Mirror or RAIDZ2 for 4 disks?

[loozhengyuan]

I am currently thinking of upgrading my pool and server by adding more disks and changing into the HP Microserver Gen8 which has 4 slots, so i'm thinking:

Mirror Configuration
Pool 1: 2 x 2TB (1 redundant)
Pool 2: 2 x 2TB (1 redundant)
(Reason for splitting into two pools is so that if one vdev is lost, i will not lose the whole zpool)

RAIDz2 Configuration
Pool 1: 4 x 2TB (2 redundant)

What do you guys think? Which configuration should I choose and why?

[b0ssman]

put the numbers in here
http://www.servethehome.com/raid-calcul ... tdl-model/
and judge for yourself what the theory says.

[riporto]

Interested in same matter.
Tried the link posted from b0ssman, couldn't figure out much more...

My server was 4x1Tb Raidz1 but I lost one disk and now I want something more solid, even if it will make me lose more space.

So my configuartion question is same as loozhengyuan.

p.s. some guides about that would be very apreciated!
Thanks.

[ChriZathens]

Two mirrors in one pool will offer better speeds (irrelevant if you use file sharing - even a large raidz3 vdev - or pool if you prefer, in case it is consisting of a single raidz3 vdev, - can saturate a Gbit NIC. Relevant if you are using it as a VM storage, and/or using 10G nics.)
Rebuilding (resilvering in ZFS terms) will also be faster. You can afford two damaged hdds in this configuration, but not two from the same mirror (in this case bye bye data)
Two mirrors in different pools is somewhat slower (still faster than raidz2, though). Two damaged disks from the same mirror will result of loss of the pool that contains them, but data in the other pool will be intact. A con in this case is that you won't have unified space.

With Raidz2 you have a pool totally capable of saturating a GB NIC, and at the same time you can afford any two damaged hdds.
In all cases the total available space is the same.

Take your pick...

[simonlebon]

Expanding RaidZx is hell (been there, gave up after two weeks of resilvering) and more expensive.
Expanding mirrored stripe is a lot easier.

The pro (and cons) for mirror vdefs vs RaidZ:
http://jrs-s.net/2015/02/06/zfs-you-sho ... not-raidz/

If you really value your data; Back up your pool!

[ChriZathens]

Many would argue that expanding mirrors is equally expensive.
The money needed is certainly less, but in at the same time you pay double money per GB.
And in OP's case, with 4 hdds and no room to add more, the space expansion can only happen by replacing existing disks with larger ones, which is the same cost either using striped mirrors or RAIDZ2

[simonlebon]

Agreed, but the cost in time and effort (resilver every drive) is a lot more (personal experience).
Plus, in a mirrored stripe you can replace 2 disks to increase the pool size instead of 4.
In that case, costs will be less.

[EDIT]
Resilvering a RaidZx puts more stress on the drives than resilvering a mirrored drive.
Samples:
-A very good friend of mine experienced the horror scenario: a drive failure in a RAID5, replaced the defective drive, but the rebuild caused another drive failure. He had a back up, but took him a week to get things running again.

4disk RaidZ2 / Mirrored Stripe comparison:
- Cost: equal
-There is no difference in the available capacity, a four disk RaidZ2 has the same size as mirrored stripe.

RaidZ2
Pro's:
- one more parity drive=reduce risk
Cons:
- to expand one has to replace all four drives
- stress during resilver (continues writing on all 'old' disks=increase risk)

Mirrored Stripe
Pro's:
- less CPU utilisation,
- expand can be done with 2 drives (or 3 in case one wants to include a spare), one can use different drive sizes/vdev size
- less stress during resilver (only reads from one old drive, writing on the new drive)
Cons:
- If two drives fail in a single vdev, entire pool is lost (this risk can be reduced by using different drive manufacturer/type)

Personally, I choose the latter to have a bit more options in the future, but it's up to the OP what he decides, just throwing in my 2 pennies.

[ChriZathens]

Agreed, but the cost in time and effort (resilver every drive) is a lot more (personal experience).
Plus, in a mirrored stripe you can replace 2 disks to increase the pool size instead of 4.
In that case, costs will be less.

Yes, this is correct

Resilvering a RaidZx puts more stress on the drives than resilvering a mirrored drive.
Samples:
-A very good friend of mine experienced the horror scenario: a drive failure in a RAID5, replaced the defective drive, but the rebuild caused another drive failure. He had a back up, but took him a week to get things running again.

This can actually happen also while resilvering a mirror. There is still only one drive failure tolerance, and If the disk is to fail, less or more stress is almost irrelevant - it will fail.
In that perspective, you are more safe if you have dual parity

In general, for home use, the most recommended setup is a raidz2.(But for more cost effective setups, if you want to reduce cost per GB, at least 6 disks should be used)
In any case, it is safer than any other configuration (desides raidz3, of course, or triple/quadriple mirrors - not a typical home use senario)
And this is actually valid even with only 4 disks (statistics don't lie)
Of course anyone's experience may be different, but in general the above is a fact.

[simonlebon]

This can actually happen also while resilvering a mirror. There is still only one drive failure tolerance, and If the disk is to fail, less or more stress is almost irrelevant - it will fail.
In that perspective, you are more safe if you have dual parity

Kalispera!

Less or more stress can make a difference, in general, a drive that writes gets hotter and is more likely to fail. So I wouldn't say that it's irrelevant.
But dual parity is indeed safer than single parity, can't disagree to that.

In general, for home use, the most recommended setup is a raidz2.(But for more cost effective setups, if you want to reduce cost per GB, at least 6 disks should be used)
In any case, it is safer than any other configuration (desides raidz3, of course, or triple/quadriple mirrors - not a typical home use senario)
And this is actually valid even with only 4 disks (statistics don't lie)

In my personal experience, saving a few bucks (or €) first, will fire back later. Upgrading 6 (Raidz2)disks (or 5) in one time vs replacing 2 disks. Plus, you wil have different drives in your pool (reduce risk of failing disks).

As a home user, your need for space increases in time.
My latest strategy for upgrading is to order two different drives of the same size.

[simonlebon]

Resilvering a RaidZx puts more stress on the drives than resilvering a mirrored drive.
Samples:
-A very good friend of mine experienced the horror scenario: a drive failure in a RAID5, replaced the defective drive, but the rebuild caused another drive failure. He had a back up, but took him a week to get things running again.

This can actually happen also while resilvering a mirror. There is still only one drive failure tolerance, and If the disk is to fail, less or more stress is almost irrelevant - it will fail.
In that perspective, you are more safe if you have dual parity

In general, for home use, the most recommended setup is a raidz2.(But for more cost effective setups, if you want to reduce cost per GB, at least 6 disks should be used)
In any case, it is safer than any other configuration (desides raidz3, of course, or triple/quadriple mirrors - not a typical home use senario)
And this is actually valid even with only 4 disks (statistics don't lie)
Of course anyone's experience may be different, but in general the above is a fact.

When upgrading RAIDZ2, you only have 1 disk fault tolerance (the other one is replaced by the new disk) plus the additional stress to the system, resulting in low performance during resilvering. In addition to that, it takes way more time to resilver a RAIDZ2 than resilvering a mirror.

Besides that, when you upgrade a mirror, you can first add a third (the new bigger) drive to the mirror. Then you have two drive failure tolerance. This is even safer.

After the resilvering is done you can replace an old disk with a new disk. After resilvering with that one, you remove the last old disk from the mirror (and even could add that one another mirror).

Another possibility (i.e. only 4 SATA ports) is to swap one drive in the mirror with a newer drive, resilver, then swap the other old drive with a new one and resilver.

The benefits of striped mirror: saves a LOT of time, performance is great and doesn't collapse like RAIDZx, flexible upgrading.
The downside: Only 50% of the total disk space is available
When I buy new disks, I look at the price per Gigabyte. The sweet spot shifts every year.
I have a 6 disk stripe mirror (2x2TB, 2x4TB, 2x6TB) and I'm glad I did it like this. It's blazing fast, saturates the Gbit network completely and If I need more storage, I can replace the 2 TB disks, (after that the 4TB's) and keep my savings account healthy.

In OP's case, one pair of 2 TB's (=4 TB usable storage space) can be replaced by a pair of 5 TB's (=7 TB) and in the future the other pair of 2 TB's can be replaced by 8 TB's (=13 TB). Or other drives depending on the sweet price per Gigabyte spot, of course.

In a 4 disk RAIDZ2, also 50% of the total disk space is available, but if you want to expand you will have to replace 4 disks instead of 2. The two drive fault tolerance isn't twice as better as the a striped mirror one drive tolerance.

Conclusion: If the OP has to choose between 4disk RAIDZ2 or stripe mirror (50%=2TB), I would recommend stripe mirror, esp for the sake of flexibility in future expansion. If the OP wants more storage efficiency (and a bit more risk), he could consider 4 disk RAIDZ1 (75%=3TB). RAIDZ2 in a 4 disk situation has hardly any benifits.

Links about mirror vs RAIDZ:
http://constantin.glez.de/blog/2010/01/ ... still-best
http://jrs-s.net/2015/02/06/zfs-you-sho ... not-raidz/