4 x 2 TB or 3 x 3 TB in RAID-Z?

[Nikotine]

What would be best to buy when using ZFS: 4 drives of 2 TB or 3 drives of 3 TB? Both give 6 TB total capacity.
The total price difference is neglegible. The drives I'm looking at are WD20EFRX vs WD30EFRX.
Would there be any performance difference?

[shakky4711]

Hi,

according ZFS best practices guide these setups are recommended:

Code: Select all

Optimal RAID-Zx pool member per vdev rule 2^n + p
Where n is 1, 2, 3, 4, . . .
And p is the parity: p=1 for raid-z1, p=2 for raid-z2 and p=3 for raid-z3
RAID-Z  = (2^1 + 1) … (2^n + 1) = 3, 5, 9,  17, …
RAID-Z2 = (2^1 + 2) … (2^n + 2) = 4, 6, 10, 18, …
RAID-Z3 = (2^1 + 3) … (2^n + 3) = 5, 7, 11, 19, …

So for a raid-z1 the setup with the 3 drives is better. Furthermore you reduce power consumption, heat and vibrations with 3 drives compared to 4 drives.

Shakky

[raulfg3]

allways use sollution with big disk = less problems, cheap to upgrade = less disk to upgrade, more SATA Conectors Free to upgrade.

Speed are the same in my experience, but perhaps others can know exactly what difference is in speed.

[Nikotine]

Alright, I thought that balance would tilt towards 4 drives, because you loose less "drive" (only 2TB isn't available, compared to 3TB). And replacing a 2 TB drive is cheaper than 3 TB.
But you have a point about less heat and power consumption.

Now if only I could decide between the greens or reds because there the price difference is significant...

[ChriZathens]

I haven't heard many good reviews about wd green drives and nas systems. People are saying that they are great but not for a raid array. I personally have a wd green 2tb hdd inside my pch c-200 and it is spinning for more than two years now without issues. Having this in mind, I thought about buying 3TB greens for my second nas, but I am having second thoughts now..
On the other hand I have read good reviews for the red drives, but they are 15-20% more expensive than the green ones..
Just my 2c..

[fsbruva]

My vote is to use 4 drives at 2T, and put them into two mirrors. 4T of storage with the ability to lose 2 disks, and double read speeds. Also, this is more scalable, as you can upgrade incrementally.

[ku-gew]

Pro 4x2TB: You can increase the size of the array easily: you replace all of them with (for example) 4 TB disks and you DOUBLE the size of the pool. If you use 3x3 TB, you can upgrade to only 50% more. In other words, with 3 disks you have to add another 3 disks to get meaningful upgrades and you will have 6 disks total (it's a lot).

Disadvantage 4x2TB: if one breaks, you have higher probability to experience an error during rebuild because you have THREE disks that can break, not two. In other words, you have 50% higher probability to experience problems and maybe **lose the whole array**.

Concerning speed they are both bad: RAIDZn is as fast as the slowest drive when it matters (small file reads). Maximum throughput increases with the number of disks, but not the speed for small reads (and THESE are the one that make the computer feel slow, unless you work 90% of the time with video files or 30 MB pictures).

I don't know what I would choose, in my case I have 4 disks but I used RAID10 (two RAID1 striped) and I wouldn't consider anything else except RAIDZ2.

[Nikotine]

Tnx for the input guys.
So many choices! I wasn't even considering RAID1+0 up until now, I thought RAID-Z was the new kid on the block so I should go with that. But it seems that the only advantage of RAID-Z is redundancy, while for other (though more complex) RAID situations the advantage is speed on top of redundancy.
Is that correct?

[ku-gew]

RAID10 has more redundancy: 2 disks (but not two on the same vdev). It has also more speed.
RAIDZ2 has the same redundancy but independent from the position of the disk and less speed.
RAIDZ1 has less redundancy then RAID10 or RAIDZ2 and the same speed of RAIDZ2 but higher space efficiency compared to RAID10 or RAIDZ1.

[shakky4711]

Hello,

So many choices! I wasn't even considering RAID1+0 up until now, I thought RAID-Z was the new kid on the block so I should go with that. But it seems that the only advantage of RAID-Z is redundancy, while for other (though more complex) RAID situations the advantage is speed on top of redundancy.

Recommendation what to choose is easy: Simply use the right tool for the job

A bunch of raid1 mirrors is good for speed, so ideal for database servers or similar situation, large disadvantage is that you loose 50% from the gross capacity.
Raid-z1 or z2 or z3 advantage is that you loose less capacity, so ideal for larger storage where absolute top-speed is not needed, for example your movie collection.

BTW: Many people overrate needed speed for streaming HD-movies, I calculated a 25GB movie with 100minutes playtime, result was 5MB/sec needed, this job can be handled by a P3 system within a 100mbit network

I love this decision matrix, original found here https://sites.google.com/site/eonstorage/zpool_notes

Code: Select all

Choose any 2: speed | reliability | cost
If you pick speed and reliability, it will not be cheap
If you pick reliable and cost effective, it will not be fast
If you pick speed and cost effective, it will not be reliable

Now if only I could decide between the greens or reds because there the price difference is significant...

Greens are absolute cheapest drives, I would never put into a storage application. Again the hint to use the right tool for the job, WD-RED are developed to operate in NAS arrays, 24/7 certified, special firmware according the environment in closed arrays, longer warranty.... Do not safe money on the wrong place, buy the RED or in business area Black and RE and put green into Windows desktop where it does not matter when it fails, people have learned to live with failing Windows systems so this is acceptable

Shakky

[ku-gew]

Exactly. If the NAS is used only for storage, go with RAIDZ2 starting from 6 disks or RAIDZ1 if you have only 3 disks.
I chose RAID10 because the NAS is actually the repository for ALL my data, since it is more stable and with ZFS data is safer. In this case I want good performances also with small files.
Not to mention that RAID10 can be expanded with 2 disks at a time, without replacing the others, while RAIDZx requires you to either add groups of disks (not very easy: you have to add 3 or 6 disks at time) or to replace each single disk and rebuild everything, a very long process.
Not to mention that rebuilding a single RAID1 pair is quick (4-5 hours) and not stressful for the disks, less probability to have it break.
http://constantin.glez.de/blog/2010/01/ ... still-best

[Nikotine]

Great tips!
I'm a single user and the NAS will be storing mostly movies and music, so speed will not be that critical after all.
So I'm picking cost and reliability as priorities. This brings me to 3 drives with RAID-Z1. If I need more speed, I'll add one more drive and use RAID1+0.

[ku-gew]

You cannot: once the raid has data, you cannot convert it. You will add either 3 other disks or you will relace the existing ones.