*New 12.1 series Release:
2019-11-08: XigmaNAS 12.1.0.4.7091 - released!

*New 11.3 series Release:
2019-10-19: XigmaNAS 11.3.0.4.7014 - released


We really need "Your" help on XigmaNAS https://translations.launchpad.net/xigmanas translations. Please help today!

Producing and hosting XigmaNAS costs money. Please consider donating for our project so that we can continue to offer you the best.
We need your support! eg: PAYPAL

When is RAIDZ3 required?

Forum rules
Set-Up GuideFAQsForum Rules
Post Reply
Onichan
Advanced User
Advanced User
Posts: 238
Joined: 04 Jul 2012 21:41
Status: Offline

When is RAIDZ3 required?

#1

Post by Onichan » 20 Feb 2013 23:27

With 1TB and greater drives, dual parity is recommended so you don't lose all your data during a rebuild, but my question is at what point is triple parity recommended? Would using RAIDZ2 with 4TB drives still be safe? What about 5TB drives? With 4TB drives already out and 5 possibly coming out the end of this year I think it's an important question.

rostreich
Status: Offline

Re: When is RAIDZ3 required?

#2

Post by rostreich » 21 Feb 2013 02:03

is at what point is triple parity recommended?
only to be safe on your data....z2 is good, z3 is better. :)

Onichan
Advanced User
Advanced User
Posts: 238
Joined: 04 Jul 2012 21:41
Status: Offline

Re: When is RAIDZ3 required?

#3

Post by Onichan » 21 Feb 2013 03:23

rostreich wrote:
is at what point is triple parity recommended?
only to be safe on your data....z2 is good, z3 is better. :)
While I agree more parity would be safer I was more looking for if I have a 6 disk pool, what size of a drive is there a say 5% or greater chance of you loosing all your data with dual parity.

rostreich
Status: Offline

Re: When is RAIDZ3 required?

#4

Post by rostreich » 21 Feb 2013 22:41

Lets say your server is on the other end of the world and you have only 1 time in a year to have physical access to it. the more parity you have, the longer is your windows of time to survive a possible degraded state.

alexplatform
Starter
Starter
Posts: 38
Joined: 26 Jun 2012 21:21
Status: Offline

Re: When is RAIDZ3 required?

#5

Post by alexplatform » 22 Feb 2013 00:30

RAID Z is the ZFS parity raid mechanism, with Z2 and Z3 referring to dual and triple parity, respectively. Parity has repercussions both in terms of fault tolerance and performance. Parity generation has a performance cost as well as resource demand (each parity member requires a disk and a host bus channel) so generally should be accounted for in terms of sensitivity to downtime (each additional parity bit results in an additional disk that can be lost without affecting data availability) within the requirements for usable space and available mechanical resources.

Add to that the fact that data is written and read in blocks in binary arrangement, leading to the need to align data to powers of two. Since RAID stripes consist of a number of data blocks for each (set of) parity block(s), optimal RAID arrangements necessarily follow powers of two for maximum efficiency. Since one parity block for 2 drives is very costly in terms of hardware, optimal RAID configuration therefore follow in 4s- 1 parity disk per 4 data stripes. This provides the most commonly optimal arrangement of aligned stripes with sufficient fault tolerance. So double parity (Z2) would be optimally applied to 10 disk RAIDsets, and triple parity (Z3) for 15 disk RAIDsets.

Above 15 you'd want to split your data into separate raidsets, or create compound (nested) RAID arrangements; on occasion it may makes sense to double up if the application is not sensitive to the greater exposure to fault-
eg 9 disks for a Z1, 18 for a Z2, 27 for Z3.

Darkfall
NewUser
NewUser
Posts: 4
Joined: 26 Feb 2013 06:16
Status: Offline

Re: When is RAIDZ3 required?

#6

Post by Darkfall » 26 Feb 2013 08:05

The answer to your question is best answered by the people that pioneered ZFS. Here's a link to the answer to your question (and many other questions):

http://www.solarisinternals.com/wiki/in ... mendations

The questions isn't so much about disk size, but about number of disks (ie: the number of points of failure). The more disks you have, the more redundancy you want to protect you from the growing number of disks that could fail, which increases your risk of problems.

The rule of thumb is to start a RAIDZ at 3 disks (2 data + 1 parity), start a RAIDZ2 at 6 disks (4 data + 2 parity), and start a RAIDZ3 at 9 disks (6 data + 3 parity). Beyond 9 disks, start creating groups of RAIDZ/RAIDZ2/RAIDZ3 and stripe them together. That way you have parity disks in each group, thus many parity disks per array (for example, striping together 2 x 6 disk RAIDZ2 as a single array would give you 4 parity disks total, thus you can lose 2 disks per group before you have problems. Keep in mind, though, that if you lose one of the groups entirely (say 3 disks die in one of your RAIDZ2 groups), you lose the entire array).

In the real world, disks are cheap. I wouldn't use RAIDZ for anything anymore, unless it's totally unimportant scratch space. With single parity RAID, if you lose one disk and put in a new disk and start a resilvering process and another disk is discovered bad during that resilvering, you'll have 2 failed disks at the same time and the array will be trashed. That being said, ZFS is much more intelligent in how it rebuilds disks than legacy RAID controllers. Legacy controllers will rebuild every block, even if it's an empty block (which is a risk during rebuilds, since those areas have never been touched before and thus errors in those areas are unknown). ZFS resilvers only areas where data resides, which is much safer. Further, ZFS resilvers from the root of the file system up, so if something horrible happens in the middle of a resilver, everything that was done up until the issue occurred is safe (whereas legacy RAID will trash the entire array, regardless of where things went bad).

The general rule of thumb with single parity arrays is to immediately back them up if a disk fails, due to the risk of a secondary problem showing up during the recovery process. Thus, for RAID5 or RAIDZ arrays, having a hot spare ready to jump in automatically is generally a terrible idea, unless you just hate your data and want it all to go away. ;)

The resilvering of only used space with ZFS results in faster resilvers (as you might expect), as long as the disks aren't mostly full. If they're mostly full, you probably won't safe any time. If the disks are completely full, ZFS will actually take longer than legacy RAID to complete the resilver process. But, that's not all bad news! ZFS, during the resilver, also checksums the data in the process, looking for any silent data corruption (bad writes that usually go unnoticed by legacy file systems). If something is read during the resilver that doesn't add up, ZFS figures out which disk is returning bad data and fixes it on the fly.

One thing to be aware of is that a resilver with ZFS will, by default, resilver in low priority mode so that array performance is impacted as little as possible. This means that it can take quite a while to complete - sometimes long enough for people to start asking if something is wrong. It's not - ZFS is just trying not to bring your array to it's knees in the process of fixing itself. If you have double or triple parity active, then no worries - you're safe. Just let it hum away at it until it's done.

rostreich
Status: Offline

Re: When is RAIDZ3 required?

#7

Post by rostreich » 26 Feb 2013 09:31

+1 for this quality answer! :D

Post Reply

Return to “ZFS (only!)”