Is it still possible to improve ZFS write performance

[mfbatzap]

Hi!

I've been using ZFS for a while now (in RAIDZ2 mode) and I'm wondering what are the ways to improve the write speed of my current setup?

My setup is as such:
Intel Celeron 1.8Ghz
3 x Sata3 1TB WD Red in RaidZ-2 (5400 rpm)
8GB DDR3 RAM
1Gb Intel PCI-e NIC card

I am getting around 40-50MB/s write speed. When I write to the NAS, I am only seeing around 30%-40% cpu utilization and less than 30% DDR utilization.

Is this the max. write speed I can get with this setup or is there a setting where I can push the write speeds higher?

Thanks!

[lindsay]

I get 110-112MB/s but it depends sometimes on client if it do other things like P2P

[raulfg3]

I suppose that your actual bottleneck is your vdev.

you use 3 disk in a RAIDZ2 config that is not recomended, so you only have one HD available for data and two for reundancy, so you need to write at same time in three disk.


you can have better write througput if use 3 disk in RaidZ1, or 2 vdev of 3 disk in raidZ2 in a big pool.

some interesting links:

https://constantin.glez.de/2010/06/04/a ... rformance/
https://forums.freenas.org/index.php?th ... age.44068/

and the most interesting for your case:

https://calomel.org/zfs_raid_speed_capacity.html


rule of dumb:

When building a RAID it is common practice to use the "power of two plus parity" raid infrastructure to maximize parity striping, speed and well as capacity. When using ZFS the standard RAID rules may not apply, especially when LZ4 compression is active. ZFS can vary the size of the stripes on each disk and compression can make those stripes unpredictable. The current rule of thumb when making a ZFS raid is:

MIRROR (raid1) used with two(2) to four(4) disks or more.
RAIDZ-1 (raid5) used with five(5) disks or more.
RAIDZ-2 (raid6) used with six(6) disk or more.
RAIDZ-3 (raid7) used with eleven(11) disks or more.

[mfbatzap]

Thanks guys!

Unfortunately I only have 4 sata ports in my machine. The part I do not understand is...

raid6 or raidz2 distributes parity along with the data and can lose two physical drives instead of just one like raid 5. Because more parity needs to be calculated raid 6 is slower then raid5, but raid6 is safer. raidz2 requires at least four disks and will use two(2) disks of space for parity.

but

RAIDZ-2 (raid6) used with six(6) disk or more.

If it needs at least four disks, why 6 disks is recommended? So that the raid can break the reads/writes to more disks, hence speed up the read/write?

[raulfg3]

for RaidZ2 4 disk is the minimun, but lose 50% of space in parity, so 6 disk is the recomended, ( read & write speed is better with 6 disk), easy to understand.

with same 50% loss , your speed are better if chose mirror ( 2 mirror of two disk each) in a big Pool