4 drive zfs softraid no better than 1 drive with free nas

[oph1066]

Can anyone help / advise me on my ZFS / NAS 4 Free performance issue.

I used to use freenas on a low spec Pentium with 1GIG of RAM, running ESXi over iscsi I could get about 80MB/s to a single 1TB Seagate drive.

I upgraded to an i3 with 8GB RAM and 4 x 1 TB Drives using ZFS as RAID 5 I have exported iscsi luns to two ESXI hosts, but I still see around the same sort of performance give or take a little.

(As I was using 3 separate drives one EXT lun each I am in fact getting a reduction in overall performance as now the combined LUN (data,iso and swap share this 80MB/s max)

I have checked all the networking and have tried two different ESXi boxes (one ibm x3655 server and one Intel Core 2 desktop (all three use two nics one onboard one off board and round robin for lun access) – there is a slight improvement on the server but not massive.

When I am testing using Bench32 in the guest os (server 2008R2) It does not max out the ESX Host so it is not being CPU constrained.

So is there something I need to tune with ZFS maybe (this is really disappointing)
Thanks

[raulfg3]

perhaps help: viewtopic.php?f=66&t=3977

[alexplatform]

I used to use freenas on a low spec Pentium with 1GIG of RAM, running ESXi over iscsi I could get about 80MB/s to a single 1TB Seagate drive.

Thats pretty impressive. How were you measuring the performance? A single gigabit link is only good for ~90MB/S and thats on well supported NICs on both sides (eg Intel Pro/1000 MT.) Other chipsets may have different expectations.

I upgraded to an i3 with 8GB RAM and 4 x 1 TB Drives using ZFS as RAID 5 I have exported iscsi luns to two ESXI hosts, but I still see around the same sort of performance give or take a little.

Thats all you can reasonably expect out of a single gigabit ethernet connection. You may be able to eek out a little more by optimizing your MTU and ZFS properties, but chances are you're bottlenecked on the recipient side anyway.

[oph1066]

It’s not a single Nic as mentioned in the post, its two nics using iSCSI and ESXi’s load balancing down two vmkernel ports each on a different subnet bound to a different nic on the server.

I install SAN for my day job and have set up small (by enterprise standard) boxes and seen them drive 200MB/s down two nics all day long (which is what makes me think the ZFS software RAID is the bottleneck as the i3 cpu is not tickled on the box).

[00Roush]

One thing you might want to try is a different block size for zfs volume. In NAS4Free the default block size is 8k but in some of my testing I found block size of 64k worked much better but this was with windows as the client. Unfortunately the web interface does not allow the block size to be set when creating a ZFS volume. Instead you have to create zfs volume at the command line specifying the size and blocksize. Then you have to synchronize the ZFS configuration in the NAS4Free webgui.

Other than that I would say check to make sure your network is working correctly (iperf tests from multiple clients) and your disk performance is good on the local NAS4Free box (dd read write test). Now that I think about it I am a little confused as to your setup. Is NAS4Free its own box or inside of ESXi VM? Are you using a LAGG for the dual NICs on NAS4Free side? Is ESXi loading from iSCSI LUN and then guest OSes access iSCSI through ESXi virtual disk or does guest OS access iSCSI directly.

00Roush

[00Roush]

Just found this too... http://n4f.siftusystems.com/index.php/2 ... e-esxi-51/