Unstable file transfers

[hqnet]

Hi,

I´ve been asked to improve the performance of a NAS4Free based server.
While I´ve tested a few NAS oriented linux distributions in the past this is my first in-depth incursion in the BSD realms.

I don´t want to steal too much time from you so I will go straight to the point:

This server is currently running v 9.1.0.1.847, has 6 drives, 3 TB each in Raidz1 mode, it is used to store video editing projects so huge files are being frequently copied on and off the Server.

I have assempled a tiny NAS4Free machine in order to famliarize myself with NAS4Free and see if I could reproduce the main problem I would need to fix, which is frequent drops during the transfers.

As it seemed to me that the main issue could be the "ZFS breathing" I have tunned the loader and sysctl configurations and I did achieve a relatively faster and more stable performance. But what I´ve found now is that SMB/CIFS transfers remain unstable while FTP transfer are much smoother, actually I believe the speed transfers could very well be hitting my very poor test-hardware limits.


So my question is if there is any documentation that specifically addresses the performance of CIFS + very large files + RaidZ1.

I have done many searches on this specific subject but for every suggestion I find another one in the opposite direction, and unfortunanly I´m against the clock to get, if not a fully optimal performance, at least one that allows people to work.


Maybe I should just drop SMB and use FTP or AFP instead? (some of them are Mac terminals).

maybe using 9.2.x would mean a quick way to get a noticeable boost in samba performance?


Thanks in advance.
HQ.

[bart357]

you never posted system specs memory etc.
personaly I would stay with v 9.1.0.1.847 at the moment. just my .02

[hqnet]

you never posted system specs memory etc.
personaly I would stay with v 9.1.0.1.847 at the moment. just my .02

Hi, thanks for the reply!

yeah, I know, the thing is that the hardware differences between my test machine and the real server are huge, so I didn´t want that to add noise to the thread.
It seems to me that Samba is starting to have serious troubles across any OSes with the increase in file sizes during the recent years


FWIW, my test machine has a 3Ghz dual core Intel processor, 4 GB of Ram, 1 GB Nic.

The 'real' server has a 6 core Xeon processor, 16 GB Ram, 1 GB Nic (I don´t have more specifics nor direct access to it ATM, sorry)

HQ

[crowi]

There are a lot of tweaks, the first thing I would try is to use ZFS kerneltune viewtopic.php?f=71&t=1278
enable prefetch and set the right amount of your servers memory there.

Additional information would help as well:
your servers specs (as mentioned before)
how many users access the files
how big are they
Copy a screenshot of the Status | Graph during or after a big file transfer with CIFS/SMB
etc

Another read with hints: https://wiki.freebsd.org/ZFSTuningGuide

It's a little unlikely that you get same troubles with completely different hardware

[00Roush]

To help with samba stability make sure you have AIO disabled. There were past problems with AIO and SMB2. Not sure if it got fixed completely.

For the best performance I recommend changing samba buffers to 0 and then install and use zfs kernel tune N4F extension. Setting buffers to 0 essentially disables them and allows the OS to autotune. ZFS extension automatically sets loader and sysctl values based on selected ram size. In my experience the ZFS extension helps reduce or eliminate the ZFS breathing you mentioned and also improves file transfer speeds.

Not sure if I would consider FTP... from what I remember it does not do the same amount of checking as other protocols as far as file integrity.

With fast disks your test setup should be able to get 100+ MB/sec. At least that is what I have seen with similar hardware.

Hope it helps.
00Roush

[hqnet]

Hi,

thanks for the input, I liked NAS4Free when I first met it (a couple of weeks ago) but getting also good, fast and gentle answers from the community is simply great!

Some of what you guys mention I have already gone through, but much of that info is new to me and looks promising.

I am aware of the implications of using so dissimilar hardware but that was the best I could do, so I appreciate your kind comments.


I will go through all of this and eventually come back with more solid data about the real Server.

Best regards,
HQ.

[crowi]

yes some information would be helpful, probably also the use of a ZIL and L2ARC can improve the transfer speed, if the people are working directly with the files stored at the server, not only copying them from and to the server.