CIFS/SMB Speed - Just some hints please :)

[^nighthawk^]

Righto, I normally pride myself in sorting things out myself but in some cases lack of knowledge and the way things work in an unfamiliar operating system, can really make something that seems like a good idea become a really bad idea!

So with that in mind, here's the setup and things I have tried thus far.

----
Network Overview:

Internet -> Rubbish Sky ADSL modem/router on (192.168.0 range) -> Asus RT-AC68U Router (DHCP for 192.168.1 range) -> all other devices on 192.168.1 network some fixed, some DHCP.

Asus RT-AC68U is gigabit, NAS4Free box has gigabit connection and is connected via CAT6 to it.

----
NAS4Free config:

Embedded x64 9.2.0.1 - on AMD A10-5800K APU.
ASRock FM2A85X-ITX board.
Installed NAS4Free on a 16GB USB stick and then uploaded into memory afaik of how NAS4Free works.
Power management is currently set to hiadaptive.
Tuning of some system variables enabled.

----
Storage:

ZFS, but a ZFS stripe. Absolutely no redundancy as its essentially just a file server and the files are backed up elsewhere.
The two disks are 1.5TB Samsung Spinpoint F2 EG, S.M.A.R.T Enabled and checks out fine on both devices. These are not 4k formatted disks, they are 512's.
Connected on SATA1 and SATA2 on the board, so no fancy raid card or anything.

zdb | grep ashift -> Returns 9, which is correct.

----
CIFS/SMB:

Send and Receive Buffers default.
Enabled Large Read/White
I have enabled Asyncronous I/O (didn't initially but hoped it would help speed, if anything it seemed to make it marginally worse)
AIO read and write are the default 4096

---

Situation:
Memory Usage 43% of 16GB RAM
CPU Usage barely gets above 5% on any core.
Transfers barely creep above 15MB/s across the network from a Windows 8.1 PC connected on Wireless AC (with Wireless AC Card, connected at around 800Mbps) to the NAS4Free SMB Share.
The Windows PC can transfer files to other windows PC's across the network at high speed.
I used to get a faster speed on a previous installation of NAS4Free that i tuned (but this was some time ago - talking a few years when i switched from the Old Freenas to Nas4Free), so i know improvements can be made.

I've run some tests but currently have a file transfer going on so those results are not really indicative of what I should get. iperf etc.
I've been following most of the things I can do on performance troubleshooting posts/articles without cancelling the current transfer (mainly because it has been running 8 hours or so now and i might as well let it finish).

Any ideas?

[Onichan]

What if you ssh into the NAS and run this on a dataset that doesn't have compression enabled or on the pool root
dd if=/dev/zero of=/mnt/poolname/somedataset/ddfile bs=4k count=500k

This should take a bit, but end up writing a ~2gb file to the pool and it should tell you average write speed. Easy way to tell if your pool has the speeds it should. If you are getting over 400MB/s then add a zero to the count as two disks don't have that kind of write speed so it might be caching something.

[kenZ71]

Lots of great advice at this link
http://n4f.siftusystems.com/index.php/2 ... /comments/

[00Roush]

You might have already done it but iperf and dd tests would probably be the first thing to look at. Need to see what the raw network and disk speeds are to make sure everything is working. From there I have found changing the Samba send and receive buffers to 0 and installing ZFS kernel tune extension make the biggest difference in speeds. My understanding is setting buffer to 0 essentially lets the OS autotune vs setting a static size for everything.

Let me know if it helps.

00Roush

[^nighthawk^]

Sorry for the tardy response, the file transfer took about three days....

Ok the results of dd if=/dev/zero of=/mnt/poolname/ddfile bs=4k count=500k:

512000+0 records in
512000+0 records out
2097152000 bytes transferred in 13.134710 secs (159664887 bytes/sec)

This is just the raw result, not modified for caching etc.

---

iperf results:

Interval Transfer Bandwidth
0.0-10.0 sec 206MBytes 173 Mbits/sec

Not ideal iperf for sure....

Still following advice you all posted here, so will post back when I have more info.

[b0ssman]

173 mbit iperf is your problem.

get good networking equipment.

a real gigabit switch and intel nics.

[^nighthawk^]

Well my networking equipment is actually pretty good, not super high end sure and mainly consumer grade - can't expect everyone to have super equipment.

Anyway, i'll go through the stages that have improved things for me:

Firstly iperf was obviously my main problem from the previous test, but i found another post by 00Roush here: viewtopic.php?f=58&t=5278#p29605
suggesting to use a larger size window and check in both directions: iperf -c 192.168.0.2 -w 128k -r
This provided better results in the low 200mbit range, still appalling though.

I set the Send/Receive buffers to 0/0 in SMB and speed improved to about 24MB/sec.

Next I followed the advice of a user called russian, he suggested determining maximum MTU size in this thread: viewtopic.php?f=21&t=1702#p8005
Please note he has put ipconfig as the command, but it is ifconfig in BSD if anyone is reading this sometime later on.
I set my MTU on the NAS to 1468 although i could go a little higher, around low 1470's before the size was too big, communication with router and main pc was fine at this level.

This resulted in an improvement but I didnt like the sounds the NAS was making or the drive activity I could constantly hear.
So I turned off the Asynchronous I/O.

Results of large file copies now top out at 34MB/sec, averaging 30 for a large file. Small file copies are erratic but I can live with that.

Results of iperf with -w 128k i now get an average of 260Mbits/sec.

Other optimisations so far are adding the following into the CIFS/SAMBA auxilary parameters:
getwd cache = yes
socket options = TCP_NODELAY IPTOS_LOWDELAY SO_KEEPALIVE

This is just to stop the directory list delay really.

Will post back any further progress, but speed has effectively doubled for large writes.

[^nighthawk^]

So I looked at the ZFS Kernel tune, but as i'm running the Embedded install i'd rather put the entries into the loader.conf using the gui if possible.

Anyone have a list of the commands the ZFS Kernel Tune uses?

I'm running 16GB of RAM if that helps with the advice.

Thx.

[b0ssman]

i would not call a 200 mbit iperf test pretty good.

but if you want to have good speed, stop using wireless.

[Onichan]

Results of large file copies now top out at 34MB/sec, averaging 30 for a large file. Small file copies are erratic but I can live with that.

Results of iperf with -w 128k i now get an average of 260Mbits/sec.

Just wanted to make sure you know iperf speeds are showing megabits per second and OS's such as Windows normally show megabytes per seconds. You divide bits by 8 to get bytes so 260/8=32.5MB/s so iperf and transfer speeds are matching. As others have said that is your bottleneck.

I am surprised it is that slow, even my crappy Realtek NIC, which are known to have not so great FreeBSD drivers, used to get around 60MB/s. I simply upgraded to Intel NIC on my NAS, still used Realtek on desktop, bought a 30$ gig switch and used jumbo frames on both ends and got around 105MB/s speeds (I now have Intel NIC on my desktop too which gave an extra few MB/s speeds). You say you have Cat6 cables and what looks like a decent router so my guess is a NIC on either end causing it.

[b0ssman]

he is using wireless