*New 12.1 series Release:
2019-11-08: XigmaNAS 12.1.0.4.7091 - released!

*New 11.3 series Release:
2019-10-19: XigmaNAS 11.3.0.4.7014 - released


We really need "Your" help on XigmaNAS https://translations.launchpad.net/xigmanas translations. Please help today!

Producing and hosting XigmaNAS costs money. Please consider donating for our project so that we can continue to offer you the best.
We need your support! eg: PAYPAL

Slow Read Speeds, Amazing Write Speeds

Forum rules
Set-Up GuideFAQsForum Rules
Post Reply
viper4444
NewUser
NewUser
Posts: 8
Joined: 01 Jul 2012 18:15
Status: Offline

Slow Read Speeds, Amazing Write Speeds

#1

Post by viper4444 » 02 Jul 2012 19:24

Hi Guys,

I'm new to this forum since I've only recently started using NAS4Free. I moved from FreeNas, which in my opinion was terrible. I find NAS4Free MUCH better. I have built a computer with the following specs:

CPU --> Intel(R) Pentium(R) CPU G620 @ 2.60GHz
RAM --> 8 GB DDR3
HDDs --> 4 x 2TB Seagate Green in RAIDZ
MB --> ASUSTeK COMPUTER INC. P8H61-M LE/CSM Bios: 3605 03/08/2012

I'm running NAS4Free 9.0.0.1 - Sandstorm (revision 141). I have also run the daoyama kernel tune with the 8GB option. On to the problem: with a test 3.5GB file (movie), I get amazing and fairly constant ~110MB/s write. My read speeds, however, are much lower at a very volatile ~60MB/s (see attached screenshots for both). Why would the write be so much higher? Isn't it usually lower for RAIDZ? How can I get my read speeds up to where my write speeds are? Any help would be greatly appreciated. If you need any more info please let me know and post it.

User avatar
raulfg3
Site Admin
Site Admin
Posts: 4968
Joined: 22 Jun 2012 22:13
Location: Madrid (ESPAÑA)
Contact:
Status: Offline

Re: Slow Read Speeds, Amazing Write Speeds

#2

Post by raulfg3 » 02 Jul 2012 22:02

To be sure if the bottleneck is disk or SMB, try to repeat the test using the same file, but this time using Ftp ( use filezilla or other similar program to test it).

Please post your results.

PD: I suspect that bottleneck is SAMBA that need some tunning.

PD2:you can compare your result with my NAS: and your write speed it's really fantastic, and your read speed it's good. viewtopic.php?f=63&t=31
12.0.0.4 (revision 6766)+OBI on SUPERMICRO X8SIL-F 8GB of ECC RAM, 12x3TB disk in 3 vdev in RaidZ1 = 32TB Raw size only 22TB usable

Wiki
Last changes

Babovand
Starter
Starter
Posts: 46
Joined: 23 Jun 2012 16:25
Status: Offline

Re: Slow Read Speeds, Amazing Write Speeds

#3

Post by Babovand » 02 Jul 2012 22:43

Having the same "problems" Have 2x 2TB western digital (WD20EARX) SATA3, as ZFS striped, (yeah i know striped, its a media server with music and videos, dont care if one HDD breaks), My write speed with CIFS/Samba is around 100MB/s, but read speed is around 25-60 MB/s it depends on which file i copy, usually the large ones (6GB+) is 25-35MB/s, small ones around 60(minus)MB/s

Currently using 200GB space, have like 3TB+ left

viper4444
NewUser
NewUser
Posts: 8
Joined: 01 Jul 2012 18:15
Status: Offline

Re: Slow Read Speeds, Amazing Write Speeds

#4

Post by viper4444 » 03 Jul 2012 03:10

raulfg3 wrote:To be sure if the bottleneck is disk or SMB, try to repeat the test using the same file, but this time using Ftp ( use filezilla or other similar program to test it).

Please post your results.

PD: I suspect that bottleneck is SAMBA that need some tunning.

PD2:you can compare your result with my NAS: and your write speed it's really fantastic, and your read speed it's good. viewtopic.php?f=63&t=31
Hi raulfg3,

Thanks for your advice. I forgot to mention that I was using AFP, not SMB. I ran the test again with SMB and FTP and both are stuck with the slower read speeds.

RamGuy
NewUser
NewUser
Posts: 5
Joined: 26 Jun 2012 19:31
Status: Offline

Re: Slow Read Speeds, Amazing Write Speeds

#5

Post by RamGuy » 03 Jul 2012 09:49

I'm faced with the same problem, I normally get 100MB/s write speeds, which is basically the limit and being bottlenecked by my HP ProCurve 1910-8G switch that are in-between my NAS4Free Server and my desktop computers network card. I could always go for a LAG (LACP) configuration, but that would mean I had to toss the ProCurve out of the equation and drag several cables everywhere... And the last time I tried a 2x LAG (LACP) with FreeNAS 8.2 BETA3 the result was actually more unstable performance compared to just running a single connection.

When it comes to read matters are far worse, it's not horrendous but it's nowhere near what I'd expect from my current setup and configuration. I normally see everything from 15-50MB/s depending on the amount of files and their sizes. Fewer lager files like 8GB MKV's and M4V are normally at 35-50MB/s, and application folders, driver folders etc.. With a bunch of smaller files the read speeds are about 15-35MB/s.


My current hardware and setup:

Intel Xeon E3-1270 @ 3.40GHz
Asus P8B-E/4L @ BIOS 1101
16384 MB RAM (4x 4096MB) 1333MHz, CL9 ECC
Asus GeForce GT 520 (I got the E3-1270, not the E3-1275, silly me...)
IBM M1015 RAID Controller flashed with LSI 9240-8i IT-firmware
7x Western Digital RE4 2TB 7200RPM SATA3 (WD2003FYYS)
3x Intel 520 Series SSD 60GB
NAS4Free 9.0.0.1 - Sandstorm (revision 141)


I've made a full installation of NAS4Free onto one of the three Intel 520 Series SSD's running of the on-board Intel SATA controller in AHCI mode, I made a 25GB partition for the NAS4Free installation and a 25GB SWAP partition.

The storage is configured with ZFS, running 2x / 2x / 2x WD2003FYYS mirrored, and all three mirrors configured in a ZFS pool making it a RAID1+0 solution. The two extra Intel 520 Series SSD acts one ZFS Cache and one ZFS log drive just to obliterate any potential bottleneck of having the log on the actual RAID itself, and some SSD caching never hurts even though the performance of such fast hard drives in a RAID1+0 configuration combined with the bottleneck that is gigabit ethernet should make the SSD caching rather useless in practice I guess.


Considering the raw power of the server itself, the speed of the actual hard drives and the fact that they are configured in a RAID1+0 config and not a RAIDZ or RAIDZ2 which would normally hurt performance compared to a RAID1+0 configuration I'm a bit disappointed with the rather slow read speeds to be honest, and I don' really see the logic with the read speeds being way lower compared to write speeds in the first place.

I'm not experienced with NAS4Free or FreeBSD in the first place, I haven't really done any kinds of kernel tuning or service tuning as I would normally break more than I optimise. I did try to disable the Power Deamon feature just to make sure that there wasn't any power savings that made the performance sub-par but disabling it didn't make any noticeable difference. I've got the "Enable tuning of some kernel variables" checked, but I guess it doesn't do much?


When it comes to my CIFS / Samba settings they are as follows:

- SMB2
- Log level: Minimum
- Local Master Browser: Yes
- Time Server: Yes
- Send Buffer Size: 64240
- Receive Buffer Size: 64240
- Large read/write: Checked
- Use sendfile: Unchecked
- EA Support: Unchecked
- Store DOS attributes: Checked
- Asynchronous I/O (AIO): Unchecked
- Auxiliary parameters: Unchecked


I will test AFP and FTP performance later, I never use FTP but I guess it's a good way to measure whether it's the CIFS / Samba service or the actual configuration that bottlenecks the speeds. My NAS4Free Server is accessed mostly by Windows 7 systems, but also by Mac OS X Mountain Lion systems but I use AFP for OS X systems and CIFS / Samba for Windows.

User avatar
raulfg3
Site Admin
Site Admin
Posts: 4968
Joined: 22 Jun 2012 22:13
Location: Madrid (ESPAÑA)
Contact:
Status: Offline

Re: Slow Read Speeds, Amazing Write Speeds

#6

Post by raulfg3 » 03 Jul 2012 14:36

Testing using same Big file using SMB/FTP/AFP is good to point if the bottleneck is your NIC and need some type of tunning, but you need to test your Drives too using "dd" see this post to see other user test of several disk brands: http://forums.freenas.org/showthread.ph ... -and-Cache
12.0.0.4 (revision 6766)+OBI on SUPERMICRO X8SIL-F 8GB of ECC RAM, 12x3TB disk in 3 vdev in RaidZ1 = 32TB Raw size only 22TB usable

Wiki
Last changes

Babovand
Starter
Starter
Posts: 46
Joined: 23 Jun 2012 16:25
Status: Offline

Re: Slow Read Speeds, Amazing Write Speeds

#7

Post by Babovand » 04 Jul 2012 00:01

tested dd with /dev/null/zero command e.t.c
Write and Read speed for me is around 180-210MB/s (ran multiple tests), 2x 2TB SATA3 (sata 6GB/s) Striped configured.

EDIT:
Machine
Processor: Intel Core i3-2120T @ 2,6GHz
Memory: 8GB corsair vengeance 1333MHz
Motherboard: Asus p8h77-i
HDD: 2x Western Digital Green (WD20EARX)

Again, Write speed with samba is around 100MB/s, from a desktop computer with a SATA3 Drive, The network is connected through a cisco Unmanaged gigabit 24-port switch, supports jumbo frames and using it on both the nas and the desktop computer, both are connected with a CAT6 cable to the switch. Read speeds is as what RamGuy described it as quoting him.
When it comes to read matters are far worse, it's not horrendous but it's nowhere near what I'd expect from my current setup and configuration. I normally see everything from 15-50MB/s depending on the amount of files and their sizes. Fewer lager files like 8GB MKV's and M4V are normally at 35-50MB/s, and application folders, driver folders etc.. With a bunch of smaller files the read speeds are about 15-35MB/s.

viper4444
NewUser
NewUser
Posts: 8
Joined: 01 Jul 2012 18:15
Status: Offline

Re: Slow Read Speeds, Amazing Write Speeds

#8

Post by viper4444 » 04 Jul 2012 01:19

raulfg3 wrote:Testing using same Big file using SMB/FTP/AFP is good to point if the bottleneck is your NIC and need some type of tunning, but you need to test your Drives too using "dd" see this post to see other user test of several disk brands: http://forums.freenas.org/showthread.ph ... -and-Cache
Tests using dd

Write:
dd if=/dev/zero of=/mnt/RAIDZ/temp.dat bs=2048k count=50k
51200+0 records in
51200+0 records out
107374182400 bytes transferred in 446.230088 secs (240625151 bytes/sec) --> Which is roughly 230MB/s if I am correct

Read:
dd if=/mnt/RAIDZ/temp.dat of=/dev/null bs=2048k count=50k
51200+0 records in
51200+0 records out
107374182400 bytes transferred in 650.708923 secs (165011080 bytes/sec) --> 157MB/s

Thanks for your help!

viper4444
NewUser
NewUser
Posts: 8
Joined: 01 Jul 2012 18:15
Status: Offline

Re: Slow Read Speeds, Amazing Write Speeds

#9

Post by viper4444 » 04 Jul 2012 04:15

By the way, for RamGuy and Babovand, apparently people said they get improved write speeds without Hyper-Threading enabled on their CPU. Mine doesn't support it in the first place so my issue is still outstanding. Maybe try disabling it in your BIOSes and post your results?

Babovand
Starter
Starter
Posts: 46
Joined: 23 Jun 2012 16:25
Status: Offline

Re: Slow Read Speeds, Amazing Write Speeds

#10

Post by Babovand » 04 Jul 2012 18:08

viper4444 wrote:By the way, for RamGuy and Babovand, apparently people said they get improved write speeds without Hyper-Threading enabled on their CPU. Mine doesn't support it in the first place so my issue is still outstanding. Maybe try disabling it in your BIOSes and post your results?
Good Idea!
I will try that for sure! will report back.

As topic says, its READ speed that is slow for us, but Iam sure that you meant read :P

RamGuy
NewUser
NewUser
Posts: 5
Joined: 26 Jun 2012 19:31
Status: Offline

Re: Slow Read Speeds, Amazing Write Speeds

#11

Post by RamGuy » 05 Jul 2012 09:34

If Hyper Threading is supposed to improve write speeds and not read I don't really see the point as write speeds are way above what my single gigabit network connection is able to handle anyway. But I'm going to update the BIOS, so I can play around with the Hyper Threading just for fun.

User avatar
raulfg3
Site Admin
Site Admin
Posts: 4968
Joined: 22 Jun 2012 22:13
Location: Madrid (ESPAÑA)
Contact:
Status: Offline

Re: Slow Read Speeds, Amazing Write Speeds

#12

Post by raulfg3 » 05 Jul 2012 11:30

12.0.0.4 (revision 6766)+OBI on SUPERMICRO X8SIL-F 8GB of ECC RAM, 12x3TB disk in 3 vdev in RaidZ1 = 32TB Raw size only 22TB usable

Wiki
Last changes

viper4444
NewUser
NewUser
Posts: 8
Joined: 01 Jul 2012 18:15
Status: Offline

Re: Slow Read Speeds, Amazing Write Speeds

#13

Post by viper4444 » 05 Jul 2012 23:37

Babovand and RamGuy,

I meant read speeds, not write. Sorry about that.

raulfg3,

Thank you for the link. I read through that whole forum and didn't find anything that I haven't tried. I've used the kernel tune, which if I am correct, would automatically set vm.kmem_size, vm.kmem_size_max, vfs.zfs.arc_min, vfs.zfs.arc_max etc to the optimal values.

Post Reply

Return to “ZFS (only!)”