HDDs Performance - opinions needed

[ChriZathens]

Hi to all...
My n4f server consists of 2 Raidz1 pools (as in my sig)
The first pool details:

Code: Select all

  pool: Media
 state: ONLINE
  scan: scrub repaired 0 in 14h5m with 0 errors on Thu Oct  3 16:15:56 2013
config:

	NAME          STATE     READ WRITE CKSUM
	Media         ONLINE       0     0     0
	  raidz1-0    ONLINE       0     0     0
	    ada0.nop  ONLINE       0     0     0
	    ada1.nop  ONLINE       0     0     0
	    ada2.nop  ONLINE       0     0     0
	    ada3.nop  ONLINE       0     0     0

errors: No known data errors

Second Pool details:

Code: Select all

  pool: Media2
 state: ONLINE
  scan: scrub repaired 0 in 5h48m with 0 errors on Fri Oct  4 07:49:05 2013
config:

	NAME          STATE     READ WRITE CKSUM
	Media2        ONLINE       0     0     0
	  raidz1-0    ONLINE       0     0     0
	    ada5.nop  ONLINE       0     0     0
	    ada6.nop  ONLINE       0     0     0
	    ada7.nop  ONLINE       0     0     0
	    ada8.nop  ONLINE       0     0     0

errors: No known data errors

In Diagnostics|S.M.A.R.T. I see that the last disk of Pool "Media" (ada3) and the first disk of Pool "Media2" (ada5), report some errors.
So I run a long test in both disks (smartctl -t long /dev/ada3 /dev/ada5). After running for about 6 hours, the results reported no problems.
However, in the past few months, I am experiencing low speeds when writing to my NAS (20-30 MB/s)
So I decided to follow siftu's performance test advice (http://n4f.siftusystems.com/index.php/2 ... leshooting)

For anyone bored of reading all this stuff, the summary is below the quoted part - the post inside the quote has more details

iperf tests reported the following

Code: Select all

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   996 MBytes   836 Mbits/sec

Seems fast enough Perhaps then I think my disks may have issues..
Started testing my pools using dd.
Results are as follows:
From the first pool (this pool consists of 4x1.5TB disks, attached to a Promise SATA 150 pci Controller)

Code: Select all

nas: ~ # dd if=/dev/zero of=/mnt/Media/Video/test.dd bs=2M count=2500
5242880000 bytes transferred in 183.151005 secs (28625996 bytes/sec)

Writes to the pool are at about 27 MB/s not good in my opinion

Code: Select all

nas: ~ # dd if=/mnt/Media/Video/test.dd of=/dev/null bs=2M count=2500
5242880000 bytes transferred in 50.476739 secs (103867249 bytes/sec)

Reads from the pool at about 100 MB/s



From the second pool (this pool consists of 4x2TB disks, attached to the M/B SATA 300 ports)

Code: Select all

nas: ~ # dd if=/dev/zero of=/mnt/Media2/Video/test.dd bs=2M count=2500
5242880000 bytes transferred in 38.893526 secs (134800840 bytes/sec)

Writes to the pool are at about 128 MB/s

Code: Select all

insomnianas: ~ # dd if=/mnt/Media2/Video/test.dd of=/dev/null bs=2M count=2500
5242880000 bytes transferred in 37.791149 secs (138733014 bytes/sec)

Reads from the pool at about 130 MB/s


And here follow the results from individual disks:

Code: Select all

nas: ~ # dd if=/dev/ada0 of=/dev/null bs=2M count=2500
5242880000 bytes transferred in 62.762158 secs (83535687 bytes/sec)

Pool 1 disk 1, pci sata150 card, 80MB/s

Code: Select all

nas: ~ # dd if=/dev/ada1 of=/dev/null bs=2M count=2500
5242880000 bytes transferred in 60.998625 secs (85950790 bytes/sec)

Pool 1 disk 2, pci sata150 card, 82MB/s

Code: Select all

nas: ~ # dd if=/dev/ada2 of=/dev/null bs=2M count=2500
5242880000 bytes transferred in 60.488957 secs (86674994 bytes/sec)

Pool 1 disk 3, pci sata150 card, 83MB/s

Code: Select all

nas: ~ # dd if=/dev/ada3 of=/dev/null bs=2M count=2500
5242880000 bytes transferred in 66.608684 secs (78711659 bytes/sec)

Pool 1 disk 1, pci sata150 card, 75MB/s

Code: Select all

nas: ~ # dd if=/dev/ada5 of=/dev/null bs=2M count=2500
5242880000 bytes transferred in 36.521652 secs (143555390 bytes/sec)

Pool 2 disk 1, M/B sata300, 136MB/s

Code: Select all

nas: ~ # dd if=/dev/ada6 of=/dev/null bs=2M count=2500
5242880000 bytes transferred in 36.995314 secs (141717407 bytes/sec)

Pool 2 disk 2, M/B sata300, 135MB/s

Code: Select all

nas: ~ # dd if=/dev/ada7 of=/dev/null bs=2M count=2500
5242880000 bytes transferred in 41.554745 secs (126168023 bytes/sec)

Pool 2 disk 3, M/B sata300, 120MB/s

Code: Select all

nas: ~ # dd if=/dev/ada8 of=/dev/null bs=2M count=2500
5242880000 bytes transferred in 35.504256 secs (147669057 bytes/sec)

Pool 2 disk 4, M/B sata300, 140MB/s

Let me sum up the results, in order to make the whole thing readable:

iperf test 836 Mbits/sec

first pool writes 27 MB/s - reads 100 MB/s
second pool writes 128 MB/s reads 130 MB/s


Pool 1 4x1.5TB Hdds connected via a SATA150 PCI controller
ada0 80MB/s (Seagate @5900rpm)
ada1 82MB/s (Seagate @5900rpm)
ada2 83MB/s (Seagate @5900rpm)
ada3 75MB/s (Samsung @5400rpm)

Pool 2 4x2TB Hdds connected to M/B SATA300 ports
ada5 136MB/s (Samsung @5400rpm)
ada6 135MB/s (Samsung @5400rpm)
ada7 120MB/s (Samsung @5400rpm)
ada8 140MB/s (Samsung @5400rpm)


Just a note, all of these test are while the NAS is in adaptive power mode (low power consumption)

What I saw here, is that S.M.A.R.T. complained about ada3 and indeed seems to be the slowest, but not sure if it has a problem, since all other disks in the pool are Seagate ones, running @5900rpm, while this one is a Samsung running @5400rpm.
But in second pool, while S.M.A.R.T. complained about ada5 having errors, the slowest disks appear to be ada7. And in this pool all disks are exactly the same brand and model (SAMSUNG HD204UI @5400rpm )

So I have come to a point that I don't really understand what may be wrong and I am really more confused that I was at the beginning.. So any insight by anyone is highly appreciated..
My apologies for the very long post..

[kenZ71]

What happens if you up the power mode ?

[ChriZathens]

Hello, Ken...
I am actually in the process of running the tests having set my nas to high performance.
I will post the summarized results once I have the tests completed..

[ChriZathens]

OK, just finished testing with high performance setting.
I am posting both results for comparison.
Normal results: adaptive mode (low power consumption)
Red results: maximum mode (lhighest performance)

first pool writes 27 MB/s - reads 100 MB/s
first pool writes 29 MB/s - reads at 115 MB/s
second pool writes 128 MB/s reads 130 MB/s
second pool write 164 MB/s read 216 MB/s


Pool 1 4x1.5TB Hdds connected via a SATA150 PCI controller
ada0 80MB/s (Seagate @5900rpm)
ada0 80MB/s (Seagate @5900rpm)
ada1 82MB/s (Seagate @5900rpm)
ada1 82MB/s (Seagate @5900rpm)
ada2 83MB/s (Seagate @5900rpm)
ada2 87MB/s (Seagate @5900rpm)
ada3 75MB/s (Samsung @5400rpm)
ada3 77MB/s (Samsung @5400rpm)

Pool 2 4x2TB Hdds connected to M/B SATA300 ports
ada5 136MB/s (Samsung @5400rpm)
ada5 137MB/s (Samsung @5400rpm)
ada6 135MB/s (Samsung @5400rpm)
ada6 135MB/s (Samsung @5400rpm)
ada7 120MB/s (Samsung @5400rpm)
ada7 122MB/s (Samsung @5400rpm)
ada8 140MB/s (Samsung @5400rpm)
ada8 141MB/s (Samsung @5400rpm)


I see great impact in the pool performance ..

Regarding the first pool: the write performance increased by 7% and the read by 15%
The increase in CPU power seems to help in this case, but not dramatically. Seems that the bottleneck in this case is the PCI controller the disks are attached to..

Regarding the second pool: the write performance increased by 28% and the read by 40%
Pretty impressive IMHO..


Individual disk testing seems unaffected from the power mode changing. Increase of 1Mb/s in not worth mentioning...

ada7 seems to have the lowest performance of all my second pool disks. And since they are identical, this maybe indicates the beginning of the end for this hdd ( )
S.M.A.R.T. reports nothing for that matter. On the other hand, disks that S.M.A.R.T. reports that may have problems, have no decrease in their performance..
Go figure...

Oh, well... what do you people think about these numbers?
Should I be worried about something? Do you see something going really wrong that I should take care off???