*New 11.4 series Release:
2020-07-03: XigmaNAS 11.4.0.4.7633 - released!

*New 12.1 series Release:
2020-04-17: XigmaNAS 12.1.0.4.7542 - released


We really need "Your" help on XigmaNAS https://translations.launchpad.net/xigmanas translations. Please help today!

Producing and hosting XigmaNAS costs money. Please consider donating for our project so that we can continue to offer you the best.
We need your support! eg: PAYPAL

Safe and reliable with 16TB

Show off that XigmaNAS system!
Forum rules
Set-Up GuideFAQsForum Rules
Post Reply
m_seitz
Starter
Starter
Posts: 21
Joined: 01 Mar 2015 15:36
Status: Offline

Safe and reliable with 16TB

#1

Post by m_seitz »

After many years, I finally realized a dream of mine with NAS4Free :D

Here is the set-up:

Asus M5A78L-M/USB3
AMD Phenom II X2 550
32GB ECC-RAM (4x Kingston KVR16E11/8, unbuffered)
6x 4TB HDD (HGST H3IK40003272SE)
450W Power supply (Cooler Master V450S)
UPS (CyberPower CP1300EPFCLCD, connected via USB and usbhid-ups)

Power consumption:
Start up: 180-200W (peak consumption)
Idle: 84W
Idle (Advanced Power Management set to 192): 82W
Idle (Advanced Power Management set to 128): 75W (HDDs emit high pitched noise when waking from suspend)
File transfer (one big file, CPU unrestricted): 130W (100MB/s, CPU @ 80%)
File transfer (one big file, CPU restricted to 2.4GHz): 115W (98MB/s, CPU @ 80%)
File transfer (many small files, CPU restricted to 2.4GHz): 105-110W (<0.2MB/s, CPU @ 40%)

The HDDs were combined to a raidZ2 (LZJB compressed), yielding 16TB. I hope this, together with ECC-RAM and a UPS, will keep my data safe from hardware failures.
Everything was working out of the box and I am currently filling the pool with data.
Big files reach 100MB/s (SMB), with the CPU power mode set to adaptive and everything staying "cool". Still have to find out what is limiting transfer speeds, but so far I am more than pleased with speed. Even though the CPU is quite old, I assume it will only reach its boundaries when something goes wrong with an HDD and the raid has to be rebuild or repaired...

May 2015 and the system is still up and running. The NAS has been powered on most of the time, even though the HDDs are of the desktop and not the NAS type. To stress them a little, I started a scrub and it finished without errors. The pool is filled with 10 TB data and scrubbing took 13 h at 335 MB/s (the numbers don't really sum up, but I assume that's due to compression). The CPU was busy with a load of 60 % and transferring big files was slowed down to 30-90 MB/s.
I also reduced the memory to 24 GB and it is still more than enough. Since I bought 32 GB for future projects (like using old HDDs as backup medium), I thought it would be best to have 8 GB ECC RAM in my desktop PC for now. Apparently the Gigabyte GA-870A-UD3 (rev. 2.1) runs well with ECC RAM too :D

Old:
Since I am still waiting for the case to be delivered, everything is laying around on the floor :D
nas4free_1.jpg
You can see my old storage set-up (HDD tower to the right). Once the case for my NAS4Free system arrives, I plan to use a similar solution for the HDDs and place this HDD tower inside the case. Keeping HDDs together with strong aluminium profiles keeps them cool and eliminates vibration noises. Although I don't know if it is good for the individual disks to couple them this way. Time will show :D

In the future, I might attach the old HDDs as a raid10. I would actually like to use them for backups but I don't know how I would connect them to the NAS4Free server temporarily. Maybe I will connect them permanently and set a relatively low time-out for them to spin down?


Hope this helps someone finding compatible hardware :)
You do not have the required permissions to view the files attached to this post.
Last edited by m_seitz on 04 May 2015 21:31, edited 9 times in total.
My Nas
MB: Asus M5A78L-M/USB3 RAM: 24GB unbuffered ECC (3x Kingston KVR16E11/8)
CPU: AMD Phenom II X2 550 Storage: 6x 4TB HGST H3IK40003272SE (raidZ2, one single pool)
PSU: Cooler Master V450S (450W) UPS: CyberPower CP1300EPFCLCD (USB via usbhid-ups)
OS: NAS4Free 10.2.0.2 - Prester x64-embedded
viewtopic.php?f=63&t=8482

User avatar
ChriZathens
Forum Moderator
Forum Moderator
Posts: 799
Joined: 23 Jun 2012 09:14
Location: Athens, Greece
Contact:
Status: Offline

Re: Safe and reliable with 16TB

#2

Post by ChriZathens »

Nice....What case did you get?
I am planning on purchasing the same mobo for my main NAS in order to use ECC.(My backup NAS is the one with the files that matter the most and it does have ECC RAM. )
My CPU is less powerful than yours (see my Sig for my NAS specs) and while resilvering (had to replace a failed disk a couple of months ago) never got above 40% utilisation, so I think even in that case your horsepower is enough.
Once all in in place inside the chassis, we are expecting more pictures :D
Cheers!
My Nas
  1. Case: Fractal Design Define R2
  2. M/B: Supermicro x9scl-f
  3. CPU: Intel Celeron G1620
  4. RAM: 16GB DDR3 ECC (2 x Kingston KVR1333D3E9S/8G)
  5. PSU: Chieftec 850w 80+ modular
  6. Storage: 8x2TB HDDs in a RaidZ2 array ~ 10.1 TB usable disk space
  7. O/S: XigmaNAS 11.2.0.4.6625 -amd64 embedded
  8. Extra H/W: Dell Perc H310 SAS controller, crosflashed to LSI 9211-8i IT mode, 8GB Innodisk D150SV SATADOM for O/S

Backup Nas: U-NAS NSC-400, Gigabyte MB10-DS4 (4x4TB Seagate Exos disks in RaidZ configuration - 32GB RAM)

m_seitz
Starter
Starter
Posts: 21
Joined: 01 Mar 2015 15:36
Status: Offline

Re: Safe and reliable with 16TB

#3

Post by m_seitz »

Don't worry, I will post pictures :-) But I wouldn't expect too much, I ordered a cheap midi-tower because I don't care too much about looks ;-)

It's good to know that I am on the safe side with my "old" CPU. I can recommend the M5A78L-M/USB3 with Kingston ECC. Seeing the system working without problems actually surprised me. Not just the hardware of course, also NAS4Free surprised me positively. Looks like I will have to donate some $$$ soon :-D

Now I only hope that I got lucky with the HDDs. The HGST drives they used at Backblaze were different models. At least the H3IK40003272SE drives were retail and came in nice (and safe?) boxes. Too bad that Samsung went out of business with HDDs. And with HGST belonging to Western Digital, I fear that there will be only Seagate and WD available soon...
My Nas
MB: Asus M5A78L-M/USB3 RAM: 24GB unbuffered ECC (3x Kingston KVR16E11/8)
CPU: AMD Phenom II X2 550 Storage: 6x 4TB HGST H3IK40003272SE (raidZ2, one single pool)
PSU: Cooler Master V450S (450W) UPS: CyberPower CP1300EPFCLCD (USB via usbhid-ups)
OS: NAS4Free 10.2.0.2 - Prester x64-embedded
viewtopic.php?f=63&t=8482

chrisf4lc0n
Advanced User
Advanced User
Posts: 264
Joined: 07 May 2013 13:15
Location: West Drayton (London)
Status: Offline

Re: Safe and reliable with 16TB

#4

Post by chrisf4lc0n »

Nice, but I would still invest some extra coins into the PSU, your one looks scary generic ;)
Watercooling is just the beginning ;)

m_seitz
Starter
Starter
Posts: 21
Joined: 01 Mar 2015 15:36
Status: Offline

Re: Safe and reliable with 16TB

#5

Post by m_seitz »

Hehe, this is an old PSU that I found in the basement :-)
Together with the case, I ordered a Coolermaster RS450 (80+ Gold). Hope that one will be better suited for the job. Although it is supposed to have a fan without ball bearings ...
My Nas
MB: Asus M5A78L-M/USB3 RAM: 24GB unbuffered ECC (3x Kingston KVR16E11/8)
CPU: AMD Phenom II X2 550 Storage: 6x 4TB HGST H3IK40003272SE (raidZ2, one single pool)
PSU: Cooler Master V450S (450W) UPS: CyberPower CP1300EPFCLCD (USB via usbhid-ups)
OS: NAS4Free 10.2.0.2 - Prester x64-embedded
viewtopic.php?f=63&t=8482

m_seitz
Starter
Starter
Posts: 21
Joined: 01 Mar 2015 15:36
Status: Offline

Re: Safe and reliable with 16TB

#6

Post by m_seitz »

Finally, the case and power supply were delivered.
IMG_20150307_174151.jpg
IMG_20150307_174151.jpg
The case is a Cooler Master Elite 342 ... and anything but "elite" :D It was a tight fit and there are only 2-3mm between the mainboard and the HDDs. Didn't think this through when I ordered, but it turned out alright. The case is made of very thin sheet metal, but that actually made it easy to modify it. The aluminium profiles holding the HDDs are screwed to the top of the case. Since the HGST HDDs get quite hot, I had to install a fan. The fan is inaudible, but I will have to cut a hole into the side of the case ... if I ever decide to close the case, that is.

I added some more information to the first posting, and I plan to update it every now and then.

NAS4Free is still running OK. The only error I encountered was a problem with Samba that I could fix with some help from the forum. I added "kernel change notify = no" to the auxiliary parameters and the system log has calmed down since. Hopefully, Samba will work without hiccups from now on ... but I still have to test the system for some days to be sure it won't hang again.
You do not have the required permissions to view the files attached to this post.
My Nas
MB: Asus M5A78L-M/USB3 RAM: 24GB unbuffered ECC (3x Kingston KVR16E11/8)
CPU: AMD Phenom II X2 550 Storage: 6x 4TB HGST H3IK40003272SE (raidZ2, one single pool)
PSU: Cooler Master V450S (450W) UPS: CyberPower CP1300EPFCLCD (USB via usbhid-ups)
OS: NAS4Free 10.2.0.2 - Prester x64-embedded
viewtopic.php?f=63&t=8482

chrisf4lc0n
Advanced User
Advanced User
Posts: 264
Joined: 07 May 2013 13:15
Location: West Drayton (London)
Status: Offline

Re: Safe and reliable with 16TB

#7

Post by chrisf4lc0n »

That looks better ;)
Watercooling is just the beginning ;)

Post Reply

Return to “Gallery”