*New 12.1 series Release:
2020-04-17: XigmaNAS 12.1.0.4.7542 - released

*New 11.3 series Release:
2020-04-16: XigmaNAS 11.3.0.4.7538 - released!


We really need "Your" help on XigmaNAS https://translations.launchpad.net/xigmanas translations. Please help today!

Producing and hosting XigmaNAS costs money. Please consider donating for our project so that we can continue to offer you the best.
We need your support! eg: PAYPAL

My small beast of a nas4free box

Show off that XigmaNAS system!
Forum rules
Set-Up GuideFAQsForum Rules
Post Reply
User avatar
lindsay
Forum Moderator
Forum Moderator
Posts: 299
Joined: 23 Jun 2012 09:59
Location: Steinkjer,Norway
Status: Offline

My small beast of a nas4free box

#1

Post by lindsay »

Mainboard ASUS Sabertooth 990FX R2.0, Socket-AM3+
System hdd Corsair Force LS SSD 111GB
CPU AMD FX-6300 Black Edition, Socket AM3+ 6-Core
Memory 4XKingston DDR3 1600MHz 8GB ECC
PSU Corsair RM1000i, 1000W PSU Modular (great option when you build) ;)
Controller 2X8-port LSI SATA/SAS
Network 1Xquadport(intel)+1xsingleport(intel) 1Gbps+1xPCI 1Gbps+internal (realtek) 1Gbps
hdd`s 16 of 2 and 4TB western digital reds

Extentions :
Extended GUI
RRDGraphs
SyncThing
ZFS Kernel Tune
TheBrig

Jails
Not desided yet

Things wich do not work yet
When you have set to poweroff at a given time (Scheduled shutdown) it do not poweroff it reboots.
Solution is to disable poweron by PCIe device in BIOS. You loose the abillity to use WOL

The rack i was choosing is this one from xcase

Image

Going to get 2 LCD like this one as i read it is working(my old one is broken.
The other LCD i want to use with jail

Image

Here is it up and running for the first time with the new parts.
The power deamon is working without any errors like before.
Note there are only 16GB memory for the moment.
Image

With all 4 memory banks in use
Image

The spindown of hdd`s is set at 30 minutes

Code: Select all

/usr/local/sbin/spindown -b -p /var/run/spindown.pid -t 1800 -d da0 -d da1 -d da2 -d da3 -d da4 -d da5 -d da6 -d da7 -d da8 -d da9 -d da10 -d da11 -d da12 -d da13 -d da14 -d da15 -i 30 
Temperature of hdd`s when no fan is in use and rack not closed
Image

Temps with fans and the case closed
Image

The temp of the hdd`s do not go over 35 when they are in use, but 4TB Western do get about 4 degree hotter then 2TB ;)

What is remaining is LCD screens

When it comes to the interfaces i am planning LACP (not tested)

NEW:
Got a used quad port intelcard on our local ebay
Image

Image

Image

em0 and re0 is not part of the LAGG0
Image

He has also a used supermicro server 24bay 4U wich i am interesting in byuing cheap.
As i know him from another forum i know he is trustful enough to buy from
Last edited by lindsay on 07 Dec 2015 07:57, edited 35 times in total.
Protected by smoothiebox Red,Green,
Purple,Orange Zones/VLAN`s
Powered by AMD A10-6700T


XigmaNAS Box-1 11.2.0.4 - Omnius (revision 6625)
Platform : x64-embedded on 2X Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
Motherboard: ASUS Z10PA-D8, 2xSocket-2011-3
SATA Controllers : 1X Avago Technologies (LSI) SAS2008 and 1x Avago Technologies (LSI) SAS2308
Pool 1 (Media-Pool) 8X4TB in raidz2
Pool 2 (Media-Pool-2) 4X2TB in raidz2 and 2X2TB in mirror mirror and 2X3TB in mirror
Pool 3 (Media-Pool-3) 2X2TB in mirror and 2X4TB in mirror and 2X1TB in mirror

VictorB
Starter
Starter
Posts: 28
Joined: 28 Oct 2012 22:56
Location: The Netherlands
Contact:
Status: Offline

Re: My nas4free box

#2

Post by VictorB »

Air cooling is enough for your server. Much simpler and easy to maintain and low noice if your choose big size fans.

Keep us updated! :)
ZFS Server hardware build video https://www.youtube.com/watch?v=kBYMVUNrvDY

User avatar
lindsay
Forum Moderator
Forum Moderator
Posts: 299
Joined: 23 Jun 2012 09:59
Location: Steinkjer,Norway
Status: Offline

Re: My nas4free box

#3

Post by lindsay »

I was ordering Corsair H100i Hydro

Image

And Kingston DDR3 HyperX blu 1600MHz so i get 32GB memory

Image

This will no come before january though

One thing i have thought about is to slowly get to the goal as this one will be my primary backup server solution
And there you also have to have an UPS
Protected by smoothiebox Red,Green,
Purple,Orange Zones/VLAN`s
Powered by AMD A10-6700T


XigmaNAS Box-1 11.2.0.4 - Omnius (revision 6625)
Platform : x64-embedded on 2X Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
Motherboard: ASUS Z10PA-D8, 2xSocket-2011-3
SATA Controllers : 1X Avago Technologies (LSI) SAS2008 and 1x Avago Technologies (LSI) SAS2308
Pool 1 (Media-Pool) 8X4TB in raidz2
Pool 2 (Media-Pool-2) 4X2TB in raidz2 and 2X2TB in mirror mirror and 2X3TB in mirror
Pool 3 (Media-Pool-3) 2X2TB in mirror and 2X4TB in mirror and 2X1TB in mirror

User avatar
lindsay
Forum Moderator
Forum Moderator
Posts: 299
Joined: 23 Jun 2012 09:59
Location: Steinkjer,Norway
Status: Offline

Re: My small beast of a nas4free box

#4

Post by lindsay »

An little update is that i have for now skipped the mainboard + the CPU.

I have cleaned a little bit in the server room for some pictures soon.
I have rater spent the money on new WD 2TB reds (15 of those).
9 of them is beeing shipped on the 19 of january

The mainboard is an ASUS F2A55-M FM2 based.

Got the LCD wich also workes, only thing to comment is that the frame is cheap.
Have also invested in 2 external USB 3.0 cases to hold some hdd`s beside more PCIe cards to expand hdd`s

Next post will i show with pictures about my project as i can store them on the server.
Why have it took so long time is that it have been a pain in the b**t as the disc managment change for each boot.
Also some new discs wich i do belive have som errors (not sure)
Protected by smoothiebox Red,Green,
Purple,Orange Zones/VLAN`s
Powered by AMD A10-6700T


XigmaNAS Box-1 11.2.0.4 - Omnius (revision 6625)
Platform : x64-embedded on 2X Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
Motherboard: ASUS Z10PA-D8, 2xSocket-2011-3
SATA Controllers : 1X Avago Technologies (LSI) SAS2008 and 1x Avago Technologies (LSI) SAS2308
Pool 1 (Media-Pool) 8X4TB in raidz2
Pool 2 (Media-Pool-2) 4X2TB in raidz2 and 2X2TB in mirror mirror and 2X3TB in mirror
Pool 3 (Media-Pool-3) 2X2TB in mirror and 2X4TB in mirror and 2X1TB in mirror

al562
Advanced User
Advanced User
Posts: 221
Joined: 12 Dec 2012 08:02
Location: New Jersey, U.S.A.
Contact:
Status: Offline

Re: My small beast of a nas4free box

#5

Post by al562 »

Very nice build, I look forward to more pics.
lindsay wrote:it have been a pain in the b**t as the disc managment change for each boot.
Yes, this is a pain, but remember that FreeBSD assigns the devices based on the next available so if you have:

ada1 - ada2 - ada3 - ada4 - ada5

and then you unplug ada3 and don't plug anything back in. ada3 is now available. When you reboot, ada4 is now ada3, ada5 is now ada4 and you don't have an ada5 anymore. This is the same way Windows does it.

All the best,
Al

User avatar
lindsay
Forum Moderator
Forum Moderator
Posts: 299
Joined: 23 Jun 2012 09:59
Location: Steinkjer,Norway
Status: Offline

Re: My small beast of a nas4free box

#6

Post by lindsay »

One thing i have to be sure is this errors have to be related to the PCIe controller
The HDD`s is almost brand new only a week or so old only set up once as ZFS raidz

Code: Select all

[code]Jan 9 18:49:52	nas4free	kernel: ahcich7: Poll timeout on slot 0 port 0
Jan 9 18:49:52	nas4free	kernel: ahcich7: is 00000000 cs 00000001 ss 00000000 rs 00000001 tfd 50 serr 00000000 cmd 1000c017
Jan 9 18:49:52	nas4free	kernel: (aprobe0:ahcich7:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
Jan 9 18:49:52	nas4free	kernel: (aprobe0:ahcich7:0:0:0): CAM status: Command timeout
Jan 9 18:49:52	nas4free	kernel: (aprobe0:ahcich7:0:0:0): Error 5, Retries exhausted
Jan 9 18:49:52	nas4free	kernel: (probe0:umass-sim0:0:0:0): REPORT LUNS. CDB: a0 0 0 0 0 0 0 0 0 10 0 0
Jan 9 18:49:52	nas4free	kernel: (probe0:umass-sim0:0:0:0): CAM status: SCSI Status Error
Jan 9 18:49:52	nas4free	kernel: (probe0:umass-sim0:0:0:0): SCSI status: Check Condition
Jan 9 18:49:52	nas4free	kernel: (probe0:umass-sim0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command operation code)
Jan 9 18:49:52	nas4free	kernel: (probe0:umass-sim0:0:0:0): Error 22, Unretryable error
[/code]
Protected by smoothiebox Red,Green,
Purple,Orange Zones/VLAN`s
Powered by AMD A10-6700T


XigmaNAS Box-1 11.2.0.4 - Omnius (revision 6625)
Platform : x64-embedded on 2X Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
Motherboard: ASUS Z10PA-D8, 2xSocket-2011-3
SATA Controllers : 1X Avago Technologies (LSI) SAS2008 and 1x Avago Technologies (LSI) SAS2308
Pool 1 (Media-Pool) 8X4TB in raidz2
Pool 2 (Media-Pool-2) 4X2TB in raidz2 and 2X2TB in mirror mirror and 2X3TB in mirror
Pool 3 (Media-Pool-3) 2X2TB in mirror and 2X4TB in mirror and 2X1TB in mirror

VictorB
Starter
Starter
Posts: 28
Joined: 28 Oct 2012 22:56
Location: The Netherlands
Contact:
Status: Offline

Re: My small beast of a nas4free box

#7

Post by VictorB »

Which controller?
ZFS Server hardware build video https://www.youtube.com/watch?v=kBYMVUNrvDY

User avatar
Earendil
Moderator
Moderator
Posts: 47
Joined: 23 Jun 2012 15:57
Location: near Boston, MA, USA.
Status: Offline

Re: My small beast of a nas4free box

#8

Post by Earendil »

So I have some comments and questions. Understand I am trying to be helpful and constructively critical, I am not being rude, mean or insultingly critical. :D
  • Some people would like to see very detailed info on your systems parts, if you wish to share. Perhaps a list in the first post or in your signature block. Some place that has all the information in that one place.
  • I, for one, would be very interested in where you fit and connect 21 HDDs and all other components. Certainly not the box the CPU and mainboard is in plus the two eSATA boxes! Did I read you will have two USB3 boxes as well? What is your ZFS RAIDZ layout to be?
  • I used to work for APC and I know that rackmount UPS is not cheap. It has good output (stepped square wave) but for non-green HDDs don't expect long battery run time. Power for the main box, the two eSATA boxes and the two USB3 boxes with those non-green HDDs may overload the USB (750 VA is apparent power, real power in watts will be less).
  • Why an 8 core CPU with so many non-green (i.e. faster) HDDs but no mention of multiple Ethernet connections to take advantage of such throughput? Especially when you mention it's a 100 mbps Ethernet. Without experience I would say that is overkill but my two core is maxxed at barely 30 MB/sec. Still, with experience I know that is my untuned system in both ZFS and CIFS/SMB areas dragging throughput down and loading up the CPU.
  • I would love to hear of your performance with those USB3 boxes!
  • I think we'd all be interested in you total usable file space. Some of us would like to dream!
  • I would agree with VictorB. My N4F is the only computer I've every had that isn't a pre-assembled system (like my Qnaps or something from a first tier vendor like HP) but has the heatsinks that come with the CPU. Every other system (over the past 20 years that's over 20 systems) have had upgraded (big fans and all-copper) air-cooled heatsinks. Why? As VictorB said my N4F system rarely goes to full load except when file transferring between other systems (not streaming to the media player). This is very rare and temperature doesn't get much above 60C even then, otherwise it usually hovers at 35C to 40C. Remember, assuming you are running 24/7, even a little power used adds up over years. And I've been running my NAS for nearly three years straight. :mrgreen:
  • N4F has had problems with SATA port multipliers in the past. Most heavy multiple HDD small NAS systems use SAS. Beside the type of eSATA controller VictorB asked about, post any throughput figures as well when you get them, please.
  • Sweet system, please keep the details coming!
Earendil

XigmaNAS server:
-AMD A10-7860K APU
-Gigabyte F2A88XM-D3HP w/16GB RAM
-pool0 - 4x 2 TB WD green HDDs
-pool1 - 6x 8 TB WD white HDDs
-Ziyituod (used to be Ubit) SA3014 PCI-e 1x SATA card
-External Orico USB 3.0 5 bay HDD external enclosure set at RAID 5
--5x 2TB
-650W power supply

al562
Advanced User
Advanced User
Posts: 221
Joined: 12 Dec 2012 08:02
Location: New Jersey, U.S.A.
Contact:
Status: Offline

Re: My small beast of a nas4free box

#9

Post by al562 »

lindsay wrote:One thing i have to be sure is this errors have to be related to the PCIe controller
I am not sure, looks like the motherboard controller to me, but only way to be sure is to remove the other controllers and see if the errors are still there.

Regards,
Al

User avatar
lindsay
Forum Moderator
Forum Moderator
Posts: 299
Joined: 23 Jun 2012 09:59
Location: Steinkjer,Norway
Status: Offline

Re: My small beast of a nas4free box

#10

Post by lindsay »

What is remaining is LCD screens and replace the 2 Seagate 1TB hdd`s, thinking about replace them with 4TB or maybe 6TB Red`s
When it comes to speed i can not complain, but sadly i can not remove the GPU card yet as it do not boot properly.
But it will not be online 24/7 anyway

What is remaining is also to split the network so it do not share clients at all on the same port in Cisco and 3Com/HP Switches i use.
Thinking about replace smoothwall with pfSense also because of IPv6 and captive portal, VLAN so i can replace the box from my ISP
Protected by smoothiebox Red,Green,
Purple,Orange Zones/VLAN`s
Powered by AMD A10-6700T


XigmaNAS Box-1 11.2.0.4 - Omnius (revision 6625)
Platform : x64-embedded on 2X Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
Motherboard: ASUS Z10PA-D8, 2xSocket-2011-3
SATA Controllers : 1X Avago Technologies (LSI) SAS2008 and 1x Avago Technologies (LSI) SAS2308
Pool 1 (Media-Pool) 8X4TB in raidz2
Pool 2 (Media-Pool-2) 4X2TB in raidz2 and 2X2TB in mirror mirror and 2X3TB in mirror
Pool 3 (Media-Pool-3) 2X2TB in mirror and 2X4TB in mirror and 2X1TB in mirror

Post Reply

Return to “Gallery”