This is the old XigmaNAS forum in read only mode,
it will taken offline by the end of march 2021!



I like to aks Users and Admins to rewrite/take over important post from here into the new fresh main forum!
Its not possible for us to export from here and import it to the main forum!

Strange behaviour when using ZFS stripe

Forum rules
Set-Up GuideFAQsForum Rules
Post Reply
fallen2109
Starter
Starter
Posts: 51
Joined: 27 Oct 2014 22:38
Location: The Netherlands
Status: Offline

Strange behaviour when using ZFS stripe

Post by fallen2109 »

I am experiencing strange issues (CAM errors) when configuring and using 2 identical HGST 6TB HDDs in ZFS Stripe.
Once I start writing to the ZFS volume - I am getting CAM errors on da4 (the 2 drives are /dev/da4 and /dev/da5).
No matter what I do - I can not get rid of this errors and soon da4 is even disconnected and my stripe is of course
corrupted. I did several tests on single drives - both for /dev/da4 and /dev/da5, but the drives are healthy.
Constant SMART monitoring also does not reveal anything - in the end the drives are factory new!

On /dev/da0, da1 and da2 I have my other ZFS Z1 Primary data array (WD Reds - also 6 TB in size) which runs
without problems. It even has a cache device on /dev/da3 (1TB Samsung EVO 840 SSD) .

Another anomaly that I am facing is the enumeration of my SATA devices. All of the above drives are connected to
Avago (LSI-2008) card which is in a Non-RAID mode, but while da0, da1 and da2 drives are indeed connected to
ports 0, 1 and 2 of the controller da3 is connected to port 5. So theoretically it should be enumerated as /dev/da5,
but because it was installed earlier in the system (before the /dev/da4 and /dev/da5) it is always enumerated as
/dev/da3 - (it's initial name), no matter to which port I connect on the LSI-2008! The two HGST drives are connected
to SAS ports 3 and 4, and are enumerated as /dev/da4 and dev/da5 as I stated above.

Can someone throw a suggestion here? What shall I do? For the time being I am using both HGST drives as separate
GPT backup drives for my data, but I would like very much to unite them in a ZFS stripe. No errors are thrown when I
use them as separate drives though! So I guess there are no issues with my LS-2008, SATA ports, SATA cables and HDDs.
It is quite a mystery to me what is wrong here :).

These are my current settings

Version 11.0.0.4 - Sayyadina (revision 3330)
Platform OS FreeBSD 11.0-RELEASE-p5 #0 r309722M: Thu Dec 8 22:52:57 CET 2016
Platform x64-full on Intel(R) Xeon(R) CPU L5640 @ 2.27GHz (Dual CPUs)
System Supermicro X8DT3
RAM 48 GB
12.0.0.4 - Reticulus (revision 6928)
x64-full on Intel(R) Xeon(R) CPU L5640 @ 2.27GHz (Dual CPU)
Supermicro X8DT3, 48 GB Ram

User avatar
raulfg3
Site Admin
Site Admin
Posts: 4865
Joined: 22 Jun 2012 22:13
Location: Madrid (ESPAÑA)
Contact:
Status: Offline

Re: Strange behaviour when using ZFS stripe

Post by raulfg3 »

what version of firmware do you use on SAS2008?.

http://forums.nas4free.org/viewtopic.ph ... 9&start=90

http://brycv.com/blog/2012/flashing-it- ... as9211-8i/


and revise that your Power Supply is capable of course.
12.1.0.4 - Ingva (revision 7743) on SUPERMICRO X8SIL-F 8GB of ECC RAM, 11x3TB disk in 1 vdev = Vpool = 32TB Raw size , so 29TB usable size (I Have other NAS as Backup)

Wiki
Last changes

HP T510

fallen2109
Starter
Starter
Posts: 51
Joined: 27 Oct 2014 22:38
Location: The Netherlands
Status: Offline

Re: Strange behaviour when using ZFS stripe

Post by fallen2109 »

I am on the latest firmware 20.x.x.4 as far as I remember. The link you have outlined in your post is well known to me - it actually helped me to convert the firmware from IR to IT. Power supply is at least 750 W.
12.0.0.4 - Reticulus (revision 6928)
x64-full on Intel(R) Xeon(R) CPU L5640 @ 2.27GHz (Dual CPU)
Supermicro X8DT3, 48 GB Ram

fallen2109
Starter
Starter
Posts: 51
Joined: 27 Oct 2014 22:38
Location: The Netherlands
Status: Offline

Re: Strange behaviour when using ZFS stripe

Post by fallen2109 »

RESOLVED! My hot plug HDD cage (for 5 drives) was not having sufficient power. One of the cheep Chinese cr*p power split cables was the root cause for all of this ! Blow my whole weekend away. Now I switched to direct power in all 3 sockets of the hot spare case and everything seems to be running as if there were never problems. Jesus - so frustrating!
12.0.0.4 - Reticulus (revision 6928)
x64-full on Intel(R) Xeon(R) CPU L5640 @ 2.27GHz (Dual CPU)
Supermicro X8DT3, 48 GB Ram

Post Reply

Return to “ZFS (only!)”