*New 11.3 series Release:
2019-10-05: XigmaNAS 11.3.0.4.6928 - released, 11.2 series are soon unsupported!

*New 12.0 series Release:
2019-10-05: XigmaNAS 12.0.0.4.6928 - released!

*New 11.2 series Release:
2019-09-23: XigmaNAS 11.2.0.4.6881 - released!

We really need "Your" help on XigmaNAS https://translations.launchpad.net/xigmanas translations. Please help today!

Producing and hosting XigmaNAS costs money. Please consider donating for our project so that we can continue to offer you the best.
We need your support! eg: PAYPAL

Disk Partitions on ZFS

Forum rules
Set-Up GuideFAQsForum Rules
Post Reply
User avatar
gomario
Starter
Starter
Posts: 73
Joined: 17 Dec 2016 08:45
Status: Offline

Disk Partitions on ZFS

#1

Post by gomario » 16 Aug 2019 21:14

I have a ZFS1 server with the following:
Ada1p1
Ada2
Ada3p1
Ada4p1
Ada5p1
All drives are 4TB.
As you can see, my second disc has no partition. Server works OK but sometimes I get the "degraded" ERROR and the log shows TOO MANY ERRORS ON ADA2 and then it disconects the drive. I can easy fix the system by resilvering the drive but it always comes up as Ada2 , without any partitions! My problem is even if I replace the drive, the issues still comes back now and than! The new drive gets formated everytime without a partition and after a while I will get the same error. Clearly, there is nothing wrong with the drive. It seems there is a problem how ZFS formats this particular drive (Ada2) and how it reads the data. Even if I use another SATA port the problem persists. At first the ZFS will see the new drive as some other ADA but on reboot it will change it back to Ada2, no matter what I do.
My question is: How can I manualy format the drive so that it has a partition like all the other drives have?

cookiemonster
Advanced User
Advanced User
Posts: 164
Joined: 23 Mar 2014 02:58
Location: UK
Status: Offline

Re: Disc Patritions on ZFS

#2

Post by cookiemonster » 17 Aug 2019 21:59

How strange. I would start by comparing how the raw disks appear to the system: gpart -list
Main: Xigmanas 11.2.0.4 x64-full-RootOnZFS on Supermicro X8DT3. zroot on mirrorred pair of CRUCIAL_CT64M225. Memory: 24GB ECC; 2 Xeon E5645 CPUs; Storage: (HBA) - LSI SAS 9211-4i with 3 SATA x 1 Tb in raidZ1, 1 x 3 Tb SAS drive as single stripe.
Spare1: HP DL580 G5; 128 GB ECC RAM; 4 CPU; 8 x 500 GB disks on H210i
Spare2: HP DL360 G7; 6 GB ECC RAM; 1 Xeon CPU; 5 x 500 GB disks on H210i
Spare3: HP DL380 G7; 24 GB ECC RAM; 2 Xeon E5645 CPUs; 8 x 500 GB disks on IBM M1015 flashed to LSI9211-IT

Onichan
Advanced User
Advanced User
Posts: 237
Joined: 04 Jul 2012 21:41
Status: Offline

Re: Disc Patritions on ZFS

#3

Post by Onichan » 17 Aug 2019 22:37

I find it unlikely to be a ZFS issue, are you sure you've moved the problem drive to a completely different port and cable and still have the issue? How are you viewing that partition info? None of my drives show any partitions when looking in /dev/ada or /dev/da. Though looking at Diagnostics > Information > Partitions I do see them all showing a partition.

Also the ada labeling is assigned to each drive at boot-up and isn't static. If you shuffle around your drives the problem drive should be getting a different ada label.

cookiemonster
Advanced User
Advanced User
Posts: 164
Joined: 23 Mar 2014 02:58
Location: UK
Status: Offline

Re: Disc Patritions on ZFS

#4

Post by cookiemonster » 18 Aug 2019 00:10

Unless a permanent label assigned with glabel has been given to the disk.
Main: Xigmanas 11.2.0.4 x64-full-RootOnZFS on Supermicro X8DT3. zroot on mirrorred pair of CRUCIAL_CT64M225. Memory: 24GB ECC; 2 Xeon E5645 CPUs; Storage: (HBA) - LSI SAS 9211-4i with 3 SATA x 1 Tb in raidZ1, 1 x 3 Tb SAS drive as single stripe.
Spare1: HP DL580 G5; 128 GB ECC RAM; 4 CPU; 8 x 500 GB disks on H210i
Spare2: HP DL360 G7; 6 GB ECC RAM; 1 Xeon CPU; 5 x 500 GB disks on H210i
Spare3: HP DL380 G7; 24 GB ECC RAM; 2 Xeon E5645 CPUs; 8 x 500 GB disks on IBM M1015 flashed to LSI9211-IT

User avatar
gomario
Starter
Starter
Posts: 73
Joined: 17 Dec 2016 08:45
Status: Offline

Re: Disc Patritions on ZFS

#5

Post by gomario » 18 Aug 2019 09:23

cookiemonster wrote:
18 Aug 2019 00:10
Unless a permanent label assigned with glabel has been given to the disk.
Tell me more about it. How can I permanently assigne a ADA/DA label to a drive? I thought the ADA (for SATA ports) and DA (for USB ports), are assigned by the system only.

Onichan
Advanced User
Advanced User
Posts: 237
Joined: 04 Jul 2012 21:41
Status: Offline

Re: Disc Patritions on ZFS

#6

Post by Onichan » 18 Aug 2019 15:59

You use glabel to label drives. I've never done it, but I'm not sure if you can label it with the ada nomenclature or why you would since that's the system's device ID. Normally you give them a label that means something to you like pool1d1 pool1d2.

Here are two articles I found that talk about labeling ZFS disks
https://forums.freebsd.org/threads/howt ... tem.28181/
http://stanislavs.org/adding-disks-by-l ... ck-around/

I still think you have a hardware issue.

User avatar
Maurizio
Starter
Starter
Posts: 57
Joined: 05 Jul 2018 21:49
Location: Linate (MIlan)
Status: Offline

Re: Disk Partitions on ZFS

#7

Post by Maurizio » 19 Aug 2019 19:12

You can force the OS to assign always /dev/ada2 to the same disk modifying /boot/loader.conf.local
Please look at this post in Disk Enumeration thread
XigmaNAS 11.2.0.4 on Dell R710 144GB RAM - RootOnZFS zroot on 2x 64GB 15k HDDs in mirror, zdata on 3x 1TB SSD in RAIDZ1.
2x XigmaNAS 11.2.0.4 - RootOnZFS on HPE Proliant Microserver gen10 X3216 - 3x 4TB WD RED. In mirror with zrep.

User avatar
gomario
Starter
Starter
Posts: 73
Joined: 17 Dec 2016 08:45
Status: Offline

Re: Disk Partitions on ZFS

#8

Post by gomario » 20 Aug 2019 18:58

Thank you all for the replies. Problem sorted! To see a partition after the ADA (or DA) port number, you must ZFS format the disc without giving it a name and before you add it to the pool! If you just add the disc using the replace function than the system will not partition anything. Hence you'll only see the adapter name (ADA1 or DA1 etc)

cookiemonster
Advanced User
Advanced User
Posts: 164
Joined: 23 Mar 2014 02:58
Location: UK
Status: Offline

Re: Disk Partitions on ZFS

#9

Post by cookiemonster » 21 Aug 2019 00:18

glad is sorted, and the formatting without giving it a name is an interesting find.
There was a thread I thought I had bookmarked, but can't find it now, from this forum, where there was someone who provided their working instructions to solve this. I'll see if I can find it.
Main: Xigmanas 11.2.0.4 x64-full-RootOnZFS on Supermicro X8DT3. zroot on mirrorred pair of CRUCIAL_CT64M225. Memory: 24GB ECC; 2 Xeon E5645 CPUs; Storage: (HBA) - LSI SAS 9211-4i with 3 SATA x 1 Tb in raidZ1, 1 x 3 Tb SAS drive as single stripe.
Spare1: HP DL580 G5; 128 GB ECC RAM; 4 CPU; 8 x 500 GB disks on H210i
Spare2: HP DL360 G7; 6 GB ECC RAM; 1 Xeon CPU; 5 x 500 GB disks on H210i
Spare3: HP DL380 G7; 24 GB ECC RAM; 2 Xeon E5645 CPUs; 8 x 500 GB disks on IBM M1015 flashed to LSI9211-IT

User avatar
gomario
Starter
Starter
Posts: 73
Joined: 17 Dec 2016 08:45
Status: Offline

Re: Disk Partitions on ZFS

#10

Post by gomario » 21 Aug 2019 17:16

Another interesting thing I noticed with XigmaNAS 11.2.0.4.6766 is that it gets confused if one does mix the two formats! What I am saying is: You either ZFS format all drives before creating the ZFS Pool or you don't format any of them and let the system do the formating. However, if you mix different formated drives, at least on my Intel Mobo DQ965GF, you'll eventualy get "errors" on the system formated disc and after a while the system will disable a perfectly functioning drive! This can be fixed by resilvering the "faulty" drive but faults will apear again and you are back to square one. You can replace the disc with as many new ones as you want. The outcome will be exactly the same. To completely fix the issue, one must ZFS format the disc manually and than add it to the ZFS Pool. I did not test if the reverse is also true. In other words if you add a manually, ZFS formated disc to an automatically/system formatted ZFS Pool. I also do not know if this annomaly only happens on certain Intel mother boards or is a more general issue.

sleid
PowerUser
PowerUser
Posts: 752
Joined: 23 Jun 2012 07:36
Location: FRANCE LIMOUSIN CORREZE
Status: Offline

Re: Disk Partitions on ZFS

#11

Post by sleid » 23 Aug 2019 06:22

A ZFS pool is composed of one or more VDEVs.
A VDEV is composed of one or more DEVICES.
A DEVICE can be of the type DISK (preferred), PARTITION, FILES.
A VDEV must contain the same type of DEVICE otherwise risk of error.
You can not change the DEVICE type in a created VDEV.
A pool can use VDEVs of different composition.
In your case your pool consists of a VDEV of 4 DISK and 1 PARTITION which causes errors on the partition.
12.1.0.4 - RC1 (revision 7007)
FreeBSD 12.1-RC1 #0 r353550M: Tue Oct 15 16:23:25 CEST 2019
X64-embedded sur Intel(R) Atom(TM) CPU C2750 @ 2.40GHz Boot UEFI
ASRock C2750D4I 2 X 8GB DDR3 ECC
Pool of 2 vdev Raidz1: 3 x WDC WD40EFRX + 3 x WDC WD30EZRX

Post Reply

Return to “ZFS (only!)”