ZFS Allocation question on new pool

[^nighthawk^]

Hi all,

Quick question about how ZFS allocates things... I just created a new ZFS pool using labels (on encrypted disks).

Here is the quick run down of the setup:

geli status brings back:

Code: Select all

    Name  Status  Components
ada0.eli  ACTIVE  ada0
ada1.eli  ACTIVE  ada1
ada2.eli  ACTIVE  ada2
ada3.eli  ACTIVE  ada3
ada4.eli  ACTIVE  ada4
ada5.eli  ACTIVE  ada5

geli list will result in something like the following for each disk:
As you can see 4k block specified.

Code: Select all

Providers:
1. Name: ada5.eli
   Mediasize: 3000592977920 (2.7T)
   Sectorsize: 4096
   Mode: r1w1e2
Consumers:
1. Name: ada5
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1

Now glabel status:

Code: Select all

 
 label/2J4X     N/A  ada0.eli
 label/VJ7U     N/A  ada1.eli
 label/VL3F     N/A  ada2.eli
 label/VP01     N/A  ada3.eli
 label/97D7     N/A  ada4.eli
 label/4CA5     N/A  ada5.eli

Using WD Red drives, so if anyone is interested - these are the last 4 digits of the serial, this is actually labelled on the end of the drive so if you have a case like a LianLi Q25B you can see these 4 digits on the back of the drives in the hotswop bays. Also diskinfo on a physical drive will return a value containing these digits.

glabel list returns something similar to the following:
This results in a 4k sector size for the label rather than the 512 it would have allocated outside of the encrypted disk.

Code: Select all

Geom name: ada5.eli
Providers:
1. Name: label/4CA5
   Mediasize: 3000592973824 (2.7T)
   Sectorsize: 4096
   Mode: r1w1e1
   secoffset: 0
   offset: 0
   seclength: 5860533152
   length: 3000592973824
   index: 0
Consumers:
1. Name: ada5.eli
   Mediasize: 3000592977920 (2.7T)
   Sectorsize: 4096
   Mode: r1w1e2

ZFS setup:

So as I said I created a pool of 3 mirrors, using the labels i figure this will ensure the zfs allocation per disk is (actual disk size minus 4096 geli sector minus 4096 label sector. So zfs has the rest of the disk advertised.

Code: Select all

  pool: DataPoolA
 state: ONLINE
  scan: none requested
config:

        NAME            STATE     READ WRITE CKSUM
        DataPoolA       ONLINE       0     0     0
          mirror-0      ONLINE       0     0     0
            label/VJ7U  ONLINE       0     0     0
            label/97D7  ONLINE       0     0     0
          mirror-1      ONLINE       0     0     0
            label/VL3F  ONLINE       0     0     0
            label/4CA5  ONLINE       0     0     0
          mirror-2      ONLINE       0     0     0
            label/VP01  ONLINE       0     0     0
            label/2J4X  ONLINE       0     0     0

errors: No known data errors

I have been testing a few things, swopping physical drives around, removing drives etc to make sure things all work and i've noticed something a little off but maybe its normal behaviour, i just don't know enough about ZFS in this regard.

My blank pool after creation reported:

Code: Select all

DataPoolA 	8.16T 	276K (0%) 	8.16T 	- 	0% 	1.00x 	ONLINE 	/DataPoolA 	-

The 276k being the "allocation".

All it took was a few tests swopping drives around and a couple of reboots... and this number was at over 1MB.

I still hadn't written data to the pool and a scrub doesn't get me that data back or fix it.... what is it actually doing?

Any hints please

[^nighthawk^]

Well i still didn't quite figure this out. But if i don't start swopping disks around in the chassis it happens much less. Also running a clear and a scrub as well as removing snapshots and datastores i don't need and just created for testing purposes does appear to help.