delete zfs, recreate zfs problem report

[wiki345]

NAS4Free 9.1.0.1 (revision 323) x64-embedded
DELL PE_SC3 Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz 66560MiB RAM

bus=2 hubaddr=3 port=0 devaddr=4 interface=0 bus=1 hubaddr=1 port=0
devaddr=2 interface=0 bus=2 hubaddr=6 port=0 devaddr=6 interface=0
bus=2 hubaddr=4 port=0 devaddr=5 interface=0 bus=2 hubaddr=4 port=0
devaddr=5 interface=1 bus=6 hubaddr=3 port=0 devaddr=8 interface=0
bus=6 hubaddr=1 port=0 devaddr=7 interface=0 bus=6 hubaddr=1 port=0
devaddr=7 interface=1 bus=6 hubaddr=1 port=0 devaddr=7 interface=2
EHCI (generic) USB 2.0 controller bus=2 hubaddr=2 port=0 devaddr=3
interface=0 bus=1 hubaddr=1 port=1 devaddr=2 interface=0

---

Subject:
delete zfs, recreate zfs problem report

---

Description:
If i create zfs with hot spares disk, when delete it will show some errors!
example:
root: gnop create -S 4096 /dev/mfisyspd23
root: gnop: Provider mfisyspd23.nop already exists.

i run the cli command:
gnop destroy /dev/mfisyspd23.nop

then recreat zfs, it's OK!


my zfs pool config:
=====================================
pool: VOL01
state: ONLINE
scan: none requested
config:

NAME STATE READ WRITE CKSUM
VOL01 ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
mfisyspd4.nop ONLINE 0 0 0
mfisyspd5.nop ONLINE 0 0 0
mfisyspd6.nop ONLINE 0 0 0
mfisyspd7.nop ONLINE 0 0 0
mfisyspd8.nop ONLINE 0 0 0
mfisyspd9.nop ONLINE 0 0 0
mfisyspd10.nop ONLINE 0 0 0
mfisyspd11.nop ONLINE 0 0 0
mfisyspd12.nop ONLINE 0 0 0
mfisyspd13.nop ONLINE 0 0 0
mfisyspd14.nop ONLINE 0 0 0
mfisyspd15.nop ONLINE 0 0 0
mfisyspd16.nop ONLINE 0 0 0
mfisyspd17.nop ONLINE 0 0 0
mfisyspd18.nop ONLINE 0 0 0
mfisyspd19.nop ONLINE 0 0 0
mfisyspd20.nop ONLINE 0 0 0
mfisyspd21.nop ONLINE 0 0 0
mfisyspd22.nop ONLINE 0 0 0
logs
mirror-1 ONLINE 0 0 0
mfisyspd2 ONLINE 0 0 0
mfisyspd3 ONLINE 0 0 0
cache
mfisyspd0 ONLINE 0 0 0
mfisyspd1 ONLINE 0 0 0
spares
mfisyspd23.nop AVAIL

---

Code: Select all

root: gnop create -S 4096 /dev/mfisyspd23
root: gnop: Provider mfisyspd23.nop already exists. 

[raulfg3]

sorry not related to your problem, but I suposse that you are actually doing some test and this is not in production.

I say it because is recomended do not use more of 9 disk in a Raidz1 poll , you use 19, it's better idea to use 3x6=18 Disk ( 3 vded's of 6disk in raidz1 and all 3 vdev's in one big pool)

http://www.solarisinternals.com/wiki/in ... ices_Guide

you lose 3 disk in parity, but performance increase in x3

[wiki345]

Thanks for your advice，
right now, I use it in a production environment!
NAS4free's zfs is great, same as nexenta,better than freenas,openfiler .

One question：
how to change 19disk to 18disk online?then how to add 1disk to hot spare?

[wiki345]

I test on other server, i don't know is right now? sure it can be x3 performance?
this is the config:
pool: VOL01
state: ONLINE
scan: none requested
config:

NAME STATE READ WRITE CKSUM
VOL01 ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
mfisyspd4.nop ONLINE 0 0 0
mfisyspd5.nop ONLINE 0 0 0
mfisyspd6.nop ONLINE 0 0 0
mfisyspd7.nop ONLINE 0 0 0
mfisyspd8.nop ONLINE 0 0 0
mfisyspd9.nop ONLINE 0 0 0
raidz1-1 ONLINE 0 0 0
mfisyspd10.nop ONLINE 0 0 0
mfisyspd11.nop ONLINE 0 0 0
mfisyspd12.nop ONLINE 0 0 0
mfisyspd13.nop ONLINE 0 0 0
mfisyspd14.nop ONLINE 0 0 0
mfisyspd15.nop ONLINE 0 0 0
raidz1-2 ONLINE 0 0 0
mfisyspd16.nop ONLINE 0 0 0
mfisyspd17.nop ONLINE 0 0 0
mfisyspd18.nop ONLINE 0 0 0
mfisyspd19.nop ONLINE 0 0 0
mfisyspd20.nop ONLINE 0 0 0
mfisyspd21.nop ONLINE 0 0 0
logs
mirror-3 ONLINE 0 0 0
mfisyspd2 ONLINE 0 0 0
mfisyspd3 ONLINE 0 0 0
cache
mfisyspd0 ONLINE 0 0 0
mfisyspd1 ONLINE 0 0 0
spares
mfisyspd22.nop AVAIL
mfisyspd23.nop AVAIL

errors: No known data errors

[raulfg3]

Yes , it's right now, not exact x3 , but you need to notice Write & read improvement, because writes are stripe around 3 Vdevs.

Please post your result (a compare between old a new config can be really great for all)

[wiki345]

I use it on my Production environment now.
That's Great!
before I choose NAS4Free 9.1.0.1 - Sandstorm (revision 323) ,I had tested nexenta,freenas,openfiler.
Only the NAS4free is really perfect! NAS4free has any features i just wanted.Only NAS4free fully support dell R720 (mr_sas).
Sorry,I'm chinese. my english is poor.. 不好意思啦 ^_^,要是管理员能读懂中文就好了，我可以分享很多实践经验!

=============================================================================
why the logs's capacity alloc is 0 ?

Pool information and status
capacity operations bandwidth
pool alloc free read write read write
------------------ ----- ----- ----- ----- ----- -----
VOL01 2.47T 13.8T 27 1.56K 139K 9.29M
raidz1 843G 4.61T 9 533 46.1K 3.10M
mfisyspd4.nop - - 2 14 9.26K 1.03M
mfisyspd5.nop - - 1 14 6.20K 1.03M
mfisyspd6.nop - - 2 14 9.23K 1.03M
mfisyspd7.nop - - 1 14 6.20K 1.03M
mfisyspd8.nop - - 2 14 9.25K 1.03M
mfisyspd9.nop - - 1 14 6.20K 1.03M
raidz1 843G 4.61T 9 532 46.0K 3.10M
mfisyspd10.nop - - 2 14 9.22K 1.03M
mfisyspd11.nop - - 1 14 6.21K 1.02M
mfisyspd12.nop - - 2 14 9.23K 1.03M
mfisyspd13.nop - - 1 14 6.22K 1.02M
mfisyspd14.nop - - 2 14 9.19K 1.03M
mfisyspd15.nop - - 1 14 6.21K 1.02M
raidz1 843G 4.61T 9 532 46.4K 3.10M
mfisyspd16.nop - - 2 14 9.26K 1.03M
mfisyspd17.nop - - 1 14 6.28K 1.03M
mfisyspd18.nop - - 2 14 9.27K 1.03M
mfisyspd19.nop - - 1 14 6.30K 1.02M
mfisyspd20.nop - - 2 14 9.28K 1.03M
mfisyspd21.nop - - 1 14 6.28K 1.02M
logs - - - - - -
mirror 0 93G 0 0 0 0
mfisyspd2(ssd) - - 0 0 8 9
mfisyspd3(ssd) - - 0 0 8 9
cache - - - - - -
mfisyspd0(ssd) 93.1G 7.91M 20 45 177K 4.14M
mfisyspd1(ssd) 93.1G 7.96M 20 45 180K 4.14M

[raulfg3]

why the logs's capacity alloc is 0 ?

Not sure:

Code: Select all

logs - - - - - -
mirror 0 93G 0 0 0 0
mfisyspd2(ssd) - - 0 0 8 9
mfisyspd3(ssd) - - 0 0 8 9

PD: can you do some type of read / write test?.

Perhaps using ATTO Disk Bench?