XigmaNAS

Posted: **07 Apr 2014 07:02**

I recently had a failure of my Softraid5 array, and decided to upgrade the hardware of the system while rebuilding the server and data.

I decided to go for a ZFS Raidz1 array, as I've learned Softraid is no longer in development or recommended as a storage solution.

My hardware is as follows:

MSI K9N6PGM2-V2 mobo
AMD Athlon64 x2 5200+ CPU
4 GB NON-ECC RAM (g.Skill)
3x 3TB WD AF Red drives (with load cycle fix applied)
GB D-Link NIC
CF Card for booting NAS4Free, using N4F 9.2.0.1.943 x64 embedded

The basic configuration of the server (General and advanced config, Network config, etc. ) is performed without issue.

Disks > Management adding disks works as expected. Disks are added as unformatted devices (they have been wiped with the wipe.sh script in the FAQ), and Disks > Format formatted as ZFS Storage devices with no errors.

Disks > ZFS, the Vdev is created successfully and the checkbox for 4kb sectors is checked.

Disks > ZFS > Pool is next. Pool creates successfully, and status fields show appropriate entries for disk size, etc.

zpool status shows a normal status display, all drives online, no errors.

ZFS > Configuration shows identical field information between the Current and Detected pages, with the exception of a slight variation in the pool name between the two.

Now comes the weird part:

If I reboot the server at this point, the pool status fields ALL change to UNKNOWN, and the zpool status returns a No Pools Available error. The pool Edit screen shows normal information as it was during creation. The vdev is still there, and shows the same information as during creation as well. All disks are still present and marked as ONLINE in Disks > Management.

I posted this originally to the IRC channel, and had a response asking a couple questions:

zpool status returns No Pools Available
zpool list returns No Pools Available
Going to Disks > Management and using Clear Config and Import Disks made no difference.

I've been tinkering with this for the last 2 days, and no luck. If I try using the Import config from on-disk button in the ZFS > Configuration > Detected page, it first says to force the import. Clicking THAT button returns just the word Error. It ALMOST seems that no config information is being written to the drives by this behavior.

I know using ZFS and non-ECC ram is a risky thing, but I'd like to keep current with the storage technology. However, if I must, I will return to SoftRaid5, especially if nobody has a clue as to what is happening here.

Thanks for any help.

Regards,

Mark

Posted: **07 Apr 2014 08:32**

MarkM_Erie wrote:CF Card for booting NAS4Free, using N4F 9.2.0.1.943 x64 embedded

revise that your install CD or USB is not attached, your problem sound to me like if you use liveCD or LiveUSB version, post a jpg of your system webGUI to see what version do you use.

see plattform: http://wiki.nas4free.org/doku.php?id=do ... tus_system

Posted: **07 Apr 2014 17:45**

Hi Raulfg3...

No, I'm not using the LiveCD functionality, it's an embedded install on the CF card, which actually boots the server. As requested, I've attached the Status screenshot for you.

One point I forgot to mention in my original post...

The old NAS was using WD 1.5TB Green drives. I had one of them fail, so, I ordered the 3 3TB drives to replace the original 1.5TB drives for increased storage. I know you need to replace one drive at a time in RAID5, allow it to rebuild, then do the next drive. I had started this process, but found that the motherboard was also failing. So, rather than risk any further problems, I stopped and upgraded the mobo and related hardware.

When I decided to migrate to ZFS, I then used the WD utility to wipe the first and last 1 million sectors of the drive I had used to try to rebuild the RAID5. The other two 3TB drives were not used in any way prior to trying to set up the ZFS system.

My question is, should I do a wipe with DBAN or dd on the drive used for the aborted rebuild? Or would the wipe.sh script be effective? Or is this totally unrelated to the issue at hand?

Mark

Posted: **07 Apr 2014 21:34**

MarkM_Erie wrote:No, I'm not using the LiveCD functionality, it's an embedded install on the CF card, which actually boots the server. As requested, I've attached the Status screenshot for you.

thanks , you are right, and use embeded

MarkM_Erie wrote:My question is, should I do a wipe with DBAN or dd on the drive used for the aborted rebuild? Or would the wipe.sh script be effective? Or is this totally unrelated to the issue at hand?

try DBAN, wipe.sh normally is efective, but try other wipe tool is not a problem.

please describe steps do you use to mount a ZFS pool, ( and steps to create in first place) perhaps forget some step.

Posted: **08 Apr 2014 17:39**

The following is taken as completed prior to the below steps:
1. N4F embedded installed on CF card
2. Network interface configured (option 2 from console)
3. Basic system configurations parameters set in System > General and System > Advanced

Steps for setup of ZFS array:

1. Add disks in Disks > Management. Devices are ada0, ada1, and ada2. All had the SMART checkbox checked, and left as Unformatted. No other changes made to the settings.
2. Changes applied when all disks were added.
3. Disks > Format, all three above devices formatted to ZFS Storage Device successfully.
4. Disks > ZFS Virtual Device page: Named vdev as vRaidz1, Selected ada0, ada1, ada2 as members of the vdev. Selected 4k AF checkbox. Clicked ADD, then Apply Changes. vdev created.
5. Pools > Management. Added new pool. Named as pool1, added vRaidz1 as vdev. Clicked ADD, then Apply changes. Status fields changed from Initializing' to:
Name: pool1 Size: 8.12TB Used: 703k Free: 5.33TB Capacity: 0% Dedup: 1.00k Health: ONLINE AltRoot: -

At this point I decided to reboot to make sure all changes were applied and that the array would re-mount correctly. Before reboot, the Status > System page would show an available disk sytem with specs as noted in the Pool status fields.

Upon reboot and entry to the Status > System page, it would show No Disks Configured message instead of the array. Going to ZFS > Pool > Management, all status fields there would be marked 'Unknown' .

After trying several different procedures and combinations, I sent the original post.

UPDATE:

After some digging around online, I found that there was no entry for the pool in the FSTAB file. After more investigation, had a thought on going at things a different route.

The system, as noted in the earlier posts, is booting from a CF card now. When I had the RAID5 installation, I had an old hard drive as the boot device. As this drive was quite old (10GB capacity, from around 2000), I decided to install the CF card and adapter.

When I used the CF card, I installed N4F as Embedded, thinking that it would be good as the CF card is only 2GB capacity. It would also allow me to upgrade via the GUI firmware updte process.This is how I had it set up on the hard drive as well.

On a chance, I re-installed N4F as a FULL install on the same CF card. Did the setup again, set up the ZFS system, and upon reboot, it retained the configuration. So I now have a working ZFS install.

So, the question now changes from why the ZFS pool would vanish, to why it requires a full install of N4F to make it function, instead of the Embedded install. Additionally, if I now do an upgrade from Full to Embedded via the console, will I lose he ZFS again?

I probably should have tried the Full install a lot earlier than this. But, I really wanted to keep the embedded install to make updates easier... I have to break out my external CD drive (USB) as this server has no internal CD drive any longer. But, if that's what needs to be done, so be it.

Thanks Raulfg3 for your assistance. But as of right now, it seems to be working.

Mark

Posted: **08 Apr 2014 23:27**

Since there is a live USB version you won't need an external CDROM
But this still does not explain your previous problem.
Have you already tried installing again an embedded system and test once more?
Because the only explanation for that odd behaviour would be a corrupted installation...

Posted: **09 Apr 2014 07:51**

Haven't tried changing back to the embedded install yet. Will give it a try tomorrow and see what happens...and let you know.

Not sure if it was a corrupt install though, as I re-installed as embedded at least 5 times at different points during all this.

Mark

Posted: **09 Apr 2014 11:11**

Just for the heck of it, try the embedded on a usb stick, too... Perhaps there is some weird incompatibility with your cf card (I 've seen many strange stuff over the years..

)

Posted: **10 Apr 2014 06:52**

Wiped the CF card again, and re-installed the embedded from a new download of the LiveCD iso. Imported the config from the working full install. Upon reboot, the pool once again had gone to the Unknown state. Setting up from scratch and a reboot again produced the same results.

Unfortunately, the only USB stick I have right now is under 1 GB. Will have to try when I get a higher capacity one. For now, though, I'm ticking with the full install.

I had the drives originally in a 3Ware 4-in-3 drive cage. However, thinking that (from some of the console errors I was seeing) i had a problem with the cage or the SATA cables, I pulled all the drives from the cage and mounted them in the internal case drive bays and used new SATA cables. So far, no Timeout errors on the console.

Hopefully can grab a USB stick in the next few days for testing. I'm not moving a lot of stuff back onto the server til I finish investigating this.

Will keep you posted.

Mark

Posted: **18 Apr 2014 10:34**

Hi, same problem here on a fresh embedded install!

Hardware:
Version 9.2.0.1 - Shigawire (revision 943)
Platform OS FreeBSD 9.2-RELEASE-p3 (kern.osreldate: 902001)
Platform x86-embedded on VIA Esther processor 1200MHz
System CN700-8237R
1 GB RAM

Disks:
1 CF 1 GB embedded install
2 HD WD RED 3 TB (WDC WD30EFRX-68EUZN0 )

I followed almost exactly the same steps as MarkM_Erie:

1. N4F embedded installed on CF card
2. Network interface configured (option 2 from console)
3. Basic system configurations parameters set in System > General and System > Advanced

Steps for setup of ZFS array:

1. Add disks in Disks > Management. Devices are ada0, ada1. All had the SMART checkbox checked, and left as Unformatted. No other changes made to the settings.
2. Changes applied when all disks were added.
3. Disks > Format, the two above devices formatted to ZFS Storage Device successfully.
4. Disks > ZFS Virtual Device page: Named redstripe as Stripe, Selected ada0, ada1 as members of the vdev. NOT Selected 4k AF checkbox. Clicked ADD, then Apply Changes. vdev created.
5. Pools > Management. Added new pool. Named as redpool, added redstripe as vdev. Clicked ADD, then Apply changes. Status fields changed from Initializing' to: ...ONLINE

At this point I decided to reboot to make sure all changes were applied and that the array would re-mount correctly. Before reboot, the Status > System page would show an available disk sytem with specs as noted in the Pool status fields.

Upon reboot and entry to the Status > System page, it would show No Disks Configured message instead of the array. Going to ZFS > Pool > Management, all status fields there would be marked 'Unknown' .

nas4free: ~ # zpool status
no pools available
nas4free: ~ # zpool list
no pools available

What shall I do? Please note that I don't have any data on this system yet so I can do whatever test you'd like

Please note that this is my first experience with Nas4Free, I'd like to use ZFS with striping + creating different volumes with copies=2 so that some data would be saved on both disks, while everything else only on one of them!

Posted: **19 Apr 2014 00:24**

Hi disaster...

This response is actually for everyone that have been helping me diagnose this issue as well, as I have some further results to share..

When we last left off, it had come to the point where I was advised to try doing an embedded install on a USB thumb drive. Unfortunately at that time, I didn't have one large enough to do an install, so I had to wait til I got to a store to grab one.

Earlier today I made a stop at Radio Shack and picked up an 8GB SanDisk Cruzer Glide thumb drive. Good deal to, it's currently on sale for $7.99 here in the US.

Once I got home, I removed the CF card and installed the USB drive. I then did a clean install of the embedded version of 9.2.0.1.943 as I had been trying to before. Once it was booted and configured to access via the network, I again set up the ZFS pool, as described above.

Holding my breath, once it was up and running, I rebooted the server. When I could access the status screen, it was showing an online and operational ZFS pool, just like it should. The configuration displays in the ZFS Configuration page were also what were to be expected.

So, I am now reporting that the issue has been solved. I'm not sure why the CF card would only take a Full install. Perhaps it was the capacity of the card had something to do with it, and 2GB was not large enough. My only other CF cards are 4GB in size, but they are for my dSLR, and I can't spare one of them right now for testing.

If you're in the US and have a handy Radio Shack near you, go grab one of these drives before the sale ends. It's now running my server as we speak, and I thought I'd get this off to you before starting to reload from backups. BTW, the Radio Shack stock number is 25-3393...and the sale saves you $12 off regular price ($19.99).

To everyone that gave me suggestions, thank you...it was much appreciated.

Mark

Posted: **23 Apr 2014 12:26**

Hi,
I'm in Italy so no Radio Shack for me

I've checked and my CF is actually 4 GB and I don't think that the capacity is a problem as it's being used for less that 400 MB:

Code: Select all

nas4free: ~ # df -h
Filesystem    Size    Used   Avail Capacity  Mounted on
/dev/md0      207M    200M    7.4M    96%    /
devfs         1.0k    1.0k      0B   100%    /dev
/dev/ada2a    117M    106M     10M    91%    /cf
procfs        4.0k    4.0k      0B   100%    /proc
/dev/md1       61M    752k     59M     1%    /var

What does your usb install shows?

On monday I'll find a USB key and will install the embedded version there but I'm thinking that you were simply lucky with that install and that the problem remains.
Anything else I could try? At the moment I'm thinking of not going with ZFS as maybe the support is not mature enough

Posted: **28 Apr 2014 11:52**

Hi,
I've upgraded my CF install to 9.2.0.1 - Shigawire (revision 972) and the problem persists:
I've erased the disks, recreated the pool and the results after a reboot is the same.
This is the situation before reboot:

Code: Select all

nas4free: ~ # zpool list
NAME      SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
redpool  5.46T   952K  5.46T     0%  1.00x  ONLINE  -
nas4free: ~ # zpool status redpool
  pool: redpool
 state: ONLINE
  scan: none requested
config:

	NAME        STATE     READ WRITE CKSUM
	redpool     ONLINE       0     0     0
	  ada0.nop  ONLINE       0     0     0
	  ada1.nop  ONLINE       0     0     0

errors: No known data errors
nas4free: ~ # zpool history redpool
History for 'redpool':
2014-04-28.10:17:03 zpool create -m /mnt/redpool redpool /dev/ada0.nop /dev/ada1.nop

==> /var/log/system.log <==
Apr 28 10:16:56 nas4free root: gnop create -S 4096 /dev/ada0
Apr 28 10:16:56 nas4free root: gnop create -S 4096 /dev/ada1
Apr 28 10:16:56 nas4free kernel: GEOM_NOP: Device ada0.nop created.
Apr 28 10:16:56 nas4free root: zpool create -m /mnt/redpool  redpool /dev/ada0.nop /dev/ada1.nop   
Apr 28 10:16:56 nas4free kernel: GEOM_NOP: Device ada1.nop created.
Apr 28 10:44:37 nas4free root: zfs create -o compression=off -o dedup=off -o sync=standard -o atime=on  redpool/safe
Apr 28 10:44:39 nas4free root: /mnt/redpool/safe 0777 root wheel
Apr 28 10:46:49 nas4free root: zfs create -o compression=lzjb -o dedup=off -o sync=standard -o atime=off -o quota=1000M  redpool/dump
Apr 28 10:46:49 nas4free root: /mnt/redpool/dump 0777 root wheel
Apr 28 10:56:36 nas4free root: zfs set compression=lzjb redpool/dump
Apr 28 10:56:37 nas4free root: zfs set dedup=off redpool/dump
Apr 28 10:56:37 nas4free root: zfs set sync=standard redpool/dump
Apr 28 10:56:38 nas4free root: zfs set atime=off redpool/dump
Apr 28 10:56:38 nas4free root: zfs set canmount=on redpool/dump
Apr 28 10:56:38 nas4free root: zfs set readonly=off redpool/dump
Apr 28 10:56:39 nas4free root: zfs set reservation=none redpool/dump
Apr 28 10:56:40 nas4free root: zfs set quota=1Tb redpool/dump
Apr 28 10:56:40 nas4free root: zfs set snapdir=hidden redpool/dump
Apr 28 10:56:40 nas4free root: /mnt/redpool/dump 0777 root wheel

I noticed that the ZFS config page reported a difference between detected and current, particularly on the vdev:

current working.png

detected working.png

The current shows "redstripe", the vdev I've created using the 2 disks, while the detected shows the 2 disks, could this be the source of the problem?
On the logs fo the pool creation there is no mention of the "redstripe"

Code: Select all

Apr 28 10:16:56 nas4free root: gnop create -S 4096 /dev/ada0
Apr 28 10:16:56 nas4free root: gnop create -S 4096 /dev/ada1
Apr 28 10:16:56 nas4free kernel: GEOM_NOP: Device ada0.nop created.
Apr 28 10:16:56 nas4free root: zpool create -m /mnt/redpool  redpool /dev/ada0.nop /dev/ada1.nop   
Apr 28 10:16:56 nas4free kernel: GEOM_NOP: Device ada1.nop created.

Please note that after reboot I tried to import from on-disk ZFS config but it didn't detect anything.
I will now try with the usb key and see if there's a difference

Posted: **28 Apr 2014 12:21**

disaster wrote:I noticed that the ZFS config page reported a difference between detected and current, particularly on the vdev:

You only need to go to Syncrhonize tab (next tab to current and detected) and use Sync button at the botton page.

Once your ZFS on disk are sync with your ZFS webGUI, all your problems dissapear.

PD: Sync is intended to use when you change the board or when export and then import your disk in a diferent Nas4Free install, to sysc info on disk with webGUI config.xml

Posted: **28 Apr 2014 15:05**

Hi,
I did everything again and syncronized before reboot but after reboot the pool disappears...I really don't understand how a pool can vanish, I mean, I even wrote data inside it!

Posted: **28 Apr 2014 15:30**

Hi could the problem be related to this error?

Code: Select all

nas4free: ~ # zpool import
cannot get GEOM tree: Illegal byte sequence
nas4free: ~ # gpart list
gpart: Cannot get GEOM tree: Illegal byte sequence

I have no idea of what it means, could it be that it's trying to read something from the CF and it's not supported?

**Update**
I've looked at /etc/rc.d/zfs and I think the error is definately that one!

Code: Select all

nas4free: ~ # zpool import -d /dev -f -a
cannot get GEOM tree: Illegal byte sequence

Posted: **28 Apr 2014 16:13**

save your data in a backup.

and reformat your disk, this time be sure gpart are good in your disk, and recreate Zpool.

Posted: **28 Apr 2014 16:45**

I've formatted the 2 data disks and reboot, still the same problem:

Code: Select all

nas4free: ~ # gpart list
gpart: Cannot get GEOM tree: Illegal byte sequence

So is the CF not working?

I disconnected the SAS drives and left only the CF, still the same error:

Code: Select all

==from dmesg==
ada0 at ata0 bus 0 scbus2 target 0 lun 0
ada0: <ELITE PRO CF CARD 4GB 20071207> ATA-4 device
ada0: 66.700MB/s transfers (UDMA4, PIO 512bytes)
ada0: 3823MB (7831152 512 byte sectors: 16H 63S/T 7769C)
ada0: Previously was known as ad0

nas4free: ~ # smartctl -a /dev/ada0
smartctl 6.2 2013-07-26 r3841 [FreeBSD 9.2-RELEASE-p4 i386] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     ELITE PRO CF CARD 4GB
Serial Number:    z��z�WWWW0000819D
Firmware Version: 20071207
User Capacity:    4,009,549,824 bytes [4.00 GB]
Sector Size:      512 bytes logical/physical
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ATA/ATAPI-4 (minor revision not indicated)
Local Time is:    Mon Apr 28 16:32:37 2014 CEST
SMART support is: Ambiguous - ATA IDENTIFY DEVICE words 85-87 don't show if SMART is enabled.
A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.

I've changed the CF-to-IDE adapter...same result.
I'm now cloning the CF to a USB key

Posted: **28 Apr 2014 16:54**

you dont need to clone the cf just safe the config and install the embedded on the usb stick and restore the config

Posted: **28 Apr 2014 17:27**

I know, but I wanted to clone to be sure that the problem was not with my install.
Anyway with the USB disk the GEOM tree error disappears:

Code: Select all

nas4free: ~ # gpart list
Geom name: da0
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 15240575
first: 0
entries: 8
scheme: BSD
Providers:
1. Name: da0a
   Mediasize: 131088384 (125M)
   Sectorsize: 512
   Mode: r1w0e0
   rawtype: 7
   length: 131088384
   offset: 0
   type: freebsd-ufs
   index: 1
   end: 256031
   start: 0
Consumers:
1. Name: da0
   Mediasize: 7803174912 (7.3G)
   Sectorsize: 512
   Mode: r1w0e0

I will try with another CF just to be sure that the problem is not with my CF, but as MarkM_Erie has had the same problem I think it's not that

Posted: **28 Apr 2014 18:08**

wow...the problem was the CF!
I've reinstalled on another CF, restored my config and now it seem to be working...

Code: Select all

nas4free: ~ # gpart list
Geom name: ada0
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 7813119
first: 0
entries: 8
scheme: BSD
Providers:
1. Name: ada0a
   Mediasize: 131088384 (125M)
   Sectorsize: 512
   Mode: r1w0e0
   rawtype: 7
   length: 131088384
   offset: 0
   type: freebsd-ufs
   index: 1
   end: 256031
   start: 0
Consumers:
1. Name: ada0
   Mediasize: 4000317440 (3.7G)
   Sectorsize: 512
   Mode: r1w0e0

XigmaNAS

Pool disappears after reboot

Pool disappears after reboot

Re: Pool disappears after reboot

Re: Pool disappears after reboot

Re: Pool disappears after reboot

Re: Pool disappears after reboot

Re: Pool disappears after reboot

Re: Pool disappears after reboot

Re: Pool disappears after reboot

Re: Pool disappears after reboot

Re: Pool disappears after reboot

Re: Pool disappears after reboot

Re: Pool disappears after reboot

Re: Pool disappears after reboot

Re: Pool disappears after reboot

Re: Pool disappears after reboot

Re: Pool disappears after reboot

Re: Pool disappears after reboot

Re: Pool disappears after reboot

Re: Pool disappears after reboot

Re: Pool disappears after reboot

Re: Pool disappears after reboot