*New 12.1 series Release:
2019-11-08: XigmaNAS 12.1.0.4.7091 - released!

*New 11.3 series Release:
2019-10-19: XigmaNAS 11.3.0.4.7014 - released


We really need "Your" help on XigmaNAS https://translations.launchpad.net/xigmanas translations. Please help today!

Producing and hosting XigmaNAS costs money. Please consider donating for our project so that we can continue to offer you the best.
We need your support! eg: PAYPAL

Replacing Disk woes

Forum rules
Set-Up GuideFAQsForum Rules
Post Reply
User avatar
HorzaSe
NewUser
NewUser
Posts: 7
Joined: 25 Jun 2012 13:38
Status: Offline

Replacing Disk woes

#1

Post by HorzaSe » 29 Oct 2019 23:52

Hi, long time since I had to post anything, been running a nice N4F v9-box for a long time.

However now one of my 3 WD Red 4Tb decided to die on me during a Scrub.

I have a 4 disk RaidZ 1 ie with 1 disk redundancy. I know it's not wha tis recommended today, but I haven't gotten round to change it.

However, when I tried to fix the situation, I accidentally pressed the delete Disk config and Import.

The remaining 3 disks then all started to resilver...ok, no crisis yet.

However to add to my lucky strike the power went during the resilver.....

....and the Nas4Free 9 install as well.

ATM I have replaced the faulty disk physically and have been running an ersatz N4Free v 12.

However when trying to figure out how to replace the disk I'm hitting some unexpected snags:

Result of Zpool import:

Code: Select all

 zpool import
   pool: pool
     id: 3679356042207428670
  state: FAULTED
 status: One or more devices were being resilvered.
 action: The pool cannot be imported due to damaged devices or data.
        The pool may be active on another system, but can be imported using
        the '-f' flag.
 config:

        pool                     FAULTED  corrupted data
          raidz1-0               FAULTED  corrupted data
            ada1                 ONLINE
            ada3                 ONLINE
            4075477775972449224  UNAVAIL  cannot open
            ada2                 ONLINE
When I try to force an import:

Code: Select all

 /mnt# zpool import -f pool
internal error: Unknown error: 122
Abort
(I haven't found any mention of error 122 on the net, so no clues there.)

Also in Nas4Free WebGUI i entered Disks - Management and imported disks clearing the old information.
Yay, at least the new drive shows up, but it isn't formattable as ZFS Storage pool, option greyed out.

Now I'm pretty sure I'll get some Dataloss from the Power outage during the 3 disk resilver, but hopefully I can salvage something.

What I don't understand is that the Pool seems to be stuck in Device Resilvered/other system. And how to remedy that?
Also I don't know if I should try to Partition the new disk and just try zpool replace the old?

Any caveats?

Is the pool crapped out no matter what I do? If there is any way to mount it to get the most important files off to my SSD, then that'd a least be a consolation.

Would really appreciate any good inputs if you're a ZFS-wiz.

User avatar
HorzaSe
NewUser
NewUser
Posts: 7
Joined: 25 Jun 2012 13:38
Status: Offline

Re: Replacing Disk woes

#2

Post by HorzaSe » 31 Oct 2019 22:17

Been searching a lot on the net.

Unearthed some clues.

Seems the Seagate drive which is supposed to replace the ADA0 is impossible to attach without onlining the Pool.
But the pool can't be imported due to Disk Unavail....
Tried importing as ReadOnly to see if I could offline and export the disks and the reimport but no success.

The Orig faulty disk ADA0 is bonafide dead so no real point in putting it back in I guess. Couldn't be detected before resilvering.

Code: Select all

horza: /mnt# zpool import -f
   pool: pool
     id: 3679356042207428670
  state: FAULTED
 status: One or more devices were being resilvered.
 action: The pool cannot be imported due to damaged devices or data.
        The pool may be active on another system, but can be imported using
        the '-f' flag.
 config:

        pool                     FAULTED  corrupted data
          raidz1-0               FAULTED  corrupted data
            ada1                 ONLINE
            ada3                 ONLINE
            4075477775972449224  UNAVAIL  cannot open
            ada2                 ONLINE
            
 horza: /mnt# zpool replace pool ada0
cannot open 'pool': no such pool
horza: /mnt# zpool import -F pool
cannot import 'pool': one or more devices is currently unavailable
horza: /mnt# zpool import -o readonly=on pool
internal error: Unknown error: 122
In WebGUI:

Pools : VDEV : pool_raidz1_0 raidz1
Pools : Management : pool Unknown Unknown Unknown Unknown Unknown Unknown Unknown Unknown
Pools : Information : no pools available
Pools : Virtual Device : ada1 Entire Device WDC WD40EFRX-68WT0N0 WD-WCC4E0021450 4.00TB ata3 (Promise PDC40718 SATA300 controller) Western Digital Red Virtual device is already in use.
ada3 Entire Device WDC WD40EFRX-68WT0N0 WD-WCC4E0028369 4.00TB ata5 (Promise PDC40718 SATA300 controller) Western Digital Red Virtual device is already in use.
ada0 8TB Virtual device is already in use.
ada2 Entire Device WDC WD40EFRX-68WT0N0 WD-WCC4E0050820 4.00TB ata4 (Promise PDC40718 SATA300 controller) Western Digital Red Virtual device is already in use.

Under Pools Management is it a good idea to use "Delete Selected Pools" to clr the old data or do I affect the data on the disks?
Under Pools Virtual Deviceis it a good idea to use "Delete Selected Virtual Devices" to clr the old data or do I affect the data on the disks?

User avatar
ms49434
Developer
Developer
Posts: 747
Joined: 03 Sep 2015 18:49
Location: Neuenkirchen-Vörden, Germany - GMT+1
Contact:
Status: Offline

Re: Replacing Disk woes

#3

Post by ms49434 » 31 Oct 2019 23:57

You have a severe issue with your pool and it is likely that the pool is lost.
Losing redundancy isn't your main issue because your pool would show as degraded but it would still be functional.
Your pool shows as faulted because of data corruption.

you can try to run zpool clear -F and/or zpool import -F and see if your pool can be made importable.
1) XigmaNAS 12.0.0.4 amd64-embedded on a Dell T20 running in a VM on ESXi 6.7U2, 22GB out of 32GB ECC RAM, LSI 9300-8i IT mode in passthrough mode. Pool 1: 2x HGST 10TB, mirrored, SLOG: Samsung 850 Pro, L2ARC: Samsung 850 Pro, Pool 2: 1x Samsung 860 EVO 1TB , services: Samba AD, CIFS/SMB, ftp, ctld, rsync, syncthing, zfs snapshots.
2) XigmaNAS 12.0.0.4 amd64-embedded on a Dell T20 running in a VM on ESXi 6.7U2, 8GB out of 32GB ECC RAM, IBM M1215 crossflashed, IT mode, passthrough mode, 2x HGST 10TB , services: rsync.

User avatar
HorzaSe
NewUser
NewUser
Posts: 7
Joined: 25 Jun 2012 13:38
Status: Offline

Re: Replacing Disk woes

#4

Post by HorzaSe » 01 Nov 2019 20:07

Thanks for the suggestion I'll try that.

Something which occurred to me is how ZFS keeps track of the system it was attached to?
Since I can't boot the USB with the orig Nas4Free 9.3 install, but I can run Xigma 11 which it detects as a different system?

I thought if I could fix the 9.3 maybe I could access the Pool again? Since it'd recognize the System?
A longshot, but I'm exploring all possibilities.

testmaster
NewUser
NewUser
Posts: 9
Joined: 12 Jul 2018 21:15
Status: Offline

Re: Replacing Disk woes

#5

Post by testmaster » 01 Nov 2019 23:58

And if this doesn't help you have one more change to get your pool imported by using:
zpool import -FX <pool_name>

The "X" switch isn't documented well - I don't know why, and it seems that anyone know's about it.
This function, as far as I have figured it out without any documents, is following:
When you start to try importing your pool ZFS searches the howl drives for some information about a pool that was created.
Maybe this will help.


PS:
I also have had a power fail some time agoe due to a thunderstorm, then I have had nearly the same problem that you have know.
No matter what I have tryed, my pool won't import.
A week later, when I lost all hope to get my data back, I have read the function "zpool import -X" (but no documentation, only a smal hint post anywhere on the net)
I tryed and it worked.

User avatar
ms49434
Developer
Developer
Posts: 747
Joined: 03 Sep 2015 18:49
Location: Neuenkirchen-Vörden, Germany - GMT+1
Contact:
Status: Offline

Re: Replacing Disk woes

#6

Post by ms49434 » 02 Nov 2019 17:10

Please try parameter -F first before using option -FX. 'X' forces a deep und probably unecessary extreme rewind of transactions.
1) XigmaNAS 12.0.0.4 amd64-embedded on a Dell T20 running in a VM on ESXi 6.7U2, 22GB out of 32GB ECC RAM, LSI 9300-8i IT mode in passthrough mode. Pool 1: 2x HGST 10TB, mirrored, SLOG: Samsung 850 Pro, L2ARC: Samsung 850 Pro, Pool 2: 1x Samsung 860 EVO 1TB , services: Samba AD, CIFS/SMB, ftp, ctld, rsync, syncthing, zfs snapshots.
2) XigmaNAS 12.0.0.4 amd64-embedded on a Dell T20 running in a VM on ESXi 6.7U2, 8GB out of 32GB ECC RAM, IBM M1215 crossflashed, IT mode, passthrough mode, 2x HGST 10TB , services: rsync.

User avatar
HorzaSe
NewUser
NewUser
Posts: 7
Joined: 25 Jun 2012 13:38
Status: Offline

Re: Replacing Disk woes

#7

Post by HorzaSe » 10 Nov 2019 23:06

Thanks for suggestions. ATM I'm waiting för my Salary to buy some new NAS-gear then I'll try to get it up and running again.

I'll post back when I had the chance to try out your suggestions, as well as some from another forum.

I would be soo happy if I could at least salvage some of the files on the array.

Post Reply

Return to “ZFS (only!)”