*New 11.4 series Release:
2020-07-03: XigmaNAS 11.4.0.4.7633 - released!

*New 12.1 series Release:
2020-04-17: XigmaNAS 12.1.0.4.7542 - released


We really need "Your" help on XigmaNAS https://translations.launchpad.net/xigmanas translations. Please help today!

Producing and hosting XigmaNAS costs money. Please consider donating for our project so that we can continue to offer you the best.
We need your support! eg: PAYPAL

Raid5 won't mount after replacing a hard drive

Software RAID information and help
Forum rules
Set-Up GuideFAQsForum Rules
Post Reply
yzx
NewUser
NewUser
Posts: 5
Joined: 22 Feb 2013 10:07
Status: Offline

Raid5 won't mount after replacing a hard drive

#1

Post by yzx »

Hi All,

I am a new NAS4Free user, and need some help. I just replace a hard drive that was reported to have uncorrectable sector by SMART with a new drive (same brand, same type, and same size). I followed the two-steps procedure from here http://wiki.nas4free.org/doku.php?id=do ... aid5_array and my Raid5 status was changed from DEGRADED to REBUILDING (currently 11%). But when i tried to remount the array, the status return with "Error - Retry".
At the Disks|Management, there is this error message "there is wrong disk id in the config. please remove the disk and re-add it or use 'clear and import'" and the new added drive was marked in red although it's status was online. I followed the error message to "clear and import", first I click the "Clear config and import disks", then i found out the the disks was imported as individual disk not as RAID, so i click the "Clear config and import software raid disks", and was able to get rid of the error message and the new drive was no longer marked in red. But after that the raid still wouldn't be mounted. Do i have to wait until the rebuilding to complete to be able to mount the array or is there something wrong with the array? Any insights are greatly appreciated.

User avatar
ChriZathens
Forum Moderator
Forum Moderator
Posts: 799
Joined: 23 Jun 2012 09:14
Location: Athens, Greece
Contact:
Status: Offline

Re: Raid5 won't mount after replacing a hard drive

#2

Post by ChriZathens »

Although it is a good idea to let rebuilding finish before mounting/writing again to the array, it is not mandatory... It should mount without issues..
Can you try to mount it via CLI and see if the error there is more informative?
I am currently not using SoftRaid5, but I think that a command mount -a could do the job.
Examine /etc/fstab to see what mount points are listed there...If you like post its contents here after opening it with Advanced|File Editor
My Nas
  1. Case: Fractal Design Define R2
  2. M/B: Supermicro x9scl-f
  3. CPU: Intel Celeron G1620
  4. RAM: 16GB DDR3 ECC (2 x Kingston KVR1333D3E9S/8G)
  5. PSU: Chieftec 850w 80+ modular
  6. Storage: 8x2TB HDDs in a RaidZ2 array ~ 10.1 TB usable disk space
  7. O/S: XigmaNAS 11.2.0.4.6625 -amd64 embedded
  8. Extra H/W: Dell Perc H310 SAS controller, crosflashed to LSI 9211-8i IT mode, 8GB Innodisk D150SV SATADOM for O/S

Backup Nas: U-NAS NSC-400, Gigabyte MB10-DS4 (4x4TB Seagate Exos disks in RaidZ configuration - 32GB RAM)

yzx
NewUser
NewUser
Posts: 5
Joined: 22 Feb 2013 10:07
Status: Offline

Re: Raid5 won't mount after replacing a hard drive

#3

Post by yzx »

Hi Chrizathens, thanks for your reply.

This is the contents of /etc/fstab:
/dev/da0a /cf ufs ro 1 1
proc /proc procfs rw 0 0

So the softRAID5 mount point is not stated in the fstab.

This is the mount point stated in Disks|Mount Point :
/dev/ufsid/5121b2e2678c5726 (raid5/RAID5p1)

This is the information from Diagnostics|Log :
Feb 23 13:53:08 pandoras-box2 root: mount: /dev/ufsid/5121b2e2678c5726: No such file or directory
Feb 23 13:53:08 pandoras-box2 root: Error: Failed to mount /dev/ufsid/5121b2e2678c5726 using UFS

I tried mounting manually from NAS shell with this command "mount -t UFS /dev/raid5/RAID5p1 /mnt/data" and "mount -t UFS /dev/dev/ufsid/5121b2e2678c5726 /mnt/data", both commands returns the same error message "Operation not supported by device".

Did I screw up something? Do i have to format the new disk before insert it to the array? Because I didn't format the disk as the user guide from wiki.nas4free.org didn't mention it, so I just installed the new disk i bought to the array and then run the command insert from Disk|Software RAID|Geom Raid5|Tools.

And one more thing, today the rebuilding process seems to have restarted. yesterday it was 11%, but today it is 0.3%, then i reboot the system and now it's 0%. This is the information from Disks|Software RAID|RAID5|Information :

Geom name: RAID5
State: REBUILDING CALM
Status: Total=4, Online=4
Type: AUTOMATIC
Pending: (wqp 0 // 0)
Stripesize: 131072
MemUse: 655360 (msl 4)
Newest: 0
ID: 189793942
Providers:
1. Name: raid5/RAID5
Mediasize: 9001778675712 (8.2T)
Sectorsize: 512
Mode: r0w0e0
Consumers:
1. Name: ada3
Mediasize: 3000592982016 (2.7T)
Sectorsize: 512
Stripesize: 4096
Stripeoffset: 0
Mode: r1w1e1
DiskNo: 3
Error: No
2. Name: ada2
Mediasize: 3000592982016 (2.7T)
Sectorsize: 512
Stripesize: 4096
Stripeoffset: 0
Mode: r1w1e1
DiskNo: 2
Error: No
3. Name: ada1
Mediasize: 3000592982016 (2.7T)
Sectorsize: 512
Stripesize: 4096
Stripeoffset: 0
Mode: r1w1e1
DiskNo: 1
Error: No
4. Name: ada0
Mediasize: 3000592982016 (2.7T)
Sectorsize: 512
Stripesize: 4096
Stripeoffset: 0
Mode: r1w1e1
Synchronized: 12986744832 / 0% (p:92)
DiskNo: 0
Error: No

and btw, the new disk is ada2, but the synchronization seems to run on ada0

yzx
NewUser
NewUser
Posts: 5
Joined: 22 Feb 2013 10:07
Status: Offline

Re: Raid5 won't mount after replacing a hard drive

#4

Post by yzx »

I have a config backup of the system before I click the "Clear config and import disks" button. I have restored the config to the system, and now it is in the condition right after I installed the new disk and before the "clear config and import disks". This is the screenshot of the wrong disk ID error:
screenshot.JPG
And there is another thing that I forgot to mention in my first email. It happened that after I unplugged the ethernet cable because I don't want the system to be accessible from the network and I came back the next day to the office and plug in the cable back, the system was hang at boot with this error:
GEOM-RAID5: RAID5 ada0(0): re-sync in progress: 0.55% p:728 ETA:33313min (cause: store verify progress)
Entropy harvesting: interrupts ethernet point_to_point kickstart.
Starting file system checks:
Unknown error; help!
ERROR: ABORTING BOOT (sending SIGTERM to parent)!
Feb23 14:15:13 init: /bin/sh on /etc/rc terminated abnormally / going to single user mode
Enter full pathname of shell or RETURN for /bin/sh:

I was able to get the system to complete booting by entering the path "/etc/rc". I tried entering "/bin/sh" but that would cause the system to reboot.
This is the system log after booting:

Feb 23 14:38:13 pandoras-box2 syslogd: kernel boot file is /boot/kernel/kernel
Feb 23 14:38:13 pandoras-box2 smartd[4750]: Device: /dev/ada0, WARNING: A firmware update for this drive is available,
Feb 23 14:38:13 pandoras-box2 smartd[4750]: see the following Seagate web pages:
Feb 23 14:38:13 pandoras-box2 smartd[4750]: http://knowledge.seagate.com/articles/e ... Q/207931en
Feb 23 14:38:13 pandoras-box2 smartd[4750]: http://knowledge.seagate.com/articles/e ... Q/223651en
Feb 23 14:38:13 pandoras-box2 smartd[4750]: Device: /dev/ada1, WARNING: A firmware update for this drive is available,
Feb 23 14:38:13 pandoras-box2 smartd[4750]: see the following Seagate web pages:
Feb 23 14:38:13 pandoras-box2 smartd[4750]: http://knowledge.seagate.com/articles/e ... Q/207931en
Feb 23 14:38:13 pandoras-box2 smartd[4750]: http://knowledge.seagate.com/articles/e ... Q/223651en
Feb 23 14:38:13 pandoras-box2 smartd[4750]: Device: /dev/ada2, WARNING: A firmware update for this drive may be available,
Feb 23 14:38:13 pandoras-box2 smartd[4750]: see the following Seagate web pages:
Feb 23 14:38:13 pandoras-box2 smartd[4750]: http://knowledge.seagate.com/articles/e ... Q/207931en
Feb 23 14:38:13 pandoras-box2 smartd[4750]: http://knowledge.seagate.com/articles/e ... Q/223651en
Feb 23 14:38:13 pandoras-box2 smartd[4750]: Device: /dev/ada3, WARNING: A firmware update for this drive may be available,
Feb 23 14:38:13 pandoras-box2 smartd[4750]: see the following Seagate web pages:
Feb 23 14:38:13 pandoras-box2 smartd[4750]: http://knowledge.seagate.com/articles/e ... Q/207931en
Feb 23 14:38:13 pandoras-box2 smartd[4750]: http://knowledge.seagate.com/articles/e ... Q/223651en
Feb 23 14:38:15 pandoras-box2 sshd[4780]: error: Bind to port 22 on :: failed: Address already in use.
Feb 23 14:38:15 pandoras-box2 sshd[4780]: error: Bind to port 22 on 0.0.0.0 failed: Address already in use.
Feb 23 14:38:15 pandoras-box2 sshd[4780]: fatal: Cannot bind any address.
Feb 23 14:38:15 pandoras-box2 root: /etc/rc: WARNING: failed to start lighttpd
Feb 23 14:38:16 pandoras-box2 login: login on ttyv0 as root

I don't know what are those errors on sshd and lighttpd. I would really like to reinstall NAS4Free and setup from scratch, but can I do it without losing the data in my softRAID5? I hate to say that I don't have the data backup.
You do not have the required permissions to view the files attached to this post.

User avatar
ChriZathens
Forum Moderator
Forum Moderator
Posts: 799
Joined: 23 Jun 2012 09:14
Location: Athens, Greece
Contact:
Status: Offline

Re: Raid5 won't mount after replacing a hard drive

#5

Post by ChriZathens »

Reinstalling n4f will not harm your data. You can do it and use clear config and import raid5 to import the array. If you are using embedded, there might be a chance of a corrupt usb key or cf card. So a new installation in a new media device may be a good step.
But the rebuilding is an independent process which must be performed anyway, so let's try something else first:
From what I understand from your last post, and specifically this part:
Starting file system checks:
Unknown error; help!
the system is trying to do an fsck and aborts. This may be the reason why rebuilding stopped and why it indicates that it needs so much time. I am not 100% sure, but it may also be the reason the mount point won't mount..
So, please go in disks|mount point, select to edit the mount point for your raid.
In there there is an option called File system check ( Enable foreground/background file system consistency check during boot process.), uncheck it..
Reboot again and it must indicate that raid5 will start rebuilding. My assumption is that the indicated time will be a lot shorter this time (if I am correct, the other time it was trying to both fsck and rebuild at the same time)
Let the rebuilding finish. With 3TB disks it will take a while...
After rebuilding finishes you must manually run fsck. To do this you must unmount your raid5, but this is your case anyways...

Code: Select all

fsck_ufs -y /dev/raid5/RAID5p1
Do this 2-3 times. After the first one it should state at the end
file system marked clean
, but run fsck again just to be safe (the second time it will be faster)
After finishing try again to mount your array. It will hopefully mount normally.
My Nas
  1. Case: Fractal Design Define R2
  2. M/B: Supermicro x9scl-f
  3. CPU: Intel Celeron G1620
  4. RAM: 16GB DDR3 ECC (2 x Kingston KVR1333D3E9S/8G)
  5. PSU: Chieftec 850w 80+ modular
  6. Storage: 8x2TB HDDs in a RaidZ2 array ~ 10.1 TB usable disk space
  7. O/S: XigmaNAS 11.2.0.4.6625 -amd64 embedded
  8. Extra H/W: Dell Perc H310 SAS controller, crosflashed to LSI 9211-8i IT mode, 8GB Innodisk D150SV SATADOM for O/S

Backup Nas: U-NAS NSC-400, Gigabyte MB10-DS4 (4x4TB Seagate Exos disks in RaidZ configuration - 32GB RAM)

yzx
NewUser
NewUser
Posts: 5
Joined: 22 Feb 2013 10:07
Status: Offline

Re: Raid5 won't mount after replacing a hard drive

#6

Post by yzx »

Hi ChriZathens, sory for my late reply, I was on leave and just got back to the ofice.

Today, the rebuilding has completed, but it is still error when i tried to mount.
Then i tried your suggestion to do fsck, both from the console shell or from webGUI, and they returns the same error message;
** /dev/ufsid/5121b2e2678c5726
BAD SUPER BLOCK: VALUES IN SUPER BLOCK DISAGREE WITH THOSE IN FIRST ALTERNATE
ioctl (GCINFO): Inappropriate ioctl for device
Successful
And when i reboot the console, I notice these messages too:
GEOM: ada0: corrupt or invalid GPT detected.
GEOM: ada0: GPT rejected -- may not be recoverable.
GEOM: ada1: corrupt or invalid GPT detected.
GEOM: ada1: GPT rejected -- may not be recoverable.
FYI again, the new hdd is ada2. Please help, I am really lost and don't know how to fix the problems.

User avatar
ChriZathens
Forum Moderator
Forum Moderator
Posts: 799
Joined: 23 Jun 2012 09:14
Location: Athens, Greece
Contact:
Status: Offline

Re: Raid5 won't mount after replacing a hard drive

#7

Post by ChriZathens »

I am not really sure that the /dev/ufsid/5121b2e2678c5726 part is correct
IIRC in my case it was /dev/raid5/MyRaidp1 where MyRaid is the name of your raid5 array. Please pay attention to the p1 at the end. I am not 100% sure, but I think that I had similar errors like your first quote when I was not selecting the one with p1 at the end.
About the second quote.. Ignore it
My Nas
  1. Case: Fractal Design Define R2
  2. M/B: Supermicro x9scl-f
  3. CPU: Intel Celeron G1620
  4. RAM: 16GB DDR3 ECC (2 x Kingston KVR1333D3E9S/8G)
  5. PSU: Chieftec 850w 80+ modular
  6. Storage: 8x2TB HDDs in a RaidZ2 array ~ 10.1 TB usable disk space
  7. O/S: XigmaNAS 11.2.0.4.6625 -amd64 embedded
  8. Extra H/W: Dell Perc H310 SAS controller, crosflashed to LSI 9211-8i IT mode, 8GB Innodisk D150SV SATADOM for O/S

Backup Nas: U-NAS NSC-400, Gigabyte MB10-DS4 (4x4TB Seagate Exos disks in RaidZ configuration - 32GB RAM)

yzx
NewUser
NewUser
Posts: 5
Joined: 22 Feb 2013 10:07
Status: Offline

Re: Raid5 won't mount after replacing a hard drive

#8

Post by yzx »

I mean both the command the "fsck_ufs -y /dev/raid5/RAID5p1" and the fsck tool from WebGUI returns the same thing about
"BAD SUPER BLOCK: VALUES IN SUPER BLOCK DISAGREE WITH THOSE IN FIRST ALTERNATE"

The full error message when using command "fsck_ufs -y /dev/raid5/RAID5p1" is:
** /dev/raid/RAID5p1
BAD SUPER BLOCK: VALUES IN SUPER BLOCK DISAGREE WITH THOSE IN FIRST ALTERNATE
ioctl (GCINFO): Inappropriate ioctl for device
fsck_ufs: /dev/raid/RAID5p1: Can't read disk label
And i have read the wiki http://wiki.nas4free.org/doku.php?id=faq:0047 about “GEOM: adn: corrupt or invalid GPT detected” and it says
"These errors are expected in SoftRAID5 arrays and cannot be fixed, they are normal."
According to the wiki it cannot be fixed, but I don't understand why it says "they are normal". Can you confirm that they cannot be fixed? If you do, then I am thinking of wiping the drive and set it up again, but this time 'm thinking of using RAID1 or ZFS mirror. I found out that the drive I am using (Seagate Barracuda) is not designed for NAS nor RAID, so i'm going to play it safe next time.

Post Reply

Return to “Software RAID”