Unable to repair UFS drive

[racmar]

Hello,

I have a NAS4Free simple server for home share, with version 9.1.0.1.847. Only one 1.5 TB UFS drive with a few services like FTP, SMB, UPS...
This server was running for almost a year without any problem.

Disk has approximately 400GB of data on it. I was copying a few video files from my Windows 7 computer to a shared folder then the network share was not available anymore from Windows.
I connected a monitor to the server and saw an error message I did not wrote down but I needed to reboot the server since it was not responding at all.
After that, It would boot but stop at some point with an error about filesystem full or something like that.

I took the data HDD out and the server can boot ok. If I put it back in, I get the same error again.

Since I have another computer with Ubuntu, I installed the data drive in it and I can mount it as read only and I am able to access all the files on it. Nothing is lost.
So in that same computer with Ubuntu, I removed the Ubuntu system drive and put a spare drive on which I installed FreeBSD (PC-BSD) 10.0.

Here is what happens if I try to mount it with read/write if FreeBSD:

# mount -t ufs /dev/ada1p1 /mnt/test
mount: /dev/ada1p1: R/W mount of /mnt/HDD1500GB denied.
Filesystem is not clean - run fsck.
Forced mount will invalidate journal contents: Operation not permitted

so I ran fsck:

#fsck -t ufs /dev/ada1
** /dev/ada1
Cannot find file system superblock
ioctl (GCINFO): Inappropriate ioctl for device
fsck_ufs: /dev/ada1: can't read disk label

Tried another superblock (192) which was listed with the newfs command and it says it is not a system superblock.

I can still mount it as read only in FreeBSD and see all the data.

Is there a way to repair it without erasing data and partition ?

Thanks.

[kenZ71]

Do you have a recent data backup? Since you can mount the drive in a read only format
400GB of data would be easy to backup to a 500GB drive.

That would be my first step. Then either reformat the 1.5 TB or figure out the issue. What is the SMART status?

[racmar]

Hello kenZ71,
Data is already backed up on a portable drive from my work. I did it when it was mounted with Ubuntu. So I still have two good copies of my data with my router's drive.
SMART status is ok. A few programs I tried in FreeBSD all say that the drive is good. It seems only the first superblock is missing, maybe a little more but I can't find how to solve this for now. I spent a few hours on the net with only generic solutions where fsck was able to use another superblock and repair the drive. I don't have this option when I run fsck.
I want to find out what happened and how to solve it without reformating it. It happened once, it will happen again somehow. And I don't want to go through a reformat and data restore each time. There must be an easy way to solve this.
Thanks.

[ChriZathens]

Try to run

Code: Select all

fsck_ufs -y /dev/ada1p1

[b0ssman]

Tried another superblock (192) which was listed with the newfs command and it says it is not a system superblock.

please post the exact commands you used.

did you try another superblock after 192?

[b0ssman]

also pls post the smart values of the drive

[racmar]

Tried another superblock (192) which was listed with the newfs command and it says it is not a system superblock.

please post the exact commands you used.

did you try another superblock after 192?

Yes I did the next one in the list with same result, which was a big number.

Try to run

Code: Select all

fsck_ufs -y /dev/ada1p1

Did 'fsck_ufs /dev/ada1p1' and it corrected a few errors.
Did it twice to be sure.

Now FreeBSD can mount it read/write easily.
I deleted the last files I was copying when it happened.

Put it back in my NAS computer and boot went fine.
(how the hell did I not try the "p1" at the end..)

Server is online and I can access the system via web interface.

also pls post the smart values of the drive

Right out of NAS4Free:

Code: Select all

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Barracuda 7200.14 (AF)
Device Model:     ST1500DM003-9YN16G
Serial Number:    Z1E0X5AX
LU WWN Device Id: 5 000c50 04d8c9ab2
Firmware Version: CC4C
User Capacity:    1,500,301,910,016 bytes [1.50 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 1.5 Gb/s)
Local Time is:    Mon May  5 20:27:57 2014 EDT

==> WARNING: A firmware update for this drive is available,
see the following Seagate web pages:
http://knowledge.seagate.com/articles/en_US/FAQ/207931en
http://knowledge.seagate.com/articles/en_US/FAQ/223651en

SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00)	Offline data collection activity
					was never started.
					Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0)	The previous self-test routine completed
					without error or no self-test has ever
					been run.
Total time to complete Offline
data collection: 		(  609) seconds.
Offline data collection
capabilities: 			 (0x73) SMART execute Offline immediate.
					Auto Offline data collection on/off support.
					Suspend Offline collection upon new
					command.
					No Offline surface scan supported.
					Self-test supported.
					Conveyance Self-test supported.
					Selective Self-test supported.
SMART capabilities:            (0x0003)	Saves SMART data before entering
					power-saving mode.
					Supports SMART auto save timer.
Error logging capability:        (0x01)	Error logging supported.
					General Purpose Logging supported.
Short self-test routine
recommended polling time: 	 (   1) minutes.
Extended self-test routine
recommended polling time: 	 ( 201) minutes.
Conveyance self-test routine
recommended polling time: 	 (   2) minutes.
SCT capabilities: 	       (0x3085)	SCT Status supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   117   100   006    Pre-fail  Always       -       167695736
  3 Spin_Up_Time            0x0003   094   094   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   097   097   020    Old_age   Always       -       3400
  5 Reallocated_Sector_Ct   0x0033   100   100   036    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   100   253   030    Pre-fail  Always       -       639226
  9 Power_On_Hours          0x0032   089   089   000    Old_age   Always       -       10428
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       56
183 Runtime_Bad_Block       0x0032   100   100   000    Old_age   Always       -       0
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0 0 0
189 High_Fly_Writes         0x003a   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   071   057   045    Old_age   Always       -       29 (6 103 29 29 0)
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       32
193 Load_Cycle_Count        0x0032   096   096   000    Old_age   Always       -       8175
194 Temperature_Celsius     0x0022   029   043   000    Old_age   Always       -       29 (128 0 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       2642h+07m+06.044s
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       681042909153
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       14914355249

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%     10422         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

Can't see anything wrong. Maybe you do.
I think it was a write bug from NAS4Free.
I updated the system to 9.2.0.1.943. Hope it won't happen again.
I will copy again the files I was copying when the problem occured and see if it happens again.
Thanks a lot.

[b0ssman]

didnt see the p1 either

your smart values look fine.

the reason i asked from them is that some of the seagate 1.5tb drives are amongst the most unreliable drives.
http://blog.backblaze.com/2014/01/21/wh ... uld-i-buy/

[racmar]

the reason i asked from them is that some of the seagate 1.5tb drives are amongst the most unreliable drives.
http://blog.backblaze.com/2014/01/21/wh ... uld-i-buy/

Interresting. I'll check my SMART values more often, even if mine is not the "Green" one.
Thanks for the hint.

[racmar]

I came back to write down what most probably caused my problem.
Since then, I had problems again while copying many files in one shot or big files. Suddenly, the NAS server was rebooting in the middle of the process.
I could only copy files in small batches.
I decided to test my memory and one of the memory module had problems. I had 2GB. I removed the bad module which was a 512MB module. Now I only have 1.5GB.
No more problems since. Everything runs great. Tested a lot of operations and server does not reboot anymore.