ZFS DEGRADED

[abe1232]

I have 4 3TB HDDs. One failed and was replaced. When I resilver the the system I get the following error. Can anyone help me figure this out? If nothing can be done I can wipe the system and recreate the pool but I'd rather not.


pool: RAID
state: DEGRADED
status: One or more devices has experienced an error resulting in data corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the entire pool from backup.
see: http://illumos.org/msg/ZFS-8000-8A
scan: resilvered 216G in 1h41m with 93 errors on Fri Aug 28 14:33:59 2015
config:

NAME STATE READ WRITE CKSUM
RAID DEGRADED 0 0 18.7K
raidz1-0 DEGRADED 0 0 74.6K
ada0 ONLINE 0 0 0 block size: 512B configured, 4096B native
ada2 ONLINE 0 0 7 block size: 512B configured, 4096B native
replacing-2 DEGRADED 0 0 0
1469890870143607193 UNAVAIL 0 0 0 was /dev/ada2
ada1 ONLINE 0 0 0 block size: 512B configured, 4096B native
ada3 ONLINE 0 0 0 block size: 512B configured, 4096B native

errors: Permanent errors have been detected in the following files:

RAID:<0x0>
RAID:<0x11a02>
RAID:<0x11a05>
RAID:<0x11b07>
RAID:<0x11a08>
RAID:<0x11a26>
RAID:<0x1202c>
RAID:<0x1202f>
RAID:<0x12032>
RAID:<0x12038>
RAID:<0x1203e>
RAID:<0x13841>
RAID:<0x11f4e>
RAID:<0x1195d>
RAID:<0x11d68>
RAID:<0x1396d>
RAID:<0x11e70>
RAID:<0x11d7a>
RAID:<0x11dc2>
RAID:<0x139c7>
RAID:<0x11dda>
RAID:<0x11def>

[b0ssman]

first this is an indication that more drives are failing.

please post the smart values of all of them.

the 0x means meta data error. this is quite bad.

[abe1232]

I ran the SMART test of each drive (# /usr/local/sbin/smartctl -a -d auto /dev/adaX), I've attached each terminal output as a zip file png's below.

[b0ssman]

posting smart values as png thats a new one.
Nas4free has a Gui for that under Diagnostics|Information

and makes it VERY hard to read.

ada2 has a failing now value.

[abe1232]

Should I replace the drive and resilver my system again?

[b0ssman]

you should replace the drive.

your zfs config seems broken.

you somehow managed to get 5 drives in a 4 drive array.

i dont know how to fix that.

[abe1232]

I'm a n00b, I don't know how I did that. It's only 4 drives and the embedded flash drive for the OS. But thank you, I'll turn the system off and replace the other HDD.

[b0ssman]

well when one of your drive failed your pool did not have any parity information left.

and your other drives had errors.

this is a good read on that.
http://www.zdnet.com/article/why-raid-5 ... g-in-2009/
raidz1 is roughtly raid5