Odd Issue Causing Loss of All Data on ZFS Pool

checkbin · Post by **checkbin** » 16 Jan 2017 20:40

I searched the forums but didn't see anything related to my issue. I have 3 ZPOOL RAID-z3 datasets, each with 11 3TB drives. Running Nas4Free version 9.3.0.2. Everything has been running great for several years now until yesterday, when I noticed I lost ALL my data on the 3rd dataset. The health of the pool is fine but all the data disappeared. I have the dataset also shared by SMB and I noticed the recycle bin is also gone now. I did not delete any data nor do anything to the pool or vdevs. All the drives show up still and the pool shows as healthy. I don't know how all of my nearly 25TB of data could be gone but it appears to be. Since I didn't delete any files and oddly the .recycle directory of the SMB share is even gone, I'm hoping something strange is going on and my data is really still there.

Does anyone have any ideas of what may have happened and how I can recover all my data? And no, foolish me I don't have snapshots enabled.

Thanks in advance for any thoughts or suggestions.

ku-gew · Post by **ku-gew** » 16 Jan 2017 22:26

Try
zpool history
to see what happened.
http://docs.oracle.com/cd/E19253-01/819 ... index.html

checkbin · Post by **checkbin** » 16 Jan 2017 22:43

zpool history showed no activity on the Pool since 2015 when I last swapped out a drive.

substr · Post by **substr** » 24 Jan 2017 23:52

Try zfs list -r poolname

And see if the dataset is there.

It may not be mounted, or may be mounted somewhere else.

checkbin · Post by **checkbin** » 28 Jan 2017 15:58

# zfs list -r VPOOL3
NAME USED AVAIL REFER MOUNTPOINT
VPOOL3 516M 21.5T 442K /mnt/VPOOL3
VPOOL3/VDATASET3 7.72M 21.5T 7.72M /mnt/VPOOL3/VDATASET3

So is all hope lost for getting any data back? The thing that worries me most (aside from losing all my data) is that I still don't know how this happened and fear it will occur again. I know ZFS is not a backup solution, but with raid-z3 I hope to minimize the chance of ata loss but it didn't help in this case. Doing full backups of 25G of data on each of 3 pools to another media is not an easy task.

Post by **crowi** » 28 Jan 2017 16:17

Data usually does not disappear without a hardware failure, looks like you or anyone else deleted the stuff somehow by mistake.
Even with RAIDZ3 you need a backup, obviously :-/

checkbin · Post by **checkbin** » 28 Jan 2017 16:40

Yes it does seem to appear that's what happened, I just don't know how it did. I only access the files via CIFS/Samba thru shares on my Windows 10 PC. When I do delete a file, it always shows up in the pool's .recycle directory. In this failure, every single directory was gone (probably 50 or more of them) and dozens of files, plus the .recycle directory itself was gone. So I don't know how I could have accidentally deleted all of that without k knowing it (and I'm the only user of the system). Unless some program somehow did it, which is a scary thought too. I did run virus and malware scanners too and they didn't find anything. I'm at a total loss here (literally and figuratively).

XigmaNAS

Odd Issue Causing Loss of All Data on ZFS Pool

Odd Issue Causing Loss of All Data on ZFS Pool

Re: Odd Issue Causing Loss of All Data on ZFS Pool

Re: Odd Issue Causing Loss of All Data on ZFS Pool

Re: Odd Issue Causing Loss of All Data on ZFS Pool

Re: Odd Issue Causing Loss of All Data on ZFS Pool

Re: Odd Issue Causing Loss of All Data on ZFS Pool

Re: Odd Issue Causing Loss of All Data on ZFS Pool