*New 12.1 series Release:
2019-11-08: XigmaNAS 12.1.0.4.7091 - released!

*New 11.3 series Release:
2019-10-19: XigmaNAS 11.3.0.4.7014 - released


We really need "Your" help on XigmaNAS https://translations.launchpad.net/xigmanas translations. Please help today!

Producing and hosting XigmaNAS costs money. Please consider donating for our project so that we can continue to offer you the best.
We need your support! eg: PAYPAL

One HD left in my system ... ZFS or UFS recommenced ?

Forum rules
Set-Up GuideFAQsForum Rules
Post Reply
redline
Starter
Starter
Posts: 69
Joined: 29 Dec 2012 19:23
Status: Offline

One HD left in my system ... ZFS or UFS recommenced ?

#1

Post by redline » 28 Jan 2013 14:59

Hi
I have a problem , and asked in the german section for advice , but got no hints til now.
I have a RaidZ1 Pool with 4 devices which runs like a charm. During the build i used a 4TB hard disk in an external USB 3.0 housing.
In this enclsure a 4TB Hitachi harddisk was formatted with a UFS Filesystem.
Now i ripped this harddisk out of the housing (because i was too warm inside - 50 degrees) and planned to reuse it at the last internal Sata Port
in my well cooled NAS.

But after a few days trying i'm not able to create a new mountpoint for this UFS file system. The ufsid can't be found.
Running at the USB 3.0 port it's shown as 4k device. But at the Sata port it's listed as 512 Byte device. I can't find a way to fix this.
Googled up and down and tried several ways ... But no luck.

Meanwhile , reattached to the USB 3.0 port , i have copied the data to a safe place and can start with this harddisk from scratch.

And so the question comes up if it is maybe a wise decision to use ZFS on this single harddisk instead of UFS.
As you may understand i'm not amuuzed about this UFS port swapping behaviour and my trust goes til zero in terms of UFS.
I'm not a Freebsd profi and i don't have plans really to dive into this stuff deeper then needed.

If i would make a ZFS Pool of this singel harddisk i knew that i loose of the goodies a ZFS can offer. But wouldn't it be out of the box safer
then a UFS ? Because this ZFS Pool would have no redunancy and couldn't self heal i ask myself what happens if scrub finds someday
damaged areas. Will this single pool then immediatly be degraded and offlined or i can i still access all not damaged files for
a rescue copy ( the damaged files off course are lost )
I don't have an idea how ZFS behaves running in such a constellation and couldn't find an answer for this.

It would be really nice if someone can give me a tip if UFS or ZFS on a single disk is the way to go.
- ZFS way would offer me the possibility to make a mirror out of this single device pool (maybe in the future)

fsbruva
Advanced User
Advanced User
Posts: 383
Joined: 21 Sep 2012 14:50
Status: Offline

Re: One HD left in my system ... ZFS or UFS recommenced ?

#2

Post by fsbruva » 28 Jan 2013 15:57

In the short term, add the disk as a hot spare to the RaidZ pool. In the long term, add another 4TB drive and make a mirror.

Or, create a separate pool with just the 4TB disk, and only use it for rsync backups and such (or zfs snapshots). DO NOT add the 4TB drive to the pool that contains the raidz vdev, because zfs will write whatever data it wants there, with no redundancy.

The 512 byte device issue is to maintain compatibility with OS and filesystems that cannot deal with advanced formatting. The drive reports 512 bytes, but has 4096 byte physical sectors. When you format the drive, make sure you completely wipe it (per the wiki) to remove all old metadata. When you format it, ensure you choose advanced formatting. More than likely, this will create a GNOP device. Then, follow my guide to properly align your drive.
viewtopic.php?f=59&t=1494

redline
Starter
Starter
Posts: 69
Joined: 29 Dec 2012 19:23
Status: Offline

Re: One HD left in my system ... ZFS or UFS recommenced ?

#3

Post by redline » 28 Jan 2013 16:22

@fsbruva
>In the short term, add the disk as a hot spare to the RaidZ pool.
That's not really an option for me.

On the 4TB harddisk is no mission critical data. I have this stuff now backuped on several older smaller harddisk. This is backup enough for this kind of data.
I only want it accessable as a big collection on a single drive. I need no redunance here but i need the ability to recognize if there is a problem.
That's why i would prefer ZFS here. If the drive fails completly , i replace the disk and copy the stuff back from my external devices.

My question goes into another direction. ZFS offers at least the possibility , to report if there is a problem. I can scrub it frequently.
I don't know what happens without missing redunancy (mirror,raidz). Because there is nothing which could be repaired (I don't want to use copies=2 !!)
i would like to know if the error is only reported like -> "Terminator 2.mkv" = damaged !!
Then i can replace this single file maybe against a fresh one from the backup disk's

OR .... Is the pool then immediatly degraded , offlined and therfore not accessable anymore.

That's what i really like to know !!!!

If a frequently scrub on this pool would show up that i have a problem and which file is damaged i have the possibility to decide if i replace the file
or if it's time to replace th whole disk.

Regarding your ZFS alignment guide.
Indeed i found him for a few days and i after i had done every step my RaidZ pool was changed from ashift9 to ashift12. Really, thanx a lot for this guide !!

Btw .. Exact this behaviour of ZFS with the commands EXPORT and IMPORT pool i like. Exact such a solution i would now neeed for this UFS stuff.
Another reason to use the 4TB with a ZFS filesystem. I also tried to import my pool in 3 different motherboards as a test.
It works fine. And this shit UFS partition even didn't worked on the same mainboard at different ports.
This alone would be a reason for me to go ZFS single pool.

But i need to know how it behaves when a silent data corruption happens. If someone comes with a big hammer i have my backup harddisk anyway.

redline
Starter
Starter
Posts: 69
Joined: 29 Dec 2012 19:23
Status: Offline

Re: One HD left in my system ... ZFS or UFS recommenced ?

#4

Post by redline » 28 Jan 2013 16:29

@fsbruva
> When you format the drive, make sure you completely wipe it (per the wiki) to remove all old metadata

I don't know which Wiki entry you mean here exactly.
I found for a few days a script with the DD commando which wipes at the start and the end of the harddisk some metadata.

Is it this what you mean ?

fsbruva
Advanced User
Advanced User
Posts: 383
Joined: 21 Sep 2012 14:50
Status: Offline

Re: One HD left in my system ... ZFS or UFS recommenced ?

#5

Post by fsbruva » 28 Jan 2013 17:05

I think this is the script I meant: http://wiki.nas4free.org/doku.php?id=faq:0129

The pool and vdev would remain online, because zfs knows that only that file was affected:

# zpool status monkey
pool: monkey
state: ONLINE
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: http://www.sun.com/msg/ZFS-8000-8A
scrub: scrub completed after 0h0m with 8 errors on Tue Jul 13 13:17:32 2010
config:

NAME STATE READ WRITE CKSUM
monkey ONLINE 8 0 0
c1t1d0 ONLINE 2 0 0
c2t5d0 ONLINE 6 0 0

errors: 8 data errors, use '-v' for a list
If you follow the instruction and type with the verbose tag:

Code: Select all

zpool status -v
  pool: monkey
 state: ONLINE
status: One or more devices has experienced an error resulting in data
        corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
        entire pool from backup.
   see: http://www.sun.com/msg/ZFS-8000-8A
 scrub: scrub completed after 0h0m with 8 errors on Tue Jul 13 13:17:32 2010
config:

        NAME        STATE     READ WRITE CKSUM
        monkey      ONLINE       8     0     0
          c1t1d0    ONLINE       2     0     0
          c2t5d0    ONLINE       6     0     0

errors: Permanent errors have been detected in the following files: 

/monkey/a.txt
/monkey/bananas/b.txt
/monkey/sub/dir/d.txt
monkey/ghost/e.txt
/monkey/ghost/boo/f.txt

Code: Select all

If the object number to a file path cannot be successfully translated, either due to an error or because the object doesn't have a real file path associated with it, as is the case for a dnode_t, then the dataset name followed by the object's number is displayed. For example: monkey/dnode:<0x0>
If an object in the metaobject set (MOS) is corrupted, then a special tag of <metadata>, followed by the object number, is displayed.
Check out the zfs scripts in the scripts tips/tricks section. You could probably run an hourly pool check that would send no email when no errors. Then the email could send the list of busted files.

Once you get another 4TB drive, you can scrub the pool (verify it's error free), then attach the new drive, and you've got a mirror!

redline
Starter
Starter
Posts: 69
Joined: 29 Dec 2012 19:23
Status: Offline

Re: One HD left in my system ... ZFS or UFS recommenced ?

#6

Post by redline » 28 Jan 2013 17:17

Fantastisc Infos ... Thanx a lot.

Last question ...
I have jsut for a moment wiped the Harddisk and created over the Guid a Stripe with a single /dev/ada4 and choosed 4k advanced format,
Verifying with the zdb | grep ashift i see ashift = 12.
But there is a ada4.nop now inside.

Is it in this case nevertheless important to follow your guide and export, import the pool ? (only to get rid of the .nop)
I must admit that i followed the guide last time but don't really understand what all about Gnome, .nop etc etc.

redline
Starter
Starter
Posts: 69
Joined: 29 Dec 2012 19:23
Status: Offline

Re: One HD left in my system ... ZFS or UFS recommenced ?

#7

Post by redline » 28 Jan 2013 17:38

I followed again your guide ...

In this case i see two devices under /dev -> ad12.nop (that was previous USB 3.0) and ada4.nop (the free sata port)
After exporting, destroying and importing it's changed to ada4 as suggested. But after a restart of the NAS the .nop's reappear again.

With my Raidz Pool which consists of ada0, ada1, ada2 , ada3 the "un noped" namings stay constant after your guide.
I don't know if this is important.

fsbruva
Advanced User
Advanced User
Posts: 383
Joined: 21 Sep 2012 14:50
Status: Offline

Re: One HD left in my system ... ZFS or UFS recommenced ?

#8

Post by fsbruva » 28 Jan 2013 17:44

Do the .nop's appear as members of the vdev after reboot? Or just inside the /dev? The creation of the .nop device cannot be avoided for now. However, as long as the ada4 is the member of the vdev of the pool, you are fine.

redline
Starter
Starter
Posts: 69
Joined: 29 Dec 2012 19:23
Status: Offline

Re: One HD left in my system ... ZFS or UFS recommenced ?

#9

Post by redline » 28 Jan 2013 17:50

I'm not at home anymore. I will answer this in a few hours. Sorry

fsbruva
Advanced User
Advanced User
Posts: 383
Joined: 21 Sep 2012 14:50
Status: Offline

Re: One HD left in my system ... ZFS or UFS recommenced ?

#10

Post by fsbruva » 28 Jan 2013 18:10

Setup SSH! Putty doesn't need to be installed!

Just kidding - take your time.
To be clear:
ls /dev
will show the .nop.

zpool status should not show a .nop.

redline
Starter
Starter
Posts: 69
Joined: 29 Dec 2012 19:23
Status: Offline

Re: One HD left in my system ... ZFS or UFS recommenced ?

#11

Post by redline » 28 Jan 2013 20:04

Indeed. As you assumed the zpool status is without nop.
A /dev/ada* is with nop at ada4.nop and ad12.nop.
Just wondered why the same cure worked for a few days with the Raidz Pool permanent in both commando worlds ...

The Pool on the 4TB is now running with ZFS.
Thanx again for the clarification and the screenshots which shows exactly what i asked for.

fsbruva
Advanced User
Advanced User
Posts: 383
Joined: 21 Sep 2012 14:50
Status: Offline

Re: One HD left in my system ... ZFS or UFS recommenced ?

#12

Post by fsbruva » 28 Jan 2013 21:02

The gnop devices get created at boot time. Are you sure the didn't just last until the next reboot?

redline
Starter
Starter
Posts: 69
Joined: 29 Dec 2012 19:23
Status: Offline

Re: One HD left in my system ... ZFS or UFS recommenced ?

#13

Post by redline » 28 Jan 2013 21:15

But as long as i have the RaidZ running, i have booted several times and they allways appear without nop ...

I will take a closer look at this after the copy session and the next reboot.

fsbruva
Advanced User
Advanced User
Posts: 383
Joined: 21 Sep 2012 14:50
Status: Offline

Re: One HD left in my system ... ZFS or UFS recommenced ?

#14

Post by fsbruva » 28 Jan 2013 21:58

It's not that big a deal, honestly. One system I administer always creates the .nop devices, no matter what, but I still get very good performance with the non-.nop devices in the pool.

redline
Starter
Starter
Posts: 69
Joined: 29 Dec 2012 19:23
Status: Offline

Re: One HD left in my system ... ZFS or UFS recommenced ?

#15

Post by redline » 28 Jan 2013 23:52

> but I still get very good performance with the non-.nop devices in the pool.

Okay, it's fine enough for me. I started with 20 Mb writing and now i'm at 85 til 90 Mb/sec after a few weeks googling around
on my E-350 board.

It's now off topic but maybe you have a tip for me ...

The only point i'm missing with this NAS is caching directory entries. I have several directories with +1000 files in it.
For example under character "A" in the mp3 subdirectory. Especially when you access such a directory from a thin media
player it takes ages until it's listed first time (but well this would be okay) . Accessing again is much better but unfortunately if
you copy for example in the background , or after time, these cached infos are removed from memory. Same happens when
thumbnailing large picture directories.
Another point is when i want to check how large a directory tree is. It takes minutes which was in the past on a NTFS filesystem
done in seconds. Is there a way to force ZFS to store maybe specific directory trees in memory.

I'm not shure if an additional L2ARC SSD's could do this trick. Anyway my sata ports are now all in use.
Maybe you can hit me into the right direction ... I'm a bit lost and couldn't find infos or tricks into this topic.

Post Reply

Return to “ZFS (only!)”