Experience with backup/archiving and 8TB drives

[erik]

Being a bit paranoid with securing the family archive I wanted to extend my single nas4free system to a two system master/slave setup and cloud backup.

I was looking for the biggest possible lowest power, lowest cost setup with protection against single disk failure for a NAS that is mostly used as a read only archive.
The Seagate archive drives are cheap but have a bad write reputation so I decided to test how it works in practice
My new configuration is a AMD A4-5000 (quad core 1.5GHz CPU) with 16GByte memory, with the 2 8TB drives active it uses less then 30 Watt.
To detect eventual memory corruption I set
vfs.zfs.debug = 0x10 (ZFS_DEBUG_MODIFY)

My old NAS (Xeon E3 with 16GByte ECC memory, 6 * 2TB in RAIDZ2) uses 65 Watt so this will become by backup machine.

I started with one 8TB drive and used zrep to send the zpool datasets to the new NAS. Worked well. Transfer speed started at 100MByte/s but after about 20 GByte the transfer speed suddenly dropped to 30MByte/s as the write buffer of the 8TB drive was full and the writing to the disk became the bottle neck.
After sufficient time the transfer completed for all datasets.
With zrep I switched the sender/receiver role and now zrep is used to keep the two machines in sync by waking the Xeon NAS once a day and sending all the changes.

Then I bought the second 8TB drive and added it to the first 8TB drive to create a zfs mirror. Building the mirror again showed the slow write speed but other then that no problems.

Using virtualbox I run two virtual machines, one that runs windows home server v1 for the backup of all PC's in the home. Windows home server v1 comes with the best backup software I could find. Deduplication, incremental backup, recovery of all versions through a network shares, simple warnings if a PC does not get backup up and good bare metal restore functionality. The second virtual machine runs windows 7 with crashplan so ALL zfs datasets are send to crashplan.
As all datasets are also at the backup nas I can run the virtual machines also there if the first nas breaks down.
Syncthing is used for realtime backup from all PC's to separate zfs datasets with hourly snapshot so I will never loose more then an hour work. It even works with my laptop outside the house.

And the performance of reading/writing to the NAS is excellent. More then 80MByte/s reading and writing as long as you do not send more then say 16Gbyte new files.

Because of the windows virtual machines and syncthing the 8TB drives never will go to sleep, but give the low power consumption of only 2 drives that is not a problem.

I'm happy with nas4free!

[Parkcomm]

Interesting.

I think the issue will hit with rewrites AND as the pool capacity exceeds 80%.

I assume you have not hit either of these limits yet. Keen to see what the performance looks like then.


Sent from my foam - stupid auto correct.

[Princo]

I started with one 8TB drive and used zrep to send the zpool datasets to the new NAS. Worked well. Transfer speed started at 100MByte/s but after about 20 GByte the transfer speed suddenly dropped to 30MByte/s as the write buffer of the 8TB drive was full and the writing to the disk became the bottle neck.

Nope, you're completely wrong.
You use "zrep" for transferring, and zrep relies on ssh. Ssh is very slow because of encryption.
So you are measuring ssh transfer speeed, and not disk transfer speed.
I use netcat with zfs send/receive, and gain a constant transfer speed of 1Gb/s between two HP Proliant N54L, an one ST8000AS0002 as target drive.

Regards
Princo

[erik]

See my post on "SSH speed improvement"

[Princo]

See my post on "SSH speed improvement"

I knew this post, it is very interesting.
Unfortunately, my HP Proliants didn't support AES in Hardware, and so i had to use netcat for transferring ZFS-Snapshots with full speed.

[erik]

I think the issue will hit with rewrites AND as the pool capacity exceeds 80%.

I assume you have not hit either of these limits yet. Keen to see what the performance looks like then.
.

Have been running for some time at 90% capacity. Indeed big performance decrease when writing large files. After purging some snapshots the capacity is back to 80%. Performance is ok again.