Sorry we lost some posts because of database errors!

*New 12.1 series Release:
2020-09-01: XigmaNAS 12.1.0.4.7728 - released

*New 11.4 series Release:
2020-08-27: XigmaNAS 11.4.0.4.7718 - released!


We really need "Your" help on XigmaNAS https://translations.launchpad.net/xigmanas translations. Please help today!

Producing and hosting XigmaNAS costs money. Please consider donating for our project so that we can continue to offer you the best.
We need your support! eg: PAYPAL

Archiving data on premise solution?

Problems, solutions, software
Forum rules
Set-Up GuideFAQsForum Rules
Post Reply
bunk3m
Starter
Starter
Posts: 42
Joined: 16 May 2013 21:36
Status: Offline

Archiving data on premise solution?

#1

Post by bunk3m »

I've been reading through this forum and find that most of the questions are many years old.

After upgrading my XigmaNAS (nas4free) with larger drives (3 times), I am left with 6 drives of various sizes (2x4TB, 2x3TB, 2x1TB). The drives are a bit old but performing well. I upgrade the files because of capacity not due to r/w issues.

I'd like to archive the NAS files onto the drives but I'm having problems finding the right info to research this. I understand that many of you are using cloud storage for archive but while I'm looking into that, I'd like to also investigate an on premise solution (with drives stored off site eventually).

My thinking at the moment would be to buy a multi-disc USB enclosure, format the disks in ??? (not sure yet, ?perhaps ext3/4?) and tar the files to an archive across the disks?

But then I start to think about how those disks need to look like 1 big disk for tar to work. I don't know what you'd call a multi-disc volume that looks like 1 disk so I haven't been able to research this.

Or should I leave as individually mountable drives but then do I use a oneway rsync? But in this case, then how does rsync work with multiple drives? If it does? I haven't found this out yet either.

I've read in the forums a bit about the zfs ?snapshot? but I'd like something without zfs for the archive because I'm not quite convinced that if I needed to rescue files from this archive if zfs is universal enough to be able to mount on typical Linux or Mac systems?

Am I going in the right direction or is this a completely stupid quest?

Can you perhaps provide some suggestions on what you folks do for archiving data, what type of search terms I should use to research this more?

Thanks in advance for any suggestions and help.
9.1.0.1 - Sandstorm (revision 775)
Mirrored ZPool

cookiemonster
Advanced User
Advanced User
Posts: 291
Joined: 23 Mar 2014 02:58
Location: UK
Status: Offline

Re: Archiving data on premise solution?

#2

Post by cookiemonster »

There are two layers to consider. The first one is the media, in this case they are mechanical hard drives. They can be used together in groups and something needs to be able to manage them as a unit, being able to notice a problem with one or more and attempt to repair the groups. This is what RAID does. It can be hardware, like a raid card, an hba, or it can be software like soft raid. Search for "software raid". Loads of options.
Then the second layer, the filesystem(s). These sit on top, managing the data structures and don't care if the device it sits on one or many disks. That's the job of the raid controller, hardware or software.
ZFS is special in that is does both things. It is the raid in software or more technically, a volume manager AND the filesystem management. There are other choices like Ceph.

Then you need to consider the operating system to manage both. Here some choices are more universal than others.
For instance a raid card can be used by most major OSes. Software raid depends on OS support and not all OSes can use the filesystems natively. For instance you can attach your raid card and disks to a windows machine, but won't play nice with an ext3/4 filesystem.
Be aware however that mechanical disks can fail if left powered down for long periods of time, I don't have a number, do a search.
Main: Xigmanas 11.2.0.4 x64-full-RootOnZFS as ESXi VM with 24GB memory.
Main Host: Supermicro X8DT3 Memory: 72GB ECC; 2 Xeon E5645 CPUs; Storage: (HBA) - LSI SAS 9211-4i with 3 SATA x 1 TB in raidZ1, 1 x 3 TB SAS drive as single stripe, 3 x 4 TB SAS drives in raidZ1.
Spare1: HP DL360 G7; 6 GB ECC RAM; 1 Xeon CPU; 5 x 500 GB disks on H210i
Backup1: HP DL380 G7; 24 GB ECC RAM; 2 Xeon E5645 CPUs; 8 x 500 GB disks on IBM M1015 flashed to LSI9211-IT

bunk3m
Starter
Starter
Posts: 42
Joined: 16 May 2013 21:36
Status: Offline

Re: Archiving data on premise solution?

#3

Post by bunk3m »

Thank you @cookiemonster.
There is a lot to consider and to research.
I appreciate the guidance and will start on my quest.
Keep safe!
9.1.0.1 - Sandstorm (revision 775)
Mirrored ZPool

Post Reply

Return to “Data recovery and backups”