Latest News:
2018-11-29: XigmaNAS 11.2.0.4.6229 - released!

We really need "Your" help on XigmaNAS https://translations.launchpad.net/xigmanas translations. Please help today!

Producing and hosting XigmaNAS cost money, please consider a donation to our project so we can continue to offer you the best.
We need your support! eg: PAYPAL

Helpful scripts: Backup,Snapshot,Standby,Scrub,CheckPools...

XigmaNAS Scripts and shell tips
Forum rules
Set-Up GuideFAQsForum Rules
MRVa
NewUser
NewUser
Posts: 5
Joined: 20 Jul 2014 19:59
Status: Offline

Re: Helpful scripts: Backup,Snapshot,Standby,Scrub,CheckPools...

#151

Post by MRVa » 31 Dec 2015 13:43

Bump. Anybody out there that can find my error?

kenZ71
Advanced User
Advanced User
Posts: 400
Joined: 27 Jun 2012 20:18
Location: Northeast, USA
Status: Offline

Re: Helpful scripts: Backup,Snapshot,Standby,Scrub,CheckPools...

#152

Post by kenZ71 » 31 Dec 2015 14:01

I am not familiar with this script but have the checkSpace.sh script running.

Are you able to run others like checkSpace.sh or the snapshot script? If yes that would tell you permissions are good and back to this particular one.
11.2-RELEASE-p3 | ZFS Mirror - 2 x 8TB WD Red | 28GB ECC Ram
HP ML10v2 x64-embedded on Intel(R) Core(TM) i3-4150 CPU @ 3.50GHz

Extra memory so I can host a couple VMs
1) Unifi Controller on Ubuntu
2) Librenms on Ubuntu

carloskar
Starter
Starter
Posts: 20
Joined: 28 Oct 2013 19:56
Status: Offline

Re: Helpful scripts: Backup,Snapshot,Standby,Scrub,CheckPools...

#153

Post by carloskar » 31 Dec 2015 16:07

MRVa wrote:...
Thanks,
Mike
You have spaces between the ip-addresses:

Code: Select all

-n 6:00,23:00,3,30,192.168.0.63 +192.168.0.55 +192.168.0.56 +192.168.0.62
Remove the spaces between the ip-address and the '+' and it will work.

Code: Select all

-n 6:00,23:00,3,30,192.168.0.63+192.168.0.55+192.168.0.56+192.168.0.62
11.2.0.4 - Omnius (revision 5748)
ASRock E3C226D2I, Intel(R) Celeron(R) CPU G1820 @ 2.70GHz, 8GB DDR3 ECC
ZFS main pool: Samsung SSD 850 EVO 1TB
ZFS backup pool: 3x WD10EFRX 1TB + 1x WD20EFRX 2TB, mirror, synced daily with main pool

MRVa
NewUser
NewUser
Posts: 5
Joined: 20 Jul 2014 19:59
Status: Offline

Re: Helpful scripts: Backup,Snapshot,Standby,Scrub,CheckPools...

#154

Post by MRVa » 01 Jan 2016 17:42

@ Carloskar:

Thank you for that -- and I can report partial success. When I ran the string as you suggested, the machine would boot, but would hang as it ran through the various command line instructions at:

Preparing web-based file management

I could still reach the gui, but this didn't seem a complete fix. It did, however, eliminate the original "no mandatory arguments" error email. If I removed the command script, boot completed normally.

Having tried self-help on BSD syntax (albeit unsuccessfully), I had previously noted that some strings of this sort commonly ended in a space followed by an ampersand. So I tried that . . . the the boot completed.

So now the sting is:

/mnt/Mushkin/DATA/Scripts/manageAcpi.sh -p 120 -w 300 -a 11:50,12:10 -c 23:00,6:00,3 -n 6:00,23:00,3,30,192.168.0.63+192.168.0.55+192.168.0.56+192.168.0.62 &

But we know it can't be that simple right? Entropy is stubborn, so I now have two new symptoms.

First, I noticed that the drive operating off of the secondary Sil Sata controller on this now-old Asus Athlon 64 era motherboard had a corrupted secondary file table. I cleared it and reformatted it, but there seem to be problems with the controller dropping the drive. I don't think this has anything to do with the command string, but I'll do some test boots with and without the string and with and without the hardware to see.

Second, the machine still doesn't sleep, but I think it's trying harder now. Alas its trying so hard, it has become self-crippling.

Finding the machine running this AM, I pulled up the command interface on the monitor, and found that it seems to disconnect from all manner of peripherals at some point, including the dell mouse or keyboard and other things that we normally like to have up and running so we can interact with the machine.

The command line report was:

uhub0: at usbus0, port 1, addr 1 (disconnected)
ugen0.2: <vendor 0x413c> at usbus0 (disconnected)
ugen0,3: <DELL> at usbus0 (disconnected)
ukbd0: at uhub0, port 4, addr 3 (disconnected)
uhub1: at usbus1, port 1, addr 1 (disconnected)
Uhub2: at usbus2, port 1, addr 1 (disconnected)

On reboot, the machine comes up normally. Will have to test to see what happens if/when it attempts to sleep again. Will also check my settings again when I have time. And I have captured screenshots of the gui-based log, which has some entries that may or may not be relevant (that way I can clear it and see what happens next time). I can post it in three parts if that would help, or if there's something specific I should look for, I can try to post the relevant portion.

In sum, I thank you very much for the help, and I expect to have some time next week to have another, more comprehensive look at things.
So, I'm not necessarily asking for additional help just yet, as I have more diagnostics to do. But in the meantime, if any of this plan looks either wrong-headed enough to avoid, or accidentally brilliant in some specific that I don't understand (is that ampersand at the end good or evil?), please do comment and keep me on the diagnostic straight and narrow. Because at present, I'm feel a bit like I'm trying to built a Tesla automobile using stone tools.

Happy New Year,
Mike
Last edited by MRVa on 01 Jan 2016 22:24, edited 1 time in total.

MRVa
NewUser
NewUser
Posts: 5
Joined: 20 Jul 2014 19:59
Status: Offline

Re: Helpful scripts: Backup,Snapshot,Standby,Scrub,CheckPools...

#155

Post by MRVa » 01 Jan 2016 18:53

OK, I had time for just a bit more experimentation before moving to other chores:

First, if I remove the space before the ampersand (on the theory that spaces were the issue before . . .), same symptoms. Machine will boot, but crashes when it tries to sleep.

Next, if i remove the ampersand and allow the boot to stop at "preparing web-based file management" -- the drives are (predictably) inaccessible across the network, the basic GUI works, but the machine still crashes when it tries to sleep, apparently with the same set of errors based on "disconnection" from everything at the command line.

So either there's another problem in the script somewhere, or I've got one of the other settings wrong. Just reporting this in case it helps with the diagnostics.

I really appreciate both the script and the help. Retiring Desktop machines to server duty this way really helps them earn their keep in the long run, and keeps all that old high tech in the system and out of the recycle pile.

So absent something I should try, I'm just going to remove the script until I have time, then I'll review everything, see if I can fix the problem drive/controller issue (which does seem unrelated) and then come back with a report.

carloskar
Starter
Starter
Posts: 20
Joined: 28 Oct 2013 19:56
Status: Offline

Re: Helpful scripts: Backup,Snapshot,Standby,Scrub,CheckPools...

#156

Post by carloskar » 01 Jan 2016 23:02

MRVa wrote:...
So now the sting is:

/mnt/Mushkin/DATA/Scripts/manageAcpi.sh -p 120 -w 300 -a 11:50,12:10 -c 23:00,6:00,3 -n 6:00,23:00,3,30,192.168.0.63+192.168.0.55+192.168.0.56+192.168.0.62 &
...
Happy New Year,
Mike
I'm sorry I didn't notice this before but your "-n" argument string is wrong (at least if you use v2.0 stable of the scripts that are linked to in the first post).
The correct argument format is

Code: Select all

-n ips,delay,acpi
so your string should be

Code: Select all

-n 192.168.0.63+192.168.0.55+192.168.0.56+192.168.0.62,30,3
The "-c" argument (curfew) handles the always off time, so implicitly the client-online-polling happens between 6:00 and 23:00 for you.
Note that the delay (second parameter which is '30') is in seconds.
The third parameter is '3' meaning S3 Standby or Suspend-To-RAM, does your MB support this mode and is it enabled in the BIOS?

And yes you must have the ampersand since this script never returns, did the same mistake when I first started to use this script :)
And another thing, you do run the script PostInit and not PreInit? PreInit is the default when creating a new command script.
11.2.0.4 - Omnius (revision 5748)
ASRock E3C226D2I, Intel(R) Celeron(R) CPU G1820 @ 2.70GHz, 8GB DDR3 ECC
ZFS main pool: Samsung SSD 850 EVO 1TB
ZFS backup pool: 3x WD10EFRX 1TB + 1x WD20EFRX 2TB, mirror, synced daily with main pool

MRVa
NewUser
NewUser
Posts: 5
Joined: 20 Jul 2014 19:59
Status: Offline

Re: Helpful scripts: Backup,Snapshot,Standby,Scrub,CheckPools...

#157

Post by MRVa » 02 Jan 2016 12:33

Oh, please don't apologize. I'm slowly learning all of this from a standing start, so I'm grateful for the help. It's a long script, so mistakes are easy to miss (and easier still to make). I'll look to your note and the original instructions and see where I went wrong.

I'd been running it postinit -- I think I recall that being consistent with the directions, but I'll check and have a look at that as well.

Yes, the Asus mainboard does support S3 Sleep, and it worked in windows. But I'll check the BIOS to be sure things are properly enabled.

Thanks for the assist!

Mike

Darth.75
NewUser
NewUser
Posts: 4
Joined: 15 Jan 2014 11:41
Status: Offline

Re: Helpful scripts: Backup,Snapshot,Standby,Scrub,CheckPools...

#158

Post by Darth.75 » 09 Feb 2016 21:49

Hi,

I run into a strange problem. I have two machines, both running the same version on NAS4free (10.2) en both running the same version of te scripts (v2). Server 1 (SERVER) is used as a fileserver, server 2 (BACKUP) is used as a backup server for SERVER.

On SERVER the scripts run fine as cron jobs. I get a logfile each night plus emails about power changes (S0 to S5 and vv).
On BACKUP none of the scripts run. In the log files a see errors like "ERROR Lock folder "./tmp/locks" could not be created". I manually created the lock folder under tmp, but that does not help. After this error, none of the scripts will function. That is, no log of tmp files are created.

The filesystem the scripts are in are owned by root on both servers, the groups are identical and the rights are as well.

At the moment, I am a bit our of clues on where to look. Do I need to make additional settings on the server side that I did on SERVER (which runs fine now for about 2 years) and forgot to do on BACKUP? Any help appreciated.

kenZ71
Advanced User
Advanced User
Posts: 400
Joined: 27 Jun 2012 20:18
Location: Northeast, USA
Status: Offline

Re: Helpful scripts: Backup,Snapshot,Standby,Scrub,CheckPools...

#159

Post by kenZ71 » 10 Feb 2016 01:44

Could not be created makes me think permissions.

The scripts need to run as root. Is that happening?
11.2-RELEASE-p3 | ZFS Mirror - 2 x 8TB WD Red | 28GB ECC Ram
HP ML10v2 x64-embedded on Intel(R) Core(TM) i3-4150 CPU @ 3.50GHz

Extra memory so I can host a couple VMs
1) Unifi Controller on Ubuntu
2) Librenms on Ubuntu

Darth.75
NewUser
NewUser
Posts: 4
Joined: 15 Jan 2014 11:41
Status: Offline

Re: Helpful scripts: Backup,Snapshot,Standby,Scrub,CheckPools...

#160

Post by Darth.75 » 10 Feb 2016 06:40

Hi kenZ71,

All scripts run as root. Root owns the scripts and the permissions are set to 770. That is why it puzzles me. Settings seem equal on BACKUP and SERVER. But I must be missing something.

defcon999
experienced User
experienced User
Posts: 115
Joined: 07 Dec 2013 10:55
Status: Offline

Re: Helpful scripts: Backup,Snapshot,Standby,Scrub,CheckPools...

#161

Post by defcon999 » 22 Apr 2016 10:29

Hello,

i'm using the Fritz-Scripts since my first day with NAS4Free ... all scripts working fine, but there is one big problem with the manageSnapshots.sh-script running every hour.

Every time i get several times the message
20160422_101900 ERROR Could not start script: Another instance is running or stopped abnormally
20160422_101900 ERROR In the latter case, please delete manually the corresponding lock: "/mnt/zfs/data/scripts/temp/locks/manageSnapshots.sh.zfs.lock"
But there is no lock-file to delete! And the script started ....

I could see, that the lock-file is present in the /temp/locks-folder during the script is running. When the script ends, the lock-file has been deleted.

I'm changing the scheduler to "1 hour and 25 minutes" to prevent conflicts with other scripts, but the mails still coming with the error-message.

All scripts have root-permissions and chmod 0770.

Any hint for this problem???

defcon999
NAS: HP MicroServer Gen8 - CPU: Intel Xeon E3-1230 V2 - QuadCore 3,3 GHz ** 16 GB ECC RAM ** 4 x 1 TB WD Red RaidZ2 ** Samsung 840 120 GB SSD Cache-Device ** 1 x 3 TB WD Red Rsync lokal via USB 3.0 Inateck USB 3.0 Dualschacht Festplatten-Dockingstation ** Cloning mit 1 x 3 TB WD RED im 2. Schacht der Docking-Station ** XigmaNAS-Version: 11.2.0.4 - Omnius (Revision 6229) ** embedded on JetFlash Transcend 16GB

ekr
NewUser
NewUser
Posts: 6
Joined: 18 Apr 2016 22:29
Status: Offline

Re: Helpful scripts: Backup,Snapshot,Standby,Scrub,CheckPools...

#162

Post by ekr » 28 Apr 2016 09:04

Hello Fritz,

first, a big "thumbs up" for your very nice set of scripts. I'm especially happy about your auto-snapshots.

Two questions:

- If I create my own snapshots (with a different name than yours), will your script ignore them when deleting old snapshots?
- Is it (easily) possible to change your script so that empty snapshots (i.e., with no changes to the previous snapshot) are removed immediately after creating them (or rather, to remove the _previous_ snapshot)? Or is that for some reason not a good idea, or quite hard to implement?

Thank you!

cdebuhr
NewUser
NewUser
Posts: 1
Joined: 12 May 2015 20:32
Status: Offline

Re: Helpful scripts: Backup,Snapshot,Standby,Scrub,CheckPools...

#163

Post by cdebuhr » 07 Oct 2017 01:11

Hello Fritz,

Thank you for these scripts! They appear to be exactly what I was looking for (manageSnapshots.sh and backupData.sh in particular). I've just got one issue ... I'm trying to backup datasets on my primary storage server to a backup server. The servers are very similar in configuration, right down to the names of the pools/datasets. When I invoke backupData.sh from the command line:

./backupData.sh -s root@192.168.0.27 Pool01/Misc Pool01/Misc

Nothing much happens, and I get the following in backupData.sh.log:

20171006_165920 INFO -------------------------------------
20171006_165920 ERROR SSH connection test successful.
20171006_165920 ERROR The source filesystem "Pool01/Misc" is in the same pool than the destination filesystem "Pool01/Misc"

Am I doing something wrong? It also occurs to me that perhaps the script requires the storage pool to have a different name, even when located on another server. Any comment?

Thanks again!
Chris

fritz
experienced User
experienced User
Posts: 87
Joined: 12 Dec 2012 16:40
Contact:
Status: Offline

Re: Helpful scripts: Backup,Snapshot,Standby,Scrub,CheckPools...

#164

Post by fritz » 04 Nov 2018 12:07

ekr wrote:
28 Apr 2016 09:04
Hello Fritz,

first, a big "thumbs up" for your very nice set of scripts. I'm especially happy about your auto-snapshots.

Two questions:

- If I create my own snapshots (with a different name than yours), will your script ignore them when deleting old snapshots?
- Is it (easily) possible to change your script so that empty snapshots (i.e., with no changes to the previous snapshot) are removed immediately after creating them (or rather, to remove the _previous_ snapshot)? Or is that for some reason not a good idea, or quite hard to implement?

Thank you!
Hi,

- Your snapshots shall not be affected by the scrript has long as you use another pattern for naming then.
- With ZFS it is possible to get the size of the snapshot. If size is 0, the snapshot does not contain any change and could be deleted. This change shouldn't be too. Complex. Please do not hesitate to contribute this feature with a pool request.

fritz
O/S: NAS4Free 11.1.0.4 - Atomics (revision 5017) (Embedded 64bit), installed on 8GB USB flash drive
https://github.com/fritz-hh

fritz
experienced User
experienced User
Posts: 87
Joined: 12 Dec 2012 16:40
Contact:
Status: Offline

Re: Helpful scripts: Backup,Snapshot,Standby,Scrub,CheckPools...

#165

Post by fritz » 04 Nov 2018 12:08

cdebuhr wrote:
07 Oct 2017 01:11
Hello Fritz,

Thank you for these scripts! They appear to be exactly what I was looking for (manageSnapshots.sh and backupData.sh in particular). I've just got one issue ... I'm trying to backup datasets on my primary storage server to a backup server. The servers are very similar in configuration, right down to the names of the pools/datasets. When I invoke backupData.sh from the command line:

./backupData.sh -s root@192.168.0.27 Pool01/Misc Pool01/Misc

Nothing much happens, and I get the following in backupData.sh.log:

20171006_165920 INFO -------------------------------------
20171006_165920 ERROR SSH connection test successful.
20171006_165920 ERROR The source filesystem "Pool01/Misc" is in the same pool than the destination filesystem "Pool01/Misc"

Am I doing something wrong? It also occurs to me that perhaps the script requires the storage pool to have a different name, even when located on another server. Any comment?

Thanks again!
Chris
Hi,

This is indeed a limitation of the script. The source and destination pools must have different names.
Do not hesitate to contribute a fix (pull-request) for this in github!

fritz
O/S: NAS4Free 11.1.0.4 - Atomics (revision 5017) (Embedded 64bit), installed on 8GB USB flash drive
https://github.com/fritz-hh

Post Reply

Return to “Scripts and shell tips”