This is the old XigmaNAS forum in read only mode,
it will taken offline by the end of march 2021!



I like to aks Users and Admins to rewrite/take over important post from here into the new fresh main forum!
Its not possible for us to export from here and import it to the main forum!

rsync and cron #1

Synchronize files & directories to/from XigmaNAS with minimal data transfer.
Forum rules
Set-Up GuideFAQsForum Rules
Post Reply
ijeffsc
Starter
Starter
Posts: 23
Joined: 05 Dec 2015 13:11
Location: NY, ON, Krabi Thailand
Status: Offline

rsync and cron #1

Post by ijeffsc »

This seems like a popular topic, cron and rsync does :-).

<editorial> I’m really liking nas4free so far in that I doubt I could have set up the basics of ZFS so quickly without. All that time saved though is being burned by tasks that would be pretty routine in a normal server config. Let’s see if I can make a contribution </editorial>

Synopsis: So is issue #1 a bug and should I submit it? There is a 2nd issue but it was a layer 8 problem for your reading enjoyment.

First Cron:

The cron setup is pretty intuitive iff you know cron. Cron has always been subject to careful environment setup and I’ve usually handled that with scripts and/or absolute path names to executables. What is not intuitive (to me) is the behaviour of the “run now” GUI command and exactly what shell is running the script. As an example here is a simple touchme script:

Code: Select all

#!/usr/local/bin/bash

MYFILE=/tmp/mytouchmefile

rm $MYFILE
echo $SHELL > $MYFILE
# /usr/bin/touch $MYFILE
exit
When run from the command line it produces this result:

Code: Select all

[ijeff@nas4free ~/bin]$ ./touchme
[ijeff@nas4free ~/bin]$ ls -l /tmp/mytouchmefile ; cat /tmp/mytouchmefile 
-rw-r--r--  1 ijeff  wheel  10 Dec  6 09:48 /tmp/mytouchmefile
/bin/bash
When run from cron it produces this result:

Code: Select all

[ijeff@nas4free ~/bin]$ ls -l /tmp/mytouchmefile ; cat /tmp/mytouchmefile 
-rw-r--r--  1 ijeff  wheel  8 Dec  6 09:49 /tmp/mytouchmefile
/bin/sh
When run from the “run now” in the GUI it produces this:

Code: Select all

[ijeff@nas4free ~/bin]$ ls -l /tmp/mytouchmefile ; cat /tmp/mytouchmefile 
-rw-r--r--  1 root  wheel  10 Dec  6 09:52 /tmp/mytouchmefile
/bin/tcsh
There is a serious issue here in that the “run now” basically breaks (stops from running) the subsequent runs and has to be cleaned up before the regular cron run can function. These issues were documented in bug #97 although the permissions portion of the problem was not. There was not enough info given in #97 IMHO to reproduce the problem either.

There are two apparent problems.

#1 the file ownership is incorrect in the “run now” cron GUI case
#2 the shell is wrong in the cron case (it should be bash shouldn’t it?)

A bit more debugging and I realize that the second issue isn’t. I should have been looking a bit deeper into the SHELL variable which is the default shell. The new script is:

Code: Select all

#!/usr/local/bin/bash

MYFILE=/tmp/mytouchmefile
rm $MYFILE

ps -l >> $MYFILE
# /usr/bin/touch $MYFILE
exit
This results in cron out put of this (and clearly bash is running (is there a better way to do this?)):

Code: Select all

[ijeff@nas4free ~/bin]$ ls -l /tmp/mytouchmefile ; cat /tmp/mytouchmefile 
-rw-r--r--  1 ijeff  wheel  223 Dec  6 10:43 /tmp/mytouchmefile
 UID   PID  PPID CPU PRI NI   VSZ  RSS MWCHAN STAT TT     TIME COMMAND
1001 11120 11119   0  20  0 17640 3556 ttyin  Is+   0  0:00.29 -bash (bash)
1001 11587 11586   0  20  0 17640 3532 wait   Is    1  0:00.02 -bash (bash)
On the command line I see this:

Code: Select all

[ijeff@nas4free ~/bin]$ ./touchme
[ijeff@nas4free ~/bin]$ ls -l /tmp/mytouchmefile ; cat /tmp/mytouchmefile 
-rw-r--r--  1 ijeff  wheel  385 Dec  6 10:43 /tmp/mytouchmefile
 UID   PID  PPID CPU PRI NI   VSZ  RSS MWCHAN STAT TT     TIME COMMAND
1001 11120 11119   0  20  0 17640 3556 wait   Ss    0  0:00.30 -bash (bash)
1001 12148 11120   0  23  0 17640 3256 wait   S+    0  0:00.01 /usr/local/bin/bash ./touchme
1001 12150 12148   0  23  0 18760 2372 -      R+    0  0:00.00 ps -l
1001 11587 11586   0  20  0 17640 3532 wait   Is    1  0:00.02 -bash (bash)
The cron job is thus in config.xml:

Code: Select all

                <job>
                        <enable/>
                        <uuid>f8eab5f1-9f3d-427d-8329-60054e4ef462</uuid>
                        <desc>touchme</desc>
                        <minute/>
                        <hour/>
                        <day/>
                        <month/>
                        <weekday/>
                        <all_mins>1</all_mins>
                        <all_hours>1</all_hours>
                        <all_days>1</all_days>
                        <all_months>1</all_months>
                        <all_weekdays>1</all_weekdays>
                        <who>ijeff</who>
                        <command>/mnt/Pool1b/DS2Z/home/ijeff/bin/touchme</command>
                </job>
NAS4Free 10.3.0.3 (2617)
Intel Corporation D525MW 4GB
Promise SATA 300 TX4
HGST HTS721010A9E630 X 3 TOSHIBA MQ01ABD100 x2 + spare ZFS R5

User avatar
daoyama
Developer
Developer
Posts: 394
Joined: 25 Aug 2012 09:28
Location: Japan
Status: Offline

Re: rsync and cron #1

Post by daoyama »

Thank you for your inspection.
ijeffsc wrote: #1 the file ownership is incorrect in the “run now” cron GUI case
#2 the shell is wrong in the cron case (it should be bash shouldn’t it?)
It looks #2 is not a problem. /bin/sh should use shebang in your script.
ps display the process under terminal. But cron is not terminal of course.
(You need ps -U user or ps -a instead.)

#1 is a problem. It should use the user specified by Who field.
NAS4Free 10.2.0.2.2115 (x64-embedded), 10.2.0.2.2258 (arm), 10.2.0.2.2258(dom0)
GIGABYTE 5YASV-RH, Celeron E3400 (Dual 2.6GHz), ECC 8GB, Intel ET/CT/82566DM (on-board), ZFS mirror (2TBx2)
ASRock E350M1/USB3, 16GB, Realtek 8111E (on-board), ZFS mirror (2TBx2)
MSI MS-9666, Core i7-860(Quad 2.8GHz/HT), 32GB, Mellanox ConnectX-2 EN/Intel 82578DM (on-board), ZFS mirror (3TBx2+L2ARC/ZIL:SSD128GB)
Develop/test environment:
VirtualBox 512MB VM, ESXi 512MB-8GB VM, Raspberry Pi, Pi2, ODROID-C1

ijeffsc
Starter
Starter
Posts: 23
Joined: 05 Dec 2015 13:11
Location: NY, ON, Krabi Thailand
Status: Offline

Re: rsync and cron #1

Post by ijeffsc »

Would anyone care to submit this as a bug?
NAS4Free 10.3.0.3 (2617)
Intel Corporation D525MW 4GB
Promise SATA 300 TX4
HGST HTS721010A9E630 X 3 TOSHIBA MQ01ABD100 x2 + spare ZFS R5

ijeffsc
Starter
Starter
Posts: 23
Joined: 05 Dec 2015 13:11
Location: NY, ON, Krabi Thailand
Status: Offline

Re: rsync and cron #1

Post by ijeffsc »

submitted as bug #283.
NAS4Free 10.3.0.3 (2617)
Intel Corporation D525MW 4GB
Promise SATA 300 TX4
HGST HTS721010A9E630 X 3 TOSHIBA MQ01ABD100 x2 + spare ZFS R5

User avatar
zoon01
Developer
Developer
Posts: 724
Joined: 20 Jun 2012 21:06
Location: Netherlands
Contact:
Status: Offline

Re: rsync and cron #1

Post by zoon01 »

Thanks for bug report, please check pm
System specs: XigmaNAS 11.2.0.4 -embedded on Samsung 860 EVO 256GB and Supermicro X10SL7-F w / Bios v3.2, IPMI v.03.86 / CPU E3-1241 v3 @ 3.50GHz - 32GB Crucial DDR3L 1600mhz ECC 1.35v , LSI 2308 on PH20.00.07.00 IT mode, Storage: 5x Western Digital Red (WD30EFRX) raidz

Development system is same system in virtualbox.

ijeffsc
Starter
Starter
Posts: 23
Joined: 05 Dec 2015 13:11
Location: NY, ON, Krabi Thailand
Status: Offline

Re: rsync and cron #1

Post by ijeffsc »

Corrected in 10.3 thanks to dev team.
NAS4Free 10.3.0.3 (2617)
Intel Corporation D525MW 4GB
Promise SATA 300 TX4
HGST HTS721010A9E630 X 3 TOSHIBA MQ01ABD100 x2 + spare ZFS R5

Post Reply

Return to “RSYNC”