*New 11.3 series Release:
2019-10-05: XigmaNAS 11.3.0.4.6928 - released, 11.2 series are soon unsupported!

*New 12.0 series Release:
2019-10-05: XigmaNAS 12.0.0.4.6928 - released!

*New 11.2 series Release:
2019-09-23: XigmaNAS 11.2.0.4.6881 - released!

We really need "Your" help on XigmaNAS https://translations.launchpad.net/xigmanas translations. Please help today!

Producing and hosting XigmaNAS costs money. Please consider donating for our project so that we can continue to offer you the best.
We need your support! eg: PAYPAL

[EXTENSION] Rclone

XigmaNAS Extensions / Add-ons

Moderator: crest

Forum rules
Set-Up GuideFAQsForum Rules
User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

[EXTENSION] Rclone

#1

Post by crest » 05 Nov 2018 09:47

I'm very pleased to announce the extension Rclone (RCL) which lets users easily install and maintain the Rclone application and provides an easy to use interface which is seamless integrated into the XigmaNAS webgui (under Extensions > Rclone).

The extension
  • takes care about rclone application version check and upgrade
  • keeps track of rclone run status and execution results
  • enables creation/editing/removing of tasks
  • lets task directly starting on demand
  • add tasks to cron for scheduling
  • lets perform single rclone commands and output directly to the WebGUI page (refer to the Single Command screenshots at the bottom of this post)
  • works currently with x64 architecture (can be amended on demand)
  • works on all plattforms (embedded, full)
  • does not need jail or pkg install
  • add pages to XigmaNAS WebGUI extensions
  • includes extension maintenance (update & removal) without the need to work with the system console or shell CLI



INSTALLATION AND CONFIGURATION

Prior to the installation perform a backup of the XigmaNAS configuration via SYSTEM > BACKUP/RESTORE > Download configuration.

This extension can be easily installed with the OneButtonInstaller.
Update the extensions list manually prior to the installation if the switch Update extensions list automatically is not checked in OneButtonInstaller.

After successfull installation proceed to Extensions > Rclone
screen_20181117_061520.png
Rclone is now available to run (could be used also on the CLI, type rclone to see the available commands and options).
One can change now the path and file name of the rclone config file inside the extension to another location/share if one don't want to use it in the path /Your/Extension/Path/rclone/ (option Configuration File in the extension) and push 'Save' to store the setting.
For rclone CLI usage always use the additional parameters --config /mnt/DATA/extensions/rclone/rclone.conf and eventually --log-file /mnt/DATA/extensions/rclone/rclone.log.

Next step would be to
  • define one (or more) remotes - I attached the app RcloneBrowser (RCB) for MS Windows 64bit as well as the rclone Windows executable 64bit in /Your/Extension/Path/rclone/RcloneBrowser.
    One could run it directly from the server or copy the whole directory to a folder on your client PC and run it from there, no installation is required.
    Executables for Linux and macOS as well as installation instructions can be found at the RCB Github pages

    Running RCB for the first time it is required to setup the path to the Windows executable in the above mentioned directory and the rclone config file at your XigmaNAS server which can be found directly in /Your/Extension/Path/rclone/rclone.conf -> in RCB > Files > Preferences.
    Save the settings with OK and on the RCB main screen at the bottom hit Config to define your remotes.

    The process for this is really straight forward and in the Rclone documentation described for many supported cloud services.
    For some cloud services a browser window on your local machine will open to confirm to give access for Rclone to the service.

    After the definition of the remote(s) one could browse the remotes in RCB and perfom up/downloads etc but now we would define some tasks on the XigmaNAS server - therefore refresh the RClone extension page and the defined remote(s) should appear under Extensions > Rclone > Configuration > Defined Remotes.
  • Under Add Task provide a Name, Source, Destination, backup Mode and eventually additional Parameters - hit Save button to store the Definition.
  • Now we are ready to execute the task directly from the task list with Execute (the task will run in the background, status shows running and after finishing display the datetime of the last successful run or show failed if there were problems during execution - output will be written to rclone.log in the extension directory).
  • With Schedule to cron the appropriate commands will be written to the cron table to perform an execution of the task every night at 1:00 - change the cron job directly under System > Advanced > Cron to your needs (run daily/weekly etc ...).



RCLONE LOG VIEW

For users of the extension Extended GUI - to show the rclone log under Diagnostics > Log > Rclone one can use the sample code below, copy and paste to the file defined in Extensions > Extended GUI > Tools > User Defined Files > Logs.
Pls obey to change the 'logfile' path to the appropriate file path used for your rclone extension -> /mnt/Your/Extensions/Path/rclone/rclone.log :!:

Code: Select all

[
    'visible' => true,
    'desc' => gtext('Rclone'),
    'logfile' => "/mnt/DATA/extensions/rclone/rclone.log",
    'filename' => 'rclone.log',
    'type' => 'plain',
    "size" => '32768',
    'pattern' => '/^(\S+\s+\S+)\s+(\S+)\s+(\S+)\s+(.*)$/',
    'columns' => [
        ['title' => gtext('Date & Time'),'width' => '10%','hdrclass' => 'lhell','class' => 'lcell','param' => 'style="white-space: nowrap;"','pmid' => 1],
        ['title' => gtext('Level'),'width' => '5%','hdrclass' => 'lhell','class' => 'lcell','param' => 'style="white-space: nowrap;"','pmid' => 2],
        ['title' => gtext('Event'),'hdrclass' => 'lhebl','class' => 'lcebl','param' => '','pmid' => 4]
]],

Output in Diagnostic > Log > Rclone
screen_20181117_070523.png

To get a one-line condensed output of the rclone stats (when used the -v or -vv flag) in the log file one can append the flag --stats-one-line to the task - compare with the previous screenshot
screen_20181117_070251.png



TRANSLATIONS

Rclone is on Launchpad - if someone wants to contribute please go to Rclone on Launchpad for translation!

DISCLAIMER

This extension is provided AS-IS, I'm NOT responsible for any data loss or damage caused by the use of it, use it solely at your own risk.

VERSION HISTORY

Version 1.1 - 2018.11.16
minor GUI improvements
N: add 'mount' and 'check' commands to command dropdown
N: add task name to PID in Status for identification if more tasks are running simultaneously,
to see the names for already defined tasks they must be loaded to edit and just saved once again
N: add 'about' command (button) to 'Defined Remotes'
N: add 'Single Command' feature to perform on demand calls with output directly to the GUI page,
uses the 'Additional Parameters / Single Command' combined input field
N: add Greek translation
N: add Spanish translation
N: add Swedish translation
C: update German translation

Version 1.0 - 2018.11.05
further GUI improvements
N: add German translation

Version 0.2 - 2018.11.03
several GUI improvements
N: allow task editing
C: delete appropriate cron tab entries on task remove
F: browser button overlapped next field in some cases

Version 0.1 - 2018.10.30
initial release

N: ... new feature
C: ... changes
F: ... bug fix



Status overview
screen_20181117_061726.png


About defined remotes
screen_20181117_061901.png


Single Comand line
screen_20181117_062125.png


Output of Single Command
screen_20181117_062259.png




2DO
Currently the definition of remotes needs some extra work, there are efforts to create a rclone GUI in the rclone community - this is in a very early state but could solve some of the caveats for headless servers like XigmaNAS ...



Any feedback is highly appreciated!

Regards
crest
You do not have the required permissions to view the files attached to this post.
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

User avatar
Snufkin
Advanced User
Advanced User
Posts: 289
Joined: 01 Jul 2012 11:27
Location: Etc/GMT-3 (BSD style)
Status: Offline

Re: [EXTENSION] Rclone

#2

Post by Snufkin » 05 Nov 2018 12:31

crest wrote:
05 Nov 2018 09:47
The extension provides an easy to use interface which is seamless integrated into the XigmaNAS webgui (under Extensions > Rclone), takes care about rclone application version check and upgrade, keeps track of rclone run status and execution results, enables creation/editing/removing of tasks, lets task directly starting on demand or alternatively add tasks to cron for scheduling.
Great job, Crest thank you very much for your continuous efforts to expand XigmaNAS feature list.
All RCL extension features are implemented perfectly and work as expected.
crest wrote:
05 Nov 2018 09:47
2DO
Currently the definition of remotes needs some extra work, there are efforts to create a rclone GUI in the rclone community but this is in a very early state but could maybe solve some of the caveats for headless servers like XigmaNAS ...
Yes, creating a GUI for configuring rclone remotes is not an easy task.
Hopefully, rclone community combined efforts will lead to success.
XNAS 11.3.0.4 embedded, ASUS P5B-E, Intel DC E6600, 4 GB DDR2, 2 x HGST HDN726040ALE614, 2 x WDC WD5000AAKS, Ippon Back Power Pro 400

User avatar
JoseMR
Hardware & Software Guru
Hardware & Software Guru
Posts: 1151
Joined: 16 Apr 2014 04:15
Location: PR
Contact:
Status: Offline

Re: [EXTENSION] Rclone

#3

Post by JoseMR » 06 Nov 2018 07:58

Just another awesome extension crest, keep up the great work and support, definitely much appreciated by many.

Regards
System: FreeBSD 12 RootOnZFS, MB: Supermicro X8SI6-F, Xeon X3450, 16GB DDR3 ECC RDIMMs.
Addons at GitHub
JoseMRPubServ
Boot Environments Intro

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [EXTENSION] Rclone

#4

Post by crest » 06 Nov 2018 08:41

Thank you guys for the kind words :D

Many thanks also to Snufkin and ernie for their support, testing and inputs and to the translators of this extension - their exist already some (Greek, Swedish, ...) which will be added to the upcoming release.

Best regards
crest
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

Shperrung
experienced User
experienced User
Posts: 95
Joined: 04 Apr 2018 16:29
Status: Offline

Re: [EXTENSION] Rclone

#5

Post by Shperrung » 11 Nov 2018 06:57

Hi!
Rclone browser doesn't start in Windows 10 Home. Can you fix it?
Can you check what I'm doing wrong: - I have folder with binary rclone and 'rclone.conf' where are settings for my remote folders[attachment=0]Screenshot_2018-11-11-08-41-33-189_com.android.chrome.png
- I installed rclone from OBI and replaced path to my rclone.conf. After saving I see all my remotes in OBI-rclone.
- test upload from local folder to remote od2 (OneDrive) appears completed "succesdfully" but new files in remote od2 are not copied.
Screenshot_2018-11-11-08-42-20-215_com.android.chrome.png
Rclone binary (non-OBI) work fine with the same rclone.conf from CLI. I made script with variables related to config and log files and typical commands as doktornotor recommended.
You do not have the required permissions to view the files attached to this post.
11.2.0.4 - Omnius (revision 6177)
ASRock J3710-ITX, LAN: Realtek RTL8111GR; 16Gb RAM; WD 1Tbx2, WD 2Tb; UPS Powercom WOW500U.

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [EXTENSION] Rclone

#6

Post by crest » 11 Nov 2018 07:20

Hello Shperrung,
Shperrung wrote:
11 Nov 2018 06:57
Hi!
Rclone browser doesn't start in Windows 10 Home. Can you fix it?
Are there any errors messages? Try to copy the whole RcloneBrowser folder to your Windows PC and start RcloneBrowser.exe from that location.

Can you check what I'm doing wrong:
- I have folder with binary rclone and 'rclone.conf' where are settings for my remote folders[attachment=0]Screenshot_2018-11-11-08-41-33-189_com.android.chrome.png
- I installed rclone from OBI and replaced path to my rclone.conf. After saving I see all my remotes in OBI-rclone.
- test upload from local folder to remote od2 (OneDrive) appears completed "succesdfully" but new files in remote od2 are not copied. Screenshot_2018-11-11-08-42-20-215_com.android.chrome.png
As far as I can see on your screenshot - you must not pre-pend the remote with a slash "/"

Code: Select all

/od2:/test
since this leads to copy it to your server and not to the remote :!: use it without the leading slash

Code: Select all

od2:test

Rclone binary (non-OBI) work fine with the same rclone.conf from CLI. I made script with variables related to config and log files and typical commands as doktornotor recommended.
Pls can you refere to the recommended post ...

Regrads
crest
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [EXTENSION] Rclone

#7

Post by crest » 11 Nov 2018 08:44

For the integration of the Rclone log file to Diagnose > Log I added a code snippet to the first post of this thread, item RCLONE LOG VIEW
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

Shperrung
experienced User
experienced User
Posts: 95
Joined: 04 Apr 2018 16:29
Status: Offline

Re: [EXTENSION] Rclone

#8

Post by Shperrung » 11 Nov 2018 16:14

crest wrote:
11 Nov 2018 07:20
Hello Shperrung,
Shperrung wrote:
11 Nov 2018 06:57
Hi!
Rclone browser doesn't start in Windows 10 Home. Can you fix it?
Are there any errors messages? Try to copy the whole RcloneBrowser folder to your Windows PC and start RcloneBrowser.exe from that location.
RcloneBrowser folder was copied on pool shared via smb. Then I gave 777 permission to all files and tried to exexute. Nothing on my Windows 10 home.
You do not have the required permissions to view the files attached to this post.
11.2.0.4 - Omnius (revision 6177)
ASRock J3710-ITX, LAN: Realtek RTL8111GR; 16Gb RAM; WD 1Tbx2, WD 2Tb; UPS Powercom WOW500U.

Shperrung
experienced User
experienced User
Posts: 95
Joined: 04 Apr 2018 16:29
Status: Offline

Re: [EXTENSION] Rclone

#9

Post by Shperrung » 11 Nov 2018 18:21

crest wrote:
11 Nov 2018 07:20
Hello Shperrung,
As far as I can see on your screenshot - you must not pre-pend the remote with a slash "/"

Code: Select all

/od2:/test
since this leads to copy it to your server and not to the remote :!: use it without the leading slash

Code: Select all

od2:test
With slash /od2:/test this task creates folder od2/test in the local root.
Without slash od2:/test it returns error. Unfortunatelly I can't explain how to setup rclone log to be visible in Diagnostics - Log. Just to be clear: where I shoul past code: in the begining, at the end or instead exisiting?

Code: Select all

[
    'visible' => true,
    'desc' => gtext('Rclone'),
    'logfile' => "/mnt/DATA/extensions/rclone/rclone.log",
    'filename' => 'rclone.log',
    'type' => 'plain',
    "size" => '32768',
    'pattern' => '/^(\S+\s+\S+)\s+(\S+)\s+(.*)$/',
    'columns' => [
        ['title' => gtext('Date & Time'),'width' => '15%','hdrclass' => 'lhell','class' => 'lcell','param' => 'style="white-space: nowrap;"','pmid' => 1],
        ['title' => gtext('Level'),'width' => '10%','hdrclass' => 'lhell','class' => 'lcell','param' => 'style="white-space: nowrap;"','pmid' => 2],
        ['title' => gtext('Event'),'hdrclass' => 'lhebl','class' => 'lcebl','param' => '','pmid' => 3] 
Last edited by Shperrung on 11 Nov 2018 18:40, edited 1 time in total.
11.2.0.4 - Omnius (revision 6177)
ASRock J3710-ITX, LAN: Realtek RTL8111GR; 16Gb RAM; WD 1Tbx2, WD 2Tb; UPS Powercom WOW500U.

Shperrung
experienced User
experienced User
Posts: 95
Joined: 04 Apr 2018 16:29
Status: Offline

Re: [EXTENSION] Rclone

#10

Post by Shperrung » 11 Nov 2018 18:32

crest wrote:
11 Nov 2018 07:20
Rclone binary (non-OBI) work fine with the same rclone.conf from CLI. I made script with variables related to config and log files and typical commands as doktornotor recommended.
Pls can you refere to the recommended post ...

Regrads
crest
I'm sorry for multiple responses but I can reply on my mobile phone only. Here no possibility to reply in one post and keep quotas and format.
Here doktornotor shared sample of script for rclone. It defines cofiguration snd log files every time when started and execute a task. Using # I can write there all required commands with scpecified paths and quickly modify this script when need.
viewtopic.php?p=87376#p87376

Code: Select all

#!/bin/sh
export RCLONE_CONFIG=/path/to/rclone/config/file/rclone.conf
export RCLONE_LOG_FILE=/path/to/rclone/log/file/rclone.log
/path/to/rclone/binary/rclone copy local remote: 
11.2.0.4 - Omnius (revision 6177)
ASRock J3710-ITX, LAN: Realtek RTL8111GR; 16Gb RAM; WD 1Tbx2, WD 2Tb; UPS Powercom WOW500U.

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [EXTENSION] Rclone

#11

Post by crest » 12 Nov 2018 07:02

Shperrung wrote:
11 Nov 2018 16:14
crest wrote:
11 Nov 2018 07:20
Hello Shperrung,
Shperrung wrote:
11 Nov 2018 06:57
Hi!
Rclone browser doesn't start in Windows 10 Home. Can you fix it?
Are there any errors messages? Try to copy the whole RcloneBrowser folder to your Windows PC and start RcloneBrowser.exe from that location.
RcloneBrowser folder was copied on pool shared via smb. Then I gave 777 permission to all files and tried to exexute. Nothing on my Windows 10 home.
Do you use Windows 10 32 bit? If yes than you must download the 32bit version from RCB GitHub page AND the 32bit rclone executable (follow the links from the first post in this topic).
As already mentioned run it directly on your Windows PC!
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [EXTENSION] Rclone

#12

Post by crest » 12 Nov 2018 07:14

Shperrung wrote:
11 Nov 2018 18:21
crest wrote:
11 Nov 2018 07:20
Hello Shperrung,
As far as I can see on your screenshot - you must not pre-pend the remote with a slash "/"

Code: Select all

/od2:/test
since this leads to copy it to your server and not to the remote :!: use it without the leading slash

Code: Select all

od2:test
With slash /od2:/test this task creates folder od2/test in the local root.
This is what I mentioned ...
Without slash od2:/test it returns error.
Run it with debug infos -vv and take a look at the log which should show more infos.

Unfortunatelly I can't explain how to setup rclone log to be visible in Diagnostics - Log. Just to be clear: where I shoul past code: in the begining, at the end or instead exisiting?

Code: Select all

[
    'visible' => true,
    'desc' => gtext('Rclone'),
    'logfile' => "/mnt/DATA/extensions/rclone/rclone.log",
    'filename' => 'rclone.log',
    'type' => 'plain',
    "size" => '32768',
    'pattern' => '/^(\S+\s+\S+)\s+(\S+)\s+(.*)$/',
    'columns' => [
        ['title' => gtext('Date & Time'),'width' => '15%','hdrclass' => 'lhell','class' => 'lcell','param' => 'style="white-space: nowrap;"','pmid' => 1],
        ['title' => gtext('Level'),'width' => '10%','hdrclass' => 'lhell','class' => 'lcell','param' => 'style="white-space: nowrap;"','pmid' => 2],
        ['title' => gtext('Event'),'hdrclass' => 'lhebl','class' => 'lcebl','param' => '','pmid' => 3] 
As described under RCLONE LOG VIEW it's for Extended GUI which provides user defined logs.
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

Shperrung
experienced User
experienced User
Posts: 95
Joined: 04 Apr 2018 16:29
Status: Offline

Re: [EXTENSION] Rclone

#13

Post by Shperrung » 12 Nov 2018 10:59

Hi, crest!
I modified file in OBI and get Diagnostic-Log-Rclone but there are no alerts after my attempts to execute "test" even with -vv
I defined those rclone.conf and rclone.log that are in default folder. Both files have 777 permissions.
My Win10 is 32bit
You do not have the required permissions to view the files attached to this post.
11.2.0.4 - Omnius (revision 6177)
ASRock J3710-ITX, LAN: Realtek RTL8111GR; 16Gb RAM; WD 1Tbx2, WD 2Tb; UPS Powercom WOW500U.

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [EXTENSION] Rclone

#14

Post by crest » 13 Nov 2018 04:10

Shperrung wrote:
12 Nov 2018 10:59
Hi, crest!
I modified file in OBI and get Diagnostic-Log-Rclone but there are no alerts after my attempts to execute "test" even with -vv
I defined those rclone.conf and rclone.log that are in default folder. Both files have 777 permissions.
My Win10 is 32bit
Please provide the content of the task file /Your/Extension/Path/rclone/tasks/rclone-task-Test.sh

Do you see errors when you run the task file in the CLI?
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

User avatar
Snufkin
Advanced User
Advanced User
Posts: 289
Joined: 01 Jul 2012 11:27
Location: Etc/GMT-3 (BSD style)
Status: Offline

Re: [EXTENSION] Rclone log view

#15

Post by Snufkin » 13 Nov 2018 10:15

crest wrote:
11 Nov 2018 08:44
For the integration of the Rclone log file to Diagnose > Log I added a code snippet to the first post of this thread, item RCLONE LOG VIEW
Tested RCLONE LOG VIEW feature and found minor formatting errors.

Test case includes two RCL tasks with different verbosity level, 1st with option "-v", 2nd with option "-vv".
Below is original RClone log file and RClone log screenshot.

Option "-v"

Code: Select all

2018/11/13 11:38:32 INFO  : Yandex tmpdir: Waiting for checks to finish
2018/11/13 11:38:32 INFO  : Yandex tmpdir: Waiting for transfers to finish
2018/11/13 11:38:32 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 0
Checks:                 2 / 2, 100%
Transferred:            0 / 0, -
Elapsed time:        2.8s
Option "-vv"

Code: Select all

2018/11/13 11:38:37 DEBUG : rclone: Version "v1.44" starting with parameters ["/usr/local/bin/rclone" "--config" "/mnt/Test-NC/ex/rclone/rclone.conf" "--log-file" "/mnt/Test-NC/ex/rclone/rclone.log" "--checksum" "--no-update-modtime" "-vv" "copy" "/mnt/Test-NC/rclone-test" "yandexdisk:/tmpdir"]
2018/11/13 11:38:37 DEBUG : Using config file from "/mnt/Test-NC/ex/rclone/rclone.conf"
2018/11/13 11:38:38 DEBUG : fileintmpdir.txt: Size and MD5 of src and dst objects identical
2018/11/13 11:38:38 DEBUG : fileintmpdir.txt: Unchanged skipping
2018/11/13 11:38:39 DEBUG : tmpsubdir/fileintmpsubdir.txt: Size and MD5 of src and dst objects identical
2018/11/13 11:38:39 DEBUG : tmpsubdir/fileintmpsubdir.txt: Unchanged skipping
2018/11/13 11:38:39 INFO  : Yandex tmpdir: Waiting for checks to finish
2018/11/13 11:38:39 INFO  : Yandex tmpdir: Waiting for transfers to finish
2018/11/13 11:38:39 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 0
Checks:                 2 / 2, 100%
Transferred:            0 / 0, -
Elapsed time:        1.6s

2018/11/13 11:38:39 DEBUG : 5 go routines active
2018/11/13 11:38:39 DEBUG : rclone: Version "v1.44" finishing with parameters ["/usr/local/bin/rclone" "--config" "/mnt/Test-NC/ex/rclone/rclone.conf" "--log-file" "/mnt/Test-NC/ex/rclone/rclone.log" "--checksum" "--no-update-modtime" "-vv" "copy" "/mnt/Test-NC/rclone-test" "yandexdisk:/tmpdir"]
RClone log screenshot
rcl-log.jpg
Is there any way to change log file parsing parameters to fix formatting?
You do not have the required permissions to view the files attached to this post.
XNAS 11.3.0.4 embedded, ASUS P5B-E, Intel DC E6600, 4 GB DDR2, 2 x HGST HDN726040ALE614, 2 x WDC WD5000AAKS, Ippon Back Power Pro 400

Shperrung
experienced User
experienced User
Posts: 95
Joined: 04 Apr 2018 16:29
Status: Offline

Re: [EXTENSION] Rclone

#16

Post by Shperrung » 14 Nov 2018 06:39

crest wrote:
13 Nov 2018 04:10
Shperrung wrote:
12 Nov 2018 10:59
Hi, crest!
I modified file in OBI and get Diagnostic-Log-Rclone but there are no alerts after my attempts to execute "test" even with -vv
I defined those rclone.conf and rclone.log that are in default folder. Both files have 777 permissions.
My Win10 is 32bit
Please provide the content of the task file /Your/Extension/Path/rclone/tasks/rclone-task-Test.sh

Do you see errors when you run the task file in the CLI?
Hi!
rclone-task-Test.sh:

Code: Select all

 #!/bin/sh
# WARNING: THIS IS AN AUTOMATICALLY CREATED SCRIPT, DO NOT CHANGE THE CONTENT!
# Command for cron usage: /mnt/RAID/Ext/Extensions/rclone/tasks/rclone-task-Test.sh
logger 'Rclone task Test started'
/usr/local/bin/rclone --config '/mnt/RAID/rclone/rclone.conf' --log-file '/mnt/RAID/Ext/Extensions/rclone/rclone.log' -vv copy '/mnt/Pool/Documents/Favorites' 'od2:/Shperrung/test'
if [ $? == 0 ]; then
	logger 'Rclone task Test successfully finished'
	echo 'Test successfully finished' `date` > /mnt/RAID/Ext/Extensions/rclone/rclone_backup-date.txt
else
	logger 'Rclone task Test stopped with error(s)'
	echo '<font color='red'><b>Test stopped with error(s)</b></font>' `date` > /mnt/RAID/Ext/Extensions/rclone/rclone_backup-date.txt
fi
11.2.0.4 - Omnius (revision 6177)
ASRock J3710-ITX, LAN: Realtek RTL8111GR; 16Gb RAM; WD 1Tbx2, WD 2Tb; UPS Powercom WOW500U.

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [EXTENSION] Rclone log view

#17

Post by crest » 14 Nov 2018 06:46

Snufkin wrote:
13 Nov 2018 10:15
crest wrote:
11 Nov 2018 08:44
For the integration of the Rclone log file to Diagnose > Log I added a code snippet to the first post of this thread, item RCLONE LOG VIEW
Tested RCLONE LOG VIEW feature and found minor formatting errors.

Test case includes two RCL tasks with different verbosity level, 1st with option "-v", 2nd with option "-vv".
Below is original RClone log file and RClone log screenshot.
...
Is there any way to change log file parsing parameters to fix formatting?
For "-v" and "-vv" one kann add the flag --stats-one-line to Additional Parameters - I will add this flag as compulsory to the next release of RCL to get a consistent log view for all logs.
Also I modified the log view code snippet to get rid of the ":" in the log view, just cosmetics ;)
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

User avatar
Snufkin
Advanced User
Advanced User
Posts: 289
Joined: 01 Jul 2012 11:27
Location: Etc/GMT-3 (BSD style)
Status: Offline

Re: [EXTENSION] Rclone log view

#18

Post by Snufkin » 14 Nov 2018 08:03

crest wrote:
14 Nov 2018 06:46
For "-v" and "-vv" one kann add the flag --stats-one-line to Additional Parameters - I will add this flag as compulsory to the next release of RCL to get a consistent log view for all logs.
Also I modified the log view code snippet to get rid of the ":" in the log view, just cosmetics ;)
Please, note --stats-one-line option designed for "update the stats in a static block in the terminal" (see RClone option -P --progress). And when the output is redirected to the log file, we only get the first line from "a static block in the terminal" (see below log file).

Option "-v"

Code: Select all

2018/11/14 09:36:47 INFO  : Yandex tmpdir: Waiting for checks to finish
2018/11/14 09:36:47 INFO  : Yandex tmpdir: Waiting for transfers to finish
2018/11/14 09:36:47 INFO  :          0 / 0 Bytes, -, 0 Bytes/s, ETA -
Option "-vv"

Code: Select all

2018/11/14 09:36:51 DEBUG : rclone: Version "v1.44" starting with parameters ["/usr/local/bin/rclone" "--config" "/mnt/Test-NC/ex/rclone/rclone.conf" "--log-file" "/mnt/Test-NC/ex/rclone/rclone.log" "--checksum" "--no-update-modtime" "--stats-one-line" "-vv" "copy" "/mnt/Test-NC/rclone-test" "yandexdisk:/tmpdir"]
2018/11/14 09:36:51 DEBUG : Using config file from "/mnt/Test-NC/ex/rclone/rclone.conf"
2018/11/14 09:36:53 DEBUG : fileintmpdir.txt: Size and MD5 of src and dst objects identical
2018/11/14 09:36:53 DEBUG : fileintmpdir.txt: Unchanged skipping
2018/11/14 09:36:53 DEBUG : tmpsubdir/fileintmpsubdir.txt: Size and MD5 of src and dst objects identical
2018/11/14 09:36:53 DEBUG : tmpsubdir/fileintmpsubdir.txt: Unchanged skipping
2018/11/14 09:36:53 INFO  : Yandex tmpdir: Waiting for checks to finish
2018/11/14 09:36:53 INFO  : Yandex tmpdir: Waiting for transfers to finish
2018/11/14 09:36:53 INFO  :          0 / 0 Bytes, -, 0 Bytes/s, ETA -
2018/11/14 09:36:53 DEBUG : 5 go routines active
2018/11/14 09:36:53 DEBUG : rclone: Version "v1.44" finishing with parameters ["/usr/local/bin/rclone" "--config" "/mnt/Test-NC/ex/rclone/rclone.conf" "--log-file" "/mnt/Test-NC/ex/rclone/rclone.log" "--checksum" "--no-update-modtime" "--stats-one-line" "-vv" "copy" "/mnt/Test-NC/rclone-test" "yandexdisk:/tmpdir"]
XNAS 11.3.0.4 embedded, ASUS P5B-E, Intel DC E6600, 4 GB DDR2, 2 x HGST HDN726040ALE614, 2 x WDC WD5000AAKS, Ippon Back Power Pro 400

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [EXTENSION] Rclone log view

#19

Post by crest » 14 Nov 2018 08:41

Snufkin wrote:
14 Nov 2018 08:03
...
Please, note --stats-one-line option designed for "update the stats in a static block in the terminal" (see RClone option -P --progress). And when the output is redirected to the log file, we only get the first line from "a static block in the terminal" (see below log file).
...
The rclone docu --stats-one-line tells
--stats-one-line
When this is specified, rclone condenses the stats into a single line showing the most important stats only.
which seems to be fair enough because otherwise there is (at least atm) no other means to get a consistent log view by the rclone app which means IMHO that you could either life with ...
... and found minor formatting errors.
or get the condensed information in one row ;)
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [EXTENSION] Rclone

#20

Post by crest » 14 Nov 2018 08:47

Shperrung wrote:
14 Nov 2018 06:39
crest wrote:
13 Nov 2018 04:10
Shperrung wrote:
12 Nov 2018 10:59
Hi, crest!
I modified file in OBI and get Diagnostic-Log-Rclone but there are no alerts after my attempts to execute "test" even with -vv
I defined those rclone.conf and rclone.log that are in default folder. Both files have 777 permissions.
My Win10 is 32bit
Please provide the content of the task file /Your/Extension/Path/rclone/tasks/rclone-task-Test.sh

Do you see errors when you run the task file in the CLI?
Hi!
rclone-task-Test.sh:
...
The rclone command looks correct, did you see errors when you run the task file in the CLI and/or in the plain rclone.log file?
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [EXTENSION] Rclone

#21

Post by crest » 17 Nov 2018 06:58

Hello all,

there is a new release (v1.1) of this extension available => [EXTENSION] Rclone.
UPDATES CAN BE DONE DIRECTLY VIA THE WEBGUI (Extension Maintenance Tab).

For a list of changes take a look at the Version History.
The extension description and screenshots have been updated to reflect the changes in v1.1.

The translations base file on Launchpad has been updated - please proceed to Rclone on Launchpad for translation!
My special thanks goes to ChriZathens who supports the extension translations with light-speed, THANKS A LOT AGAIN :D ;)

If someone wants to contribute to extension translations please refer to this topic => Translations for Extensions

Regards
crest
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

User avatar
raulfg3
Site Admin
Site Admin
Posts: 4921
Joined: 22 Jun 2012 22:13
Location: Madrid (ESPAÑA)
Contact:
Status: Offline

Re: [EXTENSION] Rclone

#22

Post by raulfg3 » 17 Nov 2018 10:41

Translation into Spanish done.
12.0.0.4 (revision 6766)+OBI on SUPERMICRO X8SIL-F 8GB of ECC RAM, 12x3TB disk in 3 vdev in RaidZ1 = 32TB Raw size only 22TB usable

Wiki
Last changes

Shperrung
experienced User
experienced User
Posts: 95
Joined: 04 Apr 2018 16:29
Status: Offline

Re: [EXTENSION] Rclone

#23

Post by Shperrung » 17 Nov 2018 12:38

crest wrote:
14 Nov 2018 08:47
The rclone command looks correct, did you see errors when you run the task file in the CLI and/or in the plain rclone.log file?
Hello crest!
I found cause of my issue. Remotes in rclone.conf created in my binary rclone are not compatible with OBI-rclone.
I copied RcloneBrowser folder on shared pool (smb) and pasted my rclone.conf with remotes that doesn't work in OBI-rclone. They doesn't work here too. RBrowser can't open remote (OneDrive).
I created new remote for OneDrive using RcloneBrowser (Win10-64) and then successfully copied files in RcloneBrowser and the same did in OBI-rclone.

Thank you for this app!!
Last edited by Shperrung on 17 Nov 2018 13:20, edited 1 time in total.
11.2.0.4 - Omnius (revision 6177)
ASRock J3710-ITX, LAN: Realtek RTL8111GR; 16Gb RAM; WD 1Tbx2, WD 2Tb; UPS Powercom WOW500U.

sleid
PowerUser
PowerUser
Posts: 752
Joined: 23 Jun 2012 07:36
Location: FRANCE LIMOUSIN CORREZE
Status: Offline

Re: [EXTENSION] Rclone

#24

Post by sleid » 17 Nov 2018 13:15

Translation into french done.
12.1.0.4 - RC1 (revision 6999)
FreeBSD 12.1-RC1 #0 r353550M: Tue Oct 15 16:23:25 CEST 2019
X64-embedded sur Intel(R) Atom(TM) CPU C2750 @ 2.40GHz Boot UEFI
ASRock C2750D4I 2 X 8GB DDR3 ECC
Pool of 2 vdev Raidz1: 3 x WDC WD40EFRX + 3 x WDC WD30EZRX

Shperrung
experienced User
experienced User
Posts: 95
Joined: 04 Apr 2018 16:29
Status: Offline

Re: [EXTENSION] Rclone

#25

Post by Shperrung » 17 Nov 2018 13:18

Rclone provides an option to set password for rclone.config file. It is useful because it contains sensitive token for a remote access.
When I set password for rclone.config file in RcloneBrowser all remotes disappear from OBI-rclone gui.
Does obi-rcolne support encrypted cofiguration (password) file or not?
PS: Just tested encryption for remote. Despite of the password in settings for encrypted remote it works in OBI-rclone gui in both directions (ecrypt/decrypt) without password request.
11.2.0.4 - Omnius (revision 6177)
ASRock J3710-ITX, LAN: Realtek RTL8111GR; 16Gb RAM; WD 1Tbx2, WD 2Tb; UPS Powercom WOW500U.

User avatar
Snufkin
Advanced User
Advanced User
Posts: 289
Joined: 01 Jul 2012 11:27
Location: Etc/GMT-3 (BSD style)
Status: Offline

Re: [EXTENSION] Rclone

#26

Post by Snufkin » 29 Nov 2018 21:17

crest wrote:
17 Nov 2018 06:58
there is a new release (v1.1) of this extension available => [EXTENSION] Rclone.
...
Regards
crest
Dear Crest,

I had a chance to carry out indepth tests for latest RCL v1.1 extension.
Below are issues found.


Topic: New installation

Case: Fresh install
Steps:
  1. Have no file or empty file rclone.conf
  2. Push button "About" nearby "Defined Remotes" field
Results:
  • On scren error

    Code: Select all

    Warning: Invalid argument supplied for foreach() in /mnt/<...>/rclone/rclone-config.php on line 328
  • Numerous records in system log

    Code: Select all

    NOTICE:	file "/mnt/Test-NC/ex/rclone/rclone.conf" not found - using defaults
Case: Single command
Steps:
  1. Execute single command not saved earlier
Results:
  • On scren error

    Code: Select all

    Execute Single Command - not a valid command: --checksum --no-update-modtime -v

Topic: Broken char's

Case: Wrong locale
Steps:
  1. RCI extension or rclone itself update/upgrade
Results:
  • Record in system log

    Code: Select all

    root:	tar: Failed to set default locale
See my early report Locale error during OBI and NOC upgrade

Case: Create/save task
Steps:
  1. Create new task (could be the same from previous case "Single command")
  2. Have Task name, source directory and its subdirectory, remote path directory in ru locale, for example "Фото" (Photo)
  3. Save new task
Results:
  • Task screen name: Ok
  • Source screen name: Ok
  • Task shell script name: Ok
  • Path to source and destination in script: Ok
  • Path to directory in remote: Ok
Case: Execute non-broken task
Steps:
  1. Push button "Execute" in "Task list" for the task from previous case, could be done multiple times
Results:
  • Task screen name: Broken, no change second and subsequent times
  • Source screen name: Broken, no change second and subsequent times
  • Task shell script name: Ok
  • Path to source and destination in script: Ok
  • Path to directory in remote: Ok
Case: Schedule broken task
Steps:
  1. Push button "Schedule" in "Task list" for the task from previous case, could be done multiple times
Results:
  • Task screen name: Broken, no change second and subsequent times
  • Source screen name: Broken, no change second and subsequent times
  • Task shell script name: Ok
  • Path to source and destination in script: Ok
  • Cron job both "Command" and "Description" contain broken characters
Case: Delete broken task
Steps:
  • Push button "Delete" in "Task list" for the task from previous case
Results:
  • Task is deleted from "Task list" table
  • Task script in /rclone/tasks directory not deleted
Case: Edit broken task
Steps:
  • Create new task as described in case "Create and save task"
  • Run new task as described in case "Execute task"
  • Push button "Edit" in "Task list"
  • Do no changes, push button "Save"
Results:
  • Task screen name: Broken, no change
  • Source screen name: Broken, no change
  • Task shell script name: New script created with broken name
  • Path to source and destination in script: Broken
Case: Execute broken task
Steps:
  • Push button "Execute" in "Task list" for the task from previous case, could be done multiple times
Results:
  • Last Run status: "stopped with error(s)"
  • Record in system log

    Code: Select all

    ERROR	: error reading source directory: directory not found
  • Task screen name: Broken twice, no change second and subsequent times
  • Source screen name: Broken twice, no change second and subsequent times
  • Task shell script name: Broken, no change
  • Path to source and destination in script: Broken, no change
  • Path to directory in remote: no directory created
I hope the info above is detailed enough to diagnose and fix issues found.
Do not hesitate to ask for more info :geek:
XNAS 11.3.0.4 embedded, ASUS P5B-E, Intel DC E6600, 4 GB DDR2, 2 x HGST HDN726040ALE614, 2 x WDC WD5000AAKS, Ippon Back Power Pro 400

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [EXTENSION] Rclone

#27

Post by crest » 01 Dec 2018 10:30

Hello Snufkin,
Snufkin wrote:
29 Nov 2018 21:17
...
Dear Crest,

I had a chance to carry out indepth tests for latest RCL v1.1 extension.
Below are issues found.
Thanks for your feedback, see my comments below.


Topic: New installation

Case: Fresh install
Steps:
  1. Have no file or empty file rclone.conf
  2. Push button "About" nearby "Defined Remotes" field
Results:
  • On scren error

    Code: Select all

    Warning: Invalid argument supplied for foreach() in /mnt/<...>/rclone/rclone-config.php on line 328
  • Numerous records in system log

    Code: Select all

    NOTICE:	file "/mnt/Test-NC/ex/rclone/rclone.conf" not found - using defaults
Confirmed, corrected for next release.

Case: Single command
Steps:
  1. Execute single command not saved earlier
Results:
  • On scren error

    Code: Select all

    Execute Single Command - not a valid command: --checksum --no-update-modtime -v

Topic: Broken char's
This was not the way it was meant to be (please refere to the screenshots in the first port of this topic -> Single Comand line and Output of Single Command) - the idea was to
have a fast access to the remote e.g. to look at a remote file path/tree etc. to see informations necessary for creating a task. Therefore just use the Additional Parameters / Single Command combined input field and execute button (e.g. lsd mega:Path/You/Want).

Case: Wrong locale
Steps:
  1. RCI extension or rclone itself update/upgrade
Results:
  • Record in system log

    Code: Select all

    root:	tar: Failed to set default locale
See my early report Locale error during OBI and NOC upgrade
Confirmed

Case: Create/save task
Steps:
  1. Create new task (could be the same from previous case "Single command")
  2. Have Task name, source directory and its subdirectory, remote path directory in ru locale, for example "Фото" (Photo)
  3. Save new task
Results:
  • Task screen name: Ok
  • Source screen name: Ok
  • Task shell script name: Ok
  • Path to source and destination in script: Ok
  • Path to directory in remote: Ok
Please provide the data you used.

Case: Execute non-broken task
Steps:
  1. Push button "Execute" in "Task list" for the task from previous case, could be done multiple times
Results:
  • Task screen name: Broken, no change second and subsequent times
  • Source screen name: Broken, no change second and subsequent times
  • Task shell script name: Ok
  • Path to source and destination in script: Ok
  • Path to directory in remote: Ok
Please provide the data you used.

Case: Schedule broken task
Steps:
  1. Push button "Schedule" in "Task list" for the task from previous case, could be done multiple times
Results:
  • Task screen name: Broken, no change second and subsequent times
  • Source screen name: Broken, no change second and subsequent times
  • Task shell script name: Ok
  • Path to source and destination in script: Ok
  • Cron job both "Command" and "Description" contain broken characters
Please provide the data you used.

Case: Delete broken task
Steps:
  • Push button "Delete" in "Task list" for the task from previous case
Results:
  • Task is deleted from "Task list" table
  • Task script in /rclone/tasks directory not deleted
Please provide the data you used.

Case: Edit broken task
Steps:
  • Create new task as described in case "Create and save task"
  • Run new task as described in case "Execute task"
  • Push button "Edit" in "Task list"
  • Do no changes, push button "Save"
Results:
  • Task screen name: Broken, no change
  • Source screen name: Broken, no change
  • Task shell script name: New script created with broken name
  • Path to source and destination in script: Broken
Please provide the data you used.

Case: Execute broken task
Steps:
  • Push button "Execute" in "Task list" for the task from previous case, could be done multiple times
Results:
  • Last Run status: "stopped with error(s)"
  • Record in system log

    Code: Select all

    ERROR	: error reading source directory: directory not found
  • Task screen name: Broken twice, no change second and subsequent times
  • Source screen name: Broken twice, no change second and subsequent times
  • Task shell script name: Broken, no change
  • Path to source and destination in script: Broken, no change
  • Path to directory in remote: no directory created
Please provide the data you used.
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

User avatar
Snufkin
Advanced User
Advanced User
Posts: 289
Joined: 01 Jul 2012 11:27
Location: Etc/GMT-3 (BSD style)
Status: Offline

Re: [EXTENSION] Rclone

#28

Post by Snufkin » 01 Dec 2018 15:21

crest wrote:
01 Dec 2018 10:30
Case: Single command
Steps:
  1. Execute single command not saved earlier
...
This was not the way it was meant to be (please refere to the screenshots in the first port of this topic -> Single Comand line and Output of Single Command) - the idea was to
have a fast access to the remote e.g. to look at a remote file path/tree etc. to see informations necessary for creating a task. Therefore just use the Additional Parameters / Single Command combined input field and execute button (e.g. lsd mega:Path/You/Want).
Very handy feature! Sorry, missed the idea first.
Issue "Single command" cancelled.

Below are additional info requested.
Local test directory

Code: Select all

# ls -FR1
Фото/
./Фото:
фото1.jpg
фото2.jpg
This is first part of my reply, and I'll edit this post later to add more info (completed).

Case: Create/save task
  • Task, Source and Destination screen names: Ok
    create-save-task.jpg
  • Task shell script name: Ok

    Code: Select all

    tasks# ls -1
    rclone-task-Фото.sh*
  • Path to source and destination in script: Ok

    Code: Select all

    #!/bin/sh
    # WARNING: THIS IS AN AUTOMATICALLY CREATED SCRIPT, DO NOT CHANGE THE CONTENT!
    # Command for cron usage: /mnt/dsk/services/rclone/tasks/rclone-task-Фото.sh
    PID=$$
    mkdir -p /var/run/rclone
    echo Фото > /var/run/rclone/$PID.name
    logger 'Rclone task Фото started with PID = '$PID
    /usr/local/bin/rclone --config '/mnt/dsk/services/rclone/rclone.conf' --log-file '/mnt/dsk/services/rclone/rclone.log' --checksum --no-update-modtime -v copy '/mnt/dsk/tmp/Фото' 'yandexdisk:/tmp/Фото'
    if [ $? == 0 ]; then
            logger 'Rclone task Фото successfully finished'
            echo 'Фото successfully finished' `date` > /mnt/dsk/services/rclone/rclone_backup-date.txt
    else
            logger 'Rclone task Фото stopped with error(s)'
            echo '<font color='red'><b>Фото stopped with error(s)</b></font>' `date` > /mnt/dsk/services/rclone/rclone_backup-date.txt
    fi
    rm /var/run/rclone/$PID.name
Case: Execute non-broken task
  • Task, Source and Destination screen names: Broken, no change second and subsequent times
    execute-non-broken-task.jpg
  • Task shell script name: Ok (see case "Create/save task")
  • Path to source and destination in script: Ok (see case "Create/save task")
  • Path to directory in remote: Ok

    Code: Select all

    Command Output: tree yandexdisk:/tmp
    
    /
    └── Фото
        ├── фото1.jpg
        └── фото2.jpg
Case: Schedule broken task
  • Task, Source and Destination screen names: Broken, no change second and subsequent times (see case "Execute non-broken task")
  • Task shell script name: Ok (see case "Create/save task")
  • Path to source and destination in script: Ok (see case "Create/save task")
  • Cron job both "Command" and "Description" contain broken characters
    schedule-broken-task.jpg
Case: Delete broken task
  • Task script in /rclone/tasks directory not deleted

    Code: Select all

    tasks# ls -1
    rclone-task-Фото.sh*
Case: Edit broken task
  • Task, Source and Destination screen names: Broken, no change (see case "Execute non-broken task")
  • Task shell script name: New script created with broken name

    Code: Select all

     tasks# ls -1
    rclone-task-Фото.sh
    rclone-task-ФоÑ?о.sh
  • Path to source and destination in script: Broken

    Code: Select all

    #!/bin/sh
    # WARNING: THIS IS AN AUTOMATICALLY CREATED SCRIPT, DO NOT CHANGE THE CONTENT!
    # Command for cron usage: /mnt/dsk/services/rclone/tasks/rclone-task-Фо�о.sh
    PID=$$
    mkdir -p /var/run/rclone
    echo Фо�о > /var/run/rclone/$PID.name
    logger 'Rclone task Фо�о started with PID = '$PID
    /usr/local/bin/rclone --config '/mnt/dsk/services/rclone/rclone.conf' --log-file '/mnt/dsk/services/rclone/rclone.log' --checksum --no-update-modtime -v copy '/mnt/dsk/tmp/Фо�о' 'yandexdisk:/tmp/Фо�о'
    if [ $? == 0 ]; then
            logger 'Rclone task Фо�о successfully finished'
            echo 'Фо�о successfully finished' `date` > /mnt/dsk/services/rclone/rclone_backup-date.txt
    else
            logger 'Rclone task Фо�о stopped with error(s)'
            echo '<font color='red'><b>Фо�о stopped with error(s)</b></font>' `date` > /mnt/dsk/services/rclone/rclone_backup-date.txt
    fi
    rm /var/run/rclone/$PID.name
Case: Execute broken task
  • Last Run status: "stopped with error(s)"
  • Record in system log

    Code: Select all

    ERROR	: error reading source directory: directory not found
  • Task, Source and Destination screen names: Broken again (twice)
    execute-broken-task.jpg
  • Task shell script name: Broken, no change (see case "Edit broken task")
  • Path to source and destination in script: (see case "Edit broken task")
  • Path to directory in remote: no directory created

Edit
New test case

Case: Delete non-broken task, which was never executed
  1. Steps:
    1. Create new task as described in case "Create and save task". Do not execute it!
    2. Push button "Delete" in "Task list" for the task just created (only ONCE!)
    Results:
    • The "Task list" table still has an entry for the task
    • Task, Source and Destination screen names: Broken (see case "Execute non-broken task")
    • Task shell script: Deleted
  2. Steps:
    1. Push button "Delete" again in "Task list" for the broken task (only ONCE!)
    Results:
    • The "Task list" table still has an entry for the task
    • Task, Source and Destination screen names: Broken again (twice, see case "Edit broken task")
    • Task shell script: no file
  3. Steps:
    1. Push button "Delete" again in "Task list" for the broken task (only ONCE!)
    Results:
    • The "Task list" table still has an entry for the task
    • Task, Source and Destination screen names: Broken again (thrice)
      delete-non-broken-task-never-executed.jpg
    • Task shell script: no file
Each attempt changes task name, but does not delete the task record in the "Task list" table.
To get rid of this ghost record I had to directly edit /rclone/ext/rclone.conf file.
You do not have the required permissions to view the files attached to this post.
XNAS 11.3.0.4 embedded, ASUS P5B-E, Intel DC E6600, 4 GB DDR2, 2 x HGST HDN726040ALE614, 2 x WDC WD5000AAKS, Ippon Back Power Pro 400

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [EXTENSION] Rclone

#29

Post by crest » 01 Dec 2018 17:16

Snufkin wrote:
01 Dec 2018 15:21
crest wrote:
01 Dec 2018 10:30
Case: Single command
Steps:
  1. Execute single command not saved earlier
...
This was not the way it was meant to be (please refere to the screenshots in the first port of this topic -> Single Comand line and Output of Single Command) - the idea was to
have a fast access to the remote e.g. to look at a remote file path/tree etc. to see informations necessary for creating a task. Therefore just use the Additional Parameters / Single Command combined input field and execute button (e.g. lsd mega:Path/You/Want).
Very handy feature! Sorry, missed the idea first.
Issue "Single command" cancelled.
I will make it more clear, anyway it showed me that the usage was not obvious ... :(

Below are additional info requested.
Local test directory

Code: Select all

# ls -FR1
Фото/
./Фото:
фото1.jpg
фото2.jpg
This is first part of my reply, and I'll edit this post later to add more info (completed).
....
Each attempt changes task name, but does not delete the task record in the "Task list" table.
To get rid of this ghost record I had to directly edit /rclone/ext/rclone.conf file.
Thanks again for the very complete analyze :ugeek: ;) , was easy to comprehend - bug found, correction in the upcoming release.

Regards
crest
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

User avatar
Snufkin
Advanced User
Advanced User
Posts: 289
Joined: 01 Jul 2012 11:27
Location: Etc/GMT-3 (BSD style)
Status: Offline

Re: [EXTENSION] Rclone

#30

Post by Snufkin » 01 Dec 2018 19:44

crest wrote:
01 Dec 2018 17:16

Thanks again for the very complete analyze :ugeek: ;) , was easy to comprehend - bug found, correction in the upcoming release.

Regards
crest
You are welcome.
In fact, I'm not looking for software bugs specifically, they (bugs) find me on their own. :lol:

Looking forward for upcoming Rclone extension release.
XNAS 11.3.0.4 embedded, ASUS P5B-E, Intel DC E6600, 4 GB DDR2, 2 x HGST HDN726040ALE614, 2 x WDC WD5000AAKS, Ippon Back Power Pro 400

Post Reply

Return to “Extensions / Add-ons”