*New 12.1 series Release:
2019-11-08: XigmaNAS 12.1.0.4.7091 - released!

*New 11.3 series Release:
2019-10-19: XigmaNAS 11.3.0.4.7014 - released


We really need "Your" help on XigmaNAS https://translations.launchpad.net/xigmanas translations. Please help today!

Producing and hosting XigmaNAS costs money. Please consider donating for our project so that we can continue to offer you the best.
We need your support! eg: PAYPAL

[REQUEST] MEGA.NZ integration

Post/Debate your Suggestions & Requests of XigmaNAS here. This ONLY pertains to XigmaNAS.
Forum rules
Set-Up GuideFAQsForum Rules
Post Reply
anders.malm
Starter
Starter
Posts: 23
Joined: 01 Mar 2015 10:03
Status: Offline

[REQUEST] MEGA.NZ integration

#1

Post by anders.malm » 23 Jul 2017 07:20

Would it be possibly to integrate the ability to connect to the MEGA.NZ cloud inside NAS4FREE. I have found this [url]http://www.freshports.org/net/megatools/ , a command line tool that let you upload and download files and directories. A first step could be just the tool, but of course a GUI would be nice. Maybe a packet for the OBI is a good way.
ASUS P10-S, Intel i3 CPU, 16 GB ECC ram, 250 GB M2 SSD, RootOnZFS

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#2

Post by crest » 06 Aug 2017 13:17

Hello again anders,
anders.malm wrote:
23 Jul 2017 07:20
Would it be possibly to integrate the ability to connect to the MEGA.NZ cloud inside NAS4FREE. I have found this [url]http://www.freshports.org/net/megatools/ , a command line tool that let you upload and download files and directories. A first step could be just the tool, but of course a GUI would be nice. Maybe a packet for the OBI is a good way.
Since I plan to create an extension for rclone it would be maybe possible to include MEGA.NZ also or maybe write an own extension for this but due to lack of spare time it could take some time to implement it but I will have a look at the megatools to see what's necessary to create an extension. Do you use MEGA.NZ already?

Regards
crest
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

anders.malm
Starter
Starter
Posts: 23
Joined: 01 Mar 2015 10:03
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#3

Post by anders.malm » 07 Aug 2017 18:57

Hello crest. I´m currently using Mega from one of my Windows PC to backup my pictures. The fact that you get 50 GB for free and that the files get crypt local before they is sent to the cloud is great.

/Anders
ASUS P10-S, Intel i3 CPU, 16 GB ECC ram, 250 GB M2 SSD, RootOnZFS

Sandinist
NewUser
NewUser
Posts: 2
Joined: 31 Jul 2018 10:01
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#4

Post by Sandinist » 31 Jul 2018 10:15

Hello Crest,
How is it going with Mega cloud integration?
Did you manage to look at the magatools?
I'm using Mega too and it would be very interesting to see it integrated.
Best regards,
Alex

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#5

Post by crest » 31 Jul 2018 13:22

Sandinist wrote:
31 Jul 2018 10:15
Hello Crest,
How is it going with Mega cloud integration?
Did you manage to look at the magatools?
I'm using Mega too and it would be very interesting to see it integrated.
Best regards,
Alex
Hello and welcome to the forums!

Yes, megatools look very interesting and I am planning the next steps for several extensions, one will be the support for Mega too.
Since I must start from scratch it can take a while but you could do some testing if a first beta is available if you like?

Regards
crest
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

Sandinist
NewUser
NewUser
Posts: 2
Joined: 31 Jul 2018 10:01
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#6

Post by Sandinist » 15 Aug 2018 12:34

Hello Crest,
I would make some tests but unfortunately I am hard busy for the next couple of months and hardly will be able to find any time for this.

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#7

Post by crest » 20 Aug 2018 10:09

Sandinist wrote:
15 Aug 2018 12:34
Hello Crest,
I would make some tests but unfortunately I am hard busy for the next couple of months and hardly will be able to find any time for this.
Ok, so maybe I will postbone this a bit until you will have some time for testing ... ;)
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

anders.malm
Starter
Starter
Posts: 23
Joined: 01 Mar 2015 10:03
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#8

Post by anders.malm » 25 Aug 2018 15:26

Hello Crest,
I could do som tests on this if you want.

/Anders
ASUS P10-S, Intel i3 CPU, 16 GB ECC ram, 250 GB M2 SSD, RootOnZFS

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#9

Post by crest » 26 Aug 2018 05:12

Hello anders,
anders.malm wrote:
25 Aug 2018 15:26
Hello Crest,
I could do som tests on this if you want.

/Anders
Great, I will let you know if a beta is available for testing.

Regards
crest
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#10

Post by crest » 22 Oct 2018 09:46

Hello anders,
crest wrote:
26 Aug 2018 05:12
Hello anders,
anders.malm wrote:
25 Aug 2018 15:26
Hello Crest,
I could do som tests on this if you want.

/Anders
Great, I will let you know if a beta is available for testing.

Regards
crest
Now I started with MEGA and saw, that rclone also supports mega.
After some test it turned out that
- installing is quite simple (just to download the OS appropriate archive, unpack and create a symlink to have access from everywhere - no need to use pkg, jails etc.)
- configuring of the differnet services at the command line is straight forward (for some services so a local webbrowser is needed which could be easily performed at every PC)
therefore the question is, what would be expected to fulfill with an extension?

Maybe you could lead me with some use cases in the right direction ... ;)

Regards
crest
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

anders.malm
Starter
Starter
Posts: 23
Joined: 01 Mar 2015 10:03
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#11

Post by anders.malm » 24 Oct 2018 18:54

Hello crest,
I have done some test and successfully connected to my Mega account. I can list files and directories, create directories an copy files. My goal is to be able to use Mega as a external backup target. I think its ok too use cron for scheduling the backup.

As a person that not normaly use CLI would it be nice if it was a GUI for the config. The step to install for example Putty to bee able too config is not that easy.

/Anders
ASUS P10-S, Intel i3 CPU, 16 GB ECC ram, 250 GB M2 SSD, RootOnZFS

User avatar
Snufkin
Advanced User
Advanced User
Posts: 293
Joined: 01 Jul 2012 11:27
Location: Etc/GMT-3 (BSD style)
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#12

Post by Snufkin » 24 Oct 2018 19:50

anders.malm wrote:
24 Oct 2018 18:54
As a person that not normaly use CLI would it be nice if it was a GUI for the config.
Fully agree with you.
The idea of rclone extension is rather simple:
  • fetch rclone binary,
  • define persistent location,
  • assign rclone config and log (optional) paths to above location by default,
  • create simple rclone upload (copy) script,
  • give end user WebGUI field to enter extra variables to above script.
But rclone configuration for so called remotes requires shell access.
Have no idea how it could be implemented in XigmaNAS. RcloneBrowser
?
XNAS 11.3.0.4 embedded, ASUS P5B-E, Intel DC E6600, 4 GB DDR2, 2 x HGST HDN726040ALE614, 2 x WDC WD5000AAKS, Ippon Back Power Pro 400

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#13

Post by crest » 25 Oct 2018 11:02

Snufkin wrote:
24 Oct 2018 19:50
anders.malm wrote:
24 Oct 2018 18:54
As a person that not normaly use CLI would it be nice if it was a GUI for the config.
Fully agree with you.
The idea of rclone extension is rather simple:
  • fetch rclone binary,
  • define persistent location,
  • assign rclone config and log (optional) paths to above location by default,
  • create simple rclone upload (copy) script,
  • give end user WebGUI field to enter extra variables to above script.
But rclone configuration for so called remotes requires shell access.
Have no idea how it could be implemented in XigmaNAS. RcloneBrowser
?
The items #1 - 3 and 5 are already done, #4 is still under construction but should be ready soon.
screen_20181025_115421.png

The configuration is somehow the trickiest part and I tend to leave it to the CLI in conjunction with RcloneBrowser because there are too many different parameters according the different services which can be used with rclone.

What I tried is to provide RcloneBrowser (RCB) optional with the installation of the extension (atm only for Windows) as well as the binary for Windows - with this it is possible to use the same config for rclone and RcloneBrowser. So the config for services like OneDrive, DropBox etc can be defined via RCB and the appropriate tokens will be written to the config which is also used by rclone on the server. Sounds maybe a bit confusing but at least it is not necessary to change between server and PC applications, copy/paste tokens or copy/move the config file from the PC to the server.
screen_20181025_115949.png
Currently there are efforts to create a rclone GUI in the rclone community but this is in a very early state but could maybe solve some of the caveats for headless servers, let's see ...

As soons as #4 is in a usable state I will release a beta for testing purposes.

Regards
crest
You do not have the required permissions to view the files attached to this post.
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

User avatar
Snufkin
Advanced User
Advanced User
Posts: 293
Joined: 01 Jul 2012 11:27
Location: Etc/GMT-3 (BSD style)
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#14

Post by Snufkin » 25 Oct 2018 12:37

crest wrote:
25 Oct 2018 11:02
As soons as #4 is in a usable state I will release a beta for testing purposes.
Great, Crest!
Looking forward to the latest beta.
crest wrote:
25 Oct 2018 11:02
What I tried is to provide RcloneBrowser (RCB) optional with the installation of the extension (atm only for Windows) as well as the binary for Windows - with this it is possible to use the same config for rclone and RcloneBrowser.

So the config for services like OneDrive, DropBox etc can be defined via RCB and the appropriate tokens will be written to the config which is also used by rclone on the server.

Sounds maybe a bit confusing but at least it is not necessary to change between server and PC applications, copy/paste tokens or copy/move the config file from the PC to the server
Nice trick, Crest, I like it. It's a good way to manage remotes with GUI help, at least for MS Windows desktops.

Did I understand right RcloneBrowser installation would be part of extension?
And its home directory must be on Samba share to run it directly from Windows host?
XNAS 11.3.0.4 embedded, ASUS P5B-E, Intel DC E6600, 4 GB DDR2, 2 x HGST HDN726040ALE614, 2 x WDC WD5000AAKS, Ippon Back Power Pro 400

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#15

Post by crest » 28 Oct 2018 11:26

Snufkin wrote:
25 Oct 2018 12:37
Nice trick, Crest, I like it. It's a good way to manage remotes with GUI help, at least for MS Windows desktops.

Did I understand right RcloneBrowser installation would be part of extension?
And its home directory must be on Samba share to run it directly from Windows host?
Not neccessarily, just thought about for convenience (to have everything - browser and Windows executable - at once), maybe just for the beta, will see …
The only thing would be to manage the remotes with the config stored at the NAS System to make live easier.

First beta will be most likely available tomorrow, Installation as Always via the beta channel of OneButtonInstaller.

Regards
crest
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

User avatar
ernie
Forum Moderator
Forum Moderator
Posts: 1418
Joined: 26 Aug 2012 19:09
Location: France - Val d'Oise
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#16

Post by ernie » 28 Oct 2018 15:50

Hello
Good news crest for this new integration
I will test it the beta version when available.
BR
NAS 1&2:
System: GA-6LXGH(BIOS: R01 04/30/2014) / 16 Go ECC
XigmaNAS 12.0.0.4.6766 embedded
NAS1: Xeon E3 1241@3.5GHz, 4HDD@2To/raidz2 (WD red), 3HDD@300Go/sas/raidz1 (Hitachi), 1SSD cache, Zlog on sas mirror
NAS2: G3220@3GHz, 3HDD@2To/raidz1 (Seagate), 1SSD cache, 1HDD@300Go/UFS
UPS: APC Back-UPS RS 900G
Case : Fractal Design XL R2

Extensions & services:
NAS1: OBI (Plex, extendedGUI, BTSync, zrep, rclone), nfs, UPS,
NAS2: OBI (extendedGUI, zrep (backup mode))

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#17

Post by crest » 31 Oct 2018 02:38

Now the first release of the Rclone extension is available, please install it from the OneButtonInstaller beta channel.

After successfull installation proceed to Extensions > Rclone
screen_20181031_021922.png
Rclone is now available to run (could be used also on the CLI, type rclone to see the available commands and options).

Next step would be to define one (or more) remotes - I attached the app RcloneBrowser (RCB) for MS Windows as well as the rclone Windows executable in /Your/Extension/Path/rclone/RcloneBrowser.
One could run it directly from the server or copy the whole directory to a folder on your client PC and run it from there, no installation is required.
Executables for Linux and macOS as well as installation instructions can be found at the RCB Github pages

Running RCB for the first time it is required to setup the path to the Windows executable in the above mentioned directory and the rclone.conf file at your XigmaNAS server which can be found directly in /Your/Extension/Path/rclone/rclone.conf -> in RCB > Files > Preferences
screen_20181031_023013.png
Save the settings with OK and on the RCB main screen at the bottom hit Config to define your remotes.

The process for this is really straight forward and in the Rclone documentation described for many supported cloud services.
screen_20181031_024034.png
For some cloud services a browser window on your local machine will open to confirm to give access for Rclone to the service.

After the definition of the remote(s) one could browse the remotes in RCB and perfom up/downloads etc but now we would now define some tasks on the XigmaNAS server - therefore refresh the RClone extension page and the defined remote(s) should appear under Extensions > Rclone 0.1 > Configuration > Defined Remotes.
screen_20181031_030604.png

Under Add Task provide a Name, Source, Destination, backup Mode and eventually additional Parameters - hit Add button to store the Definition.
screen_20181031_030846.png

Now we are ready to execute the task directly from the task list with Execute (the task will run in the background, status shows running and after finishing display the datetime of the last successful run or show failed if there were problems during execution - output will be written to rclone.log in the extension directory).
screen_20181031_031958.png
With Add to cron the appropriate commands will be written to the cron table to perform an execution of the task every night at 1:00 - change the cron job directly under System > Advanced > Cron to your needs (run daily/weekly etc ...).

Any feedback is highly appreciated!

Regards
crest
You do not have the required permissions to view the files attached to this post.
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

anders.malm
Starter
Starter
Posts: 23
Joined: 01 Mar 2015 10:03
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#18

Post by anders.malm » 31 Oct 2018 18:56

Hello crest
I have done some initial testing and it mostly works as expected. I found some small issues.

Rclone can't handle paths with spaces, adding ' before and after in the copy-string solves the problem.

It would be fine if we could edit a Task.

Maybe the Add button in the tasklist should be renamed to for example "Schedule" or "To cron"


/Anders
ASUS P10-S, Intel i3 CPU, 16 GB ECC ram, 250 GB M2 SSD, RootOnZFS

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#19

Post by crest » 31 Oct 2018 21:58

Hi Anders,

thank you for taking the time for testing and your feedback!
anders.malm wrote:
31 Oct 2018 18:56
Hello crest
I have done some initial testing and it mostly works as expected. I found some small issues.

Rclone can't handle paths with spaces, adding ' before and after in the copy-string solves the problem.
Yes, confirmed, saw it already today with further tests ... ;)
It would be fine if we could edit a Task.
OK, done
Maybe the Add button in the tasklist should be renamed to for example "Schedule" or "To cron"
Done too

There are a lot of bells and whistles to add too but for the first release I would prefer to get inputs of testers like you integrated, maybe by and by we will have some more features added :D

I will wait for some other inputs and release a new version asap.

Best regards
crest
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

User avatar
Snufkin
Advanced User
Advanced User
Posts: 293
Joined: 01 Jul 2012 11:27
Location: Etc/GMT-3 (BSD style)
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#20

Post by Snufkin » 01 Nov 2018 13:21

crest wrote:
31 Oct 2018 21:58
I will wait for some other inputs and release a new version asap.
Here we are!

Beta is almost fully functional and can be used without restrictions.

What else I would do:
  1. fix minor GUI glitch, could be visible while zooming browser window/tab content;
    gui-element-overlap.jpg
  2. make RClone Browser installation optional, may be checkbox somewhere in GUI page with further apply/save;
  3. avoid explicitly specifying Source as local file system and Destination as a remote one, use cases: copy (restore) backup to local, do server side move or sync two remotes;
  4. avoid explicit reference to only backup possibility, field name Last Backup, may be Last Run;
  5. rename rclone/ext/rclone.conf to avoid confusion with rclone/rclone.conf, something like exrclone.conf;
  6. add on-screen warning/hint after adding task to crontab, like "Default is everyday/hourly, consider creating your own schedule";
  7. syncronize removal of
    • RClone task in GUI,
    • shell script in rclone/tasks directory,
    • appropriate crontab job;
  8. provide user with a way to edit an existing RClone task (already in next beta?).
It didn't work out much, did it? :roll:
You do not have the required permissions to view the files attached to this post.
XNAS 11.3.0.4 embedded, ASUS P5B-E, Intel DC E6600, 4 GB DDR2, 2 x HGST HDN726040ALE614, 2 x WDC WD5000AAKS, Ippon Back Power Pro 400

User avatar
ernie
Forum Moderator
Forum Moderator
Posts: 1418
Joined: 26 Aug 2012 19:09
Location: France - Val d'Oise
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#21

Post by ernie » 02 Nov 2018 13:22

Hello

With Mega it works via cli.
I cannot test RCB as I use Debian and I had to compile RCB.

I configured rclone via CLI. Then the extension didn’t show the defined remote. Status is also always on stopped.

When I add a task and if I execute, it doesn’t work. Last backup said Last backup failed”. Any advice ?
Source is a folder in a dataset and destination is a folder existing in Mega (Mega:test).
In cli it works
rclone copy /mnt/master/admin/script/ Mega:test
In the extension view I have
Test /mnt/master/admin/script Mega:test copy

Perhaps it lacks the last / for source in extension view.

A button Edit for a task will be good.
The button remove removes from task but not from Cron, if it was added via the extension in Cron.

Why not a display of content in the rclone view (view of results of rclone lsl ou ls) ? It will help to see the structure tree of the cloud.
BR
NAS 1&2:
System: GA-6LXGH(BIOS: R01 04/30/2014) / 16 Go ECC
XigmaNAS 12.0.0.4.6766 embedded
NAS1: Xeon E3 1241@3.5GHz, 4HDD@2To/raidz2 (WD red), 3HDD@300Go/sas/raidz1 (Hitachi), 1SSD cache, Zlog on sas mirror
NAS2: G3220@3GHz, 3HDD@2To/raidz1 (Seagate), 1SSD cache, 1HDD@300Go/UFS
UPS: APC Back-UPS RS 900G
Case : Fractal Design XL R2

Extensions & services:
NAS1: OBI (Plex, extendedGUI, BTSync, zrep, rclone), nfs, UPS,
NAS2: OBI (extendedGUI, zrep (backup mode))

User avatar
Snufkin
Advanced User
Advanced User
Posts: 293
Joined: 01 Jul 2012 11:27
Location: Etc/GMT-3 (BSD style)
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#22

Post by Snufkin » 02 Nov 2018 17:47

ernie wrote:
02 Nov 2018 13:22
I configured rclone via CLI. Then the extension didn’t show the defined remote. Status is also always on stopped.
By default RClone saves its configuration somewhere in root home directory. This is most likely the reason why your remotes are missed in WebGUI.

For XNAS embedded we have to explicitly specify path to RClone config file every time we call RClone (see more in RClone doc page).

Code: Select all

rclone config --config /OBI-path-to-store-extensions/rclone/rclone.conf
where config is a command to start RClone configuration sequence,
and --config is an option to specify path to config file.

Crest designed RClone extension directory structure as follows:

Code: Select all

/OBI-path-to-store-extensions/rclone/bin/        # dir where rclone binary is located
/OBI-path-to-store-extensions/rclone/tasks/      # dir for automatically created shell scripts
/OBI-path-to-store-extensions/rclone/rclone.conf # configuration file
/OBI-path-to-store-extensions/rclone/rclone.log  # log file
ernie wrote:
02 Nov 2018 13:22
When I add a task and if I execute, it doesn’t work. Last backup said Last backup failed”. Any advice?
I'd suggest to add -vv option in task Additional Parameters to get more verbose log output.
And when you will review rclone.log file, you can easily understand the cause of the error.
XNAS 11.3.0.4 embedded, ASUS P5B-E, Intel DC E6600, 4 GB DDR2, 2 x HGST HDN726040ALE614, 2 x WDC WD5000AAKS, Ippon Back Power Pro 400

User avatar
ernie
Forum Moderator
Forum Moderator
Posts: 1418
Joined: 26 Aug 2012 19:09
Location: France - Val d'Oise
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#23

Post by ernie » 02 Nov 2018 21:18

Snufkin wrote:
02 Nov 2018 17:47
ernie wrote:
02 Nov 2018 13:22
I configured rclone via CLI. Then the extension didn’t show the defined remote. Status is also always on stopped.
By default RClone saves its configuration somewhere in root home directory. This is most likely the reason why your remotes are missed in WebGUI.

For XNAS embedded we have to explicitly specify path to RClone config file every time we call RClone (see more in RClone doc page).

Code: Select all

rclone config --config /OBI-path-to-store-extensions/rclone/rclone.conf
where config is a command to start RClone configuration sequence,
and --config is an option to specify path to config file.

Crest designed RClone extension directory structure as follows:

Code: Select all

/OBI-path-to-store-extensions/rclone/bin/        # dir where rclone binary is located
/OBI-path-to-store-extensions/rclone/tasks/      # dir for automatically created shell scripts
/OBI-path-to-store-extensions/rclone/rclone.conf # configuration file
/OBI-path-to-store-extensions/rclone/rclone.log  # log file
ernie wrote:
02 Nov 2018 13:22
When I add a task and if I execute, it doesn’t work. Last backup said Last backup failed”. Any advice?
I'd suggest to add -vv option in task Additional Parameters to get more verbose log output.
And when you will review rclone.log file, you can easily understand the cause of the error.
Hello
Thanks. I modified the config. But now I seen the remote in the extension view, bu the cli doesn’t work:
Rclone about mega: said:
Failed to create file system for "mega:": didn't find section in config file

The log file is empty.

And in webgui it works. The test is successful for the task.

I will investigate and any advice is welcome.

I tried also an uninstall of extension and reinstall, new config via — config pathoffile/rclone.conf but not successful.

Via cli I am as root. Do I need to use a specific user or group ?

BR
NAS 1&2:
System: GA-6LXGH(BIOS: R01 04/30/2014) / 16 Go ECC
XigmaNAS 12.0.0.4.6766 embedded
NAS1: Xeon E3 1241@3.5GHz, 4HDD@2To/raidz2 (WD red), 3HDD@300Go/sas/raidz1 (Hitachi), 1SSD cache, Zlog on sas mirror
NAS2: G3220@3GHz, 3HDD@2To/raidz1 (Seagate), 1SSD cache, 1HDD@300Go/UFS
UPS: APC Back-UPS RS 900G
Case : Fractal Design XL R2

Extensions & services:
NAS1: OBI (Plex, extendedGUI, BTSync, zrep, rclone), nfs, UPS,
NAS2: OBI (extendedGUI, zrep (backup mode))

User avatar
Snufkin
Advanced User
Advanced User
Posts: 293
Joined: 01 Jul 2012 11:27
Location: Etc/GMT-3 (BSD style)
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#24

Post by Snufkin » 02 Nov 2018 22:23

ernie wrote:
02 Nov 2018 21:18
But now I seen the remote in the extension view, bu the cli doesn’t work:
Rclone about mega: said:
Failed to create file system for "mega:": didn't find section in config file

The log file is empty.

And in webgui it works. The test is successful for the task.
Look at the content of the auto created shell scripts (for cron jobs) in directory

Code: Select all

/OBI-path-to-store-extensions/rclone/tasks/
There you could find full command line to call RClone in CLI.
ernie wrote:
02 Nov 2018 21:18
Via cli I am as root. Do I need to use a specific user or group?
Root user is fine.
XNAS 11.3.0.4 embedded, ASUS P5B-E, Intel DC E6600, 4 GB DDR2, 2 x HGST HDN726040ALE614, 2 x WDC WD5000AAKS, Ippon Back Power Pro 400

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#25

Post by crest » 03 Nov 2018 04:57

Hello ernie,

thank you for taking the time for testing and feedback!
ernie wrote:
02 Nov 2018 13:22
..
With Mega it works via cli.
I cannot test RCB as I use Debian and I had to compile RCB.

I configured rclone via CLI. Then the extension didn’t show the defined remote. Status is also always on stopped.
As Snufkin already mentioned the trick is just to use the same config file to define the cloud services as the Rclone extension uses. Therefore in the brief description -> viewtopic.php?f=56&t=12698#p87585 I wrote

Code: Select all

...  and the rclone.conf file at your XigmaNAS server which can be found directly in /Your/Extension/Path/rclone/rclone.conf ...
sorry if this was not clear enough.

Furthermore one can change the path inside the extension to another location/share if one don't want to use it in the path /Your/Extension/Path/rclone/ (option Configuration File in the extension).
...
A button Edit for a task will be good.
Will be included in the next release
The button remove removes from task but not from Cron, if it was added via the extension in Cron.
Right, task shell scripts are already removed but not the cron entry, will come also with the next release.
Why not a display of content in the rclone view (view of results of rclone lsl ou ls) ? It will help to see the structure tree of the cloud.
BR
This is want I meant before, there are many bells and whistles ;) which we can add as rclone proviedes A LOT of features, by and by I will add some of them :D
... Status is also always on stopped ....
This just means that rclone is currently not running/no task is executing (rclone runs not as a daemon all the time rather than on demand with Execute or triggered by cron)

Regards
crest
Last edited by crest on 03 Nov 2018 05:54, edited 1 time in total.
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#26

Post by crest » 03 Nov 2018 05:29

Hi Snufkin,

thanks for the detailed elaboration ;)
Snufkin wrote:
01 Nov 2018 13:21
...
fix minor GUI glitch, could be visible while zooming browser window/tab content;
Done
[*]make RClone Browser installation optional, may be checkbox somewhere in GUI page with further apply/save;
Atm I'm still not sure if I will stay with this because for Linux and Mac users I have no other solution as to advice to them to refer to the Rclone documentation - to install rclone for their OS and to point to the RClone extension path when creating the definitions for the remotes - or to copy a local config created to the Rclone extension path.
[*]avoid explicitly specifying Source as local file system and Destination as a remote one, use cases: copy (restore) backup to local, do server side move or sync two remotes;
Hmm, I tend to stay with the Source - Destination definition because it's flexible and allows to use it for all cases e.g. Source could be local on the XigmaNAS server (than you could use the browser button to easily put in the path) or one can enter also a remote path e.g. remote:Path/On/Remote. The field is not restricted to server side local paths.
The same applies to the Destination input field.
[*]avoid explicit reference to only backup possibility, field name Last Backup, may be Last Run;
Agreed
[*]rename rclone/ext/rclone.conf to avoid confusion with rclone/rclone.conf, something like exrclone.conf;
In normal cases I would not expect that users travell around the extensions directory structure, therefore I added the Configuration File option to let users point to another location/share on their XigmaNAS server for the rclone config file ...
[*]add on-screen warning/hint after adding task to crontab, like "Default is everyday/hourly, consider creating your own schedule";
Agreed
[*]syncronize removal of
  • RClone task in GUI,
  • shell script in rclone/tasks directory,
Already done in the first release
[*]appropriate crontab job;[/list]
Will come with the next release
[*]provide user with a way to edit an existing RClone task (already in next beta?).
...
Will come with the next release

Thanks a lot for your feedback! :D

Regards
crest
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

User avatar
crest
Hardware & Software Guru
Hardware & Software Guru
Posts: 873
Joined: 02 Jul 2012 22:25
Location: Vienna, Austria - GMT+1
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#27

Post by crest » 03 Nov 2018 08:19

A new beta (v0.2) of the Rclone extension (RCL) is available, upgrade can be performed directly inside the Rclone extension under the Maintenance tab.

Pls share your feedback ;) :D

Regards
crest
NAS1: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI 760GM-P23; AMD Athlon(tm) II X2 250 7.58GiB RAM
NAS2: 11.2.0.4 - Omnius (Revision 6766) x64-embedded; MSI MS-7369; AMD Sempron(tm) LE-1250 8022MiB RAM
UPS: APC Back-UPS ES 550G
Extensions: OneButtonInstaller, Extended GUI, NextOwnCloud, BitTorrent Sync, Syncthing, Downloady, Midnight Commander, NCDU, MySQL, Rclone, Themes:

User avatar
ernie
Forum Moderator
Forum Moderator
Posts: 1418
Joined: 26 Aug 2012 19:09
Location: France - Val d'Oise
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#28

Post by ernie » 03 Nov 2018 09:43

Thanks Crest and Snufkin for your explanation.
Thanks Crest for the new version, It is fine now for me :
- status : idle and running is very clear now
- source and destination: the text in the field is clear (local path...)
- in the note it will be good to add that in cli the rclone command must have —config pathfileconfig —log-file pathfilelog

Perhaps it could be a service in Xigmanas by default than an extension as it is useful as rsync service.
NAS 1&2:
System: GA-6LXGH(BIOS: R01 04/30/2014) / 16 Go ECC
XigmaNAS 12.0.0.4.6766 embedded
NAS1: Xeon E3 1241@3.5GHz, 4HDD@2To/raidz2 (WD red), 3HDD@300Go/sas/raidz1 (Hitachi), 1SSD cache, Zlog on sas mirror
NAS2: G3220@3GHz, 3HDD@2To/raidz1 (Seagate), 1SSD cache, 1HDD@300Go/UFS
UPS: APC Back-UPS RS 900G
Case : Fractal Design XL R2

Extensions & services:
NAS1: OBI (Plex, extendedGUI, BTSync, zrep, rclone), nfs, UPS,
NAS2: OBI (extendedGUI, zrep (backup mode))

User avatar
Snufkin
Advanced User
Advanced User
Posts: 293
Joined: 01 Jul 2012 11:27
Location: Etc/GMT-3 (BSD style)
Status: Offline

Re: [REQUEST] MEGA.NZ integration

#29

Post by Snufkin » 03 Nov 2018 17:20

crest wrote:
03 Nov 2018 08:19
A new beta (v0.2) of the Rclone extension (RCL) is available,
Pls share your feedback ;) :D
Some considerations for configuring RClone remotes.
crest wrote:
03 Nov 2018 05:29
make RClone Browser installation optional...
Atm I'm still not sure if I will stay with this because for Linux and Mac users I have no other solution...
Most likely, RClone Browser will be irrelevant on the MS Windows too.
The author of the program, mmozeiko stopped work actively on it, see his answer on GitHub.

I am in no way a developer, just a diligent user of search engines, and my below suggestion might be stupid. Could it be possible to implement in RCL extension some kind of interaction with shell script? Not one like send command, then get an answer, but true kind and PHP based?

I've found possible answer on stackoverflow.com, Run an Interactive Shell Script in PHP.
Will it help to implement RClone remotes configuration?

Let's go through RCL 0.2 release notes.
0.2 2018.11.03 several GUI improvements
Orange color looks somehow restless, all the time catches the eye. Grey for idle and green for running would be good.
N: allow task editing
C: delete appropriate cron tab entries on task remove
F: browser button overlapped next field in some cases
All early comments are taken into account and new features work as expected.
Multiple duplications of single RCL task in crontab are handled correctly. If the script does not exist, then there should be no jobs.

Great work, Crest!
XNAS 11.3.0.4 embedded, ASUS P5B-E, Intel DC E6600, 4 GB DDR2, 2 x HGST HDN726040ALE614, 2 x WDC WD5000AAKS, Ippon Back Power Pro 400

Post Reply

Return to “Suggestions & Requests”