Latest News:
2019-02-21: XigmaNAS 11.2.0.4.6536 - released!

Latest BETA Release:
2019-02-21: XigmaNAS 12.0.0.4.6536 - BETA released!

We really need "Your" help on XigmaNAS https://translations.launchpad.net/xigmanas translations. Please help today!

Producing and hosting XigmaNAS cost money, please consider a donation to our project so we can continue to offer you the best.
We need your support! eg: PAYPAL

XigmaNAS RootOnZFS Install Manual

New installs of XigmaNAS.
Forum rules
Set-Up GuideFAQsForum Rules
User avatar
JoseMR
Hardware & Software Guru
Hardware & Software Guru
Posts: 965
Joined: 16 Apr 2014 04:15
Location: PR
Contact:
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#31

Post by JoseMR » 02 Jan 2019 12:29

KTL wrote:
01 Jan 2019 21:59
I finally got around to reinstall using (1) latest version 6315, (2) RootOnZFS, and (3) 2x 120GB SSD in mirror configurations. Imported the resynchronized the original pool and all is up and running again. I can probably relax a bit better now. Thanks and have a Happy New Year!

Hi KTL, and Happy New Year! :)

Indeed you can sit back and relax, RootOnZFS Platform is very resilient even from a single disk, so you don't have to worry about sudden power failures, OS filesystem and/or boot issues etc., just make regular config backups upon settings changes from WebGUI, and/or reboot in a new Boot Environment before manual system wide changes/customizing.

I Hope for the Boot Environments feature pages I've submitted to be reviewed and available soon, so even novices can easily work between Boot Environments from the WebGUI for convenience.

Regards
System: FreeBSD 12, MB: Supermicro X8SI6-F, Xeon X3450, 16GB DDR3 ECC RDIMMs.
Addons at GitHub
JoseMRPubServ

cookiemonster
experienced User
experienced User
Posts: 94
Joined: 23 Mar 2014 02:58
Location: UK
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#32

Post by cookiemonster » 08 Jan 2019 23:34

Hi JoseMR. I finally upgraded to RootOnZFS.
Some feedback from my setup. I was on embedded on a single USB stick with a data partition and swap on it.
1. I'm sure I read somewhere and was logical anyway to backup the data in any existing partition on the embedded installation media. I ignored the advice expecting to be able to mount it after installation but the data was not there. I put that as experience and so others don't make the same mistake. I'll have to track what was there and move on.
2. My installation has one HBA where all disks are attached to. My main pool and now my new two mirrored SSDs for the root pool. After installation and reboot I was unable to boot to new disks. For some reason the PCI slot it is in was disabled in BIOS. It took me a while and many reboots to track the problem. Solved now. Booted to Full on RootOnZFS, re-scanned for disks, force imported the pool. All good.
3. Actual feedback: I chose mirrored install on default OS partition size, chose my choice of size for mirrored swap and defaults for size of data partition. I ended up with default choices of size for OS and data that are not of my liking. They are Root: 59.59 GB, Data 57 GB, Swap 4.29 GB.
My point I chose swap 4 GB and left the installer chose the sizes for data and root. It would be even better if the installer gave the option at the end of giving the sizes it works out by choosing defaults and the option to go back to change them. Unless I didn't use the tool correctly and this was there and I missed it.
4. Question if I may: Once rebooted and the disks were presented by BIOS to chose device to boot from, in my case it gave me the option of choosing one of the two devices I installed in mirror to. I've chosen the first in the just because it is the first.
4.a. Does it matter which one I chose?
4.b. Am I right in thinking that the OS will, those two being mirrored, keep the data I put on them i,e, packages, data, configuration changes synced between the two or do I have to do it somehow?

Thanks in advance and great improvement.

paulj
Starter
Starter
Posts: 24
Joined: 12 Mar 2015 01:48
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#33

Post by paulj » 14 Jan 2019 19:15

Question.

I am using this install on a (2)128GB mirror and it appears to be working great, however…

my memory usage starts out pretty low but increases to about 97% after a few days.

I have 16GB of memory.

Is this normal?

Paul
Xigmanas 11.2.0.4 - Omnius (revision 6315)
Supermicro A1SRi x64-full on Intel(R) Atom(TM) CPU C2758 @ 2.40GHz 16GB ECC memory
ZFS on Root Install

User avatar
raulfg3
Site Admin
Site Admin
Posts: 4772
Joined: 22 Jun 2012 22:13
Location: Madrid (ESPAÑA)
Contact:
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#34

Post by raulfg3 » 14 Jan 2019 21:37

paulj wrote:
14 Jan 2019 19:15
Question.

I am using this install on a (2)128GB mirror and it appears to be working great, however…

my memory usage starts out pretty low but increases to about 97% after a few days.

I have 16GB of memory.

Is this normal?

Paul
depend, RAM is for use it.

please post more info about your system and about memory use (Status > Monitoring > Memory Usage)

Do you use ZFS on Mirror? <- If yes revise System > Advanced > loader.conf

about vfs.zfs.arc_max & min and adapt values to your 12GB system
You do not have the required permissions to view the files attached to this post.
12.0.0.4 - BETA (revision 6412)+OBI on SUPERMICRO X8SIL-F 8GB of ECC RAM, 12x3TB disk in 3 vdev in RaidZ1 = 32TB Raw size only 22TB usable

Wiki
Last changes
Old Wiki

paulj
Starter
Starter
Posts: 24
Joined: 12 Mar 2015 01:48
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#35

Post by paulj » 14 Jan 2019 23:19

depend, RAM is for use it.

please post more info about your system and about memory use (Status > Monitoring > Memory Usage)

Do you use ZFS on Mirror? <- If yes revise System > Advanced > loader.conf

about vfs.zfs.arc_max & min and adapt values to your 12GB system
System I'm running is in my sig (apparently doesn't show on quick reply):

Xigmanas 11.2.0.4 - Omnius (revision 6315)
Supermicro A1SRi x64-full on Intel(R) Atom(TM) CPU C2758 @ 2.40GHz 16GB ECC memory
ZFS on Root Install mirror on (2)128GB SSD drives

Status/Monitoring/Memory Usage…

My system doesn't have that Menu Option. I have (2) choices, System Load and CPU Load. I am running Extended GUI.

The other screenshots are attached.
You do not have the required permissions to view the files attached to this post.
Xigmanas 11.2.0.4 - Omnius (revision 6315)
Supermicro A1SRi x64-full on Intel(R) Atom(TM) CPU C2758 @ 2.40GHz 16GB ECC memory
ZFS on Root Install

User avatar
JoseMR
Hardware & Software Guru
Hardware & Software Guru
Posts: 965
Joined: 16 Apr 2014 04:15
Location: PR
Contact:
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#36

Post by JoseMR » 14 Jan 2019 23:30

cookiemonster wrote:
08 Jan 2019 23:34
Hi JoseMR. I finally upgraded to RootOnZFS.
Some feedback from my setup. I was on embedded on a single USB stick with a data partition and swap on it.
1. I'm sure I read somewhere and was logical anyway to backup the data in any existing partition on the embedded installation media. I ignored the advice expecting to be able to mount it after installation but the data was not there. I put that as experience and so others don't make the same mistake. I'll have to track what was there and move on.
2. My installation has one HBA where all disks are attached to. My main pool and now my new two mirrored SSDs for the root pool. After installation and reboot I was unable to boot to new disks. For some reason the PCI slot it is in was disabled in BIOS. It took me a while and many reboots to track the problem. Solved now. Booted to Full on RootOnZFS, re-scanned for disks, force imported the pool. All good.
3. Actual feedback: I chose mirrored install on default OS partition size, chose my choice of size for mirrored swap and defaults for size of data partition. I ended up with default choices of size for OS and data that are not of my liking. They are Root: 59.59 GB, Data 57 GB, Swap 4.29 GB.
My point I chose swap 4 GB and left the installer chose the sizes for data and root. It would be even better if the installer gave the option at the end of giving the sizes it works out by choosing defaults and the option to go back to change them. Unless I didn't use the tool correctly and this was there and I missed it.
4. Question if I may: Once rebooted and the disks were presented by BIOS to chose device to boot from, in my case it gave me the option of choosing one of the two devices I installed in mirror to. I've chosen the first in the just because it is the first.
4.a. Does it matter which one I chose?
4.b. Am I right in thinking that the OS will, those two being mirrored, keep the data I put on them i,e, packages, data, configuration changes synced between the two or do I have to do it somehow?

Thanks in advance and great improvement.

Hi cookiemonster, sorry for overlook and the late reply, but I will try to answer your questions as simple as I can.

On regards to #1, yes, the user should backup the existing "Data" partition on Embedded if this data is very important(though unlikely) before a re-install generally, this is why is strongly advised not to put any critical "Data" on Embedded data partition unless this data is mirrored which is only available in RootOnZFS.

On regards to #2, if you are using an HBA for all, ZFS Data pool and for RootOnZFS, the motherboard and/or the HBA need boot support to be enabled to boot from the RootOnZFS drive(s), in my case I personally use the onboard SATA ports for OS and my LSI HBA for the ZFS storage pool, but either way it should work with no problems.

On regards to #3, the latest ZFS installer has the most sane defaults, and if you chose Mirror you will also noticed that there are 2 modes for the SWAP, one is redundant and the other is just like a JBod or spanned, if you chose Mirrored Swap @ 2GB in size, they will remains at 2GB redundant, if you chose Multiple Swap @ 2GB they will be summed to a 4GB total, Mirrored Swap is the recommended choice for maximum reliability and redundancy, as per the "Data" partition, under ZFS there is no such "Data" partition, but rather filesystems and as a plus no need to bother any longer with mounting this(which exception to legacy mount points), if you opted to create a "Data" filesystem(a dataset) under RootOnZFS, this will be automatically available after reboot and the space is shared across the "zroot" which is the tank for the Boot Environments, /tmp, /var and Data if user opted for it, this meant that the overall "zroot" space usage meter is shared between this filesystems, OTOH, there is the option for specify the zroot space you want, this is most likely for users who want to dedicate a disk partition solely for ZFS L2ARC, ZIL etc. under more complex and/or odd scenarios.

On regards to #4 and #4a, it does not mater from what drive you boot from as long as they mirrored makes no difference, between for keep an order and for administration simplicity, most users chose the first drive as the initial BIOS/EFI boot subsequently the second one etc., in the even of a failed one, the other will remain working until administrator maintenance schedule.

On regards to #4b, all the data put on RootOnZFS Mirror(either system or personal) is exactly as safe as any ZFS Mirror with all the self-healing and check-summing features of the ZFS, though I strongly recommend to leave the personal/critical data to be stored in a separate ZFS pool, between if the hardware is limited to one or 2 drives only, the user can still use the "Data" option to put their data on it and perform regular backups whenever possible.


If you need more help in regards to RootOnZFS, please don't hesitate to ask.

Regards
System: FreeBSD 12, MB: Supermicro X8SI6-F, Xeon X3450, 16GB DDR3 ECC RDIMMs.
Addons at GitHub
JoseMRPubServ

paulj
Starter
Starter
Posts: 24
Joined: 12 Mar 2015 01:48
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#37

Post by paulj » 15 Jan 2019 04:25

Now I have another problem.

I did two things (I should know better):

1. Set my jail to start on boot in The Brig
2. set vfs.zfs.arc_max to 16G and ifs.zfs.arc_min to 8G

Reboot, now no webGUI.

I have a snapshot saved from before I made these changes. How can I boot into that snapshot from the command line?

Paul
Xigmanas 11.2.0.4 - Omnius (revision 6315)
Supermicro A1SRi x64-full on Intel(R) Atom(TM) CPU C2758 @ 2.40GHz 16GB ECC memory
ZFS on Root Install

User avatar
raulfg3
Site Admin
Site Admin
Posts: 4772
Joined: 22 Jun 2012 22:13
Location: Madrid (ESPAÑA)
Contact:
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#38

Post by raulfg3 » 15 Jan 2019 07:18

connect a monitor & keyboard and use menu.

option 1 or 2 perhaps can help.

you can try to do the same by SSH
12.0.0.4 - BETA (revision 6412)+OBI on SUPERMICRO X8SIL-F 8GB of ECC RAM, 12x3TB disk in 3 vdev in RaidZ1 = 32TB Raw size only 22TB usable

Wiki
Last changes
Old Wiki

User avatar
JoseMR
Hardware & Software Guru
Hardware & Software Guru
Posts: 965
Joined: 16 Apr 2014 04:15
Location: PR
Contact:
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#39

Post by JoseMR » 15 Jan 2019 11:07

paulj wrote:
14 Jan 2019 19:15
Question.

I am using this install on a (2)128GB mirror and it appears to be working great, however…

my memory usage starts out pretty low but increases to about 97% after a few days.

I have 16GB of memory.

Is this normal?

Paul
Hi paulj, this high ZFS usage is not tied only to RootOnZFS Platforms, but for ZFS on general and is a completely a normal behavior, however if you want to prevent ZFS from use all your RAM as cache(ARC), you can limit it based on your system avail RAM, for example if you have 16GB RAM, and you will use additional services rough on memory such as Plex, VirtualBox VM's etc., you can limit your ARC_MAX to 12G for example.

paulj wrote:
15 Jan 2019 04:25
Now I have another problem.

I did two things (I should know better):

1. Set my jail to start on boot in The Brig
2. set vfs.zfs.arc_max to 16G and ifs.zfs.arc_min to 8G

Reboot, now no webGUI.

I have a snapshot saved from before I made these changes. How can I boot into that snapshot from the command line?

Paul
Hi, some users have reported XigmaNAS boot hang after set their jails to start manually on TheBrig, also other have been reported an odd network settings as well while working on TheBrig, however you need either IPMI or a physical monitor connected to the server to solve such issue easier, perhaps in the jails sub-forum and/or TheBrig thread you may get better support, though you can also run the jails purely from CLI optionally.

As a side note, you don't have to set "vfs.zfs.arc_max" to 16G since you already have 16G system RAM, set to a lower value here(see previous replies), also you don't actually have to bother with "zfs.arc_min" leave this option turned off and let ZFS manage this automatically.

Regards
System: FreeBSD 12, MB: Supermicro X8SI6-F, Xeon X3450, 16GB DDR3 ECC RDIMMs.
Addons at GitHub
JoseMRPubServ

User avatar
raulfg3
Site Admin
Site Admin
Posts: 4772
Joined: 22 Jun 2012 22:13
Location: Madrid (ESPAÑA)
Contact:
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#40

Post by raulfg3 » 15 Jan 2019 12:52

paulj wrote:
14 Jan 2019 23:19

My system doesn't have that Menu Option. I have (2) choices, System Load and CPU Load. I am running Extended GUI.

The other screenshots are attached.
YES, you have the option, but must be enabled and configured. System | Advanced | Monitoring

referred to fs.zfs.arc_max

Please set to 8GB and enable it

and fs.zfs.arc_min set to 4GB and enable it. (or not, really have no impact on your system)

And reboot, <- 8 & 4 are normal values for 16GB RAM so 8GB are free for O.S.

Then monitorize RAM usage to be sure that meets your expect
12.0.0.4 - BETA (revision 6412)+OBI on SUPERMICRO X8SIL-F 8GB of ECC RAM, 12x3TB disk in 3 vdev in RaidZ1 = 32TB Raw size only 22TB usable

Wiki
Last changes
Old Wiki

paulj
Starter
Starter
Posts: 24
Joined: 12 Mar 2015 01:48
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#41

Post by paulj » 15 Jan 2019 14:13

Guys, thanks for the replies.

My system has an IPMI interface so I always a "monitor/keyboard" available.

There's no booth hang that I can tell. It boots normally and the Default screen is available on the console.

The IP address is the same as it has been, so there doesn't seem like a reason for the webGUI not to come up.

I get the message shown in the attachment:
You do not have the required permissions to view the files attached to this post.
Xigmanas 11.2.0.4 - Omnius (revision 6315)
Supermicro A1SRi x64-full on Intel(R) Atom(TM) CPU C2758 @ 2.40GHz 16GB ECC memory
ZFS on Root Install

User avatar
raulfg3
Site Admin
Site Admin
Posts: 4772
Joined: 22 Jun 2012 22:13
Location: Madrid (ESPAÑA)
Contact:
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#42

Post by raulfg3 » 15 Jan 2019 15:27

try http instead https unless you are sure that have configured XigmaNAS properly to use https
12.0.0.4 - BETA (revision 6412)+OBI on SUPERMICRO X8SIL-F 8GB of ECC RAM, 12x3TB disk in 3 vdev in RaidZ1 = 32TB Raw size only 22TB usable

Wiki
Last changes
Old Wiki

paulj
Starter
Starter
Posts: 24
Joined: 12 Mar 2015 01:48
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#43

Post by paulj » 15 Jan 2019 17:21

try http instead https unless you are sure that have configured XigmaNAS properly to use https
I tried that. No go.

Also reset my IP settings using Option 2 of the menu.

No web GUI.

My Jails are running normally, both Sabnzbd and Sonarr are functioning normally and are accessible through a browser interface.

I can access the host system via ssh as normal.

Where is the setting stored that enables the jail to boot on startup?

I looked in rc.conf for both the host and jail…neither has an entry to start the jail on boot. loader.conf same…I don't see any entries regarding the jail.

Paul
Xigmanas 11.2.0.4 - Omnius (revision 6315)
Supermicro A1SRi x64-full on Intel(R) Atom(TM) CPU C2758 @ 2.40GHz 16GB ECC memory
ZFS on Root Install

User avatar
JoseMR
Hardware & Software Guru
Hardware & Software Guru
Posts: 965
Joined: 16 Apr 2014 04:15
Location: PR
Contact:
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#44

Post by JoseMR » 15 Jan 2019 18:22

paulj wrote:
15 Jan 2019 17:21
try http instead https unless you are sure that have configured XigmaNAS properly to use https
I tried that. No go.

Also reset my IP settings using Option 2 of the menu.

No web GUI.

My Jails are running normally, both Sabnzbd and Sonarr are functioning normally and are accessible through a browser interface.

I can access the host system via ssh as normal.

Where is the setting stored that enables the jail to boot on startup?

I looked in rc.conf for both the host and jail…neither has an entry to start the jail on boot. loader.conf same…I don't see any entries regarding the jail.

Paul

Hi paulj, I'm afraid you will get better support in either the jails and/or the TheBrig developer thread as previously mentioned, your issue has been previously reported and I think it was also solved as well but I can't find the thread, it was an odd network setting or something alike with TheBrig + Jails auto startup IIRC, really sorry that we can't provide more support here in the RootOnZFS thread about Jails/TheBrig etc.

EDIT: I think I found the reference thread I was mentioning: viewtopic.php?f=57&t=14224&p=88309&hili ... fig#p88272

Regards
System: FreeBSD 12, MB: Supermicro X8SI6-F, Xeon X3450, 16GB DDR3 ECC RDIMMs.
Addons at GitHub
JoseMRPubServ

paulj
Starter
Starter
Posts: 24
Joined: 12 Mar 2015 01:48
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#45

Post by paulj » 15 Jan 2019 19:32

Guys, thanks, The link JoseMR provided led to a solution to the problem.

I Reset my IP as fixed instead of DHCP, deleted lines listing the extra IP address (my jail address) from /var/etc/lighttpd.conf and my webGUI came back to life.

Starting jails at boot apparently confuses either the DHCP server or lighttpd…an extra IP gets added to the config (jail IP)and its downhill from there.

I'n not sure its a DHCP problem though because my jail IP is outside of the DHCP server's IP range.

Again many thanks for all the help.

Paul
Xigmanas 11.2.0.4 - Omnius (revision 6315)
Supermicro A1SRi x64-full on Intel(R) Atom(TM) CPU C2758 @ 2.40GHz 16GB ECC memory
ZFS on Root Install

cookiemonster
experienced User
experienced User
Posts: 94
Joined: 23 Mar 2014 02:58
Location: UK
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#46

Post by cookiemonster » 18 Jan 2019 22:25

I also managed to miss your reply JoseMR. All good and thanks for the thorough explanations.

defcon999
experienced User
experienced User
Posts: 127
Joined: 07 Dec 2013 10:55
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#47

Post by defcon999 » 27 Jan 2019 11:00

After updating my ZFSonRoot-Installation yesterday, i got he following message:

2019.01.26 21:22:02 ERROR Device for mountpoint zroot/ROOT/upgrade-2019-01-26-105136 missing or config changed!

What's the problem??

defcon999
NAS: HP MicroServer Gen8 - CPU: Intel Xeon E3-1230 V2 - QuadCore 3,3 GHz ** 16 GB ECC RAM ** 4 x 1 TB WD Red RaidZ2 ** Samsung 840 120 GB SSD Cache-Device ** 1 x 3 TB WD Red Rsync lokal via USB 3.0 Inateck USB 3.0 Dualschacht Festplatten-Dockingstation ** Cloning mit 1 x 3 TB WD RED im 2. Schacht der Docking-Station ** XigmaNAS-Version: 11.2.0.4 - Omnius (Revision 6400) ** ZFSOnRoot-Installation (full) ** OneButtonInstaller: Plex & Extended GUI -- VirtualBox: Ubuntu-Server with Pi-Hole

User avatar
JoseMR
Hardware & Software Guru
Hardware & Software Guru
Posts: 965
Joined: 16 Apr 2014 04:15
Location: PR
Contact:
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#48

Post by JoseMR » 27 Jan 2019 13:35

defcon999 wrote:
27 Jan 2019 11:00
After updating my ZFSonRoot-Installation yesterday, i got he following message:

2019.01.26 21:22:02 ERROR Device for mountpoint zroot/ROOT/upgrade-2019-01-26-105136 missing or config changed!

What's the problem??

defcon999

Hi defcon999, I can't find anything related to ZFS in regards to that Error/Warning message, also if your system is booting normally, it is most likely that the Error is coming from another source unrelated to RootOnZFS.

P.S HERE is a similar issue that was related to Extended GUI at the end, if this is the case please let eGUI developer crest to know about.

Regards
System: FreeBSD 12, MB: Supermicro X8SI6-F, Xeon X3450, 16GB DDR3 ECC RDIMMs.
Addons at GitHub
JoseMRPubServ

defcon999
experienced User
experienced User
Posts: 127
Joined: 07 Dec 2013 10:55
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#49

Post by defcon999 » 27 Jan 2019 14:48

Thanks for the answer, JoseMR

The System is booting without any problems ... PLEX, VirtualBox (Pi-Hole) and Extended GUI running in the background.

defcon999
NAS: HP MicroServer Gen8 - CPU: Intel Xeon E3-1230 V2 - QuadCore 3,3 GHz ** 16 GB ECC RAM ** 4 x 1 TB WD Red RaidZ2 ** Samsung 840 120 GB SSD Cache-Device ** 1 x 3 TB WD Red Rsync lokal via USB 3.0 Inateck USB 3.0 Dualschacht Festplatten-Dockingstation ** Cloning mit 1 x 3 TB WD RED im 2. Schacht der Docking-Station ** XigmaNAS-Version: 11.2.0.4 - Omnius (Revision 6400) ** ZFSOnRoot-Installation (full) ** OneButtonInstaller: Plex & Extended GUI -- VirtualBox: Ubuntu-Server with Pi-Hole

was-armandh
experienced User
experienced User
Posts: 80
Joined: 19 Nov 2018 11:30
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#50

Post by was-armandh » 27 Jan 2019 17:37

did my first root on ZFS full install on a SSD
I will do it again with the better 120Gb HP SSD
and again with embedded
but my first test was some what slower even with stout hardware.
have you done any speed tests
comparing embedded [on ram disk]
to root on ZFS on a SSD

my several embedded Xi are about 50% faster than my first try
You do not have the required permissions to view the files attached to this post.
by using this free advise, you, your heirs, etc
absolve, save, and hold harmless the advisor.
CAVEAT EMPTOR
YMMV

User avatar
JoseMR
Hardware & Software Guru
Hardware & Software Guru
Posts: 965
Joined: 16 Apr 2014 04:15
Location: PR
Contact:
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#51

Post by JoseMR » 27 Jan 2019 20:07

was-armandh wrote:
27 Jan 2019 17:37
did my first root on ZFS full install on a SSD
I will do it again with the better 120Gb HP SSD
and again with embedded
but my first test was some what slower even with stout hardware.
have you done any speed tests
comparing embedded [on ram disk]
to root on ZFS on a SSD

my several embedded Xi are about 50% faster than my first try

Hi armand, thanks for the feedback about RootOnZFS Platform performance, and indeed nice hardware.

Here on my tests with RootOnZFS under a VM, with as low specs as just as 2GB RAM and 2 CPU vCores on purpose, the speed between Embedded and RootOnZFS is entirely negligible, not to mention the fast boot under RootOnZFS and almost 2GB that is freed up for the system, and of course the OS benefits from ZFS ARC as well in which provides additional performance for regularly accessed files.

However RootOnZFS may be a tad slower that Embedded if running from quality USB 3.0 drives(USB 2.0 non recommended for Full Platforms), so if you are using a SATA3/6 HDD/SSD you won't notice any performance issue, think like a Vanilla FreeBSD server.

Between, please can you share on what specific operations XigmaNAS RootOnZFS is slower by "50%" in comparison to Embedded Platform?

Regards
System: FreeBSD 12, MB: Supermicro X8SI6-F, Xeon X3450, 16GB DDR3 ECC RDIMMs.
Addons at GitHub
JoseMRPubServ

was-armandh
experienced User
experienced User
Posts: 80
Joined: 19 Nov 2018 11:30
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#52

Post by was-armandh » 27 Jan 2019 21:40

UPDATE
3 TESTS produce the same result. ZFS root on cheapo SSD, good SSD, and embedded from the good SSD.
less than 2.3% variation and I doubt it is the over-provisioned computer. that leaves only....
the elderly 250Gb Seagate SATA drive I was using for the tests. DUH!
earlier
backing up my main i7 drive that includes the (C) partition is what I regularly keep duration times.
it is not yet an apples to apples test
but soon I hope to do a few more tests to get results that can be equated [done]
so far I have compared my work bench [power hungry] 2 Xeon 32Gb RAM with i3 & i5 Xi with 8Gb RAM
since it is not the same hardware there may be other issues the i3 & i5 have name brand SSD.
my next test in the Xeon box will be with a HP SSD then the same HP SSD with embedded..
again the POS SSD may yet be the problem here as well accounting for the slow back-up
[NO; chief suspect is now the Data Drive]
none of my booting is USB, NADA, all SATA SSDs, Intel or HP brand [now]
Last edited by was-armandh on 28 Jan 2019 16:32, edited 1 time in total.
by using this free advise, you, your heirs, etc
absolve, save, and hold harmless the advisor.
CAVEAT EMPTOR
YMMV

User avatar
JoseMR
Hardware & Software Guru
Hardware & Software Guru
Posts: 965
Joined: 16 Apr 2014 04:15
Location: PR
Contact:
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#53

Post by JoseMR » 28 Jan 2019 16:28

Hi armand thanks for the update, really appreciated, I have to note that the previous speed difference/tests on my VM's between Embedded and *Full was merely for the operating system.

Given equal hardware, a side by side bench test between Embedded and *Full(even LiveMedia), should be the same for services such as Samba, Rsync, FTP, SSH etc., since all platforms shares the same package/settings assuming they are same version/revision.

If you are experiencing a speed difference for the backup between platforms wen testing(I assume you are using Samba), then something may be miss-configured on your end.

P.S You may want to use "dd" and "Iperf" for disk and network I/O performance measurements while benching as well.

Regards
System: FreeBSD 12, MB: Supermicro X8SI6-F, Xeon X3450, 16GB DDR3 ECC RDIMMs.
Addons at GitHub
JoseMRPubServ

was-armandh
experienced User
experienced User
Posts: 80
Joined: 19 Nov 2018 11:30
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#54

Post by was-armandh » 28 Jan 2019 16:55

I think, at this point, It is the data test-drive I was using that was slow. [unexpected result]
or network use or something at the client computer?? to be determined
and, in the past, I have avoided all the USB boot problems by using SATA connected industrial flash [now SSDs]
add to that an excellent UPS and I expect Xi to be as steady as N4F, and it was VERY steady!
since my 3 Xi boxes are for storage only, I am in no hurry to switch from embedded, but now I know how.
the last boot fail was a bad SATA connection.
but if I need redundant boot storage ...... full Xi root on ZFS will be my first choice
by using this free advise, you, your heirs, etc
absolve, save, and hold harmless the advisor.
CAVEAT EMPTOR
YMMV

was-armandh
experienced User
experienced User
Posts: 80
Joined: 19 Nov 2018 11:30
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#55

Post by was-armandh » 26 Feb 2019 08:32

Happy for KTL's/others success.
dirt cheap [small capacity] name brand SSDs
have made my day for booting XigmaNAS
no more futzing around with pacific rim junk USB sticks.
I now think most of my speed differences are due to my son's gaming hours.
by using this free advise, you, your heirs, etc
absolve, save, and hold harmless the advisor.
CAVEAT EMPTOR
YMMV

User avatar
JoseMR
Hardware & Software Guru
Hardware & Software Guru
Posts: 965
Joined: 16 Apr 2014 04:15
Location: PR
Contact:
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#56

Post by JoseMR » 26 Feb 2019 15:15

was-armandh wrote:
26 Feb 2019 08:32
Happy for KTL's/others success.
dirt cheap [small capacity] name brand SSDs
have made my day for booting XigmaNAS
no more futzing around with pacific rim junk USB sticks.
I now think most of my speed differences are due to my son's gaming hours.

Hi armandh, I've been abit absent since working extra here for my DIY audio projects, but always monitoring emails and looking around in regards to Extensions and RootOnZFS to provide support whenever possible.

To bring some more in regards to RootOnZFS testing and success, I've been working on this platform since late N4F v9.x and started a beta thread for public testing with an unofficial RootOnZFS installer on N4F 10.x before integration IIRC.

Even with very limited feedback from few forum members and the lack of WebGUI upgrades at that time(supported since Rev.4159), RootOnZFS Platform has grown and proven to be the most resilience/bulletproof installation method not only for this project, and all inherited from FreeBSD and the great beadm.

Unfortunately, even with all the bells and whistles RootOnZFS Platform can provide out-of-the-box, it may overwhelm novices and some experienced users with lack on system administration knowledge and best practices.

Regards
System: FreeBSD 12, MB: Supermicro X8SI6-F, Xeon X3450, 16GB DDR3 ECC RDIMMs.
Addons at GitHub
JoseMRPubServ

bjs
NewUser
NewUser
Posts: 3
Joined: 08 Mar 2019 12:58
Location: Virginia, USA
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#57

Post by bjs » 25 Mar 2019 15:09

Okay, so I'm seriously considering updating my rock solid FreeNAS 7.2 server to a XigmaNAS 11.2 RootOnZFS install with a couple new 120 GB SSDs and have a few questions... I'm not new to FreeBSD but have no experience with ZFS as of yet... My system is older, Intel DG33FB motherboard with a Q6600 2.44 GHz Core2 Quad chip and 8 GB of RAM...

First question is I see nothing in the thread about setting the install drives to use a 4k block size??? Is this taken care of automatically somehow or is there something extra that needs to be configured???

Second question, with only 8 GB of RAM, what would be the suggested setting for fs.zfs.arc_max??? Other than Smaba, NFS, and NUT, no other services are or planned to be run on the server...

Main use for the server is to hold media files for a couple PLEX media servers, one running on a Windows box, a second running on FreeBSD, and three local Windows system backup storage... Currently I have NFS shares mounted on the FreeBSD box for media files and for backup of PLEX's install and database... I have read where NFS, specifically NFS writes can benefit from using L2ARC... Obviously the media is only read but the occasional full PLEX install backup and the scheduled database backups every three days would be NFS writes... Last question, if needed, how much space should I leave on the SSDs to use for L2SRC based on the above recommendation for fs.zfs.arc_max???

Thanks in advanced for any thoughts, guidance, or recommendations anyone can offer...

User avatar
JoseMR
Hardware & Software Guru
Hardware & Software Guru
Posts: 965
Joined: 16 Apr 2014 04:15
Location: PR
Contact:
Status: Offline

Re: XigmaNAS RootOnZFS Install Manual

#58

Post by JoseMR » 25 Mar 2019 17:11

bjs wrote:
25 Mar 2019 15:09
Okay, so I'm seriously considering updating my rock solid FreeNAS 7.2 server to a XigmaNAS 11.2 RootOnZFS install with a couple new 120 GB SSDs and have a few questions... I'm not new to FreeBSD but have no experience with ZFS as of yet... My system is older, Intel DG33FB motherboard with a Q6600 2.44 GHz Core2 Quad chip and 8 GB of RAM...

First question is I see nothing in the thread about setting the install drives to use a 4k block size??? Is this taken care of automatically somehow or is there something extra that needs to be configured???

Second question, with only 8 GB of RAM, what would be the suggested setting for fs.zfs.arc_max??? Other than Smaba, NFS, and NUT, no other services are or planned to be run on the server...

Main use for the server is to hold media files for a couple PLEX media servers, one running on a Windows box, a second running on FreeBSD, and three local Windows system backup storage... Currently I have NFS shares mounted on the FreeBSD box for media files and for backup of PLEX's install and database... I have read where NFS, specifically NFS writes can benefit from using L2ARC... Obviously the media is only read but the occasional full PLEX install backup and the scheduled database backups every three days would be NFS writes... Last question, if needed, how much space should I leave on the SSDs to use for L2SRC based on the above recommendation for fs.zfs.arc_max???

Thanks in advanced for any thoughts, guidance, or recommendations anyone can offer...

Hi, I'm currently working but I will try to quick respond to your questions without entering into much technical side.

In regards to your first question, the OS takes care of this automatically(ashift=12), to verify you can use "zdb -C poolname | grep ashift".

Code: Select all

xigmanas: ~# uname -ori
FreeBSD 11.2-RELEASE-p8 XIGMANAS-x64
xigmanas: ~# zdb -C zroot | grep ashift
                ashift: 12
xigmanas: ~#
In regards to your second question, I've used a similar setup like you with 8GB non-ECC RAM, I've set the ZFS_ARC_MAX to "6G" and the where no SWAP hits nor performance issues running Plex, Samba, Rsync, SSH, CTL, Webserv.

In regards to the L2ARC, for this particular Home/SOHO setups with just few clients, there is no need to bother with L2ARC actually IMO, the read improvements are negligible and it may compromise performance on legacy hardware with limited RAM.

Regards
System: FreeBSD 12, MB: Supermicro X8SI6-F, Xeon X3450, 16GB DDR3 ECC RDIMMs.
Addons at GitHub
JoseMRPubServ

Post Reply

Return to “Install XigmaNAS”