*New 11.3 series Release:
2019-10-05: XigmaNAS 11.3.0.4.6928 - released, 11.2 series are soon unsupported!

*New 12.0 series Release:
2019-10-05: XigmaNAS 12.0.0.4.6928 - released!

*New 11.2 series Release:
2019-09-23: XigmaNAS 11.2.0.4.6881 - released!

We really need "Your" help on XigmaNAS https://translations.launchpad.net/xigmanas translations. Please help today!

Producing and hosting XigmaNAS costs money. Please consider donating for our project so that we can continue to offer you the best.
We need your support! eg: PAYPAL

Embedded RootOnZFS (A thought)

General information about XigmaNAS
Forum rules
Set-Up GuideFAQsForum Rules
Post Reply
User avatar
JoseMR
Hardware & Software Guru
Hardware & Software Guru
Posts: 1151
Joined: 16 Apr 2014 04:15
Location: PR
Contact:
Status: Offline

Embedded RootOnZFS (A thought)

#1

Post by JoseMR » 20 Dec 2018 00:23

Hello, first of all, I want to thank zoon01 and ms49434 for letting me be part on helping development of this project, while I has been contributing for quite some time in some enhancements such as RootOnZFS Platforms, LiveMedia, UEFI, Boot Loader, Embedded features, Extensions among others contributions, wanted to stay as a simply "Contributor", so every bits are reviewed by the main project developers. ;)

With some crazy ideas and all that, I wanted production users to use a reliable NAS distribution without the Legacy UFS filesystem, and based purely on ZFS + Boot Environments for enhanced availability and reliability, then I started the RootOnZFS Platform project.

Also till today some users has been using the standard Embedded Platform in production environments which is based in UFS mainly for USB sticks, hence inherit many of the problems related to this older filesystem unfortunately, with no redundancy at all thus killing the reliability.

However today I wan to share one of my oldest proposal that I re-started for share the idea to show the power of both Embedded plus ZFS, yes is an Embedded Platform running on ZFS, with the exception of the RAM Disk for "/usr/local" (still thinking on TMPFS or just ZFS as well) here.

I know majority of users will disagree on replace the older Embedded running on UFS(regardless of its limitations/issues) for this new platform, but time will tell eventually, so in the mean time my goal and possibly best solution is to add this new Platform Option in the existing RootOnZFS installer in the future if requested, as user selectable mode between Full or Embedded Platforms running from ZFS/Boot Environments.

While Full RootOnZFS Platform gives the user complete customization abilities, the Embedded RootOnZFS Platform will not retain user changes and/or packages since "/usr/local", "/var" and "/var/tmp" will be re-created at boot time just like Embedded thus giving the user the the feeling of an Embedded device plus all the features Full RootOnZFS brings such as redundancy, Boot Environments management, plus zero UFS related issues like filesystem errors on power unplugs, the fsck execute and pray for the data that will be ok etc etc. :mrgreen:


Ok lets talk less and just see how this looks alike:

HERE is the small Video showing the Boot Loader, Boot Environments ready for non-destructive Firmware upgrades that can also be easily managed from the WebGUI with another project I was working on, Note that Embedded Platforms running on ZFS has no "tmp" and "var" ZFS datasets as those are in RAM disk as expected.

Here is the zroot zpool df info, Embedded loads from the Boot Environment and recreates the mdlocal RAM disk, at this point we don't need MFSROOT and we just rely on the BE's, no need for any "DD" over and over in the disk each time we upgrade Firmware, also notice we have redundancy here for the OS, for the Swap plus we are re-creating "/usr/local base system", "/tmp", "/var/*" on RAM for the most demanding paranoids, ok this was a joke. :lol:
XN12-Embedded-ZFS-Info.png

Here is the Screenshot of the fully booted Embedded Platform in a fully Mirrored configuration:
xigmanas.local System Information.png


At this point all the features available in Full RootOnZFS Platforms are also available for Embedded Platforms running on ZFS. 8-)
Please share your thoughts since we like constrictive criticism and others users opinions and ideas.

Regards
JoseMR
You do not have the required permissions to view the files attached to this post.
System: FreeBSD 12 RootOnZFS, MB: Supermicro X8SI6-F, Xeon X3450, 16GB DDR3 ECC RDIMMs.
Addons at GitHub
JoseMRPubServ
Boot Environments Intro

User avatar
Snufkin
Advanced User
Advanced User
Posts: 289
Joined: 01 Jul 2012 11:27
Location: Etc/GMT-3 (BSD style)
Status: Offline

Re: Embedded RootOnZFS (Preview)

#2

Post by Snufkin » 20 Dec 2018 13:48

I love that someone wants to shake things up and change the world.

Some thoughts on "rethinking the basics".
  1. What is the strength of embedded software?
    NAS admin should only care about configuration info and user data which are uniqe, and not the operating system, that's why I like it.
    The weak point of embedded solution is that the feature set is limited, and I can live with it.
  2. Why do people want to install full solution?
    Because they want to break the limits and get complete feature set.
    This is not given for nothing, and in addition to user data, we have to think about unique combination of configuration data and system software.
    I don't like the idea of being a real server admin with all these system backups and restores.
  3. What could be the difference between embedded and full technically?
    Boot USB stick, locked package manager, single configuration file and RAM disk for extensive writes in system logs or database files.
  4. How could you go from embedded to full?
    Just unlock package manager, forget about single point of configuration, redirect all logs/databases to regular HDD/SSD and start thinking about full system backups of your full installation.
    I see no reasons to move FreeBSD and WebUI from USB to HDD/SSD, it's just wasting SATA sockets.
    In the end, one can use two USB sticks to implement Full Root on Mirror ZFS.
XigmaNAS development team has a unique knowledge of how to make FreeBSD server software work in embedded mode. And I have not heard about similar successes on Linux except specific cases like routers, multimedia players, etc.

Why not take embedded to the next level?
XNAS 11.3.0.4 embedded, ASUS P5B-E, Intel DC E6600, 4 GB DDR2, 2 x HGST HDN726040ALE614, 2 x WDC WD5000AAKS, Ippon Back Power Pro 400

User avatar
JoseMR
Hardware & Software Guru
Hardware & Software Guru
Posts: 1151
Joined: 16 Apr 2014 04:15
Location: PR
Contact:
Status: Offline

Re: Embedded RootOnZFS (Preview)

#3

Post by JoseMR » 20 Dec 2018 19:49

Snufkin wrote:
20 Dec 2018 13:48
I love that someone wants to shake things up and change the world.

Some thoughts on "rethinking the basics".
  1. What is the strength of embedded software?
    NAS admin should only care about configuration info and user data which are uniqe, and not the operating system, that's why I like it.
    The weak point of embedded solution is that the feature set is limited, and I can live with it.
  2. Why do people want to install full solution?
    Because they want to break the limits and get complete feature set.
    This is not given for nothing, and in addition to user data, we have to think about unique combination of configuration data and system software.
    I don't like the idea of being a real server admin with all these system backups and restores.
  3. What could be the difference between embedded and full technically?
    Boot USB stick, locked package manager, single configuration file and RAM disk for extensive writes in system logs or database files.
  4. How could you go from embedded to full?
    Just unlock package manager, forget about single point of configuration, redirect all logs/databases to regular HDD/SSD and start thinking about full system backups of your full installation.
    I see no reasons to move FreeBSD and WebUI from USB to HDD/SSD, it's just wasting SATA sockets.
    In the end, one can use two USB sticks to implement Full Root on Mirror ZFS.
XigmaNAS development team has a unique knowledge of how to make FreeBSD server software work in embedded mode. And I have not heard about similar successes on Linux except specific cases like routers, multimedia players, etc.

Why not take embedded to the next level?

Hi thank you for the nice and honest opinion in this thread, I just have to note that I've managed to run Fully Embedded(MFSROOT) too in my internal tests but its defeats the Boot Environments functionality which is the real benefit from this feature, especially the Incremental like Firmware Upgrades, however I could integrate "mdlocal into mfsroot" but the result is a very large to allocate in RAM disk, slow boot time and non elegant.

Also I have to disagree in the USB part here to save SATA ports, unless we are talking about obsolete and/or limited hardware, OS sitting on USB(s) is not a reliable nor an elegant solution in serious production environments, but it may fit the needs for the Home users eventually.

On the Full Platforms and Backups, there is no need for backup hassle, if a user opt to use a Full Platforms it means that the user know what hes doing, and most of the time this users have their automation scripts, just for an example, I personally use FreeBSD and wen I need reinstall, after Installing from LiveMedia, I just import my pool and call my automation script and it prepares my last working environment in just few minutes automatically in a new Boot Environment, not even need to import any config of any sort as every single bit is restored and started even remotely, (DevOps alike.). :ugeek:

Last but not least, I just wanted to give "OS on ZFS" options for Advanced users running Full UFS Platforms in productions environments to avoid the common disasters from USB sticks and UFS, for example This Guy had to take down his SAN's with over 1K of up-time because his USB stick or USB controller(who knows) wanted to play a joke, this is not a common issue for a small pairs of SSD's, but I know that it may also be abit costly/overkill for home usage.

P.S Thanks again for your honest opinions, and I'm really sorry if I sounded abit harsh which is not my intention, other than share few opinions from my point of view. :)

EDIT: When comparing more deeply on other similar FreeBSD based appliances running of root-on-zfs, I just realized that this is merely overkill and a plain limitation to the RootOnZFS platform itself, but just wanted to show how it looks alike.

Regards
System: FreeBSD 12 RootOnZFS, MB: Supermicro X8SI6-F, Xeon X3450, 16GB DDR3 ECC RDIMMs.
Addons at GitHub
JoseMRPubServ
Boot Environments Intro

was-armandh
experienced User
experienced User
Posts: 116
Joined: 19 Nov 2018 11:30
Location: St Louis County MO USA
Status: Offline

Re: Embedded RootOnZFS (A thought)

#4

Post by was-armandh » 05 Jan 2019 14:58

experiments are fun, some times problem solving, some times problems are more easily avoided.
first consider that the embedded once loaded to the ram drive is backed up by the boot media.
so as long as the power is OK and the ram ECC the boot media is redundant. not so with swap.
I have never had a failure using PATA or SATA connected flash or SSDs for the boot media, or swap.
[update: save once when a MOBO failed, only discovered when rebooting]
booting from a USB port was an afterthought, their intended use being printers, scanners, mouse, KB.
it is slow, the difference in booting USB vs PATA or SATA is about 3 times longer [in my experience]
not much to care about boot speed using embedded but what about the swap?
and I have seen, in the $25 range, 4 port SATA cards with the latest fastest SATA
so, yes, I much prefer to use SATA for booting, swap, and storage and leave the USB port for the keyboard
but it is just my preference, derived from good experience. [and the low price of last years SSDs]

ON THE OTHER HAND... overkill is great for mission critical use.
a pair of SSDs on a board with BIOS level mirroring would be good
ZFS mirroring even better.

and however redundant the boot media is, one still does not loose the ability to recover
using the config and OS [etc] stored elsewhere
by using this free advise, you, your heirs, etc
absolve, save, and hold harmless the advisor.
CAVEAT EMPTOR
YMMV

Post Reply

Return to “GENERAL INFORMATION”