*New 12.1 series Release:
2020-09-13: XigmaNAS 12.1.0.4.7743 - released

*New 11.4 series Release:
2020-09-12: XigmaNAS 11.4.0.4.7741 - released!


We really need "Your" help on XigmaNAS https://translations.launchpad.net/xigmanas translations. Please help today!

Producing and hosting XigmaNAS costs money. Please consider donating for our project so that we can continue to offer you the best.
We need your support! eg: PAYPAL

Export & detatch operations at shutdown

Encrypting information and help
Forum rules
Set-Up GuideFAQsForum Rules
Post Reply
faust
NewUser
NewUser
Posts: 4
Joined: 20 Jan 2017 03:52
Status: Offline

Export & detatch operations at shutdown

#1

Post by faust »

I've got a couple of questions about how NAS4Free handles the import/export and safe operations to do manually before shutdown. Sorry for the wall of text lead-in.

Having read a number of threads and the wiki articles on the topic of encrypted disks and zpools (being new to N4F and ZFS), I was able to set everything up successfully on my embedded install as follows:

1) Encrypt the disks: Disks > Encryption > Management
2) Format the attached encrypted disks for ZFS: Disks > Management > HDD Format
3) Add virtual devices for ZFS: Disks > ZFS > Pools > Virtual Device
4) Created the zpool: Disks > ZFS > Pools > Management
5) Created datasets: Disks > ZFS > Datasets > Dataset

I'm able to attach the disks and import the pool after a reboot. Attaching is straightforward (though tedious through the web interface). But there's always some (apparently) non-critical error messages when importing the zpools. Other forum posts advise to use the panel found at "Disks > ZFS > Configuration > Synchronize" to "synchronize the zpools." I'm not entirely sure what that means, but it apparently runs `zpool import -d /dev -a -f`

Which, to the best of my understanding, iterates through the contents of /dev searching for ZFS virtual devices, parsing them somehow and determining whether they're a part of a pool, and then does a forced import of the pool. When attempting to import a zpool directly from the shell, `zpool import mypool` I get an error suggesting that the "pool may be in use from other system" which it obviously isn't. Following the advice to force the import with `zpool import -f mypool` works fine but it makes me wonder why this error shows up every time the system is rebooted.

I experimented with doing a manual export of the zpool before a reboot (`zpool export mypool`) and after a reboot, it doesn't seem to require the force and no error message is received from the regular import. Does N4F not export the pool during shutdown? Is this an issue?

I started making some scripts to speed up the attach/import process and tried making a shutdown script to export/detach as well. It works fine so far that I can tell but I'm just wondering if there are any consequences to manually exporting the zpools at this stage of the shutdown process. I just added my script as a "Shutdown" type in System > Advanced > Command Scripts.

The script is basically this:

Code: Select all

#!/bin/bash
zpool export mypool
geli detach /dev/ada0
geli detach /dev/ada1
I'm just not sure if there are any kind of operations relevant to the zpool the run during shutdown (kicking SMB users off the system, any kind of journaling/fsck type operations, etc) that make this process a bad (or nearly bad) idea.

Just to "bottom line" my post here:
(1) Does N4F export the zpools "properly" on shutdown? Should it?
(2) Are there any issues with exporting zpools and then detaching encrypted disks with a shutdown script?

faust
NewUser
NewUser
Posts: 4
Joined: 20 Jan 2017 03:52
Status: Offline

Re: Export & detatch operations at shutdown

#2

Post by faust »

Just posting the helper scripts I made for posterity. The general use of encrypted devices makes it a bit tough to boot smoothly without such helper scripts in my opinion. Some of this feels a bit janky. I don't use FreeBSD so I don't know if there's an equivalent to /sys/block/ which makes handling of removable easy. Also, I don't know how to check if a zpool is currently being written to. I'd like to be able to have that as an interlock for safety. Oh well.

Attach encrypted devices:
Uses a hard coded list, prompts for passphrases, if you hit "enter" for the passphrase, it attempts to use the previously entered passphrase (if you have one passphrase for all devices, for instance).

Code: Select all

#!/bin/bash

# Hard-coded device list here
SECURE_DEVS=( ada0 ada1 ada6 ada7 )

echo "Attaching encrypted disks..."
echo " - Devices: "${SECURE_DEVS[@]}

PREVPASS=""
for ENCDEV in "${SECURE_DEVS[@]}"; do
	if [ ! -c /dev/$ENCDEV.eli ]; then

		echo -n "    $ENCDEV : Enter passphrase: "
		read -s PASSPHRASE

		if [ "$PASSPHRASE" = "" ]; then PASSPHRASE="$PREVPASS"; fi
		PREVPASS=$PASSPHRASE

		echo $PASSPHRASE | geli attach -j - /dev/$ENCDEV

		if [ -c /dev/$ENCDEV.eli ]; then echo " SUCCESS"
		else echo " FAIL"
		fi
	else
		echo "     $ENCDEV : Volume already attached as /dev/$ENCDEV.eli"
	fi
done
Detach encrypted devices:
Again, uses a hard-coded list. Checks to see if the device is part of an imported zpool before detaching.

Code: Select all

#!/bin/bash

# Hard-coded device list here
SECURE_DEVS=( ada0 ada1 ada6 ada7 )

for ENCDEV in "${SECURE_DEVS[@]}"; do
	IMPORTED=$(zpool status | grep "$ENCDEV.eli")
	if [ -n "$IMPORTED" ]; then
		echo " - $ENCDEV : Skipping (part of an active zpool)"
	else
		echo -n " - $ENCDEV : Detaching... "
		geli detach /dev/$ENCDEV
		if [ ! -c /dev/$ENCDEV.eli ]; then echo " SUCCESS"
		else echo " FAIL"
		fi
	fi
done
Import encrypted zpools:
Hard-coded list of zpools which are on encrypted devices.Checks to see what zpools are available to import from the list and imports them. Note that since I export my zpools on shutdown, I don't ever need to force import the zpools.

Code: Select all

#!/bin/bash

# Hard-coded zpool list here
SECURE_POOLS=( md0 md1 )

echo "Importing encrypted zpools..."
for POOL in "${SECURE_POOLS[@]}"; do
        echo -n " - $POOL "
        ISONLINE=$(zpool import -d /dev | grep "$POOL.*ONLINE")
        IMPORTED=$(zpool list | grep "$POOL")
        if [ -n "$ISONLINE" ]; then
                echo -n "(online) : Importing... "
                zpool import $POOL
                IMPORTED=$(zpool list | grep "$POOL")
                if [ -n "$IMPORTED" ]; then echo "DONE"; fi
        elif [ -n "$IMPORTED" ]; then echo "(imported) : Skipping"
        else echo " (exception) : Neither online nor imported, skipping"
        fi
done
Export encrypted zpools:
I use this as a shutdown script (I noticed from the log that the encrypted devices seem to get detached without my help so I don't add that operation). It just exports zpools from the hard-coded list if they seem to be active.

Code: Select all

#!/bin/bash

# Hard-coded zpool list here
SECURE_POOLS=( md0 md1 )

echo "Exporting encrypted zpools..."
for POOL in "${SECURE_POOLS[@]}"; do
        echo -n " - $POOL "
        ISONLINE=$(zpool import -d /dev | grep "$POOL.*ONLINE")
        IMPORTED=$(zpool list | grep "$POOL")
        if [ -n "$IMPORTED" ]; then
		echo -n "(imported) : Exporting..."
		zpool export $POOL
                IMPORTED=$(zpool list | grep "$POOL")
                if [ ! -n "$IMPORTED" ]; then echo " DONE"; fi
        elif [ -n "$ISONLINE" ]; then
                echo "(online) : Already exported, skipping... "
        else echo " (exception) : Neither online nor imported, skipping"
        fi
done
Mount encrypted volumes:
I have this set up as a postinit script. For my encrypted disks, I added a keyfile to slot 1 (slot 0 is the passphrase). I keep these keyfiles encrypted elsewhere. Before I boot up the NAS, I decrypt/extract these keyfiles, put them on a FAT32 USB drive, and plug the drive into one of the higher enumerated USB ports (my embedded OS drive is in the 0th slot). The script is neutral to the enumeration, but NAS4Free isn't if you use other partitions on the OS device. When the script runs, it looks through /dev/da[0-9] looking for disks that have unmounted partitions (not my OS drive), it tries to mount those partitions, looks for key files, and tries to use those to attach the encrypted devices, and then uses the same methodology as above to import the zpools. It tries to be very careful when it comes to creating/deleting the mount point for the key drive. There are also some reference commands at the top. I realize that these keyfiles are overkill.

Code: Select all

#!/bin/bash

# Reference
# Scan for pools in /dev and force import: zpool import -d /dev -a -f
# Set a geli keyfile in slot 1: geli setkey -v -n 1 -P -K /path/to/device0.key /dev/device0
# Detach geli device: geli detach /dev/device0
# Export zpool: zpool export poolname
# Pass a variable to geli to unlock vial passphrase: echo $PASSPHRASE | /sbin/geli attach -j - /dev/device0
# Read a passphrase without echo: read -s PASSPHRASE
# Generate keyfile: head -c 4000 /dev/urandom | uuencode -m - | head -n 65 | tail -n 64 > device0.key

# Hard-coded keypath (relative to the USB drive root)
KEYPATH="/keys/"

# Hard-coded devices and zpools here
SECURE_DEVS=( ada0 ada1 ada6 ada7 )
SECURE_POOLS=( md0 md1 )

function unlock_device {
	ATTACHEDDEVICE=false
	geli attach -p -k $KEYFILE /dev/$ENCDEV 2>/dev/null
	if [ -c /dev/$ENCDEV.eli ]; then ATTACHEDDEVICE=true; fi
}

echo "========================== Mounting Encrypted Volumes =========================="
echo "Locating keyfiles..."
echo " - Checking /dev/da[0-9]"

for DEV in $(ls /dev/da[0-9]); do
	MOUNTED=false
	echo -n " - $DEV"
	for DEVFN in $(ls $DEV[p,s]*); do
		TESTDATA=$(df $DEVFN 2>/dev/null | grep "$DEVFN")
		if [ -n "$TESTDATA" ]; then MOUNTED=true; fi
	done
	if ! $MOUNTED; then
		DEVPATH=$DEV
		echo " (OK)"
	else
		echo " (XX)"
	fi
done

if [ ! -n "$DEVPATH" ]; then
	echo "No suitable devices found, aborting."
	echo "================================================================================"
	exit 1
fi

for DEVPT in $(ls $DEVPATH[p,s]*); do	

	DEVBN=$(basename $DEVPT)
	MNTPT="/tmp-mnt-$DEVBN"

	if [ -d $MNTPT ]; then
		echo "    - Existing mount point found: $MNTPT"
	else
		echo "    - Creating mount point: $MNTPT"
		mkdir $MNTPT
	fi

	if [ -n "$(find $MNTPT -prune -empty)" ]; then
		echo "    - Mounting: $DEVPT"
		mount_msdosfs -o ro $DEVPT $MNTPT
	else
		echo "    - Mount point not empty: $MNTPT : (ABORT)"
		echo "================================================================================"
		exit 1
	fi

	if [ -d $MNTPT$KEYPATH ]; then
		echo "    - Key path found: $MNTPT$KEYPATH"
		echo "Attaching encrypted devices..."
		for ENCDEV in "${SECURE_DEVS[@]}"; do
			KEYFILE=$MNTPT$KEYPATH$ENCDEV.key
			if [ ! -c /dev/$ENCDEV.eli ] && [ -e $KEYFILE ]; then
				echo -n " - /dev/$ENCDEV (key: $KEYFILE) : "
				unlock_device
				if ! $ATTACHEDDEVICE; then echo "FAIL (geli error)"
				else echo "SUCCESS"
				fi
			elif [ ! -c /dev/$ENCDEV.eli ] && [ ! -e $KEYFILE ]; then
				echo " - /dev/$ENCDEV (key: $KEYFILE) : FAIL"
				echo "      Missing key file, unlock not attempted"
			else
				echo " - /dev/$ENCDEV (unlocked) : SKIPPING"
				echo "      Device /dev/$ENCDEV.eli already exists, device already unlocked"
			fi
		done
	else
		echo "    - Key path not found: $MNTPT$KEYPATH"
	fi

	echo "Cleaning up USB volume..."
	echo " - Unmounting: $DEVPT"
	umount $DEVPT

	if [ -n "$(find $MNTPT -prune -empty)" ]; then
		echo " - Deleting mount point: $MNTPT"
		rmdir $MNTPT
	else
		echo " - Mount point not empty: $MNTPT"
		echo "      Leaving in place (possible failed umount)"
	fi


done

echo "Importing zpools..."
for POOL in "${SECURE_POOLS[@]}"; do
	echo -n " - $POOL "
	ISONLINE=$(zpool import -d /dev | grep "$POOL.*ONLINE")
	IMPORTED=$(zpool list | grep "$POOL")
	if [ -n "$ISONLINE" ]; then
		echo -n "(online) : Importing... "
		zpool import $POOL
		IMPORTED=$(zpool list | grep "$POOL")
		if [ -n "$IMPORTED" ]; then echo "DONE"; fi
	elif [ -n "$IMPORTED" ]; then echo "(imported) : Skipping"
	else echo " (exception) : Neither online nor imported, skipping"
	fi
done
echo "================================================================================"

faust
NewUser
NewUser
Posts: 4
Joined: 20 Jan 2017 03:52
Status: Offline

Re: Export & detatch operations at shutdown

#3

Post by faust »

Just adding an updated version of the "mount encrypted volumes" script. This one will also mount encrypted UFS volumes using their file system label. The only addition is the "Mounting UFS volumes..." block at the very end and the SECURE_UFS variable at the top.

Code: Select all

#!/bin/bash

# Reference
# Scan for pools in /dev and force import: zpool import -d /dev -a -f
# Set a geli keyfile in slot 1: geli setkey -v -n 1 -P -K /path/to/device0.key /dev/device0
# Detach geli device: geli detach /dev/device0
# Export zpool: zpool export poolname
# Pass a variable to geli to unlock vial passphrase: echo $PASSPHRASE | /sbin/geli attach -j - /dev/device0
# Read a passphrase without echo: read -s PASSPHRASE
# Generate keyfile: head -c 4000 /dev/urandom | uuencode -m - | head -n 65 | tail -n 64 > device0.key

# Hard-coded keypath (relative to the USB drive root)
KEYPATH="/keys/"

# Hard-coded devices and zpools here
SECURE_DEVS=( ada0 ada1 ada2 ada7 ada8 ada3 ada4 ada5 ada6 )
SECURE_POOLS=( md0 md1 md2 )
SECURE_UFS=( sysdata )

function unlock_device {
	ATTACHEDDEVICE=false
	geli attach -p -k $KEYFILE /dev/$ENCDEV 2>/dev/null
	if [ -c /dev/$ENCDEV.eli ]; then ATTACHEDDEVICE=true; fi
}

echo "========================== Mounting Encrypted Volumes =========================="
echo "Locating keyfiles..."
echo " - Checking /dev/da[0-9]"

for DEV in $(ls /dev/da[0-9]); do
	MOUNTED=false
	echo -n " - $DEV"
	for DEVFN in $(ls $DEV[p,s]*); do
		TESTDATA=$(df $DEVFN 2>/dev/null | grep "$DEVFN")
		if [ -n "$TESTDATA" ]; then MOUNTED=true; fi
	done
	if ! $MOUNTED; then
		DEVPATH=$DEV
		echo " (OK)"
	else
		echo " (XX)"
	fi
done

if [ ! -n "$DEVPATH" ]; then
	echo "No suitable devices found, aborting."
	echo "================================================================================"
	exit 1
fi

for DEVPT in $(ls $DEVPATH[p,s]*); do	

	DEVBN=$(basename $DEVPT)
	MNTPT="/tmp-mnt-$DEVBN"

	if [ -d $MNTPT ]; then
		echo "    - Existing mount point found: $MNTPT"
	else
		echo "    - Creating mount point: $MNTPT"
		mkdir $MNTPT
	fi

	if [ -n "$(find $MNTPT -prune -empty)" ]; then
		echo "    - Mounting: $DEVPT"
		mount_msdosfs -o ro $DEVPT $MNTPT
	else
		echo "    - Mount point not empty: $MNTPT : (ABORT)"
		echo "================================================================================"
		exit 1
	fi

	if [ -d $MNTPT$KEYPATH ]; then
		echo "    - Key path found: $MNTPT$KEYPATH"
		echo "Attaching encrypted devices..."
		for ENCDEV in "${SECURE_DEVS[@]}"; do
			KEYFILE=$MNTPT$KEYPATH$ENCDEV.key
			if [ ! -c /dev/$ENCDEV.eli ] && [ -e $KEYFILE ]; then
				echo -n " - /dev/$ENCDEV (key: $KEYFILE) : "
				unlock_device
				if ! $ATTACHEDDEVICE; then echo "FAIL (geli error)"
				else echo "SUCCESS"
				fi
			elif [ ! -c /dev/$ENCDEV.eli ] && [ ! -e $KEYFILE ]; then
				echo " - /dev/$ENCDEV (key: $KEYFILE) : FAIL"
				echo "      Missing key file, unlock not attempted"
			else
				echo " - /dev/$ENCDEV (unlocked) : SKIPPING"
				echo "      Device /dev/$ENCDEV.eli already exists, device already unlocked"
			fi
		done
	else
		echo "    - Key path not found: $MNTPT$KEYPATH"
	fi

	echo "Cleaning up USB volume..."
	echo " - Unmounting: $DEVPT"
	umount $DEVPT

	if [ -n "$(find $MNTPT -prune -empty)" ]; then
		echo " - Deleting mount point: $MNTPT"
		rmdir $MNTPT
	else
		echo " - Mount point not empty: $MNTPT"
		echo "      Leaving in place (possible failed umount)"
	fi


done

echo "Importing zpools..."
for POOL in "${SECURE_POOLS[@]}"; do
	echo -n " - $POOL "
	ISONLINE=$(zpool import -d /dev | grep "$POOL.*ONLINE")
	IMPORTED=$(zpool list | grep "$POOL")
	if [ -n "$ISONLINE" ]; then
		echo -n "(online) : Importing... "
		zpool import $POOL
		IMPORTED=$(zpool list | grep "$POOL")
		if [ -n "$IMPORTED" ]; then echo "DONE"; fi
	elif [ -n "$IMPORTED" ]; then echo "(imported) : Skipping"
	else echo " (exception) : Neither online nor imported, skipping"
	fi
done

echo "Mounting UFS volumes..."
for UFSDEV in "${SECURE_UFS[@]}"; do

        DEVFN="/dev/ufs/$UFSDEV"
        MNTPT="/mnt/$UFSDEV"
        echo " - $UFSDEV : $DEVFN"

        MOUNTED=false
        TESTDATA=$(df $DEVFN 2>/dev/null | grep "$MNTPT")
        if [ -n "$TESTDATA" ]; then MOUNTED=true; fi

        if ! $MOUNTED; then

                if [ -d $MNTPT ]; then
                        echo "    - Existing mount point found: $MNTPT"
                else
                        echo "    - Creating mount point: $MNTPT"
                        mkdir $MNTPT
                fi

                if [ -n "$(find $MNTPT -prune -empty)" ]; then
                        echo -n "    - Mounting: $UFSDEV"
                        mount /dev/ufs/$UFSDEV $MNTPT

                        MOUNTOK=false
                        TESTDATA=$(df $DEVFN 2>/dev/null | grep "$MNTPT")
                        if [ -n "$TESTDATA" ]; then MOUNTOK=true; fi

                        if $MOUNTOK; then echo " (OK)"
                        else echo " (XX)"
                        fi

                else echo "    - Mount point not empty: $MNTPT : (ABORT)"
                fi

        else
                echo "    - Already mounted: $MNTPT : (DONE)"
        fi

done
echo "================================================================================"

In addition, after the "export encrypted zpools" shutdown script, I use the following script to unmount my encrypted UFS volumes (Note that it also unmounts some unionfs paths which are specific to other aspects of my setup. This is the appropriate place if they're located on the encrypted UFS volume):

Code: Select all

#!/bin/bash

SECURE_UFS=( sysdata )

echo "Unmounting unionfs on encrypted UFS volumes..."
umount /mnt/sysdata/unionfs/usr/local
umount /mnt/sysdata/unionfs/var/db/pkg
umount /mnt/sysdata/acme

echo "Unmounting encrypted UFS volumes..."
for UFSDEV in "${SECURE_UFS[@]}"; do
	echo " - $UFSDEV "
	umount /dev/ufs/$UFSDEV
done

Post Reply

Return to “Encryption”