by danmero » Mon May 04, 2009 12:02 am
Copied from the Old FreeNAS Forum - So it is archived!
The main idea is to auto-backup your important data on USB (encrypted).
1. Prepare the disk
► Create a key file
Code: Select all
$ dd if=/dev/urandom of=/temp/ukey.key count=1024
Code: Select all
$ geli init -P -s 4096 -K /temp/ukey.key /dev/da0
Code: Select all
$ geli backup da0 /temp/usb.key.backup
Code: Select all
$ geli attach -p -k /temp/ukey.key /dev/da0
► Format the disk (UFS soft-updates)
Code: Select all
$ newfs -U -L Travel -b 4096 -o space /dev/da0.eli
Reduced frags per cylinder group from 13697 to 13696 to enlarge last cyl group
/dev/da0.eli: 1926.9MB (3946360 sectors) block size 4096, fragment size 4096
using 37 cylinder groups of 53.50MB, 13696 blks, 3424 inodes.
with soft updates
super-block backups (for fsck -b #) at:
144, 109712, 219280, 328848, 438416, 547984, 657552, 767120, 876688, 986256, 1095824, 1205392, 1314960, 1424528, 1534096, 1643664,
1753232, 1862800, 1972368, 2081936, 2191504, 2301072, 2410640, 2520208, 2629776, 2739344, 2848912, 2958480, 3068048, 3177616, 3287184,
3396752, 3506320, 3615888, 3725456, 3835024, 3944592
► Add new directive to devd.conf just to find information's about your device (USB disk).
Code: Select all
attach 0 {
device-name "umass[0-9]+";
action "sleep 2; logger vendor:$vendor,product:$product,release:$release,device-name:$device-name";
};
► Next add a second directive (you can keep the first one) to act on match, call the script /temp/secret.sh in my example (see below).
Code: Select all
attach 5 {
match "vendor" "0x0930";
match "product" "0x6544";
match "release" "0x0100";
action "sleep 1; /temp/secret.sh $device-name";
};
Code: Select all
/etc/rc.d/devd restart
Code: Select all
#!/bin/bash
key_file=/temp/ukey.key
device=da${1//[a-z]/}
origin=/temp/scripts
target=/mnt/secret
_start() {
if [ ! -f /var/run/${0##*/}.pid ];then
echo $$ > /var/run/${0##*/}.pid
else
/usr/bin/logger "Error syncing $origin on $device";
echo "A16P16A16P16A16P16CP8CP8CP8A16P16A16P16A16" > /dev/speaker
_finish;
fi
}
_finish() {
if [ -f /var/run/${0##*/}.pid ];then
rm /var/run/${0##*/}.pid;
fi
exit
}
_mount() {
/sbin/geli attach -p -k ${key_file} /dev/$device
/bin/mkdir -p ${target} &&
/sbin/mount /dev/${device}.eli ${target}/ &&
/sbin/fsck_ufs -pB /mnt/${target};
}
_umount() {
/sbin/umount /dev/${device}.eli;
/bin/rm -r /mnt/${target};
/bin/sync /dev/${device}.eli
/sbin/geli detach /dev/${device};
/bin/sync /dev/${device}
}
_sync_func() {
/usr/local/bin/rsync -a --log-file=${origin}/backup.log ${origin}/ ${target}/
/usr/bin/logger "Finish syncing $origin on $device";
}
_start; _mount; _sync_func; _umount;
while [ -e /dev/${device} ];do echo "A8P8A8P8A8" > /dev/speaker;sleep 10;done
_finish;
smartfish65 Responds:
When I execute the script as user root from the shell it works perfect!
The only problem is that the "geli attach" command isn't executed when
invoked by the devd. No error message, the following line simply does nothing:
Code: Select all
/sbin/geli attach -p -k ${key_file} /dev/${device} 2>&1 >> ${origin}/backup.log
Maybe you have a cron/path problem .
Code: Select all
/sbin/geli attach -p -k ${key_file} /dev/${device} 2>&1 >> ${origin}/backup.log
smartfish65 Responds:
I got the message:
Code: Select all
geli: Cannot open /dev/da1: No such file or directory.
When I start the script manually I have to walk to my laptop in another room.
Enough time for the usb drive so spin up. That's why the script did the job
without errors when started manually.
I removed the sleep 2 in devd.conf and put a sleep 10 in the first line of
the _mount() function. So I get an immediate beep and have a longer
wait period. Great, everything is perfect!
Danmero Responds:
Try to sleep more here, adjust as needed to give the USB time to respond:
Code: Select all
attach 5 {
match "vendor" "0x0930";
match "product" "0x6544";
match "release" "0x0100";
action "sleep 10; /temp/secret.sh $device-name";
};
Is there any advantage having the sleep in devd.conf?
I put a "sleep 10" in the _mount() function of the secret.sh script
because I like the immediate sound feedback when I plugin the drive.
Works perfect now. Many thanks!
bigalnz Responds:
I have modified Danmero's script and I think I have it working, without encryption:
Code: Select all
#!/bin/bash -x
#key_file=/temp/ukey.key
device=da${1//[a-z]/}
origin=/mnt/DATA/Test/
target=/mnt/WD
echo ${device}
echo ${target}
_mount() {
# /sbin/geli attach -p -k ${key_file} /dev/$device
/bin/mkdir -p ${target} &&
/sbin/mount /dev/${device} ${target}/ &&
/sbin/fsck_ufs -pB ${target};
umount /mnt/WD;
}
_umount() {
/sbin/umount /dev/${device};
/bin/rm -r ${target};
/bin/sync /dev/${device}
# /sbin/geli detach /dev/${device};
/bin/sync /dev/${device}
}
_sync_func() {
echo "START SYNC"
/usr/local/bin/rsync -av --progress --log-file=${origin}/backup.log ${origin}/ ${target}/
/usr/bin/logger "Finish syncing $origin on $device";
}
_mount; _sync_func; _umount;
#while [ -e /dev/da0 ];do echo "A8P8A8P8A8" > /dev/speaker;sleep 10;done
#_finish;
Danmero Responds:
Code: Select all
while [ -e /dev/da0 ];do echo "A8P8A8P8A8" > /dev/speaker;sleep 10;done
bubulein Responds:
In my version of FreeNAS attach 0 is already in use, so i have to replace attach 0 with attach 1. Just as a note if some has the same issue.
bigalnz Responds:
I got a bunch of errors to do with the superblocks. No longer on my screen or I would post.
I didn't realize that I needed need to modify the HEX values (vendor: product: release:) depending on what you get from the log.
tbo69 Responds:
Alright, I've written a script and modified danmero's. One is the backup script. The other checks the staleness of the backup and will email a reminder.
Use the devd.conf method for the backup script. Add a cron job through WebGUI to run the checker once a day.
I do have a problem though, maybe danmero, you know the answer....when I plug in my External USB 1.5TB backupdrive formated as UFS, the device name passed in is umass1. It won't mount. But if I run ./backup.sh da1p1 it works fine! Why is it telling me umass1? And if your script simply took the 1 from umass1 and added da to it to give me da1, it still won't mount! mount wants: "mount /dev/da1p1 /mnt/tempBackup".....I must be missing something.
Any help would be appreciated!
The backup script:
Code: Select all
#!/bin/bash
#----------------------------------------------------------------------------
# Backup.sh
# Version: 1.0
# Date: Jan 27 2011
# Author: Trevor Skaling
# Target: FreeNAS 0.7.2 Rev. 5543
# Credit: Mostly copied from danmero's script.
# https://sourceforge.net/apps/phpbb/freenas/posting.php?mode=reply&f=12&t=2073&sid=f8aa95efea911460e50a4bdb2ceb5b4f
# USB attach, auto-mount, rsync, detach & umount(auto-backup) thread
#
# Overview:
#
# This script:
# -gets passed in one argument, the target device to use for backup
# -Mounts the device
# -backs up a list of directories to the backup device via "rsync"
# -has optional parameters to allow deletetion of no longer existing
# files on the source directory
# -Unmounts and cleans up temporary directories
# -Stores the date of the backup into a file
#
# Backup List:
#
# This script is used to read a list of directories to backup.
# Each directory to be backed up should be on a seperate line in the
# file. An optional "DEL" flag can appear after the directory to
# signal to the script to backup that directory and delete files
# no longer within the source directory.
#
# Example list:
# /home/User/Data
# /home/User/Scratch DEL
# /mnt/Data/Photos
#
# Backups are made to the target device in the exact full path specified
# in the backup list. For example, if we are using backup device da0 mounted to
# /mnt/tempBackup, the list above would create the following directory structure:
# /mnt/tempBackup/home/User/
# Data/
# Scratch/
# /mnt/tempBackup/mnt/Data/Photos/
#
# Backup timestamp:
#
# This script will store the "day" of the backup to file. This can be
# used to check the time since last backup (see checkBackup.sh script)
# and email reminders if needed. Currently limited to "day" resolution.
#
#
#-----------------------------------------------------------------------------------
# VARIABLES
#-----------------------------------------------------------------------------------
# reads the first commandline argument as the device to backup to
# for some reason, da1p1 gets parsed to da11?
#device=da${1//[a-z]/}
device=$1
# the directory to mount the above device to and to backup to
target=/mnt/tempBackup
#location of this script
scriptpath=/root/of/directories/below
#location to log to
logpath=${scriptpath}/log
#location of the file used to store the last backup timestamp
datefile=${logpath}/lastBackupDate.log
#list of directories to backup
synclist=${scriptpath}/backuplist.txt
#-----------------------------------------------------------------------------------
# FUNCTIONS
#-----------------------------------------------------------------------------------
#perform checks and actions needed to initialize this script
_start() {
#check for proper usage, need one commandline argument
if [ ! $device ]; then
echo "Usage: backup device. Example: 'backup da0p1'"
echo
echo "No device given!"
echo
_finish;
fi
/usr/bin/logger "Starting Backup syncing on device $device!";
# let the user know the device is connected and the backup started
echo "A8P8A8P8A8" > /dev/speaker
#check if this script is already running!
if [ ! -f /var/run/${0##*/}.pid ];then
echo $$ > /var/run/${0##*/}.pid
else
/usr/bin/logger "Error syncing $origin on $device";
#play tone to PC speaker to notify user of error
echo "A16P16A16P16A16P16CP8CP8CP8A16P16A16P16A16" > /dev/speaker
_finish;
fi
}
#perform shutdown actions for this script
_finish() {
if [ -f /var/run/${0##*/}.pid ];then
rm /var/run/${0##*/}.pid;
fi
exit
}
#perform the needed steps to mount the backup disk
_mount() {
#wait for the device to wake up
sleep 5
/bin/mkdir -p ${target} &&
/sbin/mount /dev/${device} ${target} &&
/sbin/fsck_ufs -pB ${target};
}
#perform the steps needed to unmount the backup disk
_umount() {
/sbin/umount /dev/${device};
/bin/rm -r ${target};
/bin/sync /dev/${device};
}
#perform the steps needed to backup the data
_sync_func() {
#for each line in the backuplist file, read the path and the optional "DEL" flag
while read curPath del_option
do
/usr/bin/logger "Starting sync of $curPath to ${target}${curPath}..."
#does the destination directory exist already, if not, make it
if [ ! -e ${target}${curPath} ]; then
/bin/mkdir -p ${target}${curPath}
fi
# was this source directory flagged to backup with "delete" enabled
if [ "$del_option" == "DEL" ]; then
/usr/bin/logger "Syncing $curPath on $device with Delete Files active!";
/usr/local/bin/rsync -a --delete --log-file=${logpath}/backup.log ${curPath}/ ${target}${curPath}/
else
/usr/bin/logger "Syncing $curPath on $device";
/usr/local/bin/rsync -a --log-file=${logpath}/backup.log ${curPath}/ ${target}${curPath}/
fi
/usr/bin/logger "Finish syncing $curPath on $device";
done < $synclist
}
#perform the steps needed to timestamp the backup
# NOTE: If you change this logic, update the checkBackup.sh script (which reads this file)
# to match the new logic.
_store_last_backup() {
# remove the exiting timestamp file
/bin/rm -f ${datefile};
# create a new empty timestamp fiile
/usr/bin/touch ${datefile};
# store the current julian day into the file
/bin/date +%j > ${datefile};
}
#-----------------------------------------------------------------------------------
# SCRIPT MAIN LOOP
#-----------------------------------------------------------------------------------
#start the script, mount the device, backup, unmount, then timestamp
_start; _mount; _sync_func; _umount; _store_last_backup;
# script is finished, sound a beep on the PC speaker every 10 seconds until the
# device is unplugged
while [ -e /dev/${device} ];do echo "A8P8A8P8A8" > /dev/speaker;sleep 10;done
#exit the script
_finish;
Code: Select all
#!/bin/bash
#----------------------------------------------------------------------------
# checkBackup.sh
# Version: 1.0
# Date: Jan 27 2011
# Author: Trevor Skaling
# Target: FreeNAS 0.7.2 Rev. 5543
#
# Overview:
#
# A script to check the time since the last backup and email reminders
# if too much time has passed since the last backup. Use this script
# in conjunction with the backup.sh script.
#
# NOTE: This script currently is limited to "day" resolution. Cannot
# determine hours/minutes since last backup.
#
# This script:
# -checks the timestamp from a specified file and computes
# the elapsed "days" since that timestamp.
# -If the elasped time is greater than a specified amount,
# an email is generated and sent.
#
#-----------------------------------------------------------------------------------
# VARIABLES
#-----------------------------------------------------------------------------------
#the location of the date timestamp file. This must match the location used in
#the backup.sh script
datefile=/same/location/as/other/script/lastBackupDate.log
#mail recipients, seperate multiple emails via semi-colon
mailto=email@email.com
#Maximum amount of days that can pass since the last backup before emails are sent
maxBackupSpanDays=31
#stores the days since the last backup. Global variable needed.
daysSinceLastBackup=0
#stores the first commandline argument
isTestMail=$1
numArgs=$#
#-----------------------------------------------------------------------------------
# FUNCTIONS
#-----------------------------------------------------------------------------------
#perform checks and actions needed to initialize this script
_start()
{
# if we cannot read the timestamp file, there is nothing we can do
if [ ! -r ${datefile} ]; then
/usr/bin/logger "Cannot read last backup date!!!: ${datefile}";
_finish;
fi
#check for proper usage, need one commandline argument
if [ "$numArgs" -eq 0 ]; then
echo "Normal Check Commencing..."
elif [ "$isTestMail" != "-test" ]; then
echo "Usage: checkBackup Optional: -test"
echo
echo "Bad argument!"
echo
_finish;
else
echo "Test email send Commencing..."
_sendReminder;
_finish;
fi
}
#perform shutdown actions for this script
_finish() {
exit
}
#perform steps needed to determine the days since the last update and send an email
#if required
_checkDate() {
# assume the timestamp julian "day" is the number on the first line of the file. This
# file is generated via the backup.sh script
lastBackupDay=`head -n 1 ${datefile}`
# get the current julian day
curDay=`/bin/date +%j`
# strip off leading zero, otherwise the number is treated a hex and the
# math will be wrong
while [[ $curDay = 0* ]]
do
curDay=${curDay#0}
done
# calculate the difference
daysSinceLastBackup=$(($curDay-$lastBackupDay))
# check for next year. If the caclulation above results in a negative number we
# have moved to the next year (eg: last backup day was 345, current day is 2)
if [ $daysSinceLastBackup -lt 0 ]; then
# recalculate for the new year. Currently ignore leap years (no worries, backups
# only get over-stale by one day!
daysSinceLastBackup=$((($curDay+365)-$lastBackupDay))
echo "Converting for year carry over: $daysSinceLastBackup"
fi
# check if the time since is over the threshold
if [ $daysSinceLastBackup -gt $maxBackupSpanDays ]; then
# send the email
_sendReminder;
else
/usr/bin/logger "Backup check complete. No action needed."
fi
}
#builds and sends the reminder email
_sendReminder() {
/usr/bin/logger "Sending email reminder for NAS Backup!"
# build the email
MSG=$(printf "From:%s\nTo:%s\nSubject:NAS Backup\nDate:%s\n\n%s\n\nThanks!\n Your NAS" \
"me" "${mailto}" "$(date "+%a, %d %b %Y %H:%M:%S %z")" \
"It has been "${daysSinceLastBackup}" days since your last backup. Please Backup!" )
# send the email
/etc/rc.d/msmtp
echo "${MSG}" | msmtp --file=${msmtp_config:-"/var/etc/msmtp.conf"} "${mailto}"
}
#-----------------------------------------------------------------------------------
# SCRIPT MAIN LOOP
#-----------------------------------------------------------------------------------
_start; _checkDate;
_finish;
The most important part is the devd.conf , once you solve this problem you can do anything else.
If attach 0 exist , use attach 1 or next available number. The following directive will send the info to logger and in the same time to /test_file
Code: Select all
attach 1 {
device-name "umass[0-9]+";
action "sleep 2; logger vendor:$vendor,product:$product,release:$release,device-name:$device-name; echo vendor:$vendor,product:$product,release:$release,device-name:$device-name >> /test_file";
};
Code: Select all
cat /test_file
This will take the numerical value from first argument(in your case umass1 will be 1) and create the device variable, having the value of da1( your case).
since you want to mount a UFS filesystem from the first GPT partition(p1) from your device the line should be:
Code: Select all
device=da${1//[a-z]/}p1
Copied from the Old FreeNAS Forum - so it's archived.
Thanks.
Larry Kraemer