Page 1 of 1

Daily, weekly, month snapshots retention script for GUI

Posted: 22 Oct 2016 17:34
by erik
I wanted to use the regular snapshot setting from gui and still have hourly, daily, weekly, monthly retention scheme.
So I created a small script that will use whatever snapshot are available containing the word 'auto' , look a the creation time and purges these according to a spacing defined in the script.
The script does not care if there are holes in the snapshots, e.g. its ok if your server is sleeping from time to time and missing snapshots.

You can choose snapshots at the frequency you like (every 5 minutes, daily, hourly, weekly, whatever)
Then run below script on the zfs filesystem to remove old snapshots in the pattern indicated at the top of the script.
The only restriction is that the retentions should overlap. E.g. >7 daily, >4 weekly, >3 quarterly, etc

Code: Select all

#!/bin/sh
#############################################################################
# Purge old snapshots
# Usage: ./zpurge.sh filesystem
# Author: Erik Kaashoek
#############################################################################

secondsinminute=60
minute=1
hour=$(($minute*60))
halfhour=$(($minute*30))
day=$(($hour*24))
halfday=$(($hour*12))
week=$(($day*7))
halfweek=$(($day*3))
month=$(($week*4))
halfmonth=$(($week*2))
quarter=$(($month*3))
halfquarter=$(($month*1))
year=$(($quarter*4))
halfyear=$(($quarter*2))

#define snapshot spacing grid
maxquarters=$((16*$quarter))
quartergap=$(($quarter-$halfmonth))
maxmonths=$((5*$month)) 
monthgap=$(($month - $halfweek))
maxweeks=$((6*$week))
weekgap=$(($week-$halfday))
maxdays=$((9*$day))
daygap=$(($day-$halfhour))
maxhours=$((26*$hour))
hourgap=$(($hour - $minute))
maxminutes=$((70*$minute))

TAG="auto"

usage() {
    echo "Usage: $(basename $0) [ -n ] [ -v ] [ -t tag ] pool"
    echo "  -n    debug (dry-run) mode"
    echo "  -v    verbose mode"
    echo "  -t    specify part of snapshot tag, default=auto"
    exit 1
}

DEBUG=""
VERBOSE=""
if [ "$1" == "-v" ] ; then
	VERBOSE=1 ; shift
fi
if [ "$1" == "-n" ] ; then
	DEBUG=1 ; shift
fi
if [ "$1" == "-v" ] ; then
	VERBOSE=1 ; shift
fi
if [ "$1" == "-t" ] ; then
	TAG="$2"
	shift ; shift ;
fi
if [ $# -ne 1 ]; then 
	usage
fi

purgeSnapshot()
{
#	echo "Purge $1 because $2"
    if [ $DEBUG ]; then
		echo "would run: zfs destroy $1 # because $2"
    else
		[ "$VERBOSE" ] && echo  "running: zfs destroy $1 # because $2"
		zfs destroy $1
	fi
}

readonly fs=$1

[ "$VERBOSE" ] && echo "Purge snapshots from $fs with pattern $quarter, $month, $week, $day, $hour and tag=$TAG"

if zfs list -H -o name| grep "$fs">/dev/null; then
	[ "$VERBOSE" ] && echo "Filesystem \"$fs\" does exist"
else
	echo "Filesystem \"$fs\" does not exist"
	return 0
fi

#get current time once to ensure equal spacing
now=`date +"%s"` 
#to ensure oldest snapshot only gets deleted when it is older then maxquarters
prevage=$now

#Loop through all snapshots, oldest first, and calculate age and gap with next older existing snapshot
zfs list -Hr -t snapshot -o name,creation -s creation  $fs | grep $TAG | while read snap creation; do
	age=$((($now -  `date -j -f "%a %b %d %H:%M %Y" "$creation" +"%s"` ) / $secondsinminute ))
	gap=$(($prevage - $age))
		
#	echo "$snap ( $prevage, $age, $gap)"
#	Purge if older then maxquarters
	if [ $age -gt $maxquarters ]; then	
		purgeSnapshot "$snap" "old"
# Or purge if older then maxmonths and too frequent
	elif [ $age -gt $maxmonths -a  $gap -lt $quartergap ]; then
		purgeSnapshot "$snap" "quarter"
# Or purge if older then maxweeks and too frequent
	elif [ $age -gt $maxweeks -a  $gap -lt $monthgap ]; then 
		purgeSnapshot "$snap" "month"
# Or purge if older then maxdays and too frequent
	elif [ $age -gt $maxdays -a $gap -lt $weekgap ]; then
		purgeSnapshot "$snap" "week"
# Or purge if older then maxhours and too frequent
	elif [ $age -gt $maxhours -a $gap -lt $daygap ]; then
		purgeSnapshot "$snap" "day"
# Or purge if older then maxminutes and too frequent
	elif [ $age -gt $maxminutes -a $gap -lt $hourgap ]; then
		purgeSnapshot "$snap" "hour" 
	else
		[ "$VERBOSE" ] &&  echo "Keep $snap ( $prevage, $age, $gap)"
		prevage=$age
		prevsnap=$snap
	fi
done

exit 0
When testing always start with -v -n so no harm will be done and you can inspect if you like what it will do.