extern/zfs-auto-snapshot
1
0
Fork 1
mirror of https://github.com/zfsonlinux/zfs-auto-snapshot.git synced 2024-11-29 03:23:16 +01:00
Code Issues Packages Projects Releases Wiki Activity

Use only name property for zfs list

Browse Source 
zfs list is very slow if it needs to retrieve other properties than name. See zfsonlinux/zfs#450 for reference.

This change uses only the name of the snapshot and uses awk to extract the snapshot date from the name and sort to sort it:

1. get all snapshot names, sorted by name
2. use grep to filter out any snapshot, that isn't correctly prefixed (maybe a check for the correct date format could also be added here)
3. use awk to extract the date and put the date before the snapshot
4. use sort to reverse sort this, first by date, then by snapshot name
5. use awk to remove the date, so that just the snapshot name is left

This significally speeds it up on my system (/root/zfs-auto-snapshot is the changed one, running with -n to get only the times for snapshot retrieval):

root@tatooine:/root# time /root/zfs-auto-snapshot -d -v // --label=frequent --keep=4 -n
Debug: Including rpool for regular snapshot.
Debug: Including rpool/DATA for regular snapshot.
Debug: Including rpool/DATA/ldap for recursive snapshot.
Debug: Including rpool/DATA/postgresql for recursive snapshot.
Debug: Including rpool/DATA/vm for regular snapshot.
Debug: Including rpool/DATA/vm/alderaan.example.net for recursive snapshot.
Debug: Including rpool/DATA/vm/bespin.example.net for recursive snapshot.
Debug: Including rpool/DATA/vm/coruscant.example.net for recursive snapshot.
Debug: Including rpool/DATA/vm/dagobah.example.net for recursive snapshot.
Debug: Including rpool/DATA/vm/dantooine.example.net for recursive snapshot.
Debug: Including rpool/DATA/vm/dev.example.net for recursive snapshot.
Debug: Including rpool/DATA/vm/monitor.example.net for recursive snapshot.
Debug: Including rpool/DATA/vm/office.example.net for recursive snapshot.
Debug: Including rpool/DATA/vm/test.example.net for recursive snapshot.
Debug: Including rpool/ROOT for recursive snapshot.
Debug: Excluding rpool/ROOT/ubuntu-1 because rpool/ROOT includes it recursively.
Doing regular snapshots of rpool rpool/DATA rpool/DATA/vm
Doing recursive snapshots of rpool/DATA/ldap rpool/DATA/postgresql rpool/DATA/vm/alderaan.example.net rpool/DATA/vm/bespin.example.net rpool/DATA/vm/coruscant.example.net rpool/DATA/vm/dagobah.example.net rpool/DATA/vm/dantooine.example.net rpool/DATA/vm/dev.example.net rpool/DATA/vm/monitor.example.net rpool/DATA/vm/office.example.net rpool/DATA/vm/test.example.net rpool/ROOT
Doing a dry run. Not running these commands...
zfs snapshot -o com.sun:auto-snapshot-desc='-'  'rpool@zfs-auto-snap_frequent-2013-04-16-2344'
zfs destroy  'rpool@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-'  'rpool/DATA@zfs-auto-snap_frequent-2013-04-16-2344'
zfs destroy  'rpool/DATA@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-'  'rpool/DATA/vm@zfs-auto-snap_frequent-2013-04-16-2344'
zfs destroy  'rpool/DATA/vm@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/ldap@zfs-auto-snap_frequent-2013-04-16-2344'
zfs destroy -r 'rpool/DATA/ldap@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/postgresql@zfs-auto-snap_frequent-2013-04-16-2344'
zfs destroy -r 'rpool/DATA/postgresql@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/vm/alderaan.example.net@zfs-auto-snap_frequent-2013-04-16-2344'
zfs destroy -r 'rpool/DATA/vm/alderaan.example.net@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/vm/bespin.example.net@zfs-auto-snap_frequent-2013-04-16-2344'
zfs destroy -r 'rpool/DATA/vm/bespin.example.net@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/vm/coruscant.example.net@zfs-auto-snap_frequent-2013-04-16-2344'
zfs destroy -r 'rpool/DATA/vm/coruscant.example.net@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/vm/dagobah.example.net@zfs-auto-snap_frequent-2013-04-16-2344'
zfs destroy -r 'rpool/DATA/vm/dagobah.example.net@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/vm/dantooine.example.net@zfs-auto-snap_frequent-2013-04-16-2344'
zfs destroy -r 'rpool/DATA/vm/dantooine.example.net@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/vm/dev.example.net@zfs-auto-snap_frequent-2013-04-16-2344'
zfs destroy -r 'rpool/DATA/vm/dev.example.net@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/vm/monitor.example.net@zfs-auto-snap_frequent-2013-04-16-2344'
zfs destroy -r 'rpool/DATA/vm/monitor.example.net@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/vm/office.example.net@zfs-auto-snap_frequent-2013-04-16-2344'
zfs destroy -r 'rpool/DATA/vm/office.example.net@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/vm/test.example.net@zfs-auto-snap_frequent-2013-04-16-2344'
zfs destroy -r 'rpool/DATA/vm/test.example.net@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/ROOT@zfs-auto-snap_frequent-2013-04-16-2344'
zfs destroy -r 'rpool/ROOT@zfs-auto-snap_frequent-2013-04-16-2315'
@zfs-auto-snap_frequent-2013-04-16-2344, 15 created, 15 destroyed, 0 warnings.

real	0m6.936s
user	0m0.076s
sys	0m0.184s
root@tatooine:/root# time /sbin/zfs-auto-snapshot -d -v // --label=frequent --keep=4 -n
Debug: Including rpool for regular snapshot.
Debug: Including rpool/DATA for regular snapshot.
Debug: Including rpool/DATA/ldap for recursive snapshot.
Debug: Including rpool/DATA/postgresql for recursive snapshot.
Debug: Including rpool/DATA/vm for regular snapshot.
Debug: Including rpool/DATA/vm/alderaan.example.net for recursive snapshot.
Debug: Including rpool/DATA/vm/bespin.example.net for recursive snapshot.
Debug: Including rpool/DATA/vm/coruscant.example.net for recursive snapshot.
Debug: Including rpool/DATA/vm/dagobah.example.net for recursive snapshot.
Debug: Including rpool/DATA/vm/dantooine.example.net for recursive snapshot.
Debug: Including rpool/DATA/vm/dev.example.net for recursive snapshot.
Debug: Including rpool/DATA/vm/monitor.example.net for recursive snapshot.
Debug: Including rpool/DATA/vm/office.example.net for recursive snapshot.
Debug: Including rpool/DATA/vm/test.example.net for recursive snapshot.
Debug: Including rpool/ROOT for recursive snapshot.
Debug: Excluding rpool/ROOT/ubuntu-1 because rpool/ROOT includes it recursively.
Doing regular snapshots of rpool rpool/DATA rpool/DATA/vm
Doing recursive snapshots of rpool/DATA/ldap rpool/DATA/postgresql rpool/DATA/vm/alderaan.example.net rpool/DATA/vm/bespin.example.net rpool/DATA/vm/coruscant.example.net rpool/DATA/vm/dagobah.example.net rpool/DATA/vm/dantooine.example.net rpool/DATA/vm/dev.example.net rpool/DATA/vm/monitor.example.net rpool/DATA/vm/office.example.net rpool/DATA/vm/test.example.net rpool/ROOT
Doing a dry run. Not running these commands...
zfs snapshot -o com.sun:auto-snapshot-desc='-'  'rpool@zfs-auto-snap_frequent-2013-04-16-2348'
zfs destroy  'rpool@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-'  'rpool/DATA@zfs-auto-snap_frequent-2013-04-16-2348'
zfs destroy  'rpool/DATA@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-'  'rpool/DATA/vm@zfs-auto-snap_frequent-2013-04-16-2348'
zfs destroy  'rpool/DATA/vm@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/ldap@zfs-auto-snap_frequent-2013-04-16-2348'
zfs destroy -r 'rpool/DATA/ldap@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/postgresql@zfs-auto-snap_frequent-2013-04-16-2348'
zfs destroy -r 'rpool/DATA/postgresql@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/vm/alderaan.example.net@zfs-auto-snap_frequent-2013-04-16-2348'
zfs destroy -r 'rpool/DATA/vm/alderaan.example.net@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/vm/bespin.example.net@zfs-auto-snap_frequent-2013-04-16-2348'
zfs destroy -r 'rpool/DATA/vm/bespin.example.net@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/vm/coruscant.example.net@zfs-auto-snap_frequent-2013-04-16-2348'
zfs destroy -r 'rpool/DATA/vm/coruscant.example.net@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/vm/dagobah.example.net@zfs-auto-snap_frequent-2013-04-16-2348'
zfs destroy -r 'rpool/DATA/vm/dagobah.example.net@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/vm/dantooine.example.net@zfs-auto-snap_frequent-2013-04-16-2348'
zfs destroy -r 'rpool/DATA/vm/dantooine.example.net@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/vm/dev.example.net@zfs-auto-snap_frequent-2013-04-16-2348'
zfs destroy -r 'rpool/DATA/vm/dev.example.net@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/vm/monitor.example.net@zfs-auto-snap_frequent-2013-04-16-2348'
zfs destroy -r 'rpool/DATA/vm/monitor.example.net@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/vm/office.example.net@zfs-auto-snap_frequent-2013-04-16-2348'
zfs destroy -r 'rpool/DATA/vm/office.example.net@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/DATA/vm/test.example.net@zfs-auto-snap_frequent-2013-04-16-2348'
zfs destroy -r 'rpool/DATA/vm/test.example.net@zfs-auto-snap_frequent-2013-04-16-2315'
zfs snapshot -o com.sun:auto-snapshot-desc='-' -r 'rpool/ROOT@zfs-auto-snap_frequent-2013-04-16-2348'
zfs destroy -r 'rpool/ROOT@zfs-auto-snap_frequent-2013-04-16-2315'
@zfs-auto-snap_frequent-2013-04-16-2348, 15 created, 15 destroyed, 0 warnings.

real	3m30.995s
user	0m0.152s
sys	0m0.792s
root@tatooine:/root# 

I'm not an awk god, so I tried a bit until I got a working version. There might be better ways. I also don't know if this catches every adge case, but it is a start for improvement. :)
This commit is contained in:
Markus Heberling 2013-04-17 01:07:00 +03:00
parent cc9f1802ed
commit b6fba51643
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
src/zfs-auto-snapshot.sh
View File

@ -339,7 +339,7 @@ ZFS_LIST=$(env LC_ALL=C zfs list -H -t filesystem,volume -s name \
-o name,com.sun:auto-snapshot,com.sun:auto-snapshot:"$opt_label") \
|| { print_log error "zfs list $?: $ZFS_LIST"; exit 136; }
SNAPSHOTS_OLD=$(env LC_ALL=C zfs list -H -t snapshot -S creation -o name) \
SNAPSHOTS_OLD=$(env LC_ALL=C zfs list -H -t snapshot -o name -s name|grep $opt_prefix |awk '{ print substr( $0, length($0) - 14, length($0) ) " " $0}' |sort -r -k1,1 -k2,2|awk '{ print substr( $0, 17, length($0) )}') \
|| { print_log error "zfs list $?: $SNAPSHOTS_OLD"; exit 137; }