Mercurial > treepkg

annotate contrib/bin/delete-old-debs.py @ 535:fb7a900a649f

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
contrib: Added script to remove old deb files from directories.
author Sascha Teichmann <teichmann@intevation.de>
date Sat, 08 Jan 2011 12:57:07 +0000
parents
children 8a61185a3357
rev   line source
535
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
1 #!/usr/bin/env python
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
2 # -*- coding: UTF-8 -*-
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
3 #
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
4 # Copyright (C) 2011 by Intevation GmbH
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
5 # Authors:
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
6 # Sascha L. Teichmann <sascha.teichmann@intevation.de>
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
7 #
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
8 # This program is free software under the GPL (>=v2)
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
9 # Read the file COPYING coming with the software for details.
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
10
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
11 import sys
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
12 import os
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
13 import re
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
14
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
15 import subprocess
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
16 import logging
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
17
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
18 from heapq import nsmallest
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
19
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
20 from optparse import OptionParser
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
21
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
22 log = logging.getLogger(__name__)
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
23 log.setLevel(logging.WARNING)
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
24 log.addHandler(logging.StreamHandler(sys.stderr))
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
25
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
26 DEFAULT_KEEP = 3
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
27
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
28 FIELD = re.compile("([a-zA-Z]+):\s*(.+)")
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
29
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
30 # map rich comparison to 'dpkg --compare-versions'
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
31 # map == to !=, < to >= and so on to reverse order in heap.
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
32 RICH_CMP = dict([
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
33 ("__%s__" % a, lambda se, ot:
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
34 subprocess.call([
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
35 "dpkg", "--compare-versions",
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
36 se.version, b, ot.version]) == 0)
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
37 for a, b in (("eq", "ne"), ("ne", "eq"),
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
38 ("lt", "ge"), ("gt", "le"),
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
39 ("le", "gt"), ("ge", "lt"))])
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
40
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
41
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
42 class DebCmp(object):
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
43 """Helper class to make deb files comparable
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
44 by there versions.
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
45 """
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
46
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
47 def __init__(self, version, path):
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
48 self.version = version
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
49 self.path = path
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
50
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
51 self.__dict__.update(RICH_CMP)
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
52
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
53
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
54 def deb_info(deb, fields=["Package", "Version"]):
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
55 """Extract some meta info from a deb file."""
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
56 po = subprocess.Popen(
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
57 ["dpkg-deb", "-f", deb] + fields,
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
58 stdout=subprocess.PIPE)
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
59 out = po.communicate()[0]
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
60 return dict([m.groups()
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
61 for m in map(FIELD.match, out.splitlines()) if m])
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
62
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
63
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
64 def oldest_debs(deb_dir, keep=DEFAULT_KEEP):
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
65 """Given directory containing deb files this function
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
66 returns the files that are older than the youngest
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
67 keep-th per package.
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
68 """
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
69
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
70 log.info("scanning dir '%s'" % deb_dir)
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
71
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
72 packages = {}
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
73
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
74 num = 1
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
75 for f in os.listdir(deb_dir):
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
76 if not f.endswith(".deb"): continue
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
77 deb = os.path.join(deb_dir, f)
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
78 if not os.path.isfile(deb): continue
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
79 info = deb_info(deb)
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
80 packages.setdefault(info['Package'], []).append(
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
81 DebCmp(info['Version'], deb))
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
82 if (num % 10) == 0:
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
83 log.info("%d debs found" % (num-1))
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
84 num += 1
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
85
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
86 if log.isEnabledFor(logging.INFO):
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
87 log.info("%d debs found" % (num-1))
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
88 log.info("number packages: %s" % len(packages))
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
89
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
90 for package, debs in packages.iteritems():
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
91 if len(debs) > keep:
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
92 # full sorting is not required
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
93 stay = frozenset([d.path for d in nsmallest(keep, debs)])
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
94
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
95 for deb in debs:
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
96 if deb.path not in stay:
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
97 yield deb.path
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
98
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
99
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
100 def main():
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
101 usage = "usage: %prog [options] dir ..."
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
102 parser = OptionParser(usage=usage)
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
103 parser.add_option(
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
104 "-v", "--verbose", action="store_true",
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
105 dest="verbose",
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
106 help="verbose output")
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
107 parser.add_option(
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
108 "-d", "--dry-run", action="store_true",
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
109 dest="dry_run",
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
110 help="don't remove the old deb files")
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
111 parser.add_option(
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
112 "-k", "--keep", action="store",
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
113 dest="keep", type="int", default=DEFAULT_KEEP,
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
114 help="number of files to keep. Default: %d" % DEFAULT_KEEP)
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
115
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
116 options, args = parser.parse_args()
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
117
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
118 remove = options.dry_run and (lambda x: None) or os.remove
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
119 keep = max(1, options.keep)
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
120 if options.verbose: log.setLevel(logging.INFO)
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
121
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
122 for deb_dir in args:
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
123
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
124 if not os.path.isdir(deb_dir):
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
125 log.warn("'%s' is not a directory" % deb_dir)
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
126 continue
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
127
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
128 for deb in oldest_debs(deb_dir, keep):
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
129 log.debug("remove '%s'" % deb)
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
130 remove(deb)
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
131 changes = deb.path[:-3] + "changes"
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
132 if os.path.isfile(changes):
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
133 log.debug("remove '%s'" % changes)
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
134 remove(changes)
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
135
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
136
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
137 if __name__ == "__main__":
fb7a900a649f contrib: Added script to remove old deb files from directories.
Sascha Teichmann <teichmann@intevation.de>
parents:
diff changeset
138 main()
This site is hosted by Intevation GmbH (Datenschutzerklärung und Impressum | Privacy Policy and Imprint)