Mercurial > treepkg > treepkg

annotate treepkg/publish.py @ 429:9cfa9f64387a treepkg-status

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
only rsync specified architectures from cachedir to publishdir
author Bjoern Ricks <bricks@intevation.de>
date Wed, 28 Jul 2010 11:38:00 +0000
parents 25ef11a79d7f
children 8e0c81870e5e
rev   line source
404
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
1 #! /usr/bin/python
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
2 # Copyright (C) 2007 - 2010 by Intevation GmbH
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
3 # Authors:
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
4 # Bernhard Herzog <bh@intevation.de>
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
5 # Bjoern Ricks <bjoern.ricks@intevation.de>
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
6 #
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
7 # This program is free software under the GPL (>=v2)
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
8 # Read the file COPYING coming with the software for details.
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
9
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
10 import os.path
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
11
429
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
12 import util
404
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
13 from treepkg.run import call, capture_output
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
14 from treepkg.cmdexpand import cmdexpand
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
15
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
16 def remove_trailing_slashes(s):
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
17 return s.rstrip("/")
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
18
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
19 def expand_filename(filename):
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
20 """
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
21 Applies os.path.expanduser and os.path.expandvars to filename
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
22 """
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
23 return os.path.expandvars(os.path.expanduser(filename))
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
24
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
25 def prefix_for_remote_command(user, host):
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
26 """Returns the ssh call needed to run a command on a remote host.
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
27 If host is empty, the function assumes the command is to be run on
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
28 the local host as the same user that exectutes this function, so
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
29 that no ssh or other call is needed.
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
30 """
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
31 prefix = []
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
32 if host:
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
33 prefix.extend(["ssh", "%s@%s" % (user, host)])
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
34 return prefix
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
35
429
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
36 def copy_arch_to_publishdir(variables, dist, section, arch=None, quiet=False,
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
37 create=True):
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
38 if not arch:
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
39 cachedir = variables["cachedir"]
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
40 else:
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
41 cachedir = os.path.join(variables["cachedir"], arch)
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
42
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
43 # if cachedir does not exist rsync will fail therefore
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
44 # it must be created or skipped. if it is created remote
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
45 # content will be deleted
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
46 if not os.path.exists(cachedir):
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
47 if create:
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
48 util.ensure_directory(cachedir)
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
49 else:
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
50 return
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
51
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
52 destdir = os.path.join(variables["publish_dir"], dist, section)
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
53 remote_destdir = destdir
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
54 if variables["publish_host"]:
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
55 remote_destdir = (("%(publish_user)s@%(publish_host)s:" % variables)
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
56 + remote_destdir)
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
57 runremote = prefix_for_remote_command(variables["publish_user"],
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
58 variables["publish_host"])
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
59
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
60 call(cmdexpand("@runremote mkdir --parents $destdir",
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
61 runremote=runremote, destdir=destdir, **variables))
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
62 rsync_flags = []
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
63 if variables["publish_remove_old_packages"]:
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
64 rsync_flags.append("--delete")
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
65 if quiet:
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
66 rsync_flags.append("--quiet")
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
67 cmd = cmdexpand("rsync @rsync_flags -r --perms --times --omit-dir-times"
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
68 " $cachedir $remote_destdir/",
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
69 rsync_flags=rsync_flags, remote_destdir=remote_destdir,
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
70 cachedir=cachedir)
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
71 #print "rsync cmd: %s" % cmd
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
72 call(cmd)
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
73
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
74
9cfa9f64387a only rsync specified architectures from cachedir to publishdir
Bjoern Ricks <bricks@intevation.de>
parents: 417
diff changeset
75
417
25ef11a79d7f incremental copying only changed debian packages
Bjoern Ricks <bricks@intevation.de>
parents: 404
diff changeset
76 def copy_to_publishdir(variables, dist, section, arch=None, quiet=False):
25ef11a79d7f incremental copying only changed debian packages
Bjoern Ricks <bricks@intevation.de>
parents: 404
diff changeset
77 if not arch:
25ef11a79d7f incremental copying only changed debian packages
Bjoern Ricks <bricks@intevation.de>
parents: 404
diff changeset
78 destdir = os.path.join(variables["publish_dir"], dist, section)
25ef11a79d7f incremental copying only changed debian packages
Bjoern Ricks <bricks@intevation.de>
parents: 404
diff changeset
79 else:
25ef11a79d7f incremental copying only changed debian packages
Bjoern Ricks <bricks@intevation.de>
parents: 404
diff changeset
80 destdir = os.path.join(variables["publish_dir"], dist, section, arch)
404
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
81 remote_destdir = destdir
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
82 if variables["publish_host"]:
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
83 remote_destdir = (("%(publish_user)s@%(publish_host)s:" % variables)
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
84 + remote_destdir)
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
85 runremote = prefix_for_remote_command(variables["publish_user"],
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
86 variables["publish_host"])
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
87
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
88 call(cmdexpand("@runremote mkdir --parents $destdir",
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
89 runremote=runremote, destdir=destdir, **variables))
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
90 rsync_flags = []
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
91 if variables["publish_remove_old_packages"]:
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
92 rsync_flags.append("--delete")
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
93 if quiet:
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
94 rsync_flags.append("--quiet")
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
95 call(cmdexpand("rsync @rsync_flags -r --perms --times --omit-dir-times"
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
96 " $cachedir/ $remote_destdir/",
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
97 rsync_flags=rsync_flags, remote_destdir=remote_destdir,
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
98 **variables))
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
99
a798edae0844 moved common publish functions to a seperate module
Bjoern Ricks <bricks@intevation.de>
parents:
diff changeset
100
This site is hosted by Intevation GmbH (Datenschutzerklärung und Impressum | Privacy Policy and Imprint)