Mercurial > treepkg

annotate bin/publishstaticweb.py @ 441:eadcb1bb54e2

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Merged revisions 388-399 via svnmerge from svn+ssh://svn.wald.intevation.org/treepkg/branches/treepkg-status ........ r388 | bricks | 2010-07-28 10:38:50 +0200 (Mi, 28 Jul 2010) | 3 lines fix if statement upload hook should be called if it's not empty ........ r389 | bricks | 2010-07-28 13:38:00 +0200 (Mi, 28 Jul 2010) | 2 lines only rsync specified architectures from cachedir to publishdir ........ r390 | bricks | 2010-07-29 14:07:57 +0200 (Do, 29 Jul 2010) | 2 lines introduced after_copy_hook and made rsync upload to publishdir optional ........ r391 | bricks | 2010-07-29 15:43:39 +0200 (Do, 29 Jul 2010) | 3 lines publish package must be default to get original behaviour be more verbose at publishing ........ r392 | bricks | 2010-07-29 16:50:53 +0200 (Do, 29 Jul 2010) | 2 lines fix handling of showing only successfull builds together with num packages ........ r393 | bricks | 2010-07-29 16:52:42 +0200 (Do, 29 Jul 2010) | 2 lines fixed typo ........ r394 | bricks | 2010-07-29 16:59:10 +0200 (Do, 29 Jul 2010) | 2 lines fixed another typo ........ r395 | bricks | 2010-07-29 18:05:54 +0200 (Do, 29 Jul 2010) | 3 lines fixed another typo treepkg is really slow with many revisions ........ r396 | bricks | 2010-08-02 12:41:55 +0200 (Mo, 02 Aug 2010) | 2 lines fixed typos ........ r397 | bricks | 2010-08-02 17:36:12 +0200 (Mo, 02 Aug 2010) | 3 lines be more error prone in listing different files not checking if a dir exists caused several errors if a build wasn't successful ........ r398 | bricks | 2010-08-05 18:21:47 +0200 (Do, 05 Aug 2010) | 2 lines review changes ........ r399 | bricks | 2010-08-06 13:06:08 +0200 (Fr, 06 Aug 2010) | 4 lines cleanup modules fix test_info testcases added testcases for remove_trailingslashes and expand_filename ........
author Bjoern Ricks <bricks@intevation.de>
date Fri, 06 Aug 2010 13:41:54 +0000
parents ef87d30468b6
children bccd69dedbe5
rev   line source
287
1fcdffbeb9de Make the #! line in the commands more portable. Use /usr/bin/python
Bernhard Herzog <bh@intevation.de>
parents: 158
diff changeset
1 #! /usr/bin/python
1fcdffbeb9de Make the #! line in the commands more portable. Use /usr/bin/python
Bernhard Herzog <bh@intevation.de>
parents: 158
diff changeset
2 # Copyright (C) 2007, 2008, 2009 by Intevation GmbH
78
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
3 # Authors:
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
4 # Bernhard Herzog <bh@intevation.de>
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
5 #
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
6 # This program is free software under the GPL (>=v2)
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
7 # Read the file COPYING coming with the software for details.
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
8
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
9 """Publishes a static web-site with a status report"""
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
10
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
11 import sys
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
12 import os
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
13 from optparse import OptionParser
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
14 from ConfigParser import SafeConfigParser
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
15
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
16 import treepkgcmd
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
17 from treepkg.readconfig import read_config_section
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
18 from treepkg.run import call
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
19 from treepkg.cmdexpand import cmdexpand
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
20 from treepkg.util import ensure_directory
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
21
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
22 def remove_trailing_slashes(s):
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
23 return s.rstrip("/")
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
24
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
25 def expand_filename(filename):
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
26 """
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
27 Applies os.path.expanduser and os.path.expandvars to filename
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
28 """
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
29 return os.path.expandvars(os.path.expanduser(filename))
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
30
158
8ec9ed76d67b Make the template used by bin/publishstaticweb.py configurable.
Bernhard Herzog <bh@intevation.de>
parents: 78
diff changeset
31 staticweb_desc = ["build_user", "build_host", "build_create", "build_template",
372
ef87d30468b6 Added the option to expose additional log files from the log directory.
Andre Heinecke <aheinecke@intevation.de>
parents: 338
diff changeset
32 "build_logs",
78
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
33 ("build_dir", remove_trailing_slashes),
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
34 "publish_user", "publish_host",
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
35 ("publish_dir", remove_trailing_slashes),
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
36 ("cachedir",
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
37 lambda s: expand_filename(remove_trailing_slashes(s)))]
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
38
372
ef87d30468b6 Added the option to expose additional log files from the log directory.
Andre Heinecke <aheinecke@intevation.de>
parents: 338
diff changeset
39 #Default values for the configuration options can be set here
ef87d30468b6 Added the option to expose additional log files from the log directory.
Andre Heinecke <aheinecke@intevation.de>
parents: 338
diff changeset
40 staticweb_defaults = [("build_logs", "build_log.txt.gz")]
ef87d30468b6 Added the option to expose additional log files from the log directory.
Andre Heinecke <aheinecke@intevation.de>
parents: 338
diff changeset
41
78
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
42 def read_config(filename):
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
43 parser = SafeConfigParser()
372
ef87d30468b6 Added the option to expose additional log files from the log directory.
Andre Heinecke <aheinecke@intevation.de>
parents: 338
diff changeset
44 parser.add_section("staticweb")
ef87d30468b6 Added the option to expose additional log files from the log directory.
Andre Heinecke <aheinecke@intevation.de>
parents: 338
diff changeset
45 for value in staticweb_defaults:
ef87d30468b6 Added the option to expose additional log files from the log directory.
Andre Heinecke <aheinecke@intevation.de>
parents: 338
diff changeset
46 parser.set("staticweb", value[0], value[1])
78
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
47 parser.read([filename])
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
48 return read_config_section(parser, "staticweb", staticweb_desc)
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
49
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
50 def parse_commandline():
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
51 parser = OptionParser()
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
52 parser.set_defaults(config_file=os.path.join(treepkgcmd.topdir,
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
53 "staticweb.cfg"))
338
28df50b267f6 Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents: 319
diff changeset
54 parser.set_defaults(update_cache_only=False)
78
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
55 parser.add_option("--config-file",
319
dba1b6b825ec Improve help string for --config-file option of publishstaticweb.
Bernhard Herzog <bh@intevation.de>
parents: 287
diff changeset
56 help=("The configuration file. Default is staticweb.cfg"))
338
28df50b267f6 Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents: 319
diff changeset
57 parser.add_option("--update-cache-only", action="store_true",
28df50b267f6 Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents: 319
diff changeset
58 help=("Stop after updating the cache."
28df50b267f6 Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents: 319
diff changeset
59 " This omits the last step of the publishing"
28df50b267f6 Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents: 319
diff changeset
60 " process"))
78
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
61 return parser.parse_args()
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
62
338
28df50b267f6 Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents: 319
diff changeset
63 def publish_static_site(config_filename, update_cache_only=False):
78
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
64 config = read_config(config_filename)
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
65
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
66 # create web-page on build host
158
8ec9ed76d67b Make the template used by bin/publishstaticweb.py configurable.
Bernhard Herzog <bh@intevation.de>
parents: 78
diff changeset
67 call(cmdexpand("ssh $build_user$@$build_host $build_create"
372
ef87d30468b6 Added the option to expose additional log files from the log directory.
Andre Heinecke <aheinecke@intevation.de>
parents: 338
diff changeset
68 " --show-logs=$build_logs"
158
8ec9ed76d67b Make the template used by bin/publishstaticweb.py configurable.
Bernhard Herzog <bh@intevation.de>
parents: 78
diff changeset
69 " --status-template=$build_template $build_dir",
78
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
70 **config))
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
71
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
72 # rsync the new web-pages to the local cache
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
73 ensure_directory(config["cachedir"])
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
74 call(cmdexpand("rsync -rL --delete $build_user$@$build_host:$build_dir/"
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
75 " $cachedir",
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
76 **config))
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
77
338
28df50b267f6 Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents: 319
diff changeset
78 if not update_cache_only:
28df50b267f6 Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents: 319
diff changeset
79 # rsync the web pages from the local cache to the publishing host
28df50b267f6 Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents: 319
diff changeset
80 call(cmdexpand("rsync -rL --delete $cachedir/"
28df50b267f6 Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents: 319
diff changeset
81 " $publish_user$@$publish_host:$publish_dir",
28df50b267f6 Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents: 319
diff changeset
82 **config))
78
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
83
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
84
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
85 def main():
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
86 options, args = parse_commandline()
338
28df50b267f6 Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents: 319
diff changeset
87 publish_static_site(options.config_file, options.update_cache_only)
78
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
88
9a602d8eaa60 initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
89 main()
This site is hosted by Intevation GmbH (Datenschutzerklärung und Impressum | Privacy Policy and Imprint)