Mercurial > treepkg
annotate bin/publishstaticweb.py @ 440:383d23c6bfde
Merged revisions 346-386 via svnmerge from
svn+ssh://svn.wald.intevation.org/treepkg/branches/treepkg-status
........
r346 | bricks | 2010-07-07 17:40:48 +0200 (Mi, 07 Jul 2010) | 2 lines
added classed for treepkg status xml generation
........
r347 | bricks | 2010-07-08 12:07:39 +0200 (Do, 08 Jul 2010) | 10 lines
renamed new status dir to info because of a naming conflict with status.py
let the user specify a treepkg name in the config
the name is propagated to PackagerGroup
[treepkg]
name: <treepkgname>
becomes:
pg = PackagerGroup(...)
pg.name
........
r348 | bricks | 2010-07-08 15:25:07 +0200 (Do, 08 Jul 2010) | 3 lines
moved test file to test main dir
fixed missing : after def statement
........
r349 | bricks | 2010-07-08 17:59:52 +0200 (Do, 08 Jul 2010) | 2 lines
added treepkg_dir andd tracks_dir attributes to PackageGroup
........
r350 | bricks | 2010-07-08 18:00:32 +0200 (Do, 08 Jul 2010) | 2 lines
changed <millpath> to <trackspath>
........
r351 | bricks | 2010-07-08 18:08:39 +0200 (Do, 08 Jul 2010) | 2 lines
check if config file exists to get usefull error output
........
r352 | bricks | 2010-07-08 18:11:13 +0200 (Do, 08 Jul 2010) | 2 lines
write first info about tracks
........
r353 | bricks | 2010-07-09 12:20:49 +0200 (Fr, 09 Jul 2010) | 2 lines
get all log files
........
r354 | bricks | 2010-07-09 14:42:42 +0200 (Fr, 09 Jul 2010) | 2 lines
added md5sum function
........
r355 | bricks | 2010-07-09 15:15:15 +0200 (Fr, 09 Jul 2010) | 4 lines
implemented nearly all info
only arch and os info are missing
........
r356 | bricks | 2010-07-09 16:19:17 +0200 (Fr, 09 Jul 2010) | 2 lines
Bugfix: source was not defined
........
r357 | bricks | 2010-07-09 16:44:52 +0200 (Fr, 09 Jul 2010) | 3 lines
fixed some typos
added missing add_checksum methond
........
r358 | bricks | 2010-07-09 17:06:15 +0200 (Fr, 09 Jul 2010) | 2 lines
fixed status line for revision
........
r359 | bricks | 2010-07-12 14:10:34 +0200 (Mo, 12 Jul 2010) | 2 lines
let the user specify num revisions that should be shown in the info
........
r360 | bricks | 2010-07-12 14:21:39 +0200 (Mo, 12 Jul 2010) | 2 lines
missed in last commit
........
r361 | bricks | 2010-07-12 17:06:22 +0200 (Mo, 12 Jul 2010) | 2 lines
add os config statement
........
r362 | bricks | 2010-07-12 17:47:42 +0200 (Mo, 12 Jul 2010) | 3 lines
moved packages and logs to revision tag
implemented os and arch info
........
r363 | bricks | 2010-07-13 14:23:28 +0200 (Di, 13 Jul 2010) | 2 lines
updated test readconfig for os config variable
........
r364 | bricks | 2010-07-13 14:24:14 +0200 (Di, 13 Jul 2010) | 2 lines
moved common publish functions to a seperate module
........
r365 | bricks | 2010-07-13 16:26:11 +0200 (Di, 13 Jul 2010) | 2 lines
moved arch info to each package info
........
r366 | bricks | 2010-07-13 18:46:17 +0200 (Di, 13 Jul 2010) | 2 lines
inital checkin for new publishpackages processing
........
r367 | bricks | 2010-07-21 17:52:15 +0200 (Mi, 21 Jul 2010) | 2 lines
implemented parsing from xml string
........
r368 | bricks | 2010-07-22 12:17:42 +0200 (Do, 22 Jul 2010) | 2 lines
copy binary-all packages in all binary-xyz dirs
........
r369 | bricks | 2010-07-22 12:26:40 +0200 (Do, 22 Jul 2010) | 2 lines
fixed small bugs
........
r370 | bricks | 2010-07-22 13:02:36 +0200 (Do, 22 Jul 2010) | 2 lines
added a new root element which is called TreepkgRoot for Treepkg Info classes
........
r371 | bricks | 2010-07-22 13:09:52 +0200 (Do, 22 Jul 2010) | 3 lines
fixed some small issues
remember: run pychecker BEFORE the commit
........
r372 | bricks | 2010-07-22 18:01:43 +0200 (Do, 22 Jul 2010) | 2 lines
improved publishdebianpackages and fixed a lot of bugs in the xml info parsing part
........
r373 | bricks | 2010-07-23 18:40:38 +0200 (Fr, 23 Jul 2010) | 2 lines
implemented CacheDb to store copied Packages from build host
........
r374 | bricks | 2010-07-26 10:28:48 +0200 (Mo, 26 Jul 2010) | 2 lines
moved data handling in publishdebianpackages into seperate module
........
r375 | bricks | 2010-07-26 11:42:07 +0200 (Mo, 26 Jul 2010) | 2 lines
fixed some typos
........
r376 | bricks | 2010-07-26 12:12:40 +0200 (Mo, 26 Jul 2010) | 2 lines
first version of incremental copying debian packages to cachedir
........
r377 | bricks | 2010-07-26 14:58:50 +0200 (Mo, 26 Jul 2010) | 2 lines
incremental copying only changed debian packages
........
r378 | bricks | 2010-07-26 15:39:10 +0200 (Mo, 26 Jul 2010) | 3 lines
use md5 instead of hashlib module
(md5 is deprecated in favour of hashlib in python 2.5)
........
r379 | bricks | 2010-07-26 15:46:19 +0200 (Mo, 26 Jul 2010) | 2 lines
make treepkg compatible to python 2.4
........
r380 | bricks | 2010-07-26 15:50:49 +0200 (Mo, 26 Jul 2010) | 2 lines
fixed last commit (db instead of sqlite3)
........
r381 | bricks | 2010-07-27 10:54:15 +0200 (Di, 27 Jul 2010) | 2 lines
fixed wrong import statement
........
r382 | bricks | 2010-07-27 10:54:34 +0200 (Di, 27 Jul 2010) | 2 lines
removed unnecessary debug output
........
r383 | bricks | 2010-07-27 18:28:22 +0200 (Di, 27 Jul 2010) | 2 lines
check if upload hook is empty
........
r384 | bricks | 2010-07-28 09:35:21 +0200 (Mi, 28 Jul 2010) | 2 lines
run rsync only an arch dirs that are present in the current treepkg
........
r385 | bricks | 2010-07-28 09:35:54 +0200 (Mi, 28 Jul 2010) | 2 lines
add helper tool to list content of a cache db
........
r386 | bricks | 2010-07-28 10:33:48 +0200 (Mi, 28 Jul 2010) | 3 lines
fixed a typo
regex fails if option is empty
........
| author | Bjoern Ricks <bricks@intevation.de> |
|---|---|
| date | Fri, 06 Aug 2010 13:28:47 +0000 |
| parents | ef87d30468b6 |
| children | bccd69dedbe5 |
| rev | line source |
|---|---|
|
287
1fcdffbeb9de
Make the #! line in the commands more portable. Use /usr/bin/python
Bernhard Herzog <bh@intevation.de>
parents:
158
diff
changeset
|
1 #! /usr/bin/python |
|
1fcdffbeb9de
Make the #! line in the commands more portable. Use /usr/bin/python
Bernhard Herzog <bh@intevation.de>
parents:
158
diff
changeset
|
2 # Copyright (C) 2007, 2008, 2009 by Intevation GmbH |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
3 # Authors: |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
4 # Bernhard Herzog <bh@intevation.de> |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
5 # |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
6 # This program is free software under the GPL (>=v2) |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
7 # Read the file COPYING coming with the software for details. |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
8 |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
9 """Publishes a static web-site with a status report""" |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
10 |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
11 import sys |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
12 import os |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
13 from optparse import OptionParser |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
14 from ConfigParser import SafeConfigParser |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
15 |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
16 import treepkgcmd |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
17 from treepkg.readconfig import read_config_section |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
18 from treepkg.run import call |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
19 from treepkg.cmdexpand import cmdexpand |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
20 from treepkg.util import ensure_directory |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
21 |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
22 def remove_trailing_slashes(s): |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
23 return s.rstrip("/") |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
24 |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
25 def expand_filename(filename): |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
26 """ |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
27 Applies os.path.expanduser and os.path.expandvars to filename |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
28 """ |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
29 return os.path.expandvars(os.path.expanduser(filename)) |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
30 |
|
158
8ec9ed76d67b
Make the template used by bin/publishstaticweb.py configurable.
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
31 staticweb_desc = ["build_user", "build_host", "build_create", "build_template", |
|
372
ef87d30468b6
Added the option to expose additional log files from the log directory.
Andre Heinecke <aheinecke@intevation.de>
parents:
338
diff
changeset
|
32 "build_logs", |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
33 ("build_dir", remove_trailing_slashes), |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
34 "publish_user", "publish_host", |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
35 ("publish_dir", remove_trailing_slashes), |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
36 ("cachedir", |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
37 lambda s: expand_filename(remove_trailing_slashes(s)))] |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
38 |
|
372
ef87d30468b6
Added the option to expose additional log files from the log directory.
Andre Heinecke <aheinecke@intevation.de>
parents:
338
diff
changeset
|
39 #Default values for the configuration options can be set here |
|
ef87d30468b6
Added the option to expose additional log files from the log directory.
Andre Heinecke <aheinecke@intevation.de>
parents:
338
diff
changeset
|
40 staticweb_defaults = [("build_logs", "build_log.txt.gz")] |
|
ef87d30468b6
Added the option to expose additional log files from the log directory.
Andre Heinecke <aheinecke@intevation.de>
parents:
338
diff
changeset
|
41 |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
42 def read_config(filename): |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
43 parser = SafeConfigParser() |
|
372
ef87d30468b6
Added the option to expose additional log files from the log directory.
Andre Heinecke <aheinecke@intevation.de>
parents:
338
diff
changeset
|
44 parser.add_section("staticweb") |
|
ef87d30468b6
Added the option to expose additional log files from the log directory.
Andre Heinecke <aheinecke@intevation.de>
parents:
338
diff
changeset
|
45 for value in staticweb_defaults: |
|
ef87d30468b6
Added the option to expose additional log files from the log directory.
Andre Heinecke <aheinecke@intevation.de>
parents:
338
diff
changeset
|
46 parser.set("staticweb", value[0], value[1]) |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
47 parser.read([filename]) |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
48 return read_config_section(parser, "staticweb", staticweb_desc) |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
49 |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
50 def parse_commandline(): |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
51 parser = OptionParser() |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
52 parser.set_defaults(config_file=os.path.join(treepkgcmd.topdir, |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
53 "staticweb.cfg")) |
|
338
28df50b267f6
Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents:
319
diff
changeset
|
54 parser.set_defaults(update_cache_only=False) |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
55 parser.add_option("--config-file", |
|
319
dba1b6b825ec
Improve help string for --config-file option of publishstaticweb.
Bernhard Herzog <bh@intevation.de>
parents:
287
diff
changeset
|
56 help=("The configuration file. Default is staticweb.cfg")) |
|
338
28df50b267f6
Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents:
319
diff
changeset
|
57 parser.add_option("--update-cache-only", action="store_true", |
|
28df50b267f6
Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents:
319
diff
changeset
|
58 help=("Stop after updating the cache." |
|
28df50b267f6
Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents:
319
diff
changeset
|
59 " This omits the last step of the publishing" |
|
28df50b267f6
Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents:
319
diff
changeset
|
60 " process")) |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
61 return parser.parse_args() |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
62 |
|
338
28df50b267f6
Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents:
319
diff
changeset
|
63 def publish_static_site(config_filename, update_cache_only=False): |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
64 config = read_config(config_filename) |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
65 |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
66 # create web-page on build host |
|
158
8ec9ed76d67b
Make the template used by bin/publishstaticweb.py configurable.
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
67 call(cmdexpand("ssh $build_user$@$build_host $build_create" |
|
372
ef87d30468b6
Added the option to expose additional log files from the log directory.
Andre Heinecke <aheinecke@intevation.de>
parents:
338
diff
changeset
|
68 " --show-logs=$build_logs" |
|
158
8ec9ed76d67b
Make the template used by bin/publishstaticweb.py configurable.
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
69 " --status-template=$build_template $build_dir", |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
70 **config)) |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
71 |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
72 # rsync the new web-pages to the local cache |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
73 ensure_directory(config["cachedir"]) |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
74 call(cmdexpand("rsync -rL --delete $build_user$@$build_host:$build_dir/" |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
75 " $cachedir", |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
76 **config)) |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
77 |
|
338
28df50b267f6
Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents:
319
diff
changeset
|
78 if not update_cache_only: |
|
28df50b267f6
Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents:
319
diff
changeset
|
79 # rsync the web pages from the local cache to the publishing host |
|
28df50b267f6
Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents:
319
diff
changeset
|
80 call(cmdexpand("rsync -rL --delete $cachedir/" |
|
28df50b267f6
Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents:
319
diff
changeset
|
81 " $publish_user$@$publish_host:$publish_dir", |
|
28df50b267f6
Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents:
319
diff
changeset
|
82 **config)) |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
83 |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
84 |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
85 def main(): |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
86 options, args = parse_commandline() |
|
338
28df50b267f6
Expanded the pkg_revision_template dictionary to include rules revision
Andre Heinecke <aheinecke@intevation.de>
parents:
319
diff
changeset
|
87 publish_static_site(options.config_file, options.update_cache_only) |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
88 |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
89 main() |