view bin/publishstaticweb.py @ 440:383d23c6bfde

Merged revisions 346-386 via svnmerge from svn+ssh://svn.wald.intevation.org/treepkg/branches/treepkg-status ........ r346 | bricks | 2010-07-07 17:40:48 +0200 (Mi, 07 Jul 2010) | 2 lines added classed for treepkg status xml generation ........ r347 | bricks | 2010-07-08 12:07:39 +0200 (Do, 08 Jul 2010) | 10 lines renamed new status dir to info because of a naming conflict with status.py let the user specify a treepkg name in the config the name is propagated to PackagerGroup [treepkg] name: <treepkgname> becomes: pg = PackagerGroup(...) pg.name ........ r348 | bricks | 2010-07-08 15:25:07 +0200 (Do, 08 Jul 2010) | 3 lines moved test file to test main dir fixed missing : after def statement ........ r349 | bricks | 2010-07-08 17:59:52 +0200 (Do, 08 Jul 2010) | 2 lines added treepkg_dir andd tracks_dir attributes to PackageGroup ........ r350 | bricks | 2010-07-08 18:00:32 +0200 (Do, 08 Jul 2010) | 2 lines changed <millpath> to <trackspath> ........ r351 | bricks | 2010-07-08 18:08:39 +0200 (Do, 08 Jul 2010) | 2 lines check if config file exists to get usefull error output ........ r352 | bricks | 2010-07-08 18:11:13 +0200 (Do, 08 Jul 2010) | 2 lines write first info about tracks ........ r353 | bricks | 2010-07-09 12:20:49 +0200 (Fr, 09 Jul 2010) | 2 lines get all log files ........ r354 | bricks | 2010-07-09 14:42:42 +0200 (Fr, 09 Jul 2010) | 2 lines added md5sum function ........ r355 | bricks | 2010-07-09 15:15:15 +0200 (Fr, 09 Jul 2010) | 4 lines implemented nearly all info only arch and os info are missing ........ r356 | bricks | 2010-07-09 16:19:17 +0200 (Fr, 09 Jul 2010) | 2 lines Bugfix: source was not defined ........ r357 | bricks | 2010-07-09 16:44:52 +0200 (Fr, 09 Jul 2010) | 3 lines fixed some typos added missing add_checksum methond ........ r358 | bricks | 2010-07-09 17:06:15 +0200 (Fr, 09 Jul 2010) | 2 lines fixed status line for revision ........ r359 | bricks | 2010-07-12 14:10:34 +0200 (Mo, 12 Jul 2010) | 2 lines let the user specify num revisions that should be shown in the info ........ r360 | bricks | 2010-07-12 14:21:39 +0200 (Mo, 12 Jul 2010) | 2 lines missed in last commit ........ r361 | bricks | 2010-07-12 17:06:22 +0200 (Mo, 12 Jul 2010) | 2 lines add os config statement ........ r362 | bricks | 2010-07-12 17:47:42 +0200 (Mo, 12 Jul 2010) | 3 lines moved packages and logs to revision tag implemented os and arch info ........ r363 | bricks | 2010-07-13 14:23:28 +0200 (Di, 13 Jul 2010) | 2 lines updated test readconfig for os config variable ........ r364 | bricks | 2010-07-13 14:24:14 +0200 (Di, 13 Jul 2010) | 2 lines moved common publish functions to a seperate module ........ r365 | bricks | 2010-07-13 16:26:11 +0200 (Di, 13 Jul 2010) | 2 lines moved arch info to each package info ........ r366 | bricks | 2010-07-13 18:46:17 +0200 (Di, 13 Jul 2010) | 2 lines inital checkin for new publishpackages processing ........ r367 | bricks | 2010-07-21 17:52:15 +0200 (Mi, 21 Jul 2010) | 2 lines implemented parsing from xml string ........ r368 | bricks | 2010-07-22 12:17:42 +0200 (Do, 22 Jul 2010) | 2 lines copy binary-all packages in all binary-xyz dirs ........ r369 | bricks | 2010-07-22 12:26:40 +0200 (Do, 22 Jul 2010) | 2 lines fixed small bugs ........ r370 | bricks | 2010-07-22 13:02:36 +0200 (Do, 22 Jul 2010) | 2 lines added a new root element which is called TreepkgRoot for Treepkg Info classes ........ r371 | bricks | 2010-07-22 13:09:52 +0200 (Do, 22 Jul 2010) | 3 lines fixed some small issues remember: run pychecker BEFORE the commit ........ r372 | bricks | 2010-07-22 18:01:43 +0200 (Do, 22 Jul 2010) | 2 lines improved publishdebianpackages and fixed a lot of bugs in the xml info parsing part ........ r373 | bricks | 2010-07-23 18:40:38 +0200 (Fr, 23 Jul 2010) | 2 lines implemented CacheDb to store copied Packages from build host ........ r374 | bricks | 2010-07-26 10:28:48 +0200 (Mo, 26 Jul 2010) | 2 lines moved data handling in publishdebianpackages into seperate module ........ r375 | bricks | 2010-07-26 11:42:07 +0200 (Mo, 26 Jul 2010) | 2 lines fixed some typos ........ r376 | bricks | 2010-07-26 12:12:40 +0200 (Mo, 26 Jul 2010) | 2 lines first version of incremental copying debian packages to cachedir ........ r377 | bricks | 2010-07-26 14:58:50 +0200 (Mo, 26 Jul 2010) | 2 lines incremental copying only changed debian packages ........ r378 | bricks | 2010-07-26 15:39:10 +0200 (Mo, 26 Jul 2010) | 3 lines use md5 instead of hashlib module (md5 is deprecated in favour of hashlib in python 2.5) ........ r379 | bricks | 2010-07-26 15:46:19 +0200 (Mo, 26 Jul 2010) | 2 lines make treepkg compatible to python 2.4 ........ r380 | bricks | 2010-07-26 15:50:49 +0200 (Mo, 26 Jul 2010) | 2 lines fixed last commit (db instead of sqlite3) ........ r381 | bricks | 2010-07-27 10:54:15 +0200 (Di, 27 Jul 2010) | 2 lines fixed wrong import statement ........ r382 | bricks | 2010-07-27 10:54:34 +0200 (Di, 27 Jul 2010) | 2 lines removed unnecessary debug output ........ r383 | bricks | 2010-07-27 18:28:22 +0200 (Di, 27 Jul 2010) | 2 lines check if upload hook is empty ........ r384 | bricks | 2010-07-28 09:35:21 +0200 (Mi, 28 Jul 2010) | 2 lines run rsync only an arch dirs that are present in the current treepkg ........ r385 | bricks | 2010-07-28 09:35:54 +0200 (Mi, 28 Jul 2010) | 2 lines add helper tool to list content of a cache db ........ r386 | bricks | 2010-07-28 10:33:48 +0200 (Mi, 28 Jul 2010) | 3 lines fixed a typo regex fails if option is empty ........
author Bjoern Ricks <bricks@intevation.de>
date Fri, 06 Aug 2010 13:28:47 +0000
parents ef87d30468b6
children bccd69dedbe5
line wrap: on
line source
#! /usr/bin/python
# Copyright (C) 2007, 2008, 2009 by Intevation GmbH
# Authors:
# Bernhard Herzog <bh@intevation.de>
#
# This program is free software under the GPL (>=v2)
# Read the file COPYING coming with the software for details.

"""Publishes a static web-site with a status report"""

import sys
import os
from optparse import OptionParser
from ConfigParser import SafeConfigParser

import treepkgcmd
from treepkg.readconfig import read_config_section
from treepkg.run import call
from treepkg.cmdexpand import cmdexpand
from treepkg.util import ensure_directory

def remove_trailing_slashes(s):
    return s.rstrip("/")

def expand_filename(filename):
    """
    Applies os.path.expanduser and os.path.expandvars to filename
    """
    return os.path.expandvars(os.path.expanduser(filename))

staticweb_desc = ["build_user", "build_host", "build_create", "build_template",
                  "build_logs",
                  ("build_dir", remove_trailing_slashes),
                  "publish_user", "publish_host",
                  ("publish_dir", remove_trailing_slashes),
                  ("cachedir",
                   lambda s: expand_filename(remove_trailing_slashes(s)))]

#Default values for the configuration options can be set here
staticweb_defaults = [("build_logs", "build_log.txt.gz")]

def read_config(filename):
    parser = SafeConfigParser()
    parser.add_section("staticweb")
    for value in staticweb_defaults:
        parser.set("staticweb", value[0], value[1])
    parser.read([filename])
    return read_config_section(parser, "staticweb", staticweb_desc)

def parse_commandline():
    parser = OptionParser()
    parser.set_defaults(config_file=os.path.join(treepkgcmd.topdir,
                                                 "staticweb.cfg"))
    parser.set_defaults(update_cache_only=False)
    parser.add_option("--config-file",
                      help=("The configuration file. Default is staticweb.cfg"))
    parser.add_option("--update-cache-only", action="store_true",
                      help=("Stop after updating the cache."
                            " This omits the last step of the publishing"
                            " process"))
    return parser.parse_args()

def publish_static_site(config_filename, update_cache_only=False):
    config = read_config(config_filename)

    # create web-page on build host
    call(cmdexpand("ssh $build_user$@$build_host $build_create"
                   " --show-logs=$build_logs"
                   " --status-template=$build_template $build_dir",
                   **config))

    # rsync the new web-pages to the local cache
    ensure_directory(config["cachedir"])
    call(cmdexpand("rsync -rL --delete $build_user$@$build_host:$build_dir/"
                   " $cachedir",
                   **config))

    if not update_cache_only:
        # rsync the web pages from the local cache to the publishing host
        call(cmdexpand("rsync -rL --delete $cachedir/"
                       " $publish_user$@$publish_host:$publish_dir",
                       **config))


def main():
    options, args = parse_commandline()
    publish_static_site(options.config_file, options.update_cache_only)

main()
This site is hosted by Intevation GmbH (Datenschutzerklärung und Impressum | Privacy Policy and Imprint)