treepkg/treepkg: treepkg/packager.py annotate

annotate treepkg/packager.py @ 99:7888fe374e11

Add support for notification mails in case of build errors This involves a new status field notification_mail to keep track of whether a notification has been sent for a particular build attempt and two programs to list the pending notifications and to send the pending notifications (similar to how the static web pages are published) as well as the corresponding configuration files.

author	Bernhard Herzog <bh@intevation.de>
date	Tue, 19 Feb 2008 19:19:23 +0000
parents	f7b9c7113c46
children	66cbfc772f84

rev	line source
99 7888fe374e11 Add support for notification mails in case of build errors Bernhard Herzog <bh@intevation.de> parents: 98 diff changeset	1 # Copyright (C) 2007, 2008 by Intevation GmbH
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	2 # Authors:
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	3 # Bernhard Herzog <bh@intevation.de>
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	4 #
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	5 # This program is free software under the GPL (>=v2)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	6 # Read the file COPYING coming with the software for details.
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	7
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	8 """Classes to automatically build debian packages from subversion checkouts"""
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	9
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	10 import os
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	11 import time
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	12 import re
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	13 import logging
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	14 import shutil
16 7c55f3879c4d Use the new status class and report start/stop time too Bernhard Herzog <bh@intevation.de> parents: 14 diff changeset	15 import datetime
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	16
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	17 import util
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	18 import subversion
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	19 import run
16 7c55f3879c4d Use the new status class and report start/stop time too Bernhard Herzog <bh@intevation.de> parents: 14 diff changeset	20 import status
45 3e610233ccfe use cmdexpand when calling subprocesses Bernhard Herzog <bh@intevation.de> parents: 41 diff changeset	21 from cmdexpand import cmdexpand
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	22
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	23 def _filenameproperty(relative_dir):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	24 def get(self):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	25 return os.path.join(self.base_dir, relative_dir)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	26 return property(get)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	27
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	28
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	29 class SourcePackager(object):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	30
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	31 # Derived classes must supply the package basename
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	32 pkg_basename = None
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	33
53 74cd21b6400b rename some variables from pkg_track to track Bernhard Herzog <bh@intevation.de> parents: 52 diff changeset	34 def __init__(self, track, status, work_dir, src_dir, revision):
74cd21b6400b rename some variables from pkg_track to track Bernhard Herzog <bh@intevation.de> parents: 52 diff changeset	35 self.track = track
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	36 self.status = status
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	37 self.work_dir = work_dir
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	38 self.src_dir = src_dir
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	39 self.revision = revision
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	40 assert(self.pkg_basename)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	41
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	42 def determine_package_version(self, directory):
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	43 """Returns the version number of the new package as a string
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	44
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	45 The directory parameter is the name of the directory containing
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	46 the newly exported sources. The sources were exported with the
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	47 export_sources method.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	48
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	49 The default implementation simply returns the revision converted
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	50 to a string.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	51 """
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	52 return str(self.revision)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	53
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	54 def export_sources(self):
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	55 """Export the sources from the subversion working directory
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	56
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	57 This method first exports the sources to a temporary directory
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	58 and then renames the directory. The new name is of the form
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	59
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	60 <pkg_basename>-<version>
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	61
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	62 Where pkg_basename is the value of self.pkg_basename and version
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	63 is the return value of the determine_package_version() method.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	64 """
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	65 temp_dir = os.path.join(self.work_dir, "temp")
53 74cd21b6400b rename some variables from pkg_track to track Bernhard Herzog <bh@intevation.de> parents: 52 diff changeset	66 self.track.export_sources(temp_dir)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	67
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	68 pkgbaseversion = self.determine_package_version(temp_dir)
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	69 pkgbasedir = os.path.join(self.work_dir,
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	70 self.pkg_basename + "-" + pkgbaseversion)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	71
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	72 os.rename(temp_dir, pkgbasedir)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	73 return pkgbaseversion, pkgbasedir
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	74
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	75
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	76 def update_version_numbers(self, pkgbasedir):
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	77 """Updates the version numbers in the code in pkgbasedir.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	78
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	79 The default implementation does nothing. Derived classes should
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	80 override this method if necessary.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	81 """
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	82
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	83 def create_tarball(self, tarballname, workdir, basedir):
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	84 """Creates a new tarball.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	85
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	86 Parameters:
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	87
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	88 tarballname -- the filename of the new tarball
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	89 workdir -- The directory into which to change before running tar.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	90 (actually this is done with GNUI tar's -C option)
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	91 basedir -- The basedirectory of the files that are packaged
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	92 into the tarfile. This should be a relative
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	93 filename directly in workdir.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	94 """
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	95 logging.info("Creating tarball %r", tarballname)
45 3e610233ccfe use cmdexpand when calling subprocesses Bernhard Herzog <bh@intevation.de> parents: 41 diff changeset	96 run.call(cmdexpand("tar czf $tarballname -C $workdir $basedir",
3e610233ccfe use cmdexpand when calling subprocesses Bernhard Herzog <bh@intevation.de> parents: 41 diff changeset	97 **locals()))
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	98
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	99 def copy_debian_directory(self, pkgbasedir, pkgbaseversion, changemsg):
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	100 """Copies the debian directory and updates the copy's changelog
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	101
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	102 Parameter:
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	103 pkgbasedir -- The directory holding the unpacked source package
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	104 pkgbaseversion -- The version to update the changelog to
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	105 changemsg -- The message for the changelog
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	106
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	107 When determining the actual version for the new package, this
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	108 function looks at the previous version in the changelog. If it
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	109 has a prefix separated from the version number by a colon this
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	110 prefix is prepended to the pkgbaseversion parameter. Debian
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	111 uses such prefixes for the kde packages.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	112 """
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	113 debian_dir = os.path.join(pkgbasedir, "debian")
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	114 changelog = os.path.join(debian_dir, "changelog")
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	115
53 74cd21b6400b rename some variables from pkg_track to track Bernhard Herzog <bh@intevation.de> parents: 52 diff changeset	116 self.track.copy_debian_directory(debian_dir)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	117
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	118 logging.info("Updating %r", changelog)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	119 oldversion = util.debian_changelog_version(changelog)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	120 if ":" in oldversion:
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	121 oldversionprefix = oldversion.split(":")[0] + ":"
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	122 else:
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	123 oldversionprefix = ""
93 73c67372c7f7 Make the prefix used in the debian revision number configurable. Bernhard Herzog <bh@intevation.de> parents: 91 diff changeset	124 debrev = self.track.debrevision_prefix + "1"
45 3e610233ccfe use cmdexpand when calling subprocesses Bernhard Herzog <bh@intevation.de> parents: 41 diff changeset	125 run.call(cmdexpand("debchange -c $changelog"
93 73c67372c7f7 Make the prefix used in the debian revision number configurable. Bernhard Herzog <bh@intevation.de> parents: 91 diff changeset	126 " -v ${oldversionprefix}${pkgbaseversion}-${debrev}"
45 3e610233ccfe use cmdexpand when calling subprocesses Bernhard Herzog <bh@intevation.de> parents: 41 diff changeset	127 " $changemsg", **locals()),
53 74cd21b6400b rename some variables from pkg_track to track Bernhard Herzog <bh@intevation.de> parents: 52 diff changeset	128 env=self.track.debian_environment())
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	129
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	130
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	131 def create_source_package(self, pkgbasedir, origtargz):
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	132 """Creates a new source package from pkgbasedir and origtargz"""
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	133 logging.info("Creating new source package")
45 3e610233ccfe use cmdexpand when calling subprocesses Bernhard Herzog <bh@intevation.de> parents: 41 diff changeset	134 run.call(cmdexpand("dpkg-source -b $directory $tarball",
3e610233ccfe use cmdexpand when calling subprocesses Bernhard Herzog <bh@intevation.de> parents: 41 diff changeset	135 directory=os.path.basename(pkgbasedir),
3e610233ccfe use cmdexpand when calling subprocesses Bernhard Herzog <bh@intevation.de> parents: 41 diff changeset	136 tarball=os.path.basename(origtargz)),
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	137 cwd=os.path.dirname(pkgbasedir),
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	138 suppress_output=True,
53 74cd21b6400b rename some variables from pkg_track to track Bernhard Herzog <bh@intevation.de> parents: 52 diff changeset	139 env=self.track.debian_environment())
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	140
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	141 def move_source_package(self, pkgbasename):
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	142 """Moves the new source package from the work_dir to the src_dir"""
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	143 logging.info("Moving source package to %r", self.src_dir)
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	144 util.ensure_directory(self.src_dir)
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	145 for filename in [filename for filename in os.listdir(self.work_dir)
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	146 if filename.startswith(pkgbasename)]:
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	147 os.rename(os.path.join(self.work_dir, filename),
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	148 os.path.join(self.src_dir, filename))
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	149
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	150 def package(self):
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	151 """Creates a source package from a subversion checkout.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	152
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	153 After setting up the working directory, this method calls the
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	154 do_package method to do the actual packaging. Afterwards the
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	155 work directory is removed.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	156 """
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	157 util.ensure_directory(self.work_dir)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	158 try:
41 f7ec40638a06 use the enums for the status field of RevisionStatus Bernhard Herzog <bh@intevation.de> parents: 36 diff changeset	159 self.status.creating_source_package()
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	160 self.do_package()
41 f7ec40638a06 use the enums for the status field of RevisionStatus Bernhard Herzog <bh@intevation.de> parents: 36 diff changeset	161 self.status.source_package_created()
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	162 finally:
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	163 logging.info("Removing workdir %r", self.work_dir)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	164 shutil.rmtree(self.work_dir)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	165
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	166 def do_package(self):
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	167 """Does the work of creating a source package
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	168 This method must be overriden by derived classes.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	169
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	170 The method should do the work in self.work_dir. When the
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	171 package is done, the source package files should be in
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	172 self.src_dir.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	173 """
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	174 raise NotImplementedError
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	175
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	176
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	177 class BinaryPackager(object):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	178
53 74cd21b6400b rename some variables from pkg_track to track Bernhard Herzog <bh@intevation.de> parents: 52 diff changeset	179 def __init__(self, track, status, binary_dir, dsc_file, logfile):
74cd21b6400b rename some variables from pkg_track to track Bernhard Herzog <bh@intevation.de> parents: 52 diff changeset	180 self.track = track
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	181 self.status = status
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	182 self.binary_dir = binary_dir
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	183 self.dsc_file = dsc_file
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	184 self.logfile = logfile
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	185
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	186 def package(self):
41 f7ec40638a06 use the enums for the status field of RevisionStatus Bernhard Herzog <bh@intevation.de> parents: 36 diff changeset	187 self.status.creating_binary_package()
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	188 util.ensure_directory(self.binary_dir)
54 b0014b8f6029 fix typo in log message Bernhard Herzog <bh@intevation.de> parents: 53 diff changeset	189 logging.info("Building binary package; logging to %r", self.logfile)
45 3e610233ccfe use cmdexpand when calling subprocesses Bernhard Herzog <bh@intevation.de> parents: 41 diff changeset	190 run.call(cmdexpand("@rootcmd /usr/sbin/pbuilder build"
51 e1028211d34b merge Bernhard Herzog <bh@intevation.de> parents: 45 49 diff changeset	191 " --configfile $pbuilderrc"
45 3e610233ccfe use cmdexpand when calling subprocesses Bernhard Herzog <bh@intevation.de> parents: 41 diff changeset	192 " --logfile $logfile --buildresult $bindir $dsc",
55 6b5f7f7575f6 merge Bernhard Herzog <bh@intevation.de> parents: 51 54 diff changeset	193 rootcmd=self.track.root_cmd,
6b5f7f7575f6 merge Bernhard Herzog <bh@intevation.de> parents: 51 54 diff changeset	194 pbuilderrc=self.track.pbuilderrc,
51 e1028211d34b merge Bernhard Herzog <bh@intevation.de> parents: 45 49 diff changeset	195 logfile=self.logfile, bindir=self.binary_dir,
e1028211d34b merge Bernhard Herzog <bh@intevation.de> parents: 45 49 diff changeset	196 dsc=self.dsc_file),
45 3e610233ccfe use cmdexpand when calling subprocesses Bernhard Herzog <bh@intevation.de> parents: 41 diff changeset	197 suppress_output=True)
84 98a7d70746a9 Make BinaryPackager remove all files that do not belong to the binary Bernhard Herzog <bh@intevation.de> parents: 80 diff changeset	198 # remove the source package files put into the binary directory
98a7d70746a9 Make BinaryPackager remove all files that do not belong to the binary Bernhard Herzog <bh@intevation.de> parents: 80 diff changeset	199 # by pbuilder
98a7d70746a9 Make BinaryPackager remove all files that do not belong to the binary Bernhard Herzog <bh@intevation.de> parents: 80 diff changeset	200 for filename in os.listdir(self.binary_dir):
98a7d70746a9 Make BinaryPackager remove all files that do not belong to the binary Bernhard Herzog <bh@intevation.de> parents: 80 diff changeset	201 if os.path.splitext(filename)[1] not in (".deb", ".changes"):
98a7d70746a9 Make BinaryPackager remove all files that do not belong to the binary Bernhard Herzog <bh@intevation.de> parents: 80 diff changeset	202 os.remove(os.path.join(self.binary_dir, filename))
41 f7ec40638a06 use the enums for the status field of RevisionStatus Bernhard Herzog <bh@intevation.de> parents: 36 diff changeset	203 self.status.binary_package_created()
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	204
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	205
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	206 class RevisionPackager(object):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	207
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	208 source_packager_cls = SourcePackager
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	209 binary_packager_cls = BinaryPackager
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	210
53 74cd21b6400b rename some variables from pkg_track to track Bernhard Herzog <bh@intevation.de> parents: 52 diff changeset	211 def __init__(self, track, revision):
74cd21b6400b rename some variables from pkg_track to track Bernhard Herzog <bh@intevation.de> parents: 52 diff changeset	212 self.track = track
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	213 self.revision = revision
53 74cd21b6400b rename some variables from pkg_track to track Bernhard Herzog <bh@intevation.de> parents: 52 diff changeset	214 self.base_dir = self.track.pkg_dir_for_revision(self.revision, 1)
36 086c68ca51d2 rename Status to RevisionStatus Bernhard Herzog <bh@intevation.de> parents: 26 diff changeset	215 self.status = status.RevisionStatus(os.path.join(self.base_dir,
086c68ca51d2 rename Status to RevisionStatus Bernhard Herzog <bh@intevation.de> parents: 26 diff changeset	216 "status"))
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	217
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	218 work_dir = _filenameproperty("work")
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	219 binary_dir = _filenameproperty("binary")
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	220 src_dir = _filenameproperty("src")
18 d5c24cfce05e Improve access to a RevisionPackager's build_log Bernhard Herzog <bh@intevation.de> parents: 16 diff changeset	221 build_log = _filenameproperty("build.log")
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	222
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	223 def find_dsc_file(self):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	224 for filename in os.listdir(self.src_dir):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	225 if filename.endswith(".dsc"):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	226 return os.path.join(self.src_dir, filename)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	227 return None
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	228
18 d5c24cfce05e Improve access to a RevisionPackager's build_log Bernhard Herzog <bh@intevation.de> parents: 16 diff changeset	229 def has_build_log(self):
d5c24cfce05e Improve access to a RevisionPackager's build_log Bernhard Herzog <bh@intevation.de> parents: 16 diff changeset	230 return os.path.exists(self.build_log)
d5c24cfce05e Improve access to a RevisionPackager's build_log Bernhard Herzog <bh@intevation.de> parents: 16 diff changeset	231
88 3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	232 def list_source_files(self):
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	233 """Returns a list with the names of the files of the source package.
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	234 The implementation assumes that all files in self.src_dir belong
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	235 to the source package.
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	236 """
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	237 return sorted(util.listdir_abs(self.src_dir))
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	238
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	239 def list_binary_files(self):
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	240 """Returns a list with the names of the files of the binary packages.
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	241 The implementation assumes that all files in self.binary_dir belong
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	242 to the binary packages.
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	243 """
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	244 return sorted(util.listdir_abs(self.binary_dir))
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	245
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	246 def package(self):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	247 try:
16 7c55f3879c4d Use the new status class and report start/stop time too Bernhard Herzog <bh@intevation.de> parents: 14 diff changeset	248 util.ensure_directory(self.work_dir)
7c55f3879c4d Use the new status class and report start/stop time too Bernhard Herzog <bh@intevation.de> parents: 14 diff changeset	249 self.status.start = datetime.datetime.utcnow()
53 74cd21b6400b rename some variables from pkg_track to track Bernhard Herzog <bh@intevation.de> parents: 52 diff changeset	250 src_packager = self.source_packager_cls(self.track, self.status,
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	251 self.work_dir, self.src_dir,
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	252 self.revision)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	253 src_packager.package()
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	254
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	255 dsc_file = self.find_dsc_file()
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	256 if dsc_file is None:
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	257 raise RuntimeError("Cannot find dsc File in %r" % self.src_dir)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	258
53 74cd21b6400b rename some variables from pkg_track to track Bernhard Herzog <bh@intevation.de> parents: 52 diff changeset	259 bin_packager = self.binary_packager_cls(self.track, self.status,
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	260 self.binary_dir, dsc_file,
18 d5c24cfce05e Improve access to a RevisionPackager's build_log Bernhard Herzog <bh@intevation.de> parents: 16 diff changeset	261 self.build_log)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	262 bin_packager.package()
16 7c55f3879c4d Use the new status class and report start/stop time too Bernhard Herzog <bh@intevation.de> parents: 14 diff changeset	263 self.status.stop = datetime.datetime.utcnow()
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	264 except:
41 f7ec40638a06 use the enums for the status field of RevisionStatus Bernhard Herzog <bh@intevation.de> parents: 36 diff changeset	265 self.status.error()
16 7c55f3879c4d Use the new status class and report start/stop time too Bernhard Herzog <bh@intevation.de> parents: 14 diff changeset	266 self.status.stop = datetime.datetime.utcnow()
99 7888fe374e11 Add support for notification mails in case of build errors Bernhard Herzog <bh@intevation.de> parents: 98 diff changeset	267 # set the notification status last to avoid race conditions.
7888fe374e11 Add support for notification mails in case of build errors Bernhard Herzog <bh@intevation.de> parents: 98 diff changeset	268 # The pending notification is for now the only situation
7888fe374e11 Add support for notification mails in case of build errors Bernhard Herzog <bh@intevation.de> parents: 98 diff changeset	269 # where another process might modify the status file (the
7888fe374e11 Add support for notification mails in case of build errors Bernhard Herzog <bh@intevation.de> parents: 98 diff changeset	270 # listpendingnotifications program will set it to
7888fe374e11 Add support for notification mails in case of build errors Bernhard Herzog <bh@intevation.de> parents: 98 diff changeset	271 # "notification_sent")
7888fe374e11 Add support for notification mails in case of build errors Bernhard Herzog <bh@intevation.de> parents: 98 diff changeset	272 self.status.notification_pending()
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	273 raise
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	274
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	275 def remove_package_dir(self):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	276 logging.info("Removing pkgdir %r", self.base_dir)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	277 shutil.rmtree(self.base_dir)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	278
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	279
52 78cf5f6778ec Rename 'packagel ine' -> 'package track' Bernhard Herzog <bh@intevation.de> parents: 49 diff changeset	280 class PackageTrack(object):
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	281
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	282 revision_packager_cls = RevisionPackager
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	283
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	284 svn_external_subdirs = []
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	285
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	286 extra_config_desc = []
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	287
47 2802be410156 add config options pbuilderrc and use it when calling pbuilder Bernhard Herzog <bh@intevation.de> parents: 16 diff changeset	288 def __init__(self, name, base_dir, svn_url, root_cmd, pbuilderrc, deb_email,
93 73c67372c7f7 Make the prefix used in the debian revision number configurable. Bernhard Herzog <bh@intevation.de> parents: 91 diff changeset	289 deb_fullname, packager_class="treepkg.packager",
73c67372c7f7 Make the prefix used in the debian revision number configurable. Bernhard Herzog <bh@intevation.de> parents: 91 diff changeset	290 debrevision_prefix="treepkg"):
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	291 self.name = name
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	292 self.base_dir = base_dir
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	293 self.svn_url = svn_url
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	294 self.root_cmd = root_cmd
47 2802be410156 add config options pbuilderrc and use it when calling pbuilder Bernhard Herzog <bh@intevation.de> parents: 16 diff changeset	295 self.pbuilderrc = pbuilderrc
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	296 self.deb_email = deb_email
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	297 self.deb_fullname = deb_fullname
93 73c67372c7f7 Make the prefix used in the debian revision number configurable. Bernhard Herzog <bh@intevation.de> parents: 91 diff changeset	298 self.debrevision_prefix = debrevision_prefix
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	299 self.pkg_dir_template = "%(revision)d-%(increment)d"
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	300 self.pkg_dir_regex \
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	301 = re.compile(r"(?P<revision>[0-9]+)-(?P<increment>[0-9]+)$")
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	302
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	303 checkout_dir = _filenameproperty("checkout")
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	304 debian_dir = _filenameproperty("debian")
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	305 pkg_dir = _filenameproperty("pkg")
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	306
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	307 def pkg_dir_for_revision(self, revision, increment):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	308 return os.path.join(self.pkg_dir,
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	309 self.pkg_dir_template % locals())
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	310
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	311 def last_changed_revision(self):
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	312 revisions = []
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	313 for directory in [self.checkout_dir] + self.svn_external_subdirs:
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	314 directory = os.path.join(self.checkout_dir, directory)
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	315 revisions.append(subversion.last_changed_revision(directory))
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	316 return max(revisions)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	317
11 6efe0bd3d8c1 Add method RevisionPackager.get_revision_numbers Bernhard Herzog <bh@intevation.de> parents: 10 diff changeset	318 def get_revision_numbers(self):
6efe0bd3d8c1 Add method RevisionPackager.get_revision_numbers Bernhard Herzog <bh@intevation.de> parents: 10 diff changeset	319 """Returns a list of the numbers of the packaged revisions"""
6efe0bd3d8c1 Add method RevisionPackager.get_revision_numbers Bernhard Herzog <bh@intevation.de> parents: 10 diff changeset	320 revisions = []
6efe0bd3d8c1 Add method RevisionPackager.get_revision_numbers Bernhard Herzog <bh@intevation.de> parents: 10 diff changeset	321 if os.path.exists(self.pkg_dir):
6efe0bd3d8c1 Add method RevisionPackager.get_revision_numbers Bernhard Herzog <bh@intevation.de> parents: 10 diff changeset	322 for filename in os.listdir(self.pkg_dir):
6efe0bd3d8c1 Add method RevisionPackager.get_revision_numbers Bernhard Herzog <bh@intevation.de> parents: 10 diff changeset	323 match = self.pkg_dir_regex.match(filename)
6efe0bd3d8c1 Add method RevisionPackager.get_revision_numbers Bernhard Herzog <bh@intevation.de> parents: 10 diff changeset	324 if match:
6efe0bd3d8c1 Add method RevisionPackager.get_revision_numbers Bernhard Herzog <bh@intevation.de> parents: 10 diff changeset	325 revisions.append(int(match.group("revision")))
85 31b0567df051 Make PackageTrack.get_revision_numbers return the revisions as a sorted Bernhard Herzog <bh@intevation.de> parents: 84 diff changeset	326 revisions.sort()
11 6efe0bd3d8c1 Add method RevisionPackager.get_revision_numbers Bernhard Herzog <bh@intevation.de> parents: 10 diff changeset	327 return revisions
6efe0bd3d8c1 Add method RevisionPackager.get_revision_numbers Bernhard Herzog <bh@intevation.de> parents: 10 diff changeset	328
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	329 def last_packaged_revision(self):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	330 """Returns the revision number of the highest packaged revision.
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	331
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	332 If the revision cannot be determined because no already packaged
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	333 revisions can be found, the function returns -1.
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	334 """
11 6efe0bd3d8c1 Add method RevisionPackager.get_revision_numbers Bernhard Herzog <bh@intevation.de> parents: 10 diff changeset	335 return max([-1] + self.get_revision_numbers())
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	336
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	337 def debian_source(self):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	338 return util.extract_value_for_key(open(os.path.join(self.debian_dir,
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	339 "control")),
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	340 "Source:")
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	341
80 0af2fa3790e0 Add revision parameter to PackagerGroup.__init__ and some other methods Bernhard Herzog <bh@intevation.de> parents: 78 diff changeset	342 def update_checkout(self, revision=None):
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	343 """Updates the working copy of self.svn_url in self.checkout_dir.
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	344
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	345 If self.checkout_dir doesn't exist yet, self.svn_url is checked
80 0af2fa3790e0 Add revision parameter to PackagerGroup.__init__ and some other methods Bernhard Herzog <bh@intevation.de> parents: 78 diff changeset	346 out into that directory. The value of the revision parameter is
0af2fa3790e0 Add revision parameter to PackagerGroup.__init__ and some other methods Bernhard Herzog <bh@intevation.de> parents: 78 diff changeset	347 passed through to subversion.update.
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	348 """
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	349 localdir = self.checkout_dir
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	350 if os.path.exists(localdir):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	351 logging.info("Updating the working copy in %r", localdir)
80 0af2fa3790e0 Add revision parameter to PackagerGroup.__init__ and some other methods Bernhard Herzog <bh@intevation.de> parents: 78 diff changeset	352 subversion.update(localdir, revision=revision)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	353 else:
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	354 logging.info("The working copy in %r doesn't exist yet."
26 06fba656dde8 fix typo Bernhard Herzog <bh@intevation.de> parents: 18 diff changeset	355 " Checking out from %r", localdir,
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	356 self.svn_url)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	357 subversion.checkout(self.svn_url, localdir)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	358
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	359 def export_sources(self, to_dir):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	360 logging.info("Exporting sources for tarball to %r", to_dir)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	361 subversion.export(self.checkout_dir, to_dir)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	362 # some versions of svn (notably version 1.4.2 shipped with etch)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	363 # do export externals such as the admin subdirectory. We may
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	364 # have to do that in an extra step.
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	365 for subdir in self.svn_external_subdirs:
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	366 absdir = os.path.join(to_dir, subdir)
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	367 if not os.path.isdir(absdir):
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	368 subversion.export(os.path.join(self.checkout_dir, subdir),
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	369 absdir)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	370
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	371 def copy_debian_directory(self, to_dir):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	372 logging.info("Copying debian directory to %r", to_dir)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	373 shutil.copytree(self.debian_dir, to_dir)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	374
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	375 def debian_environment(self):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	376 """Returns the environment variables for the debian commands"""
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	377 env = os.environ.copy()
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	378 env["DEBFULLNAME"] = self.deb_fullname
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	379 env["DEBEMAIL"] = self.deb_email
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	380 return env
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	381
80 0af2fa3790e0 Add revision parameter to PackagerGroup.__init__ and some other methods Bernhard Herzog <bh@intevation.de> parents: 78 diff changeset	382 def package_if_updated(self, revision=None):
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	383 """Checks if the checkout changed and returns a new packager if so"""
80 0af2fa3790e0 Add revision parameter to PackagerGroup.__init__ and some other methods Bernhard Herzog <bh@intevation.de> parents: 78 diff changeset	384 self.update_checkout(revision=revision)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	385 current_revision = self.last_changed_revision()
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	386 logging.info("New revision is %d", current_revision)
98 f7b9c7113c46 Make packaging a specific revision work even if newer revisions have already Bernhard Herzog <bh@intevation.de> parents: 93 diff changeset	387 if current_revision not in self.get_revision_numbers():
f7b9c7113c46 Make packaging a specific revision work even if newer revisions have already Bernhard Herzog <bh@intevation.de> parents: 93 diff changeset	388 logging.info("Revision %d has not been packaged yet",
f7b9c7113c46 Make packaging a specific revision work even if newer revisions have already Bernhard Herzog <bh@intevation.de> parents: 93 diff changeset	389 current_revision)
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	390 return self.revision_packager_cls(self, current_revision)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	391 else:
98 f7b9c7113c46 Make packaging a specific revision work even if newer revisions have already Bernhard Herzog <bh@intevation.de> parents: 93 diff changeset	392 logging.info("Revision %d has already been packaged.",
f7b9c7113c46 Make packaging a specific revision work even if newer revisions have already Bernhard Herzog <bh@intevation.de> parents: 93 diff changeset	393 current_revision)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	394
14 dfd89f81e66c Add simple status report tool. Still using the old status files Bernhard Herzog <bh@intevation.de> parents: 11 diff changeset	395 def get_revisions(self):
dfd89f81e66c Add simple status report tool. Still using the old status files Bernhard Herzog <bh@intevation.de> parents: 11 diff changeset	396 """Returns RevisionPackager objects for each packaged revision"""
dfd89f81e66c Add simple status report tool. Still using the old status files Bernhard Herzog <bh@intevation.de> parents: 11 diff changeset	397 return [self.revision_packager_cls(self, revision)
dfd89f81e66c Add simple status report tool. Still using the old status files Bernhard Herzog <bh@intevation.de> parents: 11 diff changeset	398 for revision in self.get_revision_numbers()]
dfd89f81e66c Add simple status report tool. Still using the old status files Bernhard Herzog <bh@intevation.de> parents: 11 diff changeset	399
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	400
52 78cf5f6778ec Rename 'packagel ine' -> 'package track' Bernhard Herzog <bh@intevation.de> parents: 49 diff changeset	401 def create_package_track(packager_class, **kw):
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	402 module = util.import_dotted_name(packager_class)
52 78cf5f6778ec Rename 'packagel ine' -> 'package track' Bernhard Herzog <bh@intevation.de> parents: 49 diff changeset	403 return module.PackageTrack(**kw)
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	404
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	405
7 96f4f58c62b5 Rename the Packager class to PackagerGroup Bernhard Herzog <bh@intevation.de> parents: 4 diff changeset	406 class PackagerGroup(object):
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	407
91 3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	408 def __init__(self, package_tracks, check_interval, revision=None,
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	409 instructions_file=None):
52 78cf5f6778ec Rename 'packagel ine' -> 'package track' Bernhard Herzog <bh@intevation.de> parents: 49 diff changeset	410 self.package_tracks = package_tracks
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	411 self.check_interval = check_interval
80 0af2fa3790e0 Add revision parameter to PackagerGroup.__init__ and some other methods Bernhard Herzog <bh@intevation.de> parents: 78 diff changeset	412 self.revision = revision
91 3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	413 self.instructions_file = instructions_file
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	414 self.instructions_file_removed = False
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	415
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	416 def run(self):
7 96f4f58c62b5 Rename the Packager class to PackagerGroup Bernhard Herzog <bh@intevation.de> parents: 4 diff changeset	417 """Runs the packager group indefinitely"""
78 9a602d8eaa60 initial revision of the subversion repository Thomas Arendsen Hein <thomas@intevation.de> parents: 55 diff changeset	418 logging.info("Starting in periodic check mode."
9a602d8eaa60 initial revision of the subversion repository Thomas Arendsen Hein <thomas@intevation.de> parents: 55 diff changeset	419 " Will check every %d seconds", self.check_interval)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	420 last_check = -1
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	421 while 1:
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	422 now = time.time()
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	423 if now > last_check + self.check_interval:
91 3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	424 if self.check_package_tracks():
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	425 break
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	426 last_check = now
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	427 next_check = now + self.check_interval
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	428 to_sleep = next_check - time.time()
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	429 if to_sleep > 0:
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	430 logging.info("Next check at %s",
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	431 time.strftime("%Y-%m-%d %H:%M:%S",
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	432 time.localtime(next_check)))
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	433 time.sleep(to_sleep)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	434 else:
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	435 logging.info("Next check now")
91 3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	436 if self.should_stop():
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	437 logging.info("Received stop instruction. Stopping.")
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	438 return
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	439
52 78cf5f6778ec Rename 'packagel ine' -> 'package track' Bernhard Herzog <bh@intevation.de> parents: 49 diff changeset	440 def check_package_tracks(self):
78cf5f6778ec Rename 'packagel ine' -> 'package track' Bernhard Herzog <bh@intevation.de> parents: 49 diff changeset	441 logging.info("Checking package tracks")
91 3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	442 self.clear_instruction()
52 78cf5f6778ec Rename 'packagel ine' -> 'package track' Bernhard Herzog <bh@intevation.de> parents: 49 diff changeset	443 for track in self.package_tracks:
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	444 try:
80 0af2fa3790e0 Add revision parameter to PackagerGroup.__init__ and some other methods Bernhard Herzog <bh@intevation.de> parents: 78 diff changeset	445 packager = track.package_if_updated(revision=self.revision)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	446 if packager:
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	447 packager.package()
91 3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	448 if self.should_stop():
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	449 logging.info("Received stop instruction. Stopping.")
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	450 return True
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	451 except:
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	452 logging.exception("An error occurred while"
52 78cf5f6778ec Rename 'packagel ine' -> 'package track' Bernhard Herzog <bh@intevation.de> parents: 49 diff changeset	453 " checking packager track %r", track.name)
78cf5f6778ec Rename 'packagel ine' -> 'package track' Bernhard Herzog <bh@intevation.de> parents: 49 diff changeset	454 logging.info("Checked all package tracks")
14 dfd89f81e66c Add simple status report tool. Still using the old status files Bernhard Herzog <bh@intevation.de> parents: 11 diff changeset	455
52 78cf5f6778ec Rename 'packagel ine' -> 'package track' Bernhard Herzog <bh@intevation.de> parents: 49 diff changeset	456 def get_package_tracks(self):
78cf5f6778ec Rename 'packagel ine' -> 'package track' Bernhard Herzog <bh@intevation.de> parents: 49 diff changeset	457 return self.package_tracks
91 3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	458
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	459 def read_instruction(self):
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	460 if not self.instructions_file:
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	461 return ""
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	462 try:
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	463 f = open(self.instructions_file)
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	464 except (IOError, OSError):
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	465 return ""
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	466 try:
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	467 return f.read().strip()
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	468 finally:
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	469 f.close()
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	470 self.clear_instruction()
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	471
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	472 def clear_instruction(self, force=False):
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	473 if self.instructions_file and (not self.instructions_file_removed
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	474 or force):
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	475 util.writefile(self.instructions_file, "")
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	476 self.instructions_file_removed = True
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	477
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	478 def should_stop(self):
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	479 return self.read_instruction() == "stop"

Mercurial > treepkg > treepkg

annotate treepkg/packager.py @ 99:7888fe374e11