treepkg/treepkg: treepkg/packager.py annotate

annotate treepkg/packager.py @ 462:058856954e2d

Make the compress_all_logs call builder independent and also compress if an error occurs before the builders are executed.

author	Andre Heinecke <aheinecke@intevation.de>
date	Tue, 31 Aug 2010 15:56:46 +0000
parents	5c06e0a0d329
children	5fda6768bef6

rev	line source
222 01c043f13f13 Remove the unused PackageTrack.last_packaged_revision method and Bernhard Herzog <bh@intevation.de> parents: 207 diff changeset	1 # Copyright (C) 2007, 2008, 2009 by Intevation GmbH
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	2 # Authors:
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	3 # Bernhard Herzog <bh@intevation.de>
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	4 #
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	5 # This program is free software under the GPL (>=v2)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	6 # Read the file COPYING coming with the software for details.
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	7
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	8 """Classes to automatically build debian packages from subversion checkouts"""
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	9
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	10 import os
437 48577b11375f be more error prone in listing different files Bjoern Ricks <bricks@intevation.de> parents: 401 diff changeset	11 import os.path
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	12 import time
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	13 import re
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	14 import logging
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	15 import shutil
16 7c55f3879c4d Use the new status class and report start/stop time too Bernhard Herzog <bh@intevation.de> parents: 14 diff changeset	16 import datetime
113 312949e7628d Add a smarter way to load the packager modules: Add the function Bernhard Herzog <bh@intevation.de> parents: 112 diff changeset	17 import new
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	18
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	19 import util
231 7dbf0a3443bb Use treepkg.subversion.ManualWorkingCopy for the debian directory in a Bernhard Herzog <bh@intevation.de> parents: 229 diff changeset	20 from subversion import SvnRepository, SvnWorkingCopy, ManualWorkingCopy
321 092925ff75d7 Added basic Git support, configuration options: Andre Heinecke <aheinecke@intevation.de> parents: 312 diff changeset	21 from git import GitRepository, GitWorkingCopy
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	22 import run
16 7c55f3879c4d Use the new status class and report start/stop time too Bernhard Herzog <bh@intevation.de> parents: 14 diff changeset	23 import status
128 5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	24 import debian
45 3e610233ccfe use cmdexpand when calling subprocesses Bernhard Herzog <bh@intevation.de> parents: 41 diff changeset	25 from cmdexpand import cmdexpand
112 cea98d4e4a6a Abstract the pbuilder calls into the new class treepkg.builder.PBuilder. Bernhard Herzog <bh@intevation.de> parents: 106 diff changeset	26 from builder import PBuilder
378 41226e427823 Added an option to configure the used builder class from the treepkg.cfg. This Andre Heinecke <aheinecke@intevation.de> parents: 372 diff changeset	27 from sbuilder import SbdmockBuilder
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	28
135 e5b4dea52297 Make more of RevisionPackager available to SourcePackager and Bernhard Herzog <bh@intevation.de> parents: 131 diff changeset	29 def _fromparent(attr):
e5b4dea52297 Make more of RevisionPackager available to SourcePackager and Bernhard Herzog <bh@intevation.de> parents: 131 diff changeset	30 """Creates a property that delegates its value to self.parent.<attr>"""
e5b4dea52297 Make more of RevisionPackager available to SourcePackager and Bernhard Herzog <bh@intevation.de> parents: 131 diff changeset	31 def get(self):
e5b4dea52297 Make more of RevisionPackager available to SourcePackager and Bernhard Herzog <bh@intevation.de> parents: 131 diff changeset	32 return getattr(self.parent, attr)
e5b4dea52297 Make more of RevisionPackager available to SourcePackager and Bernhard Herzog <bh@intevation.de> parents: 131 diff changeset	33 return property(get)
e5b4dea52297 Make more of RevisionPackager available to SourcePackager and Bernhard Herzog <bh@intevation.de> parents: 131 diff changeset	34
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	35
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	36 class SourcePackager(object):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	37
299 c32dc72ba979 Turn the SourcePackager class attribute pkg_basename into a per-track Bernhard Herzog <bh@intevation.de> parents: 297 diff changeset	38 pkg_basename = property(lambda self: self.track.pkg_basename)
300 e82fb08781a2 Turn the SourcePackager class attribute changemsg_template into a Bernhard Herzog <bh@intevation.de> parents: 299 diff changeset	39 changemsg_template = property(lambda self:
e82fb08781a2 Turn the SourcePackager class attribute changemsg_template into a Bernhard Herzog <bh@intevation.de> parents: 299 diff changeset	40 self.track.changelog_msg_template)
135 e5b4dea52297 Make more of RevisionPackager available to SourcePackager and Bernhard Herzog <bh@intevation.de> parents: 131 diff changeset	41 track = _fromparent("track")
e5b4dea52297 Make more of RevisionPackager available to SourcePackager and Bernhard Herzog <bh@intevation.de> parents: 131 diff changeset	42 revision = _fromparent("revision")
281 2b9d94f0ccad Pass the package revision as a parameter to the RevisionPackager Bernhard Herzog <bh@intevation.de> parents: 276 diff changeset	43 pkg_revision = _fromparent("pkg_revision")
135 e5b4dea52297 Make more of RevisionPackager available to SourcePackager and Bernhard Herzog <bh@intevation.de> parents: 131 diff changeset	44 status = _fromparent("status")
136 5598014b2a1d Add a log/ subdir for each revision. The filename is available as the Bernhard Herzog <bh@intevation.de> parents: 135 diff changeset	45 log_dir = _fromparent("log_dir")
135 e5b4dea52297 Make more of RevisionPackager available to SourcePackager and Bernhard Herzog <bh@intevation.de> parents: 131 diff changeset	46 work_dir = _fromparent("work_dir")
e5b4dea52297 Make more of RevisionPackager available to SourcePackager and Bernhard Herzog <bh@intevation.de> parents: 131 diff changeset	47 src_dir = _fromparent("src_dir")
e5b4dea52297 Make more of RevisionPackager available to SourcePackager and Bernhard Herzog <bh@intevation.de> parents: 131 diff changeset	48
e5b4dea52297 Make more of RevisionPackager available to SourcePackager and Bernhard Herzog <bh@intevation.de> parents: 131 diff changeset	49 def __init__(self, parent):
e5b4dea52297 Make more of RevisionPackager available to SourcePackager and Bernhard Herzog <bh@intevation.de> parents: 131 diff changeset	50 self.parent = parent
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	51
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	52 def determine_package_version(self, directory):
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	53 """Returns the version number of the new package as a string
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	54
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	55 The directory parameter is the name of the directory containing
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	56 the newly exported sources. The sources were exported with the
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	57 export_sources method.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	58
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	59 The default implementation simply returns the revision converted
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	60 to a string.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	61 """
450 5c06e0a0d329 Enable more variables to be used in the version template. Andre Heinecke <aheinecke@intevation.de> parents: 441 diff changeset	62 revision = self.revision
5c06e0a0d329 Enable more variables to be used in the version template. Andre Heinecke <aheinecke@intevation.de> parents: 441 diff changeset	63 rules_revision = self.parent.rules_revision
5c06e0a0d329 Enable more variables to be used in the version template. Andre Heinecke <aheinecke@intevation.de> parents: 441 diff changeset	64 pkg_revision = self.parent.pkg_revision
5c06e0a0d329 Enable more variables to be used in the version template. Andre Heinecke <aheinecke@intevation.de> parents: 441 diff changeset	65 pkg_date = time.strftime("%Y%m%d", time.localtime())
5c06e0a0d329 Enable more variables to be used in the version template. Andre Heinecke <aheinecke@intevation.de> parents: 441 diff changeset	66 return self.track.version_template % locals()
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	67
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	68 def export_sources(self):
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	69 """Export the sources from the subversion working directory
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	70
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	71 This method first exports the sources to a temporary directory
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	72 and then renames the directory. The new name is of the form
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	73
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	74 <pkg_basename>-<version>
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	75
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	76 Where pkg_basename is the value of self.pkg_basename and version
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	77 is the return value of the determine_package_version() method.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	78 """
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	79 temp_dir = os.path.join(self.work_dir, "temp")
53 74cd21b6400b rename some variables from pkg_track to track Bernhard Herzog <bh@intevation.de> parents: 52 diff changeset	80 self.track.export_sources(temp_dir)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	81
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	82 pkgbaseversion = self.determine_package_version(temp_dir)
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	83 pkgbasedir = os.path.join(self.work_dir,
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	84 self.pkg_basename + "-" + pkgbaseversion)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	85
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	86 os.rename(temp_dir, pkgbasedir)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	87 return pkgbaseversion, pkgbasedir
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	88
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	89
276 f3dee156e3e3 Add pkgbaseversion parameter to the prepare_sources_for_tarball method Bernhard Herzog <bh@intevation.de> parents: 274 diff changeset	90 def prepare_sources_for_tarball(self, pkgbasedir, pkgbaseversion):
274 2676abfc0e1d Refactoring: Implement do_package in treepkg.packager.SourcePackager. Bernhard Herzog <bh@intevation.de> parents: 261 diff changeset	91 """Prepare the exported sources prior to creating the tarball.
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	92
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	93 The default implementation does nothing. Derived classes should
274 2676abfc0e1d Refactoring: Implement do_package in treepkg.packager.SourcePackager. Bernhard Herzog <bh@intevation.de> parents: 261 diff changeset	94 override this method if necessary to e.g. update the version
2676abfc0e1d Refactoring: Implement do_package in treepkg.packager.SourcePackager. Bernhard Herzog <bh@intevation.de> parents: 261 diff changeset	95 numbers in the code.
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	96 """
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	97
207 b82fe4d25893 SourcePackager.create_tarball: add parameter to specify compression method Bernhard Herzog <bh@intevation.de> parents: 202 diff changeset	98 def create_tarball(self, tarballname, workdir, basedir, compression="gz"):
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	99 """Creates a new tarball.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	100
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	101 Parameters:
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	102
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	103 tarballname -- the filename of the new tarball
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	104 workdir -- The directory into which to change before running tar.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	105 (actually this is done with GNUI tar's -C option)
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	106 basedir -- The basedirectory of the files that are packaged
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	107 into the tarfile. This should be a relative
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	108 filename directly in workdir.
207 b82fe4d25893 SourcePackager.create_tarball: add parameter to specify compression method Bernhard Herzog <bh@intevation.de> parents: 202 diff changeset	109 compression -- The compression method to use as a string.
b82fe4d25893 SourcePackager.create_tarball: add parameter to specify compression method Bernhard Herzog <bh@intevation.de> parents: 202 diff changeset	110 Suported are 'gz' for gzip compression (the
b82fe4d25893 SourcePackager.create_tarball: add parameter to specify compression method Bernhard Herzog <bh@intevation.de> parents: 202 diff changeset	111 default) and 'bz2' for bzip2.
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	112 """
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	113 logging.info("Creating tarball %r", tarballname)
207 b82fe4d25893 SourcePackager.create_tarball: add parameter to specify compression method Bernhard Herzog <bh@intevation.de> parents: 202 diff changeset	114 if compression == "gz":
b82fe4d25893 SourcePackager.create_tarball: add parameter to specify compression method Bernhard Herzog <bh@intevation.de> parents: 202 diff changeset	115 compression_flag = "z"
b82fe4d25893 SourcePackager.create_tarball: add parameter to specify compression method Bernhard Herzog <bh@intevation.de> parents: 202 diff changeset	116 elif compression == "bz2":
b82fe4d25893 SourcePackager.create_tarball: add parameter to specify compression method Bernhard Herzog <bh@intevation.de> parents: 202 diff changeset	117 compression_flag = "j"
b82fe4d25893 SourcePackager.create_tarball: add parameter to specify compression method Bernhard Herzog <bh@intevation.de> parents: 202 diff changeset	118 else:
b82fe4d25893 SourcePackager.create_tarball: add parameter to specify compression method Bernhard Herzog <bh@intevation.de> parents: 202 diff changeset	119 raise ValueError("Unknown compression method %r" % compression)
b82fe4d25893 SourcePackager.create_tarball: add parameter to specify compression method Bernhard Herzog <bh@intevation.de> parents: 202 diff changeset	120
b82fe4d25893 SourcePackager.create_tarball: add parameter to specify compression method Bernhard Herzog <bh@intevation.de> parents: 202 diff changeset	121 run.call(cmdexpand("tar c -$compression_flag -f $tarballname"
b82fe4d25893 SourcePackager.create_tarball: add parameter to specify compression method Bernhard Herzog <bh@intevation.de> parents: 202 diff changeset	122 " -C $workdir $basedir", **locals()))
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	123
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	124 def copy_debian_directory(self, pkgbasedir, pkgbaseversion, changemsg):
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	125 """Copies the debian directory and updates the copy's changelog
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	126
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	127 Parameter:
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	128 pkgbasedir -- The directory holding the unpacked source package
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	129 pkgbaseversion -- The version to update the changelog to
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	130 changemsg -- The message for the changelog
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	131
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	132 When determining the actual version for the new package, this
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	133 function looks at the previous version in the changelog. If it
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	134 has a prefix separated from the version number by a colon this
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	135 prefix is prepended to the pkgbaseversion parameter. Debian
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	136 uses such prefixes for the kde packages.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	137 """
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	138 debian_dir = os.path.join(pkgbasedir, "debian")
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	139 changelog = os.path.join(debian_dir, "changelog")
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	140
53 74cd21b6400b rename some variables from pkg_track to track Bernhard Herzog <bh@intevation.de> parents: 52 diff changeset	141 self.track.copy_debian_directory(debian_dir)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	142
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	143 logging.info("Updating %r", changelog)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	144 oldversion = util.debian_changelog_version(changelog)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	145 if ":" in oldversion:
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	146 oldversionprefix = oldversion.split(":")[0] + ":"
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	147 else:
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	148 oldversionprefix = ""
281 2b9d94f0ccad Pass the package revision as a parameter to the RevisionPackager Bernhard Herzog <bh@intevation.de> parents: 276 diff changeset	149 debrev = self.pkg_revision
45 3e610233ccfe use cmdexpand when calling subprocesses Bernhard Herzog <bh@intevation.de> parents: 41 diff changeset	150 run.call(cmdexpand("debchange -c $changelog"
93 73c67372c7f7 Make the prefix used in the debian revision number configurable. Bernhard Herzog <bh@intevation.de> parents: 91 diff changeset	151 " -v ${oldversionprefix}${pkgbaseversion}-${debrev}"
45 3e610233ccfe use cmdexpand when calling subprocesses Bernhard Herzog <bh@intevation.de> parents: 41 diff changeset	152 " $changemsg", **locals()),
53 74cd21b6400b rename some variables from pkg_track to track Bernhard Herzog <bh@intevation.de> parents: 52 diff changeset	153 env=self.track.debian_environment())
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	154
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	155
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	156 def create_source_package(self, pkgbasedir, origtargz):
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	157 """Creates a new source package from pkgbasedir and origtargz"""
194 86360f3d5611 Log the output of the dpkg-source call to dpkg_source.txt. Bernhard Herzog <bh@intevation.de> parents: 191 diff changeset	158 util.ensure_directory(self.log_dir)
86360f3d5611 Log the output of the dpkg-source call to dpkg_source.txt. Bernhard Herzog <bh@intevation.de> parents: 191 diff changeset	159 dpkg_source_log = os.path.join(self.log_dir, "dpkg_source.txt")
86360f3d5611 Log the output of the dpkg-source call to dpkg_source.txt. Bernhard Herzog <bh@intevation.de> parents: 191 diff changeset	160 logging.info("Creating new source package; logging to %s",
86360f3d5611 Log the output of the dpkg-source call to dpkg_source.txt. Bernhard Herzog <bh@intevation.de> parents: 191 diff changeset	161 dpkg_source_log)
367 fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	162
fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	163 format = self.get_debian_source_format(pkgbasedir)
fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	164 if format == "1.0":
fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	165 run.call(cmdexpand("dpkg-source -b $directory $tarball",
45 3e610233ccfe use cmdexpand when calling subprocesses Bernhard Herzog <bh@intevation.de> parents: 41 diff changeset	166 directory=os.path.basename(pkgbasedir),
3e610233ccfe use cmdexpand when calling subprocesses Bernhard Herzog <bh@intevation.de> parents: 41 diff changeset	167 tarball=os.path.basename(origtargz)),
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	168 cwd=os.path.dirname(pkgbasedir),
194 86360f3d5611 Log the output of the dpkg-source call to dpkg_source.txt. Bernhard Herzog <bh@intevation.de> parents: 191 diff changeset	169 logfile=dpkg_source_log,
53 74cd21b6400b rename some variables from pkg_track to track Bernhard Herzog <bh@intevation.de> parents: 52 diff changeset	170 env=self.track.debian_environment())
367 fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	171 elif format == "3.0 (quilt)":
fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	172 run.call(cmdexpand("dpkg-source -b $directory",
fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	173 directory=os.path.basename(pkgbasedir)),
fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	174 cwd=os.path.dirname(pkgbasedir),
fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	175 logfile=dpkg_source_log,
fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	176 env=self.track.debian_environment())
fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	177 else:
fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	178 raise RuntimeError("debian source format %s is not supported by treepkg" % format)
fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	179
fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	180 def get_debian_source_format(self, pkgbasedir):
fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	181 formatfile = os.path.join(pkgbasedir, "debian", "source", "format")
fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	182 if not os.path.exists(formatfile):
fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	183 return "1.0"
fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	184 else:
fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	185 file = open(formatfile, "r")
fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	186 line = file.readline()
fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	187 file.close()
fb92f3c1b238 treepkg is now able to handle debian source format 1.0 and source format 3.0 (quilt) Bjoern Ricks <bricks@intevation.de> parents: 344 diff changeset	188 return line.strip()
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	189
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	190 def move_source_package(self, pkgbasename):
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	191 """Moves the new source package from the work_dir to the src_dir"""
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	192 logging.info("Moving source package to %r", self.src_dir)
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	193 util.ensure_directory(self.src_dir)
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	194 for filename in [filename for filename in os.listdir(self.work_dir)
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	195 if filename.startswith(pkgbasename)]:
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	196 os.rename(os.path.join(self.work_dir, filename),
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	197 os.path.join(self.src_dir, filename))
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	198
179 952d366f7b14 Implement optional signing of .dsc and .changes files Bernhard Herzog <bh@intevation.de> parents: 176 diff changeset	199 def sign_package(self):
952d366f7b14 Implement optional signing of .dsc and .changes files Bernhard Herzog <bh@intevation.de> parents: 176 diff changeset	200 """Signs the .dsc file created buy the instance"""
344 f06f707d9fda merged branches/scratchbox into trunk Bjoern Ricks <bricks@intevation.de> parents: 340 diff changeset	201 src_files = util.listdir_abs(self.src_dir, "*.dsc")
f06f707d9fda merged branches/scratchbox into trunk Bjoern Ricks <bricks@intevation.de> parents: 340 diff changeset	202 if not src_files:
f06f707d9fda merged branches/scratchbox into trunk Bjoern Ricks <bricks@intevation.de> parents: 340 diff changeset	203 raise RuntimeError("Could not find .dsc file in source"
f06f707d9fda merged branches/scratchbox into trunk Bjoern Ricks <bricks@intevation.de> parents: 340 diff changeset	204 " directory %s" % self.src_dir)
f06f707d9fda merged branches/scratchbox into trunk Bjoern Ricks <bricks@intevation.de> parents: 340 diff changeset	205 self.track.sign_file(src_files[0])
179 952d366f7b14 Implement optional signing of .dsc and .changes files Bernhard Herzog <bh@intevation.de> parents: 176 diff changeset	206
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	207 def package(self):
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	208 """Creates a source package from a subversion checkout.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	209
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	210 After setting up the working directory, this method calls the
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	211 do_package method to do the actual packaging. Afterwards the
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	212 work directory is removed.
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	213 """
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	214 util.ensure_directory(self.work_dir)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	215 try:
41 f7ec40638a06 use the enums for the status field of RevisionStatus Bernhard Herzog <bh@intevation.de> parents: 36 diff changeset	216 self.status.creating_source_package()
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	217 self.do_package()
179 952d366f7b14 Implement optional signing of .dsc and .changes files Bernhard Herzog <bh@intevation.de> parents: 176 diff changeset	218 self.sign_package()
41 f7ec40638a06 use the enums for the status field of RevisionStatus Bernhard Herzog <bh@intevation.de> parents: 36 diff changeset	219 self.status.source_package_created()
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	220 finally:
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	221 logging.info("Removing workdir %r", self.work_dir)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	222 shutil.rmtree(self.work_dir)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	223
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	224 def do_package(self):
274 2676abfc0e1d Refactoring: Implement do_package in treepkg.packager.SourcePackager. Bernhard Herzog <bh@intevation.de> parents: 261 diff changeset	225 """Does the work of creating a source package."""
2676abfc0e1d Refactoring: Implement do_package in treepkg.packager.SourcePackager. Bernhard Herzog <bh@intevation.de> parents: 261 diff changeset	226 pkgbaseversion, pkgbasedir = self.export_sources()
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	227
274 2676abfc0e1d Refactoring: Implement do_package in treepkg.packager.SourcePackager. Bernhard Herzog <bh@intevation.de> parents: 261 diff changeset	228 pkgbasename = self.pkg_basename + "_" + pkgbaseversion
2676abfc0e1d Refactoring: Implement do_package in treepkg.packager.SourcePackager. Bernhard Herzog <bh@intevation.de> parents: 261 diff changeset	229 origtargz = os.path.join(self.work_dir,
2676abfc0e1d Refactoring: Implement do_package in treepkg.packager.SourcePackager. Bernhard Herzog <bh@intevation.de> parents: 261 diff changeset	230 pkgbasename + ".orig.tar.gz")
2676abfc0e1d Refactoring: Implement do_package in treepkg.packager.SourcePackager. Bernhard Herzog <bh@intevation.de> parents: 261 diff changeset	231
276 f3dee156e3e3 Add pkgbaseversion parameter to the prepare_sources_for_tarball method Bernhard Herzog <bh@intevation.de> parents: 274 diff changeset	232 self.prepare_sources_for_tarball(pkgbasedir, pkgbaseversion)
274 2676abfc0e1d Refactoring: Implement do_package in treepkg.packager.SourcePackager. Bernhard Herzog <bh@intevation.de> parents: 261 diff changeset	233
2676abfc0e1d Refactoring: Implement do_package in treepkg.packager.SourcePackager. Bernhard Herzog <bh@intevation.de> parents: 261 diff changeset	234 self.create_tarball(origtargz, self.work_dir,
2676abfc0e1d Refactoring: Implement do_package in treepkg.packager.SourcePackager. Bernhard Herzog <bh@intevation.de> parents: 261 diff changeset	235 os.path.basename(pkgbasedir))
2676abfc0e1d Refactoring: Implement do_package in treepkg.packager.SourcePackager. Bernhard Herzog <bh@intevation.de> parents: 261 diff changeset	236
344 f06f707d9fda merged branches/scratchbox into trunk Bjoern Ricks <bricks@intevation.de> parents: 340 diff changeset	237 changemsg = self.get_change_msg()
274 2676abfc0e1d Refactoring: Implement do_package in treepkg.packager.SourcePackager. Bernhard Herzog <bh@intevation.de> parents: 261 diff changeset	238 self.copy_debian_directory(pkgbasedir, pkgbaseversion,
2676abfc0e1d Refactoring: Implement do_package in treepkg.packager.SourcePackager. Bernhard Herzog <bh@intevation.de> parents: 261 diff changeset	239 changemsg)
2676abfc0e1d Refactoring: Implement do_package in treepkg.packager.SourcePackager. Bernhard Herzog <bh@intevation.de> parents: 261 diff changeset	240
2676abfc0e1d Refactoring: Implement do_package in treepkg.packager.SourcePackager. Bernhard Herzog <bh@intevation.de> parents: 261 diff changeset	241 self.create_source_package(pkgbasedir, origtargz)
2676abfc0e1d Refactoring: Implement do_package in treepkg.packager.SourcePackager. Bernhard Herzog <bh@intevation.de> parents: 261 diff changeset	242 self.move_source_package(pkgbasename)
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	243
344 f06f707d9fda merged branches/scratchbox into trunk Bjoern Ricks <bricks@intevation.de> parents: 340 diff changeset	244 def get_change_msg(self):
f06f707d9fda merged branches/scratchbox into trunk Bjoern Ricks <bricks@intevation.de> parents: 340 diff changeset	245 return self.changemsg_template % dict(revision=self.revision)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	246
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	247 class BinaryPackager(object):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	248
135 e5b4dea52297 Make more of RevisionPackager available to SourcePackager and Bernhard Herzog <bh@intevation.de> parents: 131 diff changeset	249 track = _fromparent("track")
e5b4dea52297 Make more of RevisionPackager available to SourcePackager and Bernhard Herzog <bh@intevation.de> parents: 131 diff changeset	250 status = _fromparent("status")
136 5598014b2a1d Add a log/ subdir for each revision. The filename is available as the Bernhard Herzog <bh@intevation.de> parents: 135 diff changeset	251 log_dir = _fromparent("log_dir")
135 e5b4dea52297 Make more of RevisionPackager available to SourcePackager and Bernhard Herzog <bh@intevation.de> parents: 131 diff changeset	252 binary_dir = _fromparent("binary_dir")
e5b4dea52297 Make more of RevisionPackager available to SourcePackager and Bernhard Herzog <bh@intevation.de> parents: 131 diff changeset	253
e5b4dea52297 Make more of RevisionPackager available to SourcePackager and Bernhard Herzog <bh@intevation.de> parents: 131 diff changeset	254 def __init__(self, parent, dsc_file, logfile):
e5b4dea52297 Make more of RevisionPackager available to SourcePackager and Bernhard Herzog <bh@intevation.de> parents: 131 diff changeset	255 self.parent = parent
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	256 self.dsc_file = dsc_file
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	257 self.logfile = logfile
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	258
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	259 def package(self):
41 f7ec40638a06 use the enums for the status field of RevisionStatus Bernhard Herzog <bh@intevation.de> parents: 36 diff changeset	260 self.status.creating_binary_package()
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	261 util.ensure_directory(self.binary_dir)
149 4526105f81a7 Make sure the log dir exists before attempting to run pbuilder build Bernhard Herzog <bh@intevation.de> parents: 143 diff changeset	262 util.ensure_directory(self.log_dir)
54 b0014b8f6029 fix typo in log message Bernhard Herzog <bh@intevation.de> parents: 53 diff changeset	263 logging.info("Building binary package; logging to %r", self.logfile)
297 4dd6ec3a1151 Make it possible to use parallel builds for packages that support it: Bernhard Herzog <bh@intevation.de> parents: 293 diff changeset	264 self.track.builder.build(self.dsc_file, self.binary_dir, self.logfile,
4dd6ec3a1151 Make it possible to use parallel builds for packages that support it: Bernhard Herzog <bh@intevation.de> parents: 293 diff changeset	265 extra_env=self.track.debian_environment())
179 952d366f7b14 Implement optional signing of .dsc and .changes files Bernhard Herzog <bh@intevation.de> parents: 176 diff changeset	266 self.sign_package()
41 f7ec40638a06 use the enums for the status field of RevisionStatus Bernhard Herzog <bh@intevation.de> parents: 36 diff changeset	267 self.status.binary_package_created()
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	268
179 952d366f7b14 Implement optional signing of .dsc and .changes files Bernhard Herzog <bh@intevation.de> parents: 176 diff changeset	269 def sign_package(self):
370 0bee91b435ab Fix typos in doc-strings. Bernhard Herzog <bh@intevation.de> parents: 367 diff changeset	270 """Signs the .changes file created by the instance"""
344 f06f707d9fda merged branches/scratchbox into trunk Bjoern Ricks <bricks@intevation.de> parents: 340 diff changeset	271 dirs = util.listdir_abs(self.binary_dir, "*.changes")
f06f707d9fda merged branches/scratchbox into trunk Bjoern Ricks <bricks@intevation.de> parents: 340 diff changeset	272 if not dirs:
f06f707d9fda merged branches/scratchbox into trunk Bjoern Ricks <bricks@intevation.de> parents: 340 diff changeset	273 raise RuntimeError("Cannot find changes File in %r" % self.binary_dir)
f06f707d9fda merged branches/scratchbox into trunk Bjoern Ricks <bricks@intevation.de> parents: 340 diff changeset	274 self.track.sign_file(dirs[0])
179 952d366f7b14 Implement optional signing of .dsc and .changes files Bernhard Herzog <bh@intevation.de> parents: 176 diff changeset	275
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	276
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	277 class RevisionPackager(object):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	278
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	279 source_packager_cls = SourcePackager
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	280 binary_packager_cls = BinaryPackager
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	281
281 2b9d94f0ccad Pass the package revision as a parameter to the RevisionPackager Bernhard Herzog <bh@intevation.de> parents: 276 diff changeset	282 def __init__(self, track, revision, rules_revision, pkg_revision=None,
2b9d94f0ccad Pass the package revision as a parameter to the RevisionPackager Bernhard Herzog <bh@intevation.de> parents: 276 diff changeset	283 tag=""):
53 74cd21b6400b rename some variables from pkg_track to track Bernhard Herzog <bh@intevation.de> parents: 52 diff changeset	284 self.track = track
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	285 self.revision = revision
229 653a45adda50 Prepare for svn managed debian rules directories. So far, the directory Bernhard Herzog <bh@intevation.de> parents: 224 diff changeset	286 self.rules_revision = rules_revision
281 2b9d94f0ccad Pass the package revision as a parameter to the RevisionPackager Bernhard Herzog <bh@intevation.de> parents: 276 diff changeset	287
2b9d94f0ccad Pass the package revision as a parameter to the RevisionPackager Bernhard Herzog <bh@intevation.de> parents: 276 diff changeset	288 if pkg_revision is None:
293 faeeac2c4c71 Replace debrevision_prefix with pkg_revision_template. Their meaning is Bernhard Herzog <bh@intevation.de> parents: 281 diff changeset	289 pkg_revision = (self.track.pkg_revision_template
338 28df50b267f6 Expanded the pkg_revision_template dictionary to include rules revision Andre Heinecke <aheinecke@intevation.de> parents: 336 diff changeset	290 % dict(pkg_revision=1,
28df50b267f6 Expanded the pkg_revision_template dictionary to include rules revision Andre Heinecke <aheinecke@intevation.de> parents: 336 diff changeset	291 rules_revision=rules_revision))
281 2b9d94f0ccad Pass the package revision as a parameter to the RevisionPackager Bernhard Herzog <bh@intevation.de> parents: 276 diff changeset	292 self.pkg_revision = pkg_revision
2b9d94f0ccad Pass the package revision as a parameter to the RevisionPackager Bernhard Herzog <bh@intevation.de> parents: 276 diff changeset	293
229 653a45adda50 Prepare for svn managed debian rules directories. So far, the directory Bernhard Herzog <bh@intevation.de> parents: 224 diff changeset	294 self.base_dir = self.track.pkg_dir_for_revision(self.revision,
653a45adda50 Prepare for svn managed debian rules directories. So far, the directory Bernhard Herzog <bh@intevation.de> parents: 224 diff changeset	295 rules_revision)
36 086c68ca51d2 rename Status to RevisionStatus Bernhard Herzog <bh@intevation.de> parents: 26 diff changeset	296 self.status = status.RevisionStatus(os.path.join(self.base_dir,
086c68ca51d2 rename Status to RevisionStatus Bernhard Herzog <bh@intevation.de> parents: 26 diff changeset	297 "status"))
260 6aeae11ca7fe Add tag parameter to RevisionPackager constructor. The tag is stored in Bernhard Herzog <bh@intevation.de> parents: 238 diff changeset	298 if tag:
6aeae11ca7fe Add tag parameter to RevisionPackager constructor. The tag is stored in Bernhard Herzog <bh@intevation.de> parents: 238 diff changeset	299 util.ensure_directory(self.base_dir)
6aeae11ca7fe Add tag parameter to RevisionPackager constructor. The tag is stored in Bernhard Herzog <bh@intevation.de> parents: 238 diff changeset	300 self.status.tags = tag
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	301
172 06af36f915f2 Move the filenameproperty factory from treepkg/packager.py to Bernhard Herzog <bh@intevation.de> parents: 168 diff changeset	302 log_dir = util.filenameproperty("log")
06af36f915f2 Move the filenameproperty factory from treepkg/packager.py to Bernhard Herzog <bh@intevation.de> parents: 168 diff changeset	303 work_dir = util.filenameproperty("work")
06af36f915f2 Move the filenameproperty factory from treepkg/packager.py to Bernhard Herzog <bh@intevation.de> parents: 168 diff changeset	304 binary_dir = util.filenameproperty("binary")
06af36f915f2 Move the filenameproperty factory from treepkg/packager.py to Bernhard Herzog <bh@intevation.de> parents: 168 diff changeset	305 src_dir = util.filenameproperty("src")
06af36f915f2 Move the filenameproperty factory from treepkg/packager.py to Bernhard Herzog <bh@intevation.de> parents: 168 diff changeset	306 build_log = util.filenameproperty("build_log.txt", dir_attr="log_dir")
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	307
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	308 def find_dsc_file(self):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	309 for filename in os.listdir(self.src_dir):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	310 if filename.endswith(".dsc"):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	311 return os.path.join(self.src_dir, filename)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	312 return None
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	313
18 d5c24cfce05e Improve access to a RevisionPackager's build_log Bernhard Herzog <bh@intevation.de> parents: 16 diff changeset	314 def has_build_log(self):
336 5fa56edeb606 Changed the Build Logs to be gziped before they are published Andre Heinecke <aheinecke@intevation.de> parents: 328 diff changeset	315 return os.path.exists(self.get_log_file())
5fa56edeb606 Changed the Build Logs to be gziped before they are published Andre Heinecke <aheinecke@intevation.de> parents: 328 diff changeset	316
372 ef87d30468b6 Added the option to expose additional log files from the log directory. Andre Heinecke <aheinecke@intevation.de> parents: 370 diff changeset	317 def get_log_title(self, f):
ef87d30468b6 Added the option to expose additional log files from the log directory. Andre Heinecke <aheinecke@intevation.de> parents: 370 diff changeset	318 if not os.path.isfile(f):
ef87d30468b6 Added the option to expose additional log files from the log directory. Andre Heinecke <aheinecke@intevation.de> parents: 370 diff changeset	319 return None
ef87d30468b6 Added the option to expose additional log files from the log directory. Andre Heinecke <aheinecke@intevation.de> parents: 370 diff changeset	320 title = os.path.basename(f)
ef87d30468b6 Added the option to expose additional log files from the log directory. Andre Heinecke <aheinecke@intevation.de> parents: 370 diff changeset	321 title = title.replace("_"," ")
ef87d30468b6 Added the option to expose additional log files from the log directory. Andre Heinecke <aheinecke@intevation.de> parents: 370 diff changeset	322 title = title[:title.find(".")]
ef87d30468b6 Added the option to expose additional log files from the log directory. Andre Heinecke <aheinecke@intevation.de> parents: 370 diff changeset	323 title = title.title()
ef87d30468b6 Added the option to expose additional log files from the log directory. Andre Heinecke <aheinecke@intevation.de> parents: 370 diff changeset	324 return title
ef87d30468b6 Added the option to expose additional log files from the log directory. Andre Heinecke <aheinecke@intevation.de> parents: 370 diff changeset	325
336 5fa56edeb606 Changed the Build Logs to be gziped before they are published Andre Heinecke <aheinecke@intevation.de> parents: 328 diff changeset	326 def get_log_file(self):
5fa56edeb606 Changed the Build Logs to be gziped before they are published Andre Heinecke <aheinecke@intevation.de> parents: 328 diff changeset	327 if os.path.exists(self.build_log + ".gz"):
5fa56edeb606 Changed the Build Logs to be gziped before they are published Andre Heinecke <aheinecke@intevation.de> parents: 328 diff changeset	328 return self.build_log + ".gz"
5fa56edeb606 Changed the Build Logs to be gziped before they are published Andre Heinecke <aheinecke@intevation.de> parents: 328 diff changeset	329 return self.build_log
18 d5c24cfce05e Improve access to a RevisionPackager's build_log Bernhard Herzog <bh@intevation.de> parents: 16 diff changeset	330
393 5fe26e7f6e2d get all log files Bjoern Ricks <bricks@intevation.de> parents: 389 diff changeset	331 def get_log_files(self, logs=None):
372 ef87d30468b6 Added the option to expose additional log files from the log directory. Andre Heinecke <aheinecke@intevation.de> parents: 370 diff changeset	332 files = []
437 48577b11375f be more error prone in listing different files Bjoern Ricks <bricks@intevation.de> parents: 401 diff changeset	333 if os.path.isdir(self.log_dir):
48577b11375f be more error prone in listing different files Bjoern Ricks <bricks@intevation.de> parents: 401 diff changeset	334 for f in os.listdir(self.log_dir):
48577b11375f be more error prone in listing different files Bjoern Ricks <bricks@intevation.de> parents: 401 diff changeset	335 if logs is None or f in logs:
48577b11375f be more error prone in listing different files Bjoern Ricks <bricks@intevation.de> parents: 401 diff changeset	336 f = os.path.join(self.log_dir,f)
48577b11375f be more error prone in listing different files Bjoern Ricks <bricks@intevation.de> parents: 401 diff changeset	337 if os.path.isfile(f):
48577b11375f be more error prone in listing different files Bjoern Ricks <bricks@intevation.de> parents: 401 diff changeset	338 files.append((self.get_log_title(f),f))
372 ef87d30468b6 Added the option to expose additional log files from the log directory. Andre Heinecke <aheinecke@intevation.de> parents: 370 diff changeset	339 return files
ef87d30468b6 Added the option to expose additional log files from the log directory. Andre Heinecke <aheinecke@intevation.de> parents: 370 diff changeset	340
ef87d30468b6 Added the option to expose additional log files from the log directory. Andre Heinecke <aheinecke@intevation.de> parents: 370 diff changeset	341 def list_log_files(self, logs):
140 0da76aee8035 Add RevisionPackager method list_log_files which returns a description Bernhard Herzog <bh@intevation.de> parents: 139 diff changeset	342 """Returns a list describing the logfiles available for the revision.
0da76aee8035 Add RevisionPackager method list_log_files which returns a description Bernhard Herzog <bh@intevation.de> parents: 139 diff changeset	343 Each list item is a tuple of the form (TITLE, FILENAME) where
372 ef87d30468b6 Added the option to expose additional log files from the log directory. Andre Heinecke <aheinecke@intevation.de> parents: 370 diff changeset	344 TITLE is a string with the filename without directory or ending in
ef87d30468b6 Added the option to expose additional log files from the log directory. Andre Heinecke <aheinecke@intevation.de> parents: 370 diff changeset	345 which _ is replaced with a whitespace and words are capitalized.
140 0da76aee8035 Add RevisionPackager method list_log_files which returns a description Bernhard Herzog <bh@intevation.de> parents: 139 diff changeset	346 FILENAME is the absolute filename of the log file.
0da76aee8035 Add RevisionPackager method list_log_files which returns a description Bernhard Herzog <bh@intevation.de> parents: 139 diff changeset	347 """
372 ef87d30468b6 Added the option to expose additional log files from the log directory. Andre Heinecke <aheinecke@intevation.de> parents: 370 diff changeset	348 files = self.get_log_files(logs)
ef87d30468b6 Added the option to expose additional log files from the log directory. Andre Heinecke <aheinecke@intevation.de> parents: 370 diff changeset	349 if not files:
ef87d30468b6 Added the option to expose additional log files from the log directory. Andre Heinecke <aheinecke@intevation.de> parents: 370 diff changeset	350 return []
140 0da76aee8035 Add RevisionPackager method list_log_files which returns a description Bernhard Herzog <bh@intevation.de> parents: 139 diff changeset	351 return files
0da76aee8035 Add RevisionPackager method list_log_files which returns a description Bernhard Herzog <bh@intevation.de> parents: 139 diff changeset	352
88 3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	353 def list_source_files(self):
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	354 """Returns a list with the names of the files of the source package.
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	355 The implementation assumes that all files in self.src_dir belong
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	356 to the source package.
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	357 """
437 48577b11375f be more error prone in listing different files Bjoern Ricks <bricks@intevation.de> parents: 401 diff changeset	358 files = []
48577b11375f be more error prone in listing different files Bjoern Ricks <bricks@intevation.de> parents: 401 diff changeset	359 if os.path.isdir(self.src_dir):
48577b11375f be more error prone in listing different files Bjoern Ricks <bricks@intevation.de> parents: 401 diff changeset	360 files = sorted(util.listdir_abs(self.src_dir))
48577b11375f be more error prone in listing different files Bjoern Ricks <bricks@intevation.de> parents: 401 diff changeset	361 return files
88 3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	362
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	363 def list_binary_files(self):
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	364 """Returns a list with the names of the files of the binary packages.
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	365 The implementation assumes that all files in self.binary_dir belong
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	366 to the binary packages.
3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	367 """
437 48577b11375f be more error prone in listing different files Bjoern Ricks <bricks@intevation.de> parents: 401 diff changeset	368 files = []
48577b11375f be more error prone in listing different files Bjoern Ricks <bricks@intevation.de> parents: 401 diff changeset	369 if os.path.isdir(self.binary_dir):
48577b11375f be more error prone in listing different files Bjoern Ricks <bricks@intevation.de> parents: 401 diff changeset	370 files = sorted(util.listdir_abs(self.binary_dir))
48577b11375f be more error prone in listing different files Bjoern Ricks <bricks@intevation.de> parents: 401 diff changeset	371 return files
88 3ae54f99db26 Add methods RevisionPackager.list_source_files and Bernhard Herzog <bh@intevation.de> parents: 85 diff changeset	372
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	373 def package(self):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	374 try:
462 058856954e2d Make the compress_all_logs call builder independent and also compress Andre Heinecke <aheinecke@intevation.de> parents: 450 diff changeset	375 try:
058856954e2d Make the compress_all_logs call builder independent and also compress Andre Heinecke <aheinecke@intevation.de> parents: 450 diff changeset	376 util.ensure_directory(self.work_dir)
058856954e2d Make the compress_all_logs call builder independent and also compress Andre Heinecke <aheinecke@intevation.de> parents: 450 diff changeset	377 self.status.start = datetime.datetime.utcnow()
058856954e2d Make the compress_all_logs call builder independent and also compress Andre Heinecke <aheinecke@intevation.de> parents: 450 diff changeset	378 src_packager = self.source_packager_cls(self)
058856954e2d Make the compress_all_logs call builder independent and also compress Andre Heinecke <aheinecke@intevation.de> parents: 450 diff changeset	379 src_packager.package()
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	380
462 058856954e2d Make the compress_all_logs call builder independent and also compress Andre Heinecke <aheinecke@intevation.de> parents: 450 diff changeset	381 dsc_file = self.find_dsc_file()
058856954e2d Make the compress_all_logs call builder independent and also compress Andre Heinecke <aheinecke@intevation.de> parents: 450 diff changeset	382 if dsc_file is None:
058856954e2d Make the compress_all_logs call builder independent and also compress Andre Heinecke <aheinecke@intevation.de> parents: 450 diff changeset	383 raise RuntimeError("Cannot find dsc File in %r" % self.src_dir)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	384
462 058856954e2d Make the compress_all_logs call builder independent and also compress Andre Heinecke <aheinecke@intevation.de> parents: 450 diff changeset	385 bin_packager = self.binary_packager_cls(self, dsc_file, self.build_log)
058856954e2d Make the compress_all_logs call builder independent and also compress Andre Heinecke <aheinecke@intevation.de> parents: 450 diff changeset	386 bin_packager.package()
058856954e2d Make the compress_all_logs call builder independent and also compress Andre Heinecke <aheinecke@intevation.de> parents: 450 diff changeset	387 self.status.stop = datetime.datetime.utcnow()
058856954e2d Make the compress_all_logs call builder independent and also compress Andre Heinecke <aheinecke@intevation.de> parents: 450 diff changeset	388 finally:
058856954e2d Make the compress_all_logs call builder independent and also compress Andre Heinecke <aheinecke@intevation.de> parents: 450 diff changeset	389 util.compress_all_logs(self.log_dir)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	390 except:
41 f7ec40638a06 use the enums for the status field of RevisionStatus Bernhard Herzog <bh@intevation.de> parents: 36 diff changeset	391 self.status.error()
16 7c55f3879c4d Use the new status class and report start/stop time too Bernhard Herzog <bh@intevation.de> parents: 14 diff changeset	392 self.status.stop = datetime.datetime.utcnow()
99 7888fe374e11 Add support for notification mails in case of build errors Bernhard Herzog <bh@intevation.de> parents: 98 diff changeset	393 # set the notification status last to avoid race conditions.
7888fe374e11 Add support for notification mails in case of build errors Bernhard Herzog <bh@intevation.de> parents: 98 diff changeset	394 # The pending notification is for now the only situation
7888fe374e11 Add support for notification mails in case of build errors Bernhard Herzog <bh@intevation.de> parents: 98 diff changeset	395 # where another process might modify the status file (the
7888fe374e11 Add support for notification mails in case of build errors Bernhard Herzog <bh@intevation.de> parents: 98 diff changeset	396 # listpendingnotifications program will set it to
7888fe374e11 Add support for notification mails in case of build errors Bernhard Herzog <bh@intevation.de> parents: 98 diff changeset	397 # "notification_sent")
7888fe374e11 Add support for notification mails in case of build errors Bernhard Herzog <bh@intevation.de> parents: 98 diff changeset	398 self.status.notification_pending()
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	399 raise
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	400
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	401 def remove_package_dir(self):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	402 logging.info("Removing pkgdir %r", self.base_dir)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	403 shutil.rmtree(self.base_dir)
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	404
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	405
52 78cf5f6778ec Rename 'packagel ine' -> 'package track' Bernhard Herzog <bh@intevation.de> parents: 49 diff changeset	406 class PackageTrack(object):
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	407
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	408 revision_packager_cls = RevisionPackager
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	409
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	410 svn_external_subdirs = []
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	411
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	412 extra_config_desc = []
fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	413
344 f06f707d9fda merged branches/scratchbox into trunk Bjoern Ricks <bricks@intevation.de> parents: 340 diff changeset	414 def __init__(self, name, base_dir, root_cmd, builderconfig, deb_email,
321 092925ff75d7 Added basic Git support, configuration options: Andre Heinecke <aheinecke@intevation.de> parents: 312 diff changeset	415 deb_fullname, svn_url="", packager_class="treepkg.packager",
378 41226e427823 Added an option to configure the used builder class from the treepkg.cfg. This Andre Heinecke <aheinecke@intevation.de> parents: 372 diff changeset	416 version_template="%(revision)s", builder_cls="PBuilder",
340 a83a78a93fd6 Changed default message templates to the correct types Andre Heinecke <aheinecke@intevation.de> parents: 338 diff changeset	417 pkg_revision_template="treepkg%(pkg_revision)s",
293 faeeac2c4c71 Replace debrevision_prefix with pkg_revision_template. Their meaning is Bernhard Herzog <bh@intevation.de> parents: 281 diff changeset	418 handle_dependencies=False, signing_key_id="", do_build=True,
300 e82fb08781a2 Turn the SourcePackager class attribute changemsg_template into a Bernhard Herzog <bh@intevation.de> parents: 299 diff changeset	419 rules_svn_url=None, deb_build_options="", pkg_basename="",
304 6cffb43a28ca Add a way to specify svn subset checkouts where only parts of a source Bernhard Herzog <bh@intevation.de> parents: 300 diff changeset	420 changelog_msg_template="Update to r%(revision)s",
401 2db42a2a9db9 add os config statement Bjoern Ricks <bricks@intevation.de> parents: 393 diff changeset	421 svn_subset=(), svn_externals=(), git_branch="", git_url="",
2db42a2a9db9 add os config statement Bjoern Ricks <bricks@intevation.de> parents: 393 diff changeset	422 os=""):
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	423 self.name = name
378 41226e427823 Added an option to configure the used builder class from the treepkg.cfg. This Andre Heinecke <aheinecke@intevation.de> parents: 372 diff changeset	424
41226e427823 Added an option to configure the used builder class from the treepkg.cfg. This Andre Heinecke <aheinecke@intevation.de> parents: 372 diff changeset	425 # Convert the builder_cls option to a class
41226e427823 Added an option to configure the used builder class from the treepkg.cfg. This Andre Heinecke <aheinecke@intevation.de> parents: 372 diff changeset	426 if builder_cls.upper() == "SBDMOCKBUILDER" or \
41226e427823 Added an option to configure the used builder class from the treepkg.cfg. This Andre Heinecke <aheinecke@intevation.de> parents: 372 diff changeset	427 builder_cls.upper() == "SBDMOCK":
41226e427823 Added an option to configure the used builder class from the treepkg.cfg. This Andre Heinecke <aheinecke@intevation.de> parents: 372 diff changeset	428 builder_class = SbdmockBuilder
41226e427823 Added an option to configure the used builder class from the treepkg.cfg. This Andre Heinecke <aheinecke@intevation.de> parents: 372 diff changeset	429 elif builder_cls.upper() == "PBUILDER":
41226e427823 Added an option to configure the used builder class from the treepkg.cfg. This Andre Heinecke <aheinecke@intevation.de> parents: 372 diff changeset	430 builder_class = PBuilder
41226e427823 Added an option to configure the used builder class from the treepkg.cfg. This Andre Heinecke <aheinecke@intevation.de> parents: 372 diff changeset	431 else:
41226e427823 Added an option to configure the used builder class from the treepkg.cfg. This Andre Heinecke <aheinecke@intevation.de> parents: 372 diff changeset	432 # If the builder option is explicitly set with an unknown builder
41226e427823 Added an option to configure the used builder class from the treepkg.cfg. This Andre Heinecke <aheinecke@intevation.de> parents: 372 diff changeset	433 # a warning is printed.
41226e427823 Added an option to configure the used builder class from the treepkg.cfg. This Andre Heinecke <aheinecke@intevation.de> parents: 372 diff changeset	434 logging.warning("Track: %s Builder option %s could not be parsed \
41226e427823 Added an option to configure the used builder class from the treepkg.cfg. This Andre Heinecke <aheinecke@intevation.de> parents: 372 diff changeset	435 defaulting to pbuilder" % (name, builder_cls))
41226e427823 Added an option to configure the used builder class from the treepkg.cfg. This Andre Heinecke <aheinecke@intevation.de> parents: 372 diff changeset	436 builder_class = PBuilder
299 c32dc72ba979 Turn the SourcePackager class attribute pkg_basename into a per-track Bernhard Herzog <bh@intevation.de> parents: 297 diff changeset	437 if not pkg_basename:
c32dc72ba979 Turn the SourcePackager class attribute pkg_basename into a per-track Bernhard Herzog <bh@intevation.de> parents: 297 diff changeset	438 pkg_basename = name
c32dc72ba979 Turn the SourcePackager class attribute pkg_basename into a per-track Bernhard Herzog <bh@intevation.de> parents: 297 diff changeset	439 self.pkg_basename = pkg_basename
300 e82fb08781a2 Turn the SourcePackager class attribute changemsg_template into a Bernhard Herzog <bh@intevation.de> parents: 299 diff changeset	440 self.changelog_msg_template = changelog_msg_template
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	441 self.base_dir = base_dir
378 41226e427823 Added an option to configure the used builder class from the treepkg.cfg. This Andre Heinecke <aheinecke@intevation.de> parents: 372 diff changeset	442 self.builder = builder_class(builderconfig, root_cmd,
176 7bde59aa611e Make PBuilder.update_extra_pkg_dir create Release and Release.gpg files Bernhard Herzog <bh@intevation.de> parents: 172 diff changeset	443 release_signing_keyid=signing_key_id)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	444 self.deb_email = deb_email
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	445 self.deb_fullname = deb_fullname
297 4dd6ec3a1151 Make it possible to use parallel builds for packages that support it: Bernhard Herzog <bh@intevation.de> parents: 293 diff changeset	446 self.deb_build_options = deb_build_options
305 3781e9958eba Add per-track configuration option version_template used by the Bernhard Herzog <bh@intevation.de> parents: 304 diff changeset	447 self.version_template = version_template
293 faeeac2c4c71 Replace debrevision_prefix with pkg_revision_template. Their meaning is Bernhard Herzog <bh@intevation.de> parents: 281 diff changeset	448 self.pkg_revision_template = pkg_revision_template
167 36004ee0b3a1 Introduce package track config option signing_key_id to specify the Bernhard Herzog <bh@intevation.de> parents: 149 diff changeset	449 self.signing_key_id = signing_key_id
191 94fb3f3ab58b When building a subset of tracks, make sure new packages are added to Bernhard Herzog <bh@intevation.de> parents: 190 diff changeset	450 self.do_build = do_build
128 5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	451 self.handle_dependencies = handle_dependencies
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	452 self.dependencies = None
401 2db42a2a9db9 add os config statement Bjoern Ricks <bricks@intevation.de> parents: 393 diff changeset	453 self.os = os
328 dd2bd0ccd674 Revisions are now handled as strings Andre Heinecke <aheinecke@intevation.de> parents: 321 diff changeset	454 self.pkg_dir_template = "%(revision)s-%(rules_revision)s"
dd2bd0ccd674 Revisions are now handled as strings Andre Heinecke <aheinecke@intevation.de> parents: 321 diff changeset	455 self.pkg_dir_regex = re.compile(r"(?P<revision>[0-9a-f]+)"
dd2bd0ccd674 Revisions are now handled as strings Andre Heinecke <aheinecke@intevation.de> parents: 321 diff changeset	456 r"-(?P<rules_revision>[0-9a-f]+)$")
306 163f0d8b64eb Make the svn external subdirectories configurable in the configuration Bernhard Herzog <bh@intevation.de> parents: 305 diff changeset	457 externals = svn_externals
163f0d8b64eb Make the svn external subdirectories configurable in the configuration Bernhard Herzog <bh@intevation.de> parents: 305 diff changeset	458 if not externals:
163f0d8b64eb Make the svn external subdirectories configurable in the configuration Bernhard Herzog <bh@intevation.de> parents: 305 diff changeset	459 externals = self.svn_external_subdirs
321 092925ff75d7 Added basic Git support, configuration options: Andre Heinecke <aheinecke@intevation.de> parents: 312 diff changeset	460 if svn_url:
092925ff75d7 Added basic Git support, configuration options: Andre Heinecke <aheinecke@intevation.de> parents: 312 diff changeset	461 repo = SvnRepository(svn_url, externals, subset=svn_subset)
092925ff75d7 Added basic Git support, configuration options: Andre Heinecke <aheinecke@intevation.de> parents: 312 diff changeset	462 self.working_copy = SvnWorkingCopy(repo, self.checkout_dir,
092925ff75d7 Added basic Git support, configuration options: Andre Heinecke <aheinecke@intevation.de> parents: 312 diff changeset	463 logger=logging)
092925ff75d7 Added basic Git support, configuration options: Andre Heinecke <aheinecke@intevation.de> parents: 312 diff changeset	464 elif git_url:
092925ff75d7 Added basic Git support, configuration options: Andre Heinecke <aheinecke@intevation.de> parents: 312 diff changeset	465 repo = GitRepository(git_url, branch=git_branch)
092925ff75d7 Added basic Git support, configuration options: Andre Heinecke <aheinecke@intevation.de> parents: 312 diff changeset	466 self.working_copy = GitWorkingCopy(repo, self.checkout_dir,
092925ff75d7 Added basic Git support, configuration options: Andre Heinecke <aheinecke@intevation.de> parents: 312 diff changeset	467 logger=logging)
234 eaa696629a91 Add a way to specify the svn URL of the repository with the debian rules Bernhard Herzog <bh@intevation.de> parents: 232 diff changeset	468 if rules_svn_url:
eaa696629a91 Add a way to specify the svn URL of the repository with the debian rules Bernhard Herzog <bh@intevation.de> parents: 232 diff changeset	469 repo = SvnRepository(rules_svn_url)
eaa696629a91 Add a way to specify the svn URL of the repository with the debian rules Bernhard Herzog <bh@intevation.de> parents: 232 diff changeset	470 self.rules_working_copy = SvnWorkingCopy(repo, self.debian_dir,
eaa696629a91 Add a way to specify the svn URL of the repository with the debian rules Bernhard Herzog <bh@intevation.de> parents: 232 diff changeset	471 logger=logging)
eaa696629a91 Add a way to specify the svn URL of the repository with the debian rules Bernhard Herzog <bh@intevation.de> parents: 232 diff changeset	472 else:
eaa696629a91 Add a way to specify the svn URL of the repository with the debian rules Bernhard Herzog <bh@intevation.de> parents: 232 diff changeset	473 self.rules_working_copy = ManualWorkingCopy(self.debian_dir)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	474
172 06af36f915f2 Move the filenameproperty factory from treepkg/packager.py to Bernhard Herzog <bh@intevation.de> parents: 168 diff changeset	475 checkout_dir = util.filenameproperty("checkout")
06af36f915f2 Move the filenameproperty factory from treepkg/packager.py to Bernhard Herzog <bh@intevation.de> parents: 168 diff changeset	476 debian_dir = util.filenameproperty("debian")
06af36f915f2 Move the filenameproperty factory from treepkg/packager.py to Bernhard Herzog <bh@intevation.de> parents: 168 diff changeset	477 pkg_dir = util.filenameproperty("pkg")
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	478
106 66cbfc772f84 Add bin/inittreepkg.py, a script to automate some of the installation Bernhard Herzog <bh@intevation.de> parents: 99 diff changeset	479 def init_treepkg(self):
66cbfc772f84 Add bin/inittreepkg.py, a script to automate some of the installation Bernhard Herzog <bh@intevation.de> parents: 99 diff changeset	480 print "Initializing", self.name
66cbfc772f84 Add bin/inittreepkg.py, a script to automate some of the installation Bernhard Herzog <bh@intevation.de> parents: 99 diff changeset	481 if not os.path.exists(self.base_dir):
66cbfc772f84 Add bin/inittreepkg.py, a script to automate some of the installation Bernhard Herzog <bh@intevation.de> parents: 99 diff changeset	482 print "creating %s" % (self.base_dir,)
66cbfc772f84 Add bin/inittreepkg.py, a script to automate some of the installation Bernhard Herzog <bh@intevation.de> parents: 99 diff changeset	483 util.ensure_directory(self.base_dir)
234 eaa696629a91 Add a way to specify the svn URL of the repository with the debian rules Bernhard Herzog <bh@intevation.de> parents: 232 diff changeset	484 # TODO: handle case where debian directory is in version control
106 66cbfc772f84 Add bin/inittreepkg.py, a script to automate some of the installation Bernhard Herzog <bh@intevation.de> parents: 99 diff changeset	485 if not os.path.exists(self.debian_dir):
66cbfc772f84 Add bin/inittreepkg.py, a script to automate some of the installation Bernhard Herzog <bh@intevation.de> parents: 99 diff changeset	486 print ("TODO: the debian directory %s still has to be created"
66cbfc772f84 Add bin/inittreepkg.py, a script to automate some of the installation Bernhard Herzog <bh@intevation.de> parents: 99 diff changeset	487 % (self.debian_dir,))
66cbfc772f84 Add bin/inittreepkg.py, a script to automate some of the installation Bernhard Herzog <bh@intevation.de> parents: 99 diff changeset	488
128 5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	489 def determine_dependencies(self):
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	490 if self.dependencies is not None:
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	491 return
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	492
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	493 requires = ()
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	494 provides = ()
131 bea9f1cc0bef Try to determine dependencies only when treepkg is fully configured. Bernhard Herzog <bh@intevation.de> parents: 128 diff changeset	495 # only try to parse the control file if the debian directory
bea9f1cc0bef Try to determine dependencies only when treepkg is fully configured. Bernhard Herzog <bh@intevation.de> parents: 128 diff changeset	496 # exists. If the debian directory doesn't exist yet, the tree
bea9f1cc0bef Try to determine dependencies only when treepkg is fully configured. Bernhard Herzog <bh@intevation.de> parents: 128 diff changeset	497 # packager is likely still being configured and this code may be
bea9f1cc0bef Try to determine dependencies only when treepkg is fully configured. Bernhard Herzog <bh@intevation.de> parents: 128 diff changeset	498 # run indirectly from e. g. bin/inittreepkg.py in which case the
bea9f1cc0bef Try to determine dependencies only when treepkg is fully configured. Bernhard Herzog <bh@intevation.de> parents: 128 diff changeset	499 # init_treepkg method will report the missing debian
bea9f1cc0bef Try to determine dependencies only when treepkg is fully configured. Bernhard Herzog <bh@intevation.de> parents: 128 diff changeset	500 if self.handle_dependencies and os.path.exists(self.debian_dir):
128 5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	501 control = debian.DebianControlFile(os.path.join(self.debian_dir,
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	502 "control"))
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	503 requires = control.build_depends
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	504 provides = (pkg[0] for pkg in control.packages)
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	505 self.dependencies = (set(requires), set(provides))
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	506 logging.debug("Track %s: build depends: %s", self.name,
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	507 " ".join(self.dependencies[0]))
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	508 logging.debug("Track %s: provides: %s", self.name,
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	509 " ".join(self.dependencies[1]))
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	510
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	511 def dependencies_required(self):
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	512 """Returns a list of required packages"""
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	513 self.determine_dependencies()
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	514 return self.dependencies[0]
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	515
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	516 def dependencies_provided(self):
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	517 """Returns a list of provided packages"""
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	518 self.determine_dependencies()
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	519 return self.dependencies[1]
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	520
229 653a45adda50 Prepare for svn managed debian rules directories. So far, the directory Bernhard Herzog <bh@intevation.de> parents: 224 diff changeset	521 def pkg_dir_for_revision(self, revision, rules_revision):
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	522 return os.path.join(self.pkg_dir,
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	523 self.pkg_dir_template % locals())
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	524
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	525 def last_changed_revision(self):
224 6bac65dcf258 Handle the svn repositories and working copies in a more object oriented way Bernhard Herzog <bh@intevation.de> parents: 222 diff changeset	526 return self.working_copy.last_changed_revision()
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	527
11 6efe0bd3d8c1 Add method RevisionPackager.get_revision_numbers Bernhard Herzog <bh@intevation.de> parents: 10 diff changeset	528 def get_revision_numbers(self):
328 dd2bd0ccd674 Revisions are now handled as strings Andre Heinecke <aheinecke@intevation.de> parents: 321 diff changeset	529 """Returns a list of the packaged revisions"""
11 6efe0bd3d8c1 Add method RevisionPackager.get_revision_numbers Bernhard Herzog <bh@intevation.de> parents: 10 diff changeset	530 revisions = []
6efe0bd3d8c1 Add method RevisionPackager.get_revision_numbers Bernhard Herzog <bh@intevation.de> parents: 10 diff changeset	531 if os.path.exists(self.pkg_dir):
6efe0bd3d8c1 Add method RevisionPackager.get_revision_numbers Bernhard Herzog <bh@intevation.de> parents: 10 diff changeset	532 for filename in os.listdir(self.pkg_dir):
6efe0bd3d8c1 Add method RevisionPackager.get_revision_numbers Bernhard Herzog <bh@intevation.de> parents: 10 diff changeset	533 match = self.pkg_dir_regex.match(filename)
6efe0bd3d8c1 Add method RevisionPackager.get_revision_numbers Bernhard Herzog <bh@intevation.de> parents: 10 diff changeset	534 if match:
328 dd2bd0ccd674 Revisions are now handled as strings Andre Heinecke <aheinecke@intevation.de> parents: 321 diff changeset	535 revisions.append((match.group("revision"),
dd2bd0ccd674 Revisions are now handled as strings Andre Heinecke <aheinecke@intevation.de> parents: 321 diff changeset	536 match.group("rules_revision")))
dd2bd0ccd674 Revisions are now handled as strings Andre Heinecke <aheinecke@intevation.de> parents: 321 diff changeset	537 return sorted(revisions)
11 6efe0bd3d8c1 Add method RevisionPackager.get_revision_numbers Bernhard Herzog <bh@intevation.de> parents: 10 diff changeset	538
80 0af2fa3790e0 Add revision parameter to PackagerGroup.__init__ and some other methods Bernhard Herzog <bh@intevation.de> parents: 78 diff changeset	539 def update_checkout(self, revision=None):
224 6bac65dcf258 Handle the svn repositories and working copies in a more object oriented way Bernhard Herzog <bh@intevation.de> parents: 222 diff changeset	540 """Updates the working copy.
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	541
224 6bac65dcf258 Handle the svn repositories and working copies in a more object oriented way Bernhard Herzog <bh@intevation.de> parents: 222 diff changeset	542 If the checkout_dir doesn't exist yet, a new checkout is made
6bac65dcf258 Handle the svn repositories and working copies in a more object oriented way Bernhard Herzog <bh@intevation.de> parents: 222 diff changeset	543 into that directory. The value of the revision parameter is
6bac65dcf258 Handle the svn repositories and working copies in a more object oriented way Bernhard Herzog <bh@intevation.de> parents: 222 diff changeset	544 passed through to the update method.
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	545 """
224 6bac65dcf258 Handle the svn repositories and working copies in a more object oriented way Bernhard Herzog <bh@intevation.de> parents: 222 diff changeset	546 self.working_copy.update_or_checkout(revision=revision)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	547
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	548 def export_sources(self, to_dir):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	549 logging.info("Exporting sources for tarball to %r", to_dir)
224 6bac65dcf258 Handle the svn repositories and working copies in a more object oriented way Bernhard Herzog <bh@intevation.de> parents: 222 diff changeset	550 self.working_copy.export(to_dir)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	551
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	552 def copy_debian_directory(self, to_dir):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	553 logging.info("Copying debian directory to %r", to_dir)
231 7dbf0a3443bb Use treepkg.subversion.ManualWorkingCopy for the debian directory in a Bernhard Herzog <bh@intevation.de> parents: 229 diff changeset	554 self.rules_working_copy.export(to_dir)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	555
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	556 def debian_environment(self):
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	557 """Returns the environment variables for the debian commands"""
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	558 env = os.environ.copy()
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	559 env["DEBFULLNAME"] = self.deb_fullname
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	560 env["DEBEMAIL"] = self.deb_email
297 4dd6ec3a1151 Make it possible to use parallel builds for packages that support it: Bernhard Herzog <bh@intevation.de> parents: 293 diff changeset	561 env["DEB_BUILD_OPTIONS"] = self.deb_build_options
4dd6ec3a1151 Make it possible to use parallel builds for packages that support it: Bernhard Herzog <bh@intevation.de> parents: 293 diff changeset	562 # cdbs requires DEB_BUILD_PARALLEL set to something non-empty,
4dd6ec3a1151 Make it possible to use parallel builds for packages that support it: Bernhard Herzog <bh@intevation.de> parents: 293 diff changeset	563 # otherwise it will ignore any parallel=<n> setting in
4dd6ec3a1151 Make it possible to use parallel builds for packages that support it: Bernhard Herzog <bh@intevation.de> parents: 293 diff changeset	564 # DEB_BUILD_OPTIONS.
4dd6ec3a1151 Make it possible to use parallel builds for packages that support it: Bernhard Herzog <bh@intevation.de> parents: 293 diff changeset	565 env["DEB_BUILD_PARALLEL"] = "true"
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	566 return env
f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	567
261 e574b03a2957 Move the code that creates the builder for a new revision to a separate method. Bernhard Herzog <bh@intevation.de> parents: 260 diff changeset	568 def packager_for_new_revision(self):
e574b03a2957 Move the code that creates the builder for a new revision to a separate method. Bernhard Herzog <bh@intevation.de> parents: 260 diff changeset	569 current_revision = (self.last_changed_revision(),
e574b03a2957 Move the code that creates the builder for a new revision to a separate method. Bernhard Herzog <bh@intevation.de> parents: 260 diff changeset	570 self.rules_working_copy.last_changed_revision())
e574b03a2957 Move the code that creates the builder for a new revision to a separate method. Bernhard Herzog <bh@intevation.de> parents: 260 diff changeset	571 logging.info("New revision is %s", current_revision)
e574b03a2957 Move the code that creates the builder for a new revision to a separate method. Bernhard Herzog <bh@intevation.de> parents: 260 diff changeset	572 if current_revision not in self.get_revision_numbers():
e574b03a2957 Move the code that creates the builder for a new revision to a separate method. Bernhard Herzog <bh@intevation.de> parents: 260 diff changeset	573 logging.info("Revision %s has not been packaged yet",
e574b03a2957 Move the code that creates the builder for a new revision to a separate method. Bernhard Herzog <bh@intevation.de> parents: 260 diff changeset	574 current_revision)
e574b03a2957 Move the code that creates the builder for a new revision to a separate method. Bernhard Herzog <bh@intevation.de> parents: 260 diff changeset	575 return self.revision_packager_cls(self, *current_revision)
e574b03a2957 Move the code that creates the builder for a new revision to a separate method. Bernhard Herzog <bh@intevation.de> parents: 260 diff changeset	576 else:
e574b03a2957 Move the code that creates the builder for a new revision to a separate method. Bernhard Herzog <bh@intevation.de> parents: 260 diff changeset	577 logging.info("Revision %s has already been packaged.",
e574b03a2957 Move the code that creates the builder for a new revision to a separate method. Bernhard Herzog <bh@intevation.de> parents: 260 diff changeset	578 current_revision)
e574b03a2957 Move the code that creates the builder for a new revision to a separate method. Bernhard Herzog <bh@intevation.de> parents: 260 diff changeset	579
190 e83db4482aab Add runtreepkg.py command line option --no-svn-update to inhibit updates Bernhard Herzog <bh@intevation.de> parents: 179 diff changeset	580 def package_if_updated(self, revision=None, do_svn_update=True):
e83db4482aab Add runtreepkg.py command line option --no-svn-update to inhibit updates Bernhard Herzog <bh@intevation.de> parents: 179 diff changeset	581 """Returns a new packager if the working copy has not been packaged yet.
e83db4482aab Add runtreepkg.py command line option --no-svn-update to inhibit updates Bernhard Herzog <bh@intevation.de> parents: 179 diff changeset	582 If do_svn_update is true -- the default -- update the working
370 0bee91b435ab Fix typos in doc-strings. Bernhard Herzog <bh@intevation.de> parents: 367 diff changeset	583 copy to the revision specified with the revision parameter
190 e83db4482aab Add runtreepkg.py command line option --no-svn-update to inhibit updates Bernhard Herzog <bh@intevation.de> parents: 179 diff changeset	584 or if revision is None, the latest revision in the repository."""
191 94fb3f3ab58b When building a subset of tracks, make sure new packages are added to Bernhard Herzog <bh@intevation.de> parents: 190 diff changeset	585 if not self.do_build:
94fb3f3ab58b When building a subset of tracks, make sure new packages are added to Bernhard Herzog <bh@intevation.de> parents: 190 diff changeset	586 return None
190 e83db4482aab Add runtreepkg.py command line option --no-svn-update to inhibit updates Bernhard Herzog <bh@intevation.de> parents: 179 diff changeset	587 if do_svn_update:
e83db4482aab Add runtreepkg.py command line option --no-svn-update to inhibit updates Bernhard Herzog <bh@intevation.de> parents: 179 diff changeset	588 self.update_checkout(revision=revision)
232 e3cda08d2619 When checking whether to build a new revision, also update the rules Bernhard Herzog <bh@intevation.de> parents: 231 diff changeset	589 # TODO: what should happen with the debian checkout, if a
e3cda08d2619 When checking whether to build a new revision, also update the rules Bernhard Herzog <bh@intevation.de> parents: 231 diff changeset	590 # revision for the source checkoute was given?
e3cda08d2619 When checking whether to build a new revision, also update the rules Bernhard Herzog <bh@intevation.de> parents: 231 diff changeset	591 self.rules_working_copy.update_or_checkout()
261 e574b03a2957 Move the code that creates the builder for a new revision to a separate method. Bernhard Herzog <bh@intevation.de> parents: 260 diff changeset	592 return self.packager_for_new_revision()
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	593
14 dfd89f81e66c Add simple status report tool. Still using the old status files Bernhard Herzog <bh@intevation.de> parents: 11 diff changeset	594 def get_revisions(self):
dfd89f81e66c Add simple status report tool. Still using the old status files Bernhard Herzog <bh@intevation.de> parents: 11 diff changeset	595 """Returns RevisionPackager objects for each packaged revision"""
229 653a45adda50 Prepare for svn managed debian rules directories. So far, the directory Bernhard Herzog <bh@intevation.de> parents: 224 diff changeset	596 return [self.revision_packager_cls(self, revision, rules_revision)
653a45adda50 Prepare for svn managed debian rules directories. So far, the directory Bernhard Herzog <bh@intevation.de> parents: 224 diff changeset	597 for revision, rules_revision in self.get_revision_numbers()]
14 dfd89f81e66c Add simple status report tool. Still using the old status files Bernhard Herzog <bh@intevation.de> parents: 11 diff changeset	598
179 952d366f7b14 Implement optional signing of .dsc and .changes files Bernhard Herzog <bh@intevation.de> parents: 176 diff changeset	599 def sign_file(self, filename):
952d366f7b14 Implement optional signing of .dsc and .changes files Bernhard Herzog <bh@intevation.de> parents: 176 diff changeset	600 """Signs a file using the debian.sign_file function.
952d366f7b14 Implement optional signing of .dsc and .changes files Bernhard Herzog <bh@intevation.de> parents: 176 diff changeset	601 The file is signed with the key indicated by the track's
952d366f7b14 Implement optional signing of .dsc and .changes files Bernhard Herzog <bh@intevation.de> parents: 176 diff changeset	602 signing_key_id attribute. If that is empty, the file is not
952d366f7b14 Implement optional signing of .dsc and .changes files Bernhard Herzog <bh@intevation.de> parents: 176 diff changeset	603 signed.
952d366f7b14 Implement optional signing of .dsc and .changes files Bernhard Herzog <bh@intevation.de> parents: 176 diff changeset	604 """
952d366f7b14 Implement optional signing of .dsc and .changes files Bernhard Herzog <bh@intevation.de> parents: 176 diff changeset	605 if self.signing_key_id:
952d366f7b14 Implement optional signing of .dsc and .changes files Bernhard Herzog <bh@intevation.de> parents: 176 diff changeset	606 logging.info("Signing %r with key %r", filename,
952d366f7b14 Implement optional signing of .dsc and .changes files Bernhard Herzog <bh@intevation.de> parents: 176 diff changeset	607 self.signing_key_id)
952d366f7b14 Implement optional signing of .dsc and .changes files Bernhard Herzog <bh@intevation.de> parents: 176 diff changeset	608 debian.sign_file(filename, self.signing_key_id)
952d366f7b14 Implement optional signing of .dsc and .changes files Bernhard Herzog <bh@intevation.de> parents: 176 diff changeset	609
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	610
113 312949e7628d Add a smarter way to load the packager modules: Add the function Bernhard Herzog <bh@intevation.de> parents: 112 diff changeset	611 def import_packager_module(packager_class):
312949e7628d Add a smarter way to load the packager modules: Add the function Bernhard Herzog <bh@intevation.de> parents: 112 diff changeset	612 """Import the packager module named by packager_class.
312949e7628d Add a smarter way to load the packager modules: Add the function Bernhard Herzog <bh@intevation.de> parents: 112 diff changeset	613
312949e7628d Add a smarter way to load the packager modules: Add the function Bernhard Herzog <bh@intevation.de> parents: 112 diff changeset	614 The packager_class must be the full absolute module name for the
125 34e08d52956e Make the import_packager_module function more flexible. The modules may Bernhard Herzog <bh@intevation.de> parents: 113 diff changeset	615 packager. The function tries to find or create a suitable
34e08d52956e Make the import_packager_module function more flexible. The modules may Bernhard Herzog <bh@intevation.de> parents: 113 diff changeset	616 PackageTrack class from this module using the following rules:
34e08d52956e Make the import_packager_module function more flexible. The modules may Bernhard Herzog <bh@intevation.de> parents: 113 diff changeset	617
34e08d52956e Make the import_packager_module function more flexible. The modules may Bernhard Herzog <bh@intevation.de> parents: 113 diff changeset	618 - If the module contains a class called PackageTrack, use that.
34e08d52956e Make the import_packager_module function more flexible. The modules may Bernhard Herzog <bh@intevation.de> parents: 113 diff changeset	619
34e08d52956e Make the import_packager_module function more flexible. The modules may Bernhard Herzog <bh@intevation.de> parents: 113 diff changeset	620 - Otherwise create one using the module's RevisionPackager class,
34e08d52956e Make the import_packager_module function more flexible. The modules may Bernhard Herzog <bh@intevation.de> parents: 113 diff changeset	621 creating RevisionPackager if necessary.
34e08d52956e Make the import_packager_module function more flexible. The modules may Bernhard Herzog <bh@intevation.de> parents: 113 diff changeset	622
34e08d52956e Make the import_packager_module function more flexible. The modules may Bernhard Herzog <bh@intevation.de> parents: 113 diff changeset	623 - If RevisionPackager needs to be created, it uses the module's
34e08d52956e Make the import_packager_module function more flexible. The modules may Bernhard Herzog <bh@intevation.de> parents: 113 diff changeset	624 SourcePackager as source_packager_cls and if present also the
34e08d52956e Make the import_packager_module function more flexible. The modules may Bernhard Herzog <bh@intevation.de> parents: 113 diff changeset	625 module's BinaryPackager as binary_packager_cls. If the module
34e08d52956e Make the import_packager_module function more flexible. The modules may Bernhard Herzog <bh@intevation.de> parents: 113 diff changeset	626 does not have a BinaryPackager, the default BinaryPackager is
34e08d52956e Make the import_packager_module function more flexible. The modules may Bernhard Herzog <bh@intevation.de> parents: 113 diff changeset	627 used.
113 312949e7628d Add a smarter way to load the packager modules: Add the function Bernhard Herzog <bh@intevation.de> parents: 112 diff changeset	628 """
312949e7628d Add a smarter way to load the packager modules: Add the function Bernhard Herzog <bh@intevation.de> parents: 112 diff changeset	629 module = util.import_dotted_name(packager_class)
312949e7628d Add a smarter way to load the packager modules: Add the function Bernhard Herzog <bh@intevation.de> parents: 112 diff changeset	630 if not hasattr(module, "PackageTrack"):
125 34e08d52956e Make the import_packager_module function more flexible. The modules may Bernhard Herzog <bh@intevation.de> parents: 113 diff changeset	631 if not hasattr(module, "RevisionPackager"):
34e08d52956e Make the import_packager_module function more flexible. The modules may Bernhard Herzog <bh@intevation.de> parents: 113 diff changeset	632 binary_packager = getattr(module, "BinaryPackager", BinaryPackager)
34e08d52956e Make the import_packager_module function more flexible. The modules may Bernhard Herzog <bh@intevation.de> parents: 113 diff changeset	633 module.RevisionPackager \
34e08d52956e Make the import_packager_module function more flexible. The modules may Bernhard Herzog <bh@intevation.de> parents: 113 diff changeset	634 = new.classobj("RevisionPackager", (RevisionPackager,),
34e08d52956e Make the import_packager_module function more flexible. The modules may Bernhard Herzog <bh@intevation.de> parents: 113 diff changeset	635 dict(source_packager_cls=module.SourcePackager,
34e08d52956e Make the import_packager_module function more flexible. The modules may Bernhard Herzog <bh@intevation.de> parents: 113 diff changeset	636 binary_packager_cls=binary_packager))
113 312949e7628d Add a smarter way to load the packager modules: Add the function Bernhard Herzog <bh@intevation.de> parents: 112 diff changeset	637 module.PackageTrack \
312949e7628d Add a smarter way to load the packager modules: Add the function Bernhard Herzog <bh@intevation.de> parents: 112 diff changeset	638 = new.classobj("PackageTrack", (PackageTrack,),
312949e7628d Add a smarter way to load the packager modules: Add the function Bernhard Herzog <bh@intevation.de> parents: 112 diff changeset	639 dict(revision_packager_cls=module.RevisionPackager))
312949e7628d Add a smarter way to load the packager modules: Add the function Bernhard Herzog <bh@intevation.de> parents: 112 diff changeset	640 return module
312949e7628d Add a smarter way to load the packager modules: Add the function Bernhard Herzog <bh@intevation.de> parents: 112 diff changeset	641
52 78cf5f6778ec Rename 'packagel ine' -> 'package track' Bernhard Herzog <bh@intevation.de> parents: 49 diff changeset	642 def create_package_track(packager_class, **kw):
113 312949e7628d Add a smarter way to load the packager modules: Add the function Bernhard Herzog <bh@intevation.de> parents: 112 diff changeset	643 module = import_packager_module(packager_class)
52 78cf5f6778ec Rename 'packagel ine' -> 'package track' Bernhard Herzog <bh@intevation.de> parents: 49 diff changeset	644 return module.PackageTrack(**kw)
4 fee641fec94e Separate the kolab specific parts. Bernhard Herzog <bh@intevation.de> parents: 2 diff changeset	645
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	646
128 5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	647 class CyclicDependencyError(Exception):
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	648
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	649 """Exception thrown when a cycle is detected in the track dependencies"""
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	650
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	651 def __init__(self, tracks):
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	652 Exception.__init__(self,
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	653 "Cyclic dependencies between" " tracks (%s)"
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	654 % ", ".join([track.name for track in tracks]))
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	655
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	656
7 96f4f58c62b5 Rename the Packager class to PackagerGroup Bernhard Herzog <bh@intevation.de> parents: 4 diff changeset	657 class PackagerGroup(object):
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	658
91 3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	659 def __init__(self, package_tracks, check_interval, revision=None,
307 5f442b0cf3a4 New command line argument --stop-on-error for runtreepkg.py and Bernhard Herzog <bh@intevation.de> parents: 306 diff changeset	660 instructions_file=None, do_svn_update=True,
389 a690fc689f2f added treepkg_dir andd tracks_dir attributes to PackageGroup Bjoern Ricks <bricks@intevation.de> parents: 387 diff changeset	661 stop_on_error=False, name="", treepkg_dir=None,
a690fc689f2f added treepkg_dir andd tracks_dir attributes to PackageGroup Bjoern Ricks <bricks@intevation.de> parents: 387 diff changeset	662 tracks_dir=None):
52 78cf5f6778ec Rename 'packagel ine' -> 'package track' Bernhard Herzog <bh@intevation.de> parents: 49 diff changeset	663 self.package_tracks = package_tracks
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	664 self.check_interval = check_interval
80 0af2fa3790e0 Add revision parameter to PackagerGroup.__init__ and some other methods Bernhard Herzog <bh@intevation.de> parents: 78 diff changeset	665 self.revision = revision
190 e83db4482aab Add runtreepkg.py command line option --no-svn-update to inhibit updates Bernhard Herzog <bh@intevation.de> parents: 179 diff changeset	666 self.do_svn_update = do_svn_update
307 5f442b0cf3a4 New command line argument --stop-on-error for runtreepkg.py and Bernhard Herzog <bh@intevation.de> parents: 306 diff changeset	667 self.stop_on_error = stop_on_error
91 3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	668 self.instructions_file = instructions_file
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	669 self.instructions_file_removed = False
387 c1f3be727f9d renamed new status dir to info because of a naming conflict with status.py Bjoern Ricks <bricks@intevation.de> parents: 378 diff changeset	670 self.name = name
389 a690fc689f2f added treepkg_dir andd tracks_dir attributes to PackageGroup Bjoern Ricks <bricks@intevation.de> parents: 387 diff changeset	671 self.treepkg_dir = treepkg_dir
a690fc689f2f added treepkg_dir andd tracks_dir attributes to PackageGroup Bjoern Ricks <bricks@intevation.de> parents: 387 diff changeset	672 self.tracks_dir = tracks_dir
128 5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	673 self.sort_tracks()
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	674
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	675 def sort_tracks(self):
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	676 """Sorts tracks for dependency handling"""
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	677 todo = self.package_tracks[:]
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	678 sorted = []
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	679 seen = set()
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	680
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	681 # dependencies that can be solved by one of the tracks
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	682 known = set()
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	683 for track in todo:
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	684 known \|= track.dependencies_provided()
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	685
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	686 while todo:
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	687 todo_again = []
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	688 for track in todo:
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	689 if not track.handle_dependencies:
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	690 sorted.append(track)
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	691 continue
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	692
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	693 unmet = (track.dependencies_required() & known) - seen
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	694 if unmet:
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	695 todo_again.append(track)
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	696 else:
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	697 sorted.append(track)
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	698 seen \|= track.dependencies_provided()
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	699 if todo_again == todo:
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	700 raise CyclicDependencyError(todo)
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	701 todo = todo_again
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	702
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	703 self.package_tracks = sorted
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	704 self.needed_binaries = set()
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	705 for track in self.package_tracks:
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	706 self.needed_binaries \|= track.dependencies_required()
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	707 self.needed_binaries &= known
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	708
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	709 logging.info("sorted track order: %s",
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	710 " ".join(track.name for track in sorted))
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	711 logging.info("binary packages needed as build dependencies: %s",
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	712 " ".join(self.needed_binaries))
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	713
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	714 def run(self):
7 96f4f58c62b5 Rename the Packager class to PackagerGroup Bernhard Herzog <bh@intevation.de> parents: 4 diff changeset	715 """Runs the packager group indefinitely"""
78 9a602d8eaa60 initial revision of the subversion repository Thomas Arendsen Hein <thomas@intevation.de> parents: 55 diff changeset	716 logging.info("Starting in periodic check mode."
9a602d8eaa60 initial revision of the subversion repository Thomas Arendsen Hein <thomas@intevation.de> parents: 55 diff changeset	717 " Will check every %d seconds", self.check_interval)
312 2d6915bce473 Rework the packager mainloop. it's a bit simpler now and stopping should Bernhard Herzog <bh@intevation.de> parents: 310 diff changeset	718 next_check = time.time()
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	719 while 1:
312 2d6915bce473 Rework the packager mainloop. it's a bit simpler now and stopping should Bernhard Herzog <bh@intevation.de> parents: 310 diff changeset	720 if self.should_stop():
2d6915bce473 Rework the packager mainloop. it's a bit simpler now and stopping should Bernhard Herzog <bh@intevation.de> parents: 310 diff changeset	721 logging.info("Received stop instruction. Stopping.")
2d6915bce473 Rework the packager mainloop. it's a bit simpler now and stopping should Bernhard Herzog <bh@intevation.de> parents: 310 diff changeset	722 return
2d6915bce473 Rework the packager mainloop. it's a bit simpler now and stopping should Bernhard Herzog <bh@intevation.de> parents: 310 diff changeset	723
2d6915bce473 Rework the packager mainloop. it's a bit simpler now and stopping should Bernhard Herzog <bh@intevation.de> parents: 310 diff changeset	724 this_check = time.time()
2d6915bce473 Rework the packager mainloop. it's a bit simpler now and stopping should Bernhard Herzog <bh@intevation.de> parents: 310 diff changeset	725 if this_check >= next_check:
2d6915bce473 Rework the packager mainloop. it's a bit simpler now and stopping should Bernhard Herzog <bh@intevation.de> parents: 310 diff changeset	726 logging.info("Next check is now")
91 3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	727 if self.check_package_tracks():
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	728 break
312 2d6915bce473 Rework the packager mainloop. it's a bit simpler now and stopping should Bernhard Herzog <bh@intevation.de> parents: 310 diff changeset	729 last_check = this_check
2d6915bce473 Rework the packager mainloop. it's a bit simpler now and stopping should Bernhard Herzog <bh@intevation.de> parents: 310 diff changeset	730 next_check = this_check + self.check_interval
2d6915bce473 Rework the packager mainloop. it's a bit simpler now and stopping should Bernhard Herzog <bh@intevation.de> parents: 310 diff changeset	731 else:
2d6915bce473 Rework the packager mainloop. it's a bit simpler now and stopping should Bernhard Herzog <bh@intevation.de> parents: 310 diff changeset	732 to_sleep = next_check - this_check
2d6915bce473 Rework the packager mainloop. it's a bit simpler now and stopping should Bernhard Herzog <bh@intevation.de> parents: 310 diff changeset	733 logging.info("Next check at %s",
2d6915bce473 Rework the packager mainloop. it's a bit simpler now and stopping should Bernhard Herzog <bh@intevation.de> parents: 310 diff changeset	734 time.strftime("%Y-%m-%d %H:%M:%S",
2d6915bce473 Rework the packager mainloop. it's a bit simpler now and stopping should Bernhard Herzog <bh@intevation.de> parents: 310 diff changeset	735 time.localtime(next_check)))
2d6915bce473 Rework the packager mainloop. it's a bit simpler now and stopping should Bernhard Herzog <bh@intevation.de> parents: 310 diff changeset	736 time.sleep(to_sleep)
0 f78a02e79c84 initial checkin Bernhard Herzog <bh@intevation.de> parents: diff changeset	737
52 78cf5f6778ec Rename 'packagel ine' -> 'package track' Bernhard Herzog <bh@intevation.de> parents: 49 diff changeset	738 def check_package_tracks(self):
78cf5f6778ec Rename 'packagel ine' -> 'package track' Bernhard Herzog <bh@intevation.de> parents: 49 diff changeset	739 logging.info("Checking package tracks")
91 3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	740 self.clear_instruction()
128 5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	741 repeat = True
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	742 while repeat:
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	743 repeat = False
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	744 for track in self.package_tracks:
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	745 try:
190 e83db4482aab Add runtreepkg.py command line option --no-svn-update to inhibit updates Bernhard Herzog <bh@intevation.de> parents: 179 diff changeset	746 packager = track.package_if_updated(revision=self.revision,
e83db4482aab Add runtreepkg.py command line option --no-svn-update to inhibit updates Bernhard Herzog <bh@intevation.de> parents: 179 diff changeset	747 do_svn_update=self.do_svn_update)
128 5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	748 if packager:
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	749 packager.package()
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	750 repeat = self.install_dependencies(track, packager)
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	751 except:
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	752 logging.exception("An error occurred while"
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	753 " checking packager track %r", track.name)
307 5f442b0cf3a4 New command line argument --stop-on-error for runtreepkg.py and Bernhard Herzog <bh@intevation.de> parents: 306 diff changeset	754 if self.stop_on_error:
310 26c15a0f0e52 When stopping because of an error, do not raise the exception again as Bernhard Herzog <bh@intevation.de> parents: 308 diff changeset	755 logging.info("Stopping because of errors.")
26c15a0f0e52 When stopping because of an error, do not raise the exception again as Bernhard Herzog <bh@intevation.de> parents: 308 diff changeset	756 return True
308 61d1daac23d4 Move the should_stop check out of the try/except block so that it's not Bernhard Herzog <bh@intevation.de> parents: 307 diff changeset	757 if self.should_stop():
61d1daac23d4 Move the should_stop check out of the try/except block so that it's not Bernhard Herzog <bh@intevation.de> parents: 307 diff changeset	758 logging.info("Received stop instruction. Stopping.")
61d1daac23d4 Move the should_stop check out of the try/except block so that it's not Bernhard Herzog <bh@intevation.de> parents: 307 diff changeset	759 return True
128 5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	760 if repeat:
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	761 logging.info("Built binaries needed by other tracks."
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	762 " Starting over to ensure all dependencies"
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	763 " are met")
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	764 break
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	765
52 78cf5f6778ec Rename 'packagel ine' -> 'package track' Bernhard Herzog <bh@intevation.de> parents: 49 diff changeset	766 logging.info("Checked all package tracks")
14 dfd89f81e66c Add simple status report tool. Still using the old status files Bernhard Herzog <bh@intevation.de> parents: 11 diff changeset	767
128 5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	768
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	769 def install_dependencies(self, track, packager):
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	770 """Add the binaries built by packager to the builder, if necessary.
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	771 It is necessary if any track depends on the packages. The
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	772 method simply installs all binary files built by the packger
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	773 instead of only those which are immediately required by a track.
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	774 This is done because tracks usually depend directly only on the
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	775 -dev packages which usually require another binary package built
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	776 at the same time.
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	777 """
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	778 if (track.handle_dependencies
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	779 and track.dependencies_provided() & self.needed_binaries):
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	780 # FIXME: this basically assumes that all tracks use the same
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	781 # builder. This is true for now, but it is possible to
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	782 # configure treepkg with different builders for different
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	783 # tracks and we really should be installing the newly built
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	784 # binaries into the builder of the tracks which depends on
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	785 # them
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	786 binaries = packager.list_binary_files()
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	787 track.builder.add_binaries_to_extra_pkg(binaries)
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	788 return True
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	789 return False
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	790
5155b4f9443d Add basic dependency handling to PackageTrack and PackagerGroup. Bernhard Herzog <bh@intevation.de> parents: 125 diff changeset	791
52 78cf5f6778ec Rename 'packagel ine' -> 'package track' Bernhard Herzog <bh@intevation.de> parents: 49 diff changeset	792 def get_package_tracks(self):
78cf5f6778ec Rename 'packagel ine' -> 'package track' Bernhard Herzog <bh@intevation.de> parents: 49 diff changeset	793 return self.package_tracks
91 3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	794
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	795 def read_instruction(self):
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	796 if not self.instructions_file:
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	797 return ""
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	798 try:
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	799 f = open(self.instructions_file)
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	800 except (IOError, OSError):
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	801 return ""
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	802 try:
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	803 return f.read().strip()
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	804 finally:
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	805 f.close()
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	806 self.clear_instruction()
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	807
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	808 def clear_instruction(self, force=False):
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	809 if self.instructions_file and (not self.instructions_file_removed
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	810 or force):
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	811 util.writefile(self.instructions_file, "")
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	812 self.instructions_file_removed = True
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	813
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	814 def should_stop(self):
3ed079a7174a Implement a way to stop a running treepackager. Bernhard Herzog <bh@intevation.de> parents: 90 diff changeset	815 return self.read_instruction() == "stop"

Mercurial > treepkg > treepkg

annotate treepkg/packager.py @ 462:058856954e2d