Mercurial > treepkg > treepkg
changeset 0:f78a02e79c84
initial checkin
| author | Bernhard Herzog <bh@intevation.de> |
|---|---|
| date | Tue, 06 Mar 2007 17:37:32 +0100 |
| parents | |
| children | 02d5702314a3 |
| files | .hgignore COPYING runtreepkg.py treepkg/__init__.py treepkg/packager.py treepkg/readconfig.py treepkg/run.py treepkg/subversion.py treepkg/util.py |
| diffstat | 9 files changed, 942 insertions(+), 0 deletions(-) [+] |
line wrap: on
line diff
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/.hgignore Tue Mar 06 17:37:32 2007 +0100 @@ -0,0 +1,3 @@ +.*\.pyc$ +.*~$ +.*\.cfg$
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/COPYING Tue Mar 06 17:37:32 2007 +0100 @@ -0,0 +1,340 @@ + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1989, 1991 Free Software Foundation, Inc. + 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Library General Public License instead.) You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. + + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". + +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. + + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. + + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. + + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. + +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: + + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, + + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) + +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. + + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. + + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. + + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. + + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. + + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + <one line to give the program's name and a brief idea of what it does.> + Copyright (C) <year> <name of author> + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + + +Also add information on how to contact you by electronic and paper mail. + +If the program is interactive, make it output a short notice like this +when it starts in an interactive mode: + + Gnomovision version 69, Copyright (C) year name of author + Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, the commands you use may +be called something other than `show w' and `show c'; they could even be +mouse-clicks or menu items--whatever suits your program. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the program + `Gnomovision' (which makes passes at compilers) written by James Hacker. + + <signature of Ty Coon>, 1 April 1989 + Ty Coon, President of Vice + +This General Public License does not permit incorporating your program into +proprietary programs. If your program is a subroutine library, you may +consider it more useful to permit linking proprietary applications with the +library. If this is what you want to do, use the GNU Library General +Public License instead of this License.
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/runtreepkg.py Tue Mar 06 17:37:32 2007 +0100 @@ -0,0 +1,44 @@ +#! /usr/bin/python2.4 +# Copyright (C) 2007 by Intevation GmbH +# Authors: +# Bernhard Herzog <bh@intevation.de> +# +# This program is free software under the GPL (>=v2) +# Read the file COPYING coming with the software for details. + +"""Starts the tree packager""" + +import os +import logging +from optparse import OptionParser + +from treepkg.packager import AssemblyLine, Packager +from treepkg.readconfig import read_config + +def initialize_logging(): + """Initializes the logging system""" + root = logging.getLogger() + root.setLevel(logging.DEBUG) + hdlr = logging.StreamHandler() + fmt = logging.Formatter("%(asctime)s %(levelname)s %(message)s") + hdlr.setFormatter(fmt) + root.addHandler(hdlr) + +def parse_commandline(): + parser = OptionParser() + parser.set_defaults(config_file=os.path.join(os.path.dirname(__file__), + "treepkg.cfg")) + parser.add_option("--config-file") + return parser.parse_args() + +def main(): + options, args = parse_commandline() + + initialize_logging() + + treepkg_opts, assembly_line_opts = read_config(options.config_file) + packager = Packager([AssemblyLine(**opts) for opts in assembly_line_opts], + **treepkg_opts) + packager.run() + +main()
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/treepkg/__init__.py Tue Mar 06 17:37:32 2007 +0100 @@ -0,0 +1,6 @@ +# Copyright (C) 2007 by Intevation GmbH +# Authors: +# Bernhard Herzog <bh@intevation.de> +# +# This program is free software under the GPL (>=v2) +# Read the file COPYING coming with the software for details.
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/treepkg/packager.py Tue Mar 06 17:37:32 2007 +0100 @@ -0,0 +1,340 @@ +# Copyright (C) 2007 by Intevation GmbH +# Authors: +# Bernhard Herzog <bh@intevation.de> +# +# This program is free software under the GPL (>=v2) +# Read the file COPYING coming with the software for details. + +"""Classes to automatically build debian packages from subversion checkouts""" + +import os +import time +import re +import logging +import shutil +import traceback + +import util +import subversion +import run + + +def _filenameproperty(relative_dir): + def get(self): + return os.path.join(self.base_dir, relative_dir) + return property(get) + + +class SourcePackager(object): + + def __init__(self, plant, status, work_dir, src_dir, revision): + self.plant = plant + self.status = status + self.work_dir = work_dir + self.src_dir = src_dir + self.revision = revision + self.enterprise_version = (time.strftime("%Y%m%d", time.localtime()) \ + + "." + str(self.revision)) + + def kdepim_version(self, directory): + """Determine the kdepim version. + + The version is taken from the kdepim.lsm file in the plants + checkout dir. + """ + return util.extract_lsm_version(os.path.join(directory, "kdepim.lsm")) + + def determine_package_version(self, directory): + enterprise_version = self.enterprise_version + kdepimversion = self.kdepim_version(directory) + version_template = "%(kdepimversion)s.enterprise.0.%(enterprise_version)s" + return version_template % locals() + + def export_sources(self): + temp_dir = os.path.join(self.work_dir, "temp") + self.plant.export_sources(temp_dir) + + pkgbaseversion = self.determine_package_version(temp_dir) + pkgbasedir = os.path.join(self.work_dir, "kdepim-" + pkgbaseversion) + + os.rename(temp_dir, pkgbasedir) + return pkgbaseversion, pkgbasedir + + + def update_version_numbers(self, pkgbasedir): + versionstring = "(enterprise %s)" % self.enterprise_version + for versionfile in ["kmail/kmversion.h", "kontact/src/main.cpp", + "korganizer/version.h"]: + filename = os.path.join(pkgbasedir, versionfile) + patched = re.sub("\(enterprise ([^)]*)\)", versionstring, + open(filename).read()) + f = open(filename, "w") + f.write(patched) + f.close() + + def create_tarball(self, tarballname, workdir, basedir): + logging.info("Creating tarball %r", tarballname) + run.call(["tar", "czf", tarballname, "-C", workdir, basedir]) + + def copy_debian_directory(self, pkgbasedir, pkgbaseversion, changemsg): + debian_dir = os.path.join(pkgbasedir, "debian") + changelog = os.path.join(debian_dir, "changelog") + + self.plant.copy_debian_directory(debian_dir) + + logging.info("Updating %r", changelog) + oldversion = util.debian_changelog_version(changelog) + if ":" in oldversion: + oldversionprefix = oldversion.split(":")[0] + ":" + else: + oldversionprefix = "" + run.call(["debchange", "-c", changelog, + "-v", oldversionprefix + pkgbaseversion + "-kk1", + changemsg], + env=self.plant.debian_environment()) + + + def create_source_package(self, pkgbasedir, origtargz): + logging.info("Creating new source package") + run.call(["dpkg-source", "-b", os.path.basename(pkgbasedir), + os.path.basename(origtargz)], + cwd=os.path.dirname(pkgbasedir), + suppress_output=True, + env=self.plant.debian_environment()) + + def package(self): + util.ensure_directory(self.work_dir) + try: + self.status.set("creating_source_package") + pkgbaseversion, pkgbasedir = self.export_sources() + self.update_version_numbers(pkgbasedir) + + pkgbasename = "kdepim_" + pkgbaseversion + origtargz = os.path.join(self.work_dir, + pkgbasename + ".orig.tar.gz") + self.create_tarball(origtargz, self.work_dir, + os.path.basename(pkgbasedir)) + + changemsg = ("Update to SVN enterprise branch rev. %d" + % (self.revision,)) + self.copy_debian_directory(pkgbasedir, pkgbaseversion, + changemsg) + + self.create_source_package(pkgbasedir, origtargz) + + logging.info("Moving source package to %r", self.src_dir) + util.ensure_directory(self.src_dir) + for filename in [filename for filename in os.listdir(self.work_dir) + if filename.startswith(pkgbasename)]: + os.rename(os.path.join(self.work_dir, filename), + os.path.join(self.src_dir, filename)) + self.status.set("source_package_created") + finally: + logging.info("Removing workdir %r", self.work_dir) + shutil.rmtree(self.work_dir) + + +class BinaryPackager(object): + + def __init__(self, plant, status, binary_dir, dsc_file, logfile): + self.plant = plant + self.status = status + self.binary_dir = binary_dir + self.dsc_file = dsc_file + self.logfile = logfile + + def package(self): + self.status.set("creating_binary_package") + util.ensure_directory(self.binary_dir) + logging.info("Building binary package; loging to %r", self.logfile) + cmd = [] + if self.plant.root_cmd: + cmd.append(self.plant.root_cmd) + run.call(cmd + ["/usr/sbin/pbuilder", "build", + "--logfile", self.logfile, + "--buildresult", self.binary_dir, + self.dsc_file], + suppress_output=True) + self.status.set("binary_package_created") + + +class RevisionPackager(object): + + def __init__(self, plant, revision): + self.plant = plant + self.revision = revision + self.base_dir = self.plant.pkg_dir_for_revision(self.revision, 1) + self.status = util.StatusFile(os.path.join(self.base_dir, "status")) + + work_dir = _filenameproperty("work") + binary_dir = _filenameproperty("binary") + src_dir = _filenameproperty("src") + + def find_dsc_file(self): + for filename in os.listdir(self.src_dir): + if filename.endswith(".dsc"): + return os.path.join(self.src_dir, filename) + return None + + def package(self): + try: + src_packager = SourcePackager(self.plant, self.status, + self.work_dir, self.src_dir, + self.revision) + src_packager.package() + + dsc_file = self.find_dsc_file() + if dsc_file is None: + raise RuntimeError("Cannot find dsc File in %r" % self.src_dir) + + bin_packager = BinaryPackager(self.plant, self.status, + self.binary_dir, dsc_file, + os.path.join(self.base_dir, + "build.log")) + bin_packager.package() + except: + self.status.set("failed", traceback.format_exc()) + raise + + def remove_package_dir(self): + logging.info("Removing pkgdir %r", self.base_dir) + shutil.rmtree(self.base_dir) + + +class AssemblyLine(object): + + def __init__(self, name, base_dir, svn_url, root_cmd, deb_email, + deb_fullname): + self.name = name + self.base_dir = base_dir + self.svn_url = svn_url + self.root_cmd = root_cmd + self.deb_email = deb_email + self.deb_fullname = deb_fullname + self.pkg_dir_template = "%(revision)d-%(increment)d" + self.pkg_dir_regex \ + = re.compile(r"(?P<revision>[0-9]+)-(?P<increment>[0-9]+)$") + + checkout_dir = _filenameproperty("checkout") + debian_dir = _filenameproperty("debian") + pkg_dir = _filenameproperty("pkg") + + def pkg_dir_for_revision(self, revision, increment): + return os.path.join(self.pkg_dir, + self.pkg_dir_template % locals()) + + def last_changed_revision(self): + rev1 = subversion.last_changed_revision(self.checkout_dir) + rev2 = subversion.last_changed_revision(os.path.join(self.checkout_dir, + "admin")) + return max(rev1, rev2) + + def last_packaged_revision(self): + """Returns the revision number of the highest packaged revision. + + If the revision cannot be determined because no already packaged + revisions can be found, the function returns -1. + """ + revisions = [-1] + if os.path.exists(self.pkg_dir): + for filename in os.listdir(self.pkg_dir): + match = self.pkg_dir_regex.match(filename) + if match: + revisions.append(int(match.group("revision"))) + return max(revisions) + + def debian_source(self): + return util.extract_value_for_key(open(os.path.join(self.debian_dir, + "control")), + "Source:") + + def update_checkout(self): + """Updates the working copy of self.svn_url in self.checkout_dir. + + If self.checkout_dir doesn't exist yet, self.svn_url is checked + out into that directory. + """ + localdir = self.checkout_dir + if os.path.exists(localdir): + logging.info("Updating the working copy in %r", localdir) + subversion.update(localdir) + else: + logging.info("The working copy in %r doesn't exist yet." + " Checking out fromo %r", localdir, + self.svn_url) + subversion.checkout(self.svn_url, localdir) + + def export_sources(self, to_dir): + logging.info("Exporting sources for tarball to %r", to_dir) + subversion.export(self.checkout_dir, to_dir) + # some versions of svn (notably version 1.4.2 shipped with etch) + # do export externals such as the admin subdirectory. We may + # have to do that in an extra step. + admindir = os.path.join(to_dir, "admin") + if not os.path.isdir(admindir): + subversion.export(os.path.join(self.checkout_dir, "admin"), + admindir) + + def copy_debian_directory(self, to_dir): + logging.info("Copying debian directory to %r", to_dir) + shutil.copytree(self.debian_dir, to_dir) + + def debian_environment(self): + """Returns the environment variables for the debian commands""" + env = os.environ.copy() + env["DEBFULLNAME"] = self.deb_fullname + env["DEBEMAIL"] = self.deb_email + return env + + def package_if_updated(self): + """Checks if the checkout changed and returns a new packager if so""" + self.update_checkout() + current_revision = self.last_changed_revision() + logging.info("New revision is %d", current_revision) + previous_revision = self.last_packaged_revision() + logging.info("Previously packaged revision was %d", previous_revision) + if current_revision > previous_revision: + logging.info("New revision is not packaged yet") + return RevisionPackager(self, current_revision) + else: + logging.info("New revision already packaged.") + + + +class Packager(object): + + def __init__(self, assembly_lines, check_interval): + self.assembly_lines = assembly_lines + self.check_interval = check_interval + + def run(self): + """Runs the plant indefinitely""" + logging.info("Packager start. Will check every %d seconds", + self.check_interval) + last_check = -1 + while 1: + now = time.time() + if now > last_check + self.check_interval: + self.check_assembly_lines() + last_check = now + next_check = now + self.check_interval + to_sleep = next_check - time.time() + if to_sleep > 0: + logging.info("Next check at %s", + time.strftime("%Y-%m-%d %H:%M:%S", + time.localtime(next_check))) + time.sleep(to_sleep) + else: + logging.info("Next check now") + + def check_assembly_lines(self): + logging.info("Checking assembly lines") + for line in self.assembly_lines: + try: + packager = line.package_if_updated() + if packager: + packager.package() + except: + logging.exception("An error occurred while" + " checking assembly line %r", line.name) + logging.info("Checked all assembly lines")
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/treepkg/readconfig.py Tue Mar 06 17:37:32 2007 +0100 @@ -0,0 +1,50 @@ +# Copyright (C) 2007 by Intevation GmbH +# Authors: +# Bernhard Herzog <bh@intevation.de> +# +# This program is free software under the GPL (>=v2) +# Read the file COPYING coming with the software for details. + +"""Reads the configuration file""" + +import sys +from ConfigParser import SafeConfigParser, NoOptionError + + +assembly_line_keys = ["name", "base_dir", "svn_url", "root_cmd", + "deb_email", "deb_fullname"] + +defaults = dict(root_cmd="sudo") + + + +def read_config(filename): + """Reads the packagemill configuration from filename. + + The return value is a list of (name, option) pairs, one for each + assembly line of the mill. + """ + assembly_lines = [] + parser = SafeConfigParser(defaults) + parser.read([filename]) + for section in parser.sections(): + if section.startswith("pkg_"): + pkg_defaults = dict(name=section[4:]) + options = {} + for key in assembly_line_keys: + try: + options[key] = parser.get(section, key, vars=pkg_defaults) + except NoOptionError: + print >>sys.stderr, "%s: Missing option %r in section %r" \ + % (filename, key, section) + sys.exit(1) + assembly_lines.append(options) + + treepkg_opts = dict(check_interval=parser.getint("treepkg", + "check_interval")) + + return treepkg_opts, assembly_lines + + +if __name__ == "__main__": + print read_config(sys.argv[1])
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/treepkg/run.py Tue Mar 06 17:37:32 2007 +0100 @@ -0,0 +1,49 @@ +# Copyright (C) 2007 by Intevation GmbH +# Authors: +# Bernhard Herzog <bh@intevation.de> +# +# This program is free software under the GPL (>=v2) +# Read the file COPYING coming with the software for details. + +"""Helper functions to run subprocesses in various ways""" + +import os +import subprocess + + +class SubprocessError(EnvironmentError): + + def __init__(self, command, returncode): + EnvironmentError.__init__(self, + "Command %r finished with return code %d" + % (command, returncode)) + self.returncode = returncode + + +def call(command, suppress_output=False, **kw): + """Run command as a subprocess and wait until it is finished. + + The command should be given as a list of strings to avoid problems + with shell quoting. If the command exits with a return code other + than 0, a SubprocessError is raised. + """ + if suppress_output: + kw["stdout"] = open(os.devnull, "w") + ret = subprocess.call(command, **kw) + if ret != 0: + raise SubprocessError(command, ret) + + +def capture_output(command, **kw): + """Return the stdout and stderr of the command as a string + + The command should be given as a list of strings to avoid problems + with shell quoting. If the command exits with a return code other + than 0, a SubprocessError is raised. + """ + proc = subprocess.Popen(command, stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, **kw) + output = proc.communicate()[0] + if proc.returncode != 0: + raise SubprocessError(command, proc.returncode) + return output
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/treepkg/subversion.py Tue Mar 06 17:37:32 2007 +0100 @@ -0,0 +1,37 @@ +# Copyright (C) 2007 by Intevation GmbH +# Authors: +# Bernhard Herzog <bh@intevation.de> +# +# This program is free software under the GPL (>=v2) +# Read the file COPYING coming with the software for details. + +"""Collection of subversion utility functions""" + +import os + +import run +from util import extract_value_for_key + + +def checkout(url, localdir): + """Runs svn to checkout the repository at url into the localdir""" + run.call(["svn", "checkout", "-q", url, localdir]) + +def update(localdir): + """Runs svn update on the localdir""" + run.call(["svn", "update", "-q", localdir]) + +def export(src, dest): + """Runs svn export src dest""" + run.call(["svn", "export", "-q", src, dest]) + +def last_changed_revision(svn_working_copy): + """return the last changed revision of an SVN working copy as an int""" + # Make sure we run svn under the C locale to avoid localized + # messages + env = os.environ.copy() + env["LANG"] = "C" + + output = run.capture_output(["svn", "info", svn_working_copy], env=env) + return int(extract_value_for_key(output.splitlines(), + "Last Changed Rev:"))
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/treepkg/util.py Tue Mar 06 17:37:32 2007 +0100 @@ -0,0 +1,73 @@ +# Copyright (C) 2007 by Intevation GmbH +# Authors: +# Bernhard Herzog <bh@intevation.de> +# +# This program is free software under the GPL (>=v2) +# Read the file COPYING coming with the software for details. + +"""Collection of functions that didn't fit elsewhere""" + +import os +import tempfile + +import run + + +def extract_value_for_key(lines, key): + """Parses a sequence of strings for a key and returns the associated value + + The function determines the first string in lines that starts with + key. It returns the rest of the lines stripped of leading and + trailing whitespace. + """ + for line in lines: + if line.startswith(key): + return line[len(key):].strip() + +def extract_lsm_version(lsm_file): + return extract_value_for_key(open(lsm_file), "Version:") + +def debian_changelog_version(changelog): + """Returns the newest version in a debian changelog.""" + output = run.capture_output(["dpkg-parsechangelog", "-l" + changelog]) + return extract_value_for_key(output.splitlines(), "Version:") + + +def ensure_directory(directory): + """Creates directory and all its parents. + + Unlike os.makedirs, this function doesn't throw an exception + """ + if not os.path.isdir(directory): + os.makedirs(directory) + + +def writefile(filename, contents): + """Write contents to filename in an atomic way. + + The contents are first written to a temporary file in the same + directory as filename. Once the contents are written, the temporary + file is closed and renamed to filename. + """ + dirname, basename = os.path.split(filename) + fileno, tempname = tempfile.mkstemp("", basename, dirname) + try: + os.write(fileno, contents) + if not contents.endswith("\n"): + os.write(fileno, "\n") + os.close(fileno) + os.rename(tempname, filename) + finally: + if os.path.exists(tempname): + os.remove(tempname) + + +class StatusFile(object): + + def __init__(self, filename): + assert os.path.isabs(filename) + self.filename = filename + + def set(self, status, extra=""): + ensure_directory(os.path.dirname(self.filename)) + writefile(self.filename, status + "\n" + extra)