view test/test_listpackages.py @ 440:383d23c6bfde

Merged revisions 346-386 via svnmerge from svn+ssh://svn.wald.intevation.org/treepkg/branches/treepkg-status ........ r346 | bricks | 2010-07-07 17:40:48 +0200 (Mi, 07 Jul 2010) | 2 lines added classed for treepkg status xml generation ........ r347 | bricks | 2010-07-08 12:07:39 +0200 (Do, 08 Jul 2010) | 10 lines renamed new status dir to info because of a naming conflict with status.py let the user specify a treepkg name in the config the name is propagated to PackagerGroup [treepkg] name: <treepkgname> becomes: pg = PackagerGroup(...) pg.name ........ r348 | bricks | 2010-07-08 15:25:07 +0200 (Do, 08 Jul 2010) | 3 lines moved test file to test main dir fixed missing : after def statement ........ r349 | bricks | 2010-07-08 17:59:52 +0200 (Do, 08 Jul 2010) | 2 lines added treepkg_dir andd tracks_dir attributes to PackageGroup ........ r350 | bricks | 2010-07-08 18:00:32 +0200 (Do, 08 Jul 2010) | 2 lines changed <millpath> to <trackspath> ........ r351 | bricks | 2010-07-08 18:08:39 +0200 (Do, 08 Jul 2010) | 2 lines check if config file exists to get usefull error output ........ r352 | bricks | 2010-07-08 18:11:13 +0200 (Do, 08 Jul 2010) | 2 lines write first info about tracks ........ r353 | bricks | 2010-07-09 12:20:49 +0200 (Fr, 09 Jul 2010) | 2 lines get all log files ........ r354 | bricks | 2010-07-09 14:42:42 +0200 (Fr, 09 Jul 2010) | 2 lines added md5sum function ........ r355 | bricks | 2010-07-09 15:15:15 +0200 (Fr, 09 Jul 2010) | 4 lines implemented nearly all info only arch and os info are missing ........ r356 | bricks | 2010-07-09 16:19:17 +0200 (Fr, 09 Jul 2010) | 2 lines Bugfix: source was not defined ........ r357 | bricks | 2010-07-09 16:44:52 +0200 (Fr, 09 Jul 2010) | 3 lines fixed some typos added missing add_checksum methond ........ r358 | bricks | 2010-07-09 17:06:15 +0200 (Fr, 09 Jul 2010) | 2 lines fixed status line for revision ........ r359 | bricks | 2010-07-12 14:10:34 +0200 (Mo, 12 Jul 2010) | 2 lines let the user specify num revisions that should be shown in the info ........ r360 | bricks | 2010-07-12 14:21:39 +0200 (Mo, 12 Jul 2010) | 2 lines missed in last commit ........ r361 | bricks | 2010-07-12 17:06:22 +0200 (Mo, 12 Jul 2010) | 2 lines add os config statement ........ r362 | bricks | 2010-07-12 17:47:42 +0200 (Mo, 12 Jul 2010) | 3 lines moved packages and logs to revision tag implemented os and arch info ........ r363 | bricks | 2010-07-13 14:23:28 +0200 (Di, 13 Jul 2010) | 2 lines updated test readconfig for os config variable ........ r364 | bricks | 2010-07-13 14:24:14 +0200 (Di, 13 Jul 2010) | 2 lines moved common publish functions to a seperate module ........ r365 | bricks | 2010-07-13 16:26:11 +0200 (Di, 13 Jul 2010) | 2 lines moved arch info to each package info ........ r366 | bricks | 2010-07-13 18:46:17 +0200 (Di, 13 Jul 2010) | 2 lines inital checkin for new publishpackages processing ........ r367 | bricks | 2010-07-21 17:52:15 +0200 (Mi, 21 Jul 2010) | 2 lines implemented parsing from xml string ........ r368 | bricks | 2010-07-22 12:17:42 +0200 (Do, 22 Jul 2010) | 2 lines copy binary-all packages in all binary-xyz dirs ........ r369 | bricks | 2010-07-22 12:26:40 +0200 (Do, 22 Jul 2010) | 2 lines fixed small bugs ........ r370 | bricks | 2010-07-22 13:02:36 +0200 (Do, 22 Jul 2010) | 2 lines added a new root element which is called TreepkgRoot for Treepkg Info classes ........ r371 | bricks | 2010-07-22 13:09:52 +0200 (Do, 22 Jul 2010) | 3 lines fixed some small issues remember: run pychecker BEFORE the commit ........ r372 | bricks | 2010-07-22 18:01:43 +0200 (Do, 22 Jul 2010) | 2 lines improved publishdebianpackages and fixed a lot of bugs in the xml info parsing part ........ r373 | bricks | 2010-07-23 18:40:38 +0200 (Fr, 23 Jul 2010) | 2 lines implemented CacheDb to store copied Packages from build host ........ r374 | bricks | 2010-07-26 10:28:48 +0200 (Mo, 26 Jul 2010) | 2 lines moved data handling in publishdebianpackages into seperate module ........ r375 | bricks | 2010-07-26 11:42:07 +0200 (Mo, 26 Jul 2010) | 2 lines fixed some typos ........ r376 | bricks | 2010-07-26 12:12:40 +0200 (Mo, 26 Jul 2010) | 2 lines first version of incremental copying debian packages to cachedir ........ r377 | bricks | 2010-07-26 14:58:50 +0200 (Mo, 26 Jul 2010) | 2 lines incremental copying only changed debian packages ........ r378 | bricks | 2010-07-26 15:39:10 +0200 (Mo, 26 Jul 2010) | 3 lines use md5 instead of hashlib module (md5 is deprecated in favour of hashlib in python 2.5) ........ r379 | bricks | 2010-07-26 15:46:19 +0200 (Mo, 26 Jul 2010) | 2 lines make treepkg compatible to python 2.4 ........ r380 | bricks | 2010-07-26 15:50:49 +0200 (Mo, 26 Jul 2010) | 2 lines fixed last commit (db instead of sqlite3) ........ r381 | bricks | 2010-07-27 10:54:15 +0200 (Di, 27 Jul 2010) | 2 lines fixed wrong import statement ........ r382 | bricks | 2010-07-27 10:54:34 +0200 (Di, 27 Jul 2010) | 2 lines removed unnecessary debug output ........ r383 | bricks | 2010-07-27 18:28:22 +0200 (Di, 27 Jul 2010) | 2 lines check if upload hook is empty ........ r384 | bricks | 2010-07-28 09:35:21 +0200 (Mi, 28 Jul 2010) | 2 lines run rsync only an arch dirs that are present in the current treepkg ........ r385 | bricks | 2010-07-28 09:35:54 +0200 (Mi, 28 Jul 2010) | 2 lines add helper tool to list content of a cache db ........ r386 | bricks | 2010-07-28 10:33:48 +0200 (Mi, 28 Jul 2010) | 3 lines fixed a typo regex fails if option is empty ........
author Bjoern Ricks <bricks@intevation.de>
date Fri, 06 Aug 2010 13:28:47 +0000
parents f06f707d9fda
children e5b66539f893
line wrap: on
line source
# Copyright (C) 2009 by Intevation GmbH
# Authors:
# Bernhard Herzog <bh@intevation.de>
#
# This program is free software under the GPL (>=v2)
# Read the file COPYING coming with the software for details.


import sys
import os
import difflib
import unittest
from datetime import datetime, timedelta

from filesupport import FileTestMixin

from treepkg.run import call, capture_output, SubprocessError
from treepkg.status import RevisionStatus


def package_track_files(track, version, binaries, revisions):
    src_templates = ["%s_%s.%%s%s" % (track, version, suffix)
                     for suffix in [".orig.tar.gz", "-1.diff.gz", "-1.dsc"]]
    bin_templates = ["%s_%s.%%s-1_i386.deb" % (basename, version)
                     for basename in binaries]

    start_time = datetime.now()
    build_duration = timedelta(minutes=30)
    check_interval = timedelta(minutes=60)

    package_dirs = []
    for revision in revisions:
        status = RevisionStatus(None)
        if len(revision) == 2:
            rev, rules_rev = revision
            status.binary_package_created()
        else:
            rev, rules_rev, status_name = revision
            getattr(status, status_name)()

        status.start = start_time
        status.stop = start_time + build_duration
        start_time += check_interval

        if status.status.finished and not status.status.error:
            src_files = [(template % (rev,), "") for template in src_templates]
            bin_files = [(template % (rev,), "") for template in bin_templates]
        else:
            src_files = []
            bin_files = []

        package_dirs.append(("%s-%s" % (rev, rules_rev),
                             [("status", "".join(status.serialize())),
                              ("src", src_files),
                              ("binary", bin_files)]))
    return (track,
            [("pkg",
              package_dirs)])


class ListPackagesTest(unittest.TestCase, FileTestMixin):

    treepkg_cfg_template = """\
[DEFAULT]
tracks_dir: %(tracksdir)s
root_cmd: true
builderconfig:
deb_email: packager@example.com
deb_fullname: Sample Packager
pkg_revision_template: treepkg%%%%(pkg_revision)s
handle_dependencies: False

[treepkg]
check_interval: 3600
instructions_file:

[pkg_kdepim]
svn_url: svn://anonsvn.kde.org/home/kde/branches/kdepim/enterprise/kdepim
base_dir: %%(tracks_dir)s/kdepim
packager_class: recipes.kde_enterprise_3_5.kdepim

[pkg_kdepimlibs]
svn_url: svn://anonsvn.kde.org/home/kde/branches/kdepim/enterprise/kdepimlibs
base_dir: %%(tracks_dir)s/kdepimlibs
packager_class: recipes.kde_enterprise_3_5.kdepim
"""

    track_files = []

    def setUp(self):
        self.bindir = os.path.join(os.path.dirname(__file__),
                                   os.pardir, "bin")
        self.listpackages = os.path.join(self.bindir, "listpackages.py")
        self.tracksdir = self.create_files("tracks", self.track_files)
        config = self.treepkg_cfg_template % dict(tracksdir=self.tracksdir)
        self.config_file = self.create_temp_file("treepkg.cfg", config)
        self.base_command = [sys.executable, self.listpackages,
                             "--config-file=" + self.config_file]

    def run_command(self, extra_args, expected_exit_code=0):
        cmd = self.base_command + list(extra_args)
        try:
            output = capture_output(cmd)
        except SubprocessError, exc:
            if expected_exit_code != exc.returncode:
                raise
            output = exc.output
        return output

    def check_cmd(self, args, expected):
        expected_lines = [os.path.join(self.tracksdir, filename) + "\n"
                          for filename in expected.splitlines()]
        try:
            output_lines = [line + "\n"
                            for line in self.run_command(args).splitlines()]
        except SubprocessError, exc:
            print >>sys.stderr, "Unexpected output:", exc.output
            raise

        # the order of the listed packages doesn't matter, so sort the
        # lists before comparing them
        expected_lines.sort()
        output_lines.sort()

        if expected_lines != output_lines:
            diff = difflib.unified_diff(expected_lines, output_lines,
                                        "expected", "found")
            raise self.failureException("\n" + "".join(diff))


class TestListPackagesNoPackages(ListPackagesTest):

    track_files = []

    def test_listpackages_noargs(self):
        # if nothing has been packaged yet, the output is simply empty
        self.check_cmd(["--source"], "")

    def test_listpackages_revision(self):
        # if nothing has been packaged yet, the output is simply empty
        self.assertEquals(self.run_command(["--source", "--revision=12345",
                                            "--track=kdepim"],
                                           expected_exit_code=1),
                          "No revision 12345\n")

    def test_listpackages_revision_rulesrev(self):
        # if nothing has been packaged yet, the output is simply empty
        self.assertEquals(self.run_command(["--source", "--revision=12345-321",
                                            "--track=kdepim"],
                                           expected_exit_code=1),
                          "No revision 12345-321\n")

class TestListPackagesTrackNoRevision(ListPackagesTest):

    track_files = [package_track_files("kdepim", "1.0.1",
                                       ["kdepim", "kleopatra"],
                                       [(704195, 31), (702432, 47)])]

    def test_listpackages_source(self):
        self.check_cmd(["--track=kdepim", "--source"],
                       """\
kdepim/pkg/702432-47/src/kdepim_1.0.1.702432-1.diff.gz
kdepim/pkg/702432-47/src/kdepim_1.0.1.702432-1.dsc
kdepim/pkg/702432-47/src/kdepim_1.0.1.702432.orig.tar.gz
""")

    def test_listpackages_binary(self):
        self.check_cmd(["--track=kdepim", "--binary"],
                       """\
kdepim/pkg/702432-47/binary/kdepim_1.0.1.702432-1_i386.deb
kdepim/pkg/702432-47/binary/kleopatra_1.0.1.702432-1_i386.deb
""")


class TestListPackagesTrackNoRevisionWithIncompleteBuilds(ListPackagesTest):

    track_files = [package_track_files("kdepim", "1.0.1",
                                       ["kdepim", "kleopatra"],
                                       [(702432, 47),
                                        (704195, 31, "error")])]

    def test_listpackages_source(self):
        self.check_cmd(["--track=kdepim", "--source"],
                       """\
kdepim/pkg/702432-47/src/kdepim_1.0.1.702432-1.diff.gz
kdepim/pkg/702432-47/src/kdepim_1.0.1.702432-1.dsc
kdepim/pkg/702432-47/src/kdepim_1.0.1.702432.orig.tar.gz
""")

    def test_listpackages_binary(self):
        self.check_cmd(["--track=kdepim", "--binary"],
                       """\
kdepim/pkg/702432-47/binary/kdepim_1.0.1.702432-1_i386.deb
kdepim/pkg/702432-47/binary/kleopatra_1.0.1.702432-1_i386.deb
""")


class TestListPackagesNoTrackNoRevision(ListPackagesTest):

    track_files = [package_track_files("kdepim", "1.0.1",
                                       ["kdepim", "kleopatra"],
                                       [(702432, 47),
                                        (704195, 31)]),
                   package_track_files("kdepimlibs", "1.0.1",
                                       ["kdepimlibs"],
                                       [(803691, 4321),
                                        (803692, 4322)]),
                   ]

    def test_listpackages_source(self):
        self.check_cmd(["--source"],
                       """\
kdepim/pkg/704195-31/src/kdepim_1.0.1.704195-1.diff.gz
kdepim/pkg/704195-31/src/kdepim_1.0.1.704195-1.dsc
kdepim/pkg/704195-31/src/kdepim_1.0.1.704195.orig.tar.gz
kdepimlibs/pkg/803692-4322/src/kdepimlibs_1.0.1.803692-1.diff.gz
kdepimlibs/pkg/803692-4322/src/kdepimlibs_1.0.1.803692-1.dsc
kdepimlibs/pkg/803692-4322/src/kdepimlibs_1.0.1.803692.orig.tar.gz""")

    def test_listpackages_binary(self):
        self.check_cmd(["--binary"],
                       """\
kdepim/pkg/704195-31/binary/kdepim_1.0.1.704195-1_i386.deb
kdepim/pkg/704195-31/binary/kleopatra_1.0.1.704195-1_i386.deb
kdepimlibs/pkg/803692-4322/binary/kdepimlibs_1.0.1.803692-1_i386.deb""")


class TestListPackagesNNewest(ListPackagesTest):

    track_files = [package_track_files("kdepim", "1.0.1",
                                       ["kdepim", "kleopatra"],
                                       [(701865, 28),
                                        (700345, 28),
                                        (702432, 28),
                                        (704195, 31, "error")]),
                   package_track_files("kdepimlibs", "1.0.1",
                                       ["kdepimlibs"],
                                       [(700298, 34),
                                        (700345, 34),
                                        (701866, 34, "error"),
                                        (701866, 35),
                                        (702436, 35)]),
                   ]

    def test_listpackages_source(self):
        self.check_cmd(["--source", "--newest=2"],
                       """\
kdepim/pkg/700345-28/src/kdepim_1.0.1.700345-1.diff.gz
kdepim/pkg/700345-28/src/kdepim_1.0.1.700345-1.dsc
kdepim/pkg/700345-28/src/kdepim_1.0.1.700345.orig.tar.gz
kdepim/pkg/702432-28/src/kdepim_1.0.1.702432-1.diff.gz
kdepim/pkg/702432-28/src/kdepim_1.0.1.702432-1.dsc
kdepim/pkg/702432-28/src/kdepim_1.0.1.702432.orig.tar.gz
kdepimlibs/pkg/701866-35/src/kdepimlibs_1.0.1.701866-1.diff.gz
kdepimlibs/pkg/701866-35/src/kdepimlibs_1.0.1.701866-1.dsc
kdepimlibs/pkg/701866-35/src/kdepimlibs_1.0.1.701866.orig.tar.gz
kdepimlibs/pkg/702436-35/src/kdepimlibs_1.0.1.702436-1.diff.gz
kdepimlibs/pkg/702436-35/src/kdepimlibs_1.0.1.702436-1.dsc
kdepimlibs/pkg/702436-35/src/kdepimlibs_1.0.1.702436.orig.tar.gz""")

    def test_listpackages_binary(self):
        self.check_cmd(["--binary", "--newest=3"],
                       """\
kdepim/pkg/700345-28/binary/kdepim_1.0.1.700345-1_i386.deb
kdepim/pkg/700345-28/binary/kleopatra_1.0.1.700345-1_i386.deb
kdepim/pkg/701865-28/binary/kdepim_1.0.1.701865-1_i386.deb
kdepim/pkg/701865-28/binary/kleopatra_1.0.1.701865-1_i386.deb
kdepim/pkg/702432-28/binary/kdepim_1.0.1.702432-1_i386.deb
kdepim/pkg/702432-28/binary/kleopatra_1.0.1.702432-1_i386.deb
kdepimlibs/pkg/700345-34/binary/kdepimlibs_1.0.1.700345-1_i386.deb
kdepimlibs/pkg/701866-35/binary/kdepimlibs_1.0.1.701866-1_i386.deb
kdepimlibs/pkg/702436-35/binary/kdepimlibs_1.0.1.702436-1_i386.deb""")


class TestListPackagesTrackRevision(ListPackagesTest):

    track_files = [package_track_files("kdepim", "1.0.1",
                                       ["kdepim", "kleopatra"],
                                       [(704195, 31), (702432, 47)]),
                   package_track_files("kdepimlibs", "1.0.1",
                                       ["kdepimlibs"],
                                       [(803691, 4321), (803692, 4322)]),
                   ]

    def test_listpackages_source(self):
        self.check_cmd(["--source", "--track=kdepim", "--revision=702432"],
                       """\
kdepim/pkg/702432-47/src/kdepim_1.0.1.702432-1.diff.gz
kdepim/pkg/702432-47/src/kdepim_1.0.1.702432-1.dsc
kdepim/pkg/702432-47/src/kdepim_1.0.1.702432.orig.tar.gz""")

    def test_listpackages_binary(self):
        self.check_cmd(["--binary", "--track=kdepim", "--revision=702432"],
                       """\
kdepim/pkg/702432-47/binary/kdepim_1.0.1.702432-1_i386.deb
kdepim/pkg/702432-47/binary/kleopatra_1.0.1.702432-1_i386.deb""")



class TestListPackagesMultipleRulesRevisions(ListPackagesTest):

    track_files = [package_track_files("kdepim", "1.0.1",
                                       ["kdepim", "kleopatra"],
                                       [(702432, 47),
                                        (704195, 31),
                                        (704195, 33)]),
                   package_track_files("kdepimlibs", "1.0.1",
                                       ["kdepimlibs"],
                                       [(803691, 4321),
                                        (803692, 4322)]),
                   ]

    def test_listpackages_source_no_rev(self):
        self.check_cmd(["--source", "--track=kdepim"],
                       """\
kdepim/pkg/704195-33/src/kdepim_1.0.1.704195-1.diff.gz
kdepim/pkg/704195-33/src/kdepim_1.0.1.704195-1.dsc
kdepim/pkg/704195-33/src/kdepim_1.0.1.704195.orig.tar.gz""")

    def test_listpackages_binary_no_rev(self):
        self.check_cmd(["--binary", "--track=kdepim"],
                       """\
kdepim/pkg/704195-33/binary/kdepim_1.0.1.704195-1_i386.deb
kdepim/pkg/704195-33/binary/kleopatra_1.0.1.704195-1_i386.deb""")

    def test_listpackages_source_and_binary_no_rev(self):
        self.check_cmd(["--source", "--track=kdepim", "--binary"],
                       """\
kdepim/pkg/704195-33/binary/kdepim_1.0.1.704195-1_i386.deb
kdepim/pkg/704195-33/binary/kleopatra_1.0.1.704195-1_i386.deb
kdepim/pkg/704195-33/src/kdepim_1.0.1.704195-1.diff.gz
kdepim/pkg/704195-33/src/kdepim_1.0.1.704195-1.dsc
kdepim/pkg/704195-33/src/kdepim_1.0.1.704195.orig.tar.gz
""")

    def test_listpackages_source_no_rulesrev(self):
        self.check_cmd(["--source", "--track=kdepim", "--revision=704195"],
                       """\
kdepim/pkg/704195-33/src/kdepim_1.0.1.704195-1.diff.gz
kdepim/pkg/704195-33/src/kdepim_1.0.1.704195-1.dsc
kdepim/pkg/704195-33/src/kdepim_1.0.1.704195.orig.tar.gz""")

    def test_listpackages_binary_no_rulesrev(self):
        self.check_cmd(["--binary", "--track=kdepim", "--revision=704195"],
                       """\
kdepim/pkg/704195-33/binary/kdepim_1.0.1.704195-1_i386.deb
kdepim/pkg/704195-33/binary/kleopatra_1.0.1.704195-1_i386.deb""")

    def test_listpackages_source_rulesrev(self):
        self.check_cmd(["--source", "--track=kdepim", "--revision=704195-31"],
                       """\
kdepim/pkg/704195-31/src/kdepim_1.0.1.704195-1.diff.gz
kdepim/pkg/704195-31/src/kdepim_1.0.1.704195-1.dsc
kdepim/pkg/704195-31/src/kdepim_1.0.1.704195.orig.tar.gz""")

    def test_listpackages_binary_rulesrev(self):
        self.check_cmd(["--binary", "--track=kdepim", "--revision=704195-31"],
                       """\
kdepim/pkg/704195-31/binary/kdepim_1.0.1.704195-1_i386.deb
kdepim/pkg/704195-31/binary/kleopatra_1.0.1.704195-1_i386.deb""")

This site is hosted by Intevation GmbH (Datenschutzerklärung und Impressum | Privacy Policy and Imprint)