Mercurial > treepkg
annotate README @ 99:7888fe374e11
Add support for notification mails in case of build errors
This involves a new status field notification_mail to keep track of
whether a notification has been sent for a particular build attempt and
two programs to list the pending notifications and to send the pending
notifications (similar to how the static web pages are published) as
well as the corresponding configuration files.
| author | Bernhard Herzog <bh@intevation.de> |
|---|---|
| date | Tue, 19 Feb 2008 19:19:23 +0000 |
| parents | 9a602d8eaa60 |
| children | 91deff3db739 |
| rev | line source |
|---|---|
| 42 | 1 README for TreePackager |
| 2 ======================= | |
| 3 | |
| 4 | |
| 5 TreePackager is a tool to automatically build debian packages from SVN. | |
| 6 | |
| 7 | |
| 8 Prerequisites | |
| 9 ------------- | |
| 10 | |
| 11 You need the following software to run TreePackager. In the list below, | |
| 12 parentheses contain the name of the corresponding package in Debian Etch | |
| 13 if its not the same as the software. The version required is usually | |
| 14 the one from debian etch. | |
| 15 | |
| 16 Python 2.4 (python2.4-minimal) | |
| 17 Debian devscripts (devscripts) | |
| 18 subversion | |
| 19 pbuilder | |
| 20 sudo | |
| 21 bzip2 | |
| 22 | |
| 23 For the web front-end you also need the following software: | |
| 24 | |
| 25 Genshi (python-genshi) | |
| 26 CherryPy (python-cherrypy) | |
| 27 | |
| 28 Some of the packagers require additional software. The KDEPIM | |
| 29 enterprise branch packagers require the following additional software: | |
| 30 | |
| 31 autoconf2.13 | |
| 32 automake1.9 | |
| 33 | |
| 34 | |
| 35 Installation | |
| 36 ------------ | |
| 37 | |
| 38 You can run the tree packager itself directly from the source tree. | |
| 39 However, you need to configure it first and setup pbuilder. | |
| 40 | |
| 41 | |
| 42 Configuration | |
| 43 ------------- | |
| 44 | |
| 45 To understand the configuration, first a few notes about the | |
| 46 architecture of TreePackager. The TreePackager consist of one program | |
| 47 that periodically updates svn working directories and if something has | |
| 48 changed, builds a new debian package from the working directory. The | |
| 49 program should run as a normal user. The sample configuration assumes | |
| 50 that it's the user "builder" with a home directory "/home/builder". The | |
| 51 default configuration manages a directory tree under | |
| 52 "/home/builder/enterprise". | |
| 53 | |
| 54 The binary packages are built with pbuilder. Because pbuilder uses a | |
| 55 chroot environment to build the packages, it must be run as root. The | |
| 56 tree packager therefore needs a way to run pbuilder as root even though | |
| 57 itself runs as a non-root user. By default the tree packager uses sudo, | |
| 58 so you have to setup sudo to allow the tree packager user to invoke | |
| 59 pbuilder without a password. This can be accomplished with the | |
| 60 following line in /etc/sudoers (using the default user name): | |
| 61 | |
| 62 builder ALL = NOPASSWD: /usr/sbin/pbuilder | |
| 63 | |
| 64 | |
| 65 Configure TreePackager | |
| 66 ~~~~~~~~~~~~~~~~~~~~~~ | |
| 67 | |
| 68 The file demo.cfg contains example configuration that contains most of | |
| 69 what is needed to package KDEPIM and kde-i18n from the KDEPIM enterprise | |
| 70 branch. Copy this file to treepkg.cfg and adapt it to your needs. The | |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
71 comments in the file should provide most of the hints to get you |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
72 started. Some more information is in the "Configuring a packager" |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
73 section below. |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
74 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
75 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
76 Configuring a packager |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
77 ~~~~~~~~~~~~~~~~~~~~~~ |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
78 |
| 73 | 79 The configuration file contains one section for each packager. The |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
80 section name starts with a "pkg_" prefix. The possible options are |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
81 described in demo.cfg. However there are some things that need to be |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
82 set up outside of the config file. |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
83 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
84 Each packager has a base directory (the base_dir option in the |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
85 corresponding pkg_-section). One thing needed by a packager is the |
| 72 | 86 contents of the debian sub-directory of the debian source package. When |
| 87 creating the source package, the packager simply copies the | |
| 88 sub-directory "debian" of the base_dir into the directory making up the | |
| 89 source tree. How the debian directory is created and maintained is up | |
| 90 to you. Usually it's a good idea to start with the debian sub-directory | |
| 91 an existing debian package for the software. | |
| 42 | 92 |
| 93 | |
| 94 Configure pbuilder | |
| 95 ~~~~~~~~~~~~~~~~~~ | |
| 96 | |
|
50
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
97 It's best to give the tree packager its own pbuilder configuration and |
| 72 | 98 directories. The default configuration uses a "pbuilder" sub-directory |
|
50
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
99 in /home/builder/enterprise. If you have created the treepkg.cfg file |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
100 with at least one packager and the correct root_cmd and pbuilderrc |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
101 options (the defaults for both should be OK if you use sudo as described |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
102 above), you can create the directories, the pbuilder configuration and |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
103 the chroot environment with the script initpbuilder.py like this: |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
104 |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
105 bin/initpbuilder.py --mirrorsite=<URL of preferred debian mirror> |
|
50
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
106 |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
107 You can specify some more mirrors with the --othermirror option. For |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
108 more information run "bin/initpbuilder.py --help" and consult the pbuilder |
|
50
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
109 documentation. |
| 42 | 110 |
| 111 | |
| 112 Configure the web front-end | |
| 113 ~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
| 114 | |
|
70
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
115 The web front-end consists of a single HTML-page with an overview of the |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
116 packager status and links to build logs when available. There are two |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
117 ways to publish this front-end: as little web-server with a dynamic |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
118 web-page or as a directory with a bunch of files making up a static |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
119 web-site. |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
120 |
| 72 | 121 Web-server: |
|
70
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
122 |
| 42 | 123 The default configuration should be OK in most cases. If you want you |
| 124 can customize it in cherrypy.cfg. Start the web front-end with | |
| 125 | |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
126 bin/starttreepkgweb.py |
| 42 | 127 |
| 128 starttreepkgweb has some options to specify which configuration files to use. | |
|
70
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
129 |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
130 |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
131 Static pages: |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
132 |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
133 The static pages are published using two programs, createstaticweb.py |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
134 and publishstaticweb.py. createstaticweb.py is run on the system where |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
135 the tree packager runs. publishstaticweb.py is usually run on another |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
136 system and connects via ssh and rsync to the tree packager host, creates |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
137 the files with createstaticweb.py and copies the files from the tree |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
138 packager host to a third host. |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
139 |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
140 The configuration file for publishstaticweb.py is demostaticweb.cfg. |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
141 Copy this file to staticweb.cfg and adapt it to your system. The |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
142 comments in the file describe the options. Afterwards, run the script |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
143 with |
|
70
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
144 |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
145 bin/publishstaticweb.py |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
146 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
147 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
148 Running the Tree Packager |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
149 ------------------------- |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
150 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
151 After configuration, run the tree packager with |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
152 |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
153 bin/runtreepkg.py [options] [packager...] |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
154 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
155 For each packager listed on the command line -- or all configured |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
156 packagers if none are given -- the tree packager checks out or updates |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
157 the sources and builds binary packages if the new revision hasn't been |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
158 packaged yet. |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
159 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
160 If the option --once has been given, the tree packager exits after it |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
161 has checked each packager once. Without it, the check is done |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
162 repeatedly. The interval between two checks can be set in the configuration |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
163 file. |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
164 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
165 Call runtreepkg.py with the --help option to see a list of the available |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
166 options. |