Mercurial > treepkg
annotate README @ 191:94fb3f3ab58b
When building a subset of tracks, make sure new packages are added to
pbuilder even if the tracks that depend on them are not being built. To
achieve this, the information which tracks are to be built is now stored
as the do_build flag in the individual tracks and not by passing only a
subset of the tracks ot the PackagerGroup. Otherwise the PackagerGroup
would not determine the dependencies correctly.
| author | Bernhard Herzog <bh@intevation.de> |
|---|---|
| date | Wed, 30 Jul 2008 19:23:10 +0000 |
| parents | fbfa1abb47f1 |
| children | 092925ff75d7 |
| rev | line source |
|---|---|
| 42 | 1 README for TreePackager |
| 2 ======================= | |
| 3 | |
| 4 | |
| 5 TreePackager is a tool to automatically build debian packages from SVN. | |
| 6 | |
| 7 | |
| 8 Prerequisites | |
| 9 ------------- | |
| 10 | |
| 11 You need the following software to run TreePackager. In the list below, | |
| 12 parentheses contain the name of the corresponding package in Debian Etch | |
| 13 if its not the same as the software. The version required is usually | |
| 14 the one from debian etch. | |
| 15 | |
| 16 Python 2.4 (python2.4-minimal) | |
| 17 Debian devscripts (devscripts) | |
| 18 subversion | |
| 19 pbuilder | |
| 20 sudo | |
| 21 bzip2 | |
| 22 | |
| 23 For the web front-end you also need the following software: | |
| 24 | |
| 25 Genshi (python-genshi) | |
| 26 CherryPy (python-cherrypy) | |
| 27 | |
| 28 Some of the packagers require additional software. The KDEPIM | |
| 29 enterprise branch packagers require the following additional software: | |
| 30 | |
| 31 autoconf2.13 | |
| 32 automake1.9 | |
| 33 | |
| 34 | |
| 35 Installation | |
| 36 ------------ | |
| 37 | |
| 38 You can run the tree packager itself directly from the source tree. | |
| 39 However, you need to configure it first and setup pbuilder. | |
| 40 | |
| 41 | |
| 42 Configuration | |
| 43 ------------- | |
| 44 | |
| 45 To understand the configuration, first a few notes about the | |
| 46 architecture of TreePackager. The TreePackager consist of one program | |
| 47 that periodically updates svn working directories and if something has | |
| 48 changed, builds a new debian package from the working directory. The | |
| 49 program should run as a normal user. The sample configuration assumes | |
| 50 that it's the user "builder" with a home directory "/home/builder". The | |
| 51 default configuration manages a directory tree under | |
| 52 "/home/builder/enterprise". | |
| 53 | |
| 54 The binary packages are built with pbuilder. Because pbuilder uses a | |
| 55 chroot environment to build the packages, it must be run as root. The | |
| 56 tree packager therefore needs a way to run pbuilder as root even though | |
| 57 itself runs as a non-root user. By default the tree packager uses sudo, | |
| 58 so you have to setup sudo to allow the tree packager user to invoke | |
| 59 pbuilder without a password. This can be accomplished with the | |
| 60 following line in /etc/sudoers (using the default user name): | |
| 61 | |
| 62 builder ALL = NOPASSWD: /usr/sbin/pbuilder | |
| 63 | |
| 64 | |
| 65 Configure TreePackager | |
| 66 ~~~~~~~~~~~~~~~~~~~~~~ | |
| 67 | |
| 68 The file demo.cfg contains example configuration that contains most of | |
| 69 what is needed to package KDEPIM and kde-i18n from the KDEPIM enterprise | |
| 70 branch. Copy this file to treepkg.cfg and adapt it to your needs. The | |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
71 comments in the file should provide most of the hints to get you |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
72 started. Some more information is in the "Configuring a packager" |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
73 section below. |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
74 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
75 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
76 Configuring a packager |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
77 ~~~~~~~~~~~~~~~~~~~~~~ |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
78 |
| 73 | 79 The configuration file contains one section for each packager. The |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
80 section name starts with a "pkg_" prefix. The possible options are |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
81 described in demo.cfg. However there are some things that need to be |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
82 set up outside of the config file. |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
83 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
84 Each packager has a base directory (the base_dir option in the |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
85 corresponding pkg_-section). One thing needed by a packager is the |
| 72 | 86 contents of the debian sub-directory of the debian source package. When |
| 87 creating the source package, the packager simply copies the | |
| 88 sub-directory "debian" of the base_dir into the directory making up the | |
| 89 source tree. How the debian directory is created and maintained is up | |
| 90 to you. Usually it's a good idea to start with the debian sub-directory | |
| 91 an existing debian package for the software. | |
| 42 | 92 |
|
108
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
93 Once you have configured the packagers, you can create their directories |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
94 with |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
95 |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
96 bin/inittreepkg.py |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
97 |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
98 This will create base directories of each packager and report some |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
99 missing steps that will have to be done manually, like creating a debian |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
100 subdirectory with the debian packaging information. |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
101 |
| 42 | 102 |
| 103 Configure pbuilder | |
| 104 ~~~~~~~~~~~~~~~~~~ | |
| 105 | |
|
50
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
106 It's best to give the tree packager its own pbuilder configuration and |
| 72 | 107 directories. The default configuration uses a "pbuilder" sub-directory |
|
50
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
108 in /home/builder/enterprise. If you have created the treepkg.cfg file |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
109 with at least one packager and the correct root_cmd and pbuilderrc |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
110 options (the defaults for both should be OK if you use sudo as described |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
111 above), you can create the directories, the pbuilder configuration and |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
112 the chroot environment with the script initpbuilder.py like this: |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
113 |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
114 bin/initpbuilder.py --mirrorsite=<URL of preferred debian mirror> |
|
50
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
115 |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
116 You can specify some more mirrors with the --othermirror option. For |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
117 more information run "bin/initpbuilder.py --help" and consult the pbuilder |
|
50
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
118 documentation. |
| 42 | 119 |
| 120 | |
| 121 Configure the web front-end | |
| 122 ~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
| 123 | |
|
70
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
124 The web front-end consists of a single HTML-page with an overview of the |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
125 packager status and links to build logs when available. There are two |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
126 ways to publish this front-end: as little web-server with a dynamic |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
127 web-page or as a directory with a bunch of files making up a static |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
128 web-site. |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
129 |
| 72 | 130 Web-server: |
|
70
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
131 |
| 42 | 132 The default configuration should be OK in most cases. If you want you |
| 133 can customize it in cherrypy.cfg. Start the web front-end with | |
| 134 | |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
135 bin/starttreepkgweb.py |
| 42 | 136 |
| 137 starttreepkgweb has some options to specify which configuration files to use. | |
|
70
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
138 |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
139 |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
140 Static pages: |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
141 |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
142 The static pages are published using two programs, createstaticweb.py |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
143 and publishstaticweb.py. createstaticweb.py is run on the system where |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
144 the tree packager runs. publishstaticweb.py is usually run on another |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
145 system and connects via ssh and rsync to the tree packager host, creates |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
146 the files with createstaticweb.py and copies the files from the tree |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
147 packager host to a third host. |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
148 |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
149 The configuration file for publishstaticweb.py is demostaticweb.cfg. |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
150 Copy this file to staticweb.cfg and adapt it to your system. The |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
151 comments in the file describe the options. Afterwards, run the script |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
152 with |
|
70
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
153 |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
154 bin/publishstaticweb.py |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
155 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
156 |
|
109
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
157 Notification Mails |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
158 ~~~~~~~~~~~~~~~~~~ |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
159 |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
160 The tree packager can send notification mails when a built fails. This |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
161 works in a two-step process like the creation of the static web-pages. |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
162 The sample configuration file for the notification mails is |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
163 demonotification.cfg. Copy it to notification.cfg and adapt it to your |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
164 needs. The comments in the file describe the options. In particular |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
165 you will want to customize the template for the notification mail to |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
166 include suitable email addresses. To actually send notifications, run |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
167 |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
168 bin/sendnotificationmails.py |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
169 |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
170 Note that this only tries to send mails when a build has failed. |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
171 |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
172 |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
173 Running the Tree Packager |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
174 ------------------------- |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
175 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
176 After configuration, run the tree packager with |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
177 |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
178 bin/runtreepkg.py [options] [packager...] |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
179 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
180 For each packager listed on the command line -- or all configured |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
181 packagers if none are given -- the tree packager checks out or updates |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
182 the sources and builds binary packages if the new revision hasn't been |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
183 packaged yet. |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
184 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
185 If the option --once has been given, the tree packager exits after it |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
186 has checked each packager once. Without it, the check is done |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
187 repeatedly. The interval between two checks can be set in the configuration |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
188 file. |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
189 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
190 Call runtreepkg.py with the --help option to see a list of the available |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
191 options. |