Mercurial > treepkg
annotate README @ 442:c29714c601da
use os.mkdirs to create static web directory to get rid of noisy errors if a build machine is rebooted and tmp dir is cleaned
| author | Bjoern Ricks <bricks@intevation.de> |
|---|---|
| date | Fri, 06 Aug 2010 14:27:01 +0000 |
| parents | 092925ff75d7 |
| children |
| rev | line source |
|---|---|
| 42 | 1 README for TreePackager |
| 2 ======================= | |
| 3 | |
| 4 | |
|
321
092925ff75d7
Added basic Git support, configuration options:
Andre Heinecke <aheinecke@intevation.de>
parents:
109
diff
changeset
|
5 TreePackager is a tool to automatically build debian packages. |
| 42 | 6 |
| 7 | |
| 8 Prerequisites | |
| 9 ------------- | |
| 10 | |
| 11 You need the following software to run TreePackager. In the list below, | |
| 12 parentheses contain the name of the corresponding package in Debian Etch | |
| 13 if its not the same as the software. The version required is usually | |
| 14 the one from debian etch. | |
| 15 | |
| 16 Python 2.4 (python2.4-minimal) | |
| 17 Debian devscripts (devscripts) | |
| 18 subversion | |
| 19 pbuilder | |
|
321
092925ff75d7
Added basic Git support, configuration options:
Andre Heinecke <aheinecke@intevation.de>
parents:
109
diff
changeset
|
20 git-core |
| 42 | 21 sudo |
| 22 bzip2 | |
| 23 | |
| 24 For the web front-end you also need the following software: | |
| 25 | |
| 26 Genshi (python-genshi) | |
| 27 CherryPy (python-cherrypy) | |
| 28 | |
| 29 Some of the packagers require additional software. The KDEPIM | |
| 30 enterprise branch packagers require the following additional software: | |
| 31 | |
| 32 autoconf2.13 | |
| 33 automake1.9 | |
| 34 | |
| 35 | |
| 36 Installation | |
| 37 ------------ | |
| 38 | |
| 39 You can run the tree packager itself directly from the source tree. | |
| 40 However, you need to configure it first and setup pbuilder. | |
| 41 | |
| 42 | |
| 43 Configuration | |
| 44 ------------- | |
| 45 | |
| 46 To understand the configuration, first a few notes about the | |
| 47 architecture of TreePackager. The TreePackager consist of one program | |
| 48 that periodically updates svn working directories and if something has | |
| 49 changed, builds a new debian package from the working directory. The | |
| 50 program should run as a normal user. The sample configuration assumes | |
| 51 that it's the user "builder" with a home directory "/home/builder". The | |
| 52 default configuration manages a directory tree under | |
| 53 "/home/builder/enterprise". | |
| 54 | |
| 55 The binary packages are built with pbuilder. Because pbuilder uses a | |
| 56 chroot environment to build the packages, it must be run as root. The | |
| 57 tree packager therefore needs a way to run pbuilder as root even though | |
| 58 itself runs as a non-root user. By default the tree packager uses sudo, | |
| 59 so you have to setup sudo to allow the tree packager user to invoke | |
| 60 pbuilder without a password. This can be accomplished with the | |
| 61 following line in /etc/sudoers (using the default user name): | |
| 62 | |
| 63 builder ALL = NOPASSWD: /usr/sbin/pbuilder | |
| 64 | |
| 65 | |
| 66 Configure TreePackager | |
| 67 ~~~~~~~~~~~~~~~~~~~~~~ | |
| 68 | |
| 69 The file demo.cfg contains example configuration that contains most of | |
| 70 what is needed to package KDEPIM and kde-i18n from the KDEPIM enterprise | |
| 71 branch. Copy this file to treepkg.cfg and adapt it to your needs. The | |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
72 comments in the file should provide most of the hints to get you |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
73 started. Some more information is in the "Configuring a packager" |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
74 section below. |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
75 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
76 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
77 Configuring a packager |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
78 ~~~~~~~~~~~~~~~~~~~~~~ |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
79 |
| 73 | 80 The configuration file contains one section for each packager. The |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
81 section name starts with a "pkg_" prefix. The possible options are |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
82 described in demo.cfg. However there are some things that need to be |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
83 set up outside of the config file. |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
84 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
85 Each packager has a base directory (the base_dir option in the |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
86 corresponding pkg_-section). One thing needed by a packager is the |
| 72 | 87 contents of the debian sub-directory of the debian source package. When |
| 88 creating the source package, the packager simply copies the | |
| 89 sub-directory "debian" of the base_dir into the directory making up the | |
| 90 source tree. How the debian directory is created and maintained is up | |
| 91 to you. Usually it's a good idea to start with the debian sub-directory | |
| 92 an existing debian package for the software. | |
| 42 | 93 |
|
108
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
94 Once you have configured the packagers, you can create their directories |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
95 with |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
96 |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
97 bin/inittreepkg.py |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
98 |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
99 This will create base directories of each packager and report some |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
100 missing steps that will have to be done manually, like creating a debian |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
101 subdirectory with the debian packaging information. |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
102 |
| 42 | 103 |
| 104 Configure pbuilder | |
| 105 ~~~~~~~~~~~~~~~~~~ | |
| 106 | |
|
50
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
107 It's best to give the tree packager its own pbuilder configuration and |
| 72 | 108 directories. The default configuration uses a "pbuilder" sub-directory |
|
50
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
109 in /home/builder/enterprise. If you have created the treepkg.cfg file |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
110 with at least one packager and the correct root_cmd and pbuilderrc |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
111 options (the defaults for both should be OK if you use sudo as described |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
112 above), you can create the directories, the pbuilder configuration and |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
113 the chroot environment with the script initpbuilder.py like this: |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
114 |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
115 bin/initpbuilder.py --mirrorsite=<URL of preferred debian mirror> |
|
50
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
116 |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
117 You can specify some more mirrors with the --othermirror option. For |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
118 more information run "bin/initpbuilder.py --help" and consult the pbuilder |
|
50
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
119 documentation. |
| 42 | 120 |
| 121 | |
| 122 Configure the web front-end | |
| 123 ~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
| 124 | |
|
70
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
125 The web front-end consists of a single HTML-page with an overview of the |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
126 packager status and links to build logs when available. There are two |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
127 ways to publish this front-end: as little web-server with a dynamic |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
128 web-page or as a directory with a bunch of files making up a static |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
129 web-site. |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
130 |
| 72 | 131 Web-server: |
|
70
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
132 |
| 42 | 133 The default configuration should be OK in most cases. If you want you |
| 134 can customize it in cherrypy.cfg. Start the web front-end with | |
| 135 | |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
136 bin/starttreepkgweb.py |
| 42 | 137 |
| 138 starttreepkgweb has some options to specify which configuration files to use. | |
|
70
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
139 |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
140 |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
141 Static pages: |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
142 |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
143 The static pages are published using two programs, createstaticweb.py |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
144 and publishstaticweb.py. createstaticweb.py is run on the system where |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
145 the tree packager runs. publishstaticweb.py is usually run on another |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
146 system and connects via ssh and rsync to the tree packager host, creates |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
147 the files with createstaticweb.py and copies the files from the tree |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
148 packager host to a third host. |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
149 |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
150 The configuration file for publishstaticweb.py is demostaticweb.cfg. |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
151 Copy this file to staticweb.cfg and adapt it to your system. The |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
152 comments in the file describe the options. Afterwards, run the script |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
153 with |
|
70
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
154 |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
155 bin/publishstaticweb.py |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
156 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
157 |
|
109
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
158 Notification Mails |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
159 ~~~~~~~~~~~~~~~~~~ |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
160 |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
161 The tree packager can send notification mails when a built fails. This |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
162 works in a two-step process like the creation of the static web-pages. |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
163 The sample configuration file for the notification mails is |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
164 demonotification.cfg. Copy it to notification.cfg and adapt it to your |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
165 needs. The comments in the file describe the options. In particular |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
166 you will want to customize the template for the notification mail to |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
167 include suitable email addresses. To actually send notifications, run |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
168 |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
169 bin/sendnotificationmails.py |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
170 |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
171 Note that this only tries to send mails when a build has failed. |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
172 |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
173 |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
174 Running the Tree Packager |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
175 ------------------------- |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
176 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
177 After configuration, run the tree packager with |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
178 |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
179 bin/runtreepkg.py [options] [packager...] |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
180 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
181 For each packager listed on the command line -- or all configured |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
182 packagers if none are given -- the tree packager checks out or updates |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
183 the sources and builds binary packages if the new revision hasn't been |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
184 packaged yet. |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
185 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
186 If the option --once has been given, the tree packager exits after it |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
187 has checked each packager once. Without it, the check is done |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
188 repeatedly. The interval between two checks can be set in the configuration |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
189 file. |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
190 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
191 Call runtreepkg.py with the --help option to see a list of the available |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
192 options. |