Mercurial > treepkg
annotate README @ 242:351427b28b76
Add test for bin/listpendingnotifications.py
| author | Bernhard Herzog <bh@intevation.de> |
|---|---|
| date | Fri, 16 Jan 2009 20:42:06 +0000 |
| parents | fbfa1abb47f1 |
| children | 092925ff75d7 |
| rev | line source |
|---|---|
| 42 | 1 README for TreePackager |
| 2 ======================= | |
| 3 | |
| 4 | |
| 5 TreePackager is a tool to automatically build debian packages from SVN. | |
| 6 | |
| 7 | |
| 8 Prerequisites | |
| 9 ------------- | |
| 10 | |
| 11 You need the following software to run TreePackager. In the list below, | |
| 12 parentheses contain the name of the corresponding package in Debian Etch | |
| 13 if its not the same as the software. The version required is usually | |
| 14 the one from debian etch. | |
| 15 | |
| 16 Python 2.4 (python2.4-minimal) | |
| 17 Debian devscripts (devscripts) | |
| 18 subversion | |
| 19 pbuilder | |
| 20 sudo | |
| 21 bzip2 | |
| 22 | |
| 23 For the web front-end you also need the following software: | |
| 24 | |
| 25 Genshi (python-genshi) | |
| 26 CherryPy (python-cherrypy) | |
| 27 | |
| 28 Some of the packagers require additional software. The KDEPIM | |
| 29 enterprise branch packagers require the following additional software: | |
| 30 | |
| 31 autoconf2.13 | |
| 32 automake1.9 | |
| 33 | |
| 34 | |
| 35 Installation | |
| 36 ------------ | |
| 37 | |
| 38 You can run the tree packager itself directly from the source tree. | |
| 39 However, you need to configure it first and setup pbuilder. | |
| 40 | |
| 41 | |
| 42 Configuration | |
| 43 ------------- | |
| 44 | |
| 45 To understand the configuration, first a few notes about the | |
| 46 architecture of TreePackager. The TreePackager consist of one program | |
| 47 that periodically updates svn working directories and if something has | |
| 48 changed, builds a new debian package from the working directory. The | |
| 49 program should run as a normal user. The sample configuration assumes | |
| 50 that it's the user "builder" with a home directory "/home/builder". The | |
| 51 default configuration manages a directory tree under | |
| 52 "/home/builder/enterprise". | |
| 53 | |
| 54 The binary packages are built with pbuilder. Because pbuilder uses a | |
| 55 chroot environment to build the packages, it must be run as root. The | |
| 56 tree packager therefore needs a way to run pbuilder as root even though | |
| 57 itself runs as a non-root user. By default the tree packager uses sudo, | |
| 58 so you have to setup sudo to allow the tree packager user to invoke | |
| 59 pbuilder without a password. This can be accomplished with the | |
| 60 following line in /etc/sudoers (using the default user name): | |
| 61 | |
| 62 builder ALL = NOPASSWD: /usr/sbin/pbuilder | |
| 63 | |
| 64 | |
| 65 Configure TreePackager | |
| 66 ~~~~~~~~~~~~~~~~~~~~~~ | |
| 67 | |
| 68 The file demo.cfg contains example configuration that contains most of | |
| 69 what is needed to package KDEPIM and kde-i18n from the KDEPIM enterprise | |
| 70 branch. Copy this file to treepkg.cfg and adapt it to your needs. The | |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
71 comments in the file should provide most of the hints to get you |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
72 started. Some more information is in the "Configuring a packager" |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
73 section below. |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
74 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
75 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
76 Configuring a packager |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
77 ~~~~~~~~~~~~~~~~~~~~~~ |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
78 |
| 73 | 79 The configuration file contains one section for each packager. The |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
80 section name starts with a "pkg_" prefix. The possible options are |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
81 described in demo.cfg. However there are some things that need to be |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
82 set up outside of the config file. |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
83 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
84 Each packager has a base directory (the base_dir option in the |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
85 corresponding pkg_-section). One thing needed by a packager is the |
| 72 | 86 contents of the debian sub-directory of the debian source package. When |
| 87 creating the source package, the packager simply copies the | |
| 88 sub-directory "debian" of the base_dir into the directory making up the | |
| 89 source tree. How the debian directory is created and maintained is up | |
| 90 to you. Usually it's a good idea to start with the debian sub-directory | |
| 91 an existing debian package for the software. | |
| 42 | 92 |
|
108
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
93 Once you have configured the packagers, you can create their directories |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
94 with |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
95 |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
96 bin/inittreepkg.py |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
97 |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
98 This will create base directories of each packager and report some |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
99 missing steps that will have to be done manually, like creating a debian |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
100 subdirectory with the debian packaging information. |
|
91deff3db739
Add note about bin/inittreepkg.py to README
Bernhard Herzog <bh@intevation.de>
parents:
78
diff
changeset
|
101 |
| 42 | 102 |
| 103 Configure pbuilder | |
| 104 ~~~~~~~~~~~~~~~~~~ | |
| 105 | |
|
50
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
106 It's best to give the tree packager its own pbuilder configuration and |
| 72 | 107 directories. The default configuration uses a "pbuilder" sub-directory |
|
50
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
108 in /home/builder/enterprise. If you have created the treepkg.cfg file |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
109 with at least one packager and the correct root_cmd and pbuilderrc |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
110 options (the defaults for both should be OK if you use sudo as described |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
111 above), you can create the directories, the pbuilder configuration and |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
112 the chroot environment with the script initpbuilder.py like this: |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
113 |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
114 bin/initpbuilder.py --mirrorsite=<URL of preferred debian mirror> |
|
50
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
115 |
|
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
116 You can specify some more mirrors with the --othermirror option. For |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
117 more information run "bin/initpbuilder.py --help" and consult the pbuilder |
|
50
225206553bba
update README part about pbuilder config
Bernhard Herzog <bh@intevation.de>
parents:
42
diff
changeset
|
118 documentation. |
| 42 | 119 |
| 120 | |
| 121 Configure the web front-end | |
| 122 ~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
| 123 | |
|
70
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
124 The web front-end consists of a single HTML-page with an overview of the |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
125 packager status and links to build logs when available. There are two |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
126 ways to publish this front-end: as little web-server with a dynamic |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
127 web-page or as a directory with a bunch of files making up a static |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
128 web-site. |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
129 |
| 72 | 130 Web-server: |
|
70
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
131 |
| 42 | 132 The default configuration should be OK in most cases. If you want you |
| 133 can customize it in cherrypy.cfg. Start the web front-end with | |
| 134 | |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
135 bin/starttreepkgweb.py |
| 42 | 136 |
| 137 starttreepkgweb has some options to specify which configuration files to use. | |
|
70
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
138 |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
139 |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
140 Static pages: |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
141 |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
142 The static pages are published using two programs, createstaticweb.py |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
143 and publishstaticweb.py. createstaticweb.py is run on the system where |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
144 the tree packager runs. publishstaticweb.py is usually run on another |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
145 system and connects via ssh and rsync to the tree packager host, creates |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
146 the files with createstaticweb.py and copies the files from the tree |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
147 packager host to a third host. |
|
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
148 |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
149 The configuration file for publishstaticweb.py is demostaticweb.cfg. |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
150 Copy this file to staticweb.cfg and adapt it to your system. The |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
151 comments in the file describe the options. Afterwards, run the script |
|
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
152 with |
|
70
4b537aadc5ce
add README section for the static web-page config
Bernhard Herzog <bh@intevation.de>
parents:
69
diff
changeset
|
153 |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
154 bin/publishstaticweb.py |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
155 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
156 |
|
109
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
157 Notification Mails |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
158 ~~~~~~~~~~~~~~~~~~ |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
159 |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
160 The tree packager can send notification mails when a built fails. This |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
161 works in a two-step process like the creation of the static web-pages. |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
162 The sample configuration file for the notification mails is |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
163 demonotification.cfg. Copy it to notification.cfg and adapt it to your |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
164 needs. The comments in the file describe the options. In particular |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
165 you will want to customize the template for the notification mail to |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
166 include suitable email addresses. To actually send notifications, run |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
167 |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
168 bin/sendnotificationmails.py |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
169 |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
170 Note that this only tries to send mails when a build has failed. |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
171 |
|
fbfa1abb47f1
Add note about notificatin mails to README
Bernhard Herzog <bh@intevation.de>
parents:
108
diff
changeset
|
172 |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
173 Running the Tree Packager |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
174 ------------------------- |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
175 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
176 After configuration, run the tree packager with |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
177 |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
178 bin/runtreepkg.py [options] [packager...] |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
179 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
180 For each packager listed on the command line -- or all configured |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
181 packagers if none are given -- the tree packager checks out or updates |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
182 the sources and builds binary packages if the new revision hasn't been |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
183 packaged yet. |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
184 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
185 If the option --once has been given, the tree packager exits after it |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
186 has checked each packager once. Without it, the check is done |
|
78
9a602d8eaa60
initial revision of the subversion repository
Thomas Arendsen Hein <thomas@intevation.de>
parents:
73
diff
changeset
|
187 repeatedly. The interval between two checks can be set in the configuration |
|
71
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
188 file. |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
189 |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
190 Call runtreepkg.py with the --help option to see a list of the available |
|
2eefd12526ff
flesh out the REAME file some more
Bernhard Herzog <bh@intevation.de>
parents:
70
diff
changeset
|
191 options. |