In infrastructure we have a script called bz-make-components.py that syncs the l10n and docs products (and owners) into bugzilla. This code is what the pkgdb-sync-bugzilla script is based on and performs a very similar function. However, the pkgdb-sync-bugzilla script has seen continuous updates over the years as the needs of syncing our 13K packages into bugzilla have demanded we do things more and more efficiently. The bz-make-components script, in the meantime, has puttered along without breaking so we've largely gotten to ignore it.
Unfortunately, the last bugzilla update has changed the xmlrpc api enough that the script is pretty thoroughly broken. Some functions have been replaced (actually pretty easy to update) and the parameters and return values for the xmlrpc functions have changed (a real pain to sift through). After spending a small amount of time attempting to port this old code to the newer API, I decided it would be easier to rewrite it to use the pkgdb-sync-bugzilla routines to talk to bugzilla and only keep the owners.list parsing code from the old bz-make-components script.
This is a pretty major rewrite but the code seems to work. Here's the major changes:
* The new script uses python-bugzilla. The old one talked to the bugzilla xmlrpc interface directly. * bz-make-components now uses a config file. This means in puppet the script can now be a file instead of a template. It also means the bugzilla passwords won't need to be in the script itself anymore * It's the same format as pkgdb-sync-bugzilla.cfg but I'm adding a new one because: * pkgdb-sync-bugzilla.cfg has extra info. The bz-make-components script doesn't need to map fas accounts to email addresses so it doesn't need the fas username and password that's in there. * The old script sent email to the l10n-members@fedoraproject.org email alias. The new script takes the place it sends to from the config file so we need to set a different email address for bz-make-components than pkgdb-sync-bugzilla. * The pkgdb-sync-bugzilla.cfg file is owned by root so the bz-make-components.py cron job would need to change from being run by apache to being run by root (This is, admittedly, not a big concern).
Since the old script was simply failing, I'd like a freeze break to fix this in production. The script does run on bapp02 so if something was wrong with the script it could impact other things running there. However, I've tried it out on app01.stg and it doesn't have a noticable impact on other things running there.
The patch for all these changes in puppet is attached -- only the manifest is really useful as a diff, the rest is adding the new files and removing the old ones.
Could I get two +1's?
On Tue, 21 May 2013 13:17:42 -0700 Toshio Kuratomi a.badger@gmail.com wrote:
In infrastructure we have a script called bz-make-components.py that syncs the l10n and docs products (and owners) into bugzilla. This code is what the pkgdb-sync-bugzilla script is based on and performs a very similar function. However, the pkgdb-sync-bugzilla script has seen continuous updates over the years as the needs of syncing our 13K packages into bugzilla have demanded we do things more and more efficiently. The bz-make-components script, in the meantime, has puttered along without breaking so we've largely gotten to ignore it.
Unfortunately, the last bugzilla update has changed the xmlrpc api enough that the script is pretty thoroughly broken. Some functions have been replaced (actually pretty easy to update) and the parameters and return values for the xmlrpc functions have changed (a real pain to sift through). After spending a small amount of time attempting to port this old code to the newer API, I decided it would be easier to rewrite it to use the pkgdb-sync-bugzilla routines to talk to bugzilla and only keep the owners.list parsing code from the old bz-make-components script.
This is a pretty major rewrite but the code seems to work. Here's the major changes:
- The new script uses python-bugzilla. The old one talked to the
bugzilla xmlrpc interface directly.
- bz-make-components now uses a config file. This means in puppet the script can now be a file instead of a template. It also means the bugzilla passwords won't need to be in the script itself anymore
- It's the same format as pkgdb-sync-bugzilla.cfg but I'm adding a
new one because:
- pkgdb-sync-bugzilla.cfg has extra info. The bz-make-components
script doesn't need to map fas accounts to email addresses so it doesn't need the fas username and password that's in there.
- The old script sent email to the l10n-members@fedoraproject.org
email alias. The new script takes the place it sends to from the config file so we need to set a different email address for bz-make-components than pkgdb-sync-bugzilla.
- The pkgdb-sync-bugzilla.cfg file is owned by root so the bz-make-components.py cron job would need to change from being
run by apache to being run by root (This is, admittedly, not a big concern).
Since the old script was simply failing, I'd like a freeze break to fix this in production. The script does run on bapp02 so if something was wrong with the script it could impact other things running there. However, I've tried it out on app01.stg and it doesn't have a noticable impact on other things running there.
The patch for all these changes in puppet is attached -- only the manifest is really useful as a diff, the rest is adding the new files and removing the old ones.
Could I get two +1's?
+1
-sv
infrastructure@lists.fedoraproject.org
-
Kevin Fenzi -
Seth Vidal -
Toshio Kuratomi