On Wed, 30 Jan 2008 21:23:48 +0100, Thorsten Leemhuis wrote:
00:09:26 < knurd> | so, how to move on? ask mschwendt to install is? nirik, or do you want to handle that? you likely have the needed permissions as well 00:09:42 --> | mdomsch (Matt_Domsch) has joined #fedora-meeting 00:10:11 < smooge> | mdomsch, has permission to. he can do anything :) 00:10:17 < nirik> | sure, I can ask him if we can just run it on the buildsys box, or what. 00:10:27 < knurd> | nirik, k, thx
00:23:51 < nirik> | knurd: I mailed mschewnt asking about setting his script up on the buildsys box
So, here are some public comments on that. Didn't know this was a topic in the meeting once more.
For Extras we used to execute some nohup background jobs in the post-push hook (near the bottom of the pushscript config file). That was the primary way of running scripts, particularly when pushes were done almost daily. [To push without re-running such stuff, there's option -s which also skips the repoprune and repoview steps, btw.] In the signer's group env one must be careful, though, and work around permission problems in some Python generated files (e.g. urlgrabber files created by yum) if they are shared.
Running fully automated hardly ever was a good idea because Core and Extras were not pushed at once and could be out-of-sync. Whether to include or exclude updates-testing was an unresolved matter, too. Later I switched to preventing broken deps by excluding packages from needsign with the help of the RCNeedsign.py script. That got rid of the public summaries and only mailed packagers privately (warning about whether needsign and/or test updates were included).
To run the code as a cron job somebody first needs to spend some time and determine under which circumstances it would run into error conditions. Such as metadata loading problems (also adding the lockfile check for the Plague needsign repodata lock), and then try to catch all errors. Otherwise there would be the risk that it spams packagers with wrong reports. Mission objective: also check yum api's error return codes.
I haven't done that extra work because of the Core+Extras+Test Updates split (see above) and because it has never been agreed on how often to mail broken deps reports to owners and the list. The initial work-around for repo I/O problems was to sleep 12 minutes when repoclosure returned an error and retry a few times. That seemed to work for cron as well as the background jobs. The old scripts are in fedora cvs, so copying existing bits from them would be possible.
The old repoclosure code is integrated into the pushscript and the RCNeedsign.py module -- [That's the one that installs the pkgs from the needsign queue into temporary repos to allow for proper ExcludeArch/ExclusiveArch handling prior to running repoclosure on that. Originally I wanted to test adding multilib support to it but didn't continue when koji+bodhi appeared on the radar.]
The additional Extras repoclosure scripts on the buildsys machine are a bit specific to Extras. Not all constants are fetched from the Config_*.py files. E.g. the script needed to distinguish between Core and Extras (it's a similar thing for for CentOS-or-RHEL and EPEL) through a repoid keyword, hardcoded in the rc-report.py script. Which package owners list to read was hardcoded, too. A few other features were added without any way to configure them (e.g. the history support that printed the age in days of a broken dep or the function that only sends a summary to the list if there are new reports). The used Yum is a private copy with the checkForObsoletes patch applied.
The recently published repoclosure tarball is easier to modify. It doesn't depend on the pushscript code or config files. It takes an ordinary yum.conf file as input. It can be configured with Fedora Account System account details for package owner db access outside fedora infrastructure. All it needs is Yum post 2.6.1 with checkForObsolete.
Whether and how often to run a script like that, whether or not to include "testing" or needsign (*without* multilib, ExcludeArch, Exclusive}Arch), whether to run against CentOS or the private RHEL repos, I don't want to decide. I would appreciate if somebody else took over. Sometimes people, who have a fixed package in updates-testing already, complain about a broken deps report that doesn't cover updates-testing. On the other hand, some people complain if a report covers updates-testing. Furthermore, there are packagers who believe every report to be false. The only fun of doing repoclosure stuff is to see those people who understand the reports and fix their packages silently as soon as they receive a report.