On Sun, 30 Dec 2007 17:54:51 +0100, Michael Schwendt wrote:
If in a failed job.log you see the message
Job waited too long for repo to unlock. Killing it...
please notify me.
It's a problem in the plague server code that results in a denial of
service for subsequent build jobs. I have a traceback from Dec 28th, but
in the context of the source code it doesn't make sense yet (because a few
lines earlier the code ensures that the files to be copied exist and are
readable). Buildsys runs a slightly modified version that adds a bit more
debug output in this area.
Certainly related, job #37767 just failed
Failed to copy
/srv/rpmbuild/server_work/fedora-5-epel/37767-php-pecl-memcache-2.2.1-1.el5/ppc/php-pecl-memcache-2.2.1-1.el5.ppc.rpm
to the repository directory.
and that should not happen either. I assume this is not the first time
this has happened. The additional debug details are interesting.
In the server log, the individual arch jobs print the list of files
downloaded from the builders. At that time they should be available
already:
37767 (php-pecl-memcache/ppc): Build result files - [
'php-pecl-memcache-2.2.1-1.el5.ppc.rpm',
'php-pecl-memcache-2.2.1-1.el5.src.rpm', 'root.log',
'php-pecl-memcache-debuginfo-2.2.1-1.el5.ppc.rpm', 'state.log',
'job.log', 'build.log' ]
37767 (php-pecl-memcache/x86_64): Build result files - [ 'root.log',
'build.log', 'state.log', 'php-pecl-memcache-2.2.1-1.el5.src.rpm',
'php-pecl-memcache-debuginfo-2.2.1-1.el5.x86_64.rpm',
'php-pecl-memcache-2.2.1-1.el5.x86_64.rpm', 'job.log' ]
37767 (php-pecl-memcache/i386): Build result files - [ 'root.log',
'php-pecl-memcache-debuginfo-2.2.1-1.el5.i386.rpm', 'state.log',
'php-pecl-memcache-2.2.1-1.el5.i386.rpm', 'build.log',
'php-pecl-memcache-2.2.1-1.el5.src.rpm', 'job.log' ]
Further down in the log, the fedora-5-epel repo controller is asked to
install the files into the needsign repo:
Repo 'fedora-5-epel': updating repository metadata...
At that time, the last of the ppc files is not yet accessible (i.e. it
either doesn't exist or cannot be read):
[...]
Repo:
/srv/rpmbuild/server_work/fedora-5-epel/37767-php-pecl-memcache-2.2.1-1.el5/x86_64/php-pecl-memcache-2.2.1-1.el5.x86_64.rpm
is accessible.
Repo:
/srv/rpmbuild/server_work/fedora-5-epel/37767-php-pecl-memcache-2.2.1-1.el5/ppc/php-pecl-memcache-2.2.1-1.el5.ppc.rpm
is inaccessible.
Further proof-reading of source code necessary... it should not populate
needsign before all files are downloaded.