[BUG] Koji and Lustre
by Thomas Guthmann
Hey guys,
We found a bug(?) in listTaskOutput (/usr/share/koji-hub/kojihub.py)
when used with a Lustre filesystem. This function parses all attributes
of every file of a build and is used when you want to display
build.log/root.log through the web interface. Everything returned by
listTaskOutput is returned through XML-RPC and as a result we had this :
An error has occurred while processing your request.
Fault: <Fault 1: 'exceptions.OverflowError: long int exceeds XML-RPC
limits'>
Traceback (most recent call last):
File "/usr/share/koji-web/lib/kojiweb/publisher.py", line 16, in
publish_object
return old_publish_object(req, object)
File "/usr/lib64/python2.4/site-packages/mod_python/publisher.py",
line 412, in publish_object
return publish_object(req,util.apply_fs_data(object, req.form,
req=req))
File "/usr/lib64/python2.4/site-packages/mod_python/util.py", line
439, in apply_fs_data
return object(**args)
File "/usr/share/koji-web/scripts/index.py", line 649, in getfile
output = server.listTaskOutput(taskID, stat=True)
File "/usr/lib/python2.4/site-packages/koji/__init__.py", line 1468,
in __call__
return self.__func(self.__name,args,opts)
File "/usr/lib/python2.4/site-packages/koji/__init__.py", line 1718,
in _callMethod
raise err
Fault: <Fault 1: 'exceptions.OverflowError: long int exceeds XML-RPC
limits'>
The issue comes from the st_dev value gathered by getStat (stat). In
Lustre this value can be very high and that's why it complains. To fix
that, we used the same condition than st_size, we cast the value as a
string. See attached patch.
Example (see 'Device:' the value after the '/'):
# stat build.log
File: `build.log'
Size: 106021 Blocks: 208 IO Block: 2097152 regular file
Device: e04ae70eh/3763005198d Inode: 721664 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 48/ apache) Gid: ( 48/ apache)
Access: 2010-07-13 10:44:40.000000000 +1000
Modify: 2010-07-12 12:54:15.000000000 +1000
Change: 2010-07-12 12:54:52.000000000 +1000
# That's what is read by listTaskOutput
relfilename=build.log ATTR=st_atime GETATTR=1278981616
relfilename=build.log ATTR=st_blksize GETATTR=2097152
relfilename=build.log ATTR=st_blocks GETATTR=208
relfilename=build.log ATTR=st_ctime GETATTR=1278903292
relfilename=build.log ATTR=st_dev GETATTR=3763005198 <----- /!\
relfilename=build.log ATTR=st_gid GETATTR=48
relfilename=build.log ATTR=st_ino GETATTR=721664
relfilename=build.log ATTR=st_mode GETATTR=33188
relfilename=build.log ATTR=st_mtime GETATTR=1278903255
relfilename=build.log ATTR=st_nlink GETATTR=1
relfilename=build.log ATTR=st_rdev GETATTR=0
relfilename=build.log ATTR=st_size GETATTR=106021
relfilename=build.log ATTR=st_uid GETATTR=48
Hope it helps, lost a good amount of time on that one :)
Cheers,
Thomas
12 years, 6 months
Strange mock build failure due to typo
by Paul Howarth
Hi,
today I have been preparing an update to perl-Math-Pari and came across
a very strange build failure whilst doing a local mock build on a Fedora
13 x86_64 host. My package built successfully on x86_64 but when I tried
to build for i386, the build failed during %setup but without any
diagnostics. The SRPM was installed but no attempt to install its build
requirements was made. The root.log showed an exit status of 0 for all
commands that had been run.
After much experimentation bisecting the changes I had made, I
discovered that a typo in the changelog entry was the culprit: I had set
the year to 2100 instead of 2010. So it would appear that somewhere in
the mock/yum/rpm stack there may be a year 2038 problem waiting to bite
us (though I suspect there may not be too many 32-bit builds happening
by then).
Seriously though, it would be nice to have better diagnostics for this
and perhaps an rpmlint check for changelog entries in the future?
Paul.
13 years, 2 months
cannot open Providename index using db3 - Invalid argument (22)
by Vijay N. Majagaonkar
Hi,
I am running out of strange problem at the time of creating ISO, ISO created
successfully but it failed at the time of OS installation and this what i
found in root.log file
I am not able to under stand the what is the problem "cannot open
Providename index using db3 - Invalid argument (22)"
can someone please give me some point or direction to fix this,
[Log]
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: Building
isolinux directory
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: mkdosfs 2.11 (12
Mar 2005)
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: rpmdb:
/var/lib/rpm/Providename: unsupported hash version: 9
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: error: cannot
open Providename index using db3 - Invalid argument (22)
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: rpmdb:
/var/lib/rpm/Providename: unsupported hash version: 9
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: error: cannot
open Providename index using db3 - Invalid argument (22)
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall:
http://koji.fedora.com/packages/kernel/2.6.18/164.el5/x86_64/kernel-xen-2...:
[Errno 12] Timeout: <urlopen error timed out>
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: Trying other
mirror.
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: Traceback (most
recent call last):
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File
"/usr/bin/yumdownloader", line 293, in ?
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: util =
YumDownloader()
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File
"/usr/bin/yumdownloader", line 42, in __init__
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: self.main()
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File
"/usr/bin/yumdownloader", line 80, in main
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall:
self.downloadPackages(opts)
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File
"/usr/bin/yumdownloader", line 214, in downloadPackages
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: path =
repo.getPackage(download)
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File
"/usr/lib/python2.4/site-packages/yum/yumRepo.py", line 849, in getPackage
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: cache=cache
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File
"/usr/lib/python2.4/site-packages/yum/yumRepo.py", line 811, in _getFile
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: raise
Errors.RepoError, errstr
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall:
yum.Errors.RepoError: failed to retrieve
kernel/2.6.18/164.el5/x86_64/kernel-xen-2.6.18-164.el5.x86_64.rpm from
anaconda-extrarepo-1
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: error was [Errno
12] Timeout: <urlopen error timed out>
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: kernel
(kernel-xen) doesn't exist for x86_64. skipping
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: Building
minstg.img
[/Log]
Thanks for the help
V!jay
13 years, 5 months
email notifications
by Florian La Roche
Hallo,
With koji-1.4 running on RHEL6-beta2, some email notifications
are not sent out correctly:
Traceback (most recent call last):
File "/usr/sbin/kojid", line 1437, in runTask
response = (handler.run(),)
File "/usr/sbin/kojid", line 1513, in run
return self.handler(*self.params,**self.opts)
File "/usr/sbin/kojid", line 3649, in handler
message = message.encode('utf-8')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 546: ordinal not in range(128)
Thie seems to happen e.g. on dejavu-fonts, ctags, crontabs,
cronie, amanda, akonadi.
regards,
Florian La Roche
13 years, 5 months
--tag option to spin-livecd or spin-appliance
by Jay Greguske
Hello,
This patch implements a --tag option to the cli for the spin-appliance or spin-livecd directives. It causes the <target> parameter to be interpreted as a tag rather than a target. The patch includes minor changes to the client, builder, and some simple logic in the webUI to keep the task options listing clear to the user.
Thanks,
- Jay
13 years, 6 months
Koji cli-build directory
by Alan Franzoni
Hello,
just a small question.
After several months of koji usage, i realize there's a dir,
/mnt/koji/cli-build, containing temporary directories with .src.rpm,
that's growing bigger and bigger.
I suppose Koji is storing here the files he's being requested to
build, but the dir never gets cleaned up.
Is it safe to delete it?
--
Alan Franzoni
--
contact me at public(a)[mysurname].eu
13 years, 6 months
Re: [MASH/KOJI] what about imported RPMs without a build ?
by Nathan Blackham
Did you tag the build?
------Original Message------
From: Thomas Guthmann
Sender: buildsys-bounces(a)lists.fedoraproject.org
To: buildsys(a)lists.fedoraproject.org
ReplyTo: Discussion of Fedora build system
Subject: [MASH/KOJI] what about imported RPMs without a build ?
Sent: Sep 21, 2010 5:55 PM
Hey guys,
I've just discovered that mash does not include packages without a koji
build despite the fact that a RPM has been imported (koji import
--create-build [package]).
Indeed, we have some packages already build with hudson/maven and not
koji so we integrate them in koji to centralize RPMs. But then they
don't appear in the generated mash repositories (used for cobbler).
In mash/__init__.py:286, listTaggedRPMS does not include them. Is there
a workaround (like another koji python method to use or some SQL black
magic) ?
Cheers,
Thomas
--
buildsys mailing list
buildsys(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys
Sent from my Storm2
13 years, 6 months
[MASH/KOJI] what about imported RPMs without a build ?
by Thomas Guthmann
Hey guys,
I've just discovered that mash does not include packages without a koji
build despite the fact that a RPM has been imported (koji import
--create-build [package]).
Indeed, we have some packages already build with hudson/maven and not
koji so we integrate them in koji to centralize RPMs. But then they
don't appear in the generated mash repositories (used for cobbler).
In mash/__init__.py:286, listTaggedRPMS does not include them. Is there
a workaround (like another koji python method to use or some SQL black
magic) ?
Cheers,
Thomas
13 years, 6 months
Re:buildsys Digest, Vol 67, Issue 6
by lixiao@nudt.edu.cn
In your responses,you said 'The task gets released by kojid when it restarts. its then picked up again and started over.'
Howerver,if the task gets released,other builders should pick up the task.But they do not.The task does not go on until the builder powers on.Why?
13 years, 6 months
the cache of the koji
by lixiao@nudt.edu.cn
When I buid the packages in my koji1.3.2,one of the builders powered off.I found the task in the builder did not stop.The task can go on when I power on the builder.Why?Do the koji use the cache?I find in my os there is /var/cache/mock.Could anyone tell me how koji handle it?Thanks.
13 years, 6 months