Kernel Module Packaging Standard Teleconference
by Warren Togami
After weeks of discourse on IRC and mailing list discussions, we seem to
be unable to make progress toward an amicable agreement.
What seems to be the issue is that RPM does not allow any clean solution
to the kernel module packaging problem. Both kmod and kmdl are
comprised of ugly hacks in order to overcome the design limitations of
RPM. Both kmod and kmdl have pros and cons, and neither is a clearly
agreed by all parties as being technically superior.
The FESCO side is disconcerted by this situation because a great deal of
effort over years was put into making and ratifying today's kmod
standard. On the other hand, there is some hope for middle-ground
because of stated willingness to further revise kmod to overcome
remaining limitations.
It is the decision of me, Max, and Spot, to put a hold on publishing any
further kmod modules pending the result the Kernel Module Packaging
Standard conference call. Time/date of this call is to be determined,
but we are aiming for this Friday, August 18th at a time that is
workable for both American EDT and Europe. Perhaps 1PM EDT would work,
the usual timeslot of FESCO meetings? Perhaps this timeslot wont work.
If you feel it is important to attend but you are unable to make this
time, please make it known in this thread.
In order to make this meeting manageable, we must limit the number of
attendees. Invited are the existing members of the Fedora Packaging
Committee, Thorsten Leemhuis and certain Red Hat employees relevant to
this discussion. Please talk to me directly if you have been an active
contributor to this in the past but are not included in this invitee list.
My personal feeling is that it is way too late to throw out kmod, OTOH
it is important that we discuss each point to see if kmod can be improved.
If the packaging committee cannot come to key agreements, then the
matter will be escalated to the Fedora Project Board who will make the
key decisions.
Warren Togami
wtogami(a)redhat.com
17 years, 8 months
file-conflicts on updates problem
by Thorsten Leemhuis
Hi all!
I was looking at the current kmod situation a bit again and tested some
stuff and it seems I'm to dump to reproduce the "file-conflicts on
updates problem" -- a problem that I saw myself some weeks ago once IIRC
and that was multiple times brought into the current discussions by Axel.
Seems I'm doing something wrong here. Can somebody tell me that that is?
I uploaded two srpms for testing to
http://www.leemhuis.info/files/fedorarpms/MISC.fdr/kmod/
for people to reproduce the problem locally. That's not a real kmod, but
it uses the kmod-scheme and a "kernel module" for testing which is
created by
> dd if=/dev/urandom bs=1k count=500 > $RPM_BUILD_ROOT/lib/modules/%{kverrel}${kvariant}/extra/%{kmod_name}/foo.ko
Look closer yourself below to understand my confusion. Note, all the yum
calls are on FC5 with latest updates, but without the kmod plugin:
Prepare the test rpms:
> [thl@notebook ~]$ rpmbuild -ba rpmbuild/SPECS/foo-kmod-common.spec
> [thl@notebook ~]$ rpmbuild -ba rpmbuild/SPECS/foo-kmod.spec --define 'kversion 2.6.17-1.2157_FC5' --define 'kvariants ""' --define 'thisrelease 1' --target i686
> [thl@notebook ~]$ rpmbuild -ba rpmbuild/SPECS/foo-kmod.spec --define 'kversion 2.6.17-1.2174_FC5' --define 'kvariants ""' --define 'thisrelease 1' --target i686
> [thl@notebook ~]$ rpmbuild -ba rpmbuild/SPECS/foo-kmod.spec --define 'kversion 2.6.17-1.2174_FC5' --define 'kvariants ""' --define 'thisrelease 2' --target i686
[...]
Put them in a testrepo:
> [thl@notebook ~]$ mkdir -p ~/tmp/testrepo
> [thl@notebook ~]$ cp rpmbuild/RPMS/noarch/foo-kmod-common-1.1-1.noarch.rpm rpmbuild/RPMS/i686/kmod-foo-1.1-1.2.6.17_1.2157_FC5.i686.rpm ~/tmp/testrepo/
> [thl@notebook ~]$ createrepo ~/tmp/testrepo
> 2/2 - kmod-foo-1.1-1.2.6.17_1.2157_FC5.i686.rpm
> Saving Primary metadata
> Saving file lists metadata
> Saving other metadata
Install the first one with yum:
> [thl@notebook ~]$ sudo yum --enablerepo=testrepo install kmod-foo
> Loading "installonlyn" plugin
> Setting up Install Process
> Setting up repositories
[...]
> Parsing package install arguments
> Resolving Dependencies
> --> Populating transaction set with selected packages. Please wait.
> ---> Package kmod-foo.i686 0:1.1-1.2.6.17_1.2157_FC5 set to be installed
> --> Running transaction check
> --> Processing Dependency: foo-kmod-common >= 1.1 for package: kmod-foo
> --> Restarting Dependency Resolution with new changes.
> --> Populating transaction set with selected packages. Please wait.
> ---> Package foo-kmod-common.noarch 0:1.1-1 set to be updated
> --> Running transaction check
>
> Dependencies Resolved
>
> =============================================================================
> Package Arch Version Repository Size
> =============================================================================
> Installing:
> kmod-foo i686 1.1-1.2.6.17_1.2157_FC5 testrepo 504 k
> Installing for dependencies:
> foo-kmod-common noarch 1.1-1 testrepo 2.2 k
>
> Transaction Summary
> =============================================================================
> Install 2 Package(s)
> Update 0 Package(s)
> Remove 0 Package(s)
> Total download size: 506 k
> Is this ok [y/N]: y
> Downloading Packages:
> Running Transaction Test
> Finished Transaction Test
> Transaction Test Succeeded
> Running Transaction
> Installing: foo-kmod-common ######################### [1/2]
> Installing: kmod-foo ######################### [2/2]
> WARNING: Module /lib/modules/2.6.17-1.2157_FC5/extra/foo/foo.ko is not an elf object
>
> Installed: kmod-foo.i686 0:1.1-1.2.6.17_1.2157_FC5
> Dependency Installed: foo-kmod-common.noarch 0:1.1-1
> Complete!
> [thl@notebook ~]$
Copy the second one in the repo and update:
> [thl@notebook ~]$ cp rpmbuild/RPMS/i686/kmod-foo-1.1-1.2.6.17_1.2174_FC5.i686.rpm ~/tmp/testrepo/
> [thl@notebook ~]$ createrepo ~/tmp/testrepo/
> 3/3 - kmod-foo-1.1-1.2.6.17_1.2174_FC5.i686.rpm
> Saving Primary metadata
> Saving file lists metadata
> Saving other metadata
> [thl@notebook ~]$ sudo rm -rf /var/cache/yum/testrepo/
> [thl@notebook ~]$ sudo yum --enablerepo=testrepo update
> Loading "installonlyn" plugin
> Setting up Update Process
> Setting up repositories
> macromedia [1/7]
> livna [2/7]
> testrepo [3/7]
> testrepo 100% |=========================| 951 B 00:00
> updates-testing [4/7]
> core [5/7]
> updates [6/7]
> extras [7/7]
> Reading repository metadata in from local files
> primary.xml.gz 100% |=========================| 1.0 kB 00:00
> testrepo : ################################################## 3/3
> Added 3 new packages, deleted 0 old in 0.05 seconds
> Resolving Dependencies
> --> Populating transaction set with selected packages. Please wait.
> ---> Downloading header for kmod-foo to pack into transaction set.
> kmod-foo-1.1-1.2.6.17_1.2 100% |=========================| 2.4 kB 00:00
> ---> Package kmod-foo.i686 0:1.1-1.2.6.17_1.2174_FC5 set to be installed
> --> Running transaction check
>
> Dependencies Resolved
>
> =============================================================================
> Package Arch Version Repository Size
> =============================================================================
> Installing:
> kmod-foo i686 1.1-1.2.6.17_1.2174_FC5 testrepo 504 k
>
> Transaction Summary
> =============================================================================
> Install 1 Package(s)
> Update 0 Package(s)
> Remove 0 Package(s)
> Total download size: 504 k
> Is this ok [y/N]: y
> Downloading Packages:
> Running Transaction Test
> Finished Transaction Test
> Transaction Test Succeeded
> Running Transaction
> Installing: kmod-foo ######################### [1/1]
> WARNING: Module /lib/modules/2.6.17-1.2174_FC5/extra/foo/foo.ko is not an elf object
>
> Installed: kmod-foo.i686 0:1.1-1.2.6.17_1.2174_FC5
> Complete!
> [thl@notebook ~]$
Works as expected so far. But now:
> [thl@notebook ~]$ cp rpmbuild/RPMS/i686/kmod-foo-1.1-2.2.6.17_1.2174_FC5.i686.rpm tmp/testrepo/
> [thl@notebook ~]$ sudo rm -rf /var/cache/yum/testrepo/
> [thl@notebook ~]$ createrepo ~/tmp/testrepo/
> 4/4 - kmod-foo-1.1-2.2.6.17_1.2174_FC5.i686.rpm
> Saving Primary metadata
> Saving file lists metadata
> Saving other metadata
> [thl@notebook ~]$ sudo yum --enablerepo=testrepo update
> Loading "installonlyn" plugin
> Setting up Update Process
> Setting up repositories
> macromedia [1/7]
> livna [2/7]
> testrepo [3/7]
> testrepo 100% |=========================| 951 B 00:00
> updates-testing [4/7]
> core [5/7]
> updates [6/7]
> extras [7/7]
> Reading repository metadata in from local files
> primary.xml.gz 100% |=========================| 1.1 kB 00:00
> testrepo : ################################################## 4/4
> Added 4 new packages, deleted 0 old in 0.10 seconds
> Resolving Dependencies
> --> Populating transaction set with selected packages. Please wait.
> ---> Downloading header for kmod-foo to pack into transaction set.
> kmod-foo-1.1-2.2.6.17_1.2 100% |=========================| 2.4 kB 00:00
> ---> Package kmod-foo.i686 0:1.1-2.2.6.17_1.2174_FC5 set to be installed
> --> Running transaction check
> --> Populating transaction set with selected packages. Please wait.
> ---> Package kmod-foo.i686 0:1.1-1.2.6.17_1.2157_FC5 set to be erased
> --> Running transaction check
>
> Dependencies Resolved
>
> =============================================================================
> Package Arch Version Repository Size
> =============================================================================
> Installing:
> kmod-foo i686 1.1-2.2.6.17_1.2174_FC5 testrepo 504 k
> Removing:
> kmod-foo i686 1.1-1.2.6.17_1.2157_FC5 installed 500 k
>
> Transaction Summary
> =============================================================================
> Install 1 Package(s)
> Update 0 Package(s)
> Remove 1 Package(s)
> Total download size: 504 k
> Is this ok [y/N]: y
> Downloading Packages:
> Running Transaction Test
> Finished Transaction Test
> Transaction Test Succeeded
> Running Transaction
> Removing : kmod-foo ######################### [1/2]
> Installing: kmod-foo ######################### [2/2]
> WARNING: Module /lib/modules/2.6.17-1.2174_FC5/extra/foo/foo.ko is not an elf object
>
> Removed: kmod-foo.i686 0:1.1-1.2.6.17_1.2157_FC5
> Installed: kmod-foo.i686 0:1.1-2.2.6.17_1.2174_FC5
> Complete!
> [thl@notebook ~]$
Okay, why did that work? I don't know for real. Let's check some details:
> [thl@notebook ~]$ rpm -qa kmod-foo
> kmod-foo-1.1-2.2.6.17_1.2174_FC5
> kmod-foo-1.1-1.2.6.17_1.2174_FC5
> [thl@notebook ~]$ rpm -V kmod-foo-1.1-1.2.6.17_1.2174_FC5
> ..5....T /lib/modules/2.6.17-1.2174_FC5/extra/foo/foo.ko
> [thl@notebook ~]$ rpm -V kmod-foo-1.1-2.2.6.17_1.2174_FC5
> [thl@notebook ~]$
Well, I'd call that "interesting". Seem the "file-conflicts on updates
problem" doesn't exist.
But while testing it I noticed something else. You probably noticed that
in above transaction kmod-foo.i686 0:1.1-1.2.6.17_1.2157_FC5 got removed
-- it seems that's not a rpm/yum feature. It's the installonly-plugin
that does this. Let's get back to the state we were before:
> [thl@notebook ~]$ sudo rpm -ivh /home/thl/tmp/testrepo/kmod-foo-1.1-1.2.6.17_1.2157_FC5.i686.rpm --oldpackage
> Preparing... ########################################### [100%]
> 1:kmod-foo ########################################### [100%]
> WARNING: Module /lib/modules/2.6.17-1.2157_FC5/extra/foo/foo.ko is not an elf object
> [thl@notebook ~]$ sudo rpm -e kmod-foo-1.1-1.2.6.17_1.2174_FC5 kmod-foo-1.1-2.2.6.17_1.2174_FC5
> [thl@notebook ~]$ sudo rpm -ivh /home/thl/tmp/testrepo/kmod-foo-1.1-1.2.6.17_1.2174_FC5.i686.rpm
> Preparing... ########################################### [100%]
> 1:kmod-foo ########################################### [100%]
> WARNING: Module /lib/modules/2.6.17-1.2174_FC5/extra/foo/foo.ko is not an elf object
> [thl@notebook ~]$
Now disable the plugin:
> [thl@notebook ~]$ sudo yum --noplugins --enablerepo=testrepo update
> Setting up Update Process
> Setting up repositories
> macromedia [1/7]
> livna [2/7]
> testrepo [3/7]
> updates-testing [4/7]
> core [5/7]
> updates [6/7]
> extras [7/7]
> Reading repository metadata in from local files
> Resolving Dependencies
> --> Populating transaction set with selected packages. Please wait.
> ---> Downloading header for kmod-foo to pack into transaction set.
> kmod-foo-1.1-2.2.6.17_1.2 100% |=========================| 2.4 kB 00:00
> ---> Package kmod-foo.i686 0:1.1-2.2.6.17_1.2174_FC5 set to be installed
> --> Running transaction check
>
> Dependencies Resolved
>
> =============================================================================
> Package Arch Version Repository Size
> =============================================================================
> Installing:
> kmod-foo i686 1.1-2.2.6.17_1.2174_FC5 testrepo 504 k
>
> Transaction Summary
> =============================================================================
> Install 1 Package(s)
> Update 0 Package(s)
> Remove 0 Package(s)
> Total download size: 504 k
> Is this ok [y/N]: y
> Downloading Packages:
> Running Transaction Test
> Finished Transaction Test
> Transaction Test Succeeded
> Running Transaction
> Installing: kmod-foo ######################### [1/1]
> WARNING: Module /lib/modules/2.6.17-1.2174_FC5/extra/foo/foo.ko is not an elf object
>
> Installed: kmod-foo.i686 0:1.1-2.2.6.17_1.2174_FC5
> Complete!
> [thl@notebook ~]$
Disabling the installonly-plungin in
/etc/yum/pluginconf.d/installonlyn.conf has the same effect. So the "old
modules get removed on updates" problem in reality is a bug AFAICS.
Or I'm probably doing something really wrong here -- but what?
CU
thl
17 years, 8 months
Do we need a Rule "Docs should be packaged as %doc"?
by Thorsten Leemhuis
Hi!
Subject says it all. Reason for this idea:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=199108#c33
Quote:
>> > I will prefer not to move docs from /usr/share/gutenprint/doc to
>> > /usr/share/doc/gutenprint.
>>
>> Hmmm, we don't seem to have anything in the guidelines for this AFAICS. But IMHO
>> all docs should be marked as %doc and thus should land in
>> /usr/share/doc/<packagename-version-release> (the proper place used by all other
>> packages)
>>
>> Maybe we need to add such a rule :-/
>
> Sure if you think like that. Primary looking at package said me that let that
> doc files be in /usr/share/gutenprint/doc
> Then i check under /usr/share on my system using
> find . -name doc * and i got following output
> ./sane/xsane/doc
> ./cups/doc
> ./apps/quanta/doc
> ./sgml/docbook/xsl-stylesheets-1.69.1-5/htmlhelp/doc
> ./scrollkeeper/doc
> ./vim/vim70/doc
> ./eclipse/plugins/org.python.pydev_0.9.3/PySrc/ThirdParty/logilab/common/doc
> ./eclipse/plugins/org.python.pydev_0.9.3/PySrc/ThirdParty/logilab/pylint/doc
> ./pear/doc
> ./gutenprint/doc
> where some of the entries belongs to Fedora Core packages.
>
> So it looks to me that either we have different strategy for Fedora Extras or we
> have some Guidelines that will require a major changes when a package moves from
> Fedora Extras to Fedora Core. Then i would like to see that Guidelines page.
CU
thl
17 years, 8 months
mixed-use-of-spaces-and-tabs
by Jason L Tibbitts III
Does anyone else find this rpmlint warning annoying? Do we really
care that much about how packagers make things line up, or whether
they make things line up at all? I can see a problem with mixing tabs
and spaces on a single line, but the warning triggers when it finds
both three consecutive spaces and a single tab anywhere in the
specfile.
- J<
17 years, 8 months
Friday 1PM EDT IRC Meeting
by Warren Togami
It seems that most of the stakeholders would prefer an IRC meeting
instead of a teleconference call.
We will meet in #fedora-packaging at 1PM EDT Friday (same time of day as
FESCO meetings) to discuss this topic.
Warren Togami
wtogami(a)redhat.com
17 years, 8 months
The .pc and pkgconfig issue
by Jason L Tibbitts III
I know a month or so ago we voted to add a bit to the guidelines
reinforcing the necessity of -devel packages with .pc files requiring
pkgconfig. I went to file a bug about a core package that wasn't
doing this (which resulted in bustage of a package I was reviewing):
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=202280
It seems Matthias Clasen (mclasen(a)redhat.com) wants to debate this and
I'm not sure what to do. I don't have any arguments other than
directory ownership and "that's what the committee decided".
- J<
17 years, 8 months
Re: Attention kernel module project packagers!
by Thorsten Leemhuis
David Woodhouse schrieb:
> On Tue, 2006-08-15 at 09:50 -0400, Jesse Keating wrote:
>> Given that we don't want it on Core or Extras, I'm pretty happy to
>> let random 3rd party packager do whatever they want for packaging
>> modules. I'm not interested in dictating how they should handle
>> this ugly hack.
>>
>> Your example about ntfs is not usable w/out the userland
>> (ntfsprogs), which nobody wants to touch due to legal reasons, and
>> would be obsoleted by FUSE anyway where the most recent ntfs
>> support is done entirely in userspace.
>>
>> There are many more things the packaging committee can spend time
>> worrying about. Packaging of kernel modules isn't one of them
>> IMHO.
>
> Yeah, that's a fair point. However, it would be useful if those who
> _do_ care about kernel module packages would come to an agreement
> about how it should be done, and that can be documented somewhere
> central to Fedora -- like on the Fedora wiki.
>
> We can modify our kernel RPM and yum if appropriate in order to
> support that agreed method.
That already happend -- FESCo worked out and agreed on a propsoal last
winter http://www.fedoraproject.org/wiki/Packaging/KernelModules
It's working fine.
CU
thl
17 years, 8 months
Re: atrpms kernel modules
by Axel Thimm
On Tue, Jul 18, 2006 at 01:59:34PM +0200, Eric Tanguy wrote:
> > kernels=`rpm -qf /boot/vmlinuz-* | grep -v "^file .* is not owned by any package"`
> > uname_rs=`rpm -ql $kernels | grep ^/boot/vmlinuz- | sed -e's,^/boot/vmlinuz-\(.*\)$,\1,'`
> > for kmdl in `rpm -qa \*kmdl\* | sed -e's,-kmdl-.*,-kmdl,' | sort -u`;
> > do
> > for uname_r in $uname_rs; do
> > package=${kmdl}-$uname_r
> > rpm -q $package > /dev/null 2>&1 || echo $package
> > done
> > done | xargs -r smart install -y
>
> Ok this script seems to use smart. is it working with yum ?
Yes, I think "smart install -y" -> "yum -y install" is the only change
needed. And for apt it would be "apt-get -y install".
> How is this problem handled by livna because it works fine using
> this repo : the kernel modules are installed and not updated
> It's known that there are issues with this scheme, or any scheme
> that will merge module and kernel versions.
rpm for one can't cope with it. Suppose you have two active kernels
(say 2.6.16 and 2.6.17 or xen0 and non-xen0 etc.) and already have
foo-kmdl for both in version 1. Now foo-kmdl in version 2 is
released. Both rpm -i (coinstall, no replacement of of foo-kmdl for
the same kernel) and rpm -U (remove all foo-kmdl but the highest one,
e.g. at least one kernel stay w/o any kmdl) won't work.
yum/smart/apt can be tought to be more clever than rpm and try to do
the proper thing, which they will only be able to do if the merged
versions are distangled again, e.g. hidden in Provides: and the like.
So in order for a merged-version scheme to work there is special
handling neccessary (and for atrpms' system, too, but read on). I
prefer to have rpm itself already do something sensible and since it
can only upgrade one versioning system (e.g. either the kernel's or
the module's) it's preferable to allow it to upgrade the module within
the same kernel and not touch the cross-kernel boundaries. "No harm
done" like accidentially removing kmdls for other kernels or
coinstalling several kmdls for the same kernel. You "only" need to
reinstall the missing kmdl upon each kernel upgrade.
It's far easier to get this into the brains of yum/smart/apt
(e.g. translate the 9 lines of bash above into plugings/hooks for
these depsolvers) or simply use the 9liner above than have a scheme
which is broken at the level of rpm.
Incidentially one on my personal targets is to get thie discussed in
the fedora packaging group and review the currently scheme. Of course,
I'm proposing adoption of ATrpms' kmdl scheme :)
http://www.fedoraproject.org/wiki/Packaging/GuidelinesTodo
> when i install a new kernel and are removed when the kernel is
> removed.
How the dependencies to the kernel are installed (which is responsible
for having an automatic removal on kernel removal) is a different
story, which the version merging does not break. The issue is with the
system (system=rpm and(or any higher depsolver tool,
e.g. yum/smart/apt) being able to distinguish different foo-kmdls as
_upgrades_ vs _coinstalls_.
--
Axel.Thimm at ATrpms.net
17 years, 8 months
Re: Attention kernel module project packagers!
by Jesse Keating
On Tuesday 15 August 2006 09:40, David Woodhouse wrote:
> I'm not necessarily suggesting that we shouldn't have an agreed method
> of building kernel module packages at all -- just that we shouldn't have
> any such packages in Core or Extras.
>
> There _are_ relatively sane (and legal, unlike nvidia/ati stuff) cases
> where one might want to build a separate module -- like the NTFS modules
> in Livna, for example. And other 'new drivers' which aren't yet
> upstream. Of course you're right when you agree with me that those new
> drivers shouldn't be in Core or Extras -- but that doesn't mean we
> shouldn't provide a way to package them at all.
Given that we don't want it on Core or Extras, I'm pretty happy to let random
3rd party packager do whatever they want for packaging modules. I'm not
interested in dictating how they should handle this ugly hack.
Your example about ntfs is not usable w/out the userland (ntfsprogs), which
nobody wants to touch due to legal reasons, and would be obsoleted by FUSE
anyway where the most recent ntfs support is done entirely in userspace.
There are many more things the packaging committee can spend time worrying
about. Packaging of kernel modules isn't one of them IMHO.
--
Jesse Keating
Release Engineer: Fedora
17 years, 8 months
Re: Attention kernel module project packagers!
by Jesse Keating
On Tuesday 15 August 2006 08:48, David Woodhouse wrote:
> I think this argument is invalid. We should ban _all_ module packages
> from Core and Extras anyway.
I'm with David on this one. Packaging of modules is the path to insanity. It
requires all kinds of weird hacks to how they are built, how they are named,
how they are handled by rpm and depsolvers, they always lag, they always hold
users behind when critical kernel fixes come out, etc, etc, etc...
I personally feel (as does David) that if the module isn't good enough for
upstream, then it would HAVE to live in the kernel package itself. If it's
not good enough for the kernel package itself, then it isn't good enough for
Fedora. (Same could be applied to RHEL, but that's a battle for a different
list).
Arguing over which ugly ass hack to apply to be able to package kernel modules
is a bikeshed argument.
--
Jesse Keating
Release Engineer: Fedora
17 years, 8 months
