1:01 p.m.
| https://fedoraproject.org/wiki/Packaging/ReviewGuidelines
|
| MUST: A package must own all directories that it creates. If it does not
| create a directory that it uses, then it should require a package which
| does create that directory.
|
| https://fedoraproject.org/wiki/Packaging/Guidelines#FileAndDirectoryOwner...
| https://fedoraproject.org/wiki/Packaging/UnownedDirectories
Despite being in the guidelines as above, more and more unowned
directories are found in Fedora/Rawhide.
They spring up like mushrooms.
There must be reviewers and packagers who don't understand above
guideline and who are not familiar with how to include files vs. directories
in RPM package %files lists.
For every dozen unowned directories that got fixed after reporting it in
bugzilla, it seems as if two new dozen unowned directories appear (or
reappear) in other packages. My script examines only new packages in
Rawhide (with EVR not seen before), and still there is no end to be seen
in the logs. Lots of obvious packaging mistakes which could have been
noticed easily in the .spec file already, but also lots of unowned
directories which reveal missing dependencies. For dozens of bugzilla
tickets, the packagers don't respond. Sooner or later the bug triagers
will threaten with closing such tickets. Reopening them semi-automatically
will lead to an ugly fight.
I think what I'm trying to express here is that it's a lost cause.
1:47 p.m.
Michael Schwendt wrote:
| https://fedoraproject.org/wiki/Packaging/ReviewGuidelines
|
| MUST: A package must own all directories that it creates. If it does not
| create a directory that it uses, then it should require a package which
| does create that directory.
|
|
https://fedoraproject.org/wiki/Packaging/Guidelines#FileAndDirectoryOwner...
| https://fedoraproject.org/wiki/Packaging/UnownedDirectories
Despite being in the guidelines as above, more and more unowned
directories are found in Fedora/Rawhide.
They spring up like mushrooms.
There must be reviewers and packagers who don't understand above
guideline and who are not familiar with how to include files vs.
directories in RPM package %files lists.
...
I think what I'm trying to express here is that it's a lost
cause.
It may be a dirty job, but the work you put into this is much appreciated.
Education (and subsequent fixes) takes time.
-- Rex
1:48 p.m.
On Fri, Jan 30, 2009 at 2:01 PM, Michael Schwendt <mschwendt(a)gmail.com> wrote:
I think what I'm trying to express here is that it's a lost cause.
I think the conclusion was that this problem would just go away if RPM
reference counted directories? So it doesn't have to be a lost cause;
the conclusion can be instead that it makes sense to
improve/extend/fix core infrastructure rather than having humans
hand-maintain metadata.
2:15 p.m.
On Fri, 30 Jan 2009 14:48:49 -0500, Colin wrote:
I think the conclusion was that this problem would just go away if
RPM
reference counted directories?
When will that happen?
A restrictive umask still creates inaccessible unowned directories when
installing packages. And these bad permissions of such directories cannot
be verified/discovered with "rpm --verify" either.
In other cases, unowned directories are the result of missing
dependencies. And in turn, "yum install foo" installs a non-working
package. Dir refcounting in RPM cannot fix that.
2:25 p.m.
On Fri, Jan 30, 2009 at 3:15 PM, Michael Schwendt <mschwendt(a)gmail.com> wrote:
On Fri, 30 Jan 2009 14:48:49 -0500, Colin wrote:
> I think the conclusion was that this problem would just go away if RPM
> reference counted directories?
When will that happen?
I'd guess earlier than all spec files in Fedora become perfect. I
believe Richard Jones said something recently along the lines of "we
can't treat rpm and lowlevel tools as unchangeable".
A restrictive umask still creates inaccessible unowned directories
when
installing packages. And these bad permissions of such directories cannot
be verified/discovered with "rpm --verify" either.
Yeah, I think I remember some flamewar about that. (The solution
seems obvious to me; yum should set a fixed umask, if we *really* need
to support admins installing RPMs with custom permissions, it only
applies when you use rpm -i, not yum). Or instead have a custom
--umask argument for rpm.
In other cases, unowned directories are the result of missing
dependencies. And in turn, "yum install foo" installs a non-working
package. Dir refcounting in RPM cannot fix that.
Right, but if dependencies are missing things will be broken anyways.
2:51 p.m.
On Fri, 30 Jan 2009, Michael Schwendt wrote:
On Fri, 30 Jan 2009 14:48:49 -0500, Colin wrote:
> I think the conclusion was that this problem would just go away if RPM
> reference counted directories?
When will that happen?
Depends on how high a priority folks want it to have?
A restrictive umask still creates inaccessible unowned directories
when
installing packages.
Rpm sets a default umask 0022 on initialization since 4.4.2.3. Where are
you seeing inaccessible unowned directories being created? Ok, RHEL/Centos
< 5.3 but that's not really Fedora land.
And these bad permissions of such directories cannot
be verified/discovered with "rpm --verify" either.
In other cases, unowned directories are the result of missing
dependencies. And in turn, "yum install foo" installs a non-working
package. Dir refcounting in RPM cannot fix that.
Rpm could stop the install on unowned directories like it does for
file conflicts. Not maybe the nicest user experience but non-working
package is not any better.
- Panu -
3:16 p.m.
On Fri, 30 Jan 2009 22:51:59 +0200 (EET), Panu wrote:
Rpm sets a default umask 0022 on initialization since 4.4.2.3.
That seems to be Fedora >= 9. Good to hear that.
And I can confirm it with F10!
That's one annoyance less. :)
Unremoved versioned directories after package removal is a minor issue.
The inability to RPM-verify unowned directories becomes minor if dirs
default to 0755.
Where are you seeing inaccessible unowned directories being created?
As I recall it, I could still reproduce it end of Nov 2008 in Rawhide, but
there's a small chance I was in a shell on F8 by mistake.
4:07 p.m.
On Fri, 30 Jan 2009, Michael Schwendt wrote:
On Fri, 30 Jan 2009 22:51:59 +0200 (EET), Panu wrote:
> Rpm sets a default umask 0022 on initialization since 4.4.2.3.
That seems to be Fedora >= 9. Good to hear that.
Yup, since Fedora >= 9.
And I can confirm it with F10!
That's one annoyance less. :)
Unremoved versioned directories after package removal is a minor issue.
The inability to RPM-verify unowned directories becomes minor if dirs
default to 0755.
> Where are you seeing inaccessible unowned directories being created?
As I recall it, I could still reproduce it end of Nov 2008 in Rawhide, but
there's a small chance I was in a shell on F8 by mistake.
By Nov 2008 rpm certainly set a default umask, but it only sets it once at
initialization time, so any API user (python or otherwise) has a chance to
still change it to something else. I'm not aware of anything doing that
but possible, yes - or you were on F8 shell :) And sure the umask setting
could be pushed deep into the file processing mechanism to avoid API users
messing with it if absolutely necessary.
- Panu -
3:22 p.m.
On Fri, 2009-01-30 at 22:51 +0200, Panu Matilainen wrote:
Rpm could stop the install on unowned directories like it does for
file conflicts. Not maybe the nicest user experience but non-working
package is not any better.
Why not fail the build if unowned directories are found, just like we do
for unowned files? That way you catch it at build time before you try
and do something useful with the build.
--
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
3:26 p.m.
Jesse Keating píše v Pá 30. 01. 2009 v 13:22 -0800:
Why not fail the build if unowned directories are found, just like we
do
for unowned files? That way you catch it at build time before you try
and do something useful with the build.
How do we determine which directory is
unowned and which is provided by
a dependency? If we don't, every package would have to own /usr.
Mirek
3:26 p.m.
On Fri, 2009-01-30 at 21:26 +0000, Miloslav Trmač wrote:
Jesse Keating píše v Pá 30. 01. 2009 v 13:22 -0800:
> Why not fail the build if unowned directories are found, just like we do
> for unowned files? That way you catch it at build time before you try
> and do something useful with the build.
How do we determine which directory is unowned and which is provided by
a dependency? If we don't, every package would have to own /usr.
Mirek
Good point, since we don't install the Requires at the same time we're
installing the BuildRequires to do the build that would be harder.
--
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
8:14 a.m.
Jesse Keating wrote:
On Fri, 2009-01-30 at 21:26 +0000, Miloslav Trma� wrote:
> Jesse Keating pÃÅ¡e v Pá 30. 01. 2009 v 13:22 -0800:
>> Why not fail the build if unowned directories are found, just like we do
>> for unowned files? That way you catch it at build time before you try
>> and do something useful with the build.
> How do we determine which directory is unowned and which is provided by
> a dependency? If we don't, every package would have to own /usr.
> Mirek
Good point, since we don't install the Requires at the same time we're
installing the BuildRequires to do the build that would be harder.
But not for a
computer ;)
Would there not be a post-build or pre-publish prcess that could be used
to determine whether things like this are correct, and return a 'build
is not publishable' error to the maintainer. Sort of like mock but for
the install ?
I'm guessing that is how Michaels tests work; could they become part of
the infrastructure ?
BTW, is it generally config dirs where the unowned dirs are found ?
DaveT.
5:40 p.m.
Am Sonntag, den 01.02.2009, 01:14 +1100 schrieb David Timms:
BTW, is it generally config dirs where the unowned dirs are found ?
No, most of the time something in %{_usr} or %{_libdir}. And from what
I've seen so far, most of the time the unowned dirs reveal other serious
problems with the packages in question.
Regards,
Christoph
4:36 p.m.
On Fri, 30 Jan 2009, Miloslav Trmač wrote:
Jesse Keating píše v Pá 30. 01. 2009 v 13:22 -0800:
> Why not fail the build if unowned directories are found, just like we do
> for unowned files? That way you catch it at build time before you try
> and do something useful with the build.
How do we determine which directory is unowned and which is provided by
a dependency? If we don't, every package would have to own /usr.
It'd be possible to turn the topmost unowned directory into a file (well,
directory) dependency. Either at build time, which would cause a big pile
of new file dependencies in the metadata, or rpm could generate them at
runtime. The problem with runtime generated dependencies is just that yum
& the like wouldn't be able to resolve them without fairly big changes.
- Panu -
12:21 p.m.
On Sat, 2009-01-31 at 00:36 +0200, Panu Matilainen wrote:
On Fri, 30 Jan 2009, Miloslav Trmač wrote:
> Jesse Keating píše v Pá 30. 01. 2009 v 13:22 -0800:
>> Why not fail the build if unowned directories are found, just like we do
>> for unowned files? That way you catch it at build time before you try
>> and do something useful with the build.
> How do we determine which directory is unowned and which is provided by
> a dependency? If we don't, every package would have to own /usr.
It'd be possible to turn the topmost unowned directory into a file (well,
directory) dependency. Either at build time, which would cause a big pile
of new file dependencies in the metadata, or rpm could generate them at
runtime. The problem with runtime generated dependencies is just that yum
& the like wouldn't be able to resolve them without fairly big changes.
It'd be painful to have everything have a "Requires: /usr" (or
whatever) as it would make filelists mandatory for every update ... but
it would just work, with yum.
--
James Antill <james(a)fedoraproject.org>
Fedora
12:22 p.m.
On Tue, 2009-02-03 at 13:21 -0500, James Antill wrote:
On Sat, 2009-01-31 at 00:36 +0200, Panu Matilainen wrote:
> On Fri, 30 Jan 2009, Miloslav Trmač wrote:
>
> > Jesse Keating píše v Pá 30. 01. 2009 v 13:22 -0800:
> >> Why not fail the build if unowned directories are found, just like we do
> >> for unowned files? That way you catch it at build time before you try
> >> and do something useful with the build.
> > How do we determine which directory is unowned and which is provided by
> > a dependency? If we don't, every package would have to own /usr.
>
> It'd be possible to turn the topmost unowned directory into a file (well,
> directory) dependency. Either at build time, which would cause a big pile
> of new file dependencies in the metadata, or rpm could generate them at
> runtime. The problem with runtime generated dependencies is just that yum
> & the like wouldn't be able to resolve them without fairly big changes.
It'd be painful to have everything have a "Requires: /usr" (or
whatever) as it would make filelists mandatory for every update ... but
it would just work, with yum.
Not that I'd suggest this but we _could_ add dirs to the primary
filelist....
-sv
1:03 p.m.
On Tue, 2009-02-03 at 13:22 -0500, seth vidal wrote:
On Tue, 2009-02-03 at 13:21 -0500, James Antill wrote:
> On Sat, 2009-01-31 at 00:36 +0200, Panu Matilainen wrote:
> > On Fri, 30 Jan 2009, Miloslav Trmač wrote:
> >
> > > Jesse Keating píše v Pá 30. 01. 2009 v 13:22 -0800:
> > >> Why not fail the build if unowned directories are found, just like we
do
> > >> for unowned files? That way you catch it at build time before you
try
> > >> and do something useful with the build.
> > > How do we determine which directory is unowned and which is provided by
> > > a dependency? If we don't, every package would have to own /usr.
> >
> > It'd be possible to turn the topmost unowned directory into a file (well,
> > directory) dependency. Either at build time, which would cause a big pile
> > of new file dependencies in the metadata, or rpm could generate them at
> > runtime. The problem with runtime generated dependencies is just that yum
> > & the like wouldn't be able to resolve them without fairly big
changes.
>
> It'd be painful to have everything have a "Requires: /usr" (or
> whatever) as it would make filelists mandatory for every update ... but
> it would just work, with yum.
Not that I'd suggest this but we _could_ add dirs to the primary
filelist....
I'm not sure that'd work well as you can't tell from just the path
whether it's a file or a dir.
Much better to have something like dir(/usr) provides/requires, this
should then also work with apt-rpm/smart/whatever. But that could be
stage2, if we wanted to try it ... it still might bloat the primary a
bit, but maybe not much.
--
James Antill - james(a)fedoraproject.org
"I'd just like to see a realistic approach to updates via
packages." -- Les Mikesell
3:30 p.m.
On Fri, 30 Jan 2009 13:22:10 -0800, Jesse wrote:
Why not fail the build if unowned directories are found, just like we
do
for unowned files? That way you catch it at build time before you try
and do something useful with the build.
Some directories may be provided only by dependencies. You would need to
list all such dependencies as BuildRequires for a build-failure check
to become quick'n'easy (else it would need a depsolver for a closer look).
6:45 a.m.
On Fri, Jan 30, 2009 at 9:51 PM, Panu Matilainen
<pmatilai(a)laiskiainen.org> wrote:
Rpm could stop the install on unowned directories like it does for
file
conflicts. Not maybe the nicest user experience but non-working package is
not any better.
Which it should do, this way such issues are easier to spot (just
install the package).
1:49 p.m.
Am Freitag, den 30.01.2009, 20:01 +0100 schrieb Michael Schwendt:
| https://fedoraproject.org/wiki/Packaging/ReviewGuidelines
|
| MUST: A package must own all directories that it creates. If it does not
| create a directory that it uses, then it should require a package which
| does create that directory.
|
| https://fedoraproject.org/wiki/Packaging/Guidelines#FileAndDirectoryOwner...
| https://fedoraproject.org/wiki/Packaging/UnownedDirectories
Despite being in the guidelines as above, more and more unowned
directories are found in Fedora/Rawhide.
They spring up like mushrooms.
There must be reviewers and packagers who don't understand above
guideline and who are not familiar with how to include files vs. directories
in RPM package %files lists.
This underlines what I said recently about the quality of (at least some
of) the latest reviews.
Can you give me a list of packages and/or bugs so I can start working in
that?
Regards,
Christoph
5:11 a.m.
On Fri, Jan 30, 2009 at 08:01:18PM +0100, Michael Schwendt wrote:
Despite being in the guidelines as above, more and more unowned
directories are found in Fedora/Rawhide.
I appreciate the work you are doing, but let's look at the bigger
picture here.
Why can't RPM deal with it?
It should be able to track which package(s) needed to create a
particular directory, and remove it once those packages are gone.
Rich.
--
Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones
Read my OCaml programming blog: http://camltastic.blogspot.com/
Fedora now supports 68 OCaml packages (the OPEN alternative to F#)
http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora
6:43 a.m.
On Sat, 31 Jan 2009 11:11:30 +0000, Richard wrote:
Why can't RPM deal with it?
Because it doesn't do it yet? ;)
Perhaps it's not even easy to implement in RPM's current code base?
Sometimes there are lots of reasons why something isn't done. For example,
RPM fails at replacing a symlink with a directory during updates. This has
been discussed several times before [with RPM devs saying that a real
solution would be difficult/complex]. It still hits users around the world
occasionally when package maintainers carelessly hack on their %files
lists.
It should be able to track which package(s) needed to create a
particular directory, and remove it once those packages are gone.
Yes, refcounting directories or creating missing %files %dir entries
on-the-fly sounds like a solution. It seems to be at the bottom of
the TODO list, however.
Interestingly, scanning for unowned directories also reveals misplaced
files.
8:41 a.m.
Michael Schwendt wrote:
Sometimes there are lots of reasons why something isn't done. For
example,
RPM fails at replacing a symlink with a directory during updates.
Or a directory with a symlink, which comes up even more often. It's
extremely annoying when you're trying to fix a directory layout without
breaking backwards compatibility.
Kevin Kofler
11:22 a.m.
On Sat, 2009-01-31 at 15:41 +0100, Kevin Kofler wrote:
Or a directory with a symlink, which comes up even more often.
It's
extremely annoying when you're trying to fix a directory layout without
breaking backwards compatibility.
Right, the question always become "what to do with the
files/folders/symlinks/whatever inside that directory you want to turn
into a symlink?"
--
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
12:14 p.m.
On Sat, 31 Jan 2009 09:22:12 -0800, Jesse wrote:
On Sat, 2009-01-31 at 15:41 +0100, Kevin Kofler wrote:
> Or a directory with a symlink, which comes up even more often. It's
> extremely annoying when you're trying to fix a directory layout without
> breaking backwards compatibility.
Right, the question always become "what to do with the
files/folders/symlinks/whatever inside that directory you want to turn
into a symlink?"
One more reason why packagers ought to become familiar with their
%files lists. ;)
12:32 p.m.
On Sat, 2009-01-31 at 19:14 +0100, Michael Schwendt wrote:
One more reason why packagers ought to become familiar with their
%files lists. ;)
The files a package owns are one thing, the files another package has
placed there, or a user process has placed there, or just the user
themselves have placed there are a different more difficult matter.
--
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
1:12 p.m.
On Sat, 31 Jan 2009 10:32:24 -0800, Jesse wrote:
> One more reason why packagers ought to become familiar with
their
> %files lists. ;)
The files a package owns are one thing, the files another package has
placed there, or a user process has placed there, or just the user
themselves have placed there are a different more difficult matter.
Sure, but the important thing here is to notice that a directory is
turnt into a symlink. Once noticed, packager can think twice and
reconsider. Changing all relevant programs to access the link target
path directly would be one possible solution.
%files lists which are overloaded with macros and recursively included
trees aren't readable and require packagers to use "rpm -qlv" (or rpmls)
to examine packages and understand the contents of a package.
1:34 p.m.
On Sat, Jan 31, 2009 at 01:43:47PM +0100, Michael Schwendt wrote:
On Sat, 31 Jan 2009 11:11:30 +0000, Richard wrote:
> Why can't RPM deal with it?
Because it doesn't do it yet? ;)
Perhaps it's not even easy to implement in RPM's current code base?
Sure thing. We should file a bug (feature enhancement really) against
RPM and then forget about the unowned directories thing until that bug
has been resolved one way or another.
I don't know if you agree with me, but I think the _right_ way to
solve this is once in RPM, not 48235 times over in all the packages in
Fedora. Even if the RPM implementation is a bit tricky, it's surely
easier than pushing this job down to every packager.
Rich.
--
Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones
Read my OCaml programming blog: http://camltastic.blogspot.com/
Fedora now supports 68 OCaml packages (the OPEN alternative to F#)
http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora
5:41 a.m.
Richard W.M. Jones wrote:
On Sat, Jan 31, 2009 at 01:43:47PM +0100, Michael Schwendt wrote:
> On Sat, 31 Jan 2009 11:11:30 +0000, Richard wrote:
>> Why can't RPM deal with it?
> Because it doesn't do it yet? ;)
> Perhaps it's not even easy to implement in RPM's current code base?
Sure thing. We should file a bug (feature enhancement really) against
RPM and then forget about the unowned directories thing until that bug
has been resolved one way or another.
I don't know if you agree with me, but I think the _right_ way to
solve this is once in RPM, not 48235 times over in all the packages in
Fedora. Even if the RPM implementation is a bit tricky, it's surely
easier than pushing this job down to every packager.
As far as I can see right now there is only one thing that RPM could do:
Create a file requires to all directories that the packages places files in
and is not part of the package itself. From my (RPM) perspective this is a
good, save and simple to implement solution. But it comes with the price of
yum requiring (and though downloading) the file list for each and every
transaction. So you solve one problem by creating a new one. Of course this
new problem can surly also be solved by:
1. Manually adding directories other packages are supposed to put files in
as provides (replacing pain by more pain).
2. Selecting the directories and files that createrepo adds to the
primary.xml in a more intelligent way to reduce the requirement of accessing
the file list to some rare cross repo cases (Tricky and already refused by
yum upstream in the past, AFAIK).
Florian
PS: If someone knows better solutions feel free to discuss them here or on
rpm-maint(a)lists.rpm.org
5:51 a.m.
On Sun, 01 Feb 2009 12:41:11 +0100, Florian wrote:
As far as I can see right now there is only one thing that RPM could
do:
Create a file requires to all directories that the packages places files in
and is not part of the package itself. From my (RPM) perspective this is a
good, save and simple to implement solution.
So far, so good. It would make up a %dir entry for all directories not
included in the package's %files list.
But it comes with the price of
yum requiring (and though downloading) the file list for each and every
transaction.
Why that?
There is no "Requires: /orphaned/directory" anywhere in the package.
Yum does not and would not care about the files metadata at all.
RPM would simply add the missing "%dir /orphaned/directory" on-the-fly
during package installation. That would make it possible to remove the
directory with the last package that owns it (RPM %dir refcount == 1 and
directory being empty).
2:35 a.m.
Florian Festi wrote:
Richard W.M. Jones wrote:
<snip>
> I don't know if you agree with me, but I think the _right_
way to
> solve this is once in RPM, not 48235 times over in all the packages in
> Fedora. Even if the RPM implementation is a bit tricky, it's surely
> easier than pushing this job down to every packager.
As far as I can see right now there is only one thing that RPM could do:
Create a file requires to all directories that the packages places files
in and is not part of the package itself. From my (RPM) perspective this
is a good, save and simple to implement solution. But it comes with the
price of yum requiring (and though downloading) the file list for each
and every transaction. So you solve one problem by creating a new one.
Of course this new problem can surly also be solved by:
<snip>
PS: If someone knows better solutions feel free to discuss them here or
on rpm-maint(a)lists.rpm.org
What about Panu's suggestion to just abort the install when trying to install a
file in to a non-existing directory, instead of creating the dir on the fly.
This will not catch all cases, but will catch a lot, and is really easy to
implement (I think).
We could even complement this by also refusing to remove a package if *owned*
files are still left in a dir which it owns (and no other packages own). Then
we catch all error cases AFAIK.
This means people will still need to manually get directory ownership right,
but if their just build rpm fails to install they will learn soon enough!
Regards,
Hans
3:08 a.m.
On Mon, 02 Feb 2009 09:35:37 +0100, Hans wrote:
What about Panu's suggestion to just abort the install when
trying to install a
file in to a non-existing directory, instead of creating the dir on the fly.
Confronting users with these packaging mistakes sounds very wrong to me.
Does "aborting the install" here means that the RPM transaction check
would catch these mistakes? That would be too late, it would break all
those untested mass-dist-updates and create chaos - it would be a DoS
situation for normal updates.
Some packagers rewrite their %files section in spec files so often, they
flip forth and back from owned dirs to unowned dirs, simply because %dir
and recursive inclusion are not understood [yet]. There are even people
on IRC who give wrong/misleading advice wrt dirs in %files sections, and
newbie packagers listen to them.
This will not catch all cases, but will catch a lot, and is really
easy to
implement (I think).
We could even complement this by also refusing to remove a package if *owned*
files are still left in a dir which it owns (and no other packages own). Then
we catch all error cases AFAIK.
In hope that those people, who would be burnt by such a refusal to remove
a pkg, will report all such problems instead of working around them.
No, the problem is not worse enough anymore, since RPM at least uses a
proper umask since F9. Since then it has become more interesting to
examine unowned dirs which are caused by other packaging mistakes (such
as misplaced files, missing sub-pkg deps).
This means people will still need to manually get directory ownership
right,
but if their just build rpm fails to install they will learn soon enough!
The script I'm using to detect unowned dirs in repos is released since
middle/autumn of last year
( http://mschwendt.fedorapeople.org/dircheck-remote.py ).
It has become really easy to check packages in Rawhide.
$ ./dircheck-remote.py -r rawhide -n bit-devel
=> bit-0.4.90-3.fc11.src.rpm
=> bit-devel-0.4.90-3.fc11.i386 (rawhide-development-i386)
/usr/include/bit-0.4
and it can be none or multiple -n options that each accept a regexp, so
it's easy to check all -devel pkgs, for example.
3:15 a.m.
The script I'm using to detect unowned dirs in repos is released
since
middle/autumn of last year
( http://mschwendt.fedorapeople.org/dircheck-remote.py ).
It has become really easy to check packages in Rawhide.
$ ./dircheck-remote.py -r rawhide -n bit-devel
=> bit-0.4.90-3.fc11.src.rpm
=> bit-devel-0.4.90-3.fc11.i386 (rawhide-development-i386)
/usr/include/bit-0.4
and it can be none or multiple -n options that each accept a regexp, so
it's easy to check all -devel pkgs, for example.
I wasn't on this list in middle/autumn of last year, so thanks for
talking about it one more time. Will be great, at least to check my
own packages.
With all the hype about FADs these days, we could even think about a
FAD that would be specifically about running this script and reporting
bugs against the incorrect packages :)
It can also be a great way of having users contribute: "you don't know
how to help ? run this script and report a bug for each package it
returns".
----------
Mathieu Bridon (bochecha)
French Fedora Ambassador
----------
"They who can give up essential liberty to obtain a little temporary
safety, deserve neither liberty nor safety." ~Benjamin Franklin
3:53 a.m.
Le Lun 2 février 2009 10:08, Michael Schwendt a écrit :
No, the problem is not worse enough anymore, since RPM at least uses
a
proper umask since F9. Since then it has become more interesting to
examine unowned dirs which are caused by other packaging mistakes
(such as misplaced files, missing sub-pkg deps).
BTW, I could simplify the multi-font spec template a lot if having
multiple subpackages own the same font directory was accepted.
Currently one of the major quirks many font packagers fail on is the
way we try very hard to have a single canonical owner of the common
font deployment directory for the srpm.
(IIRC people wanted queries on directory ownership to return a single
package)
--
Nicolas Mailhot
7:47 a.m.
Nicolas Mailhot wrote:
BTW, I could simplify the multi-font spec template a lot if having
multiple subpackages own the same font directory was accepted.
Uh, this is already allowed by the guidelines:
https://fedoraproject.org/wiki/Packaging:Guidelines#File_and_Directory_Ow...
Although the rule of thumb is the same: own all the directories you
create
but none of the directories of packages you depend on, there are several
instances where it's desirable for multiple packages to own a directory.
And
then it gives 2 examples. Example 2 sounds a lot like the situation with
those font subpackages.
Kevin Kofler
10:33 a.m.
On Wed, Feb 4, 2009 at 2:47 PM, Kevin Kofler <kevin.kofler(a)chello.at> wrote:
Nicolas Mailhot wrote:
> BTW, I could simplify the multi-font spec template a lot if having
> multiple subpackages own the same font directory was accepted.
What is more probably it will semplify automatically, and enforce
them, what it is required by the Perl Package Guideline in the case
described in the Directory Ownership Section.
Regards
3:48 p.m.
Just to be clear: the directory ownership page says something like "if
you have multiple packages that use the same directory and do not depend
on a common package that owns it they can all own this directory in
parallel".
But with fonts we have cases where
1. the common package exists for other reasons, or
2 it's only there to own the common directory.
In case 2. the guidelines clearly allow dropping common and using
multiple ownership. My problem is case 1.: is it ok for each subpackage
to own the directory it installs fonts to, even though it depends on a
common package that already owns it for other reasons (for example, to
put core fonts indexes in it)?
Because making the font subpackage macro auto-own the font dir in all
cases is trivial, would simplify the templates and avoid packaging
errors, but detecting the presence of a common subpackage to avoid the
auto-owning in that case is *not* trivial at all.
NB: in all this discussion the "common" subpackage is created from the
same srpm and never shared with subpackages from another srpm
--
Nicolas Mailhot
4:42 a.m.
On Mon, 2 Feb 2009, Michael Schwendt wrote:
On Mon, 02 Feb 2009 09:35:37 +0100, Hans wrote:
> What about Panu's suggestion to just abort the install when trying to install a
> file in to a non-existing directory, instead of creating the dir on the fly.
Confronting users with these packaging mistakes sounds very wrong to me.
Does "aborting the install" here means that the RPM transaction check
would catch these mistakes? That would be too late, it would break all
those untested mass-dist-updates and create chaos - it would be a DoS
situation for normal updates.
Yup, that'd be having rpm transaction check fail. And sure it's fairly
draconian, but maybe it'd teach the packagers not to do untested
mass-dist-updates after getting bombarded with bug reports a few times :P
If mock/koji did a test install of the freshly built package into the
build chroot, that'd catch these and all sorts of other silly mistakes too
(Ever made a typo in manual dependency, scriptlet or so, making the
package uninstallable? I know I have...) without exposing the poor users
to packagers mistakes.
Some packagers rewrite their %files section in spec files so often,
they
flip forth and back from owned dirs to unowned dirs, simply because %dir
and recursive inclusion are not understood [yet]. There are even people
on IRC who give wrong/misleading advice wrt dirs in %files sections, and
newbie packagers listen to them.
Would it help just to have rpmbuild *report* unowned directories at
build-time? That would of course give a great deal of false positives
(you'd probably want to filter at least directories from the filesystem
package out), but it would provide an easy way for a packager to eyeball
for anything suspicious. Such as
Note: directories not explicitly owned by package libfoo:
/usr/lib
/usr/share/foo
- Panu -
7:49 a.m.
Panu Matilainen wrote:
If mock/koji did a test install of the freshly built package into
the
build chroot, that'd catch these and all sorts of other silly mistakes too
(Ever made a typo in manual dependency, scriptlet or so, making the
package uninstallable? I know I have...) without exposing the poor users
to packagers mistakes.
That sounds like a good idea. If (and ONLY if) this is implemented, having
the transaction fail on unowned directories would also be acceptable (but
only after a mass rebuild to get all the current instances fixed).
Kevin Kofler
5:42 a.m.
On Sat, 31 Jan 2009 19:34:47 +0000, Richard wrote:
On Sat, Jan 31, 2009 at 01:43:47PM +0100, Michael Schwendt wrote:
> On Sat, 31 Jan 2009 11:11:30 +0000, Richard wrote:
> > Why can't RPM deal with it?
>
> Because it doesn't do it yet? ;)
> Perhaps it's not even easy to implement in RPM's current code base?
Sure thing. We should file a bug (feature enhancement really) against
RPM and then forget about the unowned directories thing until that bug
has been resolved one way or another.
I don't know if you agree with me, but I think the _right_ way to
solve this is once in RPM, not 48235 times over in all the packages in
Fedora. Even if the RPM implementation is a bit tricky, it's surely
easier than pushing this job down to every packager.
And meanwhile, package upgrades continue with creating empty _versioned_
%doc and include directories, for example. Orphaned empty includedirs
have broken non-mock builds of some tarballs before. /usr/share/doc
requires manual clean-up if you really want to browse it with tab
completion.
Yes, I agree, a fix in RPM would be nice to have -- though, it probably
won't fix "repoquery --whatprovides /path/dir" (or Yum's similar cmd).
Waiting for a fix in RPM is no excuse for packagers, however, who don't
know about %dir or recursive inclusion of directories, or who construct
hardly readable spec files which hide the %files lists behind lots
of macros which make it difficult to understand where files are stored.
In some packages, unowned directories appear not before a package becomes
desolate as packaging bugs pile up. A re-review would be justified, e.g.
https://bugzilla.redhat.com/473593
If packagers *and* reviewers need not care about unowned directories
anymore (perhaps because FESCo believes the umask fix is sufficient),
then the packaging guideline ought to be dropped.
Packagers and Reviewers at least ought to take a look at binary pkgs
files lists with "rpm -qlvp" or "rpmls" to understand a pkg's
file
layout.
5553
days inactive
5558
days old
42 comments
18 participants
participants (18)
-
Christoph Wickert -
Colin Walters -
Development discussions related to Fedora -
drago01 -
Florian Festi -
Hans de Goede -
James Antill -
Jesse Keating -
Kevin Kofler -
Mathieu Bridon -
Michael Schwendt -
Miloslav Trmač -
Nicolas Mailhot -
Panu Matilainen -
Rex Dieter -
Richard W.M. Jones -
Seth Vidal -
yersinia