As promised, here is my new proposed draft for handling static libs:
http://fedoraproject.org/wiki/PackagingDrafts/StaticLibraryPolicy
I know that it won't make everyone happy (it doesn't just leave static bits in -devel), but we really do want to track who is building against static libraries.
~spot
Tom "spot" Callaway wrote:
As promised, here is my new proposed draft for handling static libs:
http://fedoraproject.org/wiki/PackagingDrafts/StaticLibraryPolicy
I know that it won't make everyone happy (it doesn't just leave static bits in -devel), but we really do want to track who is building against static libraries.
~spot
-- Fedora-packaging mailing list Fedora-packaging@redhat.com https://www.redhat.com/mailman/listinfo/fedora-packaging
I can live with it (and I deal with a lot of static libraries...)
- Orion
Tom "spot" Callaway wrote:
As promised, here is my new proposed draft for handling static libs:
http://fedoraproject.org/wiki/PackagingDrafts/StaticLibraryPolicy
I know that it won't make everyone happy (it doesn't just leave static bits in -devel), but we really do want to track who is building against static libraries.
From item #2: """ If the *-static-noshared package is no longer necessary, it should be removed, and Provided/Obsoleted by the *-devel package (not by the *-static package). """
I don't think we want to be Providing *-static-noshared in this case although the Obsolete makes sense.
From item #3: """ When a package only provides static libraries you can place all the static library files in the *-devel subpackage. When doing this you also have to have a virtual Provide for the *-static and *-static-noshared packages: """
It seems like we should only have a Provide for *-static-noshared as this is a special case of item #2. Thoughts on that?
-Toshio
Toshio Kuratomi wrote:
Tom "spot" Callaway wrote: From item #3: """ When a package only provides static libraries you can place all the static library files in the *-devel subpackage. When doing this you also have to have a virtual Provide for the *-static and *-static-noshared packages: """
It seems like we should only have a Provide for *-static-noshared as this is a special case of item #2. Thoughts on that?
I actually think we only should have a Provide for *-static, so that people who want to use static libs now and in the future (when there may be a shared version) , can guarantee they will get the static version by BuildRequiring the -static, since very few packages will ever have a real *-static-noshared, having a virtual provides for this feels wrong.
Regards,
Hans
On Wed, 2008-04-09 at 14:17 +0200, Hans de Goede wrote:
Toshio Kuratomi wrote:
Tom "spot" Callaway wrote: From item #3: """ When a package only provides static libraries you can place all the static library files in the *-devel subpackage. When doing this you also have to have a virtual Provide for the *-static and *-static-noshared packages: """
It seems like we should only have a Provide for *-static-noshared as this is a special case of item #2. Thoughts on that?
I actually think we only should have a Provide for *-static, so that people who want to use static libs now and in the future (when there may be a shared version) , can guarantee they will get the static version by BuildRequiring the -static, since very few packages will ever have a real *-static-noshared, having a virtual provides for this feels wrong.
The problem is two-fold:
1. We want to be able to track when packages are building against static libraries, whether they are static or static-noshared. 2. When a package goes from only providing static libraries to providing some shared libraries (but not all), we want to be able to track these. If we have these packages BuildRequire the static provide, that won't be correct anymore (we want them to use the shared libraries + static-noshared).
Realistically, what Toshio says is correct, we strictly speaking only need the Provide for *-static-noshared there. I kept the other *-static provide since it is how we used to do it.
~spot
Tom "spot" Callaway wrote:
On Wed, 2008-04-09 at 14:17 +0200, Hans de Goede wrote:
I actually think we only should have a Provide for *-static, so that people who want to use static libs now and in the future (when there may be a shared version) , can guarantee they will get the static version by BuildRequiring the -static, since very few packages will ever have a real *-static-noshared, having a virtual provides for this feels wrong.
The problem is two-fold: ...
Thanks, I think I finally have a grasp of all the motivations here, and agree the new draft is the best way forward.
-- Rex
p.s. static libraries suck
On Wed, Apr 09, 2008 at 09:44:36AM -0400, Tom spot Callaway wrote:
- When a package goes from only providing static libraries to providing
some shared libraries (but not all), we want to be able to track these.
Does it happens? I guess that this is raised by a real life example, but is there more than one package providing some library as shared+static and some only as static?
-- Pat
On Thu, 2008-04-10 at 09:50 +0200, Patrice Dumas wrote:
On Wed, Apr 09, 2008 at 09:44:36AM -0400, Tom spot Callaway wrote:
- When a package goes from only providing static libraries to providing
some shared libraries (but not all), we want to be able to track these.
Does it happens?
This happens all the time.
The critical case would be the opposite: A package going from providing shared libs to providing static libs only.
I have never seen this happen, but provides the craziness of some upstreams, I would not exclude this to happen.
I guess that this is raised by a real life example, but is there more than one package providing some library as shared+static and some only as static?
This isn't much of a problem.
* If a library's client package BR:'s *-devel, it will pick up the shared library during the next rebuild.
* If a library's client package BR:'s *-static, it will bomb out during the next rebuild.
The only issue is library-client packages not being automatically notified that they might need to be rebuilt.
To me, this is a negligible, minor issue, your proposal is too heavy weight for to find it appropriate.
We have way more serious packaging issues than this minor detail.
Ralf
On Thu, Apr 10, 2008 at 10:23:54AM +0200, Ralf Corsepius wrote:
On Thu, 2008-04-10 at 09:50 +0200, Patrice Dumas wrote:
On Wed, Apr 09, 2008 at 09:44:36AM -0400, Tom spot Callaway wrote:
- When a package goes from only providing static libraries to providing
some shared libraries (but not all), we want to be able to track these.
Does it happens?
This happens all the time.
I don't think so. Currently there are no such packages in fedora, with a mix of shared and static libs in -devel.
I did the following: repoquery --whatprovides '*-static' --qf '%{name}' > provides_static_names for file in `grep -v -- '-static$' provides_static_names`; do repoquery -ql $file | grep '.so$'; done /usr/lib/libruby.so /usr/lib/libfrysk-junit.so /usr/lib/libnettle.so /usr/libexec/p7zip/7z.so
I filled a bug against nettle-devel, p7zip and frysk are false positives (they provides a file ending in -static), so ruby-devel is the only package doing that sort of thing, but the libraries have different names: /usr/lib/libruby-static.a /usr/lib/libruby.so
But maybe you are referring to a package providing only static libraries then providing shared libraries (in -devel) and static libraries (in -static). In that case, I think it is not different from the case when a static library is rebuilt and therefoer the package linking against that library needs to be rebuilt too. This is covered by the guidelines:
"If you link statically against a library, add yourself to the initialcc list for the library so you can watch for any security issues or bug fixes for which you'd want to rebuild your package against a new version of the library. Here are instructions for making that request."
In my opinion this also covers going from static to shared, in that case the maitainer can fill himself a bug when going shared.
The critical case would be the opposite: A package going from providing shared libs to providing static libs only.
I have never seen this happen, but provides the craziness of some upstreams, I would not exclude this to happen.
Indeed, it may happen, but just following the existing guidelines in that case seems enough to me.
- If a library's client package BR:'s *-devel,
it will pick up the shared library during the next rebuild.
- If a library's client package BR:'s *-static,
it will bomb out during the next rebuild.
The only issue is library-client packages not being automatically notified that they might need to be rebuilt.
As I said above, having the maintainer fill a bug when going shared should solve this issue, since packagers should already be in CC.
The only issue I see is when a packager doesn't know that he is building against a static library.
My investigations show that (with ruby excluded), the following -devel packages consist only of static libs and have dependent packages:
--> factory-devel libfac-0:3.0.3-2.fc9.src Macaulay2-0:1.1-1.fc9.src --> flam3-devel qosmic-0:1.3.1-3.fc9.src --> g2clib-devel ncl-0:5.0.0-10.fc9.src --> libassuan-devel gnupg2-0:2.0.9-1.fc9.src dirmngr-0:1.0.1-2.fc9.src --> libfac-devel Macaulay2-0:1.1-1.fc9.src --> libnet-devel tcptraceroute-0:1.5-0.5.beta7.fc9.src dsniff-0:2.4-0.2.b1.fc9.src etherbat-0:1.0.1-4.fc9.src libnids-0:1.22-4.fc9.src heartbeat-0:2.1.3-1.fc9.src ettercap-0:0.7.3-22.fc9.src isic-0:0.07-2.fc9.src intuitively-0:0.7-13.fc9.src firewalk-0:5.0-2.fc9.src
Also libmimedir-devel and osiv-devel requires the corresponding -static subpackages since there are no corresponding shared libraries, and uulib-static provides uulib-devel. This adds:
# repoquery --whatrequires uulib-devel --archlist=src klibido-0:0.2.5-10.fc9.src # repoquery --whatrequires uulib-static --archlist=src nget-0:0.27.1-8.fc9.src # repoquery --whatrequires libmimedir-devel --archlist=src librra-0:0.11-2.fc9.src
Also I noticed that some packages BuildRequires static packages, I filled few bugs, but most of the time, the -devel is also BuildRequires, and I guess that there is a good reason to BR the static sub package.
-- Pat
On Wed, 2008-04-09 at 09:44 -0400, Tom "spot" Callaway wrote:
On Wed, 2008-04-09 at 14:17 +0200, Hans de Goede wrote:
Toshio Kuratomi wrote:
Tom "spot" Callaway wrote: From item #3: """ When a package only provides static libraries you can place all the static library files in the *-devel subpackage. When doing this you also have to have a virtual Provide for the *-static and *-static-noshared packages: """
It seems like we should only have a Provide for *-static-noshared as this is a special case of item #2. Thoughts on that?
I actually think we only should have a Provide for *-static, so that people who want to use static libs now and in the future (when there may be a shared version) , can guarantee they will get the static version by BuildRequiring the -static, since very few packages will ever have a real *-static-noshared, having a virtual provides for this feels wrong.
The problem is two-fold:
- We want to be able to track when packages are building against static
libraries, whether they are static or static-noshared.
What for? IMO this is simply bureaucracy.
- When a package goes from only providing static libraries to providing
some shared libraries (but not all), we want to be able to track these.
But this isn't what your proposal does.
Your proposal pesters/pollutes library-clients *.specs with a library's provider's packaging details, these library-clients are not interested in.
If we have these packages BuildRequire the static provide, that won't be correct anymore (we want them to use the shared libraries + static-noshared).
Realistically, what Toshio says is correct, we strictly speaking only need the Provide for *-static-noshared there. I kept the other *-static provide since it is how we used to do it.
This is something completely different.
Here, you seem to be talking about "Additionally providing *-static-noshared" and NOT to pester library-clients with BR: *-static-nonshared".
In real world, no library-client who needs a static library, needs know if this library is being provided "*-static-noshared" or "*-static". => Such kind of Provides is useless to the library-client.
Ralf
On Tue, 2008-04-08 at 17:51 -0400, Tom "spot" Callaway wrote:
As promised, here is my new proposed draft for handling static libs:
http://fedoraproject.org/wiki/PackagingDrafts/StaticLibraryPolicy
-1
I don't understand the difference between #2 and #3.
Also I don't find the *-nonshared packages useful.
Ralf
On Tue, 2008-04-08 at 17:51 -0400, Tom "spot" Callaway wrote:
As promised, here is my new proposed draft for handling static libs:
http://fedoraproject.org/wiki/PackagingDrafts/StaticLibraryPolicy
-1
I don't understand the difference between #2 and #3.
Also I don't find the *-nonshared packages useful.
I assume the point would be to ban BR: *-static in Fedora packages, but allow BR: *-nonshared - i.e. when there is no alternative. Is that correct?
- orion
On Wed, 2008-04-09 at 00:59 +0200, Ralf Corsepius wrote:
I don't understand the difference between #2 and #3.
It's a subtle distinction.
In 2, you have some static libraries and some shared libraries, but the static librar{y,ies} don't have shared alternatives. We don't want to stuff the static ones into the -devel package as we then lose the ability to track what packages statically link against said library, and we don't want to put them in -static as we then run the risk of statically linking to /all/ the static libraries, even those that have shared alternatives. In 3, there is /only/ static libraries, which if we were to try splitting out the static libraries you'd wind up with an empty -devel subpackage. That's why it's OK then to put the static libraries directly in the -devel subpackage, but still packages which link to those static BR the -static provides.
Nonshared subpackage is needed to isolate the static with no alternative libraries from the static with alternative libraries. This way you don't run the risk of statically linking to /all/ the static libraries, even those that have shared alternatives.
On Tue, 2008-04-08 at 17:51 -0400, Tom "spot" Callaway wrote:
As promised, here is my new proposed draft for handling static libs:
http://fedoraproject.org/wiki/PackagingDrafts/StaticLibraryPolicy
I know that it won't make everyone happy (it doesn't just leave static bits in -devel), but we really do want to track who is building against static libraries.
Seems reasonable enough to me.
Later, /B
Tom spot Callaway (tcallawa@redhat.com) said:
As promised, here is my new proposed draft for handling static libs:
http://fedoraproject.org/wiki/PackagingDrafts/StaticLibraryPolicy
I know that it won't make everyone happy (it doesn't just leave static bits in -devel), but we really do want to track who is building against static libraries.
I'd rather just require them to be in -static instead of -static-noshared - they can still be tracked that way.
Bill
On Tue, 2008-04-08 at 23:12 -0400, Bill Nottingham wrote:
I'd rather just require them to be in -static instead of -static-noshared
- they can still be tracked that way.
The problem (as described to me) is that if you put them in -static, and you BR -static, you then potentially link against /all/ the static libraries, even those that have shared alternatives. The motivation was to isolate the static libraries which have no shared alternative from those that do.
We can still "track" things which BR -static-noshared just as easily as we can track those that BR -static.
On Wed, 2008-04-09 at 07:37 -0400, Jesse Keating wrote:
On Tue, 2008-04-08 at 23:12 -0400, Bill Nottingham wrote:
I'd rather just require them to be in -static instead of -static-noshared
- they can still be tracked that way.
The problem (as described to me) is that if you put them in -static, and you BR -static, you then potentially link against /all/ the static libraries, even those that have shared alternatives.
How that? Our rule has been that *-static must Require *-devel, i.e. unless a package is playing nasty games with linking (or this packaging rule is being obeyed), it will always link dynamically.
The motivation was to isolate the static libraries which have no shared alternative from those that do.
We can still "track" things which BR -static-noshared just as easily as we can track those that BR -static.
I still fail to see the usefulness of this.
Our logic had been: Client-packages who intentionally want to link statically, must BR: *-static, those who don't care should BR: *-devel.
With your approach above, client-packages will have to care about characteristics of a package providing a static library.
This doesn't make any sense to me on the client-side.
Ralf
On Wed, 2008-04-09 at 14:39 +0200, Ralf Corsepius wrote:
The problem (as described to me) is that if you put them in -static, and you BR -static, you then potentially link against /all/ the static libraries, even those that have shared alternatives.
How that? Our rule has been that *-static must Require *-devel, i.e. unless a package is playing nasty games with linking (or this packaging rule is being obeyed), it will always link dynamically.
Since spot was the person who described it to me, perhaps it would be best to get his input here. The way he stated it was that if there were static libs around at link time, they would get automatically linked, even if the didn't want them to.
The motivation was to isolate the static libraries which have no shared alternative from those that do.
We can still "track" things which BR -static-noshared just as easily as we can track those that BR -static.
I still fail to see the usefulness of this.
Our logic had been: Client-packages who intentionally want to link statically, must BR: *-static, those who don't care should BR: *-devel.
With your approach above, client-packages will have to care about characteristics of a package providing a static library.
This doesn't make any sense to me on the client-side.
It is a minor annoyance on the client side, but the client side /should/ know if they are linking to anything static. In fact, if they do, they have to be on the initialcc of said static package. It's an extra burden you take on by linking statically.
On Wed, 2008-04-09 at 09:27 -0400, Jesse Keating wrote:
Since spot was the person who described it to me, perhaps it would be best to get his input here. The way he stated it was that if there were static libs around at link time, they would get automatically linked, even if the didn't want them to.
A lot of packages will look first for static libraries, then if (and only if) they are not found, look for shared libraries. By splitting into static and static-noshared, we can safely put in -devel and -static-noshared and avoid this confusion.
~spot
On Wed, 2008-04-09 at 09:47 -0400, Tom "spot" Callaway wrote:
On Wed, 2008-04-09 at 09:27 -0400, Jesse Keating wrote:
Since spot was the person who described it to me, perhaps it would be best to get his input here. The way he stated it was that if there were static libs around at link time, they would get automatically linked, even if the didn't want them to.
A lot of packages will look first for static libraries, then if (and only if) they are not found, look for shared libraries.
Examples? I am not aware of any such case.
Also, this will never happen in a chroot unless a package BR:'s *-static or if a *-devel contains a static library.
By splitting into static and static-noshared, we can safely put in -devel and -static-noshared and avoid this confusion.
Which confusion? I don't see any such confusion. The only situation such case may occur is with packages whose maintainers have been ignorant on the *-static/*-devel rule so far.
Ralf
On Wed, 2008-04-09 at 16:10 +0200, Ralf Corsepius wrote:
Also, this will never happen in a chroot unless a package BR:'s *-static or if a *-devel contains a static library.
By splitting into static and static-noshared, we can safely put in -devel and -static-noshared and avoid this confusion.
Which confusion? I don't see any such confusion. The only situation such case may occur is with packages whose maintainers have been ignorant on the *-static/*-devel rule so far.
For the case where you have some shared, some static with matching shared, and some static only:
If you put the static only in -devel, we can't reasonably detect all the things that link against the static library. We'd have to investigate anything that BRs the -devel package. If you put the static only in with the other statics in -static you then have all the statics in the chroot and run the risk that spot talks about of accidentally statically linking to things that have shared alternatives.
Jesse Keating (jkeating@redhat.com) said:
For the case where you have some shared, some static with matching shared, and some static only:
If you put the static only in -devel, we can't reasonably detect all the things that link against the static library. We'd have to investigate anything that BRs the -devel package. If you put the static only in with the other statics in -static you then have all the statics in the chroot and run the risk that spot talks about of accidentally statically linking to things that have shared alternatives.
The only way this can happen (AFAIK) is if they're linked by:
- explicitly listing %{_libdir}/libfoo.a on the link line - explicitly passing -Wl,-bstatic
Either of these things should be auditable.
Bill
On Wed, 2008-04-09 at 22:54 -0400, Bill Nottingham wrote:
The only way this can happen (AFAIK) is if they're linked by:
- explicitly listing %{_libdir}/libfoo.a on the link line
- explicitly passing -Wl,-bstatic
Either of these things should be auditable.
Ok, if that's the case, we can relax the need for the -noshared subpackage. I'm really just going on the word of others here.
On Wed, 2008-04-09 at 23:17 -0400, Jesse Keating wrote:
On Wed, 2008-04-09 at 22:54 -0400, Bill Nottingham wrote:
The only way this can happen (AFAIK) is if they're linked by:
- explicitly listing %{_libdir}/libfoo.a on the link line
- explicitly passing -Wl,-bstatic
Either of these things should be auditable.
Ok, if that's the case, we can relax the need for the -noshared subpackage. I'm really just going on the word of others here.
I've rewritten it to remove the need for the -noshared subpackage:
http://fedoraproject.org/wiki/PackagingDrafts/StaticLibraryPolicy
In plain English:
When shared libraries exist, static libraries go into -static (whether they have a shared library counterpart or not). Any packages which link against static libraries must BuildRequire: the -static subpackage. If (and only if) there are no shared libraries provided, then the static libraries can go in -devel, but -devel must also provide -static.
It is worth noting that this draft is no longer significantly different from the current policies, but is a clarification to try and eliminate confusion.
~spot
On Tue, Apr 22, 2008 at 12:41:02PM -0400, Tom spot Callaway wrote:
I've rewritten it to remove the need for the -noshared subpackage:
http://fedoraproject.org/wiki/PackagingDrafts/StaticLibraryPolicy
If (and only if) there are no shared libraries provided, then the static libraries can go in -devel, but -devel must also provide -static.
I have 2 related questions, maybe they could be part of this, if they hapen to need a guideline. It is in the case of no shared libraries provided:
1. Is it permitted to have a subpackage called mylib-static which holds what is generally in a devel subpackage, and Provides: mylib-devel = %{version}-%{release}
2. Is it permitted to have the headers in devel and the static library in -static and have mylib-devel Requires: mylib-static = %{version}-%{release}
-- Pat
On Tue, 2008-04-22 at 19:19 +0200, Patrice Dumas wrote:
- Is it permitted to have a subpackage called
mylib-static which holds what is generally in a devel subpackage, and Provides: mylib-devel = %{version}-%{release}
No.
- Is it permitted to have the headers in
devel and the static library in -static and have mylib-devel Requires: mylib-static = %{version}-%{release}
Yes. But you'd still need to have any packages which link against the static libraries BuildRequire: mylib-static.
~spot
On Tue, Apr 22, 2008 at 01:25:07PM -0400, Tom spot Callaway wrote:
On Tue, 2008-04-22 at 19:19 +0200, Patrice Dumas wrote:
- Is it permitted to have a subpackage called
mylib-static which holds what is generally in a devel subpackage, and Provides: mylib-devel = %{version}-%{release}
No.
I think that we shouldn't say too much in the guidelines, but maybe this should be said, since currently there is a package that does that. (I have filed the bug Bug 443649, package is uudeview).
- Is it permitted to have the headers in
devel and the static library in -static and have mylib-devel Requires: mylib-static = %{version}-%{release}
Yes. But you'd still need to have any packages which link against the static libraries BuildRequire: mylib-static.
Indeed.
As a side note, I would have said the same (No for 1., Yes for 2.).
-- Pat
Tom spot Callaway (tcallawa@redhat.com) said:
Since spot was the person who described it to me, perhaps it would be best to get his input here. The way he stated it was that if there were static libs around at link time, they would get automatically linked, even if the didn't want them to.
A lot of packages will look first for static libraries, then if (and only if) they are not found, look for shared libraries. By splitting into static and static-noshared, we can safely put in -devel and -static-noshared and avoid this confusion.
That's not the case for anything that just passes -l<foo>. Can't we just fix those packages?
Bill
Bill Nottingham wrote:
Tom spot Callaway (tcallawa@redhat.com) said:
Since spot was the person who described it to me, perhaps it would be best to get his input here. The way he stated it was that if there were static libs around at link time, they would get automatically linked, even if the didn't want them to.
A lot of packages will look first for static libraries, then if (and only if) they are not found, look for shared libraries. By splitting into static and static-noshared, we can safely put in -devel and -static-noshared and avoid this confusion.
That's not the case for anything that just passes -l<foo>. Can't we just fix those packages?
+1 to bill's comment. I see static-noshared as nothing but a hack to get around what could/should be fixed in the offending packages.
Unless there is some other purpose to it that I'm missing. Heightened paranoia, err, verification about what is being statically linked?
-- Rex
On Tue, Apr 08, 2008 at 05:51:31PM -0400, Tom spot Callaway wrote:
As promised, here is my new proposed draft for handling static libs:
http://fedoraproject.org/wiki/PackagingDrafts/StaticLibraryPolicy
I know that it won't make everyone happy (it doesn't just leave static bits in -devel), but we really do want to track who is building against static libraries.
Can you remove the perjorative term "deficiency", since OCaml has good reasons for statically linking to do with software safety.
Rich.
On Fri, Apr 11, 2008 at 10:40:05AM +0100, Richard W.M. Jones wrote:
On Tue, Apr 08, 2008 at 05:51:31PM -0400, Tom spot Callaway wrote:
As promised, here is my new proposed draft for handling static libs:
http://fedoraproject.org/wiki/PackagingDrafts/StaticLibraryPolicy
I know that it won't make everyone happy (it doesn't just leave static bits in -devel), but we really do want to track who is building against static libraries.
Can you remove the perjorative term "deficiency", since OCaml has good reasons for statically linking to do with software safety.
Don't worry - I found I could edit that page myself.
Rich.
packaging@lists.fedoraproject.org
-
Bill Nottingham -
Brian Pepple -
Hans de Goede -
Jesse Keating -
Orion Poplawski -
Patrice Dumas -
Ralf Corsepius -
Rex Dieter -
Richard W.M. Jones -
Tom Callaway -
Toshio Kuratomi