1:17 p.m.
Panu Matilainen schreef op vr 26-11-2010 om 13:20 [+0200]:
In particular, I'm interested in feedback on the new, pluggable
and
enhanced dependency extration system. Documentation is scarce at the
moment but some background and examples can be found here:
http://laiskiainen.org/blog/?p=35
All mingw32 packages in Fedora contain these set of instructions in
the .spec files:
%global _use_internal_dependency_generator 0
%global __find_requires %{_mingw32_findrequires}
%global __find_provides %{_mingw32_findprovides}
Does this new dependency extraction system make these kind of
instructions obsolete?
If I understand your blog entry correctly then we (the Fedora MinGW SIG)
are recommended to use something like this:
%__mingw32_provides %{_mingw32_findprovides}
%__mingw32_requires %{_mingw32_findrequires}
Is this correct or do you recommend something different?
The macros %{_mingw32_findrequires} and %{_mingw32_findprovides} are
mentioned in the file /etc/rpm/macros.mingw32 which is part of the
mingw32-filesystem package. Both refer to a small shell script which
uses the i686-pc-mingw32-objdump tool to extract dependency information.
Kind regards,
Erik van Pienbroek
2:10 p.m.
New subject: Looking for testers: RPM 4.9 alpha
On Tue, 30 Nov 2010, Erik van Pienbroek wrote:
Panu Matilainen schreef op vr 26-11-2010 om 13:20 [+0200]:
> In particular, I'm interested in feedback on the new, pluggable and
> enhanced dependency extration system. Documentation is scarce at the
> moment but some background and examples can be found here:
> http://laiskiainen.org/blog/?p=35
All mingw32 packages in Fedora contain these set of instructions in
the .spec files:
%global _use_internal_dependency_generator 0
%global __find_requires %{_mingw32_findrequires}
%global __find_provides %{_mingw32_findprovides}
Does this new dependency extraction system make these kind of
instructions obsolete?
The above still works with the new rpm version.
If I understand your blog entry correctly then we (the Fedora MinGW SIG)
are recommended to use something like this:
%__mingw32_provides %{_mingw32_findprovides}
%__mingw32_requires %{_mingw32_findrequires}
Is this correct or do you recommend something different?
That alone wont do anything at all, to create a new "file attribute"
called mingw32 you'd add a file like this to a suitable package,
mingw32-filesystem probably:
/usr/lib/rpm/fileattrs/mingw32.attr:
%__mingw32_requires /usr/lib/rpm/mingw32-find-requires.sh
%__mingw32_provides /usr/lib/rpm/mingw32-find-provides.sh
%__mingw32_magic ^PE32 executable for MS Windows.* 80386 32-bit$
The magic rule is based on what 'file -b <file>' outputs for mingw32
executables and dll's - the above includes both, but you can make it
tighter to only include DLL's or have different extractors for DLL's and
EXE's by creating two rules instead of just one etc. Or if libmagic
strings aren't good ("fakedll" binaries from Wine would match the above
rule), path based regexes can be used too. It all depends on what makes
sense in a given scenario.
You could also easily have a mingw32-specific pkg-config dependency
extractor which uses a different namespace than the regular pkgconfig(foo)
and only activates on .pc files from the mingw32 sys-root directory.
And with necessary mingw32-specific .attr files in place through
buildrequires, there's no need for override kludges in each and every
mingw32 spec.
- Panu -
2:22 p.m.
New subject: Looking for testers: RPM 4.9 alpha
Panu Matilainen schreef op di 30-11-2010 om 22:10 [+0200]:
On Tue, 30 Nov 2010, Erik van Pienbroek wrote:
> If I understand your blog entry correctly then we (the Fedora MinGW SIG)
> are recommended to use something like this:
>
> %__mingw32_provides %{_mingw32_findprovides}
> %__mingw32_requires %{_mingw32_findrequires}
>
> Is this correct or do you recommend something different?
That alone wont do anything at all, to create a new "file attribute"
called mingw32 you'd add a file like this to a suitable package,
mingw32-filesystem probably:
/usr/lib/rpm/fileattrs/mingw32.attr:
%__mingw32_requires /usr/lib/rpm/mingw32-find-requires.sh
%__mingw32_provides /usr/lib/rpm/mingw32-find-provides.sh
%__mingw32_magic ^PE32 executable for MS Windows.* 80386 32-bit$
The magic rule is based on what 'file -b <file>' outputs for mingw32
executables and dll's - the above includes both, but you can make it
tighter to only include DLL's or have different extractors for DLL's and
EXE's by creating two rules instead of just one etc. Or if libmagic
strings aren't good ("fakedll" binaries from Wine would match the above
rule), path based regexes can be used too. It all depends on what makes
sense in a given scenario.
You could also easily have a mingw32-specific pkg-config dependency
extractor which uses a different namespace than the regular pkgconfig(foo)
and only activates on .pc files from the mingw32 sys-root directory.
And with necessary mingw32-specific .attr files in place through
buildrequires, there's no need for override kludges in each and every
mingw32 spec.
Ah yes, thanks for the detailed information. This sure looks interesting
for us! I'll try to play around a bit with this in a mock environment
and see if it's possible to update our packaging guidelines to make use
of it.
Regards,
Erik van Pienbroek
2:52 p.m.
New subject: Looking for testers: RPM 4.9 alpha
On Tue, 30 Nov 2010, Erik van Pienbroek wrote:
Panu Matilainen schreef op di 30-11-2010 om 22:10 [+0200]:
> On Tue, 30 Nov 2010, Erik van Pienbroek wrote:
>> If I understand your blog entry correctly then we (the Fedora MinGW SIG)
>> are recommended to use something like this:
>>
>> %__mingw32_provides %{_mingw32_findprovides}
>> %__mingw32_requires %{_mingw32_findrequires}
>>
>> Is this correct or do you recommend something different?
>
> That alone wont do anything at all, to create a new "file attribute"
> called mingw32 you'd add a file like this to a suitable package,
> mingw32-filesystem probably:
>
> /usr/lib/rpm/fileattrs/mingw32.attr:
>
> %__mingw32_requires /usr/lib/rpm/mingw32-find-requires.sh
> %__mingw32_provides /usr/lib/rpm/mingw32-find-provides.sh
> %__mingw32_magic ^PE32 executable for MS Windows.* 80386 32-bit$
>
> The magic rule is based on what 'file -b <file>' outputs for mingw32
> executables and dll's - the above includes both, but you can make it
> tighter to only include DLL's or have different extractors for DLL's and
> EXE's by creating two rules instead of just one etc. Or if libmagic
> strings aren't good ("fakedll" binaries from Wine would match the
above
> rule), path based regexes can be used too. It all depends on what makes
> sense in a given scenario.
>
> You could also easily have a mingw32-specific pkg-config dependency
> extractor which uses a different namespace than the regular pkgconfig(foo)
> and only activates on .pc files from the mingw32 sys-root directory.
>
> And with necessary mingw32-specific .attr files in place through
> buildrequires, there's no need for override kludges in each and every
> mingw32 spec.
Ah yes, thanks for the detailed information. This sure looks interesting
for us! I'll try to play around a bit with this in a mock environment
and see if it's possible to update our packaging guidelines to make use
of it.
Do play around with it if you can, but I'd suggest leaving the packaging
guideline considerations until the new rpm has actually landed AND gotten
past the final feature acceptance line. The mechanism is likely to grow
some enhancements before that, and also more than just mingw32 guidelines
should be revisited + revised then (and might well be F16 material)
- Panu -
5:49 a.m.
New subject: MinGW provides and requires automatic generation with RPM 4.9
Hello,
I poked a bit at the new RPM 4.9 dependency extraction system and it
looks really nice. Right now we have the following boilerplate macros at
the top of each and every mingw32- spec file:
%global _use_internal_dependency_generator 0
%global __find_requires %{_mingw32_findrequires}
%global __find_provides %{_mingw32_findprovides}
With RPM 4.9 we can, finally, get rid of them.
The way the new dependency extraction system works is that we would drop
a single mingw32.attr file in /usr/lib/rpm/fileattrs/, which defines a
new "file attribute" definition. There are both file magic and path
constraints, and if both match, our custom provides and requires
extraction scripts are automatically called.
This is the contents of the mingw32.attr file:
%__mingw32_provides %{_rpmconfigdir}/mingw32-find-provides.sh
%__mingw32_requires %{_rpmconfigdir}/mingw32-find-requires.sh
%__mingw32_magic ^PE32 executable.* Intel 80386.*, for MS Windows$
%__mingw32_path ^%{_mingw32_prefix}/.*$
It should also be possible to use the new dependency extraction system
in the mingw32-gcc package where we previously had to resort to manual
mingw32(*) provides. The reason we previously couldn't define
'_use_internal_dependency_generator 0' in that package is that the
mingw32-gcc binary packages mixed both native binaries and cross
compiled binaries; turning off dependency generator and replacing it
with a mingw32 one would have resulted in no dependency generation for
native binaries. But this should all work automagically now.
I am planning to add the new mingw32.attr file to the mingw32-filesystem
package in rawhide soon. Any objections?
It might also make sense to mass rebuild all the mingw32 packages in
rawhide with the boilerplate macros removed. This is also something I
could do; would it be a good idea?
If it all works out nicely, we could also backport the
mingw32-filesystem related changes to F15 (but not rebuild any other
packages). The rationale for doing so is that packagers could then keep
the F15 and rawhide spec files in sync, if they would want to do so. I
would also like to note that supporting the new dependency extraction
system in mingw32-filesystem shouldn't break existing packages that are
still using the boilerplate macros at the top of the spec files.
Comments?
Oh, and thanks go to Panu Matilainen for the new RPM feature.
--
Kalev
6:19 a.m.
New subject: MinGW provides and requires automatic generation with RPM 4.9
On 05/19/2011 01:49 PM, Kalev Lember wrote:
Hello,
I poked a bit at the new RPM 4.9 dependency extraction system and it
looks really nice. Right now we have the following boilerplate macros at
the top of each and every mingw32- spec file:
%global _use_internal_dependency_generator 0
%global __find_requires %{_mingw32_findrequires}
%global __find_provides %{_mingw32_findprovides}
With RPM 4.9 we can, finally, get rid of them.
The way the new dependency extraction system works is that we would drop
a single mingw32.attr file in /usr/lib/rpm/fileattrs/, which defines a
new "file attribute" definition. There are both file magic and path
constraints, and if both match, our custom provides and requires
extraction scripts are automatically called.
This is the contents of the mingw32.attr file:
%__mingw32_provides %{_rpmconfigdir}/mingw32-find-provides.sh
%__mingw32_requires %{_rpmconfigdir}/mingw32-find-requires.sh
%__mingw32_magic ^PE32 executable.* Intel 80386.*, for MS Windows$
%__mingw32_path ^%{_mingw32_prefix}/.*$
Note that this doesn't work quite as you want it to, just yet: in rpm
4.9.0 path and magic are handled as an inclusive or, whereas this (and
various other things) needs an and-rule. Adding support for this is
technically trivial, just the naming is under consideration:
http://lists.rpm.org/pipermail/rpm-maint/2011-May/003022.html
It should also be possible to use the new dependency extraction
system
in the mingw32-gcc package where we previously had to resort to manual
mingw32(*) provides. The reason we previously couldn't define
'_use_internal_dependency_generator 0' in that package is that the
mingw32-gcc binary packages mixed both native binaries and cross
compiled binaries; turning off dependency generator and replacing it
with a mingw32 one would have resulted in no dependency generation for
native binaries. But this should all work automagically now.
I am planning to add the new mingw32.attr file to the mingw32-filesystem
package in rawhide soon. Any objections?
Depends on how soon your "soon" is :) - you'll probably want to wait
until the and-rule support lands in rawhide rpm. OTOH if you're in
hurry, I guess you could get away with just defining mingw32_magic and
leaving the path out for now, Wine is probably the only thing besides
mingw32 having Windows PE32 executables (and you could have an
exclude-rule for the wine-paths just to be on the safe side).
It might also make sense to mass rebuild all the mingw32 packages in
rawhide with the boilerplate macros removed. This is also something I
could do; would it be a good idea?
If it all works out nicely, we could also backport the
mingw32-filesystem related changes to F15 (but not rebuild any other
packages). The rationale for doing so is that packagers could then keep
the F15 and rawhide spec files in sync, if they would want to do so. I
would also like to note that supporting the new dependency extraction
system in mingw32-filesystem shouldn't break existing packages that are
still using the boilerplate macros at the top of the spec files.
Comments?
Oh, and thanks go to Panu Matilainen for the new RPM feature.
Nice to see folks finding use for it :)
- Panu -
12:25 p.m.
New subject: MinGW provides and requires automatic generation with RPM 4.9
On 05/19/2011 02:19 PM, Panu Matilainen wrote:
On 05/19/2011 01:49 PM, Kalev Lember wrote:
> This is the contents of the mingw32.attr file:
> %__mingw32_provides %{_rpmconfigdir}/mingw32-find-provides.sh
> %__mingw32_requires %{_rpmconfigdir}/mingw32-find-requires.sh
> %__mingw32_magic ^PE32 executable.* Intel 80386.*, for MS Windows$
> %__mingw32_path ^%{_mingw32_prefix}/.*$
Note that this doesn't work quite as you want it to, just yet: in rpm
4.9.0 path and magic are handled as an inclusive or, whereas this (and
various other things) needs an and-rule. Adding support for this is
technically trivial, just the naming is under consideration:
http://lists.rpm.org/pipermail/rpm-maint/2011-May/003022.html
Ah, good catch. I would love to see the "and-rule" in RPM, it would
certainly make things easier.
> I am planning to add the new mingw32.attr file to the
mingw32-filesystem
> package in rawhide soon. Any objections?
Depends on how soon your "soon" is :) - you'll probably want to wait
until the and-rule support lands in rawhide rpm. OTOH if you're in
hurry, I guess you could get away with just defining mingw32_magic and
leaving the path out for now, Wine is probably the only thing besides
mingw32 having Windows PE32 executables (and you could have an
exclude-rule for the wine-paths just to be on the safe side).
Mono assemblies also have a very similar magic, which is somewhat hard
to discern from the usual PE32 magic.
mingw32 cross compiled zlib1.dll:
PE32 executable (DLL) (console) Intel 80386 (stripped to external PDB),
for MS Windows
Mono assembly nunit.core.dll:
PE32 executable (DLL) (console) Intel 80386 Mono/.Net assembly, for MS
Windows
If we currently don't have the "and-rule", I would go for a simple path
based regex that matches all the .dll and .exe files in the mingw32
sysroot and forget about the libmagic matches for now. Once the
"and-rule" makes it in rawhide, we can switch over to using that.
The advantages with plain path based regexes are:
- It's similar to how we are matching the files right now, so it would
be harder to regress something;
- It would work with RPM 4.9.0 and we would be able to backport the
mingw32-filesystem changes to F15 to provide compatibility for
maintainers who want to keep F15 and rawhide spec files in sync;
- We wouldn't have to worry about excluding strange paths where some
.dll files might be, like /usr/share/.
Depends on how soon your "soon" is :)
I would actually like to get the fileattr dependency extraction in
rather sooner than later. There's no guarantee I would have time to work
on it in a month or two and I'd rather finish it now when I have the time.
--
Kalev
12:52 p.m.
New subject: MinGW provides and requires automatic generation with RPM 4.9
On 05/19/2011 08:25 PM, Kalev Lember wrote:
On 05/19/2011 02:19 PM, Panu Matilainen wrote:
> On 05/19/2011 01:49 PM, Kalev Lember wrote:
>> This is the contents of the mingw32.attr file:
>> %__mingw32_provides %{_rpmconfigdir}/mingw32-find-provides.sh
>> %__mingw32_requires %{_rpmconfigdir}/mingw32-find-requires.sh
>> %__mingw32_magic ^PE32 executable.* Intel 80386.*, for MS Windows$
>> %__mingw32_path ^%{_mingw32_prefix}/.*$
>
> Note that this doesn't work quite as you want it to, just yet: in rpm
> 4.9.0 path and magic are handled as an inclusive or, whereas this (and
> various other things) needs an and-rule. Adding support for this is
> technically trivial, just the naming is under consideration:
> http://lists.rpm.org/pipermail/rpm-maint/2011-May/003022.html
Ah, good catch. I would love to see the "and-rule" in RPM, it would
certainly make things easier.
>> I am planning to add the new mingw32.attr file to the mingw32-filesystem
>> package in rawhide soon. Any objections?
>
> Depends on how soon your "soon" is :) - you'll probably want to wait
> until the and-rule support lands in rawhide rpm. OTOH if you're in
> hurry, I guess you could get away with just defining mingw32_magic and
> leaving the path out for now, Wine is probably the only thing besides
> mingw32 having Windows PE32 executables (and you could have an
> exclude-rule for the wine-paths just to be on the safe side).
Mono assemblies also have a very similar magic, which is somewhat hard
to discern from the usual PE32 magic.
mingw32 cross compiled zlib1.dll:
PE32 executable (DLL) (console) Intel 80386 (stripped to external PDB),
for MS Windows
Mono assembly nunit.core.dll:
PE32 executable (DLL) (console) Intel 80386 Mono/.Net assembly, for MS
Windows
Ah, mono too. Those could be filtered out with an exclude_magic rule on
Mono/.Net but...
If we currently don't have the "and-rule", I would go for a simple path
based regex that matches all the .dll and .exe files in the mingw32
sysroot and forget about the libmagic matches for now. Once the
"and-rule" makes it in rawhide, we can switch over to using that.
The advantages with plain path based regexes are:
- It's similar to how we are matching the files right now, so it would
be harder to regress something;
- It would work with RPM 4.9.0 and we would be able to backport the
mingw32-filesystem changes to F15 to provide compatibility for
maintainers who want to keep F15 and rawhide spec files in sync;
- We wouldn't have to worry about excluding strange paths where some
.dll files might be, like /usr/share/.
Indeed, for something like mingw32 where everything lives in a distinct
directory structure with easily recognizable suffixes, additional
magic-rules adds little value over plain path-based regexes.
> Depends on how soon your "soon" is :)
I would actually like to get the fileattr dependency extraction in
rather sooner than later. There's no guarantee I would have time to work
on it in a month or two and I'd rather finish it now when I have the time.
Ok. Well, if only path based regexes are used then that will work right
now in both rawhide and F15.
- Panu -
2:17 a.m.
New subject: MinGW provides and requires automatic generation with RPM 4.9
On 05/19/2011 08:52 PM, Panu Matilainen wrote:
Indeed, for something like mingw32 where everything lives in a
distinct
directory structure with easily recognizable suffixes, additional
magic-rules adds little value over plain path-based regexes.
Thanks Panu.
After discussing it with Erik van Pienbroek and Richard W.M. Jones on
IRC, I've built mingw32-filesystem with the new dep extraction support
in rawhide.
This is how the mingw32.attr file now looks:
%__mingw32_provides %{_rpmconfigdir}/mingw32-find-provides.sh
%__mingw32_requires %{_rpmconfigdir}/mingw32-find-requires.sh
%__mingw32_path ^%{_mingw32_prefix}/.*\.([Dd][Ll][Ll]|[Ee][Xx][Ee])$
--
Kalev
12:26 a.m.
New subject: MinGW provides and requires automatic generation with RPM 4.9
On 05/19/2011 08:25 PM, Kalev Lember wrote:
On 05/19/2011 02:19 PM, Panu Matilainen wrote:
> On 05/19/2011 01:49 PM, Kalev Lember wrote:
>> This is the contents of the mingw32.attr file:
>> %__mingw32_provides %{_rpmconfigdir}/mingw32-find-provides.sh
>> %__mingw32_requires %{_rpmconfigdir}/mingw32-find-requires.sh
>> %__mingw32_magic ^PE32 executable.* Intel 80386.*, for MS Windows$
>> %__mingw32_path ^%{_mingw32_prefix}/.*$
>
> Note that this doesn't work quite as you want it to, just yet: in rpm
> 4.9.0 path and magic are handled as an inclusive or, whereas this (and
> various other things) needs an and-rule. Adding support for this is
> technically trivial, just the naming is under consideration:
> http://lists.rpm.org/pipermail/rpm-maint/2011-May/003022.html
Ah, good catch. I would love to see the "and-rule" in RPM, it would
certainly make things easier.
Just FWIW, rpm >= 4.9.1.1 (currently in rawhide, updates submitted for
F15 and F16) supports the magic_and_path flag. Whether you need it or
not is a another question and up to you, but just thought to let you
know now that I happened to remember this discussion :)
- Panu -
7:11 a.m.
New subject: MinGW provides and requires automatic generation with RPM 4.9
Kalev Lember schreef op do 19-05-2011 om 13:49 [+0300]:
I am planning to add the new mingw32.attr file to the
mingw32-filesystem
package in rawhide soon. Any objections?
Hi Kalev,
Thanks for looking into this feature.
While this feature sure looks interesting for the future I think it's
too soon to switch to this yet. As Panu indicated there are some things
which still need to be implemented (the and-rule) so this makes it
tricky to backport to old Fedora releases. Another issue is that we will
lose EL6 support if we're going to implement this in all mingw packages.
EL6 support is expected soon (as far as I've heard about it, various
mingw packages have popped up recently in the RHEL6 bugzilla) so it
wouldn't be really nice to break EL6 support so soon.
I think it would be more sane to wait with implementing this future
until it has matured in RPM and is available in all current Fedora
releases.
Kind regards,
Erik van Pienbroek
12:28 p.m.
New subject: MinGW provides and requires automatic generation with RPM 4.9
On 05/19/2011 03:11 PM, Erik van Pienbroek wrote:
Hi Kalev,
Thanks for looking into this feature.
While this feature sure looks interesting for the future I think it's
too soon to switch to this yet. As Panu indicated there are some things
which still need to be implemented (the and-rule) so this makes it
tricky to backport to old Fedora releases.
The missing "and-rule" would certainly be nice to have, but it is not a
showstopper. A lot of different packages are working just fine without
it in F15 and it's easy to make the mingw32 fileattr extractor also work
without needing the "and-rule". As to exactly how, I have details in my
reply to Panu.
Most (if not all) "internal" dependency generators were converted over
to the new fileattr based generators in RPM 4.9. Also several other high
profile packages like cups, maven, and gstreamer have switched over in
F15, which makes the mingw32 packages an exception, rather than a rule.
We are the ones lagging behind.
See for yourself, the following are all new fileattr generator types in
use in F15 (yum provides '/usr/lib/rpm/fileattrs/*.attr'):
desktop.attr
elf.attr
font.attr
gstreamer.attr
libsymlink.attr
libtool.attr
maven.attr
mono.attr
ocaml.attr
perl.attr
perllib.attr
pkgconfig.attr
psdriver.attr
python.attr
script.attr
Another issue is that we will
lose EL6 support if we're going to implement this in all mingw packages.
EL6 support is expected soon (as far as I've heard about it, various
mingw packages have popped up recently in the RHEL6 bugzilla) so it
wouldn't be really nice to break EL6 support so soon.
Right, RHEL 6.1 was released today with a bunch of mingw32 packages.
They are all, however, from the F13 era and if you want to build
something mingw32 related in EPEL, it would also make sense to branch
these packages from F13. For example, the mingw32-gtk2 package we have
in rawhide / F15 would not work in EPEL 6 anyway, because the
mingw32-glib2 in RHEL is too old.
I am talking about changing the dependency generation in Fedora Rawhide
(and also later backporting it to F15 to keep full srpm compatibility);
the RHEL packages are free to keep carrying the boilerplate macros,
nobody is breaking these. We are also retaining full compatibility with
all old spec files.
I think it would be more sane to wait with implementing this future
until it has matured in RPM and is available in all current Fedora
releases.
Sorry, but I disagree. Fedora Rawhide is exactly the place to innovate
and not wait for RHEL to catch up.
Also, keep in mind that the new dependency generation is fully
compatible with all the old spec files. Older spec files that manually
disable the "internal" dependency generator in the spec file are still
going to work fine.
--
Kalev
4632
days inactive
4894
days old
11 comments
3 participants
participants (3)
-
Erik van Pienbroek -
Kalev Lember -
Panu Matilainen