8:43 p.m.
I recently helped another user repair their Fedora workation, after an
update broke gnome-shell. In their case, I believe that the problem
occurred because they had the nodejs:14 module enabled, which contained
an outdated libnghttp2 [1], but in principle, the problem can affect any
system that selectively applies updates, or any system on which packages
are installed without also applying updates [2].
I expected that if I checked, I could find some mention in the Fedora
documentation to the effect that the only supportable configuration for
a major-version stable distribution is fully patched. The only thing I
could actually find, though, was an article about applying only security
or bugfix updates [3], which is basically the opposite.
(As an aside, I *also* think that "dnf update --security" is broken on
Fedora because not only can it result in broken dependencies, but it can
be misleading. In the case that a security update is published and
subsequently obsoleted by another update, systems with an affected,
older package would not be informed that a security update was required.
The second, subsequent update does include security fixes for older
versions, but that information is lost. Maybe Fedora should retain any
package which was marked as a security fix?)
So, first, I will open a PR with some changes to Fedora's DNF
documentation [4] encouraging users to apply all updates before
installing packages. I may follow up on the Fedora Magazine article, as
well. It doesn't feel right to propose an article on Fedora Magazine
advising users not to use "dnf update --security", but it also doesn't
feel right to publish articles describing that feature without also
describing any caveats.
Second, I'd like to suggest that in the future, at least in Fedora, for
any "install" or "update" operation that dnf performs, dnf's
default
behavior should be checking all of the direct and indirect dependencies
of the packages being installed (or updated) and updating any
dependencies which have updates available.
Does anyone else have any opinions on the subject? Should I simply file
a bug against dnf proposing this behavior?
1: https://bugzilla.redhat.com/show_bug.cgi?id=2164944
2: For example:
$ podman run --rm -it fedora:37
[root@30a0f8c3f6a8 /]# rpm -q libnghttp2
libnghttp2-1.49.0-1.fc37.x86_64
[root@30a0f8c3f6a8 /]# dnf install chezdav
...
[root@30a0f8c3f6a8 /]# chezdav
chezdav: symbol lookup error: /lib64/libsoup-3.0.so.0: undefined symbol:
nghttp2_option_set_no_rfc9113_leading_and_trailing_ws_validation
[root@30a0f8c3f6a8 /]# rpm -q libsoup3 libnghttp2
libsoup3-3.2.2-2.fc37.x86_64
libnghttp2-1.49.0-1.fc37.x86_64
3:
https://fedoramagazine.org/how-to-install-only-security-and-bugfixes-upda...
4:
https://docs.fedoraproject.org/en-US/fedora/latest/system-administrators-...
2:14 a.m.
New subject: Proposal: dnf should offer to update all of the dependencies of
any package installed or updated
On Fri, Jan 27, 2023 at 18:43:56 -0800,
Gordon Messmer <gordon.messmer(a)gmail.com> wrote:
Second, I'd like to suggest that in the future, at least in Fedora,
for any "install" or "update" operation that dnf performs, dnf's
default behavior should be checking all of the direct and indirect
dependencies of the packages being installed (or updated) and updating
any dependencies which have updates available.
Does anyone else have any opinions on the subject? Should I simply
file a bug against dnf proposing this behavior?
If there is a problem with not uodating dependencies when you do an install
or an update on selected packages, the packages dependencies are not
properly defined.
I think the case where you don't want to keep the full system up to date,
but a selective update or install causes problems as well is pretty rare.
I think it might be reasonable to have an option that requests doing
a recursive update. I would consider this to be a low priority feature
request that has to compete with all of the other work being done on
dnf, rather than a bug. I don't work on dnf and the people that do might
feel differently.
11:24 a.m.
New subject: Proposal: dnf should offer to update all of the dependencies of
any package installed or updated
libsoup3 depends on libnghttp2.so.14
Apparantly, either libsoup3 should depend on the minor version (in addition to the major
version), or libnghhttp2 should have bumped major, depending on the "history" of
that symbol. Probably the former (API addition in a minor bump to the same major).
12:22 p.m.
New subject: Proposal: dnf should offer to update all of the dependencies of
any package installed or updated
On 2023-01-28 00:14, Bruno Wolff III wrote:
If there is a problem with not uodating dependencies when you do an
install or an update on selected packages, the packages dependencies
are not properly defined.
By definition, yes. But rpm auto-detects dependencies, and rpm doesn't
do symbol-level dependency detection, so it doesn't capture
minor-version dependency creep. In order for rpm to do this, you'd
probably have to throw out the current implementation of dependency
resolution that provides "libfoo.so.2()(64bit)" and instead provide a
dependency like "(foo-libs >= 2.4 with foo-libs < 3)", at least for the
cases where libraries do not provide versioned symbols -- which I
believe includes the vast majority of them. (You'd also need to forbid
restructuring packages within a stable release.)
I think the case where you don't want to keep the full system up
to
date, but a selective update or install causes problems as well is
pretty rare. I think it might be reasonable to have an option that
requests doing a recursive update. I would consider this to be a low
priority feature request that has to compete with all of the other
work being done on dnf, rather than a bug. I don't work on dnf and the
people that do might feel differently.
Yeah, I agree, it's not super high priority. But it's also not really
well understood among the community that partial updates (such as `dnf
update --security`) and package installation without a contemporaneous
update are unreliable.
I can work on some of those changes to documentation and to rpm or dnf,
but I'd like input from the developer community before starting. (And
at some point I think that Fedora, the org, should probably consider
whether `dnf update --security` is broken and unreliable.)
2:12 p.m.
New subject: Proposal: dnf should offer to update all of the dependencies of
any package installed or updated
On 2023-01-28 10:22, Gordon Messmer wrote:
In order for rpm to do this, you'd probably have to throw out the
current implementation of dependency resolution that provides
"libfoo.so.2()(64bit)" and instead provide a dependency like "(foo-libs
>= 2.4 with foo-libs < 3)", at least for the cases where libraries do
not provide versioned symbols
...or "(libfoo.so.2()(64bit) with foo-libs >= x.y.z)", where x.y.z is
the version of the package that provides libfoo.so.2 in the build root,
which is an idea that's growing on me.
3:03 p.m.
New subject: Proposal: dnf should offer to update all of the dependencies of
any package installed or updated
On Sat, Jan 28, 2023 at 12:12:40PM -0800, Gordon Messmer wrote:
On 2023-01-28 10:22, Gordon Messmer wrote:
> In order for rpm to do this, you'd probably have to throw out the
> current implementation of dependency resolution that provides
> "libfoo.so.2()(64bit)" and instead provide a dependency like
"(foo-libs
> >= 2.4 with foo-libs < 3)", at least for the cases where libraries do
> not provide versioned symbols
...or "(libfoo.so.2()(64bit) with foo-libs >= x.y.z)", where x.y.z is the
version of the package that provides libfoo.so.2 in the build root, which is
an idea that's growing on me.
This is indeed a shortcoming in the rpm symbol dependency generation scheme.
But there's a problem with the proposed approach: versioning as major.minor.micro
is just a convention, and not all upstream follow it.
There is an alternative scheme that is supported by our rpm tooling already:
symbol versioning. This is ultimately more powerful, because it allows depending
on specific versions of functions, and not just one monotonic version that
tries to describe anything. (glibc makes extensive use of this to provide
backwards comaptiblity…)
Since symbol versioning is more powerful and pretty well handled by rpm,
I think it's better to try to convince upstream to switch to that if they
change symbols a lot.
(Or even better, in my personal opinion, is to never change SOVERSION, i.e.
to never remove symbols. It is surprisingly easy to provide backwards compat,
for most project much less hassle than to deal with SOVERSION changes and
forced rebuilds.)
Zbyszek
3:49 p.m.
New subject: Proposal: dnf should offer to update all of the dependencies of
any package installed or updated
On Sat, Jan 28, 2023 at 09:03:35PM +0000, Zbigniew Jędrzejewski-Szmek wrote:
This is indeed a shortcoming in the rpm symbol dependency generation scheme.
Is it though? I'm probibly reading this too quickly and missing
something, but isn't the underlying problem here that nghttp2 changed
abi and didn't change soname? If they had, soup would have kept the
older version around, or asked to upgrade both libsoup and libnghttp2?
Or did I miss some part of this?
...snip symbol versioning...
symbol versioning does indeed sound like a better way to do things. :)
kevin
5:02 p.m.
New subject: Proposal: dnf should offer to update all of the dependencies of
any package installed or updated
On Sat, Jan 28, 2023 at 01:49:17PM -0800, Kevin Fenzi wrote:
On Sat, Jan 28, 2023 at 09:03:35PM +0000, Zbigniew Jędrzejewski-Szmek
wrote:
>
> This is indeed a shortcoming in the rpm symbol dependency generation scheme.
Is it though? I'm probibly reading this too quickly and missing
something, but isn't the underlying problem here that nghttp2 changed
abi and didn't change soname? If they had, soup would have kept the
older version around, or asked to upgrade both libsoup and libnghttp2?
No. libnghttp2 added one symbol, and (correctly, i.e. following the usual
practice), it changed the suffix from .so.14.23.0 to .so.14.24.1. (The last
number is bumped when the implementation of existing symbols changes, the middle
one when when new symbols are added, and the first one when an incompatible
change happens, i.e. symbols are removed or changed incompatibly. Lower numbers
are reset to zero whenever a higher one changes.) The SOVERSION corresponds to
the major number and does not change in the case under discussion.
As Gordon wrote, our rpm dependency generation logic does not care about the
minor or patch numbers.
Zbyszek
5:23 p.m.
New subject: Proposal: dnf should offer to update all of the dependencies of
any package installed or updated
On Sun, Jan 29, 2023 at 12:03 AM Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl> wrote:
On Sat, Jan 28, 2023 at 01:49:17PM -0800, Kevin Fenzi wrote:
> On Sat, Jan 28, 2023 at 09:03:35PM +0000, Zbigniew Jędrzejewski-Szmek wrote:
> >
> > This is indeed a shortcoming in the rpm symbol dependency generation scheme.
>
> Is it though? I'm probibly reading this too quickly and missing
> something, but isn't the underlying problem here that nghttp2 changed
> abi and didn't change soname? If they had, soup would have kept the
> older version around, or asked to upgrade both libsoup and libnghttp2?
No. libnghttp2 added one symbol, and (correctly, i.e. following the usual
practice), it changed the suffix from .so.14.23.0 to .so.14.24.1. (The last
number is bumped when the implementation of existing symbols changes, the middle
one when when new symbols are added, and the first one when an incompatible
change happens, i.e. symbols are removed or changed incompatibly. Lower numbers
are reset to zero whenever a higher one changes.) The SOVERSION corresponds to
the major number and does not change in the case under discussion.
As Gordon wrote, our rpm dependency generation logic does not care about the
minor or patch numbers.
If I understand things correctly, this is not entirely true. RPM
generates a dependency for the soname / soversion, and some projects
include not only X, but all of X.Y.Z in that, which RPM will happily
generate Provides / Requires for (whether that's a nice thing to do
for an upstream project or whether that's a bug in their build systems
I leave as an exercise to the reader).
Fabio
8:19 p.m.
New subject: Proposal: dnf should offer to update all of the dependencies of
any package installed or updated
On 2023-01-28 15:23, Fabio Valentini wrote:
If I understand things correctly, this is not entirely true. RPM
generates a dependency for the soname / soversion, and some projects
include not only X, but all of X.Y.Z in that, which RPM will happily
generate Provides / Requires for
$ ls -l /lib64/libnghttp2.so.14*
lrwxrwxrwx. 1 root root 21 Nov 21 05:45 /lib64/libnghttp2.so.14 ->
libnghttp2.so.14.24.1*
-rwxr-xr-x. 1 root root 173696 Nov 21 05:45 /lib64/libnghttp2.so.14.24.1*
$ ldd /lib64/libsoup-3.0.so.0 | grep libnghttp
libnghttp2.so.14 => /lib64/libnghttp2.so.14 (0x00007f08fb366000)
$ rpm -q libsoup3 --requires | grep libnghttp2
libnghttp2.so.14()(64bit)
1:47 a.m.
New subject: Proposal: dnf should offer to update all of the dependencies of
any package installed or updated
On Sun, Jan 29, 2023 at 12:23:49AM +0100, Fabio Valentini wrote:
On Sun, Jan 29, 2023 at 12:03 AM Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl> wrote:
>
> On Sat, Jan 28, 2023 at 01:49:17PM -0800, Kevin Fenzi wrote:
> > On Sat, Jan 28, 2023 at 09:03:35PM +0000, Zbigniew Jędrzejewski-Szmek wrote:
> > >
> > > This is indeed a shortcoming in the rpm symbol dependency generation
scheme.
> >
> > Is it though? I'm probibly reading this too quickly and missing
> > something, but isn't the underlying problem here that nghttp2 changed
> > abi and didn't change soname? If they had, soup would have kept the
> > older version around, or asked to upgrade both libsoup and libnghttp2?
>
> No. libnghttp2 added one symbol, and (correctly, i.e. following the usual
> practice), it changed the suffix from .so.14.23.0 to .so.14.24.1. (The last
> number is bumped when the implementation of existing symbols changes, the middle
> one when when new symbols are added, and the first one when an incompatible
> change happens, i.e. symbols are removed or changed incompatibly. Lower numbers
> are reset to zero whenever a higher one changes.) The SOVERSION corresponds to
> the major number and does not change in the case under discussion.
>
> As Gordon wrote, our rpm dependency generation logic does not care about the
> minor or patch numbers.
If I understand things correctly, this is not entirely true. RPM
generates a dependency for the soname / soversion, and some projects
include not only X, but all of X.Y.Z in that, which RPM will happily
generate Provides / Requires for (whether that's a nice thing to do
for an upstream project or whether that's a bug in their build systems
I leave as an exercise to the reader).
That is all true. But it's probably not a good idea to for the upstream
projects to do that, because the SONAME is used to load the library at runtime. (*)
If the SONAME changes, all programs and libraries linking to that library
need to be recompiled/relinked. This would essentially defeat the shared library
system.
(*) Or more precisely speaking, when a library is installed in the file system,
a symlink is created to the real name (e.g. libcurl.so.4.8.0) from the embedded
SONAME. Subsequently, a program that wants a given SONAME is run, the dynamic
linker looks for the SONAME in the file system and finds the symlink
(libcurl.so.4 in this case) and uses that. IIUC, the SONAME isn't actually
checked again. So SONAME is _not_ used at runtime directly, but it's still used
to set things up so that the library is found.
Earlier, when a program is being compiled, a similar indirection happens. The
compiler/linker is given a library name (e.g. -lcurl) and uses that to locate
the library as %{_libdir}/libcurl.so, reads the SONAME (libcurl.so.4) from that
file and saves the SONAME in the linked program.
Zbyszek
8:12 p.m.
New subject: Proposal: dnf should offer to update all of the dependencies of
any package installed or updated
On 2023-01-28 13:03, Zbigniew Jędrzejewski-Szmek wrote:
> ...or "(libfoo.so.2()(64bit) with foo-libs >=
x.y.z)", where x.y.z is the
> version of the package that provides libfoo.so.2 in the build root, which is
> an idea that's growing on me.
This is indeed a shortcoming in the rpm symbol dependency generation scheme.
But there's a problem with the proposed approach: versioning as major.minor.micro
is just a convention, and not all upstream follow it.
The version doesn't have to be major.minor.micro, though. The
dependency generator only needs to get the version of the package that
provides the .so, and use that version. As long as changes within a so
version are always backward compatible, and the so version is bumped
when breaking changes are made (which are already constraints on the
existing system), then the proposed solution is reliable. It's
sometimes too strict -- it might require 1.4.3 when 1.4.0 is compatible
-- but it's reliable.
There is an alternative scheme that is supported by our rpm tooling
already:
symbol versioning.
Yes, the enhanced rpm dependencies would only be necessary for libraries
that don't provide versioned symbols (which seems to be the vast
majority of them).
I don't think convincing hundreds or thousands of developers to add
symbol versioning to their libraries is a viable solution. I'd love to
see it happen, but rpm/dnf should be more reliable in the meantime.
2 a.m.
New subject: Proposal: dnf should offer to update all of the dependencies of
any package installed or updated
On Sat, Jan 28, 2023 at 06:12:11PM -0800, Gordon Messmer wrote:
On 2023-01-28 13:03, Zbigniew Jędrzejewski-Szmek wrote:
> > ...or "(libfoo.so.2()(64bit) with foo-libs >= x.y.z)", where x.y.z
is the
> > version of the package that provides libfoo.so.2 in the build root, which is
> > an idea that's growing on me.
> This is indeed a shortcoming in the rpm symbol dependency generation scheme.
> But there's a problem with the proposed approach: versioning as
major.minor.micro
> is just a convention, and not all upstream follow it.
The version doesn't have to be major.minor.micro, though. The dependency
generator only needs to get the version of the package that provides the
.so, and use that version. As long as changes within a so version are
always backward compatible, and the so version is bumped when breaking
changes are made (which are already constraints on the existing system),
then the proposed solution is reliable. It's sometimes too strict -- it
might require 1.4.3 when 1.4.0 is compatible -- but it's reliable.
OK, it sounds like something worth doing.
(One issue that I expect that there'll be some libraries which do "strange
things" and may need different handling, so some opt-out mechanism would need
to be provided.)
> There is an alternative scheme that is supported by our rpm
tooling already:
> symbol versioning.
Yes, the enhanced rpm dependencies would only be necessary for libraries
that don't provide versioned symbols (which seems to be the vast majority of
them).
I don't think convincing hundreds or thousands of developers to add symbol
versioning to their libraries is a viable solution. I'd love to see it
happen, but rpm/dnf should be more reliable in the meantime.
Ack.
Zbyszek
2:57 a.m.
New subject: Proposal: dnf should offer to update all of the dependencies of
any package installed or updated
On 2023-01-29 00:00, Zbigniew Jędrzejewski-Szmek wrote:
> The version doesn't have to be major.minor.micro, though.
The dependency
> generator only needs to get the version of the package that provides the
> .so, and use that version. As long as changes within a so version are
> always backward compatible, and the so version is bumped when breaking
> changes are made (which are already constraints on the existing system),
> then the proposed solution is reliable. It's sometimes too strict -- it
> might require 1.4.3 when 1.4.0 is compatible -- but it's reliable.
OK, it sounds like something worth doing.
OK. For discussion, I opened
https://github.com/rpm-software-management/rpm/pull/2372
I tested it by rebuilding libsoup3 in a mock build root that included a
version of rpm with (basically) that patch, and then installing the
resulting package. It looks like it functions correctly, so this seems
like a sensible place to start.
$ podman run --rm -it fedora:37
[root@ae5c590fa03a /]# rpm -qp --requires
/var/tmp/libsoup3-3.2.2-3.fc37.x86_64.rpm
(libnghttp2.so.14()(64bit) with libnghttp2 >= 1.51.0)
...
[root@ae5c590fa03a /]# dnf install
/var/tmp/libsoup3-3.2.2-3.fc37.x86_64.rpm
Dependencies resolved.
===================================================================================================================================================================================================================
Package Architecture Version
Repository Size
===================================================================================================================================================================================================================
Installing:
libsoup3 x86_64 3.2.2-3.fc37
@commandline 373 k
Upgrading:
libnghttp2 x86_64 1.51.0-1.fc37
updates 74 k
...
12:53 a.m.
New subject: Proposal: dnf should offer to update all of the dependencies of
any package installed or updated
* Gordon Messmer:
On 2023-01-28 13:03, Zbigniew Jędrzejewski-Szmek wrote:
>> ...or "(libfoo.so.2()(64bit) with foo-libs >= x.y.z)", where x.y.z
is the
>> version of the package that provides libfoo.so.2 in the build root, which is
>> an idea that's growing on me.
> This is indeed a shortcoming in the rpm symbol dependency generation scheme.
> But there's a problem with the proposed approach: versioning as
major.minor.micro
> is just a convention, and not all upstream follow it.
The version doesn't have to be major.minor.micro, though. The
dependency generator only needs to get the version of the package that
provides the .so, and use that version. As long as changes within a
so version are always backward compatible, and the so version is
bumped when breaking changes are made (which are already constraints
on the existing system), then the proposed solution is reliable. It's
sometimes too strict -- it might require 1.4.3 when 1.4.0 is
compatible -- but it's reliable.
We already do this for glibc in rawhide, for the symbol version under
development.
# Auto-generating dependencies for glibc development snapshots.
#
# A glibc development snapshot (say version 2.33.9000) may define
# symbols in its under-development symbol version (GLIBC_2.34). RPM
# automatically derives RPM dependencies such as
# libc.so.6(GLIBC_2.34)(64bit) from that. While the GLIBC_2.34
# version is under development, these dependencies may be inaccurate
# and could be satisfied by glibc RPM package versions that lack the
# symbols because they were created from an earlier development
# snapshot that had some other GLIBC_2.34 symbols. Therefore, if the
# latest, under-development ELF symbol version is detected, this
# dependency generator adds an explicit RPM dependencies on the glibc
# packaging version against which an RPM package is built.
<https://src.fedoraproject.org/rpms/glibc/blob/rawhide/f/glibc.req.in>
This is an example build:
util-linux-2.38.1-1.fc37
<https://koji.fedoraproject.org/koji/buildinfo?buildID=2042005>
(util-linux-core in particular.)
Right now, this isn't used much, but it when it is, it causes issues for
developers who use the rawhide compose in chroots and newer builds leak
into the chroot from the buildroot without a full chroot update.
Manually ELN tagging can cause similar issues, so we no longer embed the
dist tag in the auto-generated versions.
Thanks,
Florian
7:09 a.m.
New subject: Proposal: dnf should offer to update all of the dependencies of
any package installed or updated
V Sat, Jan 28, 2023 at 06:12:11PM -0800, Gordon Messmer napsal(a):
I don't think convincing hundreds or thousands of developers to add symbol
versioning to their libraries is a viable solution. I'd love to see it
happen, but rpm/dnf should be more reliable in the meantime.
Is symbol versioning portable across toolchains? I.e. is the linker script
with symbol aliases understandible by other tools than (llvm|gcc)+binutils?
I speak about upstreams who do not want to impede building e.g. on FreeBSD,
Windows, or Solaris.
-- Petr
8:17 a.m.
New subject: Proposal: dnf should offer to update all of the dependencies of
any package installed or updated
* Petr Pisar:
V Sat, Jan 28, 2023 at 06:12:11PM -0800, Gordon Messmer napsal(a):
>
> I don't think convincing hundreds or thousands of developers to add symbol
> versioning to their libraries is a viable solution. I'd love to see it
> happen, but rpm/dnf should be more reliable in the meantime.
Is symbol versioning portable across toolchains? I.e. is the linker
script with symbol aliases understandible by other tools than
(llvm|gcc)+binutils? I speak about upstreams who do not want to
impede building e.g. on FreeBSD, Windows, or Solaris.
The GNU variant of symbol versioning is GNU-specific. Solaris symbol
versioning is quite different, but maybe sufficient in this context for
dependency generation. Windows doesn't have anything like it, as far as
I know.
Thanks,
Florian
8:19 a.m.
New subject: Proposal: dnf should offer to update all of the dependencies of
any package installed or updated
On Mon, Jan 30, 2023 at 02:09:18PM +0100, Petr Pisar wrote:
V Sat, Jan 28, 2023 at 06:12:11PM -0800, Gordon Messmer napsal(a):
>
> I don't think convincing hundreds or thousands of developers to add symbol
> versioning to their libraries is a viable solution. I'd love to see it
> happen, but rpm/dnf should be more reliable in the meantime.
>
Is symbol versioning portable across toolchains? I.e. is the linker script
with symbol aliases understandible by other tools than (llvm|gcc)+binutils?
I speak about upstreams who do not want to impede building e.g. on FreeBSD,
Windows, or Solaris.
It only applies to ELF platforms. It also works fine with Clang.
FreeBSD uses ELF but their linker has issues with wildcards in the
linker script so we had to disable it there:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=190851
macOS doesn't use ELF and doesn't support linker scripts, so obviously
disabled there too.
Windows (PE) linking has entirely its own set of complexities and
strangeness, but I don't think there's anything quite like symbol
versioning.
Symbol versioning makes it tricky to backport an upstream patch to
downstream (eg. to RHEL) if the patch adds a new API. You have to
decide whether or not to add a new version to the linker script or put
the symbol into one of the old sections, and I seem to recollect there
is no always good answer.
Nevertheless I think symbol versioning is a worthwhile thing to do.
Some examples of projects using linker scripts and/or symbol
versioning:
https://gitlab.com/nbdkit/nbdkit
- linker script only, to hide symbols
https://gitlab.com/nbdkit/libnbd
- linker script + versioning, everything is generated from a
machine-readable description of the API
https://gitlab.com/libvirt/libvirt
- extensive use of linker scripts + symbol versioning, see the src/*.syms
files
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
4:56 p.m.
New subject: Proposal: dnf should offer to update all of the dependencies of
any package installed or updated
On Sat, Jan 28, 2023 at 1:23 PM Gordon Messmer <gordon.messmer(a)gmail.com> wrote:
On 2023-01-28 00:14, Bruno Wolff III wrote:
> If there is a problem with not uodating dependencies when you do an
> install or an update on selected packages, the packages dependencies
> are not properly defined.
By definition, yes. But rpm auto-detects dependencies, and rpm doesn't
do symbol-level dependency detection, so it doesn't capture
minor-version dependency creep. In order for rpm to do this, you'd
probably have to throw out the current implementation of dependency
resolution that provides "libfoo.so.2()(64bit)" and instead provide a
dependency like "(foo-libs >= 2.4 with foo-libs < 3)", at least for the
cases where libraries do not provide versioned symbols -- which I
believe includes the vast majority of them. (You'd also need to forbid
restructuring packages within a stable release.)
The RPM build process tries to do so, but it can't possibly outsmart
the morass of dependencies that "modularity" tends to create. It's why
so many have given up on modularity since its introduction in Fedora,
and one of the reasons I have trouble getting approval to do RHEL 8
and RHEL 9 updates. The various sets of "modularity" components are
not tested against all the other versions of related packages: I'm
running into this headlong with ansible collections and the morass of
dependency chains weaving into and out of modularity deployed
coponents, myself. The "modular" channels are almost invariably
incomplete and require additional RPM builds to satisfy "test" or
"doc" building steps from the same base software package.
Modularity and its inevitable build-time and deployment
inconsistencies have consistently hindered dependency management.
> I think the case where you don't want to keep the full
system up to
> date, but a selective update or install causes problems as well is
> pretty rare. I think it might be reasonable to have an option that
> requests doing a recursive update. I would consider this to be a low
> priority feature request that has to compete with all of the other
> work being done on dnf, rather than a bug. I don't work on dnf and the
> people that do might feel differently.
Yeah, I agree, it's not super high priority. But it's also not really
well understood among the community that partial updates (such as `dnf
update --security`) and package installation without a contemporaneous
update are unreliable.
Yeah. It's hard to predict exactly when it will break, but it *does*
break. Unfortunately, so do fork list upgrades, such as using a base
OS image that is three years old and then doing "dnf -y update" on it.
It's very difficult to predict which update will break what, but
updating 700 packages at once as I just saw on a base OS image of RHEL
8. It's one reason Fedora's frequent releases are useful: they provide
a much fresher, smoke tested baseline image to apply changes to,
rather than relying on upgrades in places.
I can work on some of those changes to documentation and to rpm or
dnf,
but I'd like input from the developer community before starting. (And
at some point I think that Fedora, the org, should probably consider
whether `dnf update --security` is broken and unreliable.)
It's popular. I'm remembering when the "dnf update -y --security"
broke curl, because either "curl-libs" or "crypto-policies" was not
listed in those updates, and hilarity ensued. It's why I try to
schedule regular "dnf -y update" operations, and updating my base OS
images to match. It can get a bit awkward when the base OS images at
your favorite cloud vendor fall increasingly out of date over time.
6:27 a.m.
New subject: Proposal: dnf should offer to update all of the dependencies of
any package installed or updated
On Sat, Jan 28, 2023 at 02:14:58AM -0600, Bruno Wolff III wrote:
On Fri, Jan 27, 2023 at 18:43:56 -0800,
Gordon Messmer <gordon.messmer(a)gmail.com> wrote:
>
>Second, I'd like to suggest that in the future, at least in
>Fedora, for any "install" or "update" operation that dnf
performs,
>dnf's default behavior should be checking all of the direct and
>indirect dependencies of the packages being installed (or updated)
>and updating any dependencies which have updates available.
>
>Does anyone else have any opinions on the subject? Should I
>simply file a bug against dnf proposing this behavior?
If there is a problem with not uodating dependencies when you do an
install or an update on selected packages, the packages dependencies
are not properly defined.
You're correct, but ...
It's quite often the case that updates fail because of a missing
symbol in some library that wasn't updated. I think this even happens
if you're using symbol versioning, but in any case many libraries
don't use symbol versioning[1].
Gordon's suggestion is a pretty good one for fixing this common
problem without making developers add full ">= version" dependencies
to every BuildRequires line in every package they maintain.
I think the case where you don't want to keep the full system up
to
date, but a selective update or install causes problems as well is
pretty rare. I think it might be reasonable to have an option that
requests doing a recursive update. I would consider this to be a low
priority feature request that has to compete with all of the other
work being done on dnf, rather than a bug. I don't work on dnf and
the people that do might feel differently.
Not really that rare. I do this all the time. Especially useful when
keeping a system "mainly" on a Fedora release, but selectively testing
Rawhide packages.
Rich.
[1] https://www.berrange.com/posts/2011/01/13/versioning-in-the-libvirt-library/
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
7:16 a.m.
New subject: Proposal: dnf should offer to update all of the dependencies of
any package installed or updated
On Mon, Jan 30, 2023 at 1:27 PM Richard W.M. Jones <rjones(a)redhat.com> wrote:
On Sat, Jan 28, 2023 at 02:14:58AM -0600, Bruno Wolff III wrote:
> On Fri, Jan 27, 2023 at 18:43:56 -0800,
> Gordon Messmer <gordon.messmer(a)gmail.com> wrote:
> >
> >Second, I'd like to suggest that in the future, at least in
> >Fedora, for any "install" or "update" operation that dnf
performs,
> >dnf's default behavior should be checking all of the direct and
> >indirect dependencies of the packages being installed (or updated)
> >and updating any dependencies which have updates available.
> >
> >Does anyone else have any opinions on the subject? Should I
> >simply file a bug against dnf proposing this behavior?
>
> If there is a problem with not uodating dependencies when you do an
> install or an update on selected packages, the packages dependencies
> are not properly defined.
You're correct, but ...
It's quite often the case that updates fail because of a missing
symbol in some library that wasn't updated. I think this even happens
if you're using symbol versioning, but in any case many libraries
don't use symbol versioning[1].
Gordon's suggestion is a pretty good one for fixing this common
problem without making developers add full ">= version" dependencies
to every BuildRequires line in every package they maintain.
This breaks down as soon as you talk about swapping library
implementations. The most recent example of this was SDL ->
sdl12-compat. Initially, the latter's version scheme didn't match the
former at all. If we had this, then replacing the library would have
been more painful to do.
--
真実はいつも一つ！/ Always, there's only one truth!
477
days inactive
479
days old
20 comments
11 participants
participants (11)
-
Bruno Wolff III -
Fabio Valentini -
Florian Weimer -
Gordon Messmer -
Kevin Fenzi -
Michael J Gruber -
Neal Gompa -
Nico Kadel-Garcia -
Petr Pisar -
Richard W.M. Jones -
Zbigniew Jędrzejewski-Szmek