2:53 p.m.
I know I have probably been hiding under a Rock but can anyone help me
understand Fedora's stance on circular dependencies within RPMs?
At least in the past I think circular dependencies have been kept to a minimum
as it can cause issues with rpm sorting: i.e. for two rpms A,B with a circular
dependency, in different transactions sometimes A goes before B and sometimes
B goes before A.
In Fedora 20 just running rpm against the 1500 or so packages I have installed
on my Desktop I notice that there are 47 sets of "Strongly Connected
Components";
basically 47 cases of a circular dependency.
Was there a point where this became more popular. I don't remember there being more
than a few circular deps for EL6 era rpms.
Thanks,
Dusty
3:11 p.m.
Am 22.08.2014 um 21:53 schrieb Dusty Mabe:
I know I have probably been hiding under a Rock but can anyone help
me
understand Fedora's stance on circular dependencies within RPMs?
At least in the past I think circular dependencies have been kept to a minimum
as it can cause issues with rpm sorting: i.e. for two rpms A,B with a circular
dependency, in different transactions sometimes A goes before B and sometimes
B goes before A.
In Fedora 20 just running rpm against the 1500 or so packages I have installed
on my Desktop I notice that there are 47 sets of "Strongly Connected
Components";
basically 47 cases of a circular dependency.
Was there a point where this became more popular. I don't remember there being more
than a few circular deps for EL6 era rpms.
how should it work otherwise?
* each package defines it's depndencies
* each of the dependencies have their own
* no package knows the whole picture
* each package has to pull all deps it needs
if
* A requires B
* B requires C
it's clear that removal of C would also remove A because
it removes B caused by the deps leading to remove A
if you can remove C the B has a package error which is
in theory possible because C already could have been
installed and nobody noticed the missing dep until
you remove it and nothing complais (been there)
3:23 p.m.
On Fri, Aug 22, 2014 at 10:11:25PM +0200, Reindl Harald wrote:
Am 22.08.2014 um 21:53 schrieb Dusty Mabe:
> I know I have probably been hiding under a Rock but can anyone help me
> understand Fedora's stance on circular dependencies within RPMs?
>
> At least in the past I think circular dependencies have been kept to a minimum
> as it can cause issues with rpm sorting: i.e. for two rpms A,B with a circular
> dependency, in different transactions sometimes A goes before B and sometimes
> B goes before A.
>
> In Fedora 20 just running rpm against the 1500 or so packages I have installed
> on my Desktop I notice that there are 47 sets of "Strongly Connected
Components";
> basically 47 cases of a circular dependency.
>
> Was there a point where this became more popular. I don't remember there being
more
> than a few circular deps for EL6 era rpms.
how should it work otherwise?
* each package defines it's depndencies
* each of the dependencies have their own
* no package knows the whole picture
* each package has to pull all deps it needs
Sure. Each package defines all deps it needs is how it should be.
if
* A requires B
* B requires C
The scenario I am concerned with here is:
if
* A requires B
* B requires C
* C requires A
This basically yields a case where ordering can't be properly done because
rpm doesn't know which dependency is stronger. If all of the rpms in question
just deliver files then all is well with the world but if each of them have
scriptlets the ordering can definitely matter and this would be a problem.
Dusty
3:29 p.m.
On 08/22/2014 02:23 PM, Dusty Mabe wrote:
The scenario I am concerned with here is:
if
* A requires B
* B requires C
* C requires A
This basically yields a case where ordering can't be properly done because
rpm doesn't know which dependency is stronger. If all of the rpms in question
just deliver files then all is well with the world but if each of them have
scriptlets the ordering can definitely matter and this would be a problem.
Dusty
Scriptlet dependency ordering it specified with for example:
Requires(post): pkg
Are there any circular scriptlet deps?
--
Orion Poplawski
Technical Manager 303-415-9701 x222
NWRA, Boulder/CoRA Office FAX: 303-415-9702
3380 Mitchell Lane orion(a)nwra.com
Boulder, CO 80301 http://www.nwra.com
4:15 p.m.
On Fri, Aug 22, 2014 at 02:29:31PM -0600, Orion Poplawski wrote:
On 08/22/2014 02:23 PM, Dusty Mabe wrote:
>The scenario I am concerned with here is:
>
>if
> * A requires B
> * B requires C
> * C requires A
>
>This basically yields a case where ordering can't be properly done because
>rpm doesn't know which dependency is stronger. If all of the rpms in question
>just deliver files then all is well with the world but if each of them have
>scriptlets the ordering can definitely matter and this would be a problem.
>
>Dusty
>
Scriptlet dependency ordering it specified with for example:
Requires(post): pkg
Are there any circular scriptlet deps?
So that may be the answer. This *new to me* feature solves the case of
scriptlets which is really the only place you see errors *during* the
install.
Thanks Orion
9:37 a.m.
On Fri, Aug 22, 2014 at 5:15 PM, Dusty Mabe <dustymabe(a)gmail.com> wrote:
On Fri, Aug 22, 2014 at 02:29:31PM -0600, Orion Poplawski wrote:
> On 08/22/2014 02:23 PM, Dusty Mabe wrote:
> >The scenario I am concerned with here is:
> >
> >if
> > * A requires B
> > * B requires C
> > * C requires A
> >
> >This basically yields a case where ordering can't be properly done because
> >rpm doesn't know which dependency is stronger. If all of the rpms in
question
> >just deliver files then all is well with the world but if each of them have
> >scriptlets the ordering can definitely matter and this would be a problem.
> >
> >Dusty
> >
>
> Scriptlet dependency ordering it specified with for example:
>
> Requires(post): pkg
>
> Are there any circular scriptlet deps?
So that may be the answer. This *new to me* feature solves the case of
scriptlets which is really the only place you see errors *during* the
install.
Thanks Orion
Installation scripting is not the only source of the problem. Perl
modules have been prone to this.
* Perl module A requires perl module B.
* Perl module B requires perl module C.
* One small script or macro in module C requires one small script from
module A. It may not even be a critical component of C, and may be
easily segregated, but suddenly there is a circular dependency.
The vortex enters when the author of module B updates to a new
dependency or build dependency that is not in the current version of
C, and C introduces a new dependency on A but it's based on an older
version of A, that has since discarded that macro due to a code
cleanup. Hilarity ensues.....
The underlying point is that it's sometimes very helpful to split
upstream packages to smaller, individual components, precisely to
segregate these dependencies. It's especially useful with Perl
modules. I have.... old stories about the mod_perl updates back when
HTTPD 2.x was first released and mod_perl compatibility became kind of
nutty.
3:40 a.m.
On 08/24/2014 04:37 PM, Nico Kadel-Garcia wrote:
Installation scripting is not the only source of the problem. Perl
modules have been prone to this.
* Perl module A requires perl module B.
* Perl module B requires perl module C.
* One small script or macro in module C requires one small script from
module A. It may not even be a critical component of C, and may be
easily segregated, but suddenly there is a circular dependency.
The vortex enters when the author of module B updates to a new
dependency or build dependency that is not in the current version of
C, and C introduces a new dependency on A but it's based on an older
version of A, that has since discarded that macro due to a code
cleanup. Hilarity ensues.....
The underlying point is that it's sometimes very helpful to split
upstream packages to smaller, individual components, precisely to
segregate these dependencies. It's especially useful with Perl
modules. I have.... old stories about the mod_perl updates back when
HTTPD 2.x was first released and mod_perl compatibility became kind of
nutty.
Or we can wait for F21, which will have weak dependencies in RPM. And I anticipate that
weak dependencies will break a
lot of circles.
--
Miroslav Suchy, RHCE, RHCDS
Red Hat, Senior Software Engineer, #brno, #devexp, #fedora-buildsys
5:55 a.m.
On 2014-08-25, Miroslav Suchý <msuchy(a)redhat.com> wrote:
Or we can wait for F21, which will have weak dependencies in RPM.
And
I anticipate that weak dependencies will break a lot of circles.
Does Fedora have guidelines what should and what should not be a weak
dependency?
My experience with Perl packages is that "declaring dependency because it's
recommended but the package works without it" is quite seldom. The
majority of build cycles are either build-time dependencies for tests or
hard run-time dependencies (the Perl module exits with an exception
without the dependency in some code paths.)
In my opinion, it would be much more appreciated if Fedora had
a mechanism to express "I want support for PDF" on the installed system
and then package manager would use this boolean to install or skip
affected dependencies. (This is the case of "some code paths" from
previous paragraph.)
-- Petr
12:50 p.m.
On Tue, Aug 26, 2014 at 10:55:56AM +0000, Petr Pisar wrote:
On 2014-08-25, Miroslav Suchý <msuchy(a)redhat.com> wrote:
> Or we can wait for F21, which will have weak dependencies in RPM. And
> I anticipate that weak dependencies will break a lot of circles.
>
Does Fedora have guidelines what should and what should not be a weak
dependency?
My experience with Perl packages is that "declaring dependency because it's
recommended but the package works without it" is quite seldom. The
majority of build cycles are either build-time dependencies for tests or
hard run-time dependencies (the Perl module exits with an exception
without the dependency in some code paths.)
In my opinion, it would be much more appreciated if Fedora had
a mechanism to express "I want support for PDF" on the installed system
and then package manager would use this boolean to install or skip
affected dependencies. (This is the case of "some code paths" from
previous paragraph.)
Would virtual provides give us that?
Packages A, B and C have "Provides: pdf_reader" and Package B has
"Requires:
pdf_reader". The issue would to determine which provider to install, and
do you ask the user to install it first or maybe install a "recommended"
package?
--
Darryl L. Pierce <mcpierce(a)gmail.com>
http://mcpierce.blogspot.com/
Famous last words:
"I wonder what happens if we do it this way?"
8:20 a.m.
On 2014-08-28, Darryl L. Pierce <mcpierce(a)gmail.com> wrote:
On Tue, Aug 26, 2014 at 10:55:56AM +0000, Petr Pisar wrote:
> In my opinion, it would be much more appreciated if Fedora had
> a mechanism to express "I want support for PDF" on the installed system
> and then package manager would use this boolean to install or skip
> affected dependencies. (This is the case of "some code paths" from
> previous paragraph.)
Would virtual provides give us that?
Packages A, B and C have "Provides: pdf_reader" and Package B has
"Requires: pdf_reader". The issue would to determine which provider to
install, and do you ask the user to install it first or maybe install
a "recommended" package?
You are still going in circles. The key point is to express the "Requires:
pdf_reader" at package B is a weak dependency.
Then an interactive package manager could ask "The pdf_reader
B dependency is recommended but not needed. Do you want to fullfill
it?", and user had an opportunity to skip it completely.
(Of course, you could create an empty package providing "pdf_reader", but
I feel these fake provides just lead to confusion. So I do not recommend
this way. See the fakesystemd afair.)
And because users will get tired by explaining their choises with each
package manager invocation, the answer should go into local
configuration file so the decision could be reused next time.
This per-package decision can even get default value from a global
(all-package) configuration. E.g. if an user is not interested in the PDF
support at all, then any weak dependency somehow qualified as a PDF
could be auto-answered.
And the global configuration could even replace the server roles feature
because, at least now as I know, it's only about selecting default
package set.
If you know USE flags in Gentoo, then this is it.
-- Petr
4:15 a.m.
On 2014-08-24, Nico Kadel-Garcia <nkadel(a)gmail.com> wrote:
Installation scripting is not the only source of the problem. Perl
modules have been prone to this.
* Perl module A requires perl module B.
* Perl module B requires perl module C.
* One small script or macro in module C requires one small script from
module A. It may not even be a critical component of C, and may be
easily segregated, but suddenly there is a circular dependency.
There is no algorithmic difference between Perl packages and any other
packages.
Specifically to Perl, we employ %perl_bootstrap macro which guards the
`not even a critical' dependency which allows us to break the dependency
cycle.
Perl SIG exhibits weekly rebuilds
<https://ppisar.fedorapeople.org/perl_rebuild/scratch/index.xhtml> which
allow us to watch that all Perl packages are buildable.
The vortex enters when the author of module B updates to a new
dependency or build dependency that is not in the current version of
C, and C introduces a new dependency on A but it's based on an older
version of A, that has since discarded that macro due to a code
cleanup. Hilarity ensues.....
If you need to do such an upgrade, provided you've already deployed the
%perl_bootstrap condition, you just define perl_bootstrap temporarily in
one of the upgraded packages, and after upgrading the connected
component, you remove the macro definition and rebuild the package
again.
The underlying point is that it's sometimes very helpful to
split
upstream packages to smaller, individual components, precisely to
segregate these dependencies. It's especially useful with Perl
modules.
Definitely. Once packageres realize that Perl modules can move beetween
RPM packages freely, once they realize that specifying all direct
dependencies without relying on transitive dependencies is the best
practice and a must for healthy distribution, then splitting packages is
obvious solution.
Although splitting cannot solve all issues, especially if the splitted
modules are really interdependend. Then there is no point in the split.
However that's a upstream issue.
-- Petr
3521
days inactive
3528
days old
10 comments
7 participants
participants (7)
-
Darryl L. Pierce -
Dusty Mabe -
Miroslav Suchý -
Nico Kadel-Garcia -
Orion Poplawski -
Petr Pisar -
Reindl Harald