Hi folks,
The RHEL 9 composes do not have libev-devel and libuv-devel, so we cannot build python-gevent on EPEL 9 easily.
(It's possible to package the missing -devel packages separately, and I've been doing this by automatically following the NVR changes in Stream 9's Koji for several weeks with scripts at https://github.com/ktdreyer/ceph-el9. My conclusion is that it is so painful that it's not sustainable to do this for years.)
This means that python-pytest-cov and python-pytest-xdist won't be available on epel9, since those require gevent.
Several Python packages require python-pytest-cov because upstream lists it in requirements.txt or tests-requirements.txt. I think we should just patch these out in Fedora. Even apart from RHEL's restrictions, it's not a good use of resources to run pytest-cov when no one reviews coverage reports in the Koji logs, and we'll speed up builds when mock doesn't have to install this spurious BuildRequires.
Here are a list of packages where I've removed pytest-cov:
https://src.fedoraproject.org/rpms/python-watchdog/pull-request/4 https://src.fedoraproject.org/rpms/python-cheroot/pull-request/15 https://src.fedoraproject.org/rpms/python-portend/pull-request/5 https://src.fedoraproject.org/rpms/python-typing-extensions/pull-request/3
- Ken
On Fri, Sep 24, 2021 at 3:46 PM Ken Dreyer ktdreyer@ktdreyer.com wrote:
Hi folks,
The RHEL 9 composes do not have libev-devel and libuv-devel, so we cannot build python-gevent on EPEL 9 easily.
https://odcs.stream.centos.org/production/CentOS-Stream-9-20210924.0/compose...
You could request libev-devel in the composes. I remain confused why it has to be in the compose though, because libev and it's devel package are accessible in the CentOS Stream 9 buildroots today.
josh
(It's possible to package the missing -devel packages separately, and I've been doing this by automatically following the NVR changes in Stream 9's Koji for several weeks with scripts at https://github.com/ktdreyer/ceph-el9. My conclusion is that it is so painful that it's not sustainable to do this for years.)
This means that python-pytest-cov and python-pytest-xdist won't be available on epel9, since those require gevent.
Several Python packages require python-pytest-cov because upstream lists it in requirements.txt or tests-requirements.txt. I think we should just patch these out in Fedora. Even apart from RHEL's restrictions, it's not a good use of resources to run pytest-cov when no one reviews coverage reports in the Koji logs, and we'll speed up builds when mock doesn't have to install this spurious BuildRequires.
Here are a list of packages where I've removed pytest-cov:
https://src.fedoraproject.org/rpms/python-watchdog/pull-request/4 https://src.fedoraproject.org/rpms/python-cheroot/pull-request/15 https://src.fedoraproject.org/rpms/python-portend/pull-request/5 https://src.fedoraproject.org/rpms/python-typing-extensions/pull-request/3
- Ken
epel-devel mailing list -- epel-devel@lists.fedoraproject.org To unsubscribe send an email to epel-devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
On Fri, Sep 24, 2021 at 3:59 PM Josh Boyer jwboyer@redhat.com wrote:
On Fri, Sep 24, 2021 at 3:46 PM Ken Dreyer ktdreyer@ktdreyer.com wrote:
Hi folks,
The RHEL 9 composes do not have libev-devel and libuv-devel, so we cannot build python-gevent on EPEL 9 easily.
https://odcs.stream.centos.org/production/CentOS-Stream-9-20210924.0/compose...
You could request libev-devel in the composes. I remain confused why it has to be in the compose though, because libev and it's devel package are accessible in the CentOS Stream 9 buildroots today.
We can't use them in EPEL if they're not in CRB.
On Fri, Sep 24, 2021 at 4:02 PM Neal Gompa ngompa13@gmail.com wrote:
On Fri, Sep 24, 2021 at 3:59 PM Josh Boyer jwboyer@redhat.com wrote:
On Fri, Sep 24, 2021 at 3:46 PM Ken Dreyer ktdreyer@ktdreyer.com wrote:
Hi folks,
The RHEL 9 composes do not have libev-devel and libuv-devel, so we cannot build python-gevent on EPEL 9 easily.
https://odcs.stream.centos.org/production/CentOS-Stream-9-20210924.0/compose...
You could request libev-devel in the composes. I remain confused why it has to be in the compose though, because libev and it's devel package are accessible in the CentOS Stream 9 buildroots today.
We can't use them in EPEL if they're not in CRB.
Yes, that's what everyone keeps telling me. I don't understand why.
josh
On Fri, Sep 24, 2021 at 4:03 PM Josh Boyer jwboyer@redhat.com wrote:
On Fri, Sep 24, 2021 at 4:02 PM Neal Gompa ngompa13@gmail.com wrote:
On Fri, Sep 24, 2021 at 3:59 PM Josh Boyer jwboyer@redhat.com wrote:
On Fri, Sep 24, 2021 at 3:46 PM Ken Dreyer ktdreyer@ktdreyer.com wrote:
Hi folks,
The RHEL 9 composes do not have libev-devel and libuv-devel, so we cannot build python-gevent on EPEL 9 easily.
https://odcs.stream.centos.org/production/CentOS-Stream-9-20210924.0/compose...
You could request libev-devel in the composes. I remain confused why it has to be in the compose though, because libev and it's devel package are accessible in the CentOS Stream 9 buildroots today.
We can't use them in EPEL if they're not in CRB.
Yes, that's what everyone keeps telling me. I don't understand why.
Well, because outside of RHEL, everyone wants remote and local builds to have access to the same resources and not crush the servers. Since buildroot stuff isn't going out on the mirror network (otherwise, why would it be separate from CRB?), it's obvious we shouldn't rely on it for packages that people should expect to be able to build and rebuild for RHEL.
And again, by Red Hat's own sword (policy), RHEL doesn't want to ship everything needed to build stuff, so if EPEL is intended to provide the requisite community guarantees (reproducibly buildable), we have to work with what RHEL gives us.
-- 真実はいつも一つ!/ Always, there's only one truth!
On Fri, Sep 24, 2021 at 4:09 PM Neal Gompa ngompa13@gmail.com wrote:
On Fri, Sep 24, 2021 at 4:03 PM Josh Boyer jwboyer@redhat.com wrote:
On Fri, Sep 24, 2021 at 4:02 PM Neal Gompa ngompa13@gmail.com wrote:
On Fri, Sep 24, 2021 at 3:59 PM Josh Boyer jwboyer@redhat.com wrote:
On Fri, Sep 24, 2021 at 3:46 PM Ken Dreyer ktdreyer@ktdreyer.com wrote:
Hi folks,
The RHEL 9 composes do not have libev-devel and libuv-devel, so we cannot build python-gevent on EPEL 9 easily.
https://odcs.stream.centos.org/production/CentOS-Stream-9-20210924.0/compose...
You could request libev-devel in the composes. I remain confused why it has to be in the compose though, because libev and it's devel package are accessible in the CentOS Stream 9 buildroots today.
We can't use them in EPEL if they're not in CRB.
Yes, that's what everyone keeps telling me. I don't understand why.
Well, because outside of RHEL, everyone wants remote and local builds to have access to the same resources and not crush the servers. Since buildroot stuff isn't going out on the mirror network (otherwise, why would it be separate from CRB?), it's obvious we shouldn't rely on it for packages that people should expect to be able to build and rebuild for RHEL.
So you have access to what you want, you have a way to pull it down and get it locally, but you can't depend on it because... you're worried a multi-billion dollar company can't pay it's server and CDN bills?
As to why it's separate from CRB, that's because CRB is a reflection of what is provided as part of the product. It's that simple.
And again, by Red Hat's own sword (policy), RHEL doesn't want to ship everything needed to build stuff, so if EPEL is intended to provide the requisite community guarantees (reproducibly buildable), we have to work with what RHEL gives us.
I think that is also EPEL falling on EPEL's own sword a bit. I think it fails to recognize that building and distributing software can be separate things. I can see the need for a developer community to be able to build, update, and rebuild software it distributes. Access to the buildroots facilitates this. We could even point mock configs at it, or propose a buildroot repo for it if people are really worried about "servers".
However, in the context of something like python-gevent, an EPEL *end user* isn't going to want libuv-devel or libev-devel to be installed on their system at runtime. They have no need for it to be available in a compose. They only need python-gevent and the requisite runtime libraries, which are already provided. I think separating the personas and thinking about the requirements for each might be worth doing.
I understand this is a different approach and something that looks different from the past. It's been 2+ years since the OS EPEL8 is based on has shipped and it is taking a different approach than previous releases. Every indication we have shows the next major version will continue this. I'm worried that sticking with past policies precludes EPEL from making progress on a project that we all want to see succeed.
josh
On Fri, Sep 24, 2021 at 1:03 PM Josh Boyer jwboyer@redhat.com wrote:
On Fri, Sep 24, 2021 at 4:02 PM Neal Gompa ngompa13@gmail.com wrote:
On Fri, Sep 24, 2021 at 3:59 PM Josh Boyer jwboyer@redhat.com wrote:
On Fri, Sep 24, 2021 at 3:46 PM Ken Dreyer ktdreyer@ktdreyer.com
wrote:
Hi folks,
The RHEL 9 composes do not have libev-devel and libuv-devel, so we cannot build python-gevent on EPEL 9 easily.
https://odcs.stream.centos.org/production/CentOS-Stream-9-20210924.0/compose...
You could request libev-devel in the composes. I remain confused why it has to be in the compose though, because libev and it's devel package are accessible in the CentOS Stream 9 buildroots today.
We can't use them in EPEL if they're not in CRB.
Yes, that's what everyone keeps telling me. I don't understand why.
EPEL9 will not be built again CentOS Stream 9, it will be built off RHEL 9. That is the EPEL policy. Thus, if something is not in RHEL 9, it is not available to be built in EPEL 9.
It seems pretty straightforward.
Troy
On Fri, Sep 24, 2021 at 3:05 PM Josh Boyer jwboyer@redhat.com wrote:
On Fri, Sep 24, 2021 at 4:02 PM Neal Gompa ngompa13@gmail.com wrote:
On Fri, Sep 24, 2021 at 3:59 PM Josh Boyer jwboyer@redhat.com wrote:
On Fri, Sep 24, 2021 at 3:46 PM Ken Dreyer ktdreyer@ktdreyer.com wrote:
Hi folks,
The RHEL 9 composes do not have libev-devel and libuv-devel, so we cannot build python-gevent on EPEL 9 easily.
https://odcs.stream.centos.org/production/CentOS-Stream-9-20210924.0/compose...
You could request libev-devel in the composes. I remain confused why it has to be in the compose though, because libev and it's devel package are accessible in the CentOS Stream 9 buildroots today.
We can't use them in EPEL if they're not in CRB.
Yes, that's what everyone keeps telling me. I don't understand why.
EPEL builds against published RHEL content, not the CentOS Stream buildroot. Having a package available in the CentOS Stream buildroot doesn't make it accessible to EPEL builds. We can't change EPEL to use the CentOS Stream buildroot because that will cause some EPEL packages to not be installable on RHEL.
On a related note, EPEL 9 Next _is_ being set up to build against the CentOS Stream 9 buildroot. This works for EPEL Next because it explicitly targets the next minor release of RHEL (i.e. CentOS Stream). This will allow more packages to be built, at the cost of potentially confusing packagers when their package builds successfully for EPEL 9 Next but not for EPEL 9. EPEL 8 Next currently builds against published CentOS Stream 8 content. If things go well with EPEL 9 Next using the CentOS Stream 9 buildroot, EPEL 8 Next may switch to using the CentOS Stream 8 buildroot in the future.
josh _______________________________________________ epel-devel mailing list -- epel-devel@lists.fedoraproject.org To unsubscribe send an email to epel-devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
This was originally "python-gevent and pytest-cov in el9" But I wanted to talk about and discuss this paragraph from Carl on it's own.
On Fri, Sep 24, 2021 at 7:43 PM Carl George carl@redhat.com wrote:
On a related note, EPEL 9 Next _is_ being set up to build against the CentOS Stream 9 buildroot. This works for EPEL Next because it explicitly targets the next minor release of RHEL (i.e. CentOS Stream). This will allow more packages to be built, at the cost of potentially confusing packagers when their package builds successfully for EPEL 9 Next but not for EPEL 9. EPEL 8 Next currently builds against published CentOS Stream 8 content. If things go well with EPEL 9 Next using the CentOS Stream 9 buildroot, EPEL 8 Next may switch to using the CentOS Stream 8 buildroot in the future.
Although I'm not against using the CentOS Stream buildroot for epel9-next, we're already starting to see large sections of KDE that WILL be buildable against the buildroot but WILL NOT be buildable against plain epel9.[1]
This isn't a matter of "confusing packagers", it's a matter of major delays when we switch from epel9-next to epel9.
I'll have to file all those "please move these packages into CRB" bugz after RHEL9 is out. I'll also have to request "branch and builds" for all those packages in the buildroot that are ONLY in the buildroot. And then request the dependencies for those packages. And so forth.
And that's just the packages I am working on. I can't speak for anyone else.
[possible solution]
If we are going to have the CS buildroots in any of our epel-next repo's, we need to develop a "missing build-dep" service.
Something that regularly checks to make sure epel9-next packages are built against packages that will be in epel9. And if they are not buildable, let's the package maintainer know. And let Red Hat know as well. A nice list that people can easily look at and see which "missing build-dep" is being used, and by what packages.
Troy
p.s. I'm sorry if this email sounds accusatory and/or negative. I've re-written it twice and it still sounds negative to me. Possibly cuz I'm showing the bad things that can/will happen. But I want to put the emphasis on the possible solution. More of a "hay, let's see if this would work." So, if someone finds this hurting their feelings, I'm sorry. At this point I just need to send it, or I never will.
On 01. 10. 21 22:11, Troy Dawson wrote:
I'll have to file all those "please move these packages into CRB" bugz after RHEL9 is out.
I wonder whether we should file those before it is out, if at all possible?
On Fri, Oct 1, 2021 at 5:39 PM Miro Hrončok mhroncok@redhat.com wrote:
On 01. 10. 21 22:11, Troy Dawson wrote:
I'll have to file all those "please move these packages into CRB" bugz after RHEL9 is out.
I wonder whether we should file those before it is out, if at all possible?
Ideally, yes because it's an extreme pain to deal with them post-GA. Red Hat's policies around this stuff is such that after 9.0, they can only be introduced in 6 month intervals (that is, we have to wait for 9.1 for them to show up in EPEL9). We should *never* use the buildroot repository for EPEL-Next because it massively complicates discovering this stuff and getting it straightened out immediately. Unlike CentOS SIGs that only build for CentOS Stream, EPEL needs to work on RHEL, and Red Hat will *never* ship the buildroot repo for RHEL, because it opens up the door for things to depend on that content, which is highly problematic for them.
-- 真実はいつも一つ!/ Always, there's only one truth!
*this is worth a discussion in todays EPEL Steering Committee Meeting*
It sounds like the epel9-next is going to startup by building against the CS buildroot. Changing it at this time would cause a delay.
Thus we need to write some "verify build deps are released" checker. I have an idea of how to do this, so I'm willing to volunteer to write and run something.
But, it would be good to have some discussion to determine if we want to keep using the CS buildroot for epel9-next, always. Or if we want to use it just as a bootstrap mechanism, and then switch to building just off the available CentOS Stream repos at some point.
Thoughts? Should we always use buildroot? Or just keep up until we're fairly stable?
Troy
On Wed, Oct 6, 2021 at 3:36 PM Troy Dawson tdawson@redhat.com wrote:
*this is worth a discussion in todays EPEL Steering Committee Meeting*
It sounds like the epel9-next is going to startup by building against the CS buildroot. Changing it at this time would cause a delay.
Thus we need to write some "verify build deps are released" checker. I have an idea of how to do this, so I'm willing to volunteer to write and run something.
But, it would be good to have some discussion to determine if we want to keep using the CS buildroot for epel9-next, always. Or if we want to use it just as a bootstrap mechanism, and then switch to building just off the available CentOS Stream repos at some point.
Thoughts? Should we always use buildroot? Or just keep up until we're fairly stable?
We should only use the buildroot repo for as long as we need to. The *sooner* we can switch to the published content, the better.
On Wed, Oct 6, 2021 at 7:39 PM Neal Gompa ngompa13@gmail.com wrote:
On Wed, Oct 6, 2021 at 3:36 PM Troy Dawson tdawson@redhat.com wrote:
*this is worth a discussion in todays EPEL Steering Committee Meeting*
It sounds like the epel9-next is going to startup by building against
the CS buildroot. Changing it at this time would cause a delay.
Thus we need to write some "verify build deps are released" checker. I
have an idea of how to do this, so I'm willing to volunteer to write and run something.
But, it would be good to have some discussion to determine if we want to
keep using the CS buildroot for epel9-next, always. Or if we want to use it just as a bootstrap mechanism, and then switch to building just off the available CentOS Stream repos at some point.
Thoughts? Should we always use buildroot? Or just keep up until we're fairly
stable?
We should only use the buildroot repo for as long as we need to. The *sooner* we can switch to the published content, the better.
This was discussed at the EPEL Steering Committee meeting. Here is the summary. Someone correct me if I'm wrong.
epel9-next: - starts off building off CS buildroot - I will write a "check if all build packages are in the normal repos" checker, called "will it build" [1]
epel9: - Use normal RHEL 9 repos (AppStream, BaseOS, CRB)
Checks/Tests/Future: (It's a little fuzzy on the timing of these)
- grobisplitter -- see if we really need to use grobisplitter -- I'm a little fuzzy on how or when we are going to test this
- "will it build" -- After a period of time, check and see if we are happy with the combination of CS buildroot + "will it build" ? --- Yes - determine if we want to keep it like this even after epel9 --- No - determine when the best/safest time to switch to normal repos
Troy
[1] - I plan on integrating "will it build" with a "will it install" checker. It will be called "will it" or "willit". I'm sorta excited about doing this. It's something I've wanted to do for a long time and I think I've finally got it figured out.
On Thu, Oct 7, 2021 at 9:24 AM Troy Dawson tdawson@redhat.com wrote:
On Wed, Oct 6, 2021 at 7:39 PM Neal Gompa ngompa13@gmail.com wrote:
On Wed, Oct 6, 2021 at 3:36 PM Troy Dawson tdawson@redhat.com wrote:
*this is worth a discussion in todays EPEL Steering Committee Meeting*
It sounds like the epel9-next is going to startup by building against the CS buildroot. Changing it at this time would cause a delay.
Thus we need to write some "verify build deps are released" checker. I have an idea of how to do this, so I'm willing to volunteer to write and run something.
But, it would be good to have some discussion to determine if we want to keep using the CS buildroot for epel9-next, always. Or if we want to use it just as a bootstrap mechanism, and then switch to building just off the available CentOS Stream repos at some point.
Thoughts? Should we always use buildroot? Or just keep up until we're fairly stable?
We should only use the buildroot repo for as long as we need to. The *sooner* we can switch to the published content, the better.
This was discussed at the EPEL Steering Committee meeting. Here is the summary. Someone correct me if I'm wrong.
epel9-next:
- starts off building off CS buildroot
- I will write a "check if all build packages are in the normal repos" checker, called "will it build" [1]
How are we going to know whether all the build-time and run-time packages are in the normal repos? We need to check generated dependencies too, especially now that it's possible to have dynamic BuildRequires!
epel9:
- Use normal RHEL 9 repos (AppStream, BaseOS, CRB)
Checks/Tests/Future: (It's a little fuzzy on the timing of these)
- grobisplitter
-- see if we really need to use grobisplitter -- I'm a little fuzzy on how or when we are going to test this
With the retirement of the container-tools default module, grobisplitter will not be required at all unless we want to use it to support non-default modules.
-- 真実はいつも一つ!/ Always, there's only one truth!
On Thu, Oct 7, 2021 at 6:28 AM Neal Gompa ngompa13@gmail.com wrote:
On Thu, Oct 7, 2021 at 9:24 AM Troy Dawson tdawson@redhat.com wrote:
On Wed, Oct 6, 2021 at 7:39 PM Neal Gompa ngompa13@gmail.com wrote:
On Wed, Oct 6, 2021 at 3:36 PM Troy Dawson tdawson@redhat.com wrote:
*this is worth a discussion in todays EPEL Steering Committee Meeting*
It sounds like the epel9-next is going to startup by building against
the CS buildroot. Changing it at this time would cause a delay.
Thus we need to write some "verify build deps are released" checker.
I have an idea of how to do this, so I'm willing to volunteer to write and run something.
But, it would be good to have some discussion to determine if we want
to keep using the CS buildroot for epel9-next, always. Or if we want to use it just as a bootstrap mechanism, and then switch to building just off the available CentOS Stream repos at some point.
Thoughts? Should we always use buildroot? Or just keep up until we're fairly
stable?
We should only use the buildroot repo for as long as we need to. The *sooner* we can switch to the published content, the better.
This was discussed at the EPEL Steering Committee meeting. Here is the
summary.
Someone correct me if I'm wrong.
epel9-next:
- starts off building off CS buildroot
- I will write a "check if all build packages are in the normal repos"
checker, called "will it build" [1]
How are we going to know whether all the build-time and run-time packages are in the normal repos? We need to check generated dependencies too, especially now that it's possible to have dynamic BuildRequires!
run-time dependencies: That's always been a problem, even without the buildroot. But I will also be writing a "will it install" to go along with "will it build"
build-time dependencies: Grab the root.log of the package build, and parse it. This gets around any hidden and dynamic BuildRequires. I've already written code that does this for Content Resolver, and checked it against traditional dnf/libsolve dependency generation. It was 98% equal, and those 2% were on packages where it was possible for more than one package to be installed for a dependency, and for that, I'd prefer going with the root.log.
I think I've got everything I need already written, just in three separate projects. I really want to pull that code together and make "willit"
epel9:
- Use normal RHEL 9 repos (AppStream, BaseOS, CRB)
Checks/Tests/Future: (It's a little fuzzy on the timing of these)
- grobisplitter
-- see if we really need to use grobisplitter -- I'm a little fuzzy on how or when we are going to test this
With the retirement of the container-tools default module, grobisplitter will not be required at all unless we want to use it to support non-default modules.
That is the theory, yes, that grobisplitter isn't required. But nobody was able to say that was for certain. Thus, it needs to be tested.
Troy
On Thu, Oct 7, 2021 at 9:44 AM Troy Dawson tdawson@redhat.com wrote:
On Thu, Oct 7, 2021 at 6:28 AM Neal Gompa ngompa13@gmail.com wrote:
On Thu, Oct 7, 2021 at 9:24 AM Troy Dawson tdawson@redhat.com wrote:
On Wed, Oct 6, 2021 at 7:39 PM Neal Gompa ngompa13@gmail.com wrote:
On Wed, Oct 6, 2021 at 3:36 PM Troy Dawson tdawson@redhat.com wrote:
*this is worth a discussion in todays EPEL Steering Committee Meeting*
It sounds like the epel9-next is going to startup by building against the CS buildroot. Changing it at this time would cause a delay.
Thus we need to write some "verify build deps are released" checker. I have an idea of how to do this, so I'm willing to volunteer to write and run something.
But, it would be good to have some discussion to determine if we want to keep using the CS buildroot for epel9-next, always. Or if we want to use it just as a bootstrap mechanism, and then switch to building just off the available CentOS Stream repos at some point.
Thoughts? Should we always use buildroot? Or just keep up until we're fairly stable?
We should only use the buildroot repo for as long as we need to. The *sooner* we can switch to the published content, the better.
This was discussed at the EPEL Steering Committee meeting. Here is the summary. Someone correct me if I'm wrong.
epel9-next:
- starts off building off CS buildroot
- I will write a "check if all build packages are in the normal repos" checker, called "will it build" [1]
How are we going to know whether all the build-time and run-time packages are in the normal repos? We need to check generated dependencies too, especially now that it's possible to have dynamic BuildRequires!
run-time dependencies: That's always been a problem, even without the buildroot. But I will also be writing a "will it install" to go along with "will it build"
build-time dependencies: Grab the root.log of the package build, and parse it. This gets around any hidden and dynamic BuildRequires. I've already written code that does this for Content Resolver, and checked it against traditional dnf/libsolve dependency generation. It was 98% equal, and those 2% were on packages where it was possible for more than one package to be installed for a dependency, and for that, I'd prefer going with the root.log.
I think I've got everything I need already written, just in three separate projects. I really want to pull that code together and make "willit"
epel9:
- Use normal RHEL 9 repos (AppStream, BaseOS, CRB)
Checks/Tests/Future: (It's a little fuzzy on the timing of these)
- grobisplitter
-- see if we really need to use grobisplitter -- I'm a little fuzzy on how or when we are going to test this
With the retirement of the container-tools default module, grobisplitter will not be required at all unless we want to use it to support non-default modules.
That is the theory, yes, that grobisplitter isn't required. But nobody was able to say that was for certain. Thus, it needs to be tested.
I've verified this with my internal build infrastructure, so yes, I know it's not required.
Admittedly, it's not a Koji system, but I'm also not enabling any modules in my build environment right now. I'm rebuilding content from Rawhide targeting CentOS Stream 9 to get a list of initial EPEL 9 packages to build for work, which is how some of my requests to add stuff to CRB have come about[1][2][3].
This can also be verified when using something like mock with mock-core-configs v36 or higher, because I made the necessary adjustments to test building on CentOS Stream 9 the same way that Fedora Koji and the CentOS CBS would.
[1]: https://gitlab.com/redhat/centos-stream/release-engineering/comps/-/merge_re... [2]: https://gitlab.com/redhat/centos-stream/release-engineering/comps/-/merge_re... [3]: https://gitlab.com/redhat/centos-stream/release-engineering/comps/-/merge_re...
On Thu, 7 Oct 2021 at 09:52, Neal Gompa ngompa13@gmail.com wrote:
That is the theory, yes, that grobisplitter isn't required. But nobody was able to say that was for certain. Thus, it needs to be tested.
I've verified this with my internal build infrastructure, so yes, I know it's not required.
Admittedly, it's not a Koji system, but I'm also not enabling any modules in my build environment right now. I'm rebuilding content from Rawhide targeting CentOS Stream 9 to get a list of initial EPEL 9 packages to build for work, which is how some of my requests to add stuff to CRB have come about[1][2][3].
Things have probably improved, but the lesson I learned from EPEL-8 and afterwords was that koji depsolving is weird no matter how set up. Koji sets up mock environments in a way that will do fine as long as there are NO modules in the repositories it is looking at. Once a module, even a non-default module, is there things start to go wonky because the way that koji depsolves will say that 'foobaz-3.10.1-3+module' is a better solution than 'foobax-3.9.4.<arch>' it will then try to pull that in and boom you end up with broken builds. You can change the method that koji chooses packages to be in the buildroot, but the other option ends up trying to insert things like foobax-3.9.4-i386 into an x86_64 or still does the modular change but chooses foobar-2 due to some depsolv quirk.
At the moment I think building without grobisplitter will work, but I am thinking some other solution will need to be made when EL9.x starts rolling out with modules in it.
This can also be verified when using something like mock with mock-core-configs v36 or higher, because I made the necessary adjustments to test building on CentOS Stream 9 the same way that Fedora Koji and the CentOS CBS would.
On Thu, 7 Oct 2021, Stephen John Smoogen wrote:
Things have probably improved, but the lesson I learned from EPEL-8 and afterwords was that koji depsolving is weird no matter how set up. Koji sets up mock environments in a way that will do fine as long as there are NO modules in the repositories it is looking at. Once a module, even a non-default module, is there things start to go wonky because the way that koji depsolves will say that 'foobaz-3.10.1-3+module' is a better solution than 'foobax-3.9.4.<arch>' it will then try to pull that in and boom you end up with broken builds. You can change the method that koji chooses packages to be in the buildroot, but the other option ends up trying to insert things like foobax-3.9.4-i386 into an x86_64 or still does the modular change but chooses foobar-2 due to some depsolv quirk.
Is the foobar/foobaz/foobax intended or are they typos ? If they are intended, then I think we have a partially ordered set of packages, ie it isn't always possible to say whether a > b or a < b.
Without knowing anything about koji, I'd say that whilst foobaz and foobax can provide foobar (and perhaps foobar==3) they cannot satisfy foobar>=3.9 (unless they are forks or reimplementations of foobar-3.9).
On Tue, 19 Oct 2021 at 01:22, Andrew C Aitchison andrew@aitchison.me.uk wrote:
On Thu, 7 Oct 2021, Stephen John Smoogen wrote:
Things have probably improved, but the lesson I learned from EPEL-8 and afterwords was that koji depsolving is weird no matter how set up. Koji sets up mock environments in a way that will do fine as long as there are NO modules in the repositories it is looking at. Once a module, even a non-default module, is there things start to go wonky because the way that koji depsolves will say that 'foobaz-3.10.1-3+module' is a better solution than 'foobax-3.9.4.<arch>' it will then try to pull that in and boom you end up with broken builds. You can change the method that koji chooses packages to be in the buildroot, but the other option ends up trying to insert things like foobax-3.9.4-i386 into an x86_64 or still does the modular change but chooses foobar-2 due to some depsolv quirk.
Is the foobar/foobaz/foobax intended or are they typos ? If they are intended, then I think we have a partially ordered set of packages, ie it isn't always possible to say whether a > b or a < b.
Without knowing anything about koji, I'd say that whilst foobaz and foobax can provide foobar (and perhaps foobar==3) they cannot satisfy foobar>=3.9 (unless they are forks or reimplementations of foobar-3.9).
In the case I was remembering they were forks and say inside them that they satisfy anything >= 3.9. Normally you would only have one in a buildroot but you could have alternatives in modules because they each fixed something specific to that module. If you tried to install both modules you would get conflicts so only one module was to be installed at a time.. However, Koji doesn't know that.. one solution method only knows NEVR, Provides and Requires and another tries to use a bit of dnf to do resolutions but that seemed to act weirdly also.
In the case where they are all foobar but say foobar-4.0 or foobar4 is in a non-default module, then koji and dnf might come again to different conclusions of what is needed to be in the buildroot. The only mental solution I could come up with was that there would need to be some tool to put all modules (and bare rpms dependent on them) into a RHEL-modularity tree and somehow import that into MBS so that it could deal with them. That would then allow for both EPEL modules to depend on them, and keep them out of the way for koji regular builds. The major problems with that was a) no perl/python/ruby/etc in non-modular builds (all that would be available is platform-python) b) it would break Fedora build system in ways because it assumes what is in its toolkit was built by it and we are faking these.
In any case, it is a conundrum wrapped up in an enigma surrounded by a puzzle.
-- Andrew C. Aitchison Kendal, UK andrew@aitchison.me.uk _______________________________________________ epel-devel mailing list -- epel-devel@lists.fedoraproject.org To unsubscribe send an email to epel-devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
On Fri, Sep 24, 2021 at 3:59 PM Josh Boyer jwboyer@redhat.com wrote:
https://odcs.stream.centos.org/production/CentOS-Stream-9-20210924.0/compose...
On the one hand, thank you for pointing out that this build is now available. That's good to know.
On the other hand, this points at the bigger issue that dealing with the entire problem of missing packages requires a level of scripting and bookkeeping that is very difficult to keep up when building layered projects.
You could request libev-devel in the composes.
The reason I did not do that in this case is that pytest-cov is an optional dependency, and we can just remove it from the Python packages instead. I'd rather reduce the dependencies on gevent to make everything faster.
When I looked at gevent in EPEL 8 a month or so ago, it did not look like many packages depended on it.
I remain confused why it has to be in the compose though, because libev and it's devel package are accessible in the CentOS Stream 9 buildroots today.
We could point at https://kojihub.stream.centos.org/kojifiles/repos/c9s-build/latest/ , but that location will not have GPG-signed builds, and the repo is not currently in https://github.com/rpm-software-management/mock/blob/main/mock-core-configs/...
- Ken
On 24. 09. 21 21:45, Ken Dreyer wrote:
This means that python-pytest-cov and python-pytest-xdist won't be available on epel9, since those require gevent.
Ignoring the rest of your email for now, but I don't the gevent dependency does not exsist:
$ [dnf] install python3-pytest-cov python3-pytest-xdist ... Installed: expat-2.4.1-2.fc35.x86_64 mpdecimal-2.5.1-2.fc35.x86_64 openssl1.1-1:1.1.1l-1.fc36.x86_64 python-pip-wheel-21.2.3-2.fc36.noarch python-setuptools-wheel-57.4.0-1.fc35.noarch python3-3.10.0~rc2-2.fc36.x86_64 python3-attrs-21.2.0-4.fc35.noarch python3-coverage-5.6-0.3b1.fc35.x86_64 python3-execnet-1.9.0-2.fc35.noarch python3-iniconfig-1.1.1-5.fc35.noarch python3-libs-3.10.0~rc2-2.fc36.x86_64 python3-packaging-21.0-2.fc35.noarch python3-pluggy-1.0.0-1.fc36.noarch python3-py-1.10.0-5.fc35.noarch python3-pyparsing-2.4.7-9.fc35.noarch python3-pytest-6.2.5-1.fc36.noarch python3-pytest-cov-2.12.1-1.fc36.noarch python3-pytest-forked-1.3.0-4.fc35.noarch python3-pytest-xdist-2.4.0-1.fc36.noarch python3-setuptools-57.4.0-1.fc35.noarch python3-toml-0.10.2-5.fc35.noarch
On Sun, Sep 26, 2021 at 2:25 PM Miro Hrončok mhroncok@redhat.com wrote:
On 24. 09. 21 21:45, Ken Dreyer wrote:
This means that python-pytest-cov and python-pytest-xdist won't be available on epel9, since those require gevent.
Ignoring the rest of your email for now, but I don't the gevent dependency does not exsist:
You're right, gevent is not a runtime requirement. pytest-xdist requires execnet, but execnet only BuildRequires gevent for the unit tests. So we could build execnet on el9 if we disabled %check and the "execmodel=gevent" feature. Maybe that's fine, since https://github.com/pytest-dev/execnet is almost unmaintained upstream, and users have opened unaddressed GitHub tickets in execnet and xdist regarding gevent hangs.
This is another reason why I want to reduce Fedora's dependence on pytest-xdist and pytest-cov, if execnet is growing more unstable.
- Ken
epel-devel@lists.fedoraproject.org