On Sat, Jan 15, 2022 at 1:29 PM Orion Poplawski <orion(a)nwra.com> wrote:
On 1/14/22 05:02, Josh Boyer wrote:
> On Fri, Jan 14, 2022 at 5:31 AM Miro Hrončok <mhroncok(a)redhat.com> wrote:
>> On 14. 01. 22 5:11, Orion Poplawski wrote:
>>> While working on EPEL9, it seems that even more packages are missing from
>>> than were in RHEL8. The latest I found was cppunit, which appears to be
>>> completely missing from the CS9 repos despite having been built (See
>>> presumably in the CS9/RHEL9 buildroot.
>>> So I scraped some screens from pkgs.org
>>> Stream 9:
>>> CentOS AppStream Official x86_64 8882
>>> CentOS BaseOS Official x86_64 2357
>>> CentOS CRB Official x86_64 1856
>>> Stream 8:
>>> CentOS AppStream Official x86_64 15008
>>> CentOS BaseOS Official x86_64 6721
>>> CentOS PowerTools Official x86_64 3771
>>> That's a pretty big difference. Now, I don't know how many were
>>> completely and how many are of the "buildroot only" variety. But
>>> there is a fair amount of the latter and so a lot of make-work ahead of us
> A fairly accurate list of packages removed in RHEL 9 can be found in
> our RHEL 9 Adoption documentation:
Thanks for that.
>>> Packaging for EPEL9 is starting to feel more and more like cleaning the
>>> stables with RedHat piling up more manure.
>> If there is any Python stuff that's in Buildroot only, let me know and
>> my best to persuade the maintainers to move it to CRB (but my powers are
> I will politely point out two things.
> 1) The entirety of the RHEL buildroot is available in the CentOS
> Stream koji buildroots. I know the EPEL stewards have qualms about
> using it, but they are there and the technical hurdles to consume them
> are not large.
Well, we are making use of it via epel-next. But it's the "Stream"
buildroot. Once that diverges from the current released RHEL there
could be issues with trying to do updated builds for the current release
"...could be issues...". Yes, I agree it is possible to have issues.
I do not believe the number that may be found is large enough to
cripple EPEL or cause the community to have to request dozens of
-devel packages be added to product repos just to build things. It is
my hope that EPEL-next proves this out and EPEL proper can benefit
from it in a similar manner. I can appreciate the caution the EPEL
community is taking.
> 2) Moving content to CRB in RHEL is not a silver bullet solution
> many scenarios. If it's strictly for build dependencies, CRB works
> well. If an EPEL package has a runtime requires on CRB content, that
> is less desirable. RHEL CodeReady Linux Builder (CRB) content is
> unsupported, not enabled by default, and not intended to be used at
> runtime in production. EPEL itself is clearly in the same unsupported
> category, but requiring another unsupported repo at runtime may lead
> to unintentional surprises for many users.
I don't buy any of these arguments, and it doesn't really address the
situation of "missing -devel" packages. E.g. utf8proc - it's in
I'm sorry, I did not mean to intend an argumentative tone. What I
wrote is a factual statement from a product perspective, based on
feedback we've heard directly from some customers. It's ok for you to
not share the same view, of course. Feedback from some of our other
customers highlights they don't share it either.
"AppStream" - so it's presumably "supported"
and "intended to be used at
runtime in production". But without the -devel package available we
can't build anything in EPEL that uses it. So we have to go through
contortions to make sure we build the proper version of utf8proc-devel
and keep it in sync with RHEL.
This is difficult to describe, both in approach and in actual customer
facing documentation, even though we try. Content in RHEL is
generally supported, but the scope of that support can differ based on
a number of factors. There is the largest class of content supported
for general use with a full 10 year lifecycle and ABI stability
guarantees. However, there are also classes of content that are
supported for less than the full life of the RHEL release (Application
Streams, not to be conflated with the AppStream repo). Finally, there
are classes of content supported as internal to RHEL dependencies not
intended for wider use. The latter is a smaller set, and we continue
to evaluate the overall content set based on customer and user
Is the trade off of perhaps a few less RHEL support requests about a
packages that are clearly important enough to be included directly in
RHEL really worth the ill-will being generated in the volunteers that
help support the ecosystem around RHEL?
I can understand the frustration. EL7 had a more traditional
"distribution" approach where we dumped packages, almost casually,
into the Optional channel. That wound up being expensive, both for
RHEL and for our customers for a variety of reasons and a different
approach was taken in EL8.
Fortunately, we have a process in place that allows people to request
these packages. This is working, and we've expanded the content in
CRB significantly in RHEL 8 (and already 9!), as well as making
changes to make 100% of it available in the CentOS Stream project. As
the person that reviews literally every single request in some
capacity, I know this process is slow, but it does work. I continue
to look for ways to improve this, but it is not simple or quick.
Perhaps it's unreasonable for me to be as upset about this as I
largely it's because I just can't understand the motivation behind it
and it's a deliberate action that directly makes my *volunteer* work
harder. I have put in thousands of hours of work on Fedora/EPEL over
16+ years - and generally it has just gotten better. Better tools,
better infrastructure, etc. This is the first time I can remember
having something change that makes the work harder - so maybe it's just
the shock of that. Feels like a betrayal of those 16 years of working
together towards what seemed like a common goal.
That is certainly not the intention and your work, and the work of all
the other volunteers from upstream to Fedora to EPEL and CentOS, is
Oh, and for cppunit just putting it in CRB should be just fine as it
generally does not produce runtime deps.
I see that you have filed a request in bugzilla. Thank you.