Hello all,
Following up from last week's EPEL Steering Committee meeting, I'm presenting a proposal to create a dedicated SIG to make it easier to get Fedora packages into EPEL and keep them maintained.
Using the Fedora Changes Process template for this to help structure the proposal, though this is not really a Fedora Change.
== Summary ==
Have a dedicated SIG for packagers who are interested in making more Fedora packages available for EPEL releases.
== Owner == * Name: [[User:salimma|Michel Salim]] * Email: michel@michel-slm.name
== Detailed Description ==
RHEL/CentOS releases are branched off from Fedora; since RHEL 8 this happens every 3 years. Only a subset of Fedora packages get shipped as part of RHEL, as Red Hat provides support on these packages for their paying customers; EPEL (Extra Packages for Enterprise Linux) fills in the gap; this is similar to the old split between Fedora Core and Extras.
EPEL packages are maintained in dist-git as additional branches on Fedora packages; however, unlike with Fedora releases, where by default a package gets branched for any new Fedora release, EPEL branches are only created if one of the package maintainers request it (it's opt- in).
The rationale is that many Fedora packagers do not specifically care about EL, and with their long release cycles the maintenance burden is higher (e.g. upgrading to fix a security vulnerability might not be possible if the newer fixed version has unmet dependencies, so backporting the fix might be required). EL is more often used server side too, so the average Fedora packager is not expected to be an EL user.
This poses a problem for those of us who rely on packages in EPEL -- e.g. Fedora Infrastructure, and many corporate deployments. Right now the process is as such:
* An org starts rolling out the new version of EL * It turns out a given package is not available in EPEL * Bug filed requesting that package be branched and built * Worst case scenario, no response and we need to use the non- responsive maintainer flow * That package might have other unmet dependencies, so repeat this cycle several times * Wait for each of these packages to go through the update system * For organizations that have their own internal mirrors, they now need to sync the new packages
There are several changes we can make to both streamline the process, and not increase the maintenance burden on the (other) maintainers of these packages:
* Have an EPEL SIG modeled after the SIGs centered around programming language stacks (e.g. Python, Haskell, Java) * Have an expedited flow where this SIG can request EPEL branches and admin access to packages if there are no response from package maintainers for a set period (3 days? 1 week?) * whether it should be full admin access or whether such access should be scoped to epel* branches can be discussed. Full admin would make it possible to adjust the spec in Rawhide to be more EPEL friendly, for example * Members of the EPEL SIG can then branch these packages early when the next EL release is branched * The member of the SIG doing the branching should be designated as the primary EPEL assignee for that package
One deviation we might want to have from how Python SIG works is... we probably don't want to encourage packagers to add this EPEL SIG as a comaintainer preemptively, but only as needed.
== Benefit to Fedora == === Smoother infra upgrades === Upgrading the Fedora infrastructure will get easier over time, as more of the necessary EPEL packages become co-maintained by the EPEL SIG, which reduces the amount of time needed to get these packages available on new releases.
=== Packager experience === Reduced burden on Fedora package maintainers who are not interested in EPEL - the choice is now either: * One of the existing maintainers does the work of branching and building * The requestee gets added as a maintainer and does it * The request stalls because the requestee is not a packager
=== More involvement by CentOS-using organizations === With the EPEL SIG, we should encourage organizations with a long-term need for EPEL to get their members / employees added to this SIG, so scenarios #2 and #3 basically becomes this:
* EPEL SIG gets added as co-maintainer of this package
This might encourage more contributions from companies that traditionally just consume RHEL/CentOS + EPEL, and as RHEL/CentOS is increasingly developed in the open with CentOS-Stream, eventually also make it easier to get these companies' input into the EL development process.
== Scope ==
* Proposal Owners * Ask Infra to set up EPEL SIG ACL and mailing list (for Bugzilla) * Announce this in epel-devel * Have documentation for the SIG as part of the EPEL documentation * Help onboard people to this SIG
On Fri, Sep 11, 2020 at 11:52:03AM -0700, Michel Alexandre Salim wrote: ...snip...
EPEL packages are maintained in dist-git as additional branches on Fedora packages; however, unlike with Fedora releases, where by default a package gets branched for any new Fedora release, EPEL branches are only created if one of the package maintainers request it (it's opt- in).
The rationale is that many Fedora packagers do not specifically care about EL, and with their long release cycles the maintenance burden is higher (e.g. upgrading to fix a security vulnerability might not be possible if the newer fixed version has unmet dependencies, so backporting the fix might be required). EL is more often used server side too, so the average Fedora packager is not expected to be an EL user.
I'll add that in addtion to some maintainers not wanting to maintain their fedora packages also in epel, the timelines involved sometimes make it so a package that was branched/maintained in epelX, makes no sense in epelY. ie, Xfce 4.14 would be fine now, but in 3 years, 4.16 or whatever would make more sense and the package set may well not be the same.
This poses a problem for those of us who rely on packages in EPEL -- e.g. Fedora Infrastructure, and many corporate deployments. Right now the process is as such:
- An org starts rolling out the new version of EL
- It turns out a given package is not available in EPEL
- Bug filed requesting that package be branched and built
- Worst case scenario, no response and we need to use the non-
responsive maintainer flow
- That package might have other unmet dependencies, so repeat this
cycle several times
- Wait for each of these packages to go through the update system
- For organizations that have their own internal mirrors, they now need
to sync the new packages
I wonder... could we somehow let maintainers indicate "I want my package branched for the next epel as soon as it's available"? I suppose in part thats what adding epel-sig would do? With the addition of 'epel-sig should build by packages as soon as those branches are ready'.
There are several changes we can make to both streamline the process, and not increase the maintenance burden on the (other) maintainers of these packages:
- Have an EPEL SIG modeled after the SIGs centered around programming
language stacks (e.g. Python, Haskell, Java)
- Have an expedited flow where this SIG can request EPEL branches and
admin access to packages if there are no response from package maintainers for a set period (3 days? 1 week?)
Both of those are really short... I guess a week or two might be ok.
- whether it should be full admin access or whether such access
should be scoped to epel* branches can be discussed. Full admin would make it possible to adjust the spec in Rawhide to be more EPEL friendly, for example
- Members of the EPEL SIG can then branch these packages early when the
next EL release is branched
- The member of the SIG doing the branching should be designated as the
primary EPEL assignee for that package
How would that be designated?
One deviation we might want to have from how Python SIG works is... we probably don't want to encourage packagers to add this EPEL SIG as a comaintainer preemptively, but only as needed.
That would defeat the purpose of using epel-sig packages as 'always branch' wouldn't it?
The other hazard if that different maintainers have different workflows and epel-sig folks would need to adjust to those. ie, some people want master to have epel conditionals and merge back to epel branches, some don't want that at all. I wonder if it wouldn't make sense to have some way to indicate to people what workflow is in effect for the package (I guess spec file comments?)
kevin
On Mon, 2020-09-14 at 08:54 -0700, Kevin Fenzi wrote: ...snip...
I'll add that in addtion to some maintainers not wanting to maintain their fedora packages also in epel, the timelines involved sometimes make it so a package that was branched/maintained in epelX, makes no sense in epelY. ie, Xfce 4.14 would be fine now, but in 3 years, 4.16 or whatever would make more sense and the package set may well not be the same.
Right, that's also a good reason for why branching by default is not a good option for all packages.
I wonder... could we somehow let maintainers indicate "I want my package branched for the next epel as soon as it's available"? I suppose in part thats what adding epel-sig would do? With the addition of 'epel-sig should build by packages as soon as those branches are ready'.
I wonder if these are two separate concerns though? I agree that being able to indicate a package should always be branched would be great, but... epel-sig / epel-wranglers might not find a package relevant in a new EL release (e.g. say we care about neovim, and want to carry it by default in new releases, and thus we also care about some of its dependencies that are not in (RH)EL core -- but the set of dependencies we care about in EPEL7 != the set in EPEL8 etc.
^ if that seems explicit that's because it, uh, is from personal experience.
Maybe package.cfg might be where such a metadata live? e.g. if the epel8 branch of the package has a package.cfg that says "branch for next release" it gets branched for epel9 -- and inherits that package.cfg by default. If a package gets opted in once and at some point is no longer needed in the next epel, just delete that.
This might also suggest we want to have a delay before automatically branching so EPEL SIG / Wranglers have time to adjust that package.cfg after testing the next EL beta.
There are several changes we can make to both streamline the process, and not increase the maintenance burden on the (other) maintainers of these packages:
- Have an EPEL SIG modeled after the SIGs centered around
programming language stacks (e.g. Python, Haskell, Java)
- Have an expedited flow where this SIG can request EPEL branches
and admin access to packages if there are no response from package maintainers for a set period (3 days? 1 week?)
Both of those are really short... I guess a week or two might be ok.
1-2 weeks is probably fine, yes, esp. as over time the need for such manual requests should go down.
- whether it should be full admin access or whether such access
should be scoped to epel* branches can be discussed. Full admin would make it possible to adjust the spec in Rawhide to be more EPEL friendly, for example
- Members of the EPEL SIG can then branch these packages early when
the next EL release is branched
- The member of the SIG doing the branching should be designated as
the primary EPEL assignee for that package
How would that be designated?
Hmm. So right now (correct me if I'm wrong), it seems that only the main admin for a package can override the bugzilla assignee?
I'm not sure about how this part would work yet. If at the beginning we try this out with manual infra ticket, I could imagine the EPEL SIG member that requests EPEL-SIG comaintainership for a package should ask the main admin to make them the EPEL assignee, or if it goes through an infra ticket, ask for infra to make that change?
One deviation we might want to have from how Python SIG works is... we probably don't want to encourage packagers to add this EPEL SIG as a comaintainer preemptively, but only as needed.
That would defeat the purpose of using epel-sig packages as 'always branch' wouldn't it?
Right - I wasn't thinking of the 'always branch' case (once EPEL-SIG has commit access requesting a new branch is not a significant delay anyway). So 'branch for next' in package.cfg might be a better solution all around.
How much tooling change would we need to get collaborators the ability to request branches if the branch name matches their whitelist, incidentally?
The other hazard if that different maintainers have different workflows and epel-sig folks would need to adjust to those. ie, some people want master to have epel conditionals and merge back to epel branches, some don't want that at all. I wonder if it wouldn't make sense to have some way to indicate to people what workflow is in effect for the package (I guess spec file comments?)
Maybe spec file comments as well as only granting collaborator (whitelisting epel* branches) instead of commit / admin access if they don't want EPEL conditionals?
Best regards,
On Mon, Sep 14, 2020 at 09:42:36AM -0700, Michel Alexandre Salim wrote:
I wonder if these are two separate concerns though? I agree that being able to indicate a package should always be branched would be great, but... epel-sig / epel-wranglers might not find a package relevant in a new EL release (e.g. say we care about neovim, and want to carry it by default in new releases, and thus we also care about some of its dependencies that are not in (RH)EL core -- but the set of dependencies we care about in EPEL7 != the set in EPEL8 etc.
^ if that seems explicit that's because it, uh, is from personal experience.
Yeah, I agree, not everything will want to auto branch... and in fact this will change from time to time as new versions come out, things are retired, etc.
Maybe package.cfg might be where such a metadata live? e.g. if the epel8 branch of the package has a package.cfg that says "branch for next release" it gets branched for epel9 -- and inherits that package.cfg by default. If a package gets opted in once and at some point is no longer needed in the next epel, just delete that.
I don't like that idea... I've heard many many complaints from people about package.cfg files and them messing up merges.
I would think at the pagure/src.fp.o level might be better... or if not, then just a simple 'alwaysepelbranch' file or something thats ignored except for this use.
This might also suggest we want to have a delay before automatically branching so EPEL SIG / Wranglers have time to adjust that package.cfg after testing the next EL beta.
Sure, should be a deliberate process.
....snip....
Hmm. So right now (correct me if I'm wrong), it seems that only the main admin for a package can override the bugzilla assignee?
yeah, I think thats indeed the case.
I'm not sure about how this part would work yet. If at the beginning we try this out with manual infra ticket, I could imagine the EPEL SIG member that requests EPEL-SIG comaintainership for a package should ask the main admin to make them the EPEL assignee, or if it goes through an infra ticket, ask for infra to make that change?
I suppose. I am not keen on adding more infrastructure tickets. If we can do a workflow that consolidates requests/can be automated that would be much better IMHO.
One deviation we might want to have from how Python SIG works is... we probably don't want to encourage packagers to add this EPEL SIG as a comaintainer preemptively, but only as needed.
That would defeat the purpose of using epel-sig packages as 'always branch' wouldn't it?
Right - I wasn't thinking of the 'always branch' case (once EPEL-SIG has commit access requesting a new branch is not a significant delay anyway). So 'branch for next' in package.cfg might be a better solution all around.
How much tooling change would we need to get collaborators the ability to request branches if the branch name matches their whitelist, incidentally?
Not sure. Thats a question for Pingou. :)
The other hazard if that different maintainers have different workflows and epel-sig folks would need to adjust to those. ie, some people want master to have epel conditionals and merge back to epel branches, some don't want that at all. I wonder if it wouldn't make sense to have some way to indicate to people what workflow is in effect for the package (I guess spec file comments?)
Maybe spec file comments as well as only granting collaborator (whitelisting epel* branches) instead of commit / admin access if they don't want EPEL conditionals?
yeah.
kevin
Hello Alexandre!
On 11.09.20 20:52, Michel Alexandre Salim wrote:
The rationale is that many Fedora packagers do not specifically care about EL, and with their long release cycles the maintenance burden is higher (e.g. upgrading to fix a security vulnerability might not be possible if the newer fixed version has unmet dependencies, so backporting the fix might be required). EL is more often used server side too, so the average Fedora packager is not expected to be an EL user.
I fully agree with the idea of an EPEL packaging SIG.
But I would like to extend your rational: I was doing a Fedora desktop install (KDE) for some not technical affine people. After about 5 years I no more think this was a good idea, because every half year I have to make a full upgrade.
So I now plan to change to CentOS (+ EPEL). IMO this is the better choice for non technicians if the installation should last for years.
Is it OK to call CentOS (+ EPEL) "Fedora LTS"? ;-)
But there are still some packages missing.
So I started to became a Fedora packager and I am now in the process of being sponsored. I started with the simple packages "qjackctl" and "qsynth".
Thank you for you effort.
Best Regards Christoph (pampelmuse)
epel-devel@lists.fedoraproject.org
-
Christoph Karl -
Kevin Fenzi -
Michel Alexandre Salim