So in todays (2014-08-29) meeting, we wanted to move the various policy discussions to email so that people could take their time to reply and also to allow for people who could not attend time to respond.
Going from the web-page https://fedoraproject.org/wiki/EPEL-faster-repo-ideas the following items are listed (with additions I added while writing this email.)
I would like to start with the EpSCO governance to just get that out of the way and then move to current rules as we see them. I can act as secretary to make sure that changes discussed here are put in place on the wiki or other places.
Policy questions
- EpSCO governance. - Lifetime of initial committee (9 months?) - Replacement of any exiting members (replacement by formal vote, etc). - Size of committee (4 members? 5 members?) - Meeting rules (follow https://fedoraproject.org/wiki/Fedora_Engineering_Steering_Committee )
- How many repos? (examples below) - epel, - epel-test - epel-rolling - epel-rolling-test ? - epel-edge ? - epel-edge-test ?
- What would faster moving mean?
- Would packages in this be able to conflict with epel packages? Base packages?
- When would incompatible changes be allowed in each branch?
- Different guidelines for specs/packages per branch?
- When would a package be expired or removed?
- What are the rules for EPEL packages currently?
- Currently packages require of 2 weeks in EPEL-testing before promotion to EPEL. - Can this be changed to 1 week? - What Constant Integration would be needed to give auto-karma?
- How to integrate packages shepherded by CentOS (or other SIGS) which may conflict with EPEL packages?
On Fri, 29 Aug 2014 16:08:34 -0600 Stephen John Smoogen smooge@gmail.com wrote:
So in todays (2014-08-29) meeting, we wanted to move the various policy discussions to email so that people could take their time to reply and also to allow for people who could not attend time to respond.
Right.
Going from the web-page https://fedoraproject.org/wiki/EPEL-faster-repo-ideas the following items are listed (with additions I added while writing this email.)
I would like to start with the EpSCO governance to just get that out of the way and then move to current rules as we see them. I can act as secretary to make sure that changes discussed here are put in place on the wiki or other places.
Policy questions
- EpSCO governance.
- Lifetime of initial committee (9 months?)
- Replacement of any exiting members (replacement by formal
vote, etc). - Size of committee (4 members? 5 members?) - Meeting rules (follow https://fedoraproject.org/wiki/Fedora_Engineering_Steering_Committee )
I'm happy with whatever other people want. I think making things super formal just causes more problems. I think we agreed on 4 members for now, we can add by vote of those folks, we should try and meet weekly.
- How many repos? (examples below)
- epel,
- epel-test
- epel-rolling
- epel-rolling-test ?
- epel-edge ?
- epel-edge-test ?
There's some thought from folks about doing a per point release repos. I added some comments asking for more details on that idea.
If repos we add as fast moving/whatever they may not need/want testing repos.
- What would faster moving mean?
Good question. Perhaps folks could list out some example packages and stacks that aren't happy with epel as it is and we could better come up with use cases to help those?
- Would packages in this be able to conflict with epel packages?
Base packages?
When would incompatible changes be allowed in each branch?
Different guidelines for specs/packages per branch?
I would REALLY like to avoid this. EPEL leverages the excellent (IMHO) Fedora guidelines. If we go off and do our own thing we are doing our users a disservice.
When would a package be expired or removed?
What are the rules for EPEL packages currently?
All Fedora guidelines + https://fedoraproject.org/wiki/EPEL:Packaging
- Currently packages require of 2 weeks in EPEL-testing before
promotion to EPEL. - Can this be changed to 1 week?
I guess I'd be ok moving it to a week.
- What Constant Integration would be needed to give auto-karma?
- How to integrate packages shepherded by CentOS (or other SIGS)
which may conflict with EPEL packages?
kevin
On 09/04/2014 11:13 AM, Kevin Fenzi wrote:
On Fri, 29 Aug 2014 16:08:34 -0600 Stephen John Smoogen smooge@gmail.com wrote:
So in todays (2014-08-29) meeting, we wanted to move the various policy discussions to email so that people could take their time to reply and also to allow for people who could not attend time to respond.
Right.
Going from the web-page https://fedoraproject.org/wiki/EPEL-faster-repo-ideas the following items are listed (with additions I added while writing this email.)
I would like to start with the EpSCO governance to just get that out of the way and then move to current rules as we see them. I can act as secretary to make sure that changes discussed here are put in place on the wiki or other places.
Policy questions
- EpSCO governance.
- Lifetime of initial committee (9 months?)
- Replacement of any exiting members (replacement by formal
vote, etc). - Size of committee (4 members? 5 members?) - Meeting rules (follow https://fedoraproject.org/wiki/Fedora_Engineering_Steering_Committee )
I'm happy with whatever other people want. I think making things super formal just causes more problems. I think we agreed on 4 members for now, we can add by vote of those folks, we should try and meet weekly.
+1 here as well. I don't see a need to make it overly rigid.
- How many repos? (examples below)
- epel,
- epel-test
- epel-rolling
- epel-rolling-test ?
- epel-edge ?
- epel-edge-test ?
There's some thought from folks about doing a per point release repos. I added some comments asking for more details on that idea.
Without further info on this, my gut reaction is to NOT do point releases. I could be persuaded if there's a strong enough benefit.
If repos we add as fast moving/whatever they may not need/want testing repos.
- What would faster moving mean?
Good question. Perhaps folks could list out some example packages and stacks that aren't happy with epel as it is and we could better come up with use cases to help those?
Within the CentOS SIG framework, most of the 'newer' packages would likely be for things like php, perl, python, libvirt. Example, the ceph and ovirt teams would both benefit from having a newer libvirt available to them.
- Would packages in this be able to conflict with epel packages?
Base packages?
When would incompatible changes be allowed in each branch?
Different guidelines for specs/packages per branch?
I would REALLY like to avoid this. EPEL leverages the excellent (IMHO) Fedora guidelines. If we go off and do our own thing we are doing our users a disservice.
While this is true, look at what happened with puppet for EL6 as an example. EPEL couldn't move beyond 2.6 to avoid compatibility changes, but that introduced some security issues in addition to the usual user cries for current/recent versions.
Perhaps this is an issue where a shorter life-cycle expectation needs to be set?
When would a package be expired or removed?
What are the rules for EPEL packages currently?
All Fedora guidelines + https://fedoraproject.org/wiki/EPEL:Packaging
- Currently packages require of 2 weeks in EPEL-testing before
promotion to EPEL. - Can this be changed to 1 week?
I guess I'd be ok moving it to a week.
I'd be okay with this as well, especially if there's automated testing scripts provided
- What Constant Integration would be needed to give auto-karma?
- How to integrate packages shepherded by CentOS (or other SIGS)
which may conflict with EPEL packages?
This is a good question. We need to look at both the technical and policy guidelines around this.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/04/2014 02:05 PM, Jim Perrin wrote:
Perhaps this is an issue where a shorter life-cycle expectation needs to be set?
For anything related to the new repo, this is an important part of the idea. We want to set a new expectation with the new brand. Maybe we can just tie it to the upstream-of-Fedora -- 13 months, rolls with each Fedora release, etc.?
- - Karsten - -- Karsten 'quaid' Wade .^\ CentOS Doer of Stuff http://TheOpenSourceWay.org \ http://community.redhat.com @quaid (identi.ca/twitter/IRC) \v' gpg: AD0E0C41
On 09/05/2014 12:49 AM, Karsten Wade wrote:
For anything related to the new repo, this is an important part of the idea. We want to set a new expectation with the new brand. Maybe we can just tie it to the upstream-of-Fedora -- 13 months, rolls with each Fedora release, etc.?
It would look saner to me to tie it to the minor release cycle of RHEL. Fedora is in now way involved here ( short of being the source of the packages ). BUT as we want to decouple the repos ( i.e. base+updates from the rest of the ecosystem ) and allow a shorter cycle for 3rd parties, I would not tie anything to the cycle of RHEL either. We need to allow individual SIGs to be able to push $STUFF in their own cycle
My .02 eurocents ( add 30% for US currency )
manuel
On 4 September 2014 10:13, Kevin Fenzi kevin@scrye.com wrote:
On Fri, 29 Aug 2014 16:08:34 -0600 Stephen John Smoogen smooge@gmail.com wrote:
So in todays (2014-08-29) meeting, we wanted to move the various policy discussions to email so that people could take their time to reply and also to allow for people who could not attend time to respond.
Right.
Going from the web-page https://fedoraproject.org/wiki/EPEL-faster-repo-ideas the following items are listed (with additions I added while writing this email.)
I would like to start with the EpSCO governance to just get that out of the way and then move to current rules as we see them. I can act as secretary to make sure that changes discussed here are put in place on the wiki or other places.
Policy questions
- EpSCO governance.
- Lifetime of initial committee (9 months?)
- Replacement of any exiting members (replacement by formal
vote, etc). - Size of committee (4 members? 5 members?) - Meeting rules (follow
https://fedoraproject.org/wiki/Fedora_Engineering_Steering_Committee
)I'm happy with whatever other people want. I think making things super formal just causes more problems. I think we agreed on 4 members for now, we can add by vote of those folks, we should try and meet weekly.
- How many repos? (examples below)
- epel,
- epel-test
- epel-rolling
- epel-rolling-test ?
- epel-edge ?
- epel-edge-test ?
There's some thought from folks about doing a per point release repos. I added some comments asking for more details on that idea.
If repos we add as fast moving/whatever they may not need/want testing repos.
So this is my general proposal for per point release.
Problem trying to be solved:
EPEL's original goal around a 5-7 year product has run into issues where various packages end up having shorter lifetimes than can be maintained in that period. What happens is that the upstream is changing the code too massively for any sort of 'back-port' to be possible. Some software can be 'patched' around by making it parallel installable but others (say puppet) only work if there is only one version not just on the system but throughout an entire ecosystem of systems (thus if you update one, they all have to be updated to the same version.) While EPEL has a process of saying you can announce a major change it is severely frowned on and usually ends up with various users asking 'can you just keep the older version around a bit longer...' ad infinitum. Another problem is that it isn't clear how much notice is needed for a change to be made or what changes are being made so that users and package owners end up confused and upset with each other.
Possible solution:
Make it so that updates to these sorts of packages occur at regular intervals. There are three cycles these could occur: regular date driven (July 1st of every year for example), the Fedora release cycle (Fedora 22 has been released, you can update now), and the Red Hat Enterprise release cycle (RHEL-6.6 is out, you can release now). Each of these cycles has their bonuses and minuses but I think that connecting to the RHEL cycle will be more in line with what downstream users of EPEL are going to be more in sync with.
Proposed implementation:
Every Red Hat Enterprise Dot Release has a beta period and then a release period. After the release period there is a couple of weeks until a CentOS release is available. Since only the betas and CentOS release are generally available to any person off the street I am looking at those being 'action points' where the dot release period would be done. When a RHEL beta dot release (example RHEL-6.6 beta) is announced, EPEL.dot would say that packages to be updated in the next release are going to be accepted into EPEL-dot-testing. Package owners who are needing or wanting to make major changes in their EPEL package sets would target builds to that channel. People can test as needed or at least be aware where they could have tested. Sometime after the next dot release, CentOS releases their latest 'this is the state of CentOS build' which anyone needing to test latest builds can get. This acts as a marker for when EPEL.dot will 'release' its next tree which would be 1 month after the CentOS GA. During that time, everything is rebuilt against the latest RHEL (to find any FTBS or ooops they dropped libmuffin and I needed it) and at the end of the month EPEL-6.7 is put out and EPEL-6.6 is set to be sent to archives after an appropriate time. Like CentOS, EPEL-6 -> EPEL-6.6 and then EPEL-6.7.
In the case of last planned release (RHEL-5.12?), a different mechanism could be used for when updates occur if there is enough people willing to do the work. Otherwise packages go into that tree until EPEL ends building for it.
Note only one tree is being built against. If someone wants to 'keep' EPEL-6.5 running, they can grab the src.rpms from archives and do it themselves on their own hardware.. EPEL only deals with RHEL current.
Have I made this clearer or muddier?
On Sep 04 18:20, Stephen John Smoogen wrote:
So this is my general proposal for per point release.
Problem trying to be solved:
EPEL's original goal around a 5-7 year product has run into issues where various packages end up having shorter lifetimes than can be maintained in that period. What happens is that the upstream is changing the code too massively for any sort of 'back-port' to be possible. Some software can be 'patched' around by making it parallel installable but others (say puppet) only work if there is only one version not just on the system but throughout an entire ecosystem of systems (thus if you update one, they all have to be updated to the same version.) While EPEL has a process of saying you can announce a major change it is severely frowned on and usually ends up with various users asking 'can you just keep the older version around a bit longer...' ad infinitum. Another problem is that it isn't clear how much notice is needed for a change to be made or what changes are being made so that users and package owners end up confused and upset with each other.
Possible solution:
Make it so that updates to these sorts of packages occur at regular intervals. There are three cycles these could occur: regular date driven (July 1st of every year for example), the Fedora release cycle (Fedora 22 has been released, you can update now), and the Red Hat Enterprise release cycle (RHEL-6.6 is out, you can release now). Each of these cycles has their bonuses and minuses but I think that connecting to the RHEL cycle will be more in line with what downstream users of EPEL are going to be more in sync with.
Proposed implementation:
Every Red Hat Enterprise Dot Release has a beta period and then a release period. After the release period there is a couple of weeks until a CentOS release is available. Since only the betas and CentOS release are generally available to any person off the street I am looking at those being 'action points' where the dot release period would be done. When a RHEL beta dot release (example RHEL-6.6 beta) is announced, EPEL.dot would say that packages to be updated in the next release are going to be accepted into EPEL-dot-testing. Package owners who are needing or wanting to make major changes in their EPEL package sets would target builds to that channel. People can test as needed or at least be aware where they could have tested. Sometime after the next dot release, CentOS releases their latest 'this is the state of CentOS build' which anyone needing to test latest builds can get. This acts as a marker for when EPEL.dot will 'release' its next tree which would be 1 month after the CentOS GA. During that time, everything is rebuilt against the latest RHEL (to find any FTBS or ooops they dropped libmuffin and I needed it) and at the end of the month EPEL-6.7 is put out and EPEL-6.6 is set to be sent to archives after an appropriate time. Like CentOS, EPEL-6 -> EPEL-6.6 and then EPEL-6.7.
In the case of last planned release (RHEL-5.12?), a different mechanism could be used for when updates occur if there is enough people willing to do the work. Otherwise packages go into that tree until EPEL ends building for it.
Note only one tree is being built against. If someone wants to 'keep' EPEL-6.5 running, they can grab the src.rpms from archives and do it themselves on their own hardware.. EPEL only deals with RHEL current.
Have I made this clearer or muddier?
-- Stephen J Smoogen.
I think this fits in well with the release calendar(s) that EPEL consumers are already working with. Not sure what other feedback I can give, except that I like it.
Brian
-- Brian Stinson bstinson@ksu.edu | IRC: bstinson | Bitbucket/Twitter: bstinsonmhk
On Thu, 4 Sep 2014 18:20:19 -0600 Stephen John Smoogen smooge@gmail.com wrote:
So this is my general proposal for per point release.
Problem trying to be solved:
EPEL's original goal around a 5-7 year product has run into issues where various packages end up having shorter lifetimes than can be maintained in that period. What happens is that the upstream is changing the code too massively for any sort of 'back-port' to be possible. Some software can be 'patched' around by making it parallel installable but others (say puppet) only work if there is only one version not just on the system but throughout an entire ecosystem of systems (thus if you update one, they all have to be updated to the same version.) While EPEL has a process of saying you can announce a major change it is severely frowned on and usually ends up with various users asking 'can you just keep the older version around a bit longer...' ad infinitum. Another problem is that it isn't clear how much notice is needed for a change to be made or what changes are being made so that users and package owners end up confused and upset with each other.
Possible solution:
Make it so that updates to these sorts of packages occur at regular intervals. There are three cycles these could occur: regular date driven (July 1st of every year for example), the Fedora release cycle (Fedora 22 has been released, you can update now), and the Red Hat Enterprise release cycle (RHEL-6.6 is out, you can release now). Each of these cycles has their bonuses and minuses but I think that connecting to the RHEL cycle will be more in line with what downstream users of EPEL are going to be more in sync with.
ok, so to be clear this is a change in the existing EPEL repo(s)? Or is this new seperate repos?
Proposed implementation:
Every Red Hat Enterprise Dot Release has a beta period and then a release period. After the release period there is a couple of weeks until a CentOS release is available. Since only the betas and CentOS release are generally available to any person off the street I am looking at those being 'action points' where the dot release period would be done. When a RHEL beta dot release (example RHEL-6.6 beta) is announced, EPEL.dot would say that packages to be updated in the next release are going to be accepted into EPEL-dot-testing. Package owners who are needing or wanting to make major changes in their EPEL package sets would target builds to that channel. People can test as needed or at least be aware where they could have tested. Sometime after the next dot release, CentOS releases their latest 'this is the state of CentOS build' which anyone needing to test latest builds can get. This acts as a marker for when EPEL.dot will 'release' its next tree which would be 1 month after the CentOS GA.
The problem with that IMHO is that a month after people have updated, they have forgotten about this, and get surprised when a chud of incompatible changes land. ;(
If this is a seperate repo they opt into, we could try and manage expectations for that tho.
During that time, everything is rebuilt against the latest RHEL (to find any FTBS or ooops they dropped libmuffin and I needed it) and at the end of the month EPEL-6.7 is put out and EPEL-6.6 is set to be sent to archives after an appropriate time. Like CentOS, EPEL-6 -> EPEL-6.6 and then EPEL-6.7.
In the case of last planned release (RHEL-5.12?), a different mechanism could be used for when updates occur if there is enough people willing to do the work. Otherwise packages go into that tree until EPEL ends building for it.
Note only one tree is being built against. If someone wants to 'keep' EPEL-6.5 running, they can grab the src.rpms from archives and do it themselves on their own hardware.. EPEL only deals with RHEL current.
Have I made this clearer or muddier?
Some clearer. :)
kevin
On 5 September 2014 12:52, Kevin Fenzi kevin@scrye.com wrote:
On Thu, 4 Sep 2014 18:20:19 -0600 Stephen John Smoogen smooge@gmail.com wrote:
So this is my general proposal for per point release.
Problem trying to be solved:
EPEL's original goal around a 5-7 year product has run into issues where various packages end up having shorter lifetimes than can be maintained in that period. What happens is that the upstream is changing the code too massively for any sort of 'back-port' to be possible. Some software can be 'patched' around by making it parallel installable but others (say puppet) only work if there is only one version not just on the system but throughout an entire ecosystem of systems (thus if you update one, they all have to be updated to the same version.) While EPEL has a process of saying you can announce a major change it is severely frowned on and usually ends up with various users asking 'can you just keep the older version around a bit longer...' ad infinitum. Another problem is that it isn't clear how much notice is needed for a change to be made or what changes are being made so that users and package owners end up confused and upset with each other.
Possible solution:
Make it so that updates to these sorts of packages occur at regular intervals. There are three cycles these could occur: regular date driven (July 1st of every year for example), the Fedora release cycle (Fedora 22 has been released, you can update now), and the Red Hat Enterprise release cycle (RHEL-6.6 is out, you can release now). Each of these cycles has their bonuses and minuses but I think that connecting to the RHEL cycle will be more in line with what downstream users of EPEL are going to be more in sync with.
ok, so to be clear this is a change in the existing EPEL repo(s)? Or is this new seperate repos?
This would be a new seperate repo. The current Repositories are set up and have 6+ years of expectations on them. Trying to make changes to those expectations is not possible. I tried calling the seperate repo EPEL.dot to make sure it was clear it was seperate (and when I called it EPIC people didn't like that either).
Proposed implementation:
Every Red Hat Enterprise Dot Release has a beta period and then a release period. After the release period there is a couple of weeks until a CentOS release is available. Since only the betas and CentOS release are generally available to any person off the street I am looking at those being 'action points' where the dot release period would be done. When a RHEL beta dot release (example RHEL-6.6 beta) is announced, EPEL.dot would say that packages to be updated in the next release are going to be accepted into EPEL-dot-testing. Package owners who are needing or wanting to make major changes in their EPEL package sets would target builds to that channel. People can test as needed or at least be aware where they could have tested. Sometime after the next dot release, CentOS releases their latest 'this is the state of CentOS build' which anyone needing to test latest builds can get. This acts as a marker for when EPEL.dot will 'release' its next tree which would be 1 month after the CentOS GA.
The problem with that IMHO is that a month after people have updated, they have forgotten about this, and get surprised when a chud of incompatible changes land. ;(
I agree, if I were to do this with straight EPEL it would be a problem.
Problem trying to be solved:
EPEL's original goal around a 5-7 year product has run into issues where various packages end up having shorter lifetimes than can be maintained in that period. What happens is that the upstream is changing the code too massively for any sort of 'back-port' to be possible. Some software can be 'patched' around by making it parallel installable but others (say puppet) only work if there is only one version not just on the system but throughout an entire ecosystem of systems (thus if you update one, they all have to be updated to the same version.) While EPEL has a process of saying you can announce a major change it is severely frowned on and usually ends up with various users asking 'can you just keep the older version around a bit longer...' ad infinitum. Another problem is that it isn't clear how much notice is needed for a change to be made or what changes are being made so that users and package owners end up confused and upset with each other.
Possible solution:
Make it so that updates to these sorts of packages occur at regular intervals. There are three cycles these could occur: regular date driven (July 1st of every year for example), the Fedora release cycle (Fedora 22 has been released, you can update now), and the Red Hat Enterprise release cycle (RHEL-6.6 is out, you can release now). Each of these cycles has their bonuses and minuses but I think that connecting to the RHEL cycle will be more in line with what downstream users of EPEL are going to be more in sync with.
Proposed implementation:
Every Red Hat Enterprise Dot Release has a beta period and then a release period. After the release period there is a couple of weeks until a CentOS release is available. Since only the betas and CentOS release are generally available to any person off the street I am looking at those being 'action points' where the dot release period would be done. When a RHEL beta dot release (example RHEL-6.6 beta) is announced, EPEL.dot would say that packages to be updated in the next release are going to be accepted into EPEL-dot-testing. Package owners who are needing or wanting to make major changes in their EPEL package sets would target builds to that channel. People can test as needed or at least be aware where they could have tested. Sometime after the next dot release, CentOS releases their latest 'this is the state of CentOS build' which anyone needing to test latest builds can get. This acts as a marker for when EPEL.dot will 'release' its next tree which would be 1 month after the CentOS GA. During that time, everything is rebuilt against the latest RHEL (to find any FTBS or ooops they dropped libmuffin and I needed it) and at the end of the month EPEL-6.7 is put out and EPEL-6.6 is set to be sent to archives after an appropriate time. Like CentOS, EPEL-6 -> EPEL-6.6 and then EPEL-6.7.
In the case of last planned release (RHEL-5.12?), a different mechanism could be used for when updates occur if there is enough people willing to do the work. Otherwise packages go into that tree until EPEL ends building for it.
Note only one tree is being built against. If someone wants to 'keep' EPEL-6.5 running, they can grab the src.rpms from archives and do it themselves on their own hardware.. EPEL only deals with RHEL current.
Have I made this clearer or muddier?
Thanks for starting this initiative and sorry for my late response. Anyway, my 2 cents: As maintainer of python3 in Fedora, I've already been asked by quite a lot of people to put python3 in EPEL 7. Most notably by DNF maintainer and people from Fedora Infrastructure. My problem with doing this is, that I want to able to actually update python3 point releases (e.g. from 3.4 to 3.5) at some points during EPEL 7 life. So I'm trying to understand how this proposal would help me. Here are my notes and thoughts on the situtation with python3 and the possible solution: - Generally, python3 point releases can't be safely considered backwards compatible and all depending packages need to be rebuilt. - The above point I think makes updating python3 go against current EPEL policies. If I wanted to do it "right", I would need to start with python34, then after some time add python35, python36 etc. (for each of these, I'd need to keep the whole stack of dependent libraries, which brings even more iffy problems, like "where should these packages be at dist-git", "should these conflict or be installable in parallel", "when/how do we obsolete the old versions", ...) - So I'm wondering whether we could have just "python3" and ("python3-*" extension modules and other dependent packages). The "python3" package itself would be updated at time of RHEL point releases, assuming a new minor version has been released prior to the update period (RHEL beta). - Now, I'm not sure how this should work with extension modules and other packages depending on python3. Would it make sense for them to live in the main repo, assuming they'd depend on a package that lives in the "EPEL-dot" repo? I think not, but this would basically cause all python3 depending packages to live in "EPEL-dot". I'm not sure whether that is a problem or not. It doesn't feel exactly right, but I guess it's the right thing to do... - So basically the whole python3 stack would need to be rebuilt together during the RHEL beta period, which might be very short, considering that we may be talking about hundreds of packages with dependency cycles etc. In case that the rebuild wouldn't be completed on time, I think it should be safe to just wait for the next RHEL dot release.
Does this make sense? Comments/suggestions?
Thanks a lot! Slavek
-- Stephen J Smoogen.
On 11 September 2014 08:41, Bohuslav Kabrda bkabrda@redhat.com wrote:
Problem trying to be solved:
EPEL's original goal around a 5-7 year product has run into issues where various packages end up having shorter lifetimes than can be maintained in that period. What happens is that the upstream is changing the code too massively for any sort of 'back-port' to be possible. Some software can be 'patched' around by making it parallel installable but others (say puppet) only work if there is only one version not just on the system but throughout an entire ecosystem of systems (thus if you update one, they all have to be updated to the same version.) While EPEL has a process of saying you can announce a major change it is severely frowned on and usually ends up with various users asking 'can you just keep the older version around a bit longer...' ad infinitum. Another problem is that it isn't clear how much notice is needed for a change to be made or what changes are being made so that users and package owners end up confused and upset with each other.
Possible solution:
Make it so that updates to these sorts of packages occur at regular intervals. There are three cycles these could occur: regular date driven (July 1st of every year for example), the Fedora release cycle (Fedora 22 has been released, you can update now), and the Red Hat Enterprise release cycle (RHEL-6.6 is out, you can release now). Each of these cycles has their bonuses and minuses but I think that connecting to the RHEL cycle will be more in line with what downstream users of EPEL are going to be more in sync with.
Proposed implementation:
Every Red Hat Enterprise Dot Release has a beta period and then a release period. After the release period there is a couple of weeks until a CentOS release is available. Since only the betas and CentOS release are generally available to any person off the street I am looking at those being 'action points' where the dot release period would be done. When a RHEL beta dot release (example RHEL-6.6 beta) is announced, EPEL.dot would say that packages to be updated in the next release are going to be accepted into EPEL-dot-testing. Package owners who are needing or wanting to make major changes in their EPEL package sets would target builds to that channel. People can test as needed or at least be aware where they could have tested. Sometime after the next dot release, CentOS releases their latest 'this is the state of CentOS build' which anyone needing to test latest builds can get. This acts as a marker for when EPEL.dot will 'release' its next tree which would be 1 month after the CentOS GA. During that time, everything is rebuilt against the latest RHEL (to find any FTBS or ooops they dropped libmuffin and I needed it) and at the end of the month EPEL-6.7 is put out and EPEL-6.6 is set to be sent to archives after an appropriate time. Like CentOS, EPEL-6 -> EPEL-6.6 and then EPEL-6.7.
In the case of last planned release (RHEL-5.12?), a different mechanism could be used for when updates occur if there is enough people willing to do the work. Otherwise packages go into that tree until EPEL ends building for it.
Note only one tree is being built against. If someone wants to 'keep' EPEL-6.5 running, they can grab the src.rpms from archives and do it themselves on their own hardware.. EPEL only deals with RHEL current.
Have I made this clearer or muddier?
Thanks for starting this initiative and sorry for my late response. Anyway, my 2 cents: As maintainer of python3 in Fedora, I've already been asked by quite a lot of people to put python3 in EPEL 7. Most notably by DNF maintainer and people from Fedora Infrastructure. My problem with doing this is, that I want to able to actually update python3 point releases (e.g. from 3.4 to 3.5) at some points during EPEL 7 life. So I'm trying to understand how this proposal would help me. Here are my notes and thoughts on the situtation with python3 and the possible solution:
- Generally, python3 point releases can't be safely considered backwards
compatible and all depending packages need to be rebuilt.
- The above point I think makes updating python3 go against current EPEL
policies. If I wanted to do it "right", I would need to start with python34, then after some time add python35, python36 etc. (for each of these, I'd need to keep the whole stack of dependent libraries, which brings even more iffy problems, like "where should these packages be at dist-git", "should these conflict or be installable in parallel", "when/how do we obsolete the old versions", ...)
- So I'm wondering whether we could have just "python3" and ("python3-*"
extension modules and other dependent packages). The "python3" package itself would be updated at time of RHEL point releases, assuming a new minor version has been released prior to the update period (RHEL beta).
- Now, I'm not sure how this should work with extension modules and other
packages depending on python3. Would it make sense for them to live in the main repo, assuming they'd depend on a package that lives in the "EPEL-dot" repo? I think not, but this would basically cause all python3 depending packages to live in "EPEL-dot". I'm not sure whether that is a problem or not. It doesn't feel exactly right, but I guess it's the right thing to do...
- So basically the whole python3 stack would need to be rebuilt together
during the RHEL beta period, which might be very short, considering that we may be talking about hundreds of packages with dependency cycles etc. In case that the rebuild wouldn't be completed on time, I think it should be safe to just wait for the next RHEL dot release.
So I believe that various python consumers have wanted to have seperate python trees that are subversion specific as they can move to a newer release when they wanted to or keep parallel versions around. So what I would like to do is generate the least amount of surprise for them (versus no surprise).
Here is my suggestion:
1) Make python34 currently python35 later and python36 later.. 2) The expectation will not be putting python35 out as soon as it comes out. As you state, modules and such may need to be ported to the newest version etc so we need a baking period before the next version is ready for EPEL (so if python35 were in Fedora 23, then when Fedora 24 came out it would be time for putting it in EPEL.) 3) Make all submodules versioned like the parent (pulling 3 python packages installed on my system as an example)
python35-beautifulsoup4-4.3.2-1.el7.noarch python35-bunch-1.0.1-4.el7.noarch python35-chardet-2.0.1-7.el7.noarch
Name space them appropriately in their directory space.
4) Keep the old python3 version around for a release cycle if possible. If not then it goes away when python35 replaces python34 in the 'release cycle'.
A different solution would be to try and work out a method similar to what I was told Debian uses where something like a %post helper 'recompiles' the base helper in say python-bunch for whatever versions of python are installed on the system and places those .pyc over in the appropriate trees. That seems more error prone though in case of python which relies on C code etc.
To answer your further question, I expect that EPEL.dot will only be allowed to depend on EPEL and EPEL would not be allowed to depend on EPEL.dot. So if python35 is in EPEL.dot then python3x-bunch will be in EPEL.dot.
I am saying it this way as you could end up with problems where python application built against python-3.4 may not work without changes to the code in python-3.5 and would break horribly if it just relies on python3 as its dependson or something similar. [Having fought this over and over again with changes from python24 -> python25 etc etc.
Does this make sense? Comments/suggestions?
Thanks a lot! Slavek
-- Stephen J Smoogen.
epel-devel mailing list epel-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/epel-devel
I like Stephen's idea. That is how we handle python 3 packages in the IUS repo.
python32 python32-setuptools etc.
python33 python33-setuptools etc.
python34u python34u-setuptools
To avoid conflicts/confusion with SCL or EPEL packages, all future IUS packages will be suffixed with a u (such as python34u). Currently our python3* packages all conflict with each other. We have an open request to allow multiple version of our python3* packages by using update-alternatives [0]. We may or may not implement it, but it is another option that the EPEL might want to consider.
- Carl George
[0] https://bugs.launchpad.net/ius/+bug/1358202
________________________________ From: epel-devel-bounces@lists.fedoraproject.org [epel-devel-bounces@lists.fedoraproject.org] on behalf of Stephen John Smoogen [smooge@gmail.com] Sent: Thursday, September 11, 2014 02:31 PM To: EPEL Development List Subject: Re: EPEL EpSCO Email Meeting:
On 11 September 2014 08:41, Bohuslav Kabrda <bkabrda@redhat.commailto:bkabrda@redhat.com> wrote: Problem trying to be solved:
EPEL's original goal around a 5-7 year product has run into issues where various packages end up having shorter lifetimes than can be maintained in that period. What happens is that the upstream is changing the code too massively for any sort of 'back-port' to be possible. Some software can be 'patched' around by making it parallel installable but others (say puppet) only work if there is only one version not just on the system but throughout an entire ecosystem of systems (thus if you update one, they all have to be updated to the same version.) While EPEL has a process of saying you can announce a major change it is severely frowned on and usually ends up with various users asking 'can you just keep the older version around a bit longer...' ad infinitum. Another problem is that it isn't clear how much notice is needed for a change to be made or what changes are being made so that users and package owners end up confused and upset with each other.
Possible solution:
Make it so that updates to these sorts of packages occur at regular intervals. There are three cycles these could occur: regular date driven (July 1st of every year for example), the Fedora release cycle (Fedora 22 has been released, you can update now), and the Red Hat Enterprise release cycle (RHEL-6.6 is out, you can release now). Each of these cycles has their bonuses and minuses but I think that connecting to the RHEL cycle will be more in line with what downstream users of EPEL are going to be more in sync with.
Proposed implementation:
Every Red Hat Enterprise Dot Release has a beta period and then a release period. After the release period there is a couple of weeks until a CentOS release is available. Since only the betas and CentOS release are generally available to any person off the street I am looking at those being 'action points' where the dot release period would be done. When a RHEL beta dot release (example RHEL-6.6 beta) is announced, EPEL.dot would say that packages to be updated in the next release are going to be accepted into EPEL-dot-testing. Package owners who are needing or wanting to make major changes in their EPEL package sets would target builds to that channel. People can test as needed or at least be aware where they could have tested. Sometime after the next dot release, CentOS releases their latest 'this is the state of CentOS build' which anyone needing to test latest builds can get. This acts as a marker for when EPEL.dot will 'release' its next tree which would be 1 month after the CentOS GA. During that time, everything is rebuilt against the latest RHEL (to find any FTBS or ooops they dropped libmuffin and I needed it) and at the end of the month EPEL-6.7 is put out and EPEL-6.6 is set to be sent to archives after an appropriate time. Like CentOS, EPEL-6 -> EPEL-6.6 and then EPEL-6.7.
In the case of last planned release (RHEL-5.12?), a different mechanism could be used for when updates occur if there is enough people willing to do the work. Otherwise packages go into that tree until EPEL ends building for it.
Note only one tree is being built against. If someone wants to 'keep' EPEL-6.5 running, they can grab the src.rpms from archives and do it themselves on their own hardware.. EPEL only deals with RHEL current.
Have I made this clearer or muddier?
Thanks for starting this initiative and sorry for my late response. Anyway, my 2 cents: As maintainer of python3 in Fedora, I've already been asked by quite a lot of people to put python3 in EPEL 7. Most notably by DNF maintainer and people from Fedora Infrastructure. My problem with doing this is, that I want to able to actually update python3 point releases (e.g. from 3.4 to 3.5) at some points during EPEL 7 life. So I'm trying to understand how this proposal would help me. Here are my notes and thoughts on the situtation with python3 and the possible solution: - Generally, python3 point releases can't be safely considered backwards compatible and all depending packages need to be rebuilt. - The above point I think makes updating python3 go against current EPEL policies. If I wanted to do it "right", I would need to start with python34, then after some time add python35, python36 etc. (for each of these, I'd need to keep the whole stack of dependent libraries, which brings even more iffy problems, like "where should these packages be at dist-git", "should these conflict or be installable in parallel", "when/how do we obsolete the old versions", ...) - So I'm wondering whether we could have just "python3" and ("python3-*" extension modules and other dependent packages). The "python3" package itself would be updated at time of RHEL point releases, assuming a new minor version has been released prior to the update period (RHEL beta). - Now, I'm not sure how this should work with extension modules and other packages depending on python3. Would it make sense for them to live in the main repo, assuming they'd depend on a package that lives in the "EPEL-dot" repo? I think not, but this would basically cause all python3 depending packages to live in "EPEL-dot". I'm not sure whether that is a problem or not. It doesn't feel exactly right, but I guess it's the right thing to do... - So basically the whole python3 stack would need to be rebuilt together during the RHEL beta period, which might be very short, considering that we may be talking about hundreds of packages with dependency cycles etc. In case that the rebuild wouldn't be completed on time, I think it should be safe to just wait for the next RHEL dot release.
So I believe that various python consumers have wanted to have seperate python trees that are subversion specific as they can move to a newer release when they wanted to or keep parallel versions around. So what I would like to do is generate the least amount of surprise for them (versus no surprise).
Here is my suggestion:
1) Make python34 currently python35 later and python36 later.. 2) The expectation will not be putting python35 out as soon as it comes out. As you state, modules and such may need to be ported to the newest version etc so we need a baking period before the next version is ready for EPEL (so if python35 were in Fedora 23, then when Fedora 24 came out it would be time for putting it in EPEL.) 3) Make all submodules versioned like the parent (pulling 3 python packages installed on my system as an example)
python35-beautifulsoup4-4.3.2-1.el7.noarch python35-bunch-1.0.1-4.el7.noarch python35-chardet-2.0.1-7.el7.noarch
Name space them appropriately in their directory space.
4) Keep the old python3 version around for a release cycle if possible. If not then it goes away when python35 replaces python34 in the 'release cycle'.
A different solution would be to try and work out a method similar to what I was told Debian uses where something like a %post helper 'recompiles' the base helper in say python-bunch for whatever versions of python are installed on the system and places those .pyc over in the appropriate trees. That seems more error prone though in case of python which relies on C code etc.
To answer your further question, I expect that EPEL.dot will only be allowed to depend on EPEL and EPEL would not be allowed to depend on EPEL.dot. So if python35 is in EPEL.dot then python3x-bunch will be in EPEL.dot.
I am saying it this way as you could end up with problems where python application built against python-3.4 may not work without changes to the code in python-3.5 and would break horribly if it just relies on python3 as its dependson or something similar. [Having fought this over and over again with changes from python24 -> python25 etc etc.
Does this make sense? Comments/suggestions?
Thanks a lot! Slavek
-- Stephen J Smoogen.
_______________________________________________ epel-devel mailing list epel-devel@lists.fedoraproject.orgmailto:epel-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/epel-devel
-- Stephen J Smoogen.
On Fri, Aug 29, 2014 at 3:08 PM, Stephen John Smoogen smooge@gmail.com wrote:
- EpSCO governance.
- Lifetime of initial committee (9 months?)
This is fine for me. Somewhere in the 6-12 month range seems reasonable.
etc).
- Replacement of any exiting members (replacement by formal vote,
Vote of existing committee members, or vote of the wider EPEL dev/user
community? I'm fine with having it done with a committee vote.
- Size of committee (4 members? 5 members?)
I was in favor of 4 members when it was brought up in the last meeting.
However, it occurs to me that all of the people volunteered/nominated are RedHat employees (correct me if I'm wrong?). So I'd prefer to have one more member -- a non-RH person. This is in no way a comment on you all who I trust and believe want to do best by the community, but others may not see this the same way.
- How many repos? (examples below)
- epel,
- epel-test
- epel-rolling
- epel-rolling-test ?
- epel-edge ?
- epel-edge-test ?
It seems like we need to discuss some current example packages (already started in this thread) before jumping into a "solution". My gut says, the fewer repos, the better -- and "testing" repos for very fast repos may not be necessary/helpful.
- Would packages in this be able to conflict with epel packages? Base
packages?
From Jim's comment, overwriting base and/or "slow" EPEL packages may be
required for at least the CentOS SIG use case. I can see the need for this, but obviously some thought needs to be put into the approach to make it a) easy for people to use and b) not blow up people's systems that don't know better.
When would incompatible changes be allowed in each branch?
Different guidelines for specs/packages per branch?
When would a package be expired or removed?
All good points that should be discussed as the branch discussion moves
forward -- I don't have any specific comments at the moment.
- Currently packages require of 2 weeks in EPEL-testing before
promotion to EPEL. - Can this be changed to 1 week? - What Constant Integration would be needed to give auto-karma?
Self-serving +1 vote for changing this to 1 week. And I love the idea of
using CI to do some "basic" package tests once something is pushed to "testing".
-Jeff
On 5 September 2014 08:27, Jeff Sheltren jeff@tag1consulting.com wrote:
On Fri, Aug 29, 2014 at 3:08 PM, Stephen John Smoogen smooge@gmail.com wrote:
- EpSCO governance.
- Lifetime of initial committee (9 months?)
This is fine for me. Somewhere in the 6-12 month range seems reasonable.
I am going to take your initial request for putting an end date for this emergency committee but change the end date to March 31 2015. I think that will put a good 6 months to our current work and hopefully get it to a state where the next committee can focus on regular package problems.
etc).
- Replacement of any exiting members (replacement by formal vote,
Vote of existing committee members, or vote of the wider EPEL dev/user
community? I'm fine with having it done with a committee vote.
- Size of committee (4 members? 5 members?)
I was in favor of 4 members when it was brought up in the last meeting.
However, it occurs to me that all of the people volunteered/nominated are RedHat employees (correct me if I'm wrong?). So I'd prefer to have one more member -- a non-RH person. This is in no way a comment on you all who I trust and believe want to do best by the community, but others may not see this the same way.
At this point there isn't anyone who isn't a RH employee who has said they could make regular meetings and would like to sit on a steering committee to do a lot of grundge work looking through policies and dealing with people. If there is such a person I would be happy to look at them as a 5th member (or if one of the 4 want to leave).
epel-devel@lists.fedoraproject.org
-
Bohuslav Kabrda -
Brian Stinson -
Carl George -
Jeff Sheltren -
Jim Perrin -
Karsten Wade -
Kevin Fenzi -
Manuel Wolfshant -
Stephen John Smoogen