I've added a list of Python-related features to our wiki page here: https://fedoraproject.org/wiki/SIGs/Python#Python_Features
Is anyone else working on Python-related Fedora features?
In particular, I've added the three that I'm looking at (so far...): * the debug builds that I've mentioned on this list [1]
* upgrading python from 2.6 to 2.7 (and rebuilding the world) [2], which I think we'll want to do soon: 2.7 beta 2 is out, with the final release scheduled by upstream for 2010-07-03, well within F-14's schedule.
* upgrading python3 from 3.1 to 3.2 [3]. This is more uncertain, as the upstream schedule doesn't line up so well with ours. We could potentially package an alpha release I guess. python3 isn't on the critical path, and the number of packages needing a rebuild is _much_ smaller than for Python 2, but I'd want to talk to upstream about it. By comparison, Ubuntu's next release has a slightly later feature freeze than ours, and is planning to ship a beta of 3.2 ([4] and [5]).
Thoughts?
Dave
[1] https://fedoraproject.org/wiki/Features/DebugPythonStacks [2] https://fedoraproject.org/wiki/Features/Python_2.7 [3] https://fedoraproject.org/wiki/Features/Python_3.2 [4] https://wiki.ubuntu.com/MaverickMeerkat/TechnicalOverview/Python [5] http://mail.python.org/pipermail/python-dev/2010-May/100165.html
On Thu, May 27, 2010 at 07:32:23PM -0400, David Malcolm wrote:
I've added a list of Python-related features to our wiki page here: https://fedoraproject.org/wiki/SIGs/Python#Python_Features
Is anyone else working on Python-related Fedora features?
You are!
pypy and pynie are good stories for Fedora 14. Although what would make this even better is figuring out how to package modules for both of these.
I've been working on coding kitchen (will package once I get the licensing worked out) which may or may not make a good story.... It's a library of little code that everyone decided they need to write. So it's little and eclectic by nature (not earthshattering) but at the same time we're writing it and it would be useful... I'll propose it as a feature if someone prods me otherwise I'll consider it just another package.
https://fedorahosted.org/releases/k/i/kitchen/docs
- upgrading python3 from 3.1 to 3.2 [3]. This is more uncertain, as
the upstream schedule doesn't line up so well with ours. We could potentially package an alpha release I guess. python3 isn't on the critical path, and the number of packages needing a rebuild is _much_ smaller than for Python 2, but I'd want to talk to upstream about it. By comparison, Ubuntu's next release has a slightly later feature freeze than ours, and is planning to ship a beta of 3.2 ([4] and [5]).
To decide this, we really want to have an idea of features vs risks and how long between our release and the upstream 3.2-final release.
* I have a vague remembrance that a lot of good things were added to 3.2 but the upstream whatsnew page doesn't yet list things that jog my memory. * We probably don't want to ship 3.2alpha1 if final doesn't come out for six more months but shipping a late alpha when final is scheduled for two months later could be a good tradeoff.
-Toshio
On Thu, 2010-05-27 at 20:39 -0400, Toshio Kuratomi wrote:
On Thu, May 27, 2010 at 07:32:23PM -0400, David Malcolm wrote:
I've added a list of Python-related features to our wiki page here: https://fedoraproject.org/wiki/SIGs/Python#Python_Features
Is anyone else working on Python-related Fedora features?
You are!
pypy and pynie are good stories for Fedora 14. Although what would make this even better is figuring out how to package modules for both of these.
(following up on this and on an IRC chat on #fedora-python):
I've speculatively created this feature for PyPy in Fedora: https://fedoraproject.org/wiki/Features/PyPyStack
It's not clear to me yet how far to push this: 1) just the core interpreter, or 2) sharing pure-Python add-on modules with the system Python (but with split bytecode files), or 3) a full, independent Python stack, but with just pure-Python add-on modules, or 4) a full, independent Python stack, with both pure-Python and machine-code extension modules
Doing e.g. (4) would probably involve a lot of work, though I think we'd be first to have done it, which would be good in of itself.
Speaking for myself, I don't yet know if I myself want to sign up for that. There are particular Python workloads I want to optimize (yum is one of them), and I don't yet know if PyPy is the best solution for what I need. (I need to write some systemtap scripts to instrument things).
As for Pynie, I've packaged a prerelease state of upstream SVN, and Tim Lauridsen reviewed it (bug 593069), but it's at an early stage of development, and upstream have requested that we not ship it until they ship tarballs (so I've closed that review out as DEFERRED for now).
I've been working on coding kitchen (will package once I get the licensing worked out) which may or may not make a good story.... It's a library of little code that everyone decided they need to write. So it's little and eclectic by nature (not earthshattering) but at the same time we're writing it and it would be useful... I'll propose it as a feature if someone prods me otherwise I'll consider it just another package.
Is $FOO a feature? There's a checklist at the bottom of this page: https://fedoraproject.org/wiki/Features/Policy/Definitions
"kitchen" looks useful, but I'm not convinced that it matches any of the criteria there. (Don't let me stop you though!)
- upgrading python3 from 3.1 to 3.2 [3]. This is more uncertain, as
the upstream schedule doesn't line up so well with ours. We could potentially package an alpha release I guess. python3 isn't on the critical path, and the number of packages needing a rebuild is _much_ smaller than for Python 2, but I'd want to talk to upstream about it. By comparison, Ubuntu's next release has a slightly later feature freeze than ours, and is planning to ship a beta of 3.2 ([4] and [5]).
To decide this, we really want to have an idea of features vs risks and how long between our release and the upstream 3.2-final release.
- I have a vague remembrance that a lot of good things were added to 3.2 but the upstream whatsnew page doesn't yet list things that jog my memory.
- We probably don't want to ship 3.2alpha1 if final doesn't come out for six more months but shipping a late alpha when final is scheduled for two months later could be a good tradeoff.
Lining up the schedules: DATE PYTHON 3 UPSTREAM FEDORA 2010-05-25 Fedora 13 Release 2010-05-28 >Present day 2010-06-26 Python 3.2 alpha 1 2010-07-13 F14 Feature Submission Deadline 2010-07-24 Python 3.2 alpha 2 2010-07-27 F14 Feature Freeze 2010-07-27 Branch Fedora 14 from Rawhide 2010-08-03 F14 Software String Freeze 2010-08-03 F14 Alpha Change Deadline 2010-08-17 F14 Alpha Release 2010-08-21 Python 3.2 alpha 3 2010-08-31 F14 Software Translation Deadline 2010-09-07 F14 Beta Change Deadline 2010-09-18 Python 3.2 beta 1; 2010-09-18 Upstream feature freeze 2010-09-21 F14 Beta Release 2010-10-12 F14 Final Freeze 2010-10-14 F14 Compose Release Candidate 2010-10-16 Python 3.2 beta 2 2010-10-26 F14 Fedora 14 Final Release 2010-11-13 Python 3.2 candidate 1 2010-11-27 Python 3.2 candidate 2 2010-12-11 Python 3.2 final
Assuming that they turn out to be be true to the day (often a dangerous assumption!), then at F14 feature freeze on 2010-07-27, upstream would be at 3.2 alpha 2 (or maybe still alpha 1). Upstream feature freeze is only 3 days before F14 beta release.
This also assumes that python3 is not on the critical path, and that none of the dual-build python2/python3 src.rpms that might in worst-case need a python3 rebuild have python2 subpackages that _are_ on the critical path (if that makes sense).
I think my chief concerns are (i) the possibility of ABI breaks during the 3.2 development cycle, affecting either the .pyc/.pyo format, or the ABI of .so extension modules. (ii) the possibility of irritating upstream by shipping an alpha
With rawhide branched for F15 on 2010-07-27, we could simply do this in rawhide for F15.
As for Python 2, I think the case for 2.7 for F14 is fairly clear, and it's just a case of getting ready, then doing it. I plan to help fix anything that breaks - but I'm going to be on vacation from June 9th through 20th, and hopefully _not_ on a computer, so we either want to do it ASAP, or after the 20th. It may be sanest to wait until after I'm back; upstream plan to release RC2 on the 19th.
Thoughts? Dave
On Fri, May 28, 2010 at 04:22:21PM -0400, David Malcolm wrote:
On Thu, 2010-05-27 at 20:39 -0400, Toshio Kuratomi wrote:
On Thu, May 27, 2010 at 07:32:23PM -0400, David Malcolm wrote:
I've added a list of Python-related features to our wiki page here: https://fedoraproject.org/wiki/SIGs/Python#Python_Features
Is anyone else working on Python-related Fedora features?
You are!
pypy and pynie are good stories for Fedora 14. Although what would make this even better is figuring out how to package modules for both of these.
(following up on this and on an IRC chat on #fedora-python):
I've speculatively created this feature for PyPy in Fedora: https://fedoraproject.org/wiki/Features/PyPyStack
It's not clear to me yet how far to push this:
- just the core interpreter, or
- sharing pure-Python add-on modules with the system Python (but with
split bytecode files), or 3) a full, independent Python stack, but with just pure-Python add-on modules, or 4) a full, independent Python stack, with both pure-Python and machine-code extension modules
Doing e.g. (4) would probably involve a lot of work, though I think we'd be first to have done it, which would be good in of itself.
Speaking for myself, I don't yet know if I myself want to sign up for that. There are particular Python workloads I want to optimize (yum is one of them), and I don't yet know if PyPy is the best solution for what I need. (I need to write some systemtap scripts to instrument things).
Yeah -- once we get pypy reviewed, I'll see if I have time to take a look at what we would need in Guidelines to get to the functionality in #4 (and whether we must have independent stacks or can do it with combined package sets).
As for Pynie, I've packaged a prerelease state of upstream SVN, and Tim Lauridsen reviewed it (bug 593069), but it's at an early stage of development, and upstream have requested that we not ship it until they ship tarballs (so I've closed that review out as DEFERRED for now).
Might be interesting to see if upstream would be okay with us building it for rawhide but blocking it from going into any release. We'd have to be proactive about telling releng to block it when we branch for F15, etc, though, if they don't make a release within six months.
I've been working on coding kitchen (will package once I get the licensing worked out) which may or may not make a good story.... It's a library of little code that everyone decided they need to write. So it's little and eclectic by nature (not earthshattering) but at the same time we're writing it and it would be useful... I'll propose it as a feature if someone prods me otherwise I'll consider it just another package.
Is $FOO a feature? There's a checklist at the bottom of this page: https://fedoraproject.org/wiki/Features/Policy/Definitions
"kitchen" looks useful, but I'm not convinced that it matches any of the criteria there. (Don't let me stop you though!)
Yeah -- So I don't really want it to be a feature since it's small and the feature process makes things into a bigger deal than they often are. On the other hand it does fall under: #3 as we're doing work, #3 and #5 if we're trying for a reputation as a distribution for developing python apps, and #4 and #2 if we decide to port yum, createrepo, func, certmaster, python-fedora, etc to them.
Hmm... And if we port yum, that means this is going to be a critpath package... /me gets out of a tangent.
- upgrading python3 from 3.1 to 3.2 [3]. This is more uncertain, as
the upstream schedule doesn't line up so well with ours. We could potentially package an alpha release I guess. python3 isn't on the critical path, and the number of packages needing a rebuild is _much_ smaller than for Python 2, but I'd want to talk to upstream about it. By comparison, Ubuntu's next release has a slightly later feature freeze than ours, and is planning to ship a beta of 3.2 ([4] and [5]).
To decide this, we really want to have an idea of features vs risks and how long between our release and the upstream 3.2-final release.
- I have a vague remembrance that a lot of good things were added to 3.2 but the upstream whatsnew page doesn't yet list things that jog my memory.
- We probably don't want to ship 3.2alpha1 if final doesn't come out for six more months but shipping a late alpha when final is scheduled for two months later could be a good tradeoff.
Lining up the schedules: DATE PYTHON 3 UPSTREAM FEDORA 2010-05-25 Fedora 13 Release 2010-05-28 >Present day 2010-06-26 Python 3.2 alpha 1 2010-07-13 F14 Feature Submission Deadline 2010-07-24 Python 3.2 alpha 2 2010-07-27 F14 Feature Freeze 2010-07-27 Branch Fedora 14 from Rawhide 2010-08-03 F14 Software String Freeze 2010-08-03 F14 Alpha Change Deadline 2010-08-17 F14 Alpha Release 2010-08-21 Python 3.2 alpha 3 2010-08-31 F14 Software Translation Deadline 2010-09-07 F14 Beta Change Deadline 2010-09-18 Python 3.2 beta 1; 2010-09-18 Upstream feature freeze 2010-09-21 F14 Beta Release 2010-10-12 F14 Final Freeze 2010-10-14 F14 Compose Release Candidate 2010-10-16 Python 3.2 beta 2 2010-10-26 F14 Fedora 14 Final Release 2010-11-13 Python 3.2 candidate 1 2010-11-27 Python 3.2 candidate 2 2010-12-11 Python 3.2 final
Assuming that they turn out to be be true to the day (often a dangerous assumption!), then at F14 feature freeze on 2010-07-27, upstream would be at 3.2 alpha 2 (or maybe still alpha 1). Upstream feature freeze is only 3 days before F14 beta release.
For F13 final, assuming they don't slip by more than a month, we'd likely ship Python3.2-beta1. That seems pretty good. If we were shipping an alpha, it would probably mean we want to ship a snapshot taken after alpha3 somewhere around beta release.
The nice thing about that would be that we can ship 3.2 final in F14 when it comes out. The second nice thing is that we should be able to channel more bug reports into upstream around python-3.2. The not nice thing is that we'd also have to work harder on integrating bugfixes.
This also assumes that python3 is not on the critical path, and that none of the dual-build python2/python3 src.rpms that might in worst-case need a python3 rebuild have python2 subpackages that _are_ on the critical path (if that makes sense).
I'm not so worried about rebuilds with python2 subpackages. Where we have to modify source code for python3 and python2 I do get a bit worried. I'm almost regretting that we decided to go with Guidelines that encourage packaging python2 and python3 from a single srpm :-(
I think my chief concerns are (i) the possibility of ABI breaks during the 3.2 development cycle, affecting either the .pyc/.pyo format, or the ABI of .so extension modules.
<nod>. So if upstream is good about these things during beta and we're probably going to hit the beta release before final freeze, this seems okay.
(ii) the possibility of irritating upstream by shipping an alpha
We'd need to talk to them... some upstreams welcome distributions shipping latest versions even if they are alphas. And if we hit the beta, it's even better. But... upstreams usually like this when they know we're committing to bringing in bugfixes and releasing frequent updates.... If python3 isn't critpath (I doubt that we're ready to port our critical tools to it. We don't even have a list of dependencies fr our major apps yet...) then we have the option of doing this. But I don't know if we want to... it will be more work.
With rawhide branched for F15 on 2010-07-27, we could simply do this in rawhide for F15.
This is a good point. Essentially we have nearly a year of development for any given release because of the early branching... so in some ways it would have been good to have this in F14 rawhide already.
So... if we want to be leaders for F14... we could do it. But I think we might want to hold off due to lack of resources at this time.... I still haven't found a list of the enhancements that 3.2 brings so I don't know what the rewards of moving to python-3.2 are....
As for Python 2, I think the case for 2.7 for F14 is fairly clear, and it's just a case of getting ready, then doing it. I plan to help fix anything that breaks - but I'm going to be on vacation from June 9th through 20th, and hopefully _not_ on a computer, so we either want to do it ASAP, or after the 20th. It may be sanest to wait until after I'm back; upstream plan to release RC2 on the 19th.
I think we should be pushing this out now. Upstream is on the last python-2.7 beta now we should push that out to rawhide/F14. This is because: 1) rawhide is expected to be least stable right after a release. 2) If any programs break because of the python update, this gives us the most time to diagnose and fix it in either python or the apps. Holding off once the beta goes out just reduces this window.
-Toshio
python-devel@lists.fedoraproject.org
-
David Malcolm -
Toshio Kuratomi