On Tue, Nov 05, 2013 at 07:29:35AM -0500, Bohuslav Kabrda wrote: Hmmm.... Putting on a fesco hat, I would be more than willing to approve changes for F21 now.... We've already approved Python3 as Default which has both F21 and F22 goals. I think that the F21 Schedule and Product desciptions are somewhat orthogonal to some features including this. The things I'm looking at that imply that are: * Something that either is non-specific to one of our products * Something that is an upgrade of packages already in Fedora So in my view -- submit the Change! :-) <nod> The rebuilding make me want to get this into the main repos early. I think Christian outlined what they are going to do with sha3 yesterday (but yeah, it depends on what is happening with the sha3 standard so there's a bit of uncertainty there still). Do you mean minor version as in the "4" in 3.4 or as in the micro level ("0" in 3.4.0)? We can get around bootstrapping at minor version revs if pip is only used at runtime, not at buildtime. We can still get around bootstrapping at micro version if we are careful never to include pip's code in the package. Only figure out where to find pip's code. +1 We should definitely do this. pip itself bundles code that we've had to unbundle locally -- maintaining that morass of software again (it's also bundled inside of virtualenv...) is going to lead to a lot of work anytime an update needs to happen. However, talking with ncoghlan it may not be simple. The case that makes things tricky is pyvenv. Upstream python wants to protect venv's from changes to code on the system. ie: if a user installs a backwards incompatible pip on the system, that should not affect the pip inside the existing venvs. Equally, if there are security issues found with pip, those changes would not be propogated to the pip inside the venvs. ncoghlan proposed that we create wheels in the pip package (and all the packages that pip bundles that we have to unbundle) and then when ensurepip installs into a venv, it copies those wheels in there. I'm wondering if we could do a little better. Instead of keeping around two copies of all this software on the system, if we can reconstruct a wheel zip (or possibly just copy the files in) from the wheel metadata and files on the system. I know that sysadmins and developers will hotfix software on the system for issues that they care about. Having those changes propogate to newly installed venvs seems like a better plan than having the venvs get the old copies that were built at rpm build time. -Toshio

----- Original Message ----- <snip> I already marked it ready for wrangler and I asked jreznik, who told me that the changes should be postponed. I'll try to talk to him again and see if there's any real problem about proposing this Change. Yes, exactly what I mean. We still have plenty of time, so I didn't want to spend possibly unnecessary work on rebasing the fips patch. Minor as in "4". AFAIK the bytecode format only changes with major and minor version (e.g. from 3.X to 3.Y, but _not_ from 3.X.y to 3.X.z). IMO we will need bootstrapping only for major and minor versions, not for micro versions. In other words, I think we will need bootstrapping only when bytecode format changes. <snip> Hmm, interesting idea. Once Python 3.4 test suite passes for me and Python is at Beta, I'll try to look into this :) The way that Nick is proposing in the other mail ("just copy files into the venv based on the pip and setuptools RECORD files") seems the best to me, I'll try to pursue this solution. Thanks a lot, Slavek.

Update: - DONE: I've just created a copr repo for Python 3.4 [1] and built beta1 - IN PROGRESS: I'm now working on unbundling of pip and setuptools wheels, as mentioned in thread [2]. Progress: -- DONE: created python-wheels package for Fedora (I'll build it in copr ASAP) -- DONE: added logic to python-setuptools and python-pip so that they can be built as wheels (I'll build in copr ASAP) -- TODO: Now that I have an idea how to build wheel RPMs, I need to: --- wait to see if my patches to pip that enable this get accepted [3], [4] (and perhaps communicate with upstream to make them better) --- once the patches are accepted, I'd like to talk to FPC about allowing building wheel RPMs (most likely from Fedora 21 on) and how to do that --- once that is discussed, build setuptools and pip as wheels for F21 (this is really necessary, so even if Fedora as a whole distro doesn't accept wheels usage yet, I'll try to negotiate exception for these two packages) -- TODO: in the meanwhile, I'll start working on the actual ensurepip patch and I'll propose it upstream - TODO: also in the meanwhile, I'll be updating Python 3.4 to the newest prereleases, as they are released by upstream If anyone wants to help, you can: - help push my patches to pip upstream - help with creating a proposal about building wheel RPMs for FPC (have a look at the SRPMs in the copr repo - once I've built them - to see how I'm currently doing it) - help with creating the ensurepip patch (if you want to work on any of these, please let everyone know, so that we don't duplicate our efforts) Thanks a lot, Slavek. -- Regards, Bohuslav "Slavek" Kabrda. [1] http://copr-fe.cloud.fedoraproject.org/coprs/bkabrda/python-3.4/builds/ [2] https://lists.fedoraproject.org/pipermail/python-devel/2013-November/0005... [3] https://github.com/pypa/pip/issues/991 [4] https://github.com/pypa/pip/issues/1351