-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 11/06/2013 12:50 AM, Toshio Kuratomi wrote:
On Tue, Nov 05, 2013 at 07:29:35AM -0500, Bohuslav Kabrda wrote:
-- Maybe remove the bundled pip (although this goes against the pep) and use python3-pip. E.g. everything would work as expected, but under the wraps, python3-pip package would be used. So when doing security updates, we'd just fix python3-pip. It is however true that if Fedora's pip would be different from the upstream bundled one, users may experience some behaviour differencies. I'd like to hear your opinions on how we should approach this.
We should definitely do this. pip itself bundles code that we've had to unbundle locally -- maintaining that morass of software again (it's also bundled inside of virtualenv...) is going to lead to a lot of work anytime an update needs to happen.
However, talking with ncoghlan it may not be simple. The case that makes things tricky is pyvenv. Upstream python wants to protect venv's from changes to code on the system. ie: if a user installs a backwards incompatible pip on the system, that should not affect the pip inside the existing venvs. Equally, if there are security issues found with pip, those changes would not be propogated to the pip inside the venvs.
Right - the wording in the PEP is a little stronger than the real requirement of "don't break pyvenv installing pip into virtual environments", because we wanted to make it clear that providing an appropriate wheel file is the only upstream supported way to make pyvenv work as expected in 3.4+.
There are actually other ways to make it work, the approach in the PEP is just the one that we know for sure will work properly and will cover in the upstream test suite. (Leaving room for those alternative approaches is also why the PEP explicitly states that it is OK for system pip upgrades to affect later venv creation)
ncoghlan proposed that we create wheels in the pip package (and all the packages that pip bundles that we have to unbundle) and then when ensurepip installs into a venv, it copies those wheels in there.
Technically, ensurepip uses the wheel to install pip into the venv, rather than copying the wheel anywhere.
I'm wondering if we could do a little better. Instead of keeping around two copies of all this software on the system, if we can reconstruct a wheel zip (or possibly just copy the files in) from the wheel metadata and files on the system. I know that sysadmins and developers will hotfix software on the system for issues that they care about. Having those changes propogate to newly installed venvs seems like a better plan than having the venvs get the old copies that were built at rpm build time.
Yep, that should be entirely feasible - wheel installation is just a matter of copying files, so reversing that process should work fine. You could also potentially skip the interim wheel step and just copy files into the venv based on the pip and setuptools RECORD files. The one thing to watch out for will be the files *generated* at install time - you won't want to copy those back into the wheel file or virtual environment (that should just be the script files in /usr/bin, though).
If you get this working for Fedora, I could definitely see us accepting an upstream patch to make pyvenv work that way by default in 3.5+ (i.e. copying a system installed pip if available, and only fall back to using the bundled wheel if there's no system pip). We just didn't want to make blazing that trail (since it's not officially supported by pip at this point in time) a precondition for the PEP 453 implementation :)
Cheers, Nick.
- -- Nick Coghlan Red Hat Infrastructure Engineering & Development, Brisbane
Testing Solutions Team Lead Beaker Development Lead (http://beaker-project.org/)