----- Original Message -----
On 12/11/2013 04:52 PM, Bohuslav Kabrda wrote:
>> So, here's a crazy thought: what if, rather than copying the installed
>> files directly into the virtual environment, we reverse engineered a
>> wheel archive *dynamically* from the system install and then installed
>> from that? That would avoid the problems with trying to bypass pip's
>> script generation, while still bootstrapping new virtual environments
>> based on the installed versions of pip and setuptools.
My attempt on recreating wheels:
https://github.com/bkabrda/rewheel
This implementation doesn't handle included scripts, although it should work fine with
autogenerated entrypoints.
> There is a slight problem here:
> In a stable Fedora release, we typically won't change pip's version, but
> rather backport security patches and the version will stay the same. So if
> you reconstruct the wheel from that, running "ensurepip --upgrade" in
> virtualenvs won't actually do anything, since pip will see the same
> version in virtualenv as is in the system and do nothing. There are few
> possible solutions to this:
> - if version in system is the same as in virtualenv, check file checksums
> and if some differ, reinstall
> - if version in system is the same as in virtualenv, uninstall pip in
> virtualenv and then reinstall the one from the system
> - (ugly, not upstream acceptable) do "rpm -q python3-pip" and use the
> Fedora release as a "build tag" for reconstructed wheel (similar to what
I
> proposed in the "bundled wheels" solution)
> If upstream (you :)) accepts any of these types of behaviour (the second
> one seems the easiest and should work fine) in the PEP, I'll work on that.
> Quite literally this time, we would reinvent the wheel :)
Ah, true, this is the thing we're planning to fix by formally adding
local version numbering as part of the upstream data model in metadata
2.0 (that is, pip, setuptools, etc would actually be *required* to cope
properly if the version metadata contained a number like "1.5-2" rather
than just "1.5", so that patched downstream rebuilds can have different
version numbers in the metadata than vanilla upstream versions). (This
is already in the draft metadata spec, but I haven't quite worked
through all the implications for version pinning yet)
You may even want to try that out (patching the pip and setuptools
version numbers to include a release suffix) and see if it already
works, since the scheme we're making official in metadata 2.0 is based
on the way pkg_resources.parse_version() already behaves:
>>> pkg_resources.parse_version("1.5") <
pkg_resources.parse_version("1.5-1")
>>> < pkg_resources.parse_version("1.5-2")
True
Failing that, I think the uninstall/reinstall trick (preferably combined
with some kind of marker so it only happens when there *is* an upgrade
to be installed) sounds like a reasonable workaround until we get
metadata 2.0 formalised upstream (which will hopefully happen before pip
1.6 is released mid next year).
Well yeah, my point is that there is no upstream-acceptable way other than checking the
file hashes by ensurepip, is there? If I wouldn't want to check file hashes, I'd
have to query RPM for release - or is there some other way you're thinking of?
So I'm basically back at the original idea, you got me :) Although the wheel
reconstruction feels a lot cleaner than just copying files and has some pros like the
entrypoint generation.
Cheers,
Nick.
--
Nick Coghlan
Red Hat Hosted & Shared Services
Software Engineering & Development, Brisbane
Testing Solutions Team Lead
Beaker Development Lead (
http://beaker-project.org/)
--
Regards,
Bohuslav "Slavek" Kabrda.