Hi!
On 09/06/2016 08:25 PM, Nick Coghlan wrote:
On 7 September 2016 at 02:41, Tomas Orsava <torsava(a)redhat.com>
wrote:
> Hi!
>
> I'm currently writing a PEP titled "Distributing a Subset of the Standard
> Library" to standardize and hopefully improve the behavior of Python without
> the its full standard library. This is relevant to Fedora, as we exclude
> several standard library modules into separate optional packages (e.g.
> python3-tkinter).
>
> I have a draft of the first two sections: Motivation and Specification.
>
https://fedora-python.github.io/pep-drafts/pep-A.html
Very interesting, although I see a pragmatic problem with trying to
check for explicitly missing packages only after checking for the
standard library ones: the default import system doesn't make a clear
distinction as to which sys.path entries refer to the standard library
and which refer to other directories (like site-packages), so you
can't readily intercept processing after the standard library is
checked but before the rest of sys.path is processed.
I have created a proposal for
the Reference Implementation which you can
find here:
https://github.com/torsava/cpython/pull/1
It works by checking for the .missing.py files in every directory on the
sys.path. Therefore the packager can put it into any directory they use
for stdlib, and it will work (that directory of course already is on the
sys.path). And because it checks for the .missing.py file only after it
tried every other possible file extension in each directory, it won't
shadow actually installed stdlib modules.
Tomas Orsava
However, sys.meta_path *does* let you explicitly block imports
before
the default machinery is tried by raising ImportError from find_spec:
https://docs.python.org/3/reference/import.html#the-meta-path
Now, I'm making the assumption that what we need is a model whereby
the base install includes files that tells Python "these stdlib pieces
might be missing", and then the other packages can install files that
mean those "these pieces are missing" markers don't get processed.
One possible way to do that as a pre-import check injected into the
start of sys.meta_path would be to maintain a set of static
"module_name.optional" files in the standard library directory that
included:
- a relative file path to stat to indicate that the optional module is installed
- an import error message to raise if its not found
If the new hook either didn't see an optional module entry, or if it
checked and the file path was present, it would allow the import to
continue as normal. However, if it found the optional declaration file
and the file path missing, it would raise ImportError with the given
message.
CPython by default would provide optional declarations for the modules
that are optional in the upstream build process, then downstreams like
Fedora may add more for things like tkinter, idlelib, and the test
suite.
The downside of this approach is that it does mean the initial import
of optional modules would be a touch slower (since the file path to
stat and the error message would need to be read from file), whereas
the version in the draft PEP has the virtue of having no impact on
import time for modules that are available, even when starting from a
completely cold sys.modules cache.
So if we did want to enable the draft proposal in the PEP, we'd need
to look at proposing a special directory to hold the "missing" markers
that was always placed just after the standard library directory on
sys.path, and then defining a custom path_hook to process it:
https://docs.python.org/3/reference/import.html#path-entry-finders
Cheers,
Nick.