On 09/06/2016 08:25 PM, Nick Coghlan wrote:
On 7 September 2016 at 02:41, Tomas Orsava <torsava(a)redhat.com>
wrote:
> Hi!
>
> I'm currently writing a PEP titled "Distributing a Subset of the Standard
> Library" to standardize and hopefully improve the behavior of Python without
> the its full standard library. This is relevant to Fedora, as we exclude
> several standard library modules into separate optional packages (e.g.
> python3-tkinter).
>
> I have a draft of the first two sections: Motivation and Specification.
>
https://fedora-python.github.io/pep-drafts/pep-A.html
Very interesting, although I see a pragmatic problem with trying to
check for explicitly missing packages only after checking for the
standard library ones: the default import system doesn't make a clear
distinction as to which sys.path entries refer to the standard library
and which refer to other directories (like site-packages), so you
can't readily intercept processing after the standard library is
checked but before the rest of sys.path is processed.
Distinstion between stdlib and not-stdlib is not what's being solved here.
Within each sys.path entry, these suffixes are checked, in order:
>> importlib.machinery.all_suffixes()
['.py',
'.pyc', '.cpython-35m-x86_64-linux-gnu.so', '.abi3.so',
'.so']
The proposal is to put '.missing.py' at the end, so if neither '.py' nor
'.so' is found in that particular directory (or before it on sys.path),
'.missing.py' raises ImportError.
The idea is to use it for stdlib directories. These come before
site-packages, so overrides in site-packages would be blocked (as they
should be, since they wouldn't be compatible with a "complete" install
of Python).
If someone finds a use case for putting '.missing.py' markers elsewhere,
all the power to them :)
However, sys.meta_path *does* let you explicitly block imports
before
the default machinery is tried by raising ImportError from find_spec:
https://docs.python.org/3/reference/import.html#the-meta-path
Now, I'm making the assumption that what we need is a model whereby
the base install includes files that tells Python "these stdlib pieces
might be missing", and then the other packages can install files that
mean those "these pieces are missing" markers don't get processed.
The goal is mainly to provide good error messages when a piece of stdlib
is missing, i.e. ImportError("run dnf install python3-tkinter to get the
missing piece"). And to standardize how it's done, since CPython and
Debian need this too.
I hope we can answer the "what should be in stdlib?" question with
something more like dist info, complete with dependency information and
environment markers for things like Unix-only modules, and separate from
runtime.
One possible way to do that as a pre-import check injected into the
start of sys.meta_path would be to maintain a set of static
"module_name.optional" files in the standard library directory that
included:
- a relative file path to stat to indicate that the optional module is installed
- an import error message to raise if its not found
The proposal evolved from a similar idea; after that I realized an extra
"fallback file" in the stdlib directory is enough.
--
Petr Viktorin