7:49 p.m.
Hi,
This is following up on a brief conversation I was part of in IRC
(#fedora-devel) earlier:
I've never been quite sure how to name packages written in Python when
they are just applications. Many "applications" written in Python
still install themselves using distutils or setuptools, and so install
a "module" into the site_packages directory. By a strict
interpretation of our Python packaging guidelines [1] [2], python 3
"modules" MUST be packaged as "python3-name".
Since technically any nontrivial Python application that can be
installed via distutils or setuptools likely installs a module, this
strict interpretation of the guidelines would mean that software that
happens to be written in Python but does not intend to provide a
library must be named with the python3-* prefix.
I am not sure that this is intended (since there's definitely software
in the distribution written in Python not named with a python prefix).
If this is not intended, then it would be nice if we could reword the
guideline to be more clear.
If it is intended, I'm prepared to argue that the current guideline is
too strict and should be revisited, both from a practical position
(users looking for software called "foobar" who many not care whether
or not it's written in Python will be confused if it is packaged under
the name "python-foobar" in Fedora), and from a philosophical one (we
don't put the "lib" prefix on every library package if upstream
doesn't use that as their name because we generally try to respect the
way upstream names their software).
As a counter-proposal I would suggest that, as we currently do, we
require the python3-prefix to be provided by the package, but
explicitly leave it to the packager+reviewer's discretion whether or
not the prefix must be part of the real name, too. Some other
languages do already do this: nodejs [3] and ocaml [4] both explicitly
have "if this primarily provides a tool or application" clauses in
their naming guidelines. I think it makes sense to have something
similar for Python, to help avoid confusion.
Ben Rosser
[1] https://fedoraproject.org/wiki/Packaging:Naming#Python_modules
[2] https://fedoraproject.org/wiki/Packaging:Python#Provides
[3] https://fedoraproject.org/wiki/Packaging:Node.js?rd=Node.js/#Naming_Guide...
[4] https://fedoraproject.org/wiki/Packaging:Naming#OCaml_modules
9:43 p.m.
On 10 August 2017 at 10:49, Ben Rosser <rosser.bjr(a)gmail.com> wrote:
As a counter-proposal I would suggest that, as we currently do, we
require the python3-prefix to be provided by the package, but
explicitly leave it to the packager+reviewer's discretion whether or
not the prefix must be part of the real name, too. Some other
languages do already do this: nodejs [3] and ocaml [4] both explicitly
have "if this primarily provides a tool or application" clauses in
their naming guidelines. I think it makes sense to have something
similar for Python, to help avoid confusion.
This is technically already the de facto policy (if it wasn't, a lot
of Fedora's build tooling would have very different names), but the
Python packaging policy doesn't spell it out explicitly.
So while I think you're right that we should make it officially that
it's OK for application packagers to let it be an implementation
detail whether something is written in Python or not, I also think we
would ideally take more steps than we do today to make sure that such
use of Python actually *is* an implementation detail, rather than
something that can leak out and cause compatibility issues with
upstream Python modules.
For example, some time ago, the "mock" command line tool had to rename
its supporting Python package to "mockbuild" in order to resolve a
file conflict when the python-mock package was first added to the
distro.
In that vein, the potentially more robust approach I've been
considering for managing that kind of situation is the idea of
recommending that these "the use of Python is an implementation
detail" applications be restructured to take advantage of the native
virtual environment support in Python 3 in order to put their
application specific modules in a private virtual environment, rather
than installing them directly into the system Python as generally
importable packages.
Specifically, having venv available by default allows us to do this as
part of an RPM build:
1. Use "python3 -m venv --system-site-packages --without-pip
<the_venv_dir>" to create an empty symlinked virtual environment that
can see system packages
2. From *outside* the venv, run "pip3 install --prefix <the_venv_dir>"
to install the application specific module(s)
3. Copy any generated "bin" scripts out into the FHS binary directory
(their shebang lines will automatically be set up to auto-activate the
private venv, since they'll reference the venv's Python symlink, *not*
the system level path)
Having a private venv like that available would also give application
packagers a relatively clean way to inject "before anything else runs"
behaviour via *.pth files, which will let them do things like
implicitly adjusting sys.path to include additional directories, as
well as tinkering with __main__.__requires__ in order to manipulate
the behaviour of pkg_resources parallel loading support.
The main reason I hadn't gotten around to actually proposing that is
that there are a few significant open questions where I'm not sure
what the right answer would be:
1. Where would these private venvs actually live?
2. Does the answer to (1) differ for pure Python app modules vs binary
extension modules?
3. Would we allow daisy chaining of these private venvs via *.pth
files? (see https://github.com/berdario/pew#add)
4. How would we make this manageable across Fedora/EPEL/SCLo spec files?
Cheers,
Nick.
--
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
4:46 a.m.
On 08/10/2017 02:49 AM, Ben Rosser wrote:
Hi,
This is following up on a brief conversation I was part of in IRC
(#fedora-devel) earlier:
I've never been quite sure how to name packages written in Python when
they are just applications. Many "applications" written in Python
still install themselves using distutils or setuptools, and so install
a "module" into the site_packages directory. By a strict
interpretation of our Python packaging guidelines [1] [2], python 3
"modules" MUST be packaged as "python3-name".
Since technically any nontrivial Python application that can be
installed via distutils or setuptools likely installs a module, this
strict interpretation of the guidelines would mean that software that
happens to be written in Python but does not intend to provide a
library must be named with the python3-* prefix.
I am not sure that this is intended (since there's definitely software
in the distribution written in Python not named with a python prefix).
If this is not intended, then it would be nice if we could reword the
guideline to be more clear.
If it is intended, I'm prepared to argue that the current guideline is
too strict and should be revisited, both from a practical position
(users looking for software called "foobar" who many not care whether
or not it's written in Python will be confused if it is packaged under
the name "python-foobar" in Fedora), and from a philosophical one (we
don't put the "lib" prefix on every library package if upstream
doesn't use that as their name because we generally try to respect the
way upstream names their software).
Hello,
You're right that this is currently missing from the guidelines. Would
you be interested in writing up a draft change?
Python modules are a special case of "Addon Packages" [0]:
If a new package is considered an "addon" package that
enhances or adds a new functionality to an existing Fedora
package without being useful on its own, its name should >
reflect this fact.
So, if you're not extending Python, but providing a standalone app, you
don't need to use the python3- prefix.
Specifically, the de-facto policy (which we should write down) is that
if nothing else is expected to import the module, then don't use the
python3- prefix. So, by not using the prefix, you're signaling that the
Python module is an implementation detail and can be removed (or moved,
for example to a different Python implementation) at any time.
If others *are* expected to import the module, some people make a
"python3-foobar" sub-package, and make the main "foobar" package
require it.
[0]
https://fedoraproject.org/wiki/Packaging:Naming?rd=Packaging:NamingGuidel...
As a counter-proposal I would suggest that, as we currently do, we
require the python3-prefix to be provided by the package, but
explicitly leave it to the packager+reviewer's discretion whether or
not the prefix must be part of the real name, too. Some other
languages do already do this: nodejs [3] and ocaml [4] both explicitly
have "if this primarily provides a tool or application" clauses in
their naming guidelines. I think it makes sense to have something
similar for Python, to help avoid confusion.
That's a nice idea.
I'll note that modules installed using Python's mechanisms (setuptools,
flit, pip, etc.) automatically provide "python3dist(name)". If we write
new guidelines, I think we should promote that rather than the
"python3-" prefix.
--
Petr Viktorin
1:44 p.m.
On Thu, Aug 10, 2017 at 11:46:42AM +0200, Petr Viktorin wrote:
On 08/10/2017 02:49 AM, Ben Rosser wrote:
>Hi,
>
>This is following up on a brief conversation I was part of in IRC
>(#fedora-devel) earlier:
>
>I've never been quite sure how to name packages written in Python when
>they are just applications. Many "applications" written in Python
>still install themselves using distutils or setuptools, and so install
>a "module" into the site_packages directory. By a strict
>interpretation of our Python packaging guidelines [1] [2], python 3
>"modules" MUST be packaged as "python3-name".
>
>Since technically any nontrivial Python application that can be
>installed via distutils or setuptools likely installs a module, this
>strict interpretation of the guidelines would mean that software that
>happens to be written in Python but does not intend to provide a
>library must be named with the python3-* prefix.
>
>I am not sure that this is intended (since there's definitely software
>in the distribution written in Python not named with a python prefix).
>If this is not intended, then it would be nice if we could reword the
>guideline to be more clear.
>
>If it is intended, I'm prepared to argue that the current guideline is
>too strict and should be revisited, both from a practical position
>(users looking for software called "foobar" who many not care whether
>or not it's written in Python will be confused if it is packaged under
>the name "python-foobar" in Fedora), and from a philosophical one (we
>don't put the "lib" prefix on every library package if upstream
>doesn't use that as their name because we generally try to respect the
>way upstream names their software).
Hello,
You're right that this is currently missing from the guidelines.
Would you be interested in writing up a draft change?
Python modules are a special case of "Addon Packages" [0]:
> If a new package is considered an "addon" package that
> enhances or adds a new functionality to an existing Fedora
> package without being useful on its own, its name should >
> reflect this fact.
So, if you're not extending Python, but providing a standalone app,
you don't need to use the python3- prefix.
Specifically, the de-facto policy (which we should write down) is
that if nothing else is expected to import the module, then don't
use the python3- prefix. So, by not using the prefix, you're
signaling that the Python module is an implementation detail and can
be removed (or moved, for example to a different Python
implementation) at any time.
This guidelines are just too vague, and I think this is the source of
many disagreements over naming. I think we should stop guessing, and
assume that if a package installs any importable module, something
might start using that module and behave accordingly.
I'd use the following rules:
1. If "primarily an application", use upstream name for both the srpm and main
binary rpm
2. Except for packages which are strongly python-version dependent, and
install executables for both python versions (sympy, sphinx, ipython, ...).
In this case use python2- and python3- prefixes for the binary
packages, and have one of those packages Provide the original name.
3. If "primarily a library", use python- prefix for srpm, and
python2-, python3- prefixes for the binary packages
4. For all 1., 2., and 3., always add '%python_provide python2-foobar'
or '%python_provide python3-foobar' or
'%python_provide python%{py3_pkgversion}-foobar'.
This way we provide consistent naming for installation (dnf install
python2-foobar and dnf install python3-foobar DTRT), and "applications"
still follow upstream naming.
If others *are* expected to import the module, some people make a
"python3-foobar" sub-package, and make the main "foobar" package
require it.
[0]
https://fedoraproject.org/wiki/Packaging:Naming?rd=Packaging:NamingGuidel...
>As a counter-proposal I would suggest that, as we currently do, we
>require the python3-prefix to be provided by the package, but
>explicitly leave it to the packager+reviewer's discretion whether or
>not the prefix must be part of the real name, too.
Yes! I think what I wrote
above matches this.
>Some other
>languages do already do this: nodejs [3] and ocaml [4] both explicitly
>have "if this primarily provides a tool or application" clauses in
>their naming guidelines. I think it makes sense to have something
>similar for Python, to help avoid confusion.
That's a nice idea.
I'll note that modules installed using Python's mechanisms
(setuptools, flit, pip, etc.) automatically provide
"python3dist(name)". If we write new guidelines, I think we should
promote that rather than the "python3-" prefix.
python3dist(name)
provides have no %_isa suffix.
Right now packages which are multilib and have such provides are
uncommon, but this could become problematic if the use of those tags
becomes widespread.
For example:
$ dnf repoquery --provides python3-qhexedit2-qt5.x86_64
python3-qhexedit2-qt5 = 0.8.3-2.fc26
python3-qhexedit2-qt5(x86-64) = 0.8.3-2.fc26
python3.6dist(qhexedit-qt5) = 0.8.3
python3dist(qhexedit-qt5) = 0.8.3
$ dnf repoquery --provides python3-qhexedit2-qt5.i686
python3-qhexedit2-qt5 = 0.8.3-2.fc26
python3-qhexedit2-qt5(x86-32) = 0.8.3-2.fc26
python3.6dist(qhexedit-qt5) = 0.8.3
python3dist(qhexedit-qt5) = 0.8.3
If I install foobar.i686, it'd have to depend on python3-qhexedit2-qt5(x86-64),
because python3.6dist(qhexedit-qt5) is not strong enough.
Afaics, %_isa never came up in the original discussion. Petr, Tomas, Miro,
you were owners of the change. Can you shine some light on this question?
(If you say that this is just not important enough, I can live with that ;))
Zbyszek
12:29 p.m.
>>>> "ZJ" == Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl> writes:
ZJ> This guidelines are just too vague, and I think this is the source
ZJ> of many disagreements over naming.
Well, there is
https://fedoraproject.org/wiki/Packaging:Guidelines#Libraries_and_Applica...
which I don't think is particularly vague.
ZJ> I think we should stop guessing, and assume that if a package
ZJ> installs any importable module, something might start using that
ZJ> module and behave accordingly.
Which is covered by the "Mixed Use Packages" section, isn't it?
This question isn't unique to Python, which is why it's in the main
packaging guidelines. If you believe that Python should be stricter
here or has special requirements, then you can certainly propose some
Python-specific guidelines. Languages which have more complete
automatic dependency generation don't particularly need to care outside
of the requirement to subpackage for purposes of dependency
minimization.
- J<
2450
days inactive
2451
days old
python-devel@lists.fedoraproject.org
4 comments
5 participants
participants (5)
-
Ben Rosser -
Jason L Tibbitts III -
Nick Coghlan -
Petr Viktorin -
Zbigniew Jędrzejewski-Szmek