= Proposed System Wide Change: Automatic Provides for Python RPM Packages = https://fedoraproject.org/wiki/Changes/ Automatic_Provides_for_Python_RPM_Packages Change owner(s): * Tomas Orsava <https://fedoraproject.org/wiki/User:Torsava> * Miro Hroncok <https://fedoraproject.org/wiki/User:Churchyard> * Email: python-maint(a)redhat.com Upon building Python packages containing packaging metadata, RPM will automatically detect the standardized name of the software (i.e. dist name, name on PyPI) in the canonical format [1] and create a virtual Provides tag with the value pythonX.Ydist(CANONICAL_NAME), where X.Y is the used Python version. RPM may also detect dependencies of the software from the metadata and automatically require them using the same syntax. == Detailed Description == If during the building of a Python package RPM encounters .egg-info, .egg-link or .dist-info files (provided in Python Wheels and Eggs), it will read the standardized name of the software (i.e. dist name, name on PyPI) in the canonical format and create a virtual Provides tag with the value pythonX.Ydist(CANONICAL_NAME), where X.Y is the used Python version. Note that the canonical format can differ slightly from the name displayed, for example, on PyPI.[1] RPM will also detect dependencies of the software from the aforementioned metadata files and automatically require them using the format pythonX.Ydist(). However, because these files don't always contain the full list of requirements (which are either in setup.py or requirements.txt), the dependency generator will not be conclusive. All Python packages will need to be rebuilt so that the virtual Provides tags are generated and can be used by users, scripts and the requires generator. == Scope == * Proposal owners: Prepare a draft for the Fedora Packaging Guidelines for Python * Maintainers of the RPM package: Backport the functionality from upsteram to Fedora. — Already done thanks to Florian Festi [2] * Release engineering: Targeted rebuild of Python packages. Ticket [3]. * List of deliverables: All Fedora deliverables will be affected, but only in a very minor way that in no way jeopardizes their delivery. * Policies and guidelines: Fedora Packaging Guidelines for Python need to be updated after the implementation so users know how to take advantage of the change. * Trademark approval: Not needed for this Change [1] https://fedoraproject.org/wiki/Changes/ Automatic_Provides_for_Python_RPM_Packages#cite_note-canonical-0

= Proposed System Wide Change: Automatic Provides for Python RPM Packages = https://fedoraproject.org/wiki/Changes/ Automatic_Provides_for_Python_RPM_Packages Change owner(s): * Tomas Orsava <https://fedoraproject.org/wiki/User:Torsava> * Miro Hroncok <https://fedoraproject.org/wiki/User:Churchyard> * Email: python-maint(a)redhat.com Upon building Python packages containing packaging metadata, RPM will automatically detect the standardized name of the software (i.e. dist name, name on PyPI) in the canonical format [1] and create a virtual Provides tag with the value pythonX.Ydist(CANONICAL_NAME), where X.Y is the used Python version. RPM may also detect dependencies of the software from the metadata and automatically require them using the same syntax. == Detailed Description == If during the building of a Python package RPM encounters .egg-info, .egg-link or .dist-info files (provided in Python Wheels and Eggs), it will read the standardized name of the software (i.e. dist name, name on PyPI) in the canonical format and create a virtual Provides tag with the value pythonX.Ydist(CANONICAL_NAME), where X.Y is the used Python version. Note that the canonical format can differ slightly from the name displayed, for example, on PyPI.[1] RPM will also detect dependencies of the software from the aforementioned metadata files and automatically require them using the format pythonX.Ydist(). However, because these files don't always contain the full list of requirements (which are either in setup.py or requirements.txt), the dependency generator will not be conclusive. All Python packages will need to be rebuilt so that the virtual Provides tags are generated and can be used by users, scripts and the requires generator. == Scope == * Proposal owners: Prepare a draft for the Fedora Packaging Guidelines for Python * Maintainers of the RPM package: Backport the functionality from upsteram to Fedora. — Already done thanks to Florian Festi [2] * Release engineering: Targeted rebuild of Python packages. Ticket [3]. * List of deliverables: All Fedora deliverables will be affected, but only in a very minor way that in no way jeopardizes their delivery. * Policies and guidelines: Fedora Packaging Guidelines for Python need to be updated after the implementation so users know how to take advantage of the change. * Trademark approval: Not needed for this Change [1] https://fedoraproject.org/wiki/Changes/ Automatic_Provides_for_Python_RPM_Packages#cite_note-canonical-0

On Fri, Jul 1, 2016 at 2:48 AM, Tomas Orsava <torsava(a)redhat.com&gt; wrote: > Hi Neal! > > I believe you will be able to use a macro for that so that dependencies are not tied to specific minor versions. I.e. something like: > > (Build)Requires: python%{python3_version}dist(somemodule) > > Of course that isn't very neat, so maybe 2 macros could be provided that would work thus: > > (Build)Requires: %python2dist somemodule > (Build)Requires: %python3dist somemodule > > That would solve the issue but still allow Fedora to possibly ship 2 different minor versions of Python 3 for example. > What makes that different from just enabling the --majorver-provides and allowing people to use "python3dist(somemodule)" or "python2dist(somemodule)"? That said, it's not out of the question to be able to work that way, as Ruby works similarly in openSUSE (since their system allows them to support more than one version of ruby in parallel if they want) and libraries work that way in Mageia.

I'm sorry, I could have expanded on it. The difference is that if we enable --majorver-provides than there's no way we can ship more than one Minor version per a Major version [0]. With macros, however, we have no problem shipping two minor versions, and the macros will continue to work, pointing to the one we treat as the main/default one. We can then also create macros for the additional minor version as well. [0] "The option '--majorver-provides' is intended for use if only one Python stack per major version will be available at a given time, as unexpected results may occur if there are multiple independent Python stacks per major version available." [1]

So, the "problem" with --majorver-provides occurs when either of the two conditions are true: * There's no dependency on a specific version of the Python ABI ("python(abi) = X.Y") in the modules * There's more than one package providing a python implementation per major version. (i.e. a package providing "python(abi) = 3.4" and "python(abi) = 3.5") In either case, you wind up in a scenario where it's possible to BR a module that would be mismatched to the Python implementation. That is, the module would be installed in a path that doesn't match the Python implementation. Thus, the package would be effectively broken, since it won't work.

However, in Mageia (which is where a variant of this dependency generator is already used), we've never had this problem because there's a dependency that forces it to match to the proper Python implementation (Requires on "python(abi) = X.Y").

And this generator has an additional protection (that doesn't exist in the Mageia one) in that generated requires use "pythonX.Ydist(CANONICAL_NAME)" no matter what.

As for the interpreter running the dependency generator, it defaults to whatever /usr/bin/python is, unless you override it.

>>>> "JR" == Jaroslav Reznik <jreznik(a)redhat.com&gt; writes:

= Proposed System Wide Change: Automatic Provides for Python RPM Packages = https://fedoraproject.org/wiki/Changes/ Automatic_Provides_for_Python_RPM_Packages Change owner(s): * Tomas Orsava <https://fedoraproject.org/wiki/User:Torsava> * Miro Hroncok <https://fedoraproject.org/wiki/User:Churchyard> * Email: python-maint(a)redhat.com Upon building Python packages containing packaging metadata, RPM will automatically detect the standardized name of the software (i.e. dist name, name on PyPI) in the canonical format [1] and create a virtual Provides tag with the value pythonX.Ydist(CANONICAL_NAME), where X.Y is the used Python version. RPM may also detect dependencies of the software from the metadata and automatically require them using the same syntax. == Detailed Description == If during the building of a Python package RPM encounters .egg-info, .egg-link or .dist-info files (provided in Python Wheels and Eggs), it will read the standardized name of the software (i.e. dist name, name on PyPI) in the canonical format and create a virtual Provides tag with the value pythonX.Ydist(CANONICAL_NAME), where X.Y is the used Python version. Note that the canonical format can differ slightly from the name displayed, for example, on PyPI.[1] RPM will also detect dependencies of the software from the aforementioned metadata files and automatically require them using the format pythonX.Ydist(). However, because these files don't always contain the full list of requirements (which are either in setup.py or requirements.txt), the dependency generator will not be conclusive. All Python packages will need to be rebuilt so that the virtual Provides tags are generated and can be used by users, scripts and the requires generator. == Scope == * Proposal owners: Prepare a draft for the Fedora Packaging Guidelines for Python * Maintainers of the RPM package: Backport the functionality from upsteram to Fedora. — Already done thanks to Florian Festi [2] * Release engineering: Targeted rebuild of Python packages. Ticket [3]. * List of deliverables: All Fedora deliverables will be affected, but only in a very minor way that in no way jeopardizes their delivery. * Policies and guidelines: Fedora Packaging Guidelines for Python need to be updated after the implementation so users know how to take advantage of the change. * Trademark approval: Not needed for this Change [1] https://fedoraproject.org/wiki/Changes/ Automatic_Provides_for_Python_RPM_Packages#cite_note-canonical-0 -- devel mailing list devel(a)lists.fedoraproject.org https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

On Mon, Jul 4, 2016 at 7:43 AM, Tomas Orsava <torsava(a)redhat.com&gt; wrote: > Hi! > > We have just removed the dependency generator from the proposal [0], see > diff [1]. > > The implementation of it (backported from RPM upstream) is significantly > lacking: Python dependencies are most usually found in setup.py and > requirements.txt files, whereas the dependency generator only takes > dependencies from the .dist-info or .egg-info files. There, the requirements > are often not listed at all, or if so, the listing can be incomplete. > > Such behavior, I believe, would serve only to confuse packagers due to it's > highly unintuitive nature. It could be explained in the packaging > guidelines, but would add a lot of complexity without adding significant > (and consistent) benefits. > > Therefore the dependency generator will not be turned on, and was removed > from the Fedora Change proposal. Err, the reason it takes it from that information is because requirements information is recorded in there during the %pyX_install phase. It's exactly the same information. Today, most Python programs' setup.py read the requirements.txt and record it in the info files during install. It's more reliable that way because then it doesn't have to figure out whether the requirements are in a requirements.txt file, the setup.py file, or somewhere else. It's a consistent location, and it eliminates test dependencies from being written in as runtime ones.