Hi,
After each Fedora release we do a retrospective with the LLVM package maintainers and talk about how we can improve the LLVM packages[1] in Fedora. We've come up with some ideas for Fedora 41 that we'd like to share to raise awareness and get feedback. Right now these are just ideas, and we plan to write up a formal change proposal once we have decided which of these we are going to implement:
* Spec file merge. We plan to merge the clang, compiler-rt, and libomp packages in with llvm and have them be sub-packages of the llvm package. This will allow us to use the build configuration recommended by upstream and also make it possible to optimize the packages using Profile-Guided Optimizations (PGO).
* Build compat packages (e.g. llvm18) as early as possible. When we package a new major release of llvm, we create a compat package so that packages that aren't compatible with the new version can still use the old version. In the past, we've waited to introduce the compat packages until the new version of LLVM was ready (typically during the Beta Freeze). However, this proved to be an issue this release for packages the were ready to switch to the compat packages early in the release cycle, but then had to wait for Beta freeze.
* Switch to python-style compat/main packages. In order to make the packaging more consistent between the main package (e.g. llvm) and the compat package (e.g. llvm18), we would retire the un-versioned dist-git for llvm, and create a new versioned dist-git for each new release (e.g. llvm19, llvm20, llvm21 etc.). We would then designate one of these as the 'main version', and that version would produce binary rpms that look like the current main package (i.e. llvm-libs instead of llvm19-libs).
* Invert the order of compat/main packages. Instead of having the compat package be the old version, and the main package be the new version, we would have the compat package be newer and the main package be older. This would allow us to introduce a new version of llvm without impacting other packages that depend on the main version of LLVM.
If anyone has any feedback on these ideas we'd like to hear it and are happy to discuss these more.
Thanks, Tom
[1] LLVM Packages are: llvm, clang, compiler-rt, libomp, lld, lldb, llvm-test-suite, libclc, llvm-bolt, libcxx, mlir, flang, python-lit, and polly.
On Fri, Apr 26, 2024 at 9:35 PM Tom Stellard tstellar@redhat.com wrote:
Hi,
After each Fedora release we do a retrospective with the LLVM package maintainers and talk about how we can improve the LLVM packages[1] in Fedora. We've come up with some ideas for Fedora 41 that we'd like to share to raise awareness and get feedback. Right now these are just ideas, and we plan to write up a formal change proposal once we have decided which of these we are going to implement:
Here's some feedback below for each of these ideas.
- Spec file merge. We plan to merge the clang, compiler-rt, and libomp packages
in with llvm and have them be sub-packages of the llvm package. This will allow us to use the build configuration recommended by upstream and also make it possible to optimize the packages using Profile-Guided Optimizations (PGO).
Are these actually released together or are they separately developed and lifecycled? If it's the latter, this would make things much more complex down the road because you'll have to deal with a lot of the weirdness that Nodejs deals with by having to subpackage with different versions and trying to keep the release values coherent so that every NVR of every subpackage is correctly unique. It's not worth it in that case.
- Build compat packages (e.g. llvm18) as early as possible. When we package a new
major release of llvm, we create a compat package so that packages that aren't compatible with the new version can still use the old version. In the past, we've waited to introduce the compat packages until the new version of LLVM was ready (typically during the Beta Freeze). However, this proved to be an issue this release for packages the were ready to switch to the compat packages early in the release cycle, but then had to wait for Beta freeze.
This is definitely a good idea. It would also mean you can ship the new version faster in Rawhide and use the corpus to properly influence upstream to do the right things before they enter stabilization. Right now, everyone finds out too late and there's never enough time to fix it.
- Switch to python-style compat/main packages. In order to make the packaging more
consistent between the main package (e.g. llvm) and the compat package (e.g. llvm18), we would retire the un-versioned dist-git for llvm, and create a new versioned dist-git for each new release (e.g. llvm19, llvm20, llvm21 etc.). We would then designate one of these as the 'main version', and that version would produce binary rpms that look like the current main package (i.e. llvm-libs instead of llvm19-libs).
Ehh? I guess? I don't think this buys us that much.
- Invert the order of compat/main packages. Instead of having the compat package be
the old version, and the main package be the new version, we would have the compat package be newer and the main package be older. This would allow us to introduce a new version of llvm without impacting other packages that depend on the main version of LLVM.
I don't like this idea, it makes things harder to reason about and doesn't actually solve any problems. You also have to do new package reviews for each new version instead of using the compatibility package exception to branch older releases into compatibility packages.
On 4/26/24 21:58, Neal Gompa wrote:
On Fri, Apr 26, 2024 at 9:35 PM Tom Stellard tstellar@redhat.com wrote:
Hi,
After each Fedora release we do a retrospective with the LLVM package maintainers and talk about how we can improve the LLVM packages[1] in Fedora. We've come up with some ideas for Fedora 41 that we'd like to share to raise awareness and get feedback. Right now these are just ideas, and we plan to write up a formal change proposal once we have decided which of these we are going to implement:
Here's some feedback below for each of these ideas.
- Spec file merge. We plan to merge the clang, compiler-rt, and libomp packages
in with llvm and have them be sub-packages of the llvm package. This will allow us to use the build configuration recommended by upstream and also make it possible to optimize the packages using Profile-Guided Optimizations (PGO).
Are these actually released together or are they separately developed and lifecycled? If it's the latter, this would make things much more complex down the road because you'll have to deal with a lot of the weirdness that Nodejs deals with by having to subpackage with different versions and trying to keep the release values coherent so that every NVR of every subpackage is correctly unique. It's not worth it in that case.
These projects are all part of the same git repository upstream.
-Tom
- Build compat packages (e.g. llvm18) as early as possible. When we package a new
major release of llvm, we create a compat package so that packages that aren't compatible with the new version can still use the old version. In the past, we've waited to introduce the compat packages until the new version of LLVM was ready (typically during the Beta Freeze). However, this proved to be an issue this release for packages the were ready to switch to the compat packages early in the release cycle, but then had to wait for Beta freeze.
This is definitely a good idea. It would also mean you can ship the new version faster in Rawhide and use the corpus to properly influence upstream to do the right things before they enter stabilization. Right now, everyone finds out too late and there's never enough time to fix it.
- Switch to python-style compat/main packages. In order to make the packaging more
consistent between the main package (e.g. llvm) and the compat package (e.g. llvm18), we would retire the un-versioned dist-git for llvm, and create a new versioned dist-git for each new release (e.g. llvm19, llvm20, llvm21 etc.). We would then designate one of these as the 'main version', and that version would produce binary rpms that look like the current main package (i.e. llvm-libs instead of llvm19-libs).
Ehh? I guess? I don't think this buys us that much.
- Invert the order of compat/main packages. Instead of having the compat package be
the old version, and the main package be the new version, we would have the compat package be newer and the main package be older. This would allow us to introduce a new version of llvm without impacting other packages that depend on the main version of LLVM.
I don't like this idea, it makes things harder to reason about and doesn't actually solve any problems. You also have to do new package reviews for each new version instead of using the compatibility package exception to branch older releases into compatibility packages.
On Sat, Apr 27, 2024 at 5:53 AM Tom Stellard tstellar@redhat.com wrote:
On 4/26/24 21:58, Neal Gompa wrote:
On Fri, Apr 26, 2024 at 9:35 PM Tom Stellard tstellar@redhat.com wrote:
Hi,
After each Fedora release we do a retrospective with the LLVM package maintainers and talk about how we can improve the LLVM packages[1] in Fedora. We've come up with some ideas for Fedora 41 that we'd like to share to raise awareness and get feedback. Right now these are just ideas, and we plan to write up a formal change proposal once we have decided which of these we are going to implement:
Here's some feedback below for each of these ideas.
- Spec file merge. We plan to merge the clang, compiler-rt, and libomp packages
in with llvm and have them be sub-packages of the llvm package. This will allow us to use the build configuration recommended by upstream and also make it possible to optimize the packages using Profile-Guided Optimizations (PGO).
Are these actually released together or are they separately developed and lifecycled? If it's the latter, this would make things much more complex down the road because you'll have to deal with a lot of the weirdness that Nodejs deals with by having to subpackage with different versions and trying to keep the release values coherent so that every NVR of every subpackage is correctly unique. It's not worth it in that case.
These projects are all part of the same git repository upstream.
That doesn't actually matter from the perspective of Fedora. What matters is if these components are versioned, released, and supported together.
To put it in developer terms: a monorepo is a Git repo of multiple projects that have independent release lifecycles. The only reason they are there is because the developer wants them there. That's in contrast to say the Linux kernel, which has a huge uniform repository of components developed by separate people that are released together and considered a single unit from a release lifecycle perspective and share versioning.
On 4/27/24 05:57, Neal Gompa wrote:
On Sat, Apr 27, 2024 at 5:53 AM Tom Stellard tstellar@redhat.com wrote:
On 4/26/24 21:58, Neal Gompa wrote:
On Fri, Apr 26, 2024 at 9:35 PM Tom Stellard tstellar@redhat.com wrote:
Hi,
After each Fedora release we do a retrospective with the LLVM package maintainers and talk about how we can improve the LLVM packages[1] in Fedora. We've come up with some ideas for Fedora 41 that we'd like to share to raise awareness and get feedback. Right now these are just ideas, and we plan to write up a formal change proposal once we have decided which of these we are going to implement:
Here's some feedback below for each of these ideas.
- Spec file merge. We plan to merge the clang, compiler-rt, and libomp packages
in with llvm and have them be sub-packages of the llvm package. This will allow us to use the build configuration recommended by upstream and also make it possible to optimize the packages using Profile-Guided Optimizations (PGO).
Are these actually released together or are they separately developed and lifecycled? If it's the latter, this would make things much more complex down the road because you'll have to deal with a lot of the weirdness that Nodejs deals with by having to subpackage with different versions and trying to keep the release values coherent so that every NVR of every subpackage is correctly unique. It's not worth it in that case.
These projects are all part of the same git repository upstream.
That doesn't actually matter from the perspective of Fedora. What matters is if these components are versioned, released, and supported together.
They are, this is main reason why they were put in the monorepo together.
- Tom
To put it in developer terms: a monorepo is a Git repo of multiple projects that have independent release lifecycles. The only reason they are there is because the developer wants them there. That's in contrast to say the Linux kernel, which has a huge uniform repository of components developed by separate people that are released together and considered a single unit from a release lifecycle perspective and share versioning.
On Sat, Apr 27, 2024 at 5:59 AM Tom Stellard tstellar@redhat.com wrote:
On 4/27/24 05:57, Neal Gompa wrote:
On Sat, Apr 27, 2024 at 5:53 AM Tom Stellard tstellar@redhat.com wrote:
On 4/26/24 21:58, Neal Gompa wrote:
On Fri, Apr 26, 2024 at 9:35 PM Tom Stellard tstellar@redhat.com wrote:
Hi,
After each Fedora release we do a retrospective with the LLVM package maintainers and talk about how we can improve the LLVM packages[1] in Fedora. We've come up with some ideas for Fedora 41 that we'd like to share to raise awareness and get feedback. Right now these are just ideas, and we plan to write up a formal change proposal once we have decided which of these we are going to implement:
Here's some feedback below for each of these ideas.
- Spec file merge. We plan to merge the clang, compiler-rt, and libomp packages
in with llvm and have them be sub-packages of the llvm package. This will allow us to use the build configuration recommended by upstream and also make it possible to optimize the packages using Profile-Guided Optimizations (PGO).
Are these actually released together or are they separately developed and lifecycled? If it's the latter, this would make things much more complex down the road because you'll have to deal with a lot of the weirdness that Nodejs deals with by having to subpackage with different versions and trying to keep the release values coherent so that every NVR of every subpackage is correctly unique. It's not worth it in that case.
These projects are all part of the same git repository upstream.
That doesn't actually matter from the perspective of Fedora. What matters is if these components are versioned, released, and supported together.
They are, this is main reason why they were put in the monorepo together.
Then the only tradeoff is that it makes the LLVM build take longer. But if you're okay with that, then it's fine, I suppose.
Neal Gompa ngompa13@gmail.com writes:
You also have to do new package reviews for each new version instead of using the compatibility package exception to branch older releases into compatibility packages.
I don't think this will be needed because it is one of the exceptions [1]:
The package is being created so that multiple versions of the same package can coexist in the distribution (or coexist between EPEL and RHEL). The package MUST be properly named according to the naming guidelines and MUST NOT conflict with all other versions of the same package.
AFAIU, this proposal is following all the requirements mentioned in this exception.
[1] https://docs.fedoraproject.org/en-US/packaging-guidelines/ReviewGuidelines/#...
Dne 27. 04. 24 v 6:58 Neal Gompa napsal(a):
- Switch to python-style compat/main packages. In order to make the packaging more
consistent between the main package (e.g. llvm) and the compat package (e.g. llvm18), we would retire the un-versioned dist-git for llvm, and create a new versioned dist-git for each new release (e.g. llvm19, llvm20, llvm21 etc.). We would then designate one of these as the 'main version', and that version would produce binary rpms that look like the current main package (i.e. llvm-libs instead of llvm19-libs).
Ehh? I guess? I don't think this buys us that much.
- Invert the order of compat/main packages. Instead of having the compat package be
the old version, and the main package be the new version, we would have the compat package be newer and the main package be older. This would allow us to introduce a new version of llvm without impacting other packages that depend on the main version of LLVM.
I don't like this idea, it makes things harder to reason about and doesn't actually solve any problems.
I concur with Neal here.
I we were living in ideal world, we would have just one `llvm` package. Since we are not living in ideal world, it makes sense to have some compat versions. But we should try to get as close as possible to ideal world. Versioning packages by default goes against that goal, because it encourages sticking to some specific version for no particular reason.
Vít
On Mon, May 13, 2024 at 3:23 PM Vít Ondruch vondruch@redhat.com wrote:
Dne 27. 04. 24 v 6:58 Neal Gompa napsal(a):
- Switch to python-style compat/main packages. In order to make the packaging more
consistent between the main package (e.g. llvm) and the compat package (e.g. llvm18), we would retire the un-versioned dist-git for llvm, and create a new versioned dist-git for each new release (e.g. llvm19, llvm20, llvm21 etc.). We would then designate one of these as the 'main version', and that version would produce binary rpms that look like the current main package (i.e. llvm-libs instead of llvm19-libs).
Ehh? I guess? I don't think this buys us that much.
- Invert the order of compat/main packages. Instead of having the compat package be
the old version, and the main package be the new version, we would have the compat package be newer and the main package be older. This would allow us to introduce a new version of llvm without impacting other packages that depend on the main version of LLVM.
I don't like this idea, it makes things harder to reason about and doesn't actually solve any problems.
I concur with Neal here.
I we were living in ideal world, we would have just one `llvm` package. Since we are not living in ideal world, it makes sense to have some compat versions. But we should try to get as close as possible to ideal world. Versioning packages by default goes against that goal, because it encourages sticking to some specific version for no particular reason.
For the special case of LLVM, I disagree here. Some projects are just not compatible with newer LLVM versions until upstream moves first, and that can take time. So I don't think that counts as "sticking to some specific version for no particular reason", it's "upstream doesn't support LLVM X at all yet, they only support LLVM X-1 for now". I have never seen a Fedora package that uses an LLVM compat package "for no particular reason".
Fabio
Dne 13. 05. 24 v 15:28 Fabio Valentini napsal(a):
On Mon, May 13, 2024 at 3:23 PM Vít Ondruch vondruch@redhat.com wrote:
Dne 27. 04. 24 v 6:58 Neal Gompa napsal(a):
- Switch to python-style compat/main packages. In order to make the packaging more
consistent between the main package (e.g. llvm) and the compat package (e.g. llvm18), we would retire the un-versioned dist-git for llvm, and create a new versioned dist-git for each new release (e.g. llvm19, llvm20, llvm21 etc.). We would then designate one of these as the 'main version', and that version would produce binary rpms that look like the current main package (i.e. llvm-libs instead of llvm19-libs).
Ehh? I guess? I don't think this buys us that much.
- Invert the order of compat/main packages. Instead of having the compat package be
the old version, and the main package be the new version, we would have the compat package be newer and the main package be older. This would allow us to introduce a new version of llvm without impacting other packages that depend on the main version of LLVM.
I don't like this idea, it makes things harder to reason about and doesn't actually solve any problems.
I concur with Neal here.
I we were living in ideal world, we would have just one `llvm` package. Since we are not living in ideal world, it makes sense to have some compat versions. But we should try to get as close as possible to ideal world. Versioning packages by default goes against that goal, because it encourages sticking to some specific version for no particular reason.
For the special case of LLVM, I disagree here. Some projects are just not compatible with newer LLVM versions until upstream moves first, and that can take time. So I don't think that counts as "sticking to some specific version for no particular reason", it's "upstream doesn't support LLVM X at all yet, they only support LLVM X-1 for now". I have never seen a Fedora package that uses an LLVM compat package "for no particular reason".
My point is that we can spent time maintaining llvm00 - llvm99 packages or we can spent time adjusting upstream projects to be compatible with the latest llvm. Maintaining old versions of package might IMHO cost more then adjusting the package to new version.
Vít
Fabio
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On Mon, May 13, 2024 at 3:42 PM Vít Ondruch vondruch@redhat.com wrote:
My point is that we can spent time maintaining llvm00 - llvm99 packages or we can spent time adjusting upstream projects to be compatible with the latest llvm.
There are many projects that require a fair amount of work to be ported to newer llvm versions, take intel-igc for example ( https://github.com/intel/intel-graphics-compiler ) where there is now more than a year long process to slowly get it to llvm 16 (yeah, that's not a typo).
Similarly, python-llvmlite requires llvm14, and the upcoming upstream release will require llvm15 (with a medium-term plan to get to llvm17). For complex projects that are tightly coupled to the LLVM implementation like this, there is generally *absolutely nothing* that downstream packagers can do to get upstream support for current LLVM releases to happen faster. This is a well-maintained and responsive package that is, among other things, the basis of the JIT implementation for the very popular numba package, https://numba.pydata.org/.
On 5/14/24 9:46 AM, Frantisek Zatloukal wrote:
On Mon, May 13, 2024 at 3:42 PM Vít Ondruch vondruch@redhat.com wrote:
My point is that we can spent time maintaining llvm00 - llvm99 packages or we can spent time adjusting upstream projects to be compatible with the latest llvm.
There are many projects that require a fair amount of work to be ported to newer llvm versions, take intel-igc for example ( https://github.com/intel/intel-graphics-compiler ) where there is now more than a year long process to slowly get it to llvm 16 (yeah, that's not a typo).
--
Best regards / S pozdravem,
František Zatloukal Senior Quality Engineer Red Hat
-- _______________________________________________ devel mailing list --devel@lists.fedoraproject.org To unsubscribe send an email todevel-leave@lists.fedoraproject.org Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it:https://pagure.io/fedora-infrastructure/new_issue
Dne 13. 05. 24 v 15:23 Vít Ondruch napsal(a):
Dne 27. 04. 24 v 6:58 Neal Gompa napsal(a):
- Switch to python-style compat/main packages. In order to make the packaging more
consistent between the main package (e.g. llvm) and the compat package (e.g. llvm18), we would retire the un-versioned dist-git for llvm, and create a new versioned dist-git for each new release (e.g. llvm19, llvm20, llvm21 etc.). We would then designate one of these as the 'main version', and that version would produce binary rpms that look like the current main package (i.e. llvm-libs instead of llvm19-libs).
Ehh? I guess? I don't think this buys us that much.
- Invert the order of compat/main packages. Instead of having the compat package be
the old version, and the main package be the new version, we would have the compat package be newer and the main package be older. This would allow us to introduce a new version of llvm without impacting other packages that depend on the main version of LLVM.
I don't like this idea, it makes things harder to reason about and doesn't actually solve any problems.
I concur with Neal here.
I we were living in ideal world, we would have just one `llvm` package. Since we are not living in ideal world, it makes sense to have some compat versions. But we should try to get as close as possible to ideal world. Versioning packages by default goes against that goal, because it encourages sticking to some specific version for no particular reason.
Actually, reading Miro's answer and since I spent quite some time thinking about this, maybe I should elaborate more.
I truly think that we should really have one default / rolling non-versioned version of packages and try as hard as we can to keep everything compatible with those versions. This is actually where I see the biggest benefit of distributions. And this is actually also where most of my upstream contributions went.
And if there is need for multiple versions, we should make them parallel installable with the default / rolling version. This would actually allow to have `python` package, which is the default version and likely also `python3.12` if somebody choose to stay with some specific version for specific reason. This would also allow to introduce `python3.13` and keep it for testing forward compatibility and later for backward compatibility purposes.
And no, I don't think about `python` package as a metapackage or so. It would be fully fledged package. Maybe it would be currently the same or very close to the `python3.12` package.
And TBH, for me as a Fedora used with no special interest in Python, the current Python versioning sucks hard. How am I supposed to tell what is the current version just looking at e.g. the repository? Is it `python3.12` or is it already `python3.13`? Despite I have spent with Fedora more then a decade, answering such simple question is not trivial for me.
Vít
On 13. 05. 24 15:38, Vít Ondruch wrote:
And TBH, for me as a Fedora used with no special interest in Python, the current Python versioning sucks hard. How am I supposed to tell what is the current version just looking at e.g. the repository? Is it `python3.12` or is it already `python3.13`? Despite I have spent with Fedora more then a decade, answering such simple question is not trivial for me.
I guess that for the user, the easiest way is to look at the RPMs. Users barely look into our repositories.
Dne 13. 05. 24 v 20:23 Miro Hrončok napsal(a):
On 13. 05. 24 15:38, Vít Ondruch wrote:
And TBH, for me as a Fedora used with no special interest in Python, the current Python versioning sucks hard. How am I supposed to tell what is the current version just looking at e.g. the repository? Is it `python3.12` or is it already `python3.13`? Despite I have spent with Fedora more then a decade, answering such simple question is not trivial for me.
I guess that for the user, the easiest way is to look at the RPMs. Users barely look into our repositories.
~~~
$ rpm -q python package python is not installed
~~~
Why?
Even if `# dnf install python` does something, it still won't install `python` package.
Vít
On 14. 05. 24 16:02, Vít Ondruch wrote:
Dne 13. 05. 24 v 20:23 Miro Hrončok napsal(a):
On 13. 05. 24 15:38, Vít Ondruch wrote:
And TBH, for me as a Fedora used with no special interest in Python, the current Python versioning sucks hard. How am I supposed to tell what is the current version just looking at e.g. the repository? Is it `python3.12` or is it already `python3.13`? Despite I have spent with Fedora more then a decade, answering such simple question is not trivial for me.
I guess that for the user, the easiest way is to look at the RPMs. Users barely look into our repositories.
$ rpm -q python package python is not installed
Why?
Because it is called python3.
$ rpm -q python3 python3-3.12.3-2.fc39.x86_64
I thought this discussion is about python3.12 vs python3.13, not about python vs python3. I supposed the reason it is called python3 and not python is well know at this point (but if it is not, let me know and I'll try to explain).
We are in 2024, so I suppose we could rename everything python3 to python now, I just worry that it would be a lot of effort for not much benefit.
Even if `# dnf install python` does something, it still won't install `python` package.
Well, it installs the python-unverisoned-command package. Which requires python3. So it install python. Why does it matter? What are you trying to demonstrate here? (Don't take me wrong, I always appreciate good criticism, I juts don't understand what are you suggesting we should do.)
Do you suggest to rename python-unversioned-command to python? Do you suggest to rename python3 to python? Do you suggest to rename the python3.12 component to python? (As names of the components started this discussion.) Or is it something else?
Dne 14. 05. 24 v 18:35 Miro Hrončok napsal(a):
On 14. 05. 24 16:02, Vít Ondruch wrote:
Dne 13. 05. 24 v 20:23 Miro Hrončok napsal(a):
On 13. 05. 24 15:38, Vít Ondruch wrote:
And TBH, for me as a Fedora used with no special interest in Python, the current Python versioning sucks hard. How am I supposed to tell what is the current version just looking at e.g. the repository? Is it `python3.12` or is it already `python3.13`? Despite I have spent with Fedora more then a decade, answering such simple question is not trivial for me.
I guess that for the user, the easiest way is to look at the RPMs. Users barely look into our repositories.
$ rpm -q python package python is not installed
Why?
Because it is called python3.
$ rpm -q python3 python3-3.12.3-2.fc39.x86_64
I thought this discussion is about python3.12 vs python3.13, not about python vs python3. I supposed the reason it is called python3 and not python is well know at this point (but if it is not, let me know and I'll try to explain).
We are in 2024, so I suppose we could rename everything python3 to python now, I just worry that it would be a lot of effort for not much benefit.
Even if `# dnf install python` does something, it still won't install `python` package.
Well, it installs the python-unverisoned-command package. Which requires python3. So it install python. Why does it matter? What are you trying to demonstrate here? (Don't take me wrong, I always appreciate good criticism, I juts don't understand what are you suggesting we should do.)
Do you suggest to rename python-unversioned-command to python? Do you suggest to rename python3 to python? Do you suggest to rename the python3.12 component to python? (As names of the components started this discussion.) Or is it something else?
Every time I bring up such discussion, I am told "the reason it is called python3 and not python is well know" and yes, it is know to some, including me. But advocating for less experienced users. I advocating for users which are not experts on Python ecosystem. I am advocating for conventions.
I am trying to demonstrate that things should be obvious. There is "Python" language. Not "Python 3" language. There is e.g. https://www.python.org/ not https://www.python3.org/ etc.
Therefore, I'd rather hear "you are right, that does not make too much sense (these days). It is confusing and it is about the time to make the things right (finally)". In your words "We are in 2024, so I suppose we could rename everything python3 to python now" is what I would appreciate.
Vít
On 15. 05. 24 10:08, Vít Ondruch wrote:
Dne 14. 05. 24 v 18:35 Miro Hrončok napsal(a):
On 14. 05. 24 16:02, Vít Ondruch wrote:
Dne 13. 05. 24 v 20:23 Miro Hrončok napsal(a):
On 13. 05. 24 15:38, Vít Ondruch wrote:
And TBH, for me as a Fedora used with no special interest in Python, the current Python versioning sucks hard. How am I supposed to tell what is the current version just looking at e.g. the repository? Is it `python3.12` or is it already `python3.13`? Despite I have spent with Fedora more then a decade, answering such simple question is not trivial for me.
I guess that for the user, the easiest way is to look at the RPMs. Users barely look into our repositories.
$ rpm -q python package python is not installed
Why?
Because it is called python3.
$ rpm -q python3 python3-3.12.3-2.fc39.x86_64
I thought this discussion is about python3.12 vs python3.13, not about python vs python3. I supposed the reason it is called python3 and not python is well know at this point (but if it is not, let me know and I'll try to explain).
We are in 2024, so I suppose we could rename everything python3 to python now, I just worry that it would be a lot of effort for not much benefit.
Even if `# dnf install python` does something, it still won't install `python` package.
Well, it installs the python-unverisoned-command package. Which requires python3. So it install python. Why does it matter? What are you trying to demonstrate here? (Don't take me wrong, I always appreciate good criticism, I juts don't understand what are you suggesting we should do.)
Do you suggest to rename python-unversioned-command to python? Do you suggest to rename python3 to python? Do you suggest to rename the python3.12 component to python? (As names of the components started this discussion.) Or is it something else?
Every time I bring up such discussion, I am told "the reason it is called python3 and not python is well know" and yes, it is know to some, including me. But advocating for less experienced users. I advocating for users which are not experts on Python ecosystem. I am advocating for conventions.
I am trying to demonstrate that things should be obvious. There is "Python" language. Not "Python 3" language. There is e.g. https://www.python.org/ not https://www.python3.org/ etc.
Therefore, I'd rather hear "you are right, that does not make too much sense (these days). It is confusing and it is about the time to make the things right (finally)". In your words "We are in 2024, so I suppose we could rename everything python3 to python now" is what I would appreciate.
So you say "python3" should be renamed to "python".
But this entire discussion started about component names (e.g. "python3.12") and your inability to tell which Python version is the default just by looking at the sources.
I am not disagreeing with you. I just don't see how we suddenly discuss a completely different thing.
Anyway, let me tell you:
You are right, calling the package(s) "python3" does not make too much sense any more. It might be confusing and it might be about the time to make things right by renaming ~4200 packages back to "python". Feel free to propose a detailed plan of execution.
Note that I won't do it, because I don't think the benefits outweighs the necessary work. However, if there is a volunteer to drive this, I am happy to review the proposal and share my feedback.
Dne 15. 05. 24 v 12:10 Miro Hrončok napsal(a):
On 15. 05. 24 10:08, Vít Ondruch wrote:
Dne 14. 05. 24 v 18:35 Miro Hrončok napsal(a):
On 14. 05. 24 16:02, Vít Ondruch wrote:
Dne 13. 05. 24 v 20:23 Miro Hrončok napsal(a):
On 13. 05. 24 15:38, Vít Ondruch wrote:
And TBH, for me as a Fedora used with no special interest in Python, the current Python versioning sucks hard. How am I supposed to tell what is the current version just looking at e.g. the repository? Is it `python3.12` or is it already `python3.13`? Despite I have spent with Fedora more then a decade, answering such simple question is not trivial for me.
I guess that for the user, the easiest way is to look at the RPMs. Users barely look into our repositories.
$ rpm -q python package python is not installed
Why?
Because it is called python3.
$ rpm -q python3 python3-3.12.3-2.fc39.x86_64
I thought this discussion is about python3.12 vs python3.13, not about python vs python3. I supposed the reason it is called python3 and not python is well know at this point (but if it is not, let me know and I'll try to explain).
We are in 2024, so I suppose we could rename everything python3 to python now, I just worry that it would be a lot of effort for not much benefit.
Even if `# dnf install python` does something, it still won't install `python` package.
Well, it installs the python-unverisoned-command package. Which requires python3. So it install python. Why does it matter? What are you trying to demonstrate here? (Don't take me wrong, I always appreciate good criticism, I juts don't understand what are you suggesting we should do.)
Do you suggest to rename python-unversioned-command to python? Do you suggest to rename python3 to python? Do you suggest to rename the python3.12 component to python? (As names of the components started this discussion.) Or is it something else?
Every time I bring up such discussion, I am told "the reason it is called python3 and not python is well know" and yes, it is know to some, including me. But advocating for less experienced users. I advocating for users which are not experts on Python ecosystem. I am advocating for conventions.
I am trying to demonstrate that things should be obvious. There is "Python" language. Not "Python 3" language. There is e.g. https://www.python.org/ not https://www.python3.org/ etc.
Therefore, I'd rather hear "you are right, that does not make too much sense (these days). It is confusing and it is about the time to make the things right (finally)". In your words "We are in 2024, so I suppose we could rename everything python3 to python now" is what I would appreciate.
So you say "python3" should be renamed to "python".
But this entire discussion started about component names (e.g. "python3.12") and your inability to tell which Python version is the default just by looking at the sources.
I am not disagreeing with you. I just don't see how we suddenly discuss a completely different thing.
The whole discussion started with LLVM and Python was used as an example. I am saying that Python is bad example and nobody should follow it.
The problem I see is that there is no `python` package which would be coming from `python` SRPM and `python` repository. On top of that, there is by default no `python` command while Python is installed on the system. That is not intuitive.
BTW the `Why?` was rhetoric question, wondering why it so unintuitive. After all, I know the status, history, commands etc. I also acknowledge the amount of invested work to get to the point where we are.
Vít
Anyway, let me tell you:
You are right, calling the package(s) "python3" does not make too much sense any more. It might be confusing and it might be about the time to make things right by renaming ~4200 packages back to "python". Feel free to propose a detailed plan of execution.
Note that I won't do it, because I don't think the benefits outweighs the necessary work. However, if there is a volunteer to drive this, I am happy to review the proposal and share my feedback.
On 15. 05. 24 13:31, Vít Ondruch wrote:
I am saying that Python is bad example and nobody should follow it.
I respectfully disagree. The LLVM maintainers think it is a good example worth following. So did the NodeJS maintainers. Name-versioning all the components makes things so much easier for the maintainers.
The component name does not matter to the "normal" users. It only matters to experienced packagers, but those should be capable of dealing with that.
(The Python 3 vs Python thing obviously is horrible, and nobody should ever do anything like that again.)
On Wed, May 15, 2024 at 2:02 PM Miro Hrončok mhroncok@redhat.com wrote:
On 15. 05. 24 13:31, Vít Ondruch wrote:
I am saying that Python is bad example and nobody should follow it.
I respectfully disagree. The LLVM maintainers think it is a good example worth following. So did the NodeJS maintainers. Name-versioning all the components makes things so much easier for the maintainers.
Right - IMO the Python stack is the *best* example of how to provide multiple versions of something in Fedora, and for how transitions to new major versions are handled in Rawhide. (And any remaining Python vs. Python 3 confusions are an orthogonal problem.) Being able to use both newer and older versions of Python on different branches of Fedora is *awesome*, for example for running tests against different Python versions with tox.
Fabio
On Wed, 15 May 2024 at 04:09, Vít Ondruch vondruch@redhat.com wrote:
Every time I bring up such discussion, I am told "the reason it is called python3 and not python is well know" and yes, it is know to some, including me. But advocating for less experienced users. I advocating for users which are not experts on Python ecosystem. I am advocating for conventions.
Going from the convention I am finding on my MacOS-X brew and the Windows box.. the convention is some variant of calling both the command and package some variant of python3. Brew only gives you a python3 command and will say the package is some name like python@3.<version>-3.<version>. Windows does have a python package but also calls it python3 in places.
Also going from things I run into regularly.. python2 is still the most used language in many places with it being provided as /usr/local/bin/python or some other variant. Scripts which call /usr/bin/python are still mostly python2 scripts and will break if run with python3 on the first `print ""` statement or some other thing.
With EL7 and similar long lived python2 distributions going EOL this year.. I expect that this python2 will start to move, but I figure we are still a decade out (going from how long I was dealing with perl4 after perl5 had been out).
I am trying to demonstrate that things should be obvious. There is "Python" language. Not "Python 3" language. There is e.g. https://www.python.org/ not https://www.python3.org/ etc.
Therefore, I'd rather hear "you are right, that does not make too much sense (these days). It is confusing and it is about the time to make the things right (finally)". In your words "We are in 2024, so I suppose we could rename everything python3 to python now" is what I would appreciate.
Vít
-- _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On Sat, Apr 27, 2024 at 12:35 AM Tom Stellard tstellar@redhat.com wrote:
Hi,
After each Fedora release we do a retrospective with the LLVM package maintainers and talk about how we can improve the LLVM packages[1] in Fedora. We've come up with some ideas for Fedora 41 that we'd like to share to raise awareness and get feedback. Right now these are just ideas, and we plan to write up a formal change proposal once we have decided which of these we are going to implement:
- Spec file merge. We plan to merge the clang, compiler-rt, and libomp packages
in with llvm and have them be sub-packages of the llvm package. This will allow us to use the build configuration recommended by upstream and also make it possible to optimize the packages using Profile-Guided Optimizations (PGO).
- Build compat packages (e.g. llvm18) as early as possible. When we package a new
major release of llvm, we create a compat package so that packages that aren't compatible with the new version can still use the old version. In the past, we've waited to introduce the compat packages until the new version of LLVM was ready (typically during the Beta Freeze). However, this proved to be an issue this release for packages the were ready to switch to the compat packages early in the release cycle, but then had to wait for Beta freeze.
From other tools and venues and history with gcc and gnutls releaes, I like it.
- Switch to python-style compat/main packages. In order to make the packaging more
consistent between the main package (e.g. llvm) and the compat package (e.g. llvm18), we would retire the un-versioned dist-git for llvm, and create a new versioned dist-git for each new release (e.g. llvm19, llvm20, llvm21 etc.). We would then designate one of these as the 'main version', and that version would produce binary rpms that look like the current main package (i.e. llvm-libs instead of llvm19-libs).
- Invert the order of compat/main packages. Instead of having the compat package be
the old version, and the main package be the new version, we would have the compat package be newer and the main package be older. This would allow us to introduce a new version of llvm without impacting other packages that depend on the main version of LLVM.
My first thought is "don't make me hurt you". So are my second and third thoughts. Please do not leave the nominally obsolete version as the default cnotemporary version, the "main" release should always be the defult. New, pre-release versions should be as short-lived as possible.
If anyone has any feedback on these ideas we'd like to hear it and are happy to discuss these more.
Thanks, Tom
[1] LLVM Packages are: llvm, clang, compiler-rt, libomp, lld, lldb, llvm-test-suite, libclc, llvm-bolt, libcxx, mlir, flang, python-lit, and polly. -- _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Nico Kadel-Garcia nkadel@gmail.com writes:
On Sat, Apr 27, 2024 at 12:35 AM Tom Stellard tstellar@redhat.com wrote:
- Invert the order of compat/main packages. Instead of having the compat package be
the old version, and the main package be the new version, we would have the compat package be newer and the main package be older. This would allow us to introduce a new version of llvm without impacting other packages that depend on the main version of LLVM.
My first thought is "don't make me hurt you". So are my second and third thoughts. Please do not leave the nominally obsolete version as the default cnotemporary version, the "main" release should always be the defult.
I'm not sure I understood this part or if there was a miscommunication somewhere.
Considering that LLVM releases usually happen very late in Fedora's development cycle, if the default LLVM version is changed, packages may start to FTBFS very late in the development cycle if they buildrequire the default LLVM version.
Notice that, in this proposal, packages that would prefer to use the new version may still update them by buildrequiring the new versioned package.
With that said: do you really think that it's better to let packages FTBFS late in the Fedora development cycle? If that's still true, could you elaborate it, please?
New, pre-release versions should be as short-lived as possible.
AFAIU, there are no plans to increase the time pre-release version will be kept.
On Mon, Apr 29, 2024 at 2:25 PM Tulio Magno Quites Machado Filho tuliom@ascii.art.br wrote:
Considering that LLVM releases usually happen very late in Fedora's development cycle, if the default LLVM version is changed, packages may start to FTBFS very late in the development cycle if they buildrequire the default LLVM version.
Notice that, in this proposal, packages that would prefer to use the new version may still update them by buildrequiring the new versioned package.
I would rather see the llvm base package(s) always be the latest (and perhaps greatest), and for there to be something like a llvm-not-the-latest (or some other well known name) so that those whose packages are known to be llvm version sensitive can make a one-time change to use the not-the-latest version of llvm (i.e. put the onus of using not-the-latest with the package(r)s that need not-the-latest, or some specific version) so that they can be more assured of not having last minute FTBFS issues.
Do we have any idea how many code bases are actually sensitive to the specific llvm version? I suspect that there are a few likely well known and expected code bases, and most code bases are (mostly) agnostic.
On 29/04/2024 16:41, Gary Buhrmaster wrote:
Do we have any idea how many code bases are actually sensitive to the specific llvm version?
Both of my LLVM dependent packages: iwyu and pocl. On every LLVM major release they break and I have to wait for the upstream to release a new version.
On Mon, Apr 29, 2024 at 4:38 PM Vitaly Zaitsev via devel devel@lists.fedoraproject.org wrote:
Both of my LLVM dependent packages: iwyu and pocl. On every LLVM major release they break and I have to wait for the upstream to release a new version.
I would hope that there are more examples than O(1), as processes should not be determined by O(1) numbers.
In any case, since this is *every* release, is there any good reason these are not somewhere in the LLVM CI/QA workflows? Sounds like good test cases, and good test cases are typically hard to find.
On 27. 04. 24 6:34, Tom Stellard wrote:
...
- Switch to python-style compat/main packages. In order to make the packaging
more consistent between the main package (e.g. llvm) and the compat package (e.g. llvm18), we would retire the un-versioned dist-git for llvm, and create a new versioned dist-git for each new release (e.g. llvm19, llvm20, llvm21 etc.). We would then designate one of these as the 'main version', and that version would produce binary rpms that look like the current main package (i.e. llvm-libs instead of llvm19-libs).
In Python, we did it this way because it was hard to keep one "python3" component that was different version in different Fedoras + multiple "python3.X" components that did (not) exist on certain Fedoras. Git cherry-pciks between branches were... hard. Merges were impossible.
Having the component names release-agnostic simplified things us a lot.
- Invert the order of compat/main packages. Instead of having the compat
package be the old version, and the main package be the new version, we would have the compat package be newer and the main package be older. This would allow us to introduce a new version of llvm without impacting other packages that depend on the main version of LLVM.
This allowed us to package new pre-releases of Python as soon as alpha 1 was out (we could even do pre-alphas, but so far there was not enough motivation).
That makes it easier for users to test things early. It also allows *us* to test the RPM build early.
It also allowed users of e.g. Fedora 38 to use Python 3.12 if they needed to (however, without the entire RPM-Python packages stack on top), despite Python 3.12 being the main Python in F39+ only.
Overall, it works quite nicely.
If anyone has any feedback on these ideas we'd like to hear it and are happy to discuss these more.
+100
Any chance this gets partially implemented in older Fedoras? I'd appreciate llvm18 and clang18 in Fedora 39 (for Python 3.13 experimental JIT).
https://github.com/python/cpython/blob/3.13/Tools/jit/README.md
Thanks,
On Saturday 27 April 2024 05:34:29 BST Tom Stellard wrote:
Hi,
- Build compat packages (e.g. llvm18) as early as possible. When we package
a new major release of llvm, we create a compat package so that packages that aren't compatible with the new version can still use the old version. In the past, we've waited to introduce the compat packages until the new version of LLVM was ready (typically during the Beta Freeze). However, this proved to be an issue this release for packages the were ready to switch to the compat packages early in the release cycle, but then had to wait for Beta freeze.
- Switch to python-style compat/main packages. In order to make the
packaging more consistent between the main package (e.g. llvm) and the compat package (e.g. llvm18), we would retire the un-versioned dist-git for llvm, and create a new versioned dist-git for each new release (e.g. llvm19, llvm20, llvm21 etc.). We would then designate one of these as the 'main version', and that version would produce binary rpms that look like the current main package (i.e. llvm-libs instead of llvm19-libs).
As a variant on these two ideas, could the main package "Provides" the compat version that it'll become if it's kept around after the main package moves on?
In other words, llvm-libs for LLVM 21 also has a "Provides: llvm21-libs", so that as a consumer of LLVM, I can depend on llvm-libs (meaning I don't really care which version of LLVM, and I'd like the best you offer, or I can depend on llvm21-libs from day 1 if I know that I'm going to need time to move to LLVM 22.
Then, if you do decide to offer llvm21-libs, you can do so with no-one being affected; if you're looking for data on affected packages, you can use dnf repoquery to tell you how many packages depend on llvm21-libs as opposed to llvm-libs.
- Invert the order of compat/main packages. Instead of having the compat
package be the old version, and the main package be the new version, we would have the compat package be newer and the main package be older. This would allow us to introduce a new version of llvm without impacting other packages that depend on the main version of LLVM.
This, I dislike; the unversioned package should, IMO, be the latest supported by Fedora version, with older versions provided as compat packages where desirable.
If anyone has any feedback on these ideas we'd like to hear it and are happy to discuss these more.
Thanks, Tom
On 5/13/24 07:16, Simon Farnsworth via devel wrote:
On Saturday 27 April 2024 05:34:29 BST Tom Stellard wrote:
Hi,
- Build compat packages (e.g. llvm18) as early as possible. When we package
a new major release of llvm, we create a compat package so that packages that aren't compatible with the new version can still use the old version. In the past, we've waited to introduce the compat packages until the new version of LLVM was ready (typically during the Beta Freeze). However, this proved to be an issue this release for packages the were ready to switch to the compat packages early in the release cycle, but then had to wait for Beta freeze.
- Switch to python-style compat/main packages. In order to make the
packaging more consistent between the main package (e.g. llvm) and the compat package (e.g. llvm18), we would retire the un-versioned dist-git for llvm, and create a new versioned dist-git for each new release (e.g. llvm19, llvm20, llvm21 etc.). We would then designate one of these as the 'main version', and that version would produce binary rpms that look like the current main package (i.e. llvm-libs instead of llvm19-libs).
As a variant on these two ideas, could the main package "Provides" the compat version that it'll become if it's kept around after the main package moves on?
In other words, llvm-libs for LLVM 21 also has a "Provides: llvm21-libs", so that as a consumer of LLVM, I can depend on llvm-libs (meaning I don't really care which version of LLVM, and I'd like the best you offer, or I can depend on llvm21-libs from day 1 if I know that I'm going to need time to move to LLVM 22.
Then, if you do decide to offer llvm21-libs, you can do so with no-one being affected; if you're looking for data on affected packages, you can use dnf repoquery to tell you how many packages depend on llvm21-libs as opposed to llvm-libs.
llvm-libs may not be the best example, because the requires are auto-generated based on the shared object soname, and you aren't supposed to be explicitly requiring them.
For other sub-packages, you can already do
BuildRequires: clang(major) = 18
And it will pull in the either clang or clang18 depending on your Fedora version.
-Tom
- Invert the order of compat/main packages. Instead of having the compat
package be the old version, and the main package be the new version, we would have the compat package be newer and the main package be older. This would allow us to introduce a new version of llvm without impacting other packages that depend on the main version of LLVM.
This, I dislike; the unversioned package should, IMO, be the latest supported by Fedora version, with older versions provided as compat packages where desirable.
If anyone has any feedback on these ideas we'd like to hear it and are happy to discuss these more.
Thanks, Tom
On 27/04/2024 06:34, Tom Stellard wrote:
If anyone has any feedback on these ideas we'd like to hear it and are happy to discuss these more.
Can you fix the ${LLVM_LIBRARY_DIR} variable in LLVM's CMake config?
It broke after switching LLVM to use /usr/lib. It still points to /usr/lib64 and we need to patch it manually[1].
[1]: https://src.fedoraproject.org/rpms/iwyu/blob/rawhide/f/iwyu.spec#_51
On 5/21/24 09:33, Vitaly Zaitsev via devel wrote:
On 27/04/2024 06:34, Tom Stellard wrote:
If anyone has any feedback on these ideas we'd like to hear it and are happy to discuss these more.
Can you fix the ${LLVM_LIBRARY_DIR} variable in LLVM's CMake config?
It broke after switching LLVM to use /usr/lib. It still points to /usr/lib64 and we need to patch it manually[1].
I looked in the iwyu source, and it's not using ${LLVM_LIBRARY_DIR} correctly. That variable points to where the libraries are installed, but iwyu is using it to look up the resource directory. iwyu should be using `clang -print-resource-dir` instead.
-Tom
On 22/05/2024 01:45, Tom Stellard wrote:
I looked in the iwyu source, and it's not using ${LLVM_LIBRARY_DIR} correctly. That variable points to where the libraries are installed, but iwyu is using it to look up the resource directory. iwyu should be using `clang -print-resource-dir` instead.
Can you submit a pull request with a fix to upstream?
Vitaly Zaitsev via devel devel@lists.fedoraproject.org writes:
On 22/05/2024 01:45, Tom Stellard wrote:
I looked in the iwyu source, and it's not using ${LLVM_LIBRARY_DIR} correctly. That variable points to where the libraries are installed, but iwyu is using it to look up the resource directory. iwyu should be using `clang -print-resource-dir` instead.
Can you submit a pull request with a fix to upstream?
Done! https://github.com/include-what-you-use/include-what-you-use/pull/1542