The introduction of perl 5.38.0 broke polymake beyond repair. Several symbols formerly used by polymake have been marked as internal APIs. They now have the ELF hidden attribute, so we can't even cheat by adding prototypes to the polymake code.
Polymake upstream is aware of the issue. For the time being, they are advising their downstreams to stay on perl 5.36.0, which is not feasible for Fedora. In the long term, they plan to remove the mandatory perl bindings from polymake. However, they as yet have no timeline for that effort. It might be years.
I don't see that I have any choice but to retire polymake from Rawhide. This will have some repercussions: - gap-pkg-polymaking, python-jupymake, and python-jupyter-polymake will also be retired - sagemath and Macaulay2 will be rebuilt without polymake support - packages that I have maintained solely for use by polymake will be orphaned: azove, permlib, plantri, sympol, vinci
I could try begging the perl package maintainers to add a downstream patch making the affected symbols visible again. However, since those symbols are now internal only, the perl maintainers are free to alter or remove them at any time, so that would not be a good long term solution.
On Mon, Jul 17, 2023 at 04:13:59PM -0600, Jerry James wrote:
The introduction of perl 5.38.0 broke polymake beyond repair. Several symbols formerly used by polymake have been marked as internal APIs. They now have the ELF hidden attribute, so we can't even cheat by adding prototypes to the polymake code.
Do you have a list of the deprecated APIs they are using? Or a link to a bug on the topic? I happen to have far too much experience of Perl internals and I might be able to suggest replacements (although not promising anything ...)
Rich.
Polymake upstream is aware of the issue. For the time being, they are advising their downstreams to stay on perl 5.36.0, which is not feasible for Fedora. In the long term, they plan to remove the mandatory perl bindings from polymake. However, they as yet have no timeline for that effort. It might be years.
I don't see that I have any choice but to retire polymake from Rawhide. This will have some repercussions:
- gap-pkg-polymaking, python-jupymake, and python-jupyter-polymake
will also be retired
- sagemath and Macaulay2 will be rebuilt without polymake support
- packages that I have maintained solely for use by polymake will be
orphaned: azove, permlib, plantri, sympol, vinci
I could try begging the perl package maintainers to add a downstream patch making the affected symbols visible again. However, since those symbols are now internal only, the perl maintainers are free to alter or remove them at any time, so that would not be a good long term solution. -- Jerry James, who is currently channeling Billy Joel http://www.jamezone.org/ _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On Tue, Jul 18, 2023 at 08:23:54AM +0100, Richard W.M. Jones wrote:
On Mon, Jul 17, 2023 at 04:13:59PM -0600, Jerry James wrote:
The introduction of perl 5.38.0 broke polymake beyond repair. Several symbols formerly used by polymake have been marked as internal APIs. They now have the ELF hidden attribute, so we can't even cheat by adding prototypes to the polymake code.
Do you have a list of the deprecated APIs they are using? Or a link to a bug on the topic? I happen to have far too much experience of Perl internals and I might be able to suggest replacements (although not promising anything ...)
This is the only discussion I can find:
https://forum.polymake.org/viewtopic.php?t=1914
Let me remove that configure block and try to see what actually fails when it builds ...
Rich.
Polymake upstream is aware of the issue. For the time being, they are advising their downstreams to stay on perl 5.36.0, which is not feasible for Fedora. In the long term, they plan to remove the mandatory perl bindings from polymake. However, they as yet have no timeline for that effort. It might be years.
I don't see that I have any choice but to retire polymake from Rawhide. This will have some repercussions:
- gap-pkg-polymaking, python-jupymake, and python-jupyter-polymake
will also be retired
- sagemath and Macaulay2 will be rebuilt without polymake support
- packages that I have maintained solely for use by polymake will be
orphaned: azove, permlib, plantri, sympol, vinci
I could try begging the perl package maintainers to add a downstream patch making the affected symbols visible again. However, since those symbols are now internal only, the perl maintainers are free to alter or remove them at any time, so that would not be a good long term solution. -- Jerry James, who is currently channeling Billy Joel http://www.jamezone.org/ _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On Tue, Jul 18, 2023 at 08:27:19AM +0100, Richard W.M. Jones wrote:
On Tue, Jul 18, 2023 at 08:23:54AM +0100, Richard W.M. Jones wrote:
On Mon, Jul 17, 2023 at 04:13:59PM -0600, Jerry James wrote:
The introduction of perl 5.38.0 broke polymake beyond repair. Several symbols formerly used by polymake have been marked as internal APIs. They now have the ELF hidden attribute, so we can't even cheat by adding prototypes to the polymake code.
Do you have a list of the deprecated APIs they are using? Or a link to a bug on the topic? I happen to have far too much experience of Perl internals and I might be able to suggest replacements (although not promising anything ...)
This is the only discussion I can find:
https://forum.polymake.org/viewtopic.php?t=1914
Let me remove that configure block and try to see what actually fails when it builds ...
The first error is:
/home/rjones/d/fedora/polymake/rawhide/polymake-4.10/lib/core/src/perl/RefHash.xxs:737:11: error: ‘Perl_ck_fun’ was not declared in this scope; did you mean ‘Perl_cx_dup’? 737 | return Perl_ck_fun(aTHX_ o); | ^~~~~~~~~~~ | Perl_cx_dup ninja: build stopped: subcommand failed.
As far as I can see that's the only missing / hidden Perl symbol.
This symbol was hidden in: https://github.com/Perl/perl5/commit/0351a629e71de127cbfd1b142e9eaa6069deabf...
As for what it does, that's more tricky. I believe what it's doing is while Perl is parsing the input script, it is used to check that the thing you are calling is a function. Unfortunately it's slightly more complicated than that because it can convert some function-like things to functions (returning the updated OP*). Also Perl_ck_fun is complicated, to say the least:
https://github.com/Perl/perl5/blob/a6d10131eee6ee336e4bd63f22a378e9d5ae40bd/...
For these reasons we can't just replace 'return Perl_ck_fun(aTHX_ o)' with 'return aTHX_ o', nor can be copy the function into the polymake source (since it calls other internal functions, but also for licensing reasons).
So I think this does require upstream attention.
Another note is this package requires ocaml-tplib which we orphaned.
Rich.
Rich.
Polymake upstream is aware of the issue. For the time being, they are advising their downstreams to stay on perl 5.36.0, which is not feasible for Fedora. In the long term, they plan to remove the mandatory perl bindings from polymake. However, they as yet have no timeline for that effort. It might be years.
I don't see that I have any choice but to retire polymake from Rawhide. This will have some repercussions:
- gap-pkg-polymaking, python-jupymake, and python-jupyter-polymake
will also be retired
- sagemath and Macaulay2 will be rebuilt without polymake support
- packages that I have maintained solely for use by polymake will be
orphaned: azove, permlib, plantri, sympol, vinci
I could try begging the perl package maintainers to add a downstream patch making the affected symbols visible again. However, since those symbols are now internal only, the perl maintainers are free to alter or remove them at any time, so that would not be a good long term solution. -- Jerry James, who is currently channeling Billy Joel http://www.jamezone.org/ _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On Tue, Jul 18, 2023 at 5:41 AM Richard W.M. Jones rjones@redhat.com wrote:
The first error is:
/home/rjones/d/fedora/polymake/rawhide/polymake-4.10/lib/core/src/perl/RefHash.xxs:737:11: error: ‘Perl_ck_fun’ was not declared in this scope; did you mean ‘Perl_cx_dup’? 737 | return Perl_ck_fun(aTHX_ o); | ^~~~~~~~~~~ | Perl_cx_dup ninja: build stopped: subcommand failed.
As far as I can see that's the only missing / hidden Perl symbol.
This symbol was hidden in: https://github.com/Perl/perl5/commit/0351a629e71de127cbfd1b142e9eaa6069deabf...
As for what it does, that's more tricky. I believe what it's doing is while Perl is parsing the input script, it is used to check that the thing you are calling is a function. Unfortunately it's slightly more complicated than that because it can convert some function-like things to functions (returning the updated OP*). Also Perl_ck_fun is complicated, to say the least:
https://github.com/Perl/perl5/blob/a6d10131eee6ee336e4bd63f22a378e9d5ae40bd/...
For these reasons we can't just replace 'return Perl_ck_fun(aTHX_ o)' with 'return aTHX_ o', nor can be copy the function into the polymake source (since it calls other internal functions, but also for licensing reasons).
So I think this does require upstream attention.
The code also calls save_pushptrptr (Pel_save_pushptrptr), another hidden symbol. Polymake has long had way too much knowledge of perl internals, which is why it has tended to break whenever a new perl version was released. It broke drastically this time.
Another note is this package requires ocaml-tplib which we orphaned.
Right. I was going to remove that dependency, but the package broke from the perl update while we were still doing the OCaml 5 builds, so I never got the chance. At this point, I don't see how to keep it in Fedora until upstream does some major surgery on their code base.
Thanks for looking at it!
Jerry James wrote:
The code also calls save_pushptrptr (Pel_save_pushptrptr), another hidden symbol.
That one is pretty short though. But I do not know whether the stuff it does can be done without invoking more internal symbols that are now hidden. The code of the function is in Perl's scope.c, the macros it uses are defined in scope.h.
Kevin Kofler
Richard W.M. Jones wrote:
/home/rjones/d/fedora/polymake/rawhide/polymake-4.10/lib/core/src/perl/RefHash.xxs:737:11:
error: ‘Perl_ck_fun’ was not declared in this scope; did you mean ‘Perl_cx_dup’? 737 | return Perl_ck_fun(aTHX_ o); | ^~~~~~~~~~~ | Perl_cx_dup ninja: build stopped: subcommand failed.
As I understand it, this function is included in the public (at least, it is declared EXT, so it should not be hidden) PL_check array (several times, the first time at index 2), so I believe something like: #define Perl_ck_fun (PL_check[2]) should work.
Kevin Kofler
On Thu, Jul 20, 2023 at 02:20:59AM +0200, Kevin Kofler via devel wrote:
Richard W.M. Jones wrote:
/home/rjones/d/fedora/polymake/rawhide/polymake-4.10/lib/core/src/perl/RefHash.xxs:737:11:
error: ‘Perl_ck_fun’ was not declared in this scope; did you mean ‘Perl_cx_dup’? 737 | return Perl_ck_fun(aTHX_ o); | ^~~~~~~~~~~ | Perl_cx_dup ninja: build stopped: subcommand failed.
As I understand it, this function is included in the public (at least, it is declared EXT, so it should not be hidden) PL_check array (several times, the first time at index 2), so I believe something like: #define Perl_ck_fun (PL_check[2]) should work.
That seems possible, although it's a bit of a hack to misuse the array like this:
https://github.com/Perl/perl5/blob/ccfa6b533228ad41e4dbc5576cd43081b8fd2a13/...
I suppose if polymake is going to fiddle with Perl internals anyway, then why not.
Rich.
On Thu, Jul 20, 2023 at 7:29 AM Richard W.M. Jones rjones@redhat.com wrote:
On Thu, Jul 20, 2023 at 02:20:59AM +0200, Kevin Kofler via devel wrote:
As I understand it, this function is included in the public (at least, it is declared EXT, so it should not be hidden) PL_check array (several times, the first time at index 2), so I believe something like: #define Perl_ck_fun (PL_check[2]) should work.
That seems possible, although it's a bit of a hack to misuse the array like this:
https://github.com/Perl/perl5/blob/ccfa6b533228ad41e4dbc5576cd43081b8fd2a13/...
I suppose if polymake is going to fiddle with Perl internals anyway, then why not.
Thank you very much, Kevin and Rich, for the hints. I decided to see if polymake could be made to work with perl 5.38.0 after all. It turns out that polymake calls quite a few functions that are now hidden. However, the function prototypes are still included in public headers, so that did not become apparent until link time. I have been able to find pointers to most of them (mostly in PL_ppaddr). A few gave me some trouble:
Perl_keyword Perl_list Perl_save_pushptrptr Perl_scalar Perl_yyerror
For each of those, I included the upstream perl implementations (nearly) verbatim in the polymake code. Yuck. But it seems to "work". I am now encountering the kinds of problems upstream would have faced with perl 5.38.0 anyway, even without functions becoming hidden. For example, I just adapted the code to this change in Perl_anoncode: https://github.com/Perl/perl5/pull/20290. There are probably more issues. Whether all of them can be addressed remains to be seen. However, I will not orphan or retire any packages for now.