On Thu, Mar 21, 2019 at 10:05:55PM +0000, Tomasz Kłoczko wrote: > Just FTR: > > [tkloczko@domek SPECS.fedora]$ egrep -w "popd|pushd" * -l| wc -l > 2843 > > Looks like many Fedora packagers forgot that .. > > [tkloczko@domek SPECS.fedora]$ rpm -E %_buildshell > /bin/sh So what? On Fedora /bin/sh is bash, and bash is a fine shell. All this nonsense of using dash for /bin/sh on Debian is IMO a pointless bunch of make-work.

In Fedora, all scriptlets can safely assume they are running under the bash shell unless a different language has been specified.

So what? On Fedora /bin/sh is bash, and bash is a fine shell. All this nonsense of using dash for /bin/sh on Debian is IMO a pointless bunch of make-work.

On Fri, Mar 22, 2019 at 6:04 PM Japheth Cleaver <cleaver(a)terabithia.org&gt; wrote: > IMO the situation that we're in now ("Assume you're running in > bash, but called as -/bin/sh") is a worst-of-both-worlds middle > ground, somewhat akin to mandating webpages be written in IE Quirks > Mode for all time. It's neither pedantically correct, nor flexible > for users and downstreams. And the resolution from all of this last > time remains lacking in the guarantees that an independent spec > should have: The result is what we call bashisms where people put /bin/sh in their shebangs but don't realize they are using GNU extensions, and that goes for "standard" programs like awk too where people are not aware of the differences regarding what's on their machine and what POSIX mandates.

So with both a default non-POSIX-compliant shell (by default, I'm aware) and non-POSIX-compliant options to the basic commands it takes more effort to write portable shell. To be clear, bash is my favorite shell and the one I use, but I'd prefer that running it as sh or /bin/sh would be enough to enter POSIX mode.

Le 2019-03-25 09:53, Jan Pokorný a écrit : > Good point, and that's something capable of making upstream > maintenance cumbersome at times (sed is a common pet peeve), > but that's an order of magnitude more demanding level when it > comes to portability, and with Fedora settled firmly just around > GNU approaches and extensions, there's hardly a pressing need for > the spec files to come anywhere close (but if so, the restrictions > should not be limited to shell interpreter alone as remarked, > since POSIX compliance is a wider topic). More accurately, what is the point of wasting energy on making a Linux system POSIX-only today? POSIX was a useful tool to make proprietary unixes more or less compatible with one another. The situation has changed since. Linux has taken over most of the marketplace. That means the common compat layer you need to target to replace it with something else, is whatever major Linux distributions agree on, and that includes all the GNU tools with all their non-POSIX extensions.

Try 1 at specification: Fedora is based on GNU tools versus strict POSIX compliant ones. As such, packagers can expect that /bin/sh is /bin/bash, /bin/awk is /bin/gawk, /bin/cc is /bin/gcc ad naseum. This means that unless specified elsewhere that a 'bashism', 'gawkism', 'gcc-ism' is not to be used, packagers may rely on tools to act as the upstream GNU tools in their spec files.

> > Fedora is based on GNU tools versus strict POSIX compliant ones. As > > such, packagers can expect that /bin/sh is /bin/bash, /bin/awk is > > /bin/gawk, /bin/cc is /bin/gcc ad naseum. This means that unless > > specified elsewhere that a 'bashism', 'gawkism', 'gcc-ism' is not to > > be used, packagers may rely on tools to act as the upstream GNU tools > > in their spec files.

So please write the spec you want to see.

Fedora is based on GNU tools versus strict POSIX compliant ones. As such, package maintainers can expect RPM build steps and scriptlets to be run with bash and that core programs come with GNU extensions. This means that unless specified elsewhere that a 'bashism', 'gawkism', 'gcc-ism' or other GNUism is not to be used, they may rely on tools to act as the upstream GNU tools in their spec files. When shipping scripts from upstream, care should be taken to check whether they rely on such GNUisms and report portability bugs upstream.

* Dridi Boukelmoune: > This is the kind of spec that leads to spoiled upstreams putting > /bin/sh in shebangs and scratching their heads when they get bug > reports for stricter systems... > > I'd be happier if Fedora was not part of the problem and maintainers > were encouraged to figure out the correct shebang (and when in doubt > use /usr/bin/bash). If you want more compatibility, you definitely can't use /usr/bin/bash. Fedora is so different from other GNU/Linux systems these days, so I'm not sure if *any* recommendation to encourage portability (at the cost of convenience to Fedora developers or users) makes sense anymore.

_______________________________________________ devel mailing list -- devel(a)lists.fedoraproject.org To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

* Stephen John Smoogen: > On Mon, 25 Mar 2019 at 08:13, Florian Weimer <fw(a)deneb.enyo.de&gt; wrote: >> >> * Dridi Boukelmoune: >> >> > This is the kind of spec that leads to spoiled upstreams putting >> > /bin/sh in shebangs and scratching their heads when they get bug >> > reports for stricter systems... >> > >> > I'd be happier if Fedora was not part of the problem and maintainers >> > were encouraged to figure out the correct shebang (and when in doubt >> > use /usr/bin/bash). >> >> If you want more compatibility, you definitely can't use /usr/bin/bash. >> >> Fedora is so different from other GNU/Linux systems these days, so I'm >> not sure if *any* recommendation to encourage portability (at the cost >> of convenience to Fedora developers or users) makes sense anymore. > > This statement constantly confuses me. Every release we are told we > are too different from other GNU/Linux systems. That drives changes > which are supposed to make us more compatible, and yet at the next > release we are again where we started from. Hmm. I haven't seen that, at least not for low-level matters. (In the compiler flags discussion, it's about *not* changing defaults from GNU upstream.) We have implemented UsrMove and we don't use multi-arch paths for ELF objects, which makes use incompatible with a fairly large chunk of the rest of the GNU/Linux ecosystem. Our Java packaging does not use the Class-Path: manifest attribute, and packages do not consistently use /usr/share/java for storing JAR files. So at the lower levels, there is not much we can do to improve compatibility. Perhaps we can adopt multi-arch paths, and just hope that the rest of the world will eventually implement mandatory UsrMove as well.

* Dridi Boukelmoune: > This is the kind of spec that leads to spoiled upstreams putting > /bin/sh in shebangs and scratching their heads when they get bug > reports for stricter systems... > > I'd be happier if Fedora was not part of the problem and maintainers > were encouraged to figure out the correct shebang (and when in doubt > use /usr/bin/bash). If you want more compatibility, you definitely can't use /usr/bin/bash.

Fedora is so different from other GNU/Linux systems these days, so I'm not sure if *any* recommendation to encourage portability (at the cost of convenience to Fedora developers or users) makes sense anymore.

Fedora is so different from other GNU/Linux systems these days, so I'm not sure if *any* recommendation to encourage portability (at the cost of convenience to Fedora developers or users) makes sense anymore. _______________________________________________

On Mon, 2019-03-25 at 12:59 +0100, Dridi Boukelmoune wrote: > And since RPM appears to be configurable for the > default interpreter, have it use /usr/bin/bash by default. TBH, it seems to me reasonable that we just do this. If our position is that we actually expect Fedora package scriptlets to be executed by bash and don't think it's a problem if they don't work when executed by some other shell, why not this make this clear and explicit in this way instead of having the default be sh, but then tell people sh must be bash?

On Mon, 25 Mar 2019 at 11:18, Kevin Fenzi <kevin(a)scrye.com&gt; wrote: > > On 3/25/19 8:02 AM, Adam Williamson wrote: > > On Mon, 2019-03-25 at 12:59 +0100, Dridi Boukelmoune wrote: > >> And since RPM appears to be configurable for the > >> default interpreter, have it use /usr/bin/bash by default. > > > > TBH, it seems to me reasonable that we just do this. > > > > If our position is that we actually expect Fedora package scriptlets to > > be executed by bash and don't think it's a problem if they don't work > > when executed by some other shell, why not this make this clear and > > explicit in this way instead of having the default be sh, but then tell > > people sh must be bash? > > Doesn't bash behave slightly differently when invoked as 'sh' ? > > Long ago it used to, but I don't know if thats still the case... > It will interpret some things differently but does not shut off things like pushd/popd and $() and various other things that are either considered non-POSIX

On Mon, 2019-03-25 at 12:59 +0100, Dridi Boukelmoune wrote: > And since RPM appears to be configurable for the > default interpreter, have it use /usr/bin/bash by default. TBH, it seems to me reasonable that we just do this. If our position is that we actually expect Fedora package scriptlets to be executed by bash and don't think it's a problem if they don't work when executed by some other shell, why not this make this clear and explicit in this way instead of having the default be sh, but then tell people sh must be bash?

On 3/21/2019 3:23 PM, Richard W.M. Jones wrote: So what? On Fedora /bin/sh is bash, and bash is a fine shell. All this nonsense of using dash for /bin/sh on Debian is IMO a pointless bunch of make-work. Fedora has certainly made a lot of make-work projects over the last decade, under arguably more questionable reasoning.

IMO the situation that we're in now ("Assume you're running in bash, but called as -/bin/sh") is a worst-of-both-worlds middle ground, somewhat akin to mandating webpages be written in IE Quirks Mode for all time. It's neither pedantically correct, nor flexible for users and downstreams. And the resolution from all of this last time remains lacking in the guarantees that an independent spec should have:

[Also when giving one graph for one type of build, could you also give a similar graph showing how it looks with dash or ksh or whatever you used?]

On Mon, Mar 25, 2019 at 08:18:34PM +0000, Tomasz Kłoczko wrote: > Switching to other than bash sh interpreter allow reduce total gcc package > build time by ~5%. OK. But that just shows that it is — possibly — worth to switch the gcc build to a different shell, by working with gcc upstream. Nothing should be extrapolated from this to other packages and in particular to their spec files.

If you can take a one-time hit to remove bashisms and get a 25-40% improvement,

Le 2019-03-25 22:47, Japheth Cleaver a écrit : > If you can take a one-time hit to > remove bashisms and get a 25-40% improvement, CPU time is cheap, packager time is not. Exchanging CPU time for "you all should learn to write POSIX-only shell scripts" would be an awful deal. The Java part of Fedora is slowly imploding right now because a lot of people pushed their complexity on packagers, and the packagers could not cope. The Fedora target should be to help packagers achieve more with less work, not achieve less with more work.

Packager time is not cheap, it's not inexhaustible, it runs out. Wasting it on bashisms is not smart.

I was rather thinking about all the damage made on the java ecosystem by the introduction of maven but fair enough.

As far as actually making changes in specs, let's try addressing that separately and perhaps asynchronously. Most of the changes for things likely to run in scriptlets are probably just straight mechanical replacements. (We won't know until someone does a full scan, but I'd bet if you're writing *really* complex stuff there, you've probably already moved that to a packaged script that's executed as part of the scriptlet instead.)

The tooling around the original ecosystem seemed to have no rhyme or reason with it, JPackage efforts never *fully* went anywhere,

On Tue, Mar 26, 2019 at 3:44 AM Nicolas Mailhot <nicolas.mailhot(a)laposte.net&gt; wrote:

> POSIX is dead as a shell compatibility target. You want to replace > bash > with something faster, by all means do it. With something that > includes > the GNU extensions like pushd/popd that most packagers expect today. Is there any reason to *ever* use pushd or popd in %build or %install today?

On Wed, Mar 27, 2019 at 8:33 AM Nicolas Mailhot <nicolas.mailhot(a)laposte.net&gt; wrote: > Le 2019-03-27 03:46, Nico Kadel-Garcia a écrit : >> On Tue, Mar 26, 2019 at 3:44 AM Nicolas Mailhot >> <nicolas.mailhot(a)laposte.net&gt; wrote: >>> POSIX is dead as a shell compatibility target. You want to replace >>> bash >>> with something faster, by all means do it. With something that >>> includes >>> the GNU extensions like pushd/popd that most packagers expect today. >> Is there any reason to *ever* use pushd or popd in %build or %install >> today?

> That is totally the wrong attitude when you want to replace an > implementation used for years by thousands of volunteer people in tens > of thousands of interdependent files. Again, not interested in the side of the discussion about replacing bash as the default interpreter. This is a reminder that I would personally prefer that bash weren't behind /bin/sh. > It is used now, ergo packagers (the people who did the work) find it > useful and convenient. You want them to do something else, you need to I strongly disagree with the shortcut you make here. The only use of pushd/popd I made in scriptlets was because it was part of the Python packaging guidelines and I happen to maintain a bunch of Python packages. So what I was doing then wasn't to choose something I would deem useful and convenient but rather follow guidelines to the best of my abilities. So sure some people may find it useful and convenient, but others may not even be aware or care that this construct is bash-specific so please refrain from authoritatively introduce causality where there isn't. And on the convenience factor: pushd somewhere ... popd is not harder than: cd somewhere ... cd - The bash construct becomes interesting only when you need to stack directories and not keep explicit track of where you are returning to. This convenience I do not wish to remove from package maintainers, even in scriptlets. > make it worth their effort to something else. Winning 10 minutes of CPU > time in a single pathological spec like gcc isn't it. > > The easiest way to make it worth their effort is to reduce the effort to > zero, ie implement the capabilities commonly people use in your target > replacement shell. That will be way way easier than trying to invent > something compelling enough for them to change their habits. 10% of > specs doing something is definitely common use. > > Another way is to take the conversion work unto yourself. But that does > not solve the ongoing effort of helping packagers that try to use bash > syntax in their spec because they need to do something, and find out it > does not work, and give up because the additional work of looking for > alternatives makes the cost/benefit analysis of packaging something > negative. We have many packages where the cost/benefit hovers next to > the limit, because we have nice volunteers that will go to their limits > for Fedora. Conversely whenever as a Fedora end user I run into a failing script on a non Fedora system because it uses GNUisms it makes my life harder to work on the multiple systems I have to deal with. By having bash as /bin/sh Fedora defers the moment when I realize this is happening and pain ensues. That is because Fedora is my main system and others are second-class citizens in my workflow. I'd hate to elect a different one as main my system for obvious reasons. > Yet another is to propose a syntax with is clearly simpler, more > expressive, more productive and better documented for humans (Not CPUs. > CPUs do not get to vote). But, that solves "new spec and macro code" > problem, not the "existing code" problem. > > Hazing people with negative terms like bashism never convinced anyone. I'm sure some comments of this nature actually got a "fair enough" response from a couple people. > Especially when others are doing the work, not you. In my language, that > is called “arriving after the battle”: complaining loudly at the people > who sweated and blooded doing some work, that they didn't do it well > enough, when you were safely somewhere else, at the time help was > needed. I am personally taking an opportunity to raise my concern about the default shell seeing this after-the-battle thread. Maybe I should start a different one on /bin/sh? I was not aware of this until the link was shared in this thread but this is an interesting read on the topic: https://wiki.ubuntu.com/DashAsBinSh If RPM goes somewhere on this topic [1] maybe that could be a change for f31? Dridi [1] https://github.com/rpm-software-management/rpm/issues/646 _______________________________________________ devel mailing list -- devel(a)lists.fedoraproject.org To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Le 2019-03-27 03:46, Nico Kadel-Garcia a écrit : > On Tue, Mar 26, 2019 at 3:44 AM Nicolas Mailhot > <nicolas.mailhot(a)laposte.net&gt; wrote: >> POSIX is dead as a shell compatibility target. You want to replace >> bash >> with something faster, by all means do it. With something that >> includes >> the GNU extensions like pushd/popd that most packagers expect today. > > Is there any reason to *ever* use pushd or popd in %build or %install > today? That is totally the wrong attitude when you want to replace an implementation used for years by thousands of volunteer people in tens of thousands of interdependent files.

It is used now, ergo packagers (the people who did the work) find it useful and convenient. You want them to do something else, you need to make it worth their effort to something else. Winning 10 minutes of CPU time in a single pathological spec like gcc isn't it.

Yet another is to propose a syntax with is clearly simpler, more expressive, more productive and better documented for humans (Not CPUs. CPUs do not get to vote). But, that solves "new spec and macro code" problem, not the "existing code" problem. Hazing people with negative terms like bashism never convinced anyone. Especially when others are doing the work, not you. In my language, that is called “arriving after the battle”: complaining loudly at the people who sweated and blooded doing some work, that they didn't do it well enough, when you were safely somewhere else, at the time help was needed.

On Mon, Mar 25, 2019 at 08:18:34PM +0000, Tomasz Kłoczko wrote: > Switching to other than bash sh interpreter allow reduce total gcc package > build time by ~5%. OK. But that just shows that it is — possibly — worth to switch the gcc build to a different shell, by working with gcc upstream. Nothing should be extrapolated from this to other packages and in particular to their spec files.

>What does this 42 means in this case? It means that during whole gcc build >are repeated 42 times some subset of *autoconf tests*. How it was possible >to loose that?!? 🤔 >gcc is quite monolithic and it should have only one configure.ac. Yes, Why?

The GCC tree contains lots of quite separate subsystems, which are maintained semi-independently by different groups of people. It makes sense for me to maintain libstdc++-v3/configure.ac and libstdc++-v3/acinclude.m4 without worrying how my changes affect the rest of the tree.

On 26/03/19 11:40 +0000, Tomasz Kłoczko wrote: >On Tue, 26 Mar 2019 at 08:57, Jonathan Wakely <jwakely(a)fedoraproject.org&gt; >wrote: >[..] > >> >What does this 42 means in this case? It means that during whole gcc build >> >are repeated 42 times some subset of *autoconf tests*. How it was possible >> >to loose that?!? 🤔 >> >gcc is quite monolithic and it should have only one configure.ac. Yes, >> >> Why? >> > >Really? >Really do you want me to answer on the question "why there is no any sense >repeat 42 times some tests on the source code configuration stage?" ?? Yes, because you repeatedly make the mistake of assuming that one dimension of a problem is the only one that matters, and that all other considerations are irrelevant.

On Wed, 27 Mar 2019 at 21:27, Jonathan Wakely <jwakely(a)fedoraproject.org&gt; wrote: > On 26/03/19 11:40 +0000, Tomasz Kłoczko wrote: > >On Tue, 26 Mar 2019 at 08:57, Jonathan Wakely <jwakely(a)fedoraproject.org&gt; > >wrote: > >[..] > > > >> >What does this 42 means in this case? It means that during whole gcc > build > >> >are repeated 42 times some subset of *autoconf tests*. How it was > possible > >> >to loose that?!? 🤔 > >> >gcc is quite monolithic and it should have only one configure.ac. Yes, > >> > >> Why? > >> > > > >Really? > >Really do you want me to answer on the question "why there is no any sense > >repeat 42 times some tests on the source code configuration stage?" ?? > > Yes, because you repeatedly make the mistake of assuming that one > dimension of a problem is the only one that matters, and that all > other considerations are irrelevant. > Just to be clear .. So you want to say that I'm making mistake because I'm assuming that speed/performance matters?

Could you please confirm that is is what you are thinking reading what I wrote? Could you please as well crush my assumption using logical arguments and facts?

This is fine, because /bin/sh is a symbolic link to /usr/bin/bash on Fedora. I always use pushd/popd in my packages.

On Thu, Mar 21, 2019 at 5:05 PM, Tomasz =?UTF-8?b?S8WCb2N6a28=?= <kloczko.tomasz(a)gmail.com&gt; wrote: > [tkloczko@domek SPECS.fedora]$ rpm -E %_buildshell > /bin/sh How can this discussion still be ongoing? Why not just change it to /bin/bash and move on?

> Japheth Cleaver explained why in response to me a couple of days ago: > apparently changing it would also change the shell used for some > scriptlets... He also posted this link: http://web.archive.org/web/20150821020837/http://rpm.org/ticket/877

If yes I can propose simpler solution like create .post_build_preserve.lst file in source build root and whatever will be added to that file should be archived in some <package>-<version>-<release>.tar.xz file to be accessible over koji web interface. If such file will be created during manually executed "rpmbuild -ba" it will not leave any garbage behind or will be no flooding stdout during build.

On Tue, 2019-03-26 at 09:08 -0500, mcatanzaro(a)gnome.org wrote: > On Thu, Mar 21, 2019 at 5:05 PM, Tomasz =?UTF-8?b?S8WCb2N6a28=?= > <kloczko.tomasz(a)gmail.com&gt; wrote: >> [tkloczko@domek SPECS.fedora]$ rpm -E %_buildshell >> /bin/sh > How can this discussion still be ongoing? Why not just change it to > /bin/bash and move on? Japheth Cleaver explained why in response to me a couple of days ago: apparently changing it would also change the shell used for some scriptlets...

On Tue, 2019-03-26 at 09:08 -0500, mcatanzaro(a)gnome.org wrote: > On Thu, Mar 21, 2019 at 5:05 PM, Tomasz =?UTF-8?b?S8WCb2N6a28=?= > <kloczko.tomasz(a)gmail.com&gt; wrote: >> [tkloczko@domek SPECS.fedora]$ rpm -E %_buildshell >> /bin/sh > How can this discussion still be ongoing? Why not just change it to > /bin/bash and move on? Japheth Cleaver explained why in response to me a couple of days ago: apparently changing it would also change the shell used for some scriptlets...