On Sun, Apr 28, 2019 at 4:57 PM Zbigniew Jędrzejewski-Szmek <zbyszek(a)in.waw.pl&gt; wrote: I've worked in distributions that have implemented this policy, and it's often much more frustrating to work with because of it. It's not always obvious that pkgconfig missing is the reason why stuff doesn't build or otherwise work correctly. I'm pretty sure the reason for having the auto-generated requirement on pkg-config was to make it easy and obvious to use stuff through pkgconfig. I'm personally not in favor of doing this. I know that recently there has been this kick to shrink the default buildroot and minimize the dependency chain in extreme ways. I'm fairly certain a chunk of this has to do with RHEL modularity, but there's also the obvious bit of speeding up buildroot construction. But to be honest, this isn't a significant drain on the dep chain (if we still had the other pkgconfig implementation, it might be, since that pulled in glib2 in the minimal tree). If the problem is pkgconf-m4, I could be persuaded to drop pkgconf-pkg-config's requirement on it. I kept it there principally because the old pkgconfig package had the m4 file in there too. The impact is minimal, it's a developer-side dependency anyway, and it's often useful to have. So from my point of view, I don't see a problem with it. And the dependency generator already makes it "/usr/bin/pkg-config", so you could theoretically swap it with any other conforming implementation. In the end, I don't see any reasonable benefit for doing this, and it just makes developer and packaging work more frustrating. Also, w.r.t. so files, we _do_ pull in the linker library. The library "ld-linux-$ARCH.so.$SOMAJOR" is always pulled in by every rpm containing a library. -- 真実はいつも一つ！/ Always, there's only one truth!

On Sun, Apr 28, 2019 at 11:04 PM Zbigniew Jędrzejewski-Szmek < zbyszek(a)in.waw.pl&gt; wrote: It would be nice to have something like Requires: (pkg-config if pkgconfig(*)) which would install pkgconf if any packages providing pkgconfig(…) are installed.. But I'm afraid we can't do that now. Zbyszek

On Mon, Apr 29, 2019 at 5:04 AM Lennart Poettering <mzerqung(a)0pointer.de&gt; wrote: Correct. pkgconf-pkg-config owns all the base paths for pkgconfig content. -- 真実はいつも一つ！/ Always, there's only one truth!

On Mon, Apr 29, 2019 at 8:05 AM Zbigniew Jędrzejewski-Szmek <zbyszek(a)in.waw.pl&gt; wrote: It does this specifically because a pc file is shipped in the main package. That's a pretty rare circumstance. -- 真実はいつも一つ！/ Always, there's only one truth!

On Mon, Apr 29, 2019 at 02:00:41PM -0500, Rex Dieter wrote: Obviously. I would do most of the work myself. Zbyszek

On Mon, Apr 29, 2019 at 9:20 PM Zbigniew Jędrzejewski-Szmek < zbyszek(a)in.waw.pl&gt; wrote: I think it would make sense to avoid the autogenerated dep just for the directory ownership. It's a fairly tedious thing to change though: update %files lists for each package that ships .pc files, update build requires for packages that now need to list BuildRequires: pkg-config, update the rpm dep generator. And then after all of that is done, verify that things still build, now that the autogenerated dep is gone from everything in the build root. It's a lot of changes to make, but if you want to go through the pain of doing all of that then I don't mind :) Also, what should the BR be? BuildRequires: /usr/bin/pkg-config BuildRequires: pkg-config BuildRequires: pkgconfig BuildRequires: pkgconf BuildRequires: pkgconf-pkg-config Kalev

On Tue, 30 Apr 2019 at 18:19, Neal Gompa <ngompa13(a)gmail.com&gt; wrote: [..] And what is wrong with just "BuildRequires: pkgconfig"? Don't you see that this forces dependencies resolution to use packages files list? IIRC Fedora already had policy to not use paths in BRs. [tkloczko@domek SPECS]$ rpm -qf /usr/bin/pkg-config pkgconf-pkg-config-1.6.1-1.fc31.x86_64 [tkloczko@domek SPECS]$ rpm -q --whatprovides pkgconfig pkgconf-pkg-config-1.6.1-1.fc31.x86_64 kloczek -- Tomasz Kłoczko | Tel: 0774 1209067 | LinkedIn: http://lnkd.in/FXPWxH

On Tue, 30 Apr 2019 at 21:04, Neal Gompa <ngompa13(a)gmail.com&gt; wrote: You just put in my hand +1 to not use paths in BuildRequires. +1 for me. Thx. If you will look across /usr/lib/macros and files in /usr/lib/rpm/macros.d you will find hundreds of other commands used with full path and only occasionally some of them are used in BRs in form "BuildRequires: /some/path/command". Are going to request/propose to change whole distribution pattern of BRs? Really? FYI: We call that thing which you are refusing to use .. abstraction. <counerargument=WRONG> rpmbuild is not using dnf xml files :-/ </counerargument> You are again talking about *install package time dependencies resolution*. Command rpmbuild which is processing spec files is using *rpm database*. We are talking about packages *build time dependencies resolution*. Please do me a favourite and execute on your system from root "rpm --rebuilddb; ls -l /var/lib/rpm/{Basenames,Providename}" and answer on the question: query to which one rpm table has chance to be shorter? kloczek -- Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

On Wed, 1 May 2019 at 03:24, Neal Gompa <ngompa13(a)gmail.com&gt; wrote: [..] Calling rpmbuild with --nodeps to be able to resolve build dependencies from outside chroot and duplicating resolution of those dependencies by adding more code in yum/dnf to handle situation when rpm database in chroot is in different version is really .. "interesting". Just checked all Fedora spec files to find few of them which are using straight rpm command in %build or %check. One of those specs is elfutils.spec in which is: %check # Record some build root versions in build.log uname -r; rpm -q glibc In https://kojipkgs.fedoraproject.org//packages/elfutils/0.176/2.fc31/data/l... is possible to find: Executing(%check): /bin/sh -e /var/tmp/rpm-tmp.F4TimI + umask 022 + cd /builddir/build/BUILD + cd elfutils-0.176 + uname -r 5.0.6-200.fc29.x86_64 + rpm -q glibc glibc-2.29.9000-17.fc31.x86_64 I'm really happy that rpm database still is available inside mock chroot (ufff). During +25 years of using rpm it was at least two times situation when rpm was in kind of transition and in all those cases to build packages using my own automation I was always able to use just chroot command, and to be honest I would never even think about use "that way". If not using simplest "BuildRequires: pkgconfig" may be (somehow) affected by above (which I'm 100% sure that it is still not the case because still above isn't by any way kind of counterargument against "BuildRequires: pkgconfig") .. I think that I'll give up. kloczek -- Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

On Wed, 1 May 2019 at 09:04, Nicolas Mailhot <nicolas.mailhot(a)laposte.net&gt; wrote: [..] IMO maintainer in this case is 100% right because all properties of the build env already can be described in with enough precision. Providing more details about build env will be duplicating current build dependencies. Discussion about such cases should start from analyse of some case(s) when such details could help with some class of issues. Then someone should go to root cause that some exact failed/produced broken packages, and then (maybe) at the end some solution can be provided. All those context details are missing in that issue ticket so (IMO) rpm maintainer had no even chance understand what caused that someone started thinking about storing such metadata in src.rpm. BTW elfutils and glibc package EVR. This detail is already part of the koji root.log. https://kojipkgs.fedoraproject.org//packages/elfutils/0.176/2.fc31/data/l... contains line: DEBUG util.py:556: glibc x86_64 2.29.9000-17.fc31 build 3.9 M Nevertheless some people forgets sometimes that spec syntax provides not only BuildRequire but BuildConflict token as well. With those two is possible with 100% accuracy describe build env properties. That is all what rpmbuild needs to provide (IMO). [tkloczko@domek SPECS.fedora]$ grep ^BuildConflict * | wc -l 38 Some packages source code build automation implements sometimes strange things. For example gawk is using libsigsegv only if autoconf is able to find it, and it is not possible to disable using it by provide configure parameter. However there is only handful of such cases and it is easier to fix those packages source code build automation than designing rpm extensions to handle such cases straight on packaging layer. Uniformity below packaging layer should be our real goal .. this why when I see that in git repos someone is using some strange tagging convention like replace dots by underscores or putting build automation deep below project root directory (examples: recently introduced meson and cmake files in lz4 and zstd) I'm always trying gently ask maintainers to change this to implement and use those bits in kind of StandardWay(tm). All because it makes packaging easier with virtually no costs on layer below. Other issue is that as Fedora keeps only last versions/releases of all packages and even if someone would be storing full list of all installed packages in build env in koji gut it will have very limited abilities to help with something as it is not possible quickly reassembly from scratch exactly the same build env out of the available packages. So question which only left from this branch of the discussion is: What was the exact initial impulse which pushed someone to idea of storing full build env properties in src.rpm? IMO someone have been only thinking that this could solve something (that someone had only incorrect impression that it would help with something). kloczek -- Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

On Thu, 2 May 2019 at 20:29, Nicolas Mailhot via devel <devel(a)lists.fedoraproject.org&gt; wrote: [..] As I wrote problem only is that without possibility really rollback to the full state described in set of exact N-E:V-Rs packages recorded data such auditing data in case if something starts going wrong such data could be used only as kind of murky vector where possible cause really is. It is nothing wrong with keeping only latest versions of the packages but with that assumption to have enough effectiveness of catching errors/issues needs to be added more test units fired straight after package build is done or during that process. Just look on some numbers below: [tkloczko@domek SPECS.fedora]$ grep ^%check *| wc -l; ls -1| wc -l 11692 21300 and the same stats on results of my own work: [tkloczko@domek SPECS.g2v]$ grep ^%check *| wc -l; ls -1| wc -l 426 498 Do you see where my thoughts are heading? With that only in last 3-4 months I've been able to catch maybe even few tenths of new issues just right on build packages (you can check stat of my accounts on gitlab and github to be able spot that part of my activity). I fully understand why Fedora keeps only latest versions of the packages and it is nothing wrong with that. Problems only is that if that small bit must be kind of assumption other parts around needs to be readjusted to create kind of strategy catching and dealing with new issues. And again .. with assumption that during packages development process never would be possible to fully rollback to the same N-E:V-Rs state storing build time metadata is kind of pointless (IMO). In other words reproducibility of those N-E:V-Rs metadata in case of Fedora are very close to *null*. BTW: storing all N-E:V-Rs of the *packages* could be easily solved by move away from rpm to IPS and I know that in case of the Fedora that kind of change is unrealistic. Adapting rpm to use packages repository like it is in case of IPS is unrealistic as well .. because to many things around needs to be changed (not only on packaging and distribution layer). You can observe this kind of effect on for example Sol 11.4 packages http://pkg.oracle.com/solaris/release/en/ The same web interface which immanent part of the IPS provides very long list of all version of the packages if you would be able to observe all Solaris SRUs (Service Recommended Updates) data. On Solaris 10 with IPS is possible even rollback to the state ~10 years old. Full rollback to the system state of all packages from the past is sometimes really important and this is why in OL repositories are never deleted older packages (yum/dnf repo allows keep multiple E:VRs of the same package in single repository). kloczek -- Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

Le vendredi 03 mai 2019 à 09:46 +0100, Tomasz Kłoczko a écrit : There's nothing murky or wrong in beiang able to answer "which static builds have used this rotten package" without needing to contact a centralised system like koji or IPS. You're assuming the only use is roolback. It's not -- Nicolas Mailhot

Le vendredi 03 mai 2019 à 12:04 +0100, Tomasz Kłoczko a écrit : You use apps in one of those languages that static build by default. There is a security alert in one code component. You want to know which packages in your repo/mirror have been build using the broken piece of source code -- Nicolas Mailhot

On Fri, May 3, 2019 at 8:26 AM Tomasz Kłoczko <kloczko.tomasz(a)gmail.com&gt; wrote: Nope. Between Go and Rust, you have literally thousands of static linked dependencies, so it does matter in that case. -- 真実はいつも一つ！/ Always, there's only one truth!

Le vendredi 03 mai 2019 à 19:59 +0200, Dridi Boukelmoune a écrit : That makes it stack-specific And anyway, the classical compiler attack (compiler that inserts backdoor while compiling) shows that special-casing some packages for special tracking does not work, pretty much anything that existed in the build root need to be tracked because it may be exploited one way or another, and spead the exploit to everything that used it. -- Nicolas Mailhot

On Tue, Apr 30, 2019, at 10:14 PM, Neal Gompa wrote: Awesome! I love learning how these things work! Thanks for sharing! V/r, James Cassell

On Tue, Apr 30, 2019 at 9:47 AM Tomasz Kłoczko <kloczko.tomasz(a)gmail.com&gt; wrote: Note that removing /usr/bin/pkg-config from the build environment also stops pkgconfig() Provides/Requires from being generated. And while I am a provenpackager and capable of making these kinds of changes, I won't if I think they're not a good idea (like this one). -- 真実はいつも一つ！/ Always, there's only one truth!

On Tue, Apr 30, 2019 at 8:50 AM Neal Gompa <ngompa13(a)gmail.com&gt; wrote: Indeed, synthesizing pkgconfig() entries based on the presence of .pc files is not necessarily sufficient enough. .pc files can define aliases using their own Provides rules. The information is best generated by using pkgconf to dump the dependency graph. Similarly, the requirements may as well be generated by using pkgconf instead of trying to parse the .pc files with grep and sed itself as the syntax is not necessarily consistent. Many of the 'hotfix' releases in pkgconf involved mitigating problems where our parser would get confused by unexpected data in .pc files. William

On Wed, May 1, 2019, 6:15 AM Zbigniew Jędrzejewski-Szmek <zbyszek(a)in.waw.pl&gt; wrote: I'm aware of that. Other dependency generators in the past have hit issues trying to parse .pc files by hand. My point is that it's better to just place the 200KB of utilities (staticly linking pkgconf may reduce that size if it is super important) RPM needs to get the job done right is better than trying to hack up a parser with shell tools. William

On Tue, Apr 30, 2019 at 2:34 AM Vít Ondruch <vondruch(a)redhat.com&gt; wrote: If it is made into a weak dependency (like Recommends or Suggests), then it doesn't show up in build roots, since we don't have weak dependencies on there. If it is Suggests, then _no one_ gets it, not even regular users... -- 真実はいつも一つ！/ Always, there's only one truth!