I'm building a pkg on COPR.
It uses forgemeta/scm (from git) for a source.
On build exec, it fails
sh: git: command not found
sh: git: command not found
sh: git: command not found
sh: git: command not found ...
( details -> https://pastebin.com/vUCVVHQq , line #1806 )
Note, the spec _does_ have "BuildRequires: git".
It appears this^ may have to do with forgemeta needing git earlier? Note that foregemeta source pulls, subsequent to the error, fail ...
Does COPR need an earlier preload of 'git' for forgemeta SCM use by builds?
If that _is_ the case, where/how does one add a 'early' git req't for COPR builds?
On Tuesday, June 30, 2020 8:46:10 PM CEST PGNet Dev wrote:
I'm building a pkg on COPR.
It uses forgemeta/scm (from git) for a source.
On build exec, it fails
sh: git: command not found
sh: git: command not found
sh: git: command not found
sh: git: command not found ...
( details -> https://pastebin.com/vUCVVHQq , line #1806 )
Note, the spec _does_ have "BuildRequires: git".
It appears this^ may have to do with forgemeta needing git earlier? Note that foregemeta source pulls, subsequent to the error, fail ...
Error output like that is usually "a bug" in macro implementation: https://github.com/rpm-software-management/rpm/issues/424 Usually those macros are no-ops at SRPM build time, so you can ignore these errors. As long as the error output really is innocent.
This build failure seems to be described by the error: error: You need to define %{version1}, or %{tag1} before the macro invocation ! error: lua script failed: /usr/lib/rpm//lua/fedora/srpm/forge.lua:165: error expanding macro
But I'm not an expert on %forgementa macros (it's pretty new thing). I'd suggest you to contact Nicolas Mailhot on fedora devel list.
Does COPR need an earlier preload of 'git' for forgemeta SCM use by builds?
Copr rebuilds srpm first, from copr dist-git - in limited buildchroot.
If that _is_ the case, where/how does one add a 'early' git req't for COPR builds?
Probably. Note that it would be better to not remove the build next time so we could experiment and try to reproduce the build.
Pavel
copr-devel mailing list -- copr-devel@lists.fedorahosted.org To unsubscribe send an email to copr-devel-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/copr-devel@lists.fedorahosted.o...
On 7/1/20 12:16 AM, Pavel Raiskup wrote:
On Tuesday, June 30, 2020 8:46:10 PM CEST PGNet Dev wrote:
I'm building a pkg on COPR.
It uses forgemeta/scm (from git) for a source.
On build exec, it fails
sh: git: command not found
sh: git: command not found
sh: git: command not found
sh: git: command not found ...
( details -> https://pastebin.com/vUCVVHQq , line #1806 )
Note, the spec _does_ have "BuildRequires: git".
It appears this^ may have to do with forgemeta needing git earlier? Note that foregemeta source pulls, subsequent to the error, fail ...
Error output like that is usually "a bug" in macro implementation: https://github.com/rpm-software-management/rpm/issues/424 Usually those macros are no-ops at SRPM build time, so you can ignore these errors. As long as the error output really is innocent.
in my current example, the build FAILS. so unable to continue ...
This build failure seems to be described by the error: error: You need to define %{version1}, or %{tag1} before the macro invocation ! error: lua script failed: /usr/lib/rpm//lua/fedora/srpm/forge.lua:165: error expanding macro
But I'm not an expert on %forgementa macros (it's pretty new thing). I'd suggest you to contact Nicolas Mailhot on fedora devel list.
in this particular case, git's needed to get/set the commit #. hence the 'need' for git 'before' ...
if I don't get this figure out soon, I'll take it to other list. thx.
Does COPR need an earlier preload of 'git' for forgemeta SCM use by builds?
Copr rebuilds srpm first, from copr dist-git - in limited buildchroot.
as does mock, right?
as I'm getting used to the Fedora build ecosystem, my current workflow is ...
rmpbuild locally
if rmpbuild ok, mock build
if mock build ok, COPR build
a mock build and COPR build _should_ be identical(-ish) ?
If that _is_ the case, where/how does one add a 'early' git req't for COPR builds?
Probably. Note that it would be better to not remove the build next time so we could experiment and try to reproduce the build.
sry, as i'm changing a lot, i'm 'cleaning house' often -- & I didn't _assume_ that anyone would bother to actually try!
in any case, here's a current fail,
https://download.copr.fedorainfracloud.org/results/pgfed/nginx-mainline/fedo...
again, that same spec builds without error _here_, as a mock build.
now to figure out the differences ...
On Wednesday, July 1, 2020 4:03:08 PM CEST PGNet Dev wrote:
Does COPR need an earlier preload of 'git' for forgemeta SCM use by builds?
Copr rebuilds srpm first, from copr dist-git - in limited buildchroot.
as does mock, right?
as I'm getting used to the Fedora build ecosystem, my current workflow is ...
rmpbuild locally
if rmpbuild ok, mock build
if mock build ok, COPR build
a mock build and COPR build _should_ be identical(-ish) ?
By saying those statements, you keep more things unspecified than specified. The situation isn't that easy:
1. Copr imports (or rather "caches") sources into it's internal dist-git:
a) By uploading only spec file (your case), you rely on the fact that copr is able to - using provided spec file - download all the needed tarballs. This is tricky part, and can take any amount of time. But we do this only once for all the selected target chroots.
b) By uploading src.rpm, copr just goes and imports the data to dist-git. This may take some time, depending on the connectivity. But it is more reliable than a) as you are 100% sure what you upload, and then build from.
2. Copr builder builds a source RPM in mock chroot from dist-git. The chroot is decided based on the target chroot you build for (so if you requested rawhide x86_64, the src.rpm is build in rawhide x86_64 as well first). This should be an easy task since all the tarballs and specfiles are already cached and available on a very close location to the builder (in copr dist-git).
3. Using src.rpm from step 2, copr builds the RPMs in target chroot.
So if you build locally using `rpmbuild`, we know nothing about your build environment, about the tarballs you have pre-downloaded, time of your build, etc. Mock then helps you to prepare the environment, but from pre-generated src.rpm! So if this succeeds, and you have correctly set local environment, you have correct src.rpm to build from.
And so if you delivered exactly that src.rpm to copr, it _should_ succeed as well as it succeeds in mock locally. But if you upload just the spec file, you need to rely on the fact the 1.a) step does the same thing you did locally.
in any case, here's a current fail,
https://download.copr.fedorainfracloud.org/results/pgfed/nginx-mainline/fedo...
again, that same spec builds without error _here_, as a mock build.
now to figure out the differences ...
... and this failure probably means that _when_ you uploaded just the spec file and copr downloaded all the sources for you automatically, _some variant_ of sources were downloaded and imported (and cached) to dist-git.
I bit later, using the same spec file and tarballs (from cache) from dist-git, copr builds source RPM in mock (see the log, --buildsrpm). But the very same spec file probably newly references _different_ sources than those cached variants, because it tries to download something (once more, it shouldn't now, all the sources should be already fetched from caches).
The question is why this is happening; why your specfile references different sources in slightly different time (or in different environment).
Perhaps you could (I'm not 100% sure) unblock your builds by enabling network connection for your RPM builds (check the project web-UI settings page). But that is a very bad thing to do, considering that the source rpm should be already correctly imported and nothing should be re-downloaded.
You need to debug what is going on here, this is behind the support edge for copr maintainers. I'd suggest you to compare the source RPMs you and copr works with. Compare the local sources, spec files, etc. Try to build from uploaded source.rpm. We can lend you some copr builder temporarily if you need it for debugging (so you can try the `copr-rpmbuild` commands manually).
Pavel
On 7/1/20 8:10 AM, Pavel Raiskup wrote:
By saying those statements, you keep more things unspecified than specified. The situation isn't that easy:
(snip)
this is a _very_ useful explanation.
i had not yet wrapped my head around these details from the docs I'd managed to find/read.
thx!
now to figure out the differences ...
...
The question is why this is happening; why your specfile references different sources in slightly different time (or in different environment).
Perhaps you could (I'm not 100% sure) unblock your builds by enabling network connection for your RPM builds (check the project web-UI settings page).
It already is.
But that is a very bad thing to do, considering that the source rpm should be already correctly imported and nothing should be re-downloaded.
Hm.
I'm used to a system (OBS) where I push the _spec_ file online, specify the source pulls (@OBS, with _service files; similar (?) to forgemeta), and specify the build env / repos used.
As long as the online build env matched my local env, the result's been the same.
I need to revisit/rethink this. It seems that SRPMS are the preferred 'input' ...
Your comments above, again, will be helpful.
You need to debug what is going on here, this is behind the support edge for copr maintainers.
My _goal_ is to get local, mock & COPR builds generating the _same_ results. Ideally, using the available forgemeta constructs.
I'm guessing that consistent build results is not an uncommon goals.
But, sure. I get that resources are limited.
I'd suggest you to compare the source RPMs you and copr works with. Compare the local sources, spec files, etc.
Well THAT much i've been doing :-)
Try to build from uploaded source.rpm.
yep, as above.
We can lend you some copr builder temporarily if you need it for debugging (so you can try the `copr-rpmbuild` commands manually).
lol. still a Fedora-land noob, here. I have no idea what 'lend you some copr builder' means or entails! yet ...
thx agn!
On Wednesday, July 1, 2020 6:22:31 PM CEST PGNet Dev wrote:
You need to debug what is going on here, this is behind the support edge for copr maintainers.
My _goal_ is to get local, mock & COPR builds generating the _same_ results. Ideally, using the available forgemeta constructs.
I'm guessing that consistent build results is not an uncommon goals.
Definitely. But the forgemeta (triggering dynamic source downloads) thing brings a lot of variables in to the game.
I need to revisit/rethink this. It seems that SRPMS are the preferred 'input' ...
I don't want to claim `src.rpm` upload is preferred method (both methods should just work), it is just more straightforward and there's less space to make a mistake (both on build submitter and copr sides).
I'd suggest you to compare the source RPMs you and copr works with. Compare the local sources, spec files, etc.
Well THAT much i've been doing :-)
I mean, compare youre src.rpm with copr-generated src.rpm (rpmdiff, e.g.). And any peculiarity.
We can lend you some copr builder temporarily if you need it for debugging (so you can try the `copr-rpmbuild` commands manually).
lol. still a Fedora-land noob, here. I have no idea what 'lend you some copr builder' means or entails! yet ...
I mean, if you (a) contact us off-list with your ssh key, (b) we can allocate a copr builder VM for you, (c) setup the ssh access there for you and (d) than throw the VM away when you don't need it. If it makes your debugging easier...
Pavel
On 7/1/20 10:23 AM, Pavel Raiskup wrote:
I'm guessing that consistent build results is not an uncommon goal.
Definitely.
good to hear.
But the forgemeta (triggering dynamic source downloads) thing brings a lot of variables in to the game.
noted.
I need to revisit/rethink this. It seems that SRPMS are the preferred 'input' ...
I don't want to claim `src.rpm` upload is preferred method (both methods should just work), it is just more straightforward and there's less space to make a mistake (both on build submitter and copr sides).
ok. then i'll continue to poke at this until both "just work".
I mean, compare youre src.rpm with copr-generated src.rpm (rpmdiff, e.g.). And any peculiarity.
yup.
We can lend you some copr builder temporarily if you need it for debugging (so you can try the `copr-rpmbuild` commands manually).
lol. still a Fedora-land noob, here. I have no idea what 'lend you some copr builder' means or entails! yet ...
I mean, if you (a) contact us off-list with your ssh key, (b) we can allocate a copr builder VM for you, (c) setup the ssh access there for you and (d) than throw the VM away when you don't need it.
ah, thx!
If it makes your debugging easier...
dunno yet.
if it does, i'll contact as offered.
o/
copr-devel@lists.fedorahosted.org
-
Pavel Raiskup -
PGNet Dev