4:06 p.m.
Just forwarding this question to more appropriate list.
---------- Forwarded Message ----------
Subject: %forgemeta support for `git` tasks in checked-out code?
Date: Thursday, June 25, 2020, 6:42:42 PM CEST
From: PGNet Dev <pgnet.dev(a)gmail.com>
To: copr-devel(a)lists.fedorahosted.org
I'm building a package using multiple sources.
I'm using $forgemeta to get/manage sources, as per example here:
https://docs.fedoraproject.org/en-US/packaging-guidelines/SourceURL/#_mul...
One of my sources pulls from upstream scm (github), and needs a subsequent 'git
submodule--init'.
Is seems that what %forgemeta pulls/tars/expands is not initilialize git repo; rather,
just the relevant target's files.
E.g., for a given scm source (here, ngx_brotli), the %forgemeta pull unpacks
tree -L 1 -a ngx_brotli
ngx_brotli
├── config
├── CONTRIBUTING.md
├── deps
├── filter
├── .gitmodules
├── LICENSE
├── README.md
├── script
├── static
└── .travis.yml
note, no ".git" folder, and no ability as is to `git submodule --init` in the
tree.
otoh, a `git clone` of the same source results, as expected, in
tree -L 1 -a ngx_brotli
ngx_brotli
├── config
├── CONTRIBUTING.md
├── deps
├── filter
!! ├── .git
├── .gitmodules
├── LICENSE
├── README.md
├── script
├── static
└── .travis.yml
currently, to workaround this, I DISABLE the forge for the source in question, and do the
pull & submodule init in %prep,
e.g., this spec
%global forgeurl0 https://github.com/nginx/nginx
Version: 1.19.0
%global tag0 release-%{version}
%global forgeurl1 https://github.com/openresty/headers-more-nginx-module
%global tag1 master
# DISABLE global forgeurl1 https://github.com/google/ngx_brotli
%global tag2 master
%forgemeta -i -z 0
%forgemeta -i -z 1
# DISABLE forgemeta -i -z 2
Name: %{name}
%define dist %{scm0}
Release: %{dist}
Summary: %{name}
License: BSD
URL: %{forgeurl0}
Source0: %{forgesource0}
Source1: %{forgesource1}
Source2:
https://github.com/google/ngx_brotli/archive/master/ngx_brotli-master.tar.gz
...
%prep
pushd .
cd %{_sourcedir}
rm -rf ngx_brotli
git clone https://github.com/google/ngx_brotli
cd ngx_brotli
git submodule update --init
cd ..
tar --remove-files -czf ngx_brotli.tar.gz ngx_brotli
popd
%setup -T -b 0 -n nginx-%{tag0}
%setup -T -b 1 -n headers-more-nginx-module-%{tag1}
...
%build
export DESTDIR=%{buildroot}
cd %{_builddir}/%{name}-%{tag0}
...
%install
cd %{_builddir}/%{name}-%{tag0}
make install DESTDIR=%{buildroot} INSTALLDIRS=vendor
...
appears to work, and the `git submodule --init` is taken care of correctly.
but, it's messy.
_is_ there a %forgemeta alternative to the manual git clone that will init/unpack the
source _as_ a repo, allowing the submodule task to be exec'd?
this isn't, of course, ONLY a COPR question, but one of rpmbuild more generally.
_my_ goal _is_ to get it working on COPR, eventually.
that said, if this^ belongs on some other list, let me know.
_______________________________________________
copr-devel mailing list -- copr-devel(a)lists.fedorahosted.org
To unsubscribe send an email to copr-devel-leave(a)lists.fedorahosted.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/copr-devel@lists.fedorahoste...
-----------------------------------------
1:58 a.m.
New subject: Fwd:%forgemetasupportfor`git`tasksinchecked-outcode?
Hi,
forgemeta works in release mode, with release archives published over
http(s). It does not talk at all to source projects using the git
protocol (and that is intentional, since a lot ot things tooling-side
do not talk the git protocol and will never talk the git protocol,
starting with rpm itself, spectool, etc).
git is not the only scm system out there. Nor its protocol is set in
stone. It would be very tricky to push it down to layers like spec
files that operate on timescales measured in years/decades and are used
by a huge amount of third-party tools that expect http(s) source URLs.
From a pure auditing side, it is also correct to have all the sources
of things you package listed explicitly, and not have recursive sources
that install themselves other sources behind auditor’s backs.
Note that your spec as it stands is inherently unsafe since you are
pulling sources from a a moving reference (master) instead of commiting
to a specific commit or tag state¹ (another proof git is not safe to
use as direct packaging layer, since it pretends to give you a fixed
state while submoduling things that are not fixed).
The correct auditable way to use git in rpm is to declare all the git
sources you actually use, with a specific tag or commit hash for each
of them.
So you should have a long list of
%forgeurlX
%tagX or %commitX
and a single %forgemeta -a at the end
That will change soonish BTW, I’m currently doing final testing on code
that will use a slightly different syntax:
%forge_urlX
%forge_tagX or %forge_commitX
%forge_patchlist %{expand:
foo1.patch
foo2.patch
}
…
%auto_init
…
%sourcelist
%auto_sources
%patchlist
%auto_patches
%prep
%auto_prep
Because not making -a the default and emphasising -z in the
documentation was comfusing users. -z should be a last resort debuging
help, not the first thing you look at when packaging multiple sources.
That will leave you with each individual source unpacked and patched in
%{_builddir}, and needing a some symlinks between %{_builddir}
subdirectories to construct a unified source trees, but that’s how
multisource works deep down in rpm, nothing I invented myself.
Regards,
¹ I made it work in the macros because I knew people would attempt to
do it, and I’d rather have it work in my code than have people butcher
it to achiever their aims, but from a Fedora packaging guidelines POW
it is a mistake).
--
Nicolas Mailhot
9:41 a.m.
New subject: Fwd:%forgemetasupportfor`git`tasksinchecked-outcode?
hi,
On 6/25/20 11:58 PM, Nicolas Mailhot wrote:
forgemeta works in release mode, with release archives published
over
http(s). It does not talk at all to source projects using the git
protocol (and that is intentional, since a lot ot things tooling-side
do not talk the git protocol and will never talk the git protocol,
starting with rpm itself, spectool, etc).
noted. not clear initially from reading the docs; this helps. thx!
git is not the only scm system out there.
(snip)
sure. i'm trying to get forge playing nice with not-included-yet hg sources atm :-)
From a pure auditing side
...
Note that your spec as it stands is inherently unsafe since
...
The correct auditable way
...
(snip)
yup. and for the moment -- while I'm getting my 'end product' sorted out --
that's intentional, and I'm not concerned with audit trail.
yet.
point taken otherwise.
So you should have a long list of
%forgeurlX
that's already changed in my latest ...
%tagX or %commitX
fair enuf, now that the above is nice-n-clear ...
and a single %forgemeta -a at the end
was having weird artifacts doing that earlier, so split into one-per-source-target.
once a forge bug (other thread) gets ironed out, that'll get revisited, too.
That will change soonish BTW, I’m currently doing final testing on
code
that will use a slightly different syntax:
%forge_urlX
(snip)
that'll get announced ... here?
do you have a _rough_ timeframe for when that'll be consistently supported/usable in
rpmbuild, mock & COPR?
bunch of moving parts to get in sync ...
Because not making -a the default and emphasising -z in the
documentation was comfusing users. -z should be a last resort debuging
help
it was helpful for debug, here. and at my early stage, necessary ...
not the first thing you look at when packaging multiple sources.
yup
That will leave you with each individual source unpacked and patched
in
%{_builddir}, and needing a some symlinks between %{_builddir}
subdirectories to construct a unified source trees, but that’s how
multisource works deep down in rpm, nothing I invented myself.
that's what i'd naively assumed was to cleanly happen when i'd started with
this multisource spec.
if this cleans that up, then +1 !
... have people butcher it to achiever their aims ...
but, that's _our_ job! ;-)
cheers.
11:01 a.m.
New subject: Fwd:%forgemetasupportfor`git`tasksinchecked-outcode?
Le vendredi 26 juin 2020 à 07:41 -0700, PGNet Dev a écrit :
hi,
On 6/25/20 11:58 PM, Nicolas Mailhot wrote:
> forgemeta works in release mode, with release archives published
> over
> http(s). It does not talk at all to source projects using the git
> protocol (and that is intentional, since a lot ot things tooling-
> side
> do not talk the git protocol and will never talk the git protocol,
> starting with rpm itself, spectool, etc).
noted. not clear initially from reading the docs; this helps. thx!
> git is not the only scm system out there.
(snip)
sure. i'm trying to get forge playing nice with not-included-yet hg
sources atm :-)
I know some people have used it successfully with birbucket hg
projects, for example. The code is pretty SCM agnostic, it "only"
constructs API URLs for the major forge services out there.
yup. and for the moment -- while I'm getting my 'end product' sorted
out -- that's intentional, and I'm not concerned with audit trail.
That’s fine as long as you know the limitations, and that is why I made
the technical side of the macros support this workflow too.
> So you should have a long list of
> %forgeurlX
that's already changed in my latest ...
> %tagX or %commitX
fair enuf, now that the above is nice-n-clear ...
that'll get announced ... here?
That will be announced on devel and packaging once there is a redhat-
rpm-config PR and a set of example copr builds to look at. The aim is
to get it done soonish, so i18n can decide if they want to push a
change that depends on it before the F33 change deadline (30th june).
First set of unit tests and test builds is green, so it should not take
long now.
do you have a _rough_ timeframe for when that'll be consistently
supported/usable in rpmbuild, mock & COPR?
This is very low-level work that only depends on rpm (and often not a
particular rpm version). So typically when it is done it just works in
all upper level tools (spectool, rpmlint, mock, copr, koji, etc).
Now get it done includes a redhat-rpm-config PR merge, and I do not
control how long those take (likewise, it will imply an update of the
forge and fonts packaging guidelines since they both use the same macro
engine, and FPC can ponder things ponderously sometimes).
Regards,
--
Nicolas Mailhot
1:31 a.m.
New subject: Fwd:%forgemetasupportfor`git`tasksinchecked-outcode?
Le vendredi 26 juin 2020 à 07:41 -0700, PGNet Dev a écrit :
hi,
On 6/25/20 11:58 PM, Nicolas Mailhot wrote:
> forgemeta works in release mode, with release archives published
> over
> http(s). It does not talk at all to source projects using the git
> protocol (and that is intentional, since a lot ot things tooling-
> side
> do not talk the git protocol and will never talk the git protocol,
> starting with rpm itself, spectool, etc).
noted. not clear initially from reading the docs; this helps. thx!
The documentation part of the initial redhat-rpm-config forge
implementation was never merged, and I gave up on refreshing it, so
you’re in traditional unwritten rpm lore land.
Regards,
--
Nicolas Mailhot
1396
days inactive
1398
days old
copr-devel@lists.fedorahosted.org
4 comments
3 participants
participants (3)
-
Nicolas Mailhot -
Pavel Raiskup -
PGNet Dev