Fonts packaging policy rewrite proposal
by Nicolas Mailhot
Hi,
A fonts packaging policy rewrite proposal has been pushed to FPC today:
https://pagure.io/packaging-committee/pull-request/934
It should be clearer, more opinionated, and take into account:
– updates of The OpenType standard
– variable fonts
– web fonts
– upstream depreciation of non OpenType formats: final stages of the
Harfbuzz consolidation decided at the 2006 Text Layout summit
https://www.freedesktop.org/wiki/TextLayout/
– appstream & fonts
– weak dependencies
– and probably more I forget here
It is based on the new fonts-rpm-macros project for automation:
This project builds on tooling enhancements in redhat-rpm-config and rpm
itself, done during the past two years for the Forge and Go sets of
packaging macros. It started 2 years ago as a fork of fontpackages,
which is the core of our current fonts packaging guidelines.
It will require putting the fonts-srpm-macros package in the default
build root, like is done for other domain-specific packaging macro
sets.
Major additions:
– better documentation (clearer and more complete)
– better automation (less packager hassle for better and more complete
results)
Major removals:
– tools and scripts
– fixing metadata with ttname
Mostly because no one seems willing to maintain those scripts, or port
ttname to python 3.
https://copr.fedorainfracloud.org/coprs/nim/fonts-rpm-macros/builds/
showcases the new policy on 62 real-world source packages, generating
139 installation packages. Some of those are badly delayed updates to
Fedora packages, others are brand-new packages ready for Fedora
inclusion. They include major font packages such as Stix, DejaVu, Droid,
IBM Plex.
Existing Fedora packages will continue to build, the old fontpackages
macros are grandfathered in fonts-rpm-macros for now. They will be
removed in a few years to give packagers time to apply the new
guidelines.
Regards,
--
Nicolas Mailhot
3 years, 2 months
Re: Ideas and proposal for removing changelog and release fields from spec file
by Nicolas Mailhot
Le samedi 29 février 2020 à 20:30 +0100, clime a écrit :
>
> What about fedpkg local and fedpkg verrel then?
Putting %{dynrel} reconciliation in the rpmbuild -bs stage using
detached file state means that fedpkg local (or checking out git state
and building in mock or copr or OBS or via plain rpmbuild -bs) will
give you the same result as launching fedpkg build. Which is exactly
what you want to make QA and testing workflows work. Those don’t live
exclusively in koji.
And because only production builds get committed back the packager can
change his mind and stage a few more changes before doing the
production build, without polluting the changelog builds that were
never pushed to users.
For fedpkg verrel you'll probably want to output a last (saved in
detached file) and next line (probably factorizable by externalizing
the dynrel bump logic in a separate command). That’s more honest
anyway, next may happen or not, when you launch fedpkg verrel, it’s
mere potential (the next commit may bump version and invalidate your
future build).
> As for changelog, generally with build system commiting back to
> dist-git, there is a problem that I can be concurrently pushing
> changes while the build system tries to push its changes and build
> system is not human so it will not know how to resolve such
> situation.
I understand, but that’s the consequence of automating changelog. Right
now the reconciliation is done by the human packager (you can get in a
similar situation by working on a change at the time a mass rebuild
runs).
> Do you have a solution for situation when I launch a build and then I
> do another change, commit, and push (the changes to changelog can be
> quite arbitrary here)?
You can set a lock at fedpkg build time, and forbid fedpkg commit in
that branch till the lock is released (fedpkg build in the branch
succeeded or not). The packager can still git commit things, as long as
he does not touch the detached changelog file. Only fedpkg commit syncs
git state with detached changelog state.
An alternate simpler strategy would be to mark the fedpkg build as
failure in koji if sync-back failed. That would work too. The build
system need not be ultra smart, just robust WRT human behaviour.
(If the packager deliberately messes with the detached changelog while
a production build for the same branch is ongoing he deserves a build
failure – the changelog state is undefined till the build ends, so he’s
doing changes on quicksand. If he tried that today he’d probably have
to rewrite his changelog if the build failed)
> I think, this is not a decision of rpm upstream but an infrastructure
> thing or a mock thing.
mock and rpm upstreams excel when they work together. My premise is
that they are better qualified than us to do rpm/buildsys boundary fine
tuning.
Populating changelog from git and syncing back to fedora git are
fedpkg/koji responsability, because Fedora git is Fedora specific
infra. Handling release autobumping and build recording belongs to the
lower layers, however. They’re the same with or without Fedora git,
they belong to lower levels.
> I think your proposal wouldn't quite work for copr because it has no
> access to those repositories (which especially for src.fp.o is good
> in your proposal, otherwise copr would be modifying src.fp.o repos).
copr does not need write access to Fedora git, its builds do not
participate to the production build lineage as long as no one re-
imports them in koji (which should be a deliberate fedpkg command, not
something driven by copr itself).
copr only needs access to its own private git to remember its own last
successful build, if people want it to autobump from last successful
state. (maybe they don’t). srpm import in copr will overwrite copr
state with the state the new srpm contains, which is fine too.
Putting state in detached srpm source files has the following super-
duper property. You can import the srpm or scm checkout between
systems, and they’ll just pick up from the point the previous system
left things, without needing a full scm import. So you do not need to
deal with incompatible scms (or lack of scm).
> So if someone wanted to have a build counter in the release, yes,
> this could be an implementation although it would be much easier to
> simply catch the information from copr's database where this info is
> stored
It is simpler from the buildsys POW, but it ties state in a specific
git and buildsystem. So it will break cross-buildsys workflows.
Cross-buildsys workflows are critical for the project success because
they enable sharing work with other distros, and allow packagers to
make the best of a palette of build systems (each with its own
constrains and limitiations). Fun fact: I noticed today than one of my
old Fedora packages was rebuilt by others for AIX. This kind of cross-
pollination is one of the strengths of the rpm ecosystem. Don’t break
it by making out packages depend on Fedora git or buildsys state.
> in fact there would never be a changelog file at
> all because nobody uses fedpkg commit in git repos that are not from
> src.fp.o so this dynamic changelog thing wouldn't be usable for copr
> at all.
When using a buildsys that didn’t automate changelog filling, you can
just write in the detached changelog file directly. The build part
won’t care how you filled this file with changes, it just needs it to
record a build point.
scm to changelog plumbing logic is going to change over time (for
example, if we drop pagure for gitlab, or git for the next best thing).
It is dangerous to set it in stone.
dynrel autobumping and recording builds in the changelog is more of an
rpm format change. It can (and should) be done in an independant way.
Regards,
--
Nicolas Mailhot
3 years, 2 months
Help with Nagios/NRPE: SELinux considerations
by Martin Jackson
Hello,
I recently adopted nagios and nrpe packages (also nagios-plugins); but
nagios and nrpe are my chief concerns at the moment.
I'm working on getting the released fedoras and epel 7/8 level on
4.4.5. There are a number of open bugs due to SELinux denials. There's
a memory leak in 4.4.3 in epel7 that I think needs to be addressed.
There was an open PR in the nagios package, which I accepted and have
been trying to deploy. It also kind of bridges into nrpe, which I
kind-of suggested and am now thinking I should not have.
I'm not really clear on how executables get their contexts. The patch
defines a nrpe_etc_t type, but how does that get associated with the
nrpe binary? Can it have multiple contexts and if so how?
I'm relatively new to this level of RPM packaging, but excited to learn
how to deal with these kinds of concerns and wanting to provide a great
nagios experience in the ecosystem.
Thanks in advance for any pointers or advice!
Marty
3 years, 2 months
Re: Ideas and proposal for removing changelog and release fields from spec file
by Nicolas Mailhot
Le samedi 29 février 2020 à 13:49 +0100, clime a écrit :
> On Sat, 29 Feb 2020 at 10:15, Nicolas Mailhot via devel
> <devel(a)lists.fedoraproject.org> wrote:
> > Hi,
> >
> > Anyway, here is what I would personnaly consider a robust,
> > inclusive,
> > and future-proof design:
>
> I will need to ask some questions to really understand it.
>
> > — a %{use_dynstate} rpm variable enables dynamic changelog/release
> > behaviour
> > — probably initialy set to false distro-wide, to let volunteers
> > test
> > the thing by setting it to true iin their specs,
> > — then to true (opt-out), when kinks have been fixed, and
> > FPC/FESCO
> > greenlights general availability
> >
> > – when activated, last changelog, evr and %{dynrel} state are saved
> > in one or several detached files
>
> You mean the last changelog, evr, and %{dynrel} are stored once
> %{use_dynstate} is set and and after one invokes fedpkg commit?
> I think there should be some explicit action (e.g. the commit) to
> generate the files after I set the %{use_dynstate} value to true in
> the spec file.
Once a spec sets %{use_dynstate} to true changelog, last computed
changelog, ev, neutralized-r, and %{dynrel} are taken from detached
files. neutralized-r is the evaluation of Release with %{dist} set to a
neutral value (for example “-”).
State is stored in files included in the srpm to be able to import to
and from Fedora git (pretty much a hard requirement if tooling is to be
kept Fedora-git neutral, which is itself a hard requirement to be able
to interact with packaging work existing outside Fedora git).
You need the last computed ev and neutralized-r to decide whether
%{dynrel} can be kept, needs bumping, or should be reset to 1.
I don’t trust mixed mode, KISS, changelog is detached or not, don’t try
to have it both ways, here lies madness and confusion.
> How is %{dynrel} computed here at this stage
Nothing is computed at this stage.
Detached changelog state, is a commit property (except for build date).
It is computed at fedpkg commit time:
– if detached changelog data is absent
→ move in-spec changelog data to the detached file if data exists,
→ otherwise init detached data to nothing
– after the first fedpkg commit with %{use_dynstate} set to true, the
detached changelog file exists and the in-spec changelog has been
removed.
– others have detailed possible fedpkg commit strategies to stuff
changelog with new info
%{dynrel} and build changelog info are a build property. They are
computed at build time:
– if computed ev ≠ last saved ev or last %{dynrel} is undefined
→ reset %{dynrel} to 1
– otherwise if computed neutralized-r = last neutralized-r
→ bump %{dynrel}
– otherwise, do nothing, something already took care of things
– save the new computed ev, computed neutralized-r and dynrel to the
detached files
– add a build line with the full evr (and full dist) to detached
changelog
– at build time, there is no relationship with magic git state or
magic buildsys info, the srpms are 100% autonomous as before. The
detached changelog has already been populated manually or by distro
automation or manual processes when it reaches build stage and a new
build is recorded (with a new dynrel value, if necessary).
> Is the intention here that with each new build of the same package,
> the value of %{dynrel} is bumped and committed back?
It is bumped (or reseted to 1) whenever comparison with last saved
state shows a bump or reset is needed.
Build-time state changes need to be commited back, of course (otherwise
the produced srpm needs to be re-imported manually)
> That means the evr file read from the repository needs to contain
> somehow outdated values at this point (when srpm is being built in
> build system), otherwise %{dynrel} would be always bumped?
Final dynrel state is not computed before a build, yes. The build will
decide if it needs to bump dynrel, or can reset it to 1. I had
forgotten to document the reset, sorry.
There is no need to compute a new dynrel before actual build, the
packager may yet change ev or r.
> > – a build line is added to the detached changelog, using the
> > current
> > date and full evr (without %{dist} neutralization)
> > — that probably requires defining a rpm variable to track whom
> > the
> > build is attributed to
> > – it can default to "Anonymous coward" or whatever if not
> > explicitely
> > set
>
> I think today's changelogs do not contain name of the person who
> built the pacakge
Today’s changelog mixes who made spec changes and who built the result.
That what confuses you. Confusion is anthetical with automation. This
proposal clearly documents that changelog can change at build time
(fedpkg build) and between builds (fedpkg commit).
From a package consumer POW, the only attribution and timestamp that
matters is who build the final package and when, because the builder is
taking the responsability of pushing a result to users.
A clean changelog is:
%changelog
* timestamp builder-id - built-EVR ← set at build time
- change ← set at commit time
- change ← set at commit time
- change ← set at commit time
Not sure at all the change lines need attribution or timestamping. From
a package user POW they are virtual potential changes that won’t
materialize before a build.
So yes before someone presses the build button the detached changelog
will look like this
%changelog
- staged change
- staged change
- staged change
* timestamp builder-id - built-EVR ← set at build time
- past change
- past change
- past change
The act of building adds the final %changelog line:
* timestamp builder-id - built-EVR
> You mean there would be different kinds of built srpms?
We already have two level of srpms, one before dynamic BuildRequires
processing and one after. It’s up to rpm upstream to decide at which
srpm level they would prefer preforming dynrel and dynchangelog
munging.
I want upstream rpm involved for dynrel and dynchangelog to be
independent from current Fedora git infra, and interoperable.
> For copr, it is not possible, because it has read-only access to git
> repositories being built.
It has some access to private git repos, you see copr initialising them
when giving it a new srpm to build.
> I think there are good examples where some things can be done without
> rpm changes - e.g. the simple-koji-ci to do automatic scratch builds
> on a new commit.
For something that is effectively a format change (moving changelog
outside the spec itself) it’s a bad bad idea to do it without rpm
upstream vetting and involvment. Sure you can kludge things without
them. Operational word: kludge. Will forever be a pain point as long as
it is not integrated upstream. Magic git-only or bbuildsys-only state
is a misfeature, (s)rpms should cut the git/buildsys cordon as soon as
they are born.
scratch build are something else entirely. They are transient non-
shared things that do not need to cleanly export or import in other
build systems
Regards,
--
Nicolas Mailhot
3 years, 2 months
Re: Ideas and proposal for removing changelog and release fields from spec file
by Nicolas Mailhot
Hi,
Anyway, here is what I would personnaly consider a robust, inclusive,
and future-proof design:
— a %{use_dynstate} rpm variable enables dynamic changelog/release
behaviour
— probably initialy set to false distro-wide, to let volunteers test
the thing by setting it to true iin their specs,
— then to true (opt-out), when kinks have been fixed, and FPC/FESCO
greenlights general availability
– when activated, last changelog, evr and %{dynrel} state are saved in
one or several detached files
— evr computed using a fixed neutral %{dist} value (for example “-”
since it’s forbidden in %{dist})
– those files are given standard rpm variable names and added to
Sources:
– manual packager Source900: %{dynstate} or whatever
– rpm later updated to allow source file declarations via macros at
to eliminate boilerplate (I need this in forge and go packages)
– or the detached files are set in stone as separate Tags with a
default value overridable via the %{dynstate} variable in a new rpm
version
– the packager adds %{dynrel} wherever he sees fit in his Release
string
– at fedpkg commit time changelog state is updated with info taken from
fedora git state, *without* evr and build date
– that’s Fedora-specific integration, exact commit/tag syntax to mark
things to inject or ignore TBD Fedora-side
– fedpkg commit sets a tag in git to mark anything older can now be
ignored for changelog generation purposes
– detached changelog state remains packager-editable, like the in-
spec changelog today. That allows pruning useless no longer relevant
stuff and fixing grammar errors
— at rpmbuild -bs stage
– evr is computed using the same neutral %{dist} value as before, and
the saved %{dynrel} value
– if it is equal to the previously saved evr %{dynrel} is bumped and
saved in the detached file
– a build line is added to the detached changelog, using the current
date and full evr (without %{dist} neutralization)
— that probably requires defining a rpm variable to track whom the
build is attributed to
– it can default to "Anonymous coward" or whatever if not explicitely
set
– Koji, OBS and Copr can set it to whoever asked them to build the
package
– the result constitutes the new srpm (either first or second level
srpm as upstream rpm sees fit)
– that’s generic non Fedora-specific behaviour that work as well in
rpmbuild -bs, mock, copr, koji, OBS, whatever, with or without Fedora
git presence
– Koji/copr need to commit the new dynamic dynrel/changelog state to
git (a build-striggered state change is commited by the build system)
And yes that requires some work rpm-side. That is necessary to maintain
the rpm decentralized design and keep srpms independent from anyone’s
git state. Personnaly, I don’t see the point of pretending we can move
our infra forward without ever touching rpm. The cardinal sin of
Modularity was attempting to create an overlay over existing rpm/dnf
behaviour, without changing this behaviour when it needed improving.
Contrat that with dynamic buildrequires or weak deps that slotted into
our workflows with hardly any ripple. Though they were major feature
changes. But, they were done with rpm upstream, instead of trying to
bypass it.
Regards,
--
Nicolas Mailhot
3 years, 2 months
Doubt about explicit compiler mention in the BuildRequires field
by José Abílio Matos
I am asking this about
https://docs.fedoraproject.org/en-US/packaging-guidelines/C_and_C++/
Is the requirement that we should explicit state the compiler or is it enough
to have it implicitly.
Case in point, R binary packages.
Each package requires R-devel. Now R-devel requires R-core-devel that requires
gcc-c++
gcc-gfortran
So the BuildRequires for gcc-c++ is there but implicitly.
If the requirement is for the BuildRequires to be explicit then it should be
said that explicitly in the guidelines page. As an example in the "Packaging
Q&A" section.
What do you think? Regards,
--
José Abílio
3 years, 3 months
Summary/Minutes from today's FPC Meeting (2020-02-20 17:10 - 18:00 UTC)
by James Antill
======================
#fedora-meeting-1: fpc
======================
Meeting started by geppetto at 17:10:17 UTC. The full logs are available
at
https://meetbot.fedoraproject.org/fedora-meeting-1/2020-02-20/fpc.2020-02...
.
Meeting summary
---------------
* Roll Call (geppetto, 17:10:18)
* #946 Bootstrap Exception for .NET Core 3.1 (dotnet3.1) (geppetto,
17:17:34)
* ACTION: Bootstrap Exception for .NET Core 3.1 (dotnet3.1) (+1:6,
0:0, -1:0) (geppetto, 17:21:51)
* #pr-940 Revisit the Python guidelines (geppetto, 17:22:35)
* LINK: https://pagure.io/packaging-committee/pull-request/940
(geppetto, 17:22:42)
* ACTION: Revisit the Python guidelines (+1:5, 0:0, -1:0) (geppetto,
17:25:50)
* Open Floor (geppetto, 17:26:12)
* LINK: https://pagure.io/packaging-committee/issue/7199 (nim-nim,
17:33:50)
* LINK: https://pagure.io/packaging-committee/pull-request/826
(decathorpe, 17:42:14)
* LINK: https://bugzilla.redhat.com/show_bug.cgi?id=1803281
(decathorpe, 17:49:55)
* ACTION: limburgher Will review macros in
https://bugzilla.redhat.com/show_bug.cgi?id=18032811 (geppetto,
17:53:06)
Meeting ended at 17:58:57 UTC.
Action Items
------------
* Bootstrap Exception for .NET Core 3.1 (dotnet3.1) (+1:6, 0:0, -1:0)
* Revisit the Python guidelines (+1:5, 0:0, -1:0)
* limburgher Will review macros in
https://bugzilla.redhat.com/show_bug.cgi?id=1803281
Action Items, by person
-----------------------
* limburgher
* limburgher Will review macros in
https://bugzilla.redhat.com/show_bug.cgi?id=1803281
* **UNASSIGNED**
* Bootstrap Exception for .NET Core 3.1 (dotnet3.1) (+1:6, 0:0, -1:0)
* Revisit the Python guidelines (+1:5, 0:0, -1:0)
People Present (lines said)
---------------------------
* geppetto (45)
* nim-nim (29)
* decathorpe (27)
* tibbs (23)
* zodbot (16)
* ignatenkobrain (6)
* limburgher (5)
* omajid (1)
Generated by `MeetBot`_ 0.1.4
.. _`MeetBot`: http://wiki.debian.org/MeetBot
3 years, 3 months
