Good Morning Everyone,
This topic has already been discussed a few times over the past month, but Adam
Saleh, Nils Philippsen and myself have had the opportunity to invest some time
on it with the hope of making the packager's life simpler as well as making it
easier to build automation around our package maintenance.
We have investigated a few ideas, the full list with their pros and cons can be
found in this document:
https://hackmd.io/8pSwJidhTYel9euQqMEKpw?viev
If you have any questions about some of these, please ask them, we'll be happy
to answer them and potentially complete this document.
For both the release and the changelog fields we believe using RPM macros would
satisfy the requirements we have for opting-in/out:
- You can easily opt-in by using the macros
- You can easily opt-out by removing the macros from your spec file
For the changelog, we believe we have a potential good solution:
- The changelog will be automatically generated using an external `changelog`
file as well as the commit history
- When you opt-in, you will simply move the existing changelog to an external
file `changelog`, which is placed in the dist-git repo and add the
appropriate macro.
- Upon building, the macro will generate the changelog using all the commits
of the repo up to the last commit touching the `changelog` file. Of all
these commits it will only consider the commits following these rules:
- There are generally two approaches regarding what to include by default:
1. commits touching only the `sources`, `.gitignore`, `dead.package`
files, `tests` folder will be ignored by default, i.e. a blacklist
2. only commits touching the spec file or patches will be included by
default, i.e. a whitelist
==> Which approach do you think is better/will work in most cases?
- empty commits (not touching any files) will be included on the assumption
that this is their purpose
- commits containing "magic keyword" (#changelog_exclude,
#changelog_include?) will be ignored or included as the case may be
- Finally the content of the changelog file specified will be appended to the
changelog generated from the git history
If you wanted to edit the changelog, you would:
- Generate the changelog locally via a command like:
`fedpkg generate-changelog > changelog`
- Edit `changelog` as desired
- Commit and push the changes
Since the macro will only consider the commits up to the first commit touching
`changelog` (in other words, the macro will only consider the commits after this
one) and include `changelog` file as is, your edits will appear in the RPM
changelog as you want.
One thing to note is that rebuilds from the same commit will have the same
%changelog, they will not get a new entry. If you want a new changelog entry,
you have to create a new commit with the desired changelog entry as commit
message (the commit itself can be empty).
However, for the release field, we are struggling a little bit more, two options
are more appealing to us:
A) The release field is automatically generated using two elements:
- the number of commits at this version
- the number of builds at this commit
- the dist-tag being added after them
The release field would thus look like:
``<number of commit at version>.<number of build at commit>%{dist}``
So we could have: (A, B, C and D being different commits)
A -- version: 0.9 -- release: ?
|
B -- version: 1.0 -- release: 1.0
|
C -- version: 1.0 -- release: 2.0
|
D -- version: 1.1 -- release: 1.0
A rebuild of the commit D, would lead to a release 1.1 (assuming the first one
succeeded)/
Pros:
- Easy to replicate locally
- Every changelog entry can be mapped to a `version-release` (No changes to the
packaging guidelines)
- Allows triggering two builds from the same commit without any manual change
(simplifies mass-rebuilds)
- Easy to link a certain build with a specific commit
- Easy to “guess” the next release value before triggering the build
Cons:
- Old packages which are no longer receiving upstream releases may need
special care (for example if they are at the release -48 but only had 45
commits leading to this)
- Relies on information from the build system for the build number (but can
be closely simulated locally since the <n_build> info is only used for
rebuilds)
- Upgrade path may be problematic if Fn is upgraded to version X with one
commit while Fn-1 is upgrade to version X with 2 commits (or more)
B) The release field is automatically generated taking existing builds in all
current Fedora releases (ie: rawhide, Fn, Fn-1...) into account. This means for
builds of the same epoch:version, find a new release that (if at all possible)
ensures upgradability from older Fedora versions to newer ones, adhering to the
structure of a release tag documented in the Versioning Guidelines[1]. Going
from the (RPM-wise) "latest build" that the new one should surpass, this can
mean bumping in the front (`pkgrel`) or the back (`minorbump`).
This allows packages from "very stable" upstreams who haven't released in
years
to still benefit from automatically generated releases.
The following examples would use a macro for the release field as outlined in a
separate document[2].
Example #1 ("normal" release progression):
Rawhide has: 2.0-1.fc32
F31 has: 1.0-1.fc31
F30 has: 1.0-1.fc30
Next release in F31: 1.0-2.fc32
Example #2 ("hotfix" in an older release, selected by an alternative macro (or
option) in the spec file):
Rawhide has: 2.0-1.fc32
F31 has: 1.0-1.fc31
F30 has: 1.0-1.fc30
Next release in F30: 1.0-1.fc30.1
Example #3 (pre-release, selected by an alternative macro (or option) in the
spec file):
Rawhide has: 2.0-1.fc32
F31 has: 1.0-1.fc31
F30 has: 1.0-1.fc30
Next release in Rawhide: 3.0-0.1.20200224git1234abcd
Pros:
- Allows triggering two builds from the same commit without manual
intervention
- Emulates what many maintainers do manually today for most use cases
- Can cater for pre-releases (e.g.: by offering different macros or macro
options for the different use cases)
Cons:
- Needs the build system for information about builds in this and other Fedora
versions
- Not easy to reproduce locally because we don’t have machine-consumable
knowledge about other builds in e.g. the dist-git repo
- Does not allow to generate changelog entries with `version-release`
information for all commits (and this will require a change in our packaging
guidelines)
So these are the results of our current investigations, we are very much eager
to get your feedback on them and even more eager if you have ideas on how to
approach/solve some of the challenges mentioned here.
Looking forward for the discussion,
Pierre
For Adam, Nils and myself
[1]:
https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/
[2]:
https://hackmd.io/kuXOPe74RfepuztBz7pBsg