Fedora 33 System-Wide Change proposal: CMake to do out-of-source builds
by Ben Cotton
https://fedoraproject.org/wiki/Changes/CMake_to_do_out-of-source_builds
== Summary ==
<code>%cmake</code> macro will be adjusted (<code>-B</code> parameter)
to use separate build folder (already standardized
<code>%{_vpath_builddir}</code> macro). Additionally,
<code>%cmake_build</code>, <code>%cmake_install</code> and
<code>%ctest</code> macro will be created (and backported to the older
supported Fedora releases) to perform various operations that are
commonly used with CMake in a backend-agnostic (Makefiles, Ninja,
etc.) way.
Packages that will stop building are trivial to fix and will be
adjusted either by maintainers or change owners.
== Owner ==
* Name: [[User:ignatenkobrain|Igor Raits]], [[User:besser82|Björn
Esser]], [[User:ngompa|Neal Gompa]]
* Email: ignatenkobrain(a)fedoraproject.org, besser82(a)fedoraproject.org,
ngompa13(a)gmail.com
== Detailed Description ==
Historically, software builds had a singular build configuration and
required running the build within the project root. Nowadays, there
are many build modes and options that can be configured in projects,
different build settings (e.g. compiler flags) / types (release,
debug) that can be applied and different tools that can be used to
actually execute builds (compilers like gcc/clang, build job
schedulers like make/ninja, and so on). Thus, CMake upstream strongly
discourages users of doing in-source builds and recommends doing
out-of-source builds.
From <code>cmake.1</code>:
<pre>
To maintain a pristine source tree, perform an out-of-source build by
using a separate dedicated build tree. An in-source build in which the
build tree is placed in the same directory as the source tree is also
supported, but discouraged.
</pre>
The other part of the change is introduction of additional macros is
creation of set of macro that can build, install and run tests in a
backend-agnostic, vpath-aware (out-of-source, in-source) way.
=== Migration ===
==== <code>%cmake</code> + <code>%(make|ninja)_(build|install)</code> ====
There are multiple paths to complete the migration:
* Add <code>-C "%{_vpath_builddir}"</code> to the <code>%(make|ninja)_*</code>
* Replace <code>%(make|ninja)_build</code> and
<code>%(make|ninja)_install</code> with <code>%cmake_build</code> and
<code>%cmake_install</code> respectively
* Redefine vpath builddir <code>%global _vpath_builddir .</code> to
continue performing in-source builds (and optionally converting to the
<code>%cmake_*</code>)
Depending on the package, one of these options may be used to adapt to
this change.
==== <code>%cmake -B builddir</code> +
<code>%(make|ninja)_(build|install) -C builddir</code> ====
No changes are needed.
== Benefit to Fedora ==
* Follow CMake upstream recommendations when building packages
* Brings Fedora package builds more in-line with how upstream projects
expect them to be built
* Improve compatibility with other RPM distributions that already do this
* Support backend-agnostic way of building CMake projects
== Scope ==
* Proposal owners: Implement necessary macros, try to build packages
that <code>BuildRequires: cmake</code> in a side tag, analyze failures
and fix the relevant ones (introduced by this change).
* Other developers: While proposal owners will try to fix all affected
packages, there might be some cases where package is already FTBFS so
the fix can't be performed. Other package maintainers will have to fix
the issue themselves after they fix FTBFS.
* Release engineering: [https://pagure.io/releng/issue/9524 #9524]
* Policies and guidelines: CMake page will be adjusted to mention
newly created macros and the documentation about relevant VPATH macros
needs to be restructured a bit (they are already documented on the
Meson page, they need to be moved to the separate page and referenced
both from CMake and Meson page).
* Trademark approval: N/A (not needed for this Change)
== Upgrade/compatibility impact ==
Existing packages can (and most likely will) become FTBFS, but
proposal owners will fix as many Fedora packages as possible. However
fixing third-party packages is not possible and out of scope.
Third-party packagers will need to adapt based on the recommendations
noted in this Change.
== How To Test ==
# Grab the new cmake RPM from the Koji sidetag (TBC)
# Try to build package that uses <code>%cmake</code>,
<code>%cmake_build</code>, <code>%cmake_install</code> and
<code>%ctest</code> macro
== User Experience ==
The end-users (non-packagers) will not notice any changes.
== Dependencies ==
There are around 1100 RPMs in Fedora that depend on CMake at
build-time. All proposal owners are provenpackagers so they are able
to commit necessary fixes. No external dependencies.
== Contingency Plan ==
* Contingency mechanism: Proposal owners will adjust macros to not do
out-of-source builds by default, but will preserve newly created macro
(essentially to bring them to the targeted state of older supported
Fedora releases).
* Contingency deadline: Beta freeze.
* Blocks release? No
== Documentation ==
The only place that needs to be adjusted is packaging guidelines.
--
Ben Cotton
He / Him / His
Senior Program Manager, Fedora & CentOS Stream
Red Hat
TZ=America/Indiana/Indianapolis
3 years, 8 months
Fedora 33 System-Wide Change proposal: swap on zram
by Ben Cotton
https://fedoraproject.org/wiki/Changes/SwapOnZRAM
== Summary ==
Swap is useful, except when it's slow. zram is a RAM drive that uses
compression. Create a swap-on-zram during start-up. And no longer use
swap partitions by default.
== Owner ==
* Name: [[User:chrismurphy| Chris Murphy]]
* Email: chrismurphy(a)fedoraproject.org
== Detailed Description ==
==== zram Basic function ====
The zram† device, typically <span style=color:brown>/dev/zram0</span>,
has a size set at create time during early boot, by zram-generator†
per its configuration file. The memory used is not preallocated. It's
dynamically allocated and deallocated, on demand. Due to compression,
a full <span style=color:brown>/dev/zram0</span> uses half as much
memory as its size.
The <span style=color:brown>/dev/zram0</span> behaves like any other
block device. It can be formatted with a file system, or mkswap, which
is the intention with this change proposal.
The system will use RAM normally up until it's full, and then start
paging out to swap-on-zram, same as a conventional swap-on-drive. The
zram driver starts to allocate memory at roughly 1/2 the rate of page
outs, due to compression. But, there is no free lunch. This means
swap-on-zram is not as effective at page eviction as swap-on-drive,
the eviction rate is ~50% instead of 100%. But it is at least an order
of magnitude faster than drive based swap.
zram has about 0.1% overhead or ~1MiB/1GiB. If the workload never
touches swap, this overhead is the sole cost. In practice when not
used at all, feature owner has experienced ~0.04% overhead.
Example: A system has 16 GiB RAM. The proposed defaults suggest the
<span style=color:brown>/dev/zram0</span> device will be 4 GiB. If the
workload completely fills up swap with 4 GiB of anonymous pages,
what's happened? The <span style=color:red>zramctl</span> command will
display the true compression ratio. If 2:1 is really obtained, it
means 4GiB swap data is compressed to 2GiB. Therefore 2GiB is the
actual RAM usage, and is also the net effective eviction. i.e. 4 GiB
anonymous pages are evicted, but are then compressed and pinned into 2
GiB RAM, for a net memory savings of 2 GiB.
†</br >
[https://www.kernel.org/doc/Documentation/blockdev/zram.txt kernel.org zram.txt]
[https://github.com/systemd/zram-generator Github zram-generator project]
==== Overview of the Feature ====
Using swap is a good idea†, but no one likes it when it's slow.
Anaconda and Fedora IoT have been using swap-on-zram by default for
years. This builds on their prior effort.
There are three components to the change:
# Install systemd rust-zram-generator† package. This does not enable
swap-on-zram, it only makes the generator available.</br >
# Install a default zram-generator configuration. When present,
swap-on-zram is set-up during startup.</br >
# Do not create swap partition/LV with default installations.
This proposal aims to apply all three, for all Fedora editions and
spins, by default.
It further aims to apply the first two, for upgrades and custom installations.
It might be useful to only make the generator available (1), should an
edition/spin wish to opt out, or as a fallback if applying the feature
to upgrades fails to withstand scrutiny.
†</br >
There is a tl;dr section at the top. Highly recommend reading the
whole article. [https://chrisdown.name/2018/01/02/in-defence-of-swap.html
In defence of swap: common misconceptions]
==== Default zram device configuration: ====
During startup, create a zram device <span
style=color:brown>/dev/zram0</span>, with a size equal to 50% RAM, but
capped† to 4 GiB, and with a higher than typical swap priority†.
These values seem reasonably conservative, and are based on prior work
in Fedora. Anaconda sets swap-on-drive sized to 50% RAM in the no
hibernation case, common outside x86. Fedora IoT's implementation also
sets swap-on-zram size to 50% RAM.
†</br >
[https://github.com/systemd/zram-generator/issues/10 RFE: should be
able to set a cap on zram device size #10]
[https://github.com/systemd/zram-generator/issues/8 RFE: should set priority #8]
==== Default installer behavior ====
The installer is currently responsible for creating a swap-on-drive
device. This will be dropped. The zram-generator + configuration file
will trigger the setup and activation of swap-on-zram. This means
hibernation isn't possible, even on systems that could support it.
Please see [https://pagure.io/fedora-workstation/blob/master/f/hibernationstatus.md
Supporting hibernation in Workstation edition] for much more detailed
information, including why it's increasingly likely hibernation isn't
possible anyway, and a path to improving hibernation support.
==== Custom/Advance partitioning installer behavior ====
The user can add swap using Custom partitioning at install time. This
is swap-on-drive. And the installer will also include the <span
style=color:red> resume=UUID </span> kernel parameter for this swap
device. No change in behavior here.
Since swap-on-zram is still enabled by default, there will be two
swaps: swap-on-zram, and swap-on-drive. The swap-on-zram will have
higher priority, thus being favored over drive based swap. The kernel
is smart enough to know it can't hibernate to a zram device, and will
instead use drive based swap.
==== How can it be disabled? ====
Immediately:</br >
<span style=color:red>swapoff /dev/zram0</span>
Permanently:</br >
<span style=color:red>rm /etc/systemd/zram-generator.conf</span>
== Feedback ==
==== You're enabling it on upgrades? ====
That's the current plan. As a technical matter, feature owner is
confident this feature will improve the experience of all users
regardless of configuration. As a non-technical matter, it's
recognized that (a) ''hey pal, you're messing with my customizations,
not cool!'' and (b) ''swap always stinks, I don't care if it has a 'Z'
in the name!'' may need more convincing.
There are possible risks.
* Workloads that expect full use of memory, and depend on 100% page
eviction. These may run slower if they really need full use of memory,
but some memory is used for the zram device instead. Such workloads
might favor zswap.
* Workloads with low compressible pages. In the worst case, this means
unnecessary work merely moving pages around.
* Workloads with memory full, and hibernation. Hibernation is already
stressful to memory-management subsystem and prone to bailing out in
such cases. The swap-on-zram will be favored for evictions in the
attempt to free memory to create the hibernation image. It could
increase instances of hibernation entry failure. This isn't a crash,
it just means the attempt doesn't succeed, and the system resumes
operation instead of hibernating.
While possible, it's difficult to estimate their probability. But this
is a significant consideration in the conservative default zram size.
Users can easily increase zram size as needed for their use case,
simply by editing <span
style=color:red>/etc/systemd/zram-generator.conf</span> and the change
takes effect at next boot.
==== Why systemd zram-generator? ====
It's the most upstream implementation to date, is fast and
lightweight. The zram-generator uses existing systemd infrastructure
to setup the zram block device, format it as swap, and swapon - all
during early boot. It's very similar in behavior to fstab-generator,
gpt-auto-generator, and cryptsetup-generator†.
Converging on one implementation avoids user confusion. And while the
alternatives are nice and work fine, a systemd generator is
particularly well suited for this use case compared to a systemd
service unit.†
Also, it's an reference implementation of a system generator written in Rust.
†</br >
[https://www.freedesktop.org/software/systemd/man/systemd.generator.html
freedesktop.org About systemd generators.]</br >
[https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...
devel@ ''Re: swap-on-zram by default'' Zbigniew Jędrzejewski-Szmek,
systemd zram-generator author/maintainer]
==== Why not a bigger zram device? ====
The main idea of being conservative is to address concerns about
upgrades. It's possible some workloads will have less compressible
data. Hence, not going with <span style=color:brown>/dev/zram0</span>
sized to 100% of RAM at this time. Even a <span
style=color:brown>/dev/zram0</span> of 200% RAM is not unreasonable
*if* the compression ratio is at least 2:1. However, it's possible a
system can get "stuck" in a kind of swap thrashing similar to
conventional swap-on-drive, except it's CPU and memory bound, rather
than IO bound. Feature owner thinks it's better to just oom, instead
of getting overly aggressive with the zram device size.
Conversely it's possible to be too conservative with the size, and
result in more instances of OOM kill. If applying the feature to
upgrades is rejected, it's probably reasonable to increase the cap to
~8GiB. Of course more feedback and testing is needed, and it will be
taken into consideration.
Note that the kernel zram doc says an excessively sized zram device
does come with overhead. Users's can increase the size easily
post-install, a capability they don't easily have with swap-on-drive.
The goal for Fedora 33 is a default that's useful and safe for the
vast majority of use cases.
==== Why not zswap? ====
Zswap† is a similar idea, but with a totally different implementation.
It is swap specific, uses a RAM cache, and requires a conventional
swap partition existing already. It might be true certain workloads
are better suited for using zswap. But swap-on-zram depends only on
volatile storage. This is simpler and it's more secure. Whereas zswap
"spills over" into swap-on-drive and will leak user data if that swap
device isn't encrypted. Some workloads may do better with zswap, and
it's a valid future feature for a new generator, or possibly extend
zram-generator to support it via the configuration file. Maybe the
generator could favor zswap when swap-on-drive already exists; and
fallback to swap-on-zram?
†</br >
[https://www.kernel.org/doc/Documentation/vm/zswap.txt kernel.org zswap.txt]
== Benefit to Fedora ==
* significantly improves system responsiveness, especially when swap
is under pressure;
* more secure, user data leaks into swap are on volatile media;
* without swap-on-drive, there's better utilization of a limited
resource: benefit of swap without the drive space consumption;
* complements on-going resource control work, including earlyoom;
* further reduces the time to out-of-memory kill, when workloads exceed limits;
* improves performance for both "no swap" and "existing swap" setups;
== Scope ==
* Proposal owners:
** add zram-generator package to comps and kickstarts as appropriate
** obsolete zram package (used by Fedora IoT)
** means of per edition/spin configurations, if needed
** coordinate a test day
* Other developers:
**Anaconda are agreeable to deprecating their built-in implementation
in favor of swap-on-zram
**RFE's for zram-generator: users are not worse off if they don't
happen. Open request for help, to make it possible. It's much
appreciated.</br >
[https://github.com/systemd/zram-generator/issues/10 RFE: should be
able to set a cap on zram device size #10]</br >
[https://github.com/systemd/zram-generator/issues/8 RFE: should set priority #8]
* Release engineering: [https://pagure.io/releng/issues #9495]
* Policies and guidelines: N/A
* Trademark approval: N/A
== Upgrade/compatibility impact ==
Add Supplements:fedora-release-common to zram-generator to pull it in
on upgrades.
Existing systems without swap will have swap-on-zram enabled.
Existing systems with swap-on-drive, will also have swap-on-zram
enabled (two swap devices), with higher priority for the zram device.
Existing swap-on-drive will not be removed.
'zram' package contains zram-swap.service and associated bash scripts,
and is currently used by Fedora IoT and ARM spins. It will be
obsoleted to avoid conflicting/duplicative swap-on-zram
implementations.
== How To Test ==
Any hardware. Any version of Fedora.
# dnf install zram-generator
# cp /usr/share/doc/zram-generator/zram-generator.conf.example
/etc/systemd/zram-generator.conf
# Edit the configuration
# Reboot
# Check that swap is on a zram device: zramctl, swapon
# Detailed check: journalctl -b -o short-monotonic | grep 'swap\|zram'
# Check that priority is higher than existing swap if two or more are
listed. ## (Enhancement is needed for this.)
Suggested configuration file values:</br >
<span style=color:red>[zram0]</span></br >
<span style=color:red>memory-limit = none</span></br >
<span style=color:red>zram-fraction = 0.5</span></br >
Feel free to run your usual workloads more aggressively or in
parallel. Suspend-to-RAM and suspend-to-drive are expected to continue
to work too (or at least hit all the same bugs as without zram being
used).
Also, you can see the actual compression ratio achieved with the
following command:</br >
<span style=color:red> zramctl </span>
==== Test Day ====
[https://pagure.io/fedora-qa/issue/632 QA: SwapOnzram Test Day] to
discover edge cases, and tweak the default configuration if necessary
to establish a good one-size-fits all approach.
== User Experience ==
The user won't notice anything displeasing. If their usual workload
causes them to dread swap thrashing, they'll be surprised that
thrashing doesn't happen. The user might get curious if they don't
find a swap entry in /etc/fstab. Or if they 'swapon' and see swap
pointing to <span style=color:brown>/dev/zram0</span> instead of a
drive partition or LV.
== Dependencies ==
N/A
== Contingency Plan ==
* Contingency mechanism: Don't ship the generator = big hammer, but
easy. Preferable to ship the generator, but only selectively ship
configuration files = scalpel, pretty easy.
* Contingency deadline: Beta freeze
* Blocks release? No.
* Blocks product? No.
== Documentation ==
Consider adding a hint in an /etc/fstab comment? There is no man page
for this, and the documentation is also minimal, besides what's in
this feature proposal. It's an open question how the user should get
more information on how to configure and tweak it. But then, they
don't have that for swap today either. There's just institutional
knowledge.
Hence, a strong test day, with a lot of people and press coverage of
the feature, might help spread the word for institutional knowledge
changes coming.
Ideas welcome.
== Release Notes ==
Pending feedback and test day.
--
Ben Cotton
He / Him / His
Senior Program Manager, Fedora & CentOS Stream
Red Hat
TZ=America/Indiana/Indianapolis
3 years, 8 months
Fedora 33 System-Wide Change proposal: Make btrfs the default file
system for desktop variants
by Ben Cotton
https://fedoraproject.org/wiki/Changes/BtrfsByDefault
== Summary ==
For laptop and workstation installs of Fedora, we want to provide file
system features to users in a transparent fashion. We want to add new
features, while reducing the amount of expertise needed to deal with
situations like [https://pagure.io/fedora-workstation/issue/152
running out of disk space.] Btrfs is well adapted to this role by
design philosophy, let's make it the default.
== Owners ==
* Names: [[User:Chrismurphy|Chris Murphy]], [[User:Ngompa|Neal
Gompa]], [[User:Josef|Josef Bacik]], [[User:Salimma|Michel Alexandre
Salim]], [[User:Dcavalca|Davide Cavalca]], [[User:eeickmeyer|Erich
Eickmeyer]], [[User:ignatenkobrain|Igor Raits]],
[[User:Raveit65|Wolfgang Ulbrich]], [[User:Zsun|Zamir SUN]],
[[User:rdieter|Rex Dieter]], [[User:grinnz|Dan Book]],
[[User:nonamedotc|Mukundan Ragavan]]
* Emails: chrismurphy(a)fedoraproject.org, ngompa13(a)gmail.com,
josef(a)toxicpanda.com, michel(a)michel-slm.name, dcavalca(a)fb.com,
erich(a)ericheickmeyer.com, ignatenkobrain(a)fedoraproject.org,
fedora(a)raveit.de, zsun(a)fedoraproject.org, rdieter(a)gmail.com,
grinnz(a)gmail.com, nonamedotc(a)gmail.com
* Products: All desktop editions, spins, and labs
* Responsible WGs: Workstation Working Group, KDE Special Interest Group
== Detailed Description ==
Fedora desktop edition/spin variants will switch to using Btrfs as the
filesystem by default for new installs. Labs derived from these
variants inherit this change, and other editions may opt into this
change.
The change is based on the installer's custom partitioning Btrfs
preset. It's been well tested for 7 years.
'''''Current partitioning'''''<br />
<span style="color: tomato">vg/root</span> LV mounted at <span
style="color: tomato">/</span> and a <span style="color:
tomato">vg/home</span> LV mounted at <span style="color:
tomato">/home</span>. These are separate file system volumes, with
separate free/used space.
'''''Proposed partitioning'''''<br />
<span style="color: tomato">root</span> subvolume mounted at <span
style="color: tomato">/</span> and <span style="color:
tomato">home</span> subvolume mounted at <span style="color:
tomato">/home</span>. Subvolumes don't have size, they act mostly like
directories, space is shared.
'''''Unchanged'''''<br />
<span style="color: tomato">/boot</span> will be a small ext4 volume.
A separate boot is needed to boot dm-crypt sysroot installations; it's
less complicated to keep the layout the same, regardless of whether
sysroot is encrypted. There will be no automatic snapshots/rollbacks.
If you select to encrypt your data, LUKS (dm-crypt) will be still used
as it is today (with the small difference that Btrfs is used instead
of LVM+Ext4). There is upstream work on getting native encryption for
Btrfs that will be considered once ready and is subject of a different
change proposal in a future Fedora release.
=== Optimizations (Optional) ===
The detailed description above is the proposal. It's intended to be a
minimalist and transparent switch. It's also the same as was
[[Features/F16BtrfsDefaultFs|proposed]] (and
[https://lwn.net/Articles/446925/ accepted]) for Fedora 16. The
following optimizations improve on the proposal, but are not critical.
They are also transparent to most users. The general idea is agree to
the base proposal first, and then consider these as enhancements.
==== Boot on Btrfs ====
* Instead of a 1G ext4 boot, create a 1G Btrfs boot.
* Advantage: Makes it possible to include in a snapshot and rollback
regime. GRUB has stable support for Btrfs for 10+ years.
* Scope: Contingent on bootloader and installer team review and
approval. blivet should use <code>mkfs.btrfs --mixed</code>.
==== Compression ====
* Enable transparent compression using zstd on select directories:
<span style="color: tomato">/usr</span> <span style="color:
tomato">/var/lib/flatpak</span> <span style="color:
tomato">~/.local/share/flatpak</span>
* Advantage: Saves space and significantly increase the lifespan of
flash-based media by reducing write amplification. It may improve
performance in some instances.
* Scope: Contingent on installer team review and approval to enhance
anaconda to perform the installation using <code>mount -o
compress=zstd</code>, then set the proper XATTR for each directory.
The XATTR can't be set until after the directories are created via:
rsync, rpm, or unsquashfs based installation.
==== Additional subvolumes ====
* <span style="color: tomato">/var/log/</span> <span style="color:
tomato">/var/lib/libvirt/images</span> and <span style="color:
tomato">~/.local/share/gnome-boxes/images/</span> will use separate
subvolumes.
* Advantage: Makes it easier to excluded them from snapshots,
rollbacks, and send/receive. (Btrfs snapshotting is not recursive, it
stops at a nested subvolume.)
* Scope: Anaconda knows how to do this already, just change the
kickstart to add additional subvolumes (minus the subvolume in <span
style="color: tomato">~/</span>. GNOME Boxes will need enhancement to
detect that the user home is on Btrfs and create <span style="color:
tomato">~/.local/share/gnome-boxes/images/</span> as a subvolume.
== Feedback ==
==== Red Hat doesn't support Btrfs? Can Fedora do this? ====
Red Hat supports Fedora well, in many ways. But Fedora already works
closely with, and depends on, upstreams. And this will be one of them.
That's an important consideration for this proposal. The community has
a stake in ensuring it is supported. Red Hat will never support Btrfs
if Fedora rejects it. Fedora necessarily needs to be first, and make
the persuasive case that it solves more problems than alternatives.
Feature owners believe it does, hands down.
The Btrfs community has users that have been using it for most of the
past decade at scale. It's been the default on openSUSE (and SUSE
Linux Enterprise) since 2014, and Facebook has been using it for all
their OS and data volumes, in their data centers, for almost as long.
Btrfs is a mature, well-understood, and battle-tested file system,
used on both desktop/container and server/cloud use-cases. We do have
developers of the Btrfs filesystem maintaining and supporting the code
in Fedora, one is a Change owner, so issues that are pinned to Btrfs
can be addressed quickly.
==== What about device-mapper alternatives? ====
dm-thin (thin provisioning):
[[https://pagure.io/fedora-workstation/issue/152 Issue #152] still
happens, because the installer won't over provision by default. It
still requires manual intervention by the user to identify and resolve
the problem. Upon growing a file system on dm-thin, the pool is over
committed, and file system sizes become a fantasy: they don't add up
to the total physical storage available. The truth of used and free
space is only known by the thin pool, and CLI and GUI programs are
unprepared for this. Integration points like rpm free space checks or
GNOME disk-space warnings would have to be adapted as well.
dm-vdo: is not yet merged, and isn't as straightforward to selectively
enable per directory and per file, as is the case on Btrfs using
<code>chattr +c</code> on <span style="color:
tomato">/var/lib/flatpaks/</span>.
Btrfs solves the problems that need solving, with few side effects or
pitfalls for users. It has more features we can take advantage of
immediately and transparently: compression, integrity, and IO
isolation. Many Btrfs features and optimizations can be opted into
selectively per directory or file, such as compression and nodatacow,
rather than as a layer that's either on or off.
==== What about UI/UX and integration in the desktop? ====
If Btrfs isn't the default file system, there's no commitment, nor
reason to work on any UI/UX integration. There are ideas to make
certain features discoverable: selective compression; systemd-homed
may take advantage of either Btrfs online resize, or near-term planned
native encryption, which could make it possible to live convert
non-encrypted homes to encrypted; and system snapshot and rollbacks.
Anaconda already has sophisticated Btrfs integration.
==== What Btrfs features are recommended and supported? ====
The primary goal of this feature is to be largely transparent to the
user. It does not require or expect users to learn new commands, or to
engage in peculiar maintenance rituals.
The full set of Btrfs features that is considered stable and enabled
by default upstream will be enabled in Fedora. Fedora is a community
project. What is supported within Fedora depends on what the community
decides to put forward in terms of resources.
The upstream [https://btrfs.wiki.kernel.org/index.php/Status Btrfs
feature status page].
==== Are subvolumes really mostly like directories? ====
Subvolumes behave like directories in terms of navigation in both the
GUI and CLI, e.g. <code>cp</code>, <code>mv</code>, <code>du</code>,
owner/permissions, and SELinux labels. They also share space, just
like a directory.
But it is an incomplete answer.
A subvolume is an independent file tree, with its own POSIX namespace,
and has its own pool of inodes. This means inode numbers repeat
themselves on a Btrfs volume. Inodes are only unique within a given
subvolume. A subvolume has its own st_dev, so if you use <code>stat
FILE</code> it reports a device value referring to the subvolume the
file is in. And it also means hard links can't be created between
subvolumes. From this perspective, subvolumes start looking more like
a separate file system. But subvolumes share most of the other trees,
so they're not truly independent file systems. They're also not block
devices.
== Benefit to Fedora ==
Problems Btrfs helps solve:
* Users running out of free space on either <span style="color:
tomato">/</span> or <span style="color: tomato">/home</span>
[https://pagure.io/fedora-workstation/issue/152 Workstation issue
#152]
** "one big file system": no hard barriers like partitions or logical volumes
** transparent compression: significantly reduces write amplification,
improves lifespan of storage hardware
** reflinks and snapshots are more efficient for use cases like
containers (Podman supports both)
* Storage devices can be flaky, resulting in data corruption
** Everything is checksummed and verified on every read
** Corrupt data results in EIO (input/output error), instead of
resulting in application confusion, and isn't replicated into backups
and archives
* Poor desktop responsiveness when under pressure
[https://pagure.io/fedora-workstation/issue/154 Workstation issue
#154]
** Currently only Btrfs has proper IO isolation capability via cgroups2
** Completes the resource control picture: memory, cpu, IO isolation
* File system resize
** Online shrink and grow are fundamental to the design
* Complex storage setups are... complicated
** Simple and comprehensive command interface. One master command
** Simpler to boot, all code is in the kernel, no initramfs complexities
** Simple and efficient file system replication, including incremental
backups, with <code>btrfs send</code> and <code>btrfs receive</code>
== Scope ==
* Proposal owners:
** Submit PR's for Anaconda to change <code>default_scheme =
BTRFS</code> to the proper product files.
** Multiple test days: build community support network
** Aid with documentation
* Other developers:
** Anaconda, review PRs and merge
** Bootloader team, review PRs and merge
** Recommended optimization <code>chattr +C</code> set on the
containing directory for virt-manager and GNOME Boxes.
* Release engineering: [https://pagure.io/releng/issue/9545 #9545]
* Policies and guidelines: N/A
* Trademark approval: N/A
== Upgrade/compatibility impact ==
Change will not affect upgrades.
Documentation will be provided for existing Btrfs users to "retrofit"
their setups to that of a default Btrfs installation (base plus any
approved options).
== How To Test ==
'''''Today'''''<br />
Do a custom partitioning installation; change the scheme drop-down
menu to Btrfs; click the blue "automatically create partitions"; and
install.<br />
Fedora 31, 32, Rawhide, on x86_64 and ARM.
'''''Once change lands'''''<br />
It should be simple enough to test, just do a normal install.
== User Experience ==
==== Pros ====
* Mostly transparent
* Space savings from compression
* Longer lifespan of hardware, also from compression.
* Utilities for used and free space, CLI and GUI, are expected to
behave the same. No special commands are required.
* More detailed information can be revealed by <code>btrfs</code>
specific commands.
==== Enhancement opportunities ====
[https://bugzilla.redhat.com/show_bug.cgi?id=906591 updatedb does not
index /home when /home is a bind mount] Also can affected rpm-ostree
installations, including Silverblue.
[https://gitlab.gnome.org/GNOME/gnome-usage/-/issues/49 GNOME Usage:
Incorrect numbers when using multiple btrfs subvolumes] This isn't
Btrfs specific, happens with "one big ext4" volume as well.
[https://gitlab.gnome.org/GNOME/gnome-boxes/-/issues/88 GNOME Boxes,
RFE: create qcow2 with 'nocow' option when on btrfs /home] This is
Btrfs specific, and is a recommended optimization for both GNOME Boxes
and virt-manager.
[https://github.com/containers/libpod/issues/6563 containers/libpod:
automatically use btrfs driver if on btrfs]
== Dependencies ==
None.
== Contingency Plan ==
* Contingency mechanism: Owner will revert changes back to LVM+ext4
* Contingency deadline: Beta freeze
* Blocks release? Yes
* Blocks product? Workstation and KDE
== Documentation ==
Strictly speaking no documentation is required reading for users. But
there will be some Fedora documentation to help get the ball rolling.
For those who want to know more:
[https://btrfs.wiki.kernel.org/index.php/Main_Page btrfs wiki main
page and full feature list.]
<code>man 5 btrfs</code> contains: mount options, features, swapfile
support, checksum algorithms, and more<br />
<code>man btrfs</code> contains an overview of the btrfs subcommands<br />
<code>man btrfs <nowiki><subcommand></nowiki></code> will show the man
page for that subcommand
NOTE: The btrfs command will accept partial subcommands, as long as
it's not ambiguous. These are equivalent commands:<br />
<code>btrfs subvolume snapshot</code><br />
<code>btrfs sub snap</code><br />
<code>btrfs su sn</code>
You'll discover your own convention. It might be preferable to write
out the full command on forums and lists, but then maybe some folks
don't learn about this useful shortcut?
For those who want to know a lot more:
[https://btrfs.wiki.kernel.org/index.php/Main_Page#Developer_documentation
Btrfs developer documentation]<br />
[https://github.com/btrfs/btrfs-dev-docs/blob/master/trees.txt Btrfs trees]
== Release Notes ==
The default file system on the desktop is Btrfs.
--
Ben Cotton
He / Him / His
Senior Program Manager, Fedora & CentOS Stream
Red Hat
TZ=America/Indiana/Indianapolis
3 years, 8 months
List of long term FTBFS packages to be retired in August
by Miro Hrončok
Dear maintainers.
Based on the current fail to build from source policy, the following packages
will be retired from Fedora 33 approximately one week before branching (August
2020).
Policy:
https://docs.fedoraproject.org/en-US/fesco/Fails_to_build_from_source_Fai...
Note that some listed packages are orphaned and hence may be retired even sooner.
The packages in rawhide were not successfully built at least since Fedora 31.
This report is based on dist tags.
Packages collected via:
https://github.com/hroncok/fedora-report-ftbfs-retirements/blob/master/ft...
If you see a package that was built, please let me know.
If you see a package that should be exempted from the process, please let me
know and we can work together to get a FESCo approval for that.
If you see a package that can be rebuilt, please do so.
Package (co)maintainers Latest build
====================================================================
OpenCoarrays jussilehtola Fedora 30
gpscorrelate till Fedora 30
js-jquery-jqplot xavierb Fedora 30
js-jquery1 nodejs-sig, patches, vondruch Fedora 30
js-jquery2 vondruch Fedora 30
js-sizzle nodejs-sig, patches, vondruch Fedora 30
nodejs-path-type jsmith, nodejs-sig Fedora 30
nodejs-temp-write jsmith Fedora 30
nodejs-unique-stream jsmith, nodejs-sig Fedora 30
ocaml-pxp orphan Fedora 30
ocaml-ulex orphan Fedora 30
orpie bowlofeggs, jaredwallace Fedora 30
rubygem-ruby-hmac humaton, mmorsi Fedora 30
xvarstar orphan Fedora 30
The following packages require above mentioned packages:
Depending on: gpscorrelate (1)
foxtrotgps (maintained by: bubeck)
foxtrotgps-1.2.2-5.fc33.x86_64 requires gpscorrelate = 1.6.1-27.fc30
Depending on: js-jquery-jqplot (1)
sympa (maintained by: xavierb)
sympa-6.2.56-1.fc33.src requires js-jquery-jqplot = 1.0.9-3.fc30
sympa-6.2.56-1.fc33.x86_64 requires js-jquery-jqplot = 1.0.9-3.fc30
Depending on: js-jquery1 (69)
R-profvis (maintained by: qulogic)
R-profvis-0.3.6-3.fc33.src requires js-jquery1 = 1.12.4-7.fc30
R-profvis-0.3.6-3.fc33.x86_64 requires js-jquery1 = 1.12.4-7.fc30
R-rmarkdown (maintained by: qulogic)
R-rmarkdown-2.2-1.fc33.noarch requires js-jquery1 = 1.12.4-7.fc30
R-rmarkdown-2.2-1.fc33.src requires js-jquery1 = 1.12.4-7.fc30
copr-frontend (maintained by: clime, copr-sig, dturecek, frostyx, msuchy, praiskup)
copr-frontend-1.166-1.fc33.noarch requires js-jquery1 = 1.12.4-7.fc30
ghc-pretty-show (maintained by: mathstuf)
ghc-pretty-show-1.9.5-3.fc32.x86_64 requires js-jquery1 = 1.12.4-7.fc30
mkdocs (maintained by: cheeselee)
mkdocs-1.1.2-1.fc33.noarch requires js-jquery1 = 1.12.4-7.fc30
mkdocs-1.1.2-1.fc33.src requires js-jquery1 = 1.12.4-7.fc30
python-XStatic-jQuery (maintained by: mrunge, openstack-sig, rdopiera)
python3-XStatic-jQuery-3.4.1.0-2.fc33.noarch requires js-jquery1 = 1.12.4-7.fc30
python-sphinx-bootstrap-theme (maintained by: besser82, sic)
python3-sphinx-bootstrap-theme-0.8.0-3.fc33.noarch requires js-jquery1 =
1.12.4-7.fc30
rubygem-apipie-rails (maintained by: jaruga, ruby-packagers-sig, vondruch)
rubygem-apipie-rails-0.5.5-6.fc32.noarch requires js-jquery1 = 1.12.4-7.fc30
R-BiocFileCache (maintained by: spot)
R-BiocFileCache-1.12.0-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-DBItest (maintained by: qulogic)
R-DBItest-1.7.0-1.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-V8 (maintained by: qulogic)
R-V8-3.1.0-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-broom (maintained by: qulogic)
R-broom-0.5.6-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-cellranger (maintained by: qulogic)
R-cellranger-1.1.0-6.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-clipr (maintained by: qulogic)
R-clipr-0.7.0-3.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-dbplyr (maintained by: qulogic)
R-dbplyr-1.4.3-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-devtools (maintained by: qulogic)
R-devtools-2.1.0-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-diffobj (maintained by: qulogic)
R-diffobj-0.3.0-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-dplyr (maintained by: qulogic)
R-dplyr-0.8.5-2.fc33~bootstrap.src requires R-rmarkdown = 2.2-1.fc33
R-dtplyr (maintained by: qulogic)
R-dtplyr-1.0.1-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-foghorn (maintained by: qulogic)
R-foghorn-1.1.0-5.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-forcats (maintained by: qulogic)
R-forcats-0.5.0-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-formatR (maintained by: qulogic)
R-formatR-1.7-5.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-fs (maintained by: qulogic)
R-fs-1.4.1-1.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-gargle (maintained by: qulogic)
R-gargle-0.5.0-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-ggplot2 (maintained by: qulogic)
R-ggplot2-3.2.1-3.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-gmailr (maintained by: qulogic)
R-gmailr-1.0.0-3.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-haven (maintained by: qulogic)
R-haven-2.3.0-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-httr (maintained by: qulogic)
R-httr-1.4.1-3.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-hunspell (maintained by: qulogic)
R-hunspell-3.0-5.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-jose (maintained by: qulogic)
R-jose-1.0-5.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-jqr (maintained by: qulogic)
R-jqr-1.1.0-5.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-later (maintained by: qulogic)
R-later-1.1.0.1-1.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-lazyeval (maintained by: qulogic)
R-lazyeval-0.2.2-4.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-lifecycle (maintained by: qulogic)
R-lifecycle-0.2.0-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-lintr (maintained by: qulogic)
R-lintr-2.0.1-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-packrat (maintained by: qulogic)
R-packrat-0.5.0-5.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-pkgdown (maintained by: qulogic)
R-pkgdown-1.5.1-2.fc33.noarch requires R(rmarkdown) = 2.2
R-pkgdown-1.5.1-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-polynom (maintained by: qulogic)
R-polynom-1.4.0-5.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-prettydoc (maintained by: qulogic)
R-prettydoc-0.3.1-3.fc33.noarch requires R(rmarkdown) = 2.2
R-prettydoc-0.3.1-3.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-promises (maintained by: qulogic)
R-promises-1.1.0-3.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-purrr (maintained by: qulogic)
R-purrr-0.3.4-2.fc33~bootstrap.src requires R-rmarkdown = 2.2-1.fc33
R-qcc (maintained by: jjmcd)
R-qcc-2.7-1.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-rcmdcheck (maintained by: qulogic)
R-rcmdcheck-1.3.3-5.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-readr (maintained by: qulogic)
R-readr-1.3.1-5.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-readxl (maintained by: qulogic)
R-readxl-1.3.1-5.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-remotes (maintained by: qulogic)
R-remotes-2.1.1-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-reprex (maintained by: qulogic)
R-reprex-0.3.0-5.fc33.noarch requires R(rmarkdown) = 2.2
R-reprex-0.3.0-5.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-reticulate (maintained by: qulogic)
R-reticulate-1.16-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-rex (maintained by: qulogic)
R-rex-1.2.0-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-rhub (maintained by: qulogic)
R-rhub-1.1.1-5.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-rlang (maintained by: qulogic)
R-rlang-0.4.6-3.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-roxygen2 (maintained by: qulogic)
R-roxygen2-7.1.0-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-rsconnect (maintained by: qulogic)
R-rsconnect-0.8.16-3.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-rvest (maintained by: qulogic)
R-rvest-0.3.5-3.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-servr (maintained by: qulogic)
R-servr-0.17-1.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-shiny (maintained by: qulogic)
R-shiny-1.4.0.2-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-showtext (maintained by: qulogic)
R-showtext-0.8.1-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-sodium (maintained by: qulogic)
R-sodium-1.1-4.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-styler (maintained by: qulogic)
R-styler-1.3.2-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-svglite (maintained by: qulogic)
R-svglite-1.2.3-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-systemfonts (maintained by: qulogic)
R-systemfonts-0.2.2-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-tibble (maintained by: qulogic)
R-tibble-3.0.1-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-tidyr (maintained by: qulogic)
R-tidyr-1.1.0-1.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-tidyselect (maintained by: qulogic)
R-tidyselect-1.1.0-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-tufte (maintained by: qulogic)
R-tufte-0.6-2.fc33.noarch requires R(rmarkdown) = 2.2
R-tufte-0.6-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-unitizer (maintained by: qulogic)
R-unitizer-1.4.10-2.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-usethis (maintained by: qulogic)
R-usethis-1.5.1-3.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-websocket (maintained by: qulogic)
R-websocket-1.1.0-3.fc33.src requires R-rmarkdown = 2.2-1.fc33
R-zeallot (maintained by: qulogic)
R-zeallot-0.1.0-5.fc33.src requires R-rmarkdown = 2.2-1.fc33
Too many dependencies for js-jquery1, not all listed here
Depending on: nodejs-path-type (1)
nodejs-read-pkg (maintained by: jsmith, nodejs-sig)
nodejs-read-pkg-2.0.0-7.fc32.noarch requires npm(path-type) = 2.0.0
Affected (co)maintainers (directly and indirectly):
besser82: js-jquery1
bowlofeggs: orpie
bubeck: gpscorrelate
cheeselee: js-jquery1
clime: js-jquery1
copr-sig: js-jquery1
dturecek: js-jquery1
frostyx: js-jquery1
humaton: rubygem-ruby-hmac
jaredwallace: orpie
jaruga: js-jquery1
jjmcd: js-jquery1
jsmith: nodejs-temp-write, nodejs-path-type, nodejs-unique-stream
jussilehtola: OpenCoarrays
mathstuf: js-jquery1
mmorsi: rubygem-ruby-hmac
mrunge: js-jquery1
msuchy: js-jquery1
nodejs-sig: js-sizzle, js-jquery1, nodejs-path-type, nodejs-unique-stream
openstack-sig: js-jquery1
patches: js-sizzle, js-jquery1
praiskup: js-jquery1
qulogic: js-jquery1
rdopiera: js-jquery1
ruby-packagers-sig: js-jquery1
sic: js-jquery1
spot: js-jquery1
till: gpscorrelate
vondruch: js-sizzle, js-jquery1, js-jquery2
xavierb: js-jquery-jqplot
3 years, 8 months
Fedora 33 Self-Contained Change proposal: GHC 8.8 and Haskell
Stackage LTS 16
by Ben Cotton
https://fedoraproject.org/wiki/Changes/GHC_8.8_and_LTS16
== Summary ==
The GHC Haskell compiler will be updated from major version 8.6 to 8.8,
and Haskell packages will be updated from Stackage LTS 14 to LTS 16 versions.
== Owner ==
* Name: [[User:Petersen| Jens Petersen]]
* Email: <petersen(a)redhat.com>
== Detailed Description ==
For Fedora 33, the GHC Haskell compiler will be updated from version
8.6.5 to 8.8.3 (based on the ghc:8.8 module stream).
Along with this Haskell packages in [https://www.stackage.org
Stackage] will be updated from the versions in LTS 14 to LTS 16.
Haskell packages not in Stackage will be updated to the latest current
version in [https://hackage.haskell.org Hackage].
== Benefit to Fedora ==
Fedora users will benefit from access to the latest stable Haskell
compiler release, package tools, and current stable Haskell packages
from Stackage LTS.
GHC 8.8 features a new code layout algorithm for x86, the final
implementation of the MonadFail proposal, and also many bugfixes (see
the Documentation links for more details).
== Scope ==
* Proposal owners:
** rebase ghc to 8.8.3
** update ghc-rpm-macros to the final version for F33 GA
** refresh packagings with the latest cabal-rpm release
** update packages to latest Stackage LTS 16 versions using cabal-rpm
** build all the packages in a Koji sidetag repo in dependency order
** When finished push all builds through Bodhi to Rawhide before the
mass rebuild
* Other developers: N/A (not a System Wide Change)
* Release engineering: N/A
* Policies and guidelines: N/A (not a System Wide Change)
* Trademark approval: N/A (not needed for this Change)
== Upgrade/compatibility impact ==
Any dropped packages will have obsoletes added.
Otherwise there should not be any direct upgrade impact.
Users' Haskell projects will get rebuilt with ghc-8.8 when they next
build them and might need minor adjustments.
== How To Test ==
* install ghc and cabal-install
* install pandoc, ShellCheck, git-annex
* install ghc-*-devel or ghc-*-prof or ghc-*-doc
* cabal-rpm builddep <favouritepackage>; cabal install <favouritepackage>
* test upgrades of F32 packages to F33
== User Experience ==
Users will have the most recent stable major version of `ghc` and
Haskell libraries and tools available to them.
This makes it easier to build the latest versions of Haskell projects.
Also these updates include a major new version 3 of the Haskell
`Cabal` library and `cabal-install` packaging tool
with important new features and enhancements, and `stack` is being
updated from 2.1 to 2.3.
== Dependencies ==
N/A (not a System Wide Change)
== Contingency Plan ==
* Contingency mechanism: (What to do? Who will do it?)
** Change owner will drop the new builds and revert back to the versions in F32.
* Contingency deadline: N/A (not a System Wide Change)
* Blocks release? N/A (not a System Wide Change)
== Documentation ==
* https://downloads.haskell.org/~ghc/8.8.3/docs/html/users_guide/8.8.1-note...
* https://hackage.haskell.org/package/cabal-install-3.0.0.0/changelog
* https://gitlab.haskell.org/ghc/ghc/-/wikis/migration/8.8
--
Ben Cotton
He / Him / His
Senior Program Manager, Fedora & CentOS Stream
Red Hat
TZ=America/Indiana/Indianapolis
3 years, 9 months
RPM-level auto release and changelog bumping - Fedora 33 System-Wide
Change proposal
by Ben Cotton
https://fedoraproject.org/wiki/Changes/rpm_level_auto_release_and_changel...
== Summary ==
redhat-rpm-config will be updated so users of the auto framework get
automated release and changelog bumping.
== Owner ==
* Name: [[User:nim| Nicolas Mailhot]]
* Email: <nicolas.mailhot at laposte.net>
== Detailed Description ==
This is a system-wide change because all packages build with
redhat-rpm-config, but it only concerns packages that opted to use
this part of redhat-rpm-config (auto framework).
The change will make those packages auto-bump and auto-changelog at
the rpm level, in an infrastructure-independent way.
== Benefit to Fedora ==
Autobumping removes a huge packager shore and makes timestamping in
changelogs more reliable.
== Scope ==
* Proposal owners: The feature is coded and works at the rpm level.
Unfortunately, mock filters away the srpms containing the bump state,
so it does not work in upper layers.
* Other developers: The feature requires buy-in by mock developers
(and probably koji developers) to lift the restrictions that block it
above the rpm level. Also, it requires a mechanism to pass the user
name and email that will be used in bumped changelogs (defining two
variables in ~/.rpmmacros is sufficient at rpm level)
* Mock issue: https://github.com/rpm-software-management/mock/issues/599
* Release engineering: https://pagure.io/releng/issue/9567
* Policies and guidelines: maybe eventually if things work out on the
technical level
* FPC issue: https://pagure.io/packaging-committee/issue/998
* Trademark approval: N/A (not needed for this Change)
== Upgrade/compatibility impact ==
This is a pure build tooling update, it changes how things are built
not what is built.
== How To Test ==
A redhat-rpm-config packages with the changes and some example
packages are available in
https://copr.fedorainfracloud.org/coprs/nim/refactoring-forge-patches-aut...
Since the mock/copr layer is currently blocking the feature, you need
to install the redhat-rpm-config and forge macro packages available in
this repo locally. Afterwards you can take any of the example packages
in the repo and rebuild them with rpmbuild -ba to your heart content,
and see the releases bump and the changelogs being updated
accordingly.
To get beautiful changelogs, you also need to add
<pre>
%buildsys_name Your name
%buildsys_email Your email
</pre>
in ~/.rpmmacros
== User Experience ==
N/A Packager experience change only
== Dependencies ==
The change is a spin-off of
https://fedoraproject.org/wiki/Changes/Patches_in_Forge_macros_-_Auto_mac...
Therefore, it depends on the success of that other change and will
probably need rebasing if the code in this other change evolves during
the redhat-rpm-config merge.
It also depends on mock / copr/ koji buy-in and changes, that may add
their own requirements.
== Contingency Plan ==
There is no contingency plan because the change will happen or not at all.
== Documentation ==
There is as much documentation as the average redhat-rpm-config change
(ie comments in the macro files themselves)
== Release Notes ==
N/A Packager productivity change only
--
Ben Cotton
He / Him / His
Senior Program Manager, Fedora & CentOS Stream
Red Hat
TZ=America/Indiana/Indianapolis
3 years, 9 months
Ditch RPM in favor of DPKG
by Dridi Boukelmoune
Greetings packagers,
I know how important RPM is to the Fedora Project, but it breaks
everything downstream and we'd be better off using DPKG as we should
have from day one.
I'm calling this initiative fedpkg: Fedora Embraces DPKG.
A bit of background here: I build both RPMs and DEBs for $DAYJOB and
until recently my workflow was quite painful because I needed extra steps
between git checkout and git push that involves a VM, because what we
ship as apt is in reality apt-rpm.
It finally got enough on my nerves to locally build the things I needed and
after a month I have already amortized my efforts with the time I save not
having to deal with needless extra hoops.
In order to successfully build debs on Fedora I needed 4 packages that
I'm now submitting for review:
https://bugzilla.redhat.com/show_bug.cgi?id=gnu-config
https://bugzilla.redhat.com/show_bug.cgi?id=strip-nondeterminism
https://bugzilla.redhat.com/show_bug.cgi?id=sbuild
https://bugzilla.redhat.com/show_bug.cgi?id=apt
I need more than reviews here.
Three of those packages are heavy on Perl code, and I'm not a Perl
Monk. I tried to CC perl-sig as per the guidelines [1] (also tried with
the mailing list address) but bugzilla replied kindly:
CC: perl-sig did not match anything
Apt is a mix of C, Perl and C++ code, so I would be reassured if I
could have a C++ co-maintainer too. I'm only a C developer so if
something goes wrong outside of the C realm that would be helpful.
Two of those packages should be runtime dependencies of debhelper.
The current apt package should be renamed to apt-rpm, I will look up
the procedure for that to happen. I understand that when someone sees
they should run "apt-get install foo" somewhere on the web it's
helpful for non-savvy users that this JustWorks(tm) [2], but apt-rpm is
dead upstream and it shouldn't be advertised as apt.
I hope I CC'd everyone that should get this heads up, and hope to find
help for the reviews and co-maintainership. The packaging does nothing
fancy, there are quirks here and there but overall it was rather easy
to put together. And of course I would be happy to help with reviews
too in exchange.
And thanks again to the mock developers, its design is so much better
than either sbuild or pdebuild that I barely have pain points left when it
comes to RPM packaging.
Thanks,
Dridi
[1] https://docs.fedoraproject.org/en-US/packaging-guidelines/Perl/#_perl_sig
[2] I'm not against apt-rpm in the base install for example
3 years, 9 months
fedora-minimal container and registry negative feedback
by Dridi Boukelmoune
Greetings,
I'm not sure whether the minimization effort is still going on but I
wanted to report the pitfalls I ran into moving from the fedora Docker
container to fedora-minimal.
For starters I was surprised by the absence of DNF and I had to find
by myself, I don't remember how, that MicroDNF was present instead.
So my first bit of feedback concerns the container registry. It looks a
bit... out of shape?
I get an empty page [1] when I click on the fedora-minimal container
and as of today it is by far not the only blank page. I had to keep
opening more until I found one [2] with contents. Ironically a docker
container for another container runtime...
Even when there is content, it doesn't tell much. This is where I
would hope to get the information about MicroDNF.
The other and more significant pitfall I ran into was the lack of a
timezone database. This manifests either as missing tz information
or reports of a corrupted tz database, depending on the application
I'm trying to run. Installing tzdata was a no-op, it turns out to be
installed by default but some of its contents are removed post
installation. Again, something trivial to fix, but it should be
documented on the registry:
microdnf reinstall tzdata
By comparison dockerhub, from which I used to pull fedora images
before moving to fedora-minimal has a nice landing page [3] and
maybe it's also failing to document pitfalls but so far the base image
never surprised me.
Other than that, it seems to hum along, thanks!
Dridi
[1] https://registry.fedoraproject.org/repo/fedora-minimal/tags/
[2] https://registry.fedoraproject.org/repo/flatpak-build-base/tags/
[3] https://hub.docker.com/_/fedora
3 years, 9 months
