RFC: Modularity Simplified
by Igor Gnatenko
Hello fellows,
After last publication on LWN about Fedora Modularity mess, I think it
is time to describe the idea I was proposing internally with few other
folks (Adam Samalik, Brian Exelbierd) back in the RH times.
Before I actually go deep, I'll try to answer main questions to myself
(so that you can understand why I am proposing this particular thing).
1. Do we want to package multiple streams only for "leaf" software or
any kind of it?
I believe that we need both, and we do support both. However, it might
not look as nice as it could:
* Need to create multiple repos for different "streams"
* Need to maintain epel7/epel8/f30/f31/master branches
* Package names have to be "mangled" (e.g. mozjs38, mozjs60) and it is
manual work
* However, those are supposed to be (according to the guidelines)
parallel-installable (and not be conflicting in any way)
Doing git merge / git cherry-pick and maintaining many git remotes
requires some advanced git knowledge and a time. And we can't actually
have multiple versions of a package with same name (without "mangled"
names) in a repo due to the way how our buildsystem works (and not
only buildsystem, with some caveats).
We do have some kind of a solution for multiple releases building from
one branch (package.cfg), however this work has been never finished,
thus there are many problems with this approach.
2. Do we want to support buildtime-only packages?
I would rather generalize this category as "less-supported packages".
I maintain 800+ Rust packages and very often I need to update them to
an incompatible version. In Rawhide I just do it, update all dependent
packages to use new version, and if I can't do that for some reason,
create "compat" package. Obviously, all patches are sent to the
upstream.
Upstreams are removing features, I need to deal with Obsoletes but I
simply can't continuously add new Obsoletes into the
fedora-obsolete-packages. And what for if they are used only during
build of other, more important, packages? Why do I have to spend time
with upgradepaths? I definitely want some mechanism which will tell to
user that "THIS PACKAGE IS NOT FULLY SUPPORTED."
Obviously, for packages which are used in runtime need a proper
support as we do today for all packages to share work (that's the
place where I agree with Kevin Kofler.)
3. Can we have different lifecycles for the software in Fedora?
Right now we have to keep all versions of software which was there at
GA point. And from the updates repo. We never remove packages
entirely. That said, if package was there at GA time, it will have to
be "supported" until the end of that Fedora release.
I don't think we can (should?) do much in this regard. If we make
packages to build from "stream" branch, we can put an information to
that branch that this package should be built for all distributions
(Fedora, EPEL) until this date. Or even store this info somewhere else
like PDC (yes, I know we want to kill it). fedpkg build will take this
into account and submit proper builds. And we can design some API in
infra which would tell until when some particular stream of a package
is supported. Much like it is done with Fedora releases & GNOME
Software.
4. Do we want to have some kind of "stream expansion" where software
builds against all Pythons, Perls and whatnot?
I think this should be conscious choice of a distribution, and in
specific cases, maintainer. Using some examples from previous threads,
why does bugzilla have to be built against 2 different versions of
perl and users could choose? I think maintainer should choose one
version of perl and let bugzilla use it. Being able to build
combinations of software is definitely nice, but I don't think this
should be standard practice. openSUSE does that with ruby and python
(they build all modules automatically for all versions) and I like
this. But having all packages built against all is just combinatoric
explosion. Given how many updates have feedback in Bodhi, I'm pretty
sure 99% of combination won't be tested (or even installed?) ever.
5. Are we still trying to be a Linux distribution or we are just
letting people to do whatever they want in our infrastructure?
This question bothers me from time to time and I don't have answer.
For example, Modularity is very flexible and I very often find people
saying that you can expand this and that, but in Fedora we limit its
usefulness. Do we actually need to develop something like this
(knowing in advance that probably nobody outside of Fedora/RHEL will
be using this software / technology)? Are we trying to create
technologies which would be very extensible and used in other
distributions instead of solving some specific problems? If former,
why don't we talk to others about things in advance and not getting
other people to work on these cool things?
===
How I would imagine having multiple streams:
* rpms/nodejs has multiple branches - 10, 11, 12
* in all of them, nodejs.spec exists as-is without "mangled" name
(just with different versions and such)
* each has package.cfg or some alternative which specifies what is the
EOL and/or for which releases to build
* when builds are submitted, Koji automatically adds suffix to a main
package and subpackages containing branch name
* during the rpmbuild, extra provides are added (for the unversioned
names and indication of a stream) and conflicts (possibly depending on
some macro which user can disable) automatically
Notes:
* If software needs to specify that it wants 9 ≤ nodejs ≤ 11 it can
describe this in standard way (Requires: (nodejs >= 9 with nodejs <=
11)) and one will be picked up automatically
* If user wants to switch from 9 to 10, he can run `dnf swap nodejs9 nodejs10`
* If that requires some conflicting dependency to be switched, it will
be switched automatically
* Packages produced from nodejs.src have Provides: stream(nodejs)
stream(nodejs:9) and Conflicts: stream(nodejs) so that it is
explicitly not possible to install 9 and 10 at the same time
* If desired, packages can depend on a specific stream via Requires:
stream(nodejs:10) without actually depending on nodejs (this requires
some small piece of code in libsolv)
* In the very similar way, user can lock themselves to a specific
stream by writing some conf file which DNF would read (if point above
is implemented, about just tens of lines of the code in libdnf, or
even libsolv can be teached about that as well)
* DNF can be teached about these special things so that you can
automatically swap conflicting dependencies, but lock yourself to some
streams of a package
* All standard Conflicts/Requires/Obsoletes/… will simply work
* With side tags on demand it is now much easier
* Mass rebuild scripts will be teached about stream branches and will
build packages properly
* After branching, there will be a script which untags all packages
which should not be supported in the new release
* fedpkg will be teached about stream branches and will show which
branches build where, so that maintainers can easily maintain them
* Packaging guidelines should be adopted to accept conflicting
packages and tooling should be improved to show the conflicts and how
to resolve them
* fedora-release will have Suggests: stream(nodejs:10) so that when
user types dnf install nodejs, for example nodejs10 would be picked up
(instead of nodejs12)
* Some macros can be added to generate virtual package with "default
stream", so that there will be actual nodejs package depending on
nodejs12 if that would be default in given Fedora release
* Going further, I would extend Koji with fedora plugin which would
deal with the build submission instead of having this logic on clients
so that all people get same result
===
Now let me quote Matthew Miller with the list of requirements and
answer how we achieve them.
1. Users should have alternate streams of software available.
I think this is achieved with what I have said above. We will ship
nodejs9, nodejs10 at the same time.
2. Those alternate streams should be able to have different lifecycles.
nodejs 9 and 10 can automatically build for all distros until EOL
(which would be stored somewhere). EOL-date aware tooling (fedpkg)
would not build for excluded versions of distros and build for distros
where EOL of distro is further than EOL of component.
3. Packaging an individual stream for multiple outputs should be
easier than before.
As I wrote above, we will have one nodejs rpm repo with multiple
branches (aka streams) and no name mangling needed. fedpkg build will
spawn builds for all releases matching criteria (EOL date, excluded
distributions).
===
This is not the exact approach we have tried back in the days, but it
is very similar to that. Unfortunately I did not save any documents
from those times but Adam or Brian probably still can find them (since
they are RH internal).
4 years, 4 months
Summary/Minutes from today's FESCo Meeting (2019-12-02)
by Justin Forbes
=====================================
#fedora-meeting-1: FESCO (2019-12-02)
=====================================
Meeting started by jforbes at 15:00:40 UTC. The full logs are available
at
https://meetbot.fedoraproject.org/fedora-meeting-1/2019-12-02/fesco.2019-...
.
Meeting summary
---------------
* init process (jforbes, 15:00:42)
* #2285 Make Eclipse Installable (jforbes, 15:03:43)
* AGREED: Eclipse can be set as the default stream for F31 and F32
(+6,2,-0) (jforbes, 15:12:11)
* Next week's chair (jforbes, 15:12:27)
* ACTION: ignatenkobrain_ will chair next meeting (jforbes, 15:13:50)
* Open Floor (jforbes, 15:13:54)
* AGREED: Python 2 exception: sugar desktop environment close the
request, and ask the submitters to reopen with a concrete list of
packages. (+7,0,0) (jforbes, 15:22:31)
Meeting ended at 15:25:06 UTC.
Action Items
------------
* ignatenkobrain_ will chair next meeting
Action Items, by person
-----------------------
* ignatenkobrain
* ignatenkobrain_ will chair next meeting
* ignatenkobrain_
* ignatenkobrain_ will chair next meeting
* **UNASSIGNED**
* (none)
People Present (lines said)
---------------------------
* jforbes (27)
* zbyszek (17)
* zodbot (15)
* sgallagh (10)
* nirik (9)
* ignatenkobrain_ (8)
* contyk (4)
* mhroncok_mobile (4)
* bookwar (3)
* mbooth (2)
* ignatenkobrain (1)
* bcotton (1)
* mhroncok (0)
* otaylor (0)
4 years, 4 months
Fedora 32 Self-Contained Change proposal: Rebase apt package from
apt-rpm to Debian's apt
by Ben Cotton
https://fedoraproject.org/wiki/Changes/Move_apt_package_from_RPM_to_DPKG_...
== Summary ==
Currently the apt package in Fedora actually installs apt-rpm,
starting with Fedora 32 it will provide the regular apt software
backed by DPKG.
== Owner ==
* Name: [[User:dridi| Dridi Boukelmoune]], [[User:ngompa | Neal Gompa]]
* Email: dridi(a)fedoraproject.org, ngompa13(a)gmail.com
== Detailed Description ==
The apt package in Fedora does not ship the mainline apt software from
Debian, but rather the apt-rpm fork instead. This allows a user to
copy and paste apt or apt-get commands often found in "Linux"
tutorials. This will usually work, apt-rpm will resolve dependencies
from the Yum/DNF repositories and since our package naming guidelines
often lead to the same package names as apt-based distributions like
Debian and Ubuntu.
The apt-rpm software is dead upstream and doesn't support rich
dependencies or modules. It also has known vulnerabilities and
according to its author other bugs that are never going to be fixed.
== Benefit to Fedora ==
By switching the Fedora apt package from apt-rpm to regular apt we
move from a dead to a living upstream. We also close security holes
and introduce a critical dependency for more packages from the DPKG
ecosystem. It is already possible to build Deb packages in Fedora,
including with pbuilder, an equivalent for mock in the DPKG ecosystem,
however pbuilder uses debootstrap to provision a build environment.
While we may lose the ability to "apt-get install" Fedora packages
from the command line, we also open the gate for sbuild, another mock
equivalent to build Debs in a clean environment. This change offers
more options to target Debian and derivative systems without leaving
the Fedora comfort zone.
== Scope ==
* Proposal owners: re-review of the apt package with the proper
upstream ([https://bugzilla.redhat.com/show_bug.cgi?id=1764813
RH#1764813]), and optionally more dependent packages.
* Other developers: N/A (not a System Wide Change)
* Release engineering: N/A (not needed for this Change)
* Policies and guidelines: As apt would conflict with DNF for the host
system, we may want to ship it without pre-configured repositories.
* Trademark approval: N/A (not needed for this Change)
== Upgrade/compatibility impact ==
Any user actively relying on apt-rpm will lose functionality that
cannot be replaced. Because apt-rpm's version is much lower than the
current apt version, this change will follow the natural upgrade path.
== How To Test ==
If sbuild is packaged in time for the beta, performing builds with
sbuild should be enough to confirm that apt was able to provision a
build root.
== User Experience ==
Anyone used to paste apt-get commands in a terminal will no longer be
able to install or remove Fedora packages this way.
On the other hand anyone needing regular apt tooling will be able to
work with it directly from Fedora.
== Dependencies ==
Apt shouldn't bring more dependencies, it will be the dependency for
more packages from the DPKG ecosystem.
== Contingency Plan ==
* Contingency mechanism: Simply retire apt (apt-rpm)
* Contingency deadline: N/A (not a System Wide Change)
* Blocks release? N/A (not a System Wide Change)
* Blocks product? N/A
== Documentation ==
Once installed, apt ships multiple manual pages available in several
languages. There will no longer be any references in the shipped apt
package documentation of handling RPMs.
== Release Notes ==
The apt package has been rebased from apt-rpm to Debian's apt. This
means that apt no longer supports handling RPMs or managing RPM-based
systems. Please use dnf for software management of RPM-based systems
and containers.
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
4 years, 4 months
Fedora rawhide compose report: 20191202.n.0 changes
by Fedora Rawhide Report
OLD: Fedora-Rawhide-20191201.n.0
NEW: Fedora-Rawhide-20191202.n.0
===== SUMMARY =====
Added images: 0
Dropped images: 3
Added packages: 1
Dropped packages: 5
Upgraded packages: 16
Downgraded packages: 0
Size of added packages: 360.38 KiB
Size of dropped packages: 3.47 MiB
Size of upgraded packages: 223.10 MiB
Size of downgraded packages: 0 B
Size change of upgraded packages: 1.52 MiB
Size change of downgraded packages: 0 B
===== ADDED IMAGES =====
===== DROPPED IMAGES =====
Image: Cloud_Base raw-xz ppc64le
Path: Cloud/ppc64le/images/Fedora-Cloud-Base-Rawhide-20191201.n.0.ppc64le.raw.xz
Image: Cloud_Base qcow2 ppc64le
Path: Cloud/ppc64le/images/Fedora-Cloud-Base-Rawhide-20191201.n.0.ppc64le.qcow2
Image: Cloud_Base vmdk ppc64le
Path: Cloud/ppc64le/images/Fedora-Cloud-Base-Rawhide-20191201.n.0.ppc64le.vmdk
===== ADDED PACKAGES =====
Package: sqlitecpp-2.4.0-1.fc32
Summary: Smart and easy to use C++ SQLite3 wrapper
RPMs: sqlitecpp sqlitecpp-devel
Size: 360.38 KiB
===== DROPPED PACKAGES =====
Package: extra166y-1.7.0-12.fc31
Summary: Concurrency JSR-166 - Collections supporting parallel operations
RPMs: extra166y extra166y-javadoc
Size: 667.11 KiB
Package: felix-osgi-foundation-1.2.0-26.fc31
Summary: Felix OSGi Foundation EE Bundle
RPMs: felix-osgi-foundation felix-osgi-foundation-javadoc
Size: 703.69 KiB
Package: jcsp-1.1-0.12.rc5.fc31
Summary: Communicating Sequential Processes for Java (JCSP)
RPMs: jcsp jcsp-javadoc
Size: 1.60 MiB
Package: multiverse-0.7.0-11.fc31
Summary: A software transactional memory implementation for the JVM
RPMs: multiverse multiverse-javadoc
Size: 499.69 KiB
Package: plexus-cli-1.6-9.fc31
Summary: Command Line Interface facilitator for Plexus
RPMs: plexus-cli plexus-cli-javadoc
Size: 45.08 KiB
===== UPGRADED PACKAGES =====
Package: atk-2.35.1-1.fc32
Old package: atk-2.34.1-1.fc32
Summary: Interfaces for accessibility support
RPMs: atk atk-devel
Size: 2.58 MiB
Size change: 5.74 KiB
Changelog:
* Mon Dec 02 2019 Kalev Lember <klember(a)redhat.com> - 2.35.1-1
- Update to 2.35.1
Package: bijiben-3.35.1-1.fc32
Old package: bijiben-3.34.1-1.fc32
Summary: Simple Note Viewer
RPMs: bijiben
Size: 2.81 MiB
Size change: -6.22 KiB
Changelog:
* Mon Dec 02 2019 Kalev Lember <klember(a)redhat.com> - 3.35.1-1
- Update to 3.35.1
Package: bluez-5.52-2.fc32
Old package: bluez-5.52-1.fc32
Summary: Bluetooth utilities
RPMs: bluez bluez-cups bluez-hid2hci bluez-libs bluez-libs-devel bluez-mesh bluez-obexd
Size: 10.15 MiB
Size change: 410.21 KiB
Changelog:
* Mon Dec 02 2019 Lubomir Rintel <lkundrak(a)v3.sk> - 5.52-2
- Package the btvirt binary
Package: breezy-3.0.2-1.fc32
Old package: breezy-3.0.1-4.fc32
Summary: Friendly distributed version control system
RPMs: breezy breezy-doc
Size: 31.17 MiB
Size change: 3.28 KiB
Changelog:
* Sun Dec 01 2019 Miro Hron��ok <mhroncok(a)redhat.com> - 3.0.2-1
- Update to 3.0.2
Package: diffoscope-111-4.fc32
Old package: diffoscope-111-3.fc32
Summary: In-depth comparison of files, archives, and directories
RPMs: diffoscope
Size: 282.83 KiB
Size change: 64 B
Changelog:
* Thu Oct 03 2019 Miro Hron��ok <mhroncok(a)redhat.com> - 111-4
- Rebuilt for Python 3.8.0rc1 (#1748018)
Package: eog-3.35.1-1.fc32
Old package: eog-3.34.1-1.fc32
Summary: Eye of GNOME image viewer
RPMs: eog eog-devel eog-tests
Size: 19.19 MiB
Size change: 2.58 KiB
Changelog:
* Mon Dec 02 2019 Kalev Lember <klember(a)redhat.com> - 3.35.1-1
- Update to 3.35.1
Package: et-6.0.4-1.fc32
Old package: et-6.0.3-1.fc32
Summary: Remote shell that survives IP roaming and disconnect
RPMs: et
Size: 4.05 MiB
Size change: -99.31 KiB
Changelog:
* Sun Dec 01 2019 Michel Alexandre Salim <salimma(a)fedoraproject.org> - 6.0.4-1
- Update to 6.0.4
Package: ht-2.1.0-6.fc32
Old package: ht-2.1.0-2.fc31
Summary: File editor/viewer/analyzer for executables
RPMs: ht
Size: 3.35 MiB
Size change: -38.85 KiB
Changelog:
* Sun Dec 01 2019 Vitaly Zaitsev <vitaly(a)easycoding.org> - 2.1.0-6
- Revived package. Fixed FTBFS.
Package: nbdkit-1.17.1-1.fc32
Old package: nbdkit-1.16.0-6.fc32
Summary: NBD server
RPMs: nbdkit nbdkit-bash-completion nbdkit-basic-filters nbdkit-basic-plugins nbdkit-curl-plugin nbdkit-devel nbdkit-example-plugins nbdkit-ext2-plugin nbdkit-guestfs-plugin nbdkit-gzip-plugin nbdkit-iso-plugin nbdkit-libvirt-plugin nbdkit-linuxdisk-plugin nbdkit-lua-plugin nbdkit-nbd-plugin nbdkit-ocaml-plugin nbdkit-ocaml-plugin-devel nbdkit-perl-plugin nbdkit-python-plugin nbdkit-ruby-plugin nbdkit-server nbdkit-ssh-plugin nbdkit-tar-plugin nbdkit-tcl-plugin nbdkit-vddk-plugin nbdkit-xz-filter
Size: 5.28 MiB
Size change: -1.20 MiB
Changelog:
* Sun Dec 01 2019 Richard W.M. Jones <rjones(a)redhat.com> - 1.17.1-1
- New upstream development version 1.17.1.
- Add nbdkit-eval-plugin.
- Add nbdkit-ip-filter.
Package: php-PhpOption-1.6.0-1.fc32
Old package: php-PhpOption-1.5.0-7.fc31
Summary: Option type for PHP
RPMs: php-PhpOption
Size: 19.35 KiB
Size change: 378 B
Changelog:
* Sun Dec 01 2019 Shawn Iwinski <shawn.iwinski(a)gmail.com> - 1.6.0-1
- Update to 1.6.0 (RHBZ #1771062)
-
Package: php-guzzlehttp-guzzle-5.3.4-1.fc32
Old package: php-guzzlehttp-guzzle-5.3.3-2.fc30
Summary: PHP HTTP client and webservice framework
RPMs: php-guzzlehttp-guzzle
Size: 98.54 KiB
Size change: -6.59 KiB
Changelog:
* Fri Jul 26 2019 Fedora Release Engineering <releng(a)fedoraproject.org> - 5.3.3-3
- Rebuilt for https://fedoraproject.org/wiki/Fedora_31_Mass_Rebuild
* Sun Dec 01 2019 Shawn Iwinski <shawn.iwinski(a)gmail.com> - 5.3.4-1
- Update to 5.3.4 (RHBZ #1766925)
Package: python-pyopengl-3.1.4-1.fc32
Old package: python-pyopengl-3.1.1a1-21.fc32
Summary: Python bindings for OpenGL
RPMs: python3-pyopengl python3-pyopengl-tk
Size: 13.21 MiB
Size change: 1.92 MiB
Changelog:
* Sun Dec 01 2019 Scott Talbert <swt(a)techie.net> - 3.1.4-1
- Update to new upstream release 3.1.4
Package: python-tw2-forms-2.2.6-4.fc32
Old package: python-tw2-forms-2.2.6-3.fc32
Summary: Forms for ToscaWidgets2
RPMs: python3-tw2-forms
Size: 113.33 KiB
Size change: 60 B
Changelog:
* Thu Oct 03 2019 Miro Hron��ok <mhroncok(a)redhat.com> - 2.2.6-4
- Rebuilt for Python 3.8.0rc1 (#1748018)
Package: skopeo-1:0.1.41-11.dev.git3ed6e83.fc32
Old package: skopeo-1:0.1.41-6.dev.git912b7e1.fc32
Summary: Inspect container images and repositories on registries
RPMs: containers-common skopeo skopeo-tests
Size: 31.38 MiB
Size change: 1.25 MiB
Changelog:
* Tue Nov 26 2019 RH Container Bot <rhcontainerbot(a)fedoraproject.org> - 1:0.1.41-7.dev.gitce6ec77
- autobuilt ce6ec77
* Wed Nov 27 2019 RH Container Bot <rhcontainerbot(a)fedoraproject.org> - 1:0.1.41-8.dev.git2bfa895
- autobuilt 2bfa895
* Thu Nov 28 2019 RH Container Bot <rhcontainerbot(a)fedoraproject.org> - 1:0.1.41-9.dev.git73248bd
- autobuilt 73248bd
* Sat Nov 30 2019 RH Container Bot <rhcontainerbot(a)fedoraproject.org> - 1:0.1.41-10.dev.git3ed6e83
- autobuilt 3ed6e83
* Mon Dec 02 2019 Dan Walsh <dwalsh(a)fedoraproject.org> - 1:0.1.41-11.dev.git3ed6e83
- Update man pages to reflect upstream sources
- Also update storage.conf to remove skip_mount_home which is no longer
supported.
Package: tortoisehg-5.1-20191201.00a9f6fd23fe.fc32
Old package: tortoisehg-5.0.2-2.fc31
Summary: Mercurial GUI command line tool thg
RPMs: tortoisehg tortoisehg-nautilus
Size: 3.20 MiB
Size change: 16.18 KiB
Changelog:
* Sun Dec 01 2019 Mads Kiilerich <mads(a)kiilerich.com> - 5.1-20191201.00a9f6fd23fe
- early 5.2 snapshot 5.1+229-00a9f6fd23fe
- migrate to py3
Package: vigra-1.11.1-18.fc32
Old package: vigra-1.11.1-17.fc32
Summary: Generic Programming for Computer Vision
RPMs: python3-vigra vigra vigra-devel
Size: 96.23 MiB
Size change: -739.47 KiB
Changelog:
* Thu Oct 03 2019 Miro Hron��ok <mhroncok(a)redhat.com> - 1.11.1-18
- Rebuilt for Python 3.8.0rc1 (#1748018)
===== DOWNGRADED PACKAGES =====
4 years, 4 months
