Small rant: installer environment size
by Adam Williamson
Hi folks! Today I woke up and found
https://bugzilla.redhat.com/show_bug.cgi?id=2151495 , which diverted me
down a bit of an "installer environment size" rabbit hole.
As of today, with that new dep in webkitgtk, Rawhide's network install
images are 703M in size. Here's a potted history of network install
image sizes:
Fedora Core 8: 103.2M (boot.iso 9.2M + stage2.img 94M)
Fedora 13: 208M
Fedora 17: 162M (last "old UI")
Fedora 18: 294M (first "new UI")
Fedora 23: 415M
Fedora 28: 583M
Fedora 33: 686M
Fedora 37: 665M
Fedora Rawhide: 703M
The installer does not really do much more in Rawhide than it did in
FC8. Even after the UI rewrite in F18, we were only at 294M. Now the
image is well over 2x as big and does...basically the same.
Why does this matter? Well, the images being large is moderately
annoying in itself just in terms of transfer times and so on. But more
importantly, AIUI at least, the entire installer environment is loaded
into RAM at startup - it kinda has to be, we don't have anywhere else
to put it. The bigger it is, the more RAM you need to install Fedora.
The size of the installer environment (for which the size of the
network install image is more or less a perfect proxy) is one of the
two key factors in this, the other being how much RAM DNF uses during
package install.
So, I did a bit of poking about into *what* is taking up all that
space. There's a variety of answers, but there's two major culprits:
1. firmware
2. yelp (which pulls in webkitgtk and its deps)
I've been using du and baobab (the GNOME visual disk usage analyzer,
which is great) to examine the filesystems, but I ran a couple of test
builds to confirm these suspects, especially after the impact of
compression (it's hard to check the *compressed* size of things in the
installer environment directly).
I did a scratch build of lorax which does not pull in firmware
packages, and had openQA build a netinst using that lorax. It came out
at 489M - 214M smaller than current netinsts, a size we last managed in
Fedora 26. I did a scratch build of anaconda with its requirement of
yelp dropped (which would break help pages), and built a netinst with
that; it came out at 662M - 41M smaller than current images. I haven't
run a combined test yet, but it ought to come out around 448M, around
the size of Fedora 24.
Even then we'd still be about 50% larger than the Fedora 18 image, for
not really any added functionality.
I've moaned about the sheer amount and size of firmware blobs in other
forums before, but 214M compressed is *really* obnoxious. We must be
able to do something to clean this up (further than it's already
cleaned up - this is *after* we dropped low-hanging fruit like
enterprise switch 'firmwares' and garbage like that; most of the
remaining size seems to be huge amounts of probably-very-similar
firmware files for AMD graphics adapters and Intel wireless adapters).
I know some folks were trying to work on this (there was talk that we
could drop quite a lot of files that would only be loaded by older
kernels no longer in Fedora); any news on how far along that effort is?
Other obvious things that take up a lot of space:
1. /usr/lib/locale/locale-archive , from glibc-all-langpacks - this is
224M uncompressed. A quick test just compressing the file with xz on my
system shows it compresses to around 11M, though, so that's probably
all it adds up to after compression (the image is an xz-compressed
squashfs)
2. /usr/lib64/libLLVM-15.so, which is 114M on its own, compresses to
23M. We are, I think, basically stuck with this for mesa-dri-drivers ,
but does it have to be so *big*?
3. libicudata.so.71.1 - 30.4M, compresses to 7M. This is in the
webkitgtk dep chain but seems to still be pulled in without it, not
sure what else is requiring it.
4. /usr/share/locale - 112M in total (uncompressed, not sure how much
compressed) of translated strings from a ton of packages. No idea how
many of these are really *needed* in the installer environment. We can
maybe come up with a way to have lorax strip some, if we can come up
with a viable way to figure out which. Obviously-fairly-large ones are
from gnupg2 and libgweather4. I do recall we have some logic somewhere
to decide which languages have a certain level of translation in
anaconda; perhaps we could only include the strings for these
languages?
--
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net
1 month, 1 week
Starting Flatpak SIG
by Kalev Lember
Hi all,
I would like to kick of a Fedora Flatpak Packaging SIG that meets
regularly and has IRC meetings.
Our flatpaks have gotten some bad rep recently and I think for a good
reason. We don't have a lot of them and we only have a handful of people
working on them. We also have a fairly obvious conflict of interest with
Flathub and a bunch of Fedora people are doing flatpaks in Flathub
instead of Fedora and advocating against using Fedora flatpaks.
Here's an attempt to try to get more people involved :) The awkward
situation here is that I am not 100% convinced myself that we should
continue doing Fedora flatpaks, but maybe with a bit more organization
around this we can get some more eyes on the problem and solve some of
the issues.
Please put your name in https://fedoraproject.org/wiki/SIGs/Flatpak if
you are interested and I'll try to organize a poll for a weekly meeting
time. I'd suggest to start the meetings on the second week of January
when the holidays are over.
I've added a few people to CC that I thought might be interested in
this, but anyone that has interest in the area is more than welcome to join.
[Edit: the email got rejected from the mailing list due to too many
recipients so I'm re-sending it without the CC list. The CC'd people
hopefully got the message already.]
Some questions I have around organization: Can someone help register the
#fedora-flatpaks IRC channel under the Fedora umbrella? I am not sure
how to best do that. Does matrix need any special sauce to get a room?
What would be a good name for a pagure.io project?
pagure.io/fedora-flatpaks or pagure.io/fedora-flatpak-sig or
pagure.io/flatpak-sig or something else?
Do we need a mailing list? My initial gut feeling is no, but I'm
interested in what other people think.
If you reply to this, please make sure to reply to the
devel(a)lists.fedoraproject.org post as I am cross-posting it to desktop@
as well.
--
Thanks,
Kalev
2 months, 1 week
[Fedora 38] Call for Test Days
by Sumantro Mukherjee
Hi Fedora users, developers, and friends!
It's time to start thinking about Test Days for Fedora 38.
For anyone who isn't aware, a Test Day is an event usually focused
around IRC for interaction and a Wiki page for instructions and results,
with the aim being to get a bunch of interested users and developers
together to test a specific feature or area of the distribution. You can
run a Test Day on just about anything for which it would be useful to do
some fairly focused testing in 'real time' with a group of testers; it
doesn't have to be code, for instance, we often run Test Days for
l10n/i18n topics. For more information on Test Days, see
https://fedoraproject.org/wiki/QA/Test_Days .
Anyone who wants to can host their own Test Day, or you can request that
the QA group helps you out with organization or any combination of the
two. To propose a Test Day, just file a ticket in fedora-qa pagure - here's
an example https://pagure.io/fedora-qa/issue/624 . For
instructions on hosting a Test Day, see
https://fedoraproject.org/wiki/QA/SOP_Test_Day_management .
You can see the schedule at https://pagure.io/fedora-qa/issues?tags=test+days .
There are many slots open right now. Consider the development
schedule, though, in deciding when you want to run your Test Day - for
some topics you may want to avoid
the time before the Beta release or the time after the feature freeze
or the Final Freeze.
We normally aim to schedule Test Days on Thursdays; however, if you want
to run a series of related Test Days, it's often a good idea to do
something like Tuesday / Wednesday / Thursday of the same week (this is
how we usually run the X Test Week, for instance). If all the Thursday
slots fill up but more people want to run Test Days, we will open up
Tuesday slots as overflows. And finally, if you really want to run a
Test Day in a specific time frame due to the development schedule, but
the Thursday slot for that week is full, we can add a slot on another
day. We're flexible! Just put in your ticket the date or time frame you'd
like, and we'll figure it out from there.
If you don't want to run your own Test Day, but you are willing to
help with another, feel free to join one or more of already accepted
Test Days:
GNOME Test Day*
i18n Test Day*
Kernel Test Week(s)*
Upgrade Test Day*
IoT Test Week*
Cloud Test Day*
Fedora CoreOS Test Week*
And don't be afraid, there are a lot of more slots available for your
own Test Day!
[*] These are the test days we run generally to make sure everything
is working fine, the dates get announced as we move into the release
cycle.
If you have any questions about the Test Day process, please don't
hesitate to contact me or any member of the Fedora QA team on test at
lists.fedoraproject.org or in #fedora-qa on IRC. Thanks!
--
//sumantro
Fedora QE
TRIED AND PERSONALLY TESTED, ERGO TRUSTED
2 months, 3 weeks
Fedora Workstation WG minutes, 2022-12-13
by Chris Murphy
==============================================
#fedora-meeting-2: Workstation WG (2022-12-13)
==============================================
Meeting started by brainycmurf at 03:55:10 UTC. The full logs are
available at
https://meetbot.fedoraproject.org/fedora-meeting-2/2022-12-19/workstation...
.
Meeting summary
---------------
* Present members: Michael, Chris, Allan, Jens, Matthias, Tomas, Neal
(brainycmurf, 03:55:24)
* Guests: (brainycmurf, 03:55:25)
* Regrets: Kalev (brainycmurf, 03:55:25)
* Secretary: Jens (brainycmurf, 03:55:25)
* Drop flathub filtering (brainycmurf, 03:55:25)
* LINK: https://pagure.io/fedora-workstation/issue/300 (brainycmurf,
03:55:25)
* Improve the quality of the desktop wallpapers & ensure they're
maintained (brainycmurf, 03:55:31)
* LINK: https://pagure.io/fedora-workstation/issue/293 (brainycmurf,
03:55:33)
* LINK: https://fedoraproject.org/wiki/Wallpapers (brainycmurf,
03:55:37)
* LINK:
https://gitlab.com/fedora/design/team/release-artwork/default-wallpaper/-...
(brainycmurf, 03:55:41)
* ACTION: Allen to talk with Mairin (brainycmurf, 03:55:43)
* Ship preininstalled apps as Flatpaks (brainycmurf, 03:55:45)
* LINK: https://pagure.io/fedora-workstation/issue/269 (brainycmurf,
03:55:47)
* LINK: https://fedoraproject.org/wiki/SIGs/Flatpak (brainycmurf,
03:55:53)
* ACTION: Neal to note down his concerns about moving to default
flatpaks (brainycmurf, 03:55:59)
* Offline automated workstation installation not possible (brainycmurf,
03:56:01)
* LINK: https://pagure.io/fedora-workstation/issue/85 (brainycmurf,
03:56:03)
* ACTION: Michael will ping Felipe and the reporter (brainycmurf,
03:56:05)
* Announcements, Status Updates (brainycmurf, 03:56:11)
* next meeting 10th Jan (brainycmurf, 03:56:13)
* The minutes from last week have been posted. (brainycmurf,
03:56:15)
* LINK:
https://meetbot.fedoraproject.org/fedora-meeting-2/2022-12-07/workstation...
(brainycmurf, 03:56:17)
Meeting ended at 03:56:40 UTC.
Action Items
------------
* Allen to talk with Mairin
* Neal to note down his concerns about moving to default flatpaks
* Michael will ping Felipe and the reporter
Action Items, by person
-----------------------
* Michael
* Michael will ping Felipe and the reporter
* **UNASSIGNED**
* Allen to talk with Mairin
* Neal to note down his concerns about moving to default flatpaks
People Present (lines said)
---------------------------
* brainycmurf (35)
* zodbot (7)
* Michael (0)
Generated by `MeetBot`_ 0.4
.. _`MeetBot`: https://fedoraproject.org/wiki/Zodbot#Meeting_Functions
3 months
Re: Small rant: installer environment size
by Florian Weimer
* Richard W. M. Jones:
> You only need network / wifi firmware blobs (although I'm sure they
> are in themselves large) and then you can fetch anything else needed
> for the hardware including graphics, right?
I think you need graphics to set up wifi.
Thanks,
Florian
3 months, 1 week
Re: Small rant: installer environment size
by drago01
On Thursday, December 8, 2022, Daniel P. Berrangé <berrange(a)redhat.com>
wrote:
> On Thu, Dec 08, 2022 at 07:59:20PM +0100, drago01 wrote:
> > On Thursday, December 8, 2022, Adam Williamson <
> adamwill(a)fedoraproject.org>
> > wrote:
> >
> > > On Thu, 2022-12-08 at 12:58 +0000, Peter Robinson wrote:
> > > >
> > > > I've done a few passes, dropping a bunch of older firmware upstream
> > > > that are no longer supported in any stable kernel release, also a
> > > > bunch of de-dupe and linking of files rather than shipping of
> multiple
> > > > copies of the same firmware. It's improved things a bit,
> unfortunately
> > > > a lot of the dead firmware was tiny compared to say average modern
> > > > devices like GPUs or WiFI.
> > > >
> > > > The problem with a lot of the firmware, and with the new nvidia "open
> > > > driver" which shoves a lot of stuff into firmware in order to have an
> > > > upstreamable driver apparently the firmwares there are going to be
> > > > 30+Mb each, is that they're needed to bring up graphics/network etc
> to
> > > > even just install so I don't know how we can get around this and
> still
> > > > have a device work enough to be able to install the needed firmware
> > > > across the network.
> > > >
> > > > Ideas on how to solve that problem welcome.
> > >
> > > Sorry if this is way off, but - do we need the GPU firmwares to run a
> > > graphical install on the fallback path, just using the framebuffer set
> > > up by the firmware? How crazy would it be to just do that - ship the
> > > installer env with no GPU firmware?
> >
> >
> > That would be very crazy, as you will have a degraded user experience
> > (laggy UI, wrong resolution, ...) to save a couple of megabytes that are
> a
> > non issue for today's hardware.
>
> Please bear in mind the difference between bare metal and virtual
> machines. The bare metal machine may have 32 GB of RAM, making a
> 800 MB install image a non-issue. For a public cloud virtual machine
> though, this could bump your VM sizing up 1 level from 2 GB quota
> to a 4 GB RAM quota, with correspondingly higher price point. Now
> most people probably don't run the installer in a public cloud,
> preferring pre-built disk images. Even in a local machine though,
> you may be using most of your 32 GB of RAM for other things (well
> firefox/chrome), so allowing extra for the VM is not without
> resource cost. If we could figure out a way to knock a few 100 MB
> off the installer RAM requirements that is valuable.
>
>
The problem I see here is not the presence of the firmware on the image,
but the fact that it seems to be loaded into memory despite not being used.
> With regards,
> Daniel
> --
> |: https://berrange.com -o- https://www.flickr.com/photos/
> dberrange :|
> |: https://libvirt.org -o-
> https://fstop138.berrange.com :|
> |: https://entangle-photo.org -o- https://www.instagram.com/
> dberrange :|
> _______________________________________________
> kernel mailing list -- kernel(a)lists.fedoraproject.org
> To unsubscribe send an email to kernel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://docs.fedoraproject.
> org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/kernel@
> lists.fedoraproject.org
> Do not reply to spam, report it: https://pagure.io/fedora-
> infrastructure/new_issue
>
3 months, 1 week
Re: Small rant: installer environment size
by Adam Williamson
On Thu, 2022-12-08 at 12:50 -0500, David Cantrell wrote:
> On Wed, Dec 07, 2022 at 04:42:05PM -0800, Adam Williamson wrote:
> > Hi folks! Today I woke up and found
> > https://bugzilla.redhat.com/show_bug.cgi?id=2151495 , which diverted me
> > down a bit of an "installer environment size" rabbit hole.
> >
> > As of today, with that new dep in webkitgtk, Rawhide's network install
> > images are 703M in size. Here's a potted history of network install
> > image sizes:
> >
> > Fedora Core 8: 103.2M (boot.iso 9.2M + stage2.img 94M)
> > Fedora 13: 208M
> > Fedora 17: 162M (last "old UI")
> > Fedora 18: 294M (first "new UI")
> > Fedora 23: 415M
> > Fedora 28: 583M
> > Fedora 33: 686M
> > Fedora 37: 665M
> > Fedora Rawhide: 703M
> >
> > The installer does not really do much more in Rawhide than it did in
> > FC8. Even after the UI rewrite in F18, we were only at 294M. Now the
> > image is well over 2x as big and does...basically the same.
>
> I take issue with this. It is not accurate to say that the installer now does
> not do much more than it did for Fedora Core 8. There is more to the
> installer than the UI.
>
> Broadly speaking, a lot of the growth came from converging the runtime
> environment for the installer with the installed system. In Fedora Core 8 and
> previous releases, the "installer environment" was a unique and stripped down
> install. This was frustrating because it was effectively maintaining a small
> mini distro for the purposes of running the distro installer.
I meant it doesn't do much more in terms of what it achieves for the
user. But we can also just take F18 as the base point, if you like.
We're still over 2x as big as that was.
> I think some curation on firmware could happen. I think probinson@ mentions
> it later in this thread. If there were a way to identify the firmware
> necessary for the installer environment that could probably simplify things.
We already do a lot of "identifying the firmware necessary for the
installer environment", see my earlier mail about all the stuff lorax
does here. We've done the easy part, unfortunately. The stuff that's
left is stuff that is, in some sense, needed - graphics card and
wireless adapter firmwares, mainly.
>
> > Other obvious things that take up a lot of space:
> >
> > 1. /usr/lib/locale/locale-archive , from glibc-all-langpacks - this is
> > 224M uncompressed. A quick test just compressing the file with xz on my
> > system shows it compresses to around 11M, though, so that's probably
> > all it adds up to after compression (the image is an xz-compressed
> > squashfs)
>
> Can this be installed compressed? I'm not sure it can.
The "compressed" size is the effective size we're concerned about,
because the installer filesystem image is an xz-compressed squashfs. So
11M is already the "effective weight" of this file, I think - if I ran
an image build with it removed, it'd probably be ~11M smaller.
>
> > 2. /usr/lib64/libLLVM-15.so, which is 114M on its own, compresses to
> > 23M. We are, I think, basically stuck with this for mesa-dri-drivers ,
> > but does it have to be so *big*?
>
> If mesa-dri-drivers is not required for installation, it could be removed from
> the installer environment.
It is required - we don't get any graphics without it.
> > 4. /usr/share/locale - 112M in total (uncompressed, not sure how much
> > compressed) of translated strings from a ton of packages. No idea how
> > many of these are really *needed* in the installer environment. We can
> > maybe come up with a way to have lorax strip some, if we can come up
> > with a viable way to figure out which. Obviously-fairly-large ones are
> > from gnupg2 and libgweather4. I do recall we have some logic somewhere
> > to decide which languages have a certain level of translation in
> > anaconda; perhaps we could only include the strings for these
> > languages?
>
> On that note, /usr/share/doc, /usr/share/man, and /usr/share/info could be
> removed from the installer image if they are present. That likely won't free
> a whole lot of space, but it's not nothing.
All of those are already stripped:
https://github.com/weldr/lorax/blob/master/share/templates.d/99-generic/r...
--
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net
3 months, 2 weeks
Re: Small rant: installer environment size
by Adam Williamson
On Thu, 2022-12-08 at 08:26 -0500, Stephen Smoogen wrote:
> >
> The only ideas I have seen which 'work'* is to ship a minimal set of
> drivers for some 'chosen' hardware and then you have a bloated kitchen-sink
> iso which has all the drivers in it. The chosen hardware could be a
> 'defined' virtual environment which needs a minimal set of firmware,
> languages, etc. Everything else can be done in the install for getting
> languages, extra firmware, etc. The kitchen-sink.iso is going to be one
> which we know is going to be large.
>
> Now I have doubled the QA, releng, and product work.. I would say we would
> focus most of the work on the mini-installer, but we all know that all the
> work will be in the kitchen-sink one.
Well, if the two images are just "with firmware" and "without firmware"
I suppose in a way the testing load shouldn't be too awful, because we
can be pretty confident of the possible behaviours - nothing except the
kernel should do anything with kernel firmware files, after all, so
you're kinda limited to "it works fine", "some hardware doesn't work
because the firmware isn't there", and the occasional "kernel blew up
because firmware was missing which it really shouldn't do", which is
bad but at least easy to spot and workaround ("you gotta boot with the
firmware, sorry"). We could probably rely mostly on automated testing
to confirm that both images at least work properly in expected cases.
Also it's general not too onerous for us to test "some random alternate
path through the installer" because we have to run several install
tests on bare metal anyway so we can just kinda add it to the 'matrix'
there - "OK, for the firmware RAID install test I'll try booting the
no-firmware route", that knocks out two tests in one. (Of course you're
assuming that firmware RAID handling isn't somehow broken when booting
with the firmware, but that seems sufficiently unlikely not to worry
about :>)
If we want to get fancy, I suppose we could ship a single ISO, but with
two filesystem images - one the main installer environment, one
containing /lib/firmware - and just have a boot arg that (tells dracut
to) mounts the firmware one, and a boot menu option to *not* do it (not
pass the boot arg), which saves memory as long as the system doesn't
need the firmwares. But then of course someone will say "hey, why don't
we build a second ISO without the firmware image included at all, so we
have a small ISO for people who know they don't need it?" and you're
back at option 1. We also I guess have to think about how things work
for things like PXE installs, and maybe update the documentation
there...
--
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net
3 months, 2 weeks
