10:21 a.m.
Hi all,
I noticed there is a Fedora minimization project which seems could be
a bit related to the thing I'm trying to do, and this could be a
generic topic.
What I'm trying to do is reduce the initramfs size used for kdump.
Kdump loads a crash kernel and kdump initramfs image in a prereseved
memory region, which get booted when current kernel crashed and
perform crash dump. The prereserved memory is limited, so initramfs
shouldn't go too big.
Kdump in Fedora use Dracut to create bootable initramfs, just hook the
final step to do kdump things instead of switch root. And to reduce
the size only the binaries and drivers required to boot and perform
kdump on current machine is installed. So long it have been working
very well.
But problem is Dracut works by reusing binaries and libraries from the
currently running system, and many userspace binaries and libraries is
keep growing and using more space. So the initramfs is also growing.
Some component, like Systemd, have grown by a lot, here is a list of
the size of part of binaries along with the binaries they required in
F31:
....
/root/image/bin/bash
4.8M .
/root/image/bin/echo
3.4M .
/root/image/bin/kmod
7.5M .
/root/image/bin/mkdir
4.5M .
/root/image/bin/mount
5.3M .
/root/image/bin/sh
4.8M .
/root/image/bin/systemctl
20M .
/root/image/usr/bin/systemctl
20M .
/root/image/usr/bin/systemd-cgls
20M .
/root/image/usr/bin/systemd-escape
20M .
/root/image/usr/bin/systemd-run
20M .
...
There are overlays between the libraries they used so when installed
into the initramfs, the total size didn't go too big yet. But we can
see the size of systemd binary and libraries it used is much bigger
than others.
And as a compare, from version 219 to 243, systemd's library
dependency increased a lot:
(v219 is 5M in total, v243 is 20M in total)
Version 243:
[root@localhost ~]# systemctl --version
systemd 243 (v243.4-1.fc31)
+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP
+LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS
+KMOD +IDN2 -IDN +PCRE2 default-hierarchy=unified
[root@localhost ~]# ldd /usr/bin/systemctl
linux-vdso.so.1 (0x00007ffe739a3000)
libc.so.6 => /lib64/libc.so.6 (0x00007fe2bbd63000)
libsystemd-shared-243.so =>
/usr/lib/systemd/libsystemd-shared-243.so (0x00007fe2bba8d000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fe2bba73000)
/lib64/ld-linux-x86-64.so.2 (0x00007fe2bbf73000)
librt.so.1 => /lib64/librt.so.1 (0x00007fe2bba68000)
libcap.so.2 => /lib64/libcap.so.2 (0x00007fe2bba61000)
libacl.so.1 => /lib64/libacl.so.1 (0x00007fe2bba56000)
libcryptsetup.so.12 => /lib64/libcryptsetup.so.12 (0x00007fe2bb9e3000)
libgcrypt.so.20 => /lib64/libgcrypt.so.20 (0x00007fe2bb8c5000)
libip4tc.so.2 => /lib64/libip4tc.so.2 (0x00007fe2bb8bb000)
libkmod.so.2 => /lib64/libkmod.so.2 (0x00007fe2bb8a1000)
libmount.so.1 => /lib64/libmount.so.1 (0x00007fe2bb83e000)
libseccomp.so.2 => /lib64/libseccomp.so.2 (0x00007fe2bb7f1000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007fe2bb7c2000)
libidn2.so.0 => /lib64/libidn2.so.0 (0x00007fe2bb7a0000)
liblzma.so.5 => /lib64/liblzma.so.5 (0x00007fe2bb777000)
liblz4.so.1 => /lib64/liblz4.so.1 (0x00007fe2bb756000)
libblkid.so.1 => /lib64/libblkid.so.1 (0x00007fe2bb6ff000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fe2bb6dd000)
libattr.so.1 => /lib64/libattr.so.1 (0x00007fe2bb6d3000)
libuuid.so.1 => /lib64/libuuid.so.1 (0x00007fe2bb6c9000)
libdevmapper.so.1.02 => /lib64/libdevmapper.so.1.02 (0x00007fe2bb66c000)
libssl.so.1.1 => /lib64/libssl.so.1.1 (0x00007fe2bb5d6000)
libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007fe2bb2f6000)
libargon2.so.1 => /lib64/libargon2.so.1 (0x00007fe2bb2ec000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fe2bb2e3000)
libjson-c.so.4 => /lib64/libjson-c.so.4 (0x00007fe2bb2d1000)
libgpg-error.so.0 => /lib64/libgpg-error.so.0 (0x00007fe2bb2ae000)
libpcap.so.1 => /lib64/libpcap.so.1 (0x00007fe2bb262000)
libz.so.1 => /lib64/libz.so.1 (0x00007fe2bb248000)
libpcre2-8.so.0 => /lib64/libpcre2-8.so.0 (0x00007fe2bb1ba000)
libunistring.so.2 => /lib64/libunistring.so.2 (0x00007fe2bb034000)
libudev.so.1 => /lib64/libudev.so.1 (0x00007fe2bb006000)
libm.so.6 => /lib64/libm.so.6 (0x00007fe2baec0000)
Version 219:
[root@localhost test]# systemctl --version
systemd 219
+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP
+LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS
+KMOD +IDN
[root@localhost test]# ldd /bin/systemctl
linux-vdso.so.1 => (0x00007fffc407f000)
librt.so.1 => /lib64/librt.so.1 (0x00007f70255bd000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f7025396000)
liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f7025170000)
liblz4.so.1 => /lib64/liblz4.so.1 (0x00007f7024f5b000)
libgcrypt.so.11 => /lib64/libgcrypt.so.11 (0x00007f7024cda000)
libcap.so.2 => /lib64/libcap.so.2 (0x00007f7024ad5000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f70248bf000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f70246a3000)
libc.so.6 => /lib64/libc.so.6 (0x00007f70242d5000)
/lib64/ld-linux-x86-64.so.2 (0x00007f7025a72000)
libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f7024073000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f7023e6f000)
libgpg-error.so.0 => /lib64/libgpg-error.so.0 (0x00007f7023c6a000)
libattr.so.1 => /lib64/libattr.so.1 (0x00007f7023a65000)
Kdump in Fedora relies heavily on Dracut, and Dracut relies on
systemd. But kdump only expect systemd to start some services. I think
there might be many features not needed by kdump being brought in.
Is there any way to have a smaller systemd binary that is just enough
to boot the initramfs into the stage before switch-root?
--
Best Regards,
Kairui Song
3:02 a.m.
On Thu, Jan 02, 2020 at 12:21:26AM +0800, Kairui Song wrote:
Some component, like Systemd, have grown by a lot, here is a list of
the size of part of binaries along with the binaries they required in
F31:
/root/image/bin/systemctl
20M .
/root/image/usr/bin/systemctl
20M .
/root/image/usr/bin/systemd-cgls
20M .
/root/image/usr/bin/systemd-escape
20M .
/root/image/usr/bin/systemd-run
20M .
...
There are overlays between the libraries they used so when installed
into the initramfs, the total size didn't go too big yet. But we can
see the size of systemd binary and libraries it used is much bigger
than others.
All systemd binaries will mostly link to the same libraries (because
they link to a private shared library, which links to various other
shared libraries). So this "20M" will be repeated over and over, but
it's the same dependencies.
While we'd all prefer for this to be smaller, 20M should is actually
not that much...
And as a compare, from version 219 to 243, systemd's library
dependency increased a lot:
(v219 is 5M in total, v243 is 20M in total)
This is slightly misleading. Code was moved from individual binaries
to libsystemd-shared-nnn.so, so if you look at the deps of just a single
binary, you'll see many more deps (because libsystemd-shared-nnn.so has
more deps). But the total number of deps when summed over all binaries
grew much less. A more useful measure would be the size with deps summed
over all systemd binaries that are installed into your image in v219 and
v243.
I don't have a link at hand, but there's work being done to use openssl
for all crypto, which would reduce the dependency list nicely.
Is there any way to have a smaller systemd binary that is just
enough
to boot the initramfs into the stage before switch-root?
We don't have anything like this now, sorry!
Zbyszek
9:45 p.m.
On 01/02/20 at 09:02am, Zbigniew Jędrzejewski-Szmek wrote:
On Thu, Jan 02, 2020 at 12:21:26AM +0800, Kairui Song wrote:
> Some component, like Systemd, have grown by a lot, here is a list of
> the size of part of binaries along with the binaries they required in
> F31:
> /root/image/bin/systemctl
> 20M .
> /root/image/usr/bin/systemctl
> 20M .
> /root/image/usr/bin/systemd-cgls
> 20M .
> /root/image/usr/bin/systemd-escape
> 20M .
> /root/image/usr/bin/systemd-run
> 20M .
> ...
>
> There are overlays between the libraries they used so when installed
> into the initramfs, the total size didn't go too big yet. But we can
> see the size of systemd binary and libraries it used is much bigger
> than others.
All systemd binaries will mostly link to the same libraries (because
they link to a private shared library, which links to various other
shared libraries). So this "20M" will be repeated over and over, but
it's the same dependencies.
While we'd all prefer for this to be smaller, 20M should is actually
not that much...
> And as a compare, from version 219 to 243, systemd's library
> dependency increased a lot:
> (v219 is 5M in total, v243 is 20M in total)
This is slightly misleading. Code was moved from individual binaries
to libsystemd-shared-nnn.so, so if you look at the deps of just a single
binary, you'll see many more deps (because libsystemd-shared-nnn.so has
more deps). But the total number of deps when summed over all binaries
grew much less. A more useful measure would be the size with deps summed
over all systemd binaries that are installed into your image in v219 and
v243.
I vaguely remember the size increased before due to linking with libidn2
previously, so those libraries contribute a lot.
Does every systemd binary depend on all libraries? Or each of the
systemd binary only depends on those libs when really needed?
Thanks
Dave
9:48 p.m.
On 01/03/20 at 11:45am, Dave Young wrote:
On 01/02/20 at 09:02am, Zbigniew Jędrzejewski-Szmek wrote:
> On Thu, Jan 02, 2020 at 12:21:26AM +0800, Kairui Song wrote:
> > Some component, like Systemd, have grown by a lot, here is a list of
> > the size of part of binaries along with the binaries they required in
> > F31:
> > /root/image/bin/systemctl
> > 20M .
> > /root/image/usr/bin/systemctl
> > 20M .
> > /root/image/usr/bin/systemd-cgls
> > 20M .
> > /root/image/usr/bin/systemd-escape
> > 20M .
> > /root/image/usr/bin/systemd-run
> > 20M .
> > ...
> >
> > There are overlays between the libraries they used so when installed
> > into the initramfs, the total size didn't go too big yet. But we can
> > see the size of systemd binary and libraries it used is much bigger
> > than others.
>
> All systemd binaries will mostly link to the same libraries (because
> they link to a private shared library, which links to various other
> shared libraries). So this "20M" will be repeated over and over, but
> it's the same dependencies.
>
> While we'd all prefer for this to be smaller, 20M should is actually
> not that much...
>
> > And as a compare, from version 219 to 243, systemd's library
> > dependency increased a lot:
> > (v219 is 5M in total, v243 is 20M in total)
>
> This is slightly misleading. Code was moved from individual binaries
> to libsystemd-shared-nnn.so, so if you look at the deps of just a single
> binary, you'll see many more deps (because libsystemd-shared-nnn.so has
> more deps). But the total number of deps when summed over all binaries
> grew much less. A more useful measure would be the size with deps summed
> over all systemd binaries that are installed into your image in v219 and
> v243.
>
I vaguely remember the size increased before due to linking with libidn2
previously, so those libraries contribute a lot.
Does every systemd binary depend on all libraries? Or each of the
systemd binary only depends on those libs when really needed?
For example if I do not need journalctl, then I can drop journalctl and
those libraries specific for journalctl?
Thanks
Dave
1:21 a.m.
On Fri, Jan 03, 2020 at 11:48:53AM +0800, Dave Young wrote:
On 01/03/20 at 11:45am, Dave Young wrote:
> On 01/02/20 at 09:02am, Zbigniew Jędrzejewski-Szmek wrote:
> > On Thu, Jan 02, 2020 at 12:21:26AM +0800, Kairui Song wrote:
> > > Some component, like Systemd, have grown by a lot, here is a list of
> > > the size of part of binaries along with the binaries they required in
> > > F31:
> > > /root/image/bin/systemctl
> > > 20M .
> > > /root/image/usr/bin/systemctl
> > > 20M .
> > > /root/image/usr/bin/systemd-cgls
> > > 20M .
> > > /root/image/usr/bin/systemd-escape
> > > 20M .
> > > /root/image/usr/bin/systemd-run
> > > 20M .
> > > ...
> > >
> > > There are overlays between the libraries they used so when installed
> > > into the initramfs, the total size didn't go too big yet. But we can
> > > see the size of systemd binary and libraries it used is much bigger
> > > than others.
> >
> > All systemd binaries will mostly link to the same libraries (because
> > they link to a private shared library, which links to various other
> > shared libraries). So this "20M" will be repeated over and over, but
> > it's the same dependencies.
> >
> > While we'd all prefer for this to be smaller, 20M should is actually
> > not that much...
> >
> > > And as a compare, from version 219 to 243, systemd's library
> > > dependency increased a lot:
> > > (v219 is 5M in total, v243 is 20M in total)
> >
> > This is slightly misleading. Code was moved from individual binaries
> > to libsystemd-shared-nnn.so, so if you look at the deps of just a single
> > binary, you'll see many more deps (because libsystemd-shared-nnn.so has
> > more deps). But the total number of deps when summed over all binaries
> > grew much less. A more useful measure would be the size with deps summed
> > over all systemd binaries that are installed into your image in v219 and
> > v243.
> >
>
> I vaguely remember the size increased before due to linking with libidn2
> previously, so those libraries contribute a lot.
>
> Does every systemd binary depend on all libraries? Or each of the
> systemd binary only depends on those libs when really needed?
For example if I do not need journalctl, then I can drop journalctl and
those libraries specific for journalctl?
It's using standard shared object linking, so yeah, for anything which
libsystemd-shared-nnn.so links to, "every systemd binary depend[s] on
all libraries", in the sense that the runtime linker will fail to start
the executable if any of the libraries are missing.
Zbyszek
11:12 p.m.
On Fri, Jan 3, 2020 at 3:23 PM Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl> wrote:
On Fri, Jan 03, 2020 at 11:48:53AM +0800, Dave Young wrote:
> On 01/03/20 at 11:45am, Dave Young wrote:
> > On 01/02/20 at 09:02am, Zbigniew Jędrzejewski-Szmek wrote:
> > > On Thu, Jan 02, 2020 at 12:21:26AM +0800, Kairui Song wrote:
> > > > Some component, like Systemd, have grown by a lot, here is a list of
> > > > the size of part of binaries along with the binaries they required
in
> > > > F31:
> > > > /root/image/bin/systemctl
> > > > 20M .
> > > > /root/image/usr/bin/systemctl
> > > > 20M .
> > > > /root/image/usr/bin/systemd-cgls
> > > > 20M .
> > > > /root/image/usr/bin/systemd-escape
> > > > 20M .
> > > > /root/image/usr/bin/systemd-run
> > > > 20M .
> > > > ...
> > > >
> > > > There are overlays between the libraries they used so when installed
> > > > into the initramfs, the total size didn't go too big yet. But we
can
> > > > see the size of systemd binary and libraries it used is much bigger
> > > > than others.
> > >
> > > All systemd binaries will mostly link to the same libraries (because
> > > they link to a private shared library, which links to various other
> > > shared libraries). So this "20M" will be repeated over and over,
but
> > > it's the same dependencies.
> > >
> > > While we'd all prefer for this to be smaller, 20M should is actually
> > > not that much...
> > >
> > > > And as a compare, from version 219 to 243, systemd's library
> > > > dependency increased a lot:
> > > > (v219 is 5M in total, v243 is 20M in total)
> > >
> > > This is slightly misleading. Code was moved from individual binaries
> > > to libsystemd-shared-nnn.so, so if you look at the deps of just a single
> > > binary, you'll see many more deps (because libsystemd-shared-nnn.so
has
> > > more deps). But the total number of deps when summed over all binaries
> > > grew much less. A more useful measure would be the size with deps summed
> > > over all systemd binaries that are installed into your image in v219 and
> > > v243.
> > >
> >
> > I vaguely remember the size increased before due to linking with libidn2
> > previously, so those libraries contribute a lot.
> >
> > Does every systemd binary depend on all libraries? Or each of the
> > systemd binary only depends on those libs when really needed?
>
> For example if I do not need journalctl, then I can drop journalctl and
> those libraries specific for journalctl?
It's using standard shared object linking, so yeah, for anything which
libsystemd-shared-nnn.so links to, "every systemd binary depend[s] on
all libraries", in the sense that the runtime linker will fail to start
the executable if any of the libraries are missing.
Hi, it has been a while since last discuss update, but a second
thought about the libsystemd-shared-nnn.so problem:
Now each systemd executable depends on libsystemd-shared-nnn.so, which
then depend on a lot of things. In older version (eg. version 219),
each individual systemd executable have it's own dependency, that make
thing much cleaner for special usages like kdump.
I'm not sure what is the purpose of this change, could there be any
work be done to minimize the lib dependency of each systemd binary?
--
Best Regards,
Kairui Song
1:07 a.m.
On Tue, Feb 25, 2020 at 01:12:08PM +0800, Kairui Song wrote:
On Fri, Jan 3, 2020 at 3:23 PM Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl> wrote:
> On Fri, Jan 03, 2020 at 11:48:53AM +0800, Dave Young wrote:
> > On 01/03/20 at 11:45am, Dave Young wrote:
> > > On 01/02/20 at 09:02am, Zbigniew Jędrzejewski-Szmek wrote:
> > > > On Thu, Jan 02, 2020 at 12:21:26AM +0800, Kairui Song wrote:
> > > > > Some component, like Systemd, have grown by a lot, here is a
list of
> > > > > the size of part of binaries along with the binaries they
required in
> > > > > F31:
> > > > > /root/image/bin/systemctl
> > > > > 20M .
> > > > > /root/image/usr/bin/systemctl
> > > > > 20M .
> > > > > /root/image/usr/bin/systemd-cgls
> > > > > 20M .
> > > > > /root/image/usr/bin/systemd-escape
> > > > > 20M .
> > > > > /root/image/usr/bin/systemd-run
> > > > > 20M .
> > > > > ...
> > > > >
> > > > > There are overlays between the libraries they used so when
installed
> > > > > into the initramfs, the total size didn't go too big yet.
But we can
> > > > > see the size of systemd binary and libraries it used is much
bigger
> > > > > than others.
> > > >
> > > > All systemd binaries will mostly link to the same libraries (because
> > > > they link to a private shared library, which links to various other
> > > > shared libraries). So this "20M" will be repeated over and
over, but
> > > > it's the same dependencies.
> > > >
> > > > While we'd all prefer for this to be smaller, 20M should is
actually
> > > > not that much...
> > > >
> > > > > And as a compare, from version 219 to 243, systemd's
library
> > > > > dependency increased a lot:
> > > > > (v219 is 5M in total, v243 is 20M in total)
> > > >
> > > > This is slightly misleading. Code was moved from individual binaries
> > > > to libsystemd-shared-nnn.so, so if you look at the deps of just a
single
> > > > binary, you'll see many more deps (because
libsystemd-shared-nnn.so has
> > > > more deps). But the total number of deps when summed over all
binaries
> > > > grew much less. A more useful measure would be the size with deps
summed
> > > > over all systemd binaries that are installed into your image in v219
and
> > > > v243.
> > > >
> > >
> > > I vaguely remember the size increased before due to linking with libidn2
> > > previously, so those libraries contribute a lot.
> > >
> > > Does every systemd binary depend on all libraries? Or each of the
> > > systemd binary only depends on those libs when really needed?
> >
> > For example if I do not need journalctl, then I can drop journalctl and
> > those libraries specific for journalctl?
>
> It's using standard shared object linking, so yeah, for anything which
> libsystemd-shared-nnn.so links to, "every systemd binary depend[s] on
> all libraries", in the sense that the runtime linker will fail to start
> the executable if any of the libraries are missing.
>
Hi, it has been a while since last discuss update, but a second
thought about the libsystemd-shared-nnn.so problem:
Now each systemd executable depends on libsystemd-shared-nnn.so, which
then depend on a lot of things. In older version (eg. version 219),
each individual systemd executable have it's own dependency, that make
thing much cleaner for special usages like kdump.
I'm not sure what is the purpose of this change, could there be any
work be done to minimize the lib dependency of each systemd binary?
libsystemd-shared-nnn.so holds code used in multiple executables. This
means that if the full suite is installed, shared code is present in
just one copy, and the total footprint of the installation is minimized
(disk, loading time, rss). OTOH, the footprint of installing just a
single executable is unfortunately increased. Our thinking was that
installing many executables is much more common (pretty even a minimal
system with systemd has at least ~30 systemd binaries), so it makes
sense to prioritize that.
See https://github.com/systemd/systemd/pull/3516 for the discussion
of the space savings back when this was originally done. Now we have
many more binaries (and even more shared code since integration of
various components is increasing...), so I expect the savings to
be even bigger.
Zbyszek
1:21 a.m.
On Tue, Feb 25, 2020 at 3:07 PM Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl> wrote:
On Tue, Feb 25, 2020 at 01:12:08PM +0800, Kairui Song wrote:
> On Fri, Jan 3, 2020 at 3:23 PM Zbigniew Jędrzejewski-Szmek
> <zbyszek(a)in.waw.pl> wrote:
> > On Fri, Jan 03, 2020 at 11:48:53AM +0800, Dave Young wrote:
> > > On 01/03/20 at 11:45am, Dave Young wrote:
> > > > On 01/02/20 at 09:02am, Zbigniew Jędrzejewski-Szmek wrote:
> > > > > On Thu, Jan 02, 2020 at 12:21:26AM +0800, Kairui Song wrote:
> > > > > > Some component, like Systemd, have grown by a lot, here is
a list of
> > > > > > the size of part of binaries along with the binaries they
required in
> > > > > > F31:
> > > > > > /root/image/bin/systemctl
> > > > > > 20M .
> > > > > > /root/image/usr/bin/systemctl
> > > > > > 20M .
> > > > > > /root/image/usr/bin/systemd-cgls
> > > > > > 20M .
> > > > > > /root/image/usr/bin/systemd-escape
> > > > > > 20M .
> > > > > > /root/image/usr/bin/systemd-run
> > > > > > 20M .
> > > > > > ...
> > > > > >
> > > > > > There are overlays between the libraries they used so when
installed
> > > > > > into the initramfs, the total size didn't go too big
yet. But we can
> > > > > > see the size of systemd binary and libraries it used is
much bigger
> > > > > > than others.
> > > > >
> > > > > All systemd binaries will mostly link to the same libraries
(because
> > > > > they link to a private shared library, which links to various
other
> > > > > shared libraries). So this "20M" will be repeated over
and over, but
> > > > > it's the same dependencies.
> > > > >
> > > > > While we'd all prefer for this to be smaller, 20M should is
actually
> > > > > not that much...
> > > > >
> > > > > > And as a compare, from version 219 to 243, systemd's
library
> > > > > > dependency increased a lot:
> > > > > > (v219 is 5M in total, v243 is 20M in total)
> > > > >
> > > > > This is slightly misleading. Code was moved from individual
binaries
> > > > > to libsystemd-shared-nnn.so, so if you look at the deps of just
a single
> > > > > binary, you'll see many more deps (because
libsystemd-shared-nnn.so has
> > > > > more deps). But the total number of deps when summed over all
binaries
> > > > > grew much less. A more useful measure would be the size with
deps summed
> > > > > over all systemd binaries that are installed into your image in
v219 and
> > > > > v243.
> > > > >
> > > >
> > > > I vaguely remember the size increased before due to linking with
libidn2
> > > > previously, so those libraries contribute a lot.
> > > >
> > > > Does every systemd binary depend on all libraries? Or each of the
> > > > systemd binary only depends on those libs when really needed?
> > >
> > > For example if I do not need journalctl, then I can drop journalctl and
> > > those libraries specific for journalctl?
> >
> > It's using standard shared object linking, so yeah, for anything which
> > libsystemd-shared-nnn.so links to, "every systemd binary depend[s] on
> > all libraries", in the sense that the runtime linker will fail to start
> > the executable if any of the libraries are missing.
> >
>
> Hi, it has been a while since last discuss update, but a second
> thought about the libsystemd-shared-nnn.so problem:
>
> Now each systemd executable depends on libsystemd-shared-nnn.so, which
> then depend on a lot of things. In older version (eg. version 219),
> each individual systemd executable have it's own dependency, that make
> thing much cleaner for special usages like kdump.
>
> I'm not sure what is the purpose of this change, could there be any
> work be done to minimize the lib dependency of each systemd binary?
libsystemd-shared-nnn.so holds code used in multiple executables. This
means that if the full suite is installed, shared code is present in
just one copy, and the total footprint of the installation is minimized
(disk, loading time, rss). OTOH, the footprint of installing just a
single executable is unfortunately increased. Our thinking was that
installing many executables is much more common (pretty even a minimal
system with systemd has at least ~30 systemd binaries), so it makes
sense to prioritize that.
Yes, that's very true, this kind of usage (single binary intallation)
is quite rare.
See https://github.com/systemd/systemd/pull/3516 for the discussion
of the space savings back when this was originally done. Now we have
many more binaries (and even more shared code since integration of
various components is increasing...), so I expect the savings to
be even bigger.
Zbyszek
Thanks for the info, the PR reference is very useful. Will check it
for more background knowledge.
--
Best Regards,
Kairui Song
11:34 p.m.
On Thu, Jan 2, 2020 at 5:04 PM Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl> wrote:
On Thu, Jan 02, 2020 at 12:21:26AM +0800, Kairui Song wrote:
> Some component, like Systemd, have grown by a lot, here is a list of
> the size of part of binaries along with the binaries they required in
> F31:
> /root/image/bin/systemctl
> 20M .
> /root/image/usr/bin/systemctl
> 20M .
> /root/image/usr/bin/systemd-cgls
> 20M .
> /root/image/usr/bin/systemd-escape
> 20M .
> /root/image/usr/bin/systemd-run
> 20M .
> ...
>
> There are overlays between the libraries they used so when installed
> into the initramfs, the total size didn't go too big yet. But we can
> see the size of systemd binary and libraries it used is much bigger
> than others.
All systemd binaries will mostly link to the same libraries (because
they link to a private shared library, which links to various other
shared libraries). So this "20M" will be repeated over and over, but
it's the same dependencies.
While we'd all prefer for this to be smaller, 20M should is actually
not that much...
> And as a compare, from version 219 to 243, systemd's library
> dependency increased a lot:
> (v219 is 5M in total, v243 is 20M in total)
This is slightly misleading. Code was moved from individual binaries
to libsystemd-shared-nnn.so, so if you look at the deps of just a single
binary, you'll see many more deps (because libsystemd-shared-nnn.so has
more deps). But the total number of deps when summed over all binaries
grew much less. A more useful measure would be the size with deps summed
over all systemd binaries that are installed into your image in v219 and
v243.
Yes, you are right.
I use following method to measure the size of the binary together with
the library it used:
Just call dracut-install to install a binary into an empty folder as
new root, that tool can also take care of resolve and installing the
libraries. The result it like this:
With systemd V243:
[root@localhost test]# for i in /usr/bin/systemd-*; do
/usr/lib/dracut/dracut-install -l -D $(pwd) $i; done
[root@localhost test]# for i in /lib/systemd/systemd-*; do
/usr/lib/dracut/dracut-install -l -D $(pwd) $i; done
[root@localhost test]# /usr/lib/dracut/dracut-install -l -D $(pwd)
/lib/systemd/systemd
[root@localhost test]# /usr/lib/dracut/dracut-install -l -D $(pwd)
/usr/bin/journalctl
[root@localhost test]# /usr/lib/dracut/dracut-install -l -D $(pwd)
/usr/bin/loginctl
[root@localhost test]# /usr/lib/dracut/dracut-install -l -D $(pwd)
/usr/bin/systemctl
[root@localhost test]# /usr/lib/dracut/dracut-install -l -D $(pwd)
/usr/bin/udevadm
[root@localhost test]# du -hs .
34M .
With V219:
[root@localhost test]# for i in /usr/bin/systemd-*; do
/usr/lib/dracut/dracut-install -l -D $(pwd) $i; done
[root@localhost test]# for i in /usr/lib/systemd/systemd-*; do
/usr/lib/dracut/dracut-install -l -D $(pwd) $i; done
[root@localhost test]# /usr/lib/dracut/dracut-install -l -D $(pwd)
/usr/lib/systemd/systemd
[root@localhost test]# /usr/lib/dracut/dracut-install -l -D $(pwd)
/usr/bin/journalctl
[root@localhost test]# /usr/lib/dracut/dracut-install -l -D $(pwd)
/usr/bin/loginctl
[root@localhost test]# /usr/lib/dracut/dracut-install -l -D $(pwd)
/usr/bin/systemctl
[root@localhost test]# /usr/lib/dracut/dracut-install -l -D $(pwd)
/usr/bin/udevadm
[root@localhost test]# du -hs .
33M .
So indeed it didn't grow in total.
But kdump's initramfs only need a subset of all binaries, so if I only
install the files needed:
With V243:
[root@localhost test]# for i in systemd systemd-coredump
systemd-cgroups-agent systemd-shutdown systemd-reply-password
systemd-fsck systemd-udevd systemd-journald systemd-sysctl
systemd-modules-load systemd-vconsole-setup; do
/usr/lib/dracut/dracut-install -l -D $(pwd) /usr/lib/systemd/$i; done
[root@localhost test]#
[root@localhost test]# for i in journalctl systemctl udevadm
systemd-run systemd-escape systemd-cgls systemd-tmpfiles; do
/usr/lib/dracut/dracut-install -l -D $(pwd) /usr/bin/$i; done
[root@localhost test]# du -hs .
24M .
With V219:
[root@localhost test]# for i in systemd systemd-coredump
systemd-cgroups-agent systemd-shutdown systemd-reply-password
systemd-fsck systemd-udevd systemd-journald systemd-sysctl
systemd-modules-load systemd-vconsole-setup; do
/usr/lib/dracut/dracut-install -l -D $(pwd) /usr/lib/systemd/$i; done
[root@localhost test]# for i in journalctl systemctl udevadm
systemd-run systemd-escape systemd-cgls systemd-tmpfiles; do
/usr/lib/dracut/dracut-install -l -D $(pwd) /usr/bin/$i; done
[root@localhost test]# du -hs .
12M .
That did grow a lot.
2:29 p.m.
Kairui Song <kasong(a)redhat.com> writes:
What I'm trying to do is reduce the initramfs size used for
kdump.
Kdump loads a crash kernel and kdump initramfs image in a prereseved
memory region, which get booted when current kernel crashed and
perform crash dump. The prereserved memory is limited, so initramfs
shouldn't go too big.
Kdump in Fedora use Dracut to create bootable initramfs, just hook the
final step to do kdump things instead of switch root. And to reduce
the size only the binaries and drivers required to boot and perform
kdump on current machine is installed. So long it have been working
very well.
But problem is Dracut works by reusing binaries and libraries from the
currently running system, and many userspace binaries and libraries is
keep growing and using more space. So the initramfs is also growing.
/root/image/bin/bash
4.8M .
/root/image/bin/sh
4.8M .
So it's not a direct comparison, but:
$ du -sh /bin/bash /bin/dash
1.2M /bin/bash
132K /bin/dash
This suggests to me that 1-3 MB could be reduced by using dash as the
shell. (dash's library dependencies are also smaller since it drops
requirements on libtinfo (200K) and libdl (36K); whether this matters I
don't know.)
Thanks,
--Robbie
4:59 p.m.
On Thu, Jan 02, 2020 at 03:29:26PM -0500, Robbie Harwood wrote:
Kairui Song <kasong(a)redhat.com> writes:
> What I'm trying to do is reduce the initramfs size used for kdump.
> Kdump loads a crash kernel and kdump initramfs image in a prereseved
> memory region, which get booted when current kernel crashed and
> perform crash dump. The prereserved memory is limited, so initramfs
> shouldn't go too big.
>
> Kdump in Fedora use Dracut to create bootable initramfs, just hook the
> final step to do kdump things instead of switch root. And to reduce
> the size only the binaries and drivers required to boot and perform
> kdump on current machine is installed. So long it have been working
> very well.
>
> But problem is Dracut works by reusing binaries and libraries from the
> currently running system, and many userspace binaries and libraries is
> keep growing and using more space. So the initramfs is also growing.
>
> /root/image/bin/bash
> 4.8M .
> /root/image/bin/sh
> 4.8M .
So it's not a direct comparison, but:
$ du -sh /bin/bash /bin/dash
1.2M /bin/bash
132K /bin/dash
This suggests to me that 1-3 MB could be reduced by using dash as the
shell. (dash's library dependencies are also smaller since it drops
requirements on libtinfo (200K) and libdl (36K); whether this matters I
don't know.)
dash doesn't support various bash extensions and syntaxes. The problem
is that many scripts which use #!/bin/sh really require #!/bin/bash.
So after switching to dash as the provider of /bin/sh various scripts
would suddenly behave differently, and those bugs would only be detected
at runtime.
Debian went through a long process of switching to dash as the default
init shell and fixing various scripts to remove bashisms so things
would run on dash (or any other /bin/sh). This was way more work than
anyone excepted and took a long time. IMO the gain of 1 MB that we
would get is not nearly enough to offset the work required and the
destabilization.
(In Debian the motivation was speed, rather than installation footprint.
So that work was mostly wasted because of the switch from sysvinit to systemd
and ensuing avoidance of shell during boot. Instead of trying to switch
shells, we should instead try to avoid shell in boot as much as possible.)
Zbyszek
8:03 p.m.
On Thu, 2020-01-02 at 22:59 +0000, Zbigniew Jędrzejewski-Szmek wrote:
(In Debian the motivation was speed, rather than installation
footprint.
So that work was mostly wasted because of the switch from sysvinit to systemd
and ensuing avoidance of shell during boot. Instead of trying to switch
shells, we should instead try to avoid shell in boot as much as possible.)
this may be tricky since dracut is just a giant ball of shell scripts
that no-one likes to touch...:P
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
1515
days inactive
1570
days old
11 comments
5 participants
participants (5)
-
Adam Williamson -
Dave Young -
Kairui Song -
Robbie Harwood -
Zbigniew Jędrzejewski-Szmek