1:48 a.m.
Heads-up: I've taken ownership over the Cloud SIG's planned
"Cloud-Friendly Kernel Packaging" (aka "Modular Kernel Packaging for
Cloud") Change [0].
(Sorry for cross-posting this on the kernel and cloud lists, but both
the kernel team and the cloud SIG need to be involved in this
discussion).
Obviously, we wish for modular kernel packaging and Josh Boyer already
reached out to the cloud SIG in this regard a while ago when we didn't
really know what we want yet [1]. We do now know that somewhat better
[2] but details could still require additional discussion. Our
motivation is to save space, time and bandwidth (the latter two
referring to required updates) as in the cloud, all three equal to
costs. And yes, we also target things other than the kernel to save
space.
So, in our case hardware drivers are rather unnecessary and the kernel
experts might know other ways to shrink the footprint for our limited
use cases. The kernel we require supports all primary architectures
(i686 and x86_64 right now, ARM likely later) and is able to run on a
hypervisor (primarily KVM and Xen but support for ESXi and Hyper-V are
becoming crucial in private clouds). Only. No bare-metal. We also
require support for Linux Containers (LXC) to enable the use of
Docker.
Now, I heard some people screaming when I said HW drivers are
unnecessary. Some people will want to make use of PCI passthrough and
while I think we don't want to ship the necessary modules by default,
they should be easily installable through a separate package. If some
more granularity is acceptable (and makes sense), I think one package
for SRIOV NICs, one for graphic cards (to enable mathematical
computation stuff) and one for everything else PCI passthrough (plus
one for all other HW drivers for the non-cloud products) would be
totally nice.
What does everybody think about this? Can it be done? How is it best
done? What's the timeframe (we'd really like to see this implemented
in F21 Beta but obviously the earlier modular kernels can be tested,
the better)? Do you require additional input?
Please try to keep the discussion as simple as possible (in terms of
us non-kernel hackers should be able to follow it). Just like Josh
requested when reaching out to us: "Explain it to me like I'm a
child."
Thanks,
Sandro
[0]
https://fedoraproject.org/wiki/Cloud_Changelist#Change:_Cloud-Friendly_Ke...
[1] https://lists.fedoraproject.org/pipermail/cloud/2013-October/002862.html
[2] https://fedoraproject.org/wiki/Cloud_PRD
3:58 a.m.
On Wed, Mar 5, 2014 at 8:48 AM, Sandro "red" Mathys
<red(a)fedoraproject.org> wrote:
Heads-up: I've taken ownership over the Cloud SIG's planned
"Cloud-Friendly Kernel Packaging" (aka "Modular Kernel Packaging for
Cloud") Change [0].
(Sorry for cross-posting this on the kernel and cloud lists, but both
the kernel team and the cloud SIG need to be involved in this
discussion).
Obviously, we wish for modular kernel packaging and Josh Boyer already
reached out to the cloud SIG in this regard a while ago when we didn't
really know what we want yet [1]. We do now know that somewhat better
[2] but details could still require additional discussion. Our
motivation is to save space, time and bandwidth (the latter two
referring to required updates) as in the cloud, all three equal to
costs. And yes, we also target things other than the kernel to save
space.
So, in our case hardware drivers are rather unnecessary and the kernel
experts might know other ways to shrink the footprint for our limited
use cases. The kernel we require supports all primary architectures
(i686 and x86_64 right now, ARM likely later) and is able to run on a
hypervisor (primarily KVM and Xen but support for ESXi and Hyper-V are
becoming crucial in private clouds). Only. No bare-metal. We also
require support for Linux Containers (LXC) to enable the use of
Docker.
Now, I heard some people screaming when I said HW drivers are
unnecessary. Some people will want to make use of PCI passthrough and
while I think we don't want to ship the necessary modules by default,
they should be easily installable through a separate package. If some
more granularity is acceptable (and makes sense), I think one package
for SRIOV NICs, one for graphic cards (to enable mathematical
computation stuff) and one for everything else PCI passthrough (plus
one for all other HW drivers for the non-cloud products) would be
totally nice.
What does everybody think about this? Can it be done? How is it best
done? What's the timeframe (we'd really like to see this implemented
in F21 Beta but obviously the earlier modular kernels can be tested,
the better)? Do you require additional input?
Please try to keep the discussion as simple as possible (in terms of
us non-kernel hackers should be able to follow it). Just like Josh
requested when reaching out to us: "Explain it to me like I'm a
child."
Some drivers are built inside the kernel (for various reasons) so if
we keep them in they can't really be modular.
If we are going down this route (ex. kernel-hw-drivers sub package) we
should have a way to handle upgrades.
You don't want to upgrade and be left out without any hardware support
because it is now a sub package that nothing pulls in.
How much are you trying to save by this? All modules (excluding extra
which is already a sub package) are ~115MB here on x86_64.
5:25 a.m.
On Wed, Mar 5, 2014 at 6:58 PM, drago01 <drago01(a)gmail.com> wrote:
On Wed, Mar 5, 2014 at 8:48 AM, Sandro "red" Mathys
<red(a)fedoraproject.org> wrote:
> Heads-up: I've taken ownership over the Cloud SIG's planned
> "Cloud-Friendly Kernel Packaging" (aka "Modular Kernel Packaging for
> Cloud") Change [0].
>
> (Sorry for cross-posting this on the kernel and cloud lists, but both
> the kernel team and the cloud SIG need to be involved in this
> discussion).
>
> Obviously, we wish for modular kernel packaging and Josh Boyer already
> reached out to the cloud SIG in this regard a while ago when we didn't
> really know what we want yet [1]. We do now know that somewhat better
> [2] but details could still require additional discussion. Our
> motivation is to save space, time and bandwidth (the latter two
> referring to required updates) as in the cloud, all three equal to
> costs. And yes, we also target things other than the kernel to save
> space.
>
> So, in our case hardware drivers are rather unnecessary and the kernel
> experts might know other ways to shrink the footprint for our limited
> use cases. The kernel we require supports all primary architectures
> (i686 and x86_64 right now, ARM likely later) and is able to run on a
> hypervisor (primarily KVM and Xen but support for ESXi and Hyper-V are
> becoming crucial in private clouds). Only. No bare-metal. We also
> require support for Linux Containers (LXC) to enable the use of
> Docker.
>
> Now, I heard some people screaming when I said HW drivers are
> unnecessary. Some people will want to make use of PCI passthrough and
> while I think we don't want to ship the necessary modules by default,
> they should be easily installable through a separate package. If some
> more granularity is acceptable (and makes sense), I think one package
> for SRIOV NICs, one for graphic cards (to enable mathematical
> computation stuff) and one for everything else PCI passthrough (plus
> one for all other HW drivers for the non-cloud products) would be
> totally nice.
>
> What does everybody think about this? Can it be done? How is it best
> done? What's the timeframe (we'd really like to see this implemented
> in F21 Beta but obviously the earlier modular kernels can be tested,
> the better)? Do you require additional input?
>
> Please try to keep the discussion as simple as possible (in terms of
> us non-kernel hackers should be able to follow it). Just like Josh
> requested when reaching out to us: "Explain it to me like I'm a
> child."
Some drivers are built inside the kernel (for various reasons) so if
we keep them in they can't really be modular.
If there are reasons they are built-in (and I think currently nothing
is being built-in without good reasons), that should not be changed.
Only just talking about what is currently a module (or otherwise a
separate file - if applicable). The vmlinuz should be kept as-is and
stay the same for all products (server, workstation and cloud).
If we are going down this route (ex. kernel-hw-drivers sub package)
we
should have a way to handle upgrades.
You don't want to upgrade and be left out without any hardware support
because it is now a sub package that nothing pulls in.
Yes, that's only too true. But it should be possible to guarantee
proper upgrades through means of packaging. And of course this will be
tested.
How much are you trying to save by this? All modules (excluding
extra
which is already a sub package) are ~115MB here on x86_64.
As much as is sensibly possible. Every MB counts but we don't want to
drive the kernel team members insane. :)
-- Sandro
8:54 a.m.
On Wed, Mar 05, 2014 at 08:25:12PM +0900, Sandro "red" Mathys wrote:
> Some drivers are built inside the kernel (for various reasons)
so if
> we keep them in they can't really be modular.
If there are reasons they are built-in (and I think currently nothing
is being built-in without good reasons), that should not be changed.
Only just talking about what is currently a module (or otherwise a
separate file - if applicable). The vmlinuz should be kept as-is and
stay the same for all products (server, workstation and cloud).
> If we are going down this route (ex. kernel-hw-drivers sub package) we
> should have a way to handle upgrades.
> You don't want to upgrade and be left out without any hardware support
> because it is now a sub package that nothing pulls in.
Yes, that's only too true. But it should be possible to guarantee
proper upgrades through means of packaging. And of course this will be
tested.
> How much are you trying to save by this? All modules (excluding extra
> which is already a sub package) are ~115MB here on x86_64.
As much as is sensibly possible. Every MB counts but we don't want to
drive the kernel team members insane. :)
I disagree with that approach. 'Every MB counts' is a difficult concept
to implement and forces folks (like the kernel) to implement radical
changes to achieve some 'pie in the sky' requirement that you are never
really sure you have achieved.
How about something more sensible, hard limits. Go through your package
requirements and allocate a chunk of space to everyone and see how that
works (well not to each userspace package individually but collectively).
If that is too big trim some places.
For example, lets start with 100MB package requirement for the kernel (and
say 2 GB for userspace). This way the kernel team can implement
reasonable changes and monitor proper usage (because things grow over
time).
If later on you realize 100 MB is not competitive enough, come back and
chop it down to say 50 MB and let the kernel team figure it out.
But please do not come in here with a 'every MB counts' approach. It is
not very sustainable for future growth nor really easy to implement from
an engineering approach.
Is that acceptable? The kernel team can start with a hard limit of 100MB
package requirement (or something reasonably less)? Let's work off budget
requirements please.
Cheers,
Don
9:02 a.m.
On Wed, Mar 5, 2014 at 9:54 AM, Don Zickus <dzickus(a)redhat.com> wrote:
On Wed, Mar 05, 2014 at 08:25:12PM +0900, Sandro "red"
Mathys wrote:
For example, lets start with 100MB package requirement for the kernel (and
say 2 GB for userspace). This way the kernel team can implement
reasonable changes and monitor proper usage (because things grow over
time).
If later on you realize 100 MB is not competitive enough, come back and
chop it down to say 50 MB and let the kernel team figure it out.
But please do not come in here with a 'every MB counts' approach. It is
not very sustainable for future growth nor really easy to implement from
an engineering approach.
Is that acceptable? The kernel team can start with a hard limit of 100MB
package requirement (or something reasonably less)? Let's work off budget
requirements please.
This is a fair point. To be honest, I've ignored the "every MB
counts" aspect entirely for now. I've instead been focusing on
required functionality, because that's likely going to be the main
driver of what the resulting size will be.
FWIW, the existing kernel package installed today (a debug kernel
even) is ~142 MB. 123MB of that is the /lib/modules content. ~6MB of
that is vmlinuz. The remaining 13MB is the initramfs, which is
actually something that composes on the system during install and not
something we can shrink from a packaging standpoint.
josh
9:12 a.m.
New subject: Modular Kernel Packaging for Cloud -> glibc-common
Am 05.03.2014 16:02, schrieb Josh Boyer:
FWIW, the existing kernel package installed today (a debug kernel
even) is ~142 MB. 123MB of that is the /lib/modules content. ~6MB of
that is vmlinuz. The remaining 13MB is the initramfs, which is
actually something that composes on the system during install and not
something we can shrink from a packaging standpoint
honestly "glibc-common" would be more useful and less critical
to split into subpackages
* missing a locale and fallback to english leaves the option
install whatever package - i would not miss anything than
"de" and "en" on all machines i maintain - cloud or not
* missing a kernel module and refuse to boot is critical
because of that CC to @devel
________________________________________________________
114,08 MB glibc-common
132,60 MB kernel
[harry@rh]$ rpm -q --filesbypkg glibc-common | grep locale | wc -l
419
[harry@rh]$ rpm -q --filesbypkg glibc-common | grep -v locale | wc -l
257
9:13 a.m.
On Wed, Mar 05, 2014 at 10:02:17AM -0500, Josh Boyer wrote:
On Wed, Mar 5, 2014 at 9:54 AM, Don Zickus <dzickus(a)redhat.com>
wrote:
> On Wed, Mar 05, 2014 at 08:25:12PM +0900, Sandro "red" Mathys wrote:
> For example, lets start with 100MB package requirement for the kernel (and
> say 2 GB for userspace). This way the kernel team can implement
> reasonable changes and monitor proper usage (because things grow over
> time).
>
> If later on you realize 100 MB is not competitive enough, come back and
> chop it down to say 50 MB and let the kernel team figure it out.
>
> But please do not come in here with a 'every MB counts' approach. It is
> not very sustainable for future growth nor really easy to implement from
> an engineering approach.
>
> Is that acceptable? The kernel team can start with a hard limit of 100MB
> package requirement (or something reasonably less)? Let's work off budget
> requirements please.
This is a fair point. To be honest, I've ignored the "every MB
counts" aspect entirely for now. I've instead been focusing on
required functionality, because that's likely going to be the main
driver of what the resulting size will be.
Of course. :-)
FWIW, the existing kernel package installed today (a debug kernel
even) is ~142 MB. 123MB of that is the /lib/modules content. ~6MB of
that is vmlinuz. The remaining 13MB is the initramfs, which is
actually something that composes on the system during install and not
something we can shrink from a packaging standpoint.
It also helps with monitoring. 3-4 years from now after all the chopping,
these pacakges bloat right back up and everyone forgets why we chopped in
the first place. Hard requirements help keep everything in check and
forces people to request more space which the cloud team can evaluate
properly and still control their enviroment.
Anyway, this was just my rant for the day. I saw the initial email and
immediately disagreed with the approach. Spent a few minutes thinking of
a better way and thought, well I wasted a few minutes thinking about it
might as well respond in email. :-)
Cheers,
Don
5:16 p.m.
On Thu, Mar 6, 2014 at 12:13 AM, Don Zickus <dzickus(a)redhat.com> wrote:
On Wed, Mar 05, 2014 at 10:02:17AM -0500, Josh Boyer wrote:
> On Wed, Mar 5, 2014 at 9:54 AM, Don Zickus <dzickus(a)redhat.com> wrote:
> > On Wed, Mar 05, 2014 at 08:25:12PM +0900, Sandro "red" Mathys wrote:
> > For example, lets start with 100MB package requirement for the kernel (and
> > say 2 GB for userspace). This way the kernel team can implement
> > reasonable changes and monitor proper usage (because things grow over
> > time).
> >
> > If later on you realize 100 MB is not competitive enough, come back and
> > chop it down to say 50 MB and let the kernel team figure it out.
> >
> > But please do not come in here with a 'every MB counts' approach. It
is
> > not very sustainable for future growth nor really easy to implement from
> > an engineering approach.
> >
> > Is that acceptable? The kernel team can start with a hard limit of 100MB
> > package requirement (or something reasonably less)? Let's work off budget
> > requirements please.
>
> This is a fair point. To be honest, I've ignored the "every MB
> counts" aspect entirely for now. I've instead been focusing on
> required functionality, because that's likely going to be the main
> driver of what the resulting size will be.
That's the point, we want a reasonably small package while still
providing the required functionality. Not sure how providing a fixed
size number is helping in this. But most of all, I didn't throw in a
number because I have no idea what is reasonably possible. I really
only just said "every MB counts" because the question came up before
(in Josh's old thread) and I hoped I could stop this discussion from
happening again before we have any numbers for this.
Of course. :-)
>
> FWIW, the existing kernel package installed today (a debug kernel
> even) is ~142 MB. 123MB of that is the /lib/modules content. ~6MB of
> that is vmlinuz. The remaining 13MB is the initramfs, which is
> actually something that composes on the system during install and not
> something we can shrink from a packaging standpoint.
It also helps with monitoring. 3-4 years from now after all the chopping,
these pacakges bloat right back up and everyone forgets why we chopped in
the first place. Hard requirements help keep everything in check and
forces people to request more space which the cloud team can evaluate
properly and still control their enviroment.
Well, if we can remember why we put up a fixed size requirement, why
can't we remember why we did the chopping? ;) Anyway, I think it's
fair to define a "kernel-core should be smaller than X MB" requirement
but I don't think it's fair to say Y MB because I like the number Y. I
also don't like that we might throw out e.g. NFS just because we're
1MB over the limit.
But if it helps the kernel team to have a fixed number, someone tell
me what we roughly save by throwing out the stuff we discussed and we
can discuss what number would long-termish make sense, I guess. Also,
I'm not sure whether we should measure the extracted files, the
compressed RPM or both.
-- Sandro
8:57 a.m.
On Thu, Mar 06, 2014 at 08:16:00AM +0900, Sandro "red" Mathys wrote:
On Thu, Mar 6, 2014 at 12:13 AM, Don Zickus
<dzickus(a)redhat.com> wrote:
> On Wed, Mar 05, 2014 at 10:02:17AM -0500, Josh Boyer wrote:
>> On Wed, Mar 5, 2014 at 9:54 AM, Don Zickus <dzickus(a)redhat.com> wrote:
>> > On Wed, Mar 05, 2014 at 08:25:12PM +0900, Sandro "red" Mathys
wrote:
>> > For example, lets start with 100MB package requirement for the kernel (and
>> > say 2 GB for userspace). This way the kernel team can implement
>> > reasonable changes and monitor proper usage (because things grow over
>> > time).
>> >
>> > If later on you realize 100 MB is not competitive enough, come back and
>> > chop it down to say 50 MB and let the kernel team figure it out.
>> >
>> > But please do not come in here with a 'every MB counts' approach.
It is
>> > not very sustainable for future growth nor really easy to implement from
>> > an engineering approach.
>> >
>> > Is that acceptable? The kernel team can start with a hard limit of 100MB
>> > package requirement (or something reasonably less)? Let's work off
budget
>> > requirements please.
>>
>> This is a fair point. To be honest, I've ignored the "every MB
>> counts" aspect entirely for now. I've instead been focusing on
>> required functionality, because that's likely going to be the main
>> driver of what the resulting size will be.
That's the point, we want a reasonably small package while still
providing the required functionality. Not sure how providing a fixed
size number is helping in this. But most of all, I didn't throw in a
number because I have no idea what is reasonably possible. I really
only just said "every MB counts" because the question came up before
(in Josh's old thread) and I hoped I could stop this discussion from
happening again before we have any numbers for this.
Ever work in the embedded space? Every MB counts there too. :-) This was
solved by creating budgets for size and memory requirements. This helped
control bloat, which is going to be your biggest problem with cloud
deployment.
What concerns me is that you don't know what size your cloud deployment is
but expect everyone to just chop chop chop. How do we know if the kernel
is already at the right size?
There is s a huge difference between re-architecting the kernel packaging
to save 1 or 2 MB (off ~143 MB size currently) vs. re-architecting to save
50 MB. The former is really a wasted exercise in the bigger picture,
wherease the latter (if proven needed) accomplishes something.
But again it comes down to understanding your environment. Understanding
your environment revovles around control. I get the impression you are
not sure what size your environment should be.
So I was proposing the kernel stay put or maybe create _one_ extras
package that gets installed in addition to the bzImage. But from the
sound of it, the chopping is really going to get you savings of about
~30MB or so.
The thing is a few years ago, the kernel converted a lot of modules to
built-ins (inside the kernel). What that means from a novice perspective
is we took a whole bunch of external bloat that could be stripped away and
stuffed it into the kernel binary itself. The goal at the time was to
speed up boot times (because loading modules was slow).
Now with the re-design of module loading in the last couple of years,
maybe this can be reverted. This could shave MBs off the kernel binary
itself. And then you can only package the modules cloud really needs.
This at least has the ability to scale across other SIGs too.
However, this all hinges on _how much chopping we should do_. I doubt
Josh really wants to embark on this thrashing without very good convincing
reason to do so.
So having some numbers to work off of, provides us the right idea how much
and what type of work needs to get done (little tweaks vs re-think the
whole approach).
> Of course. :-)
>
>>
>> FWIW, the existing kernel package installed today (a debug kernel
>> even) is ~142 MB. 123MB of that is the /lib/modules content. ~6MB of
>> that is vmlinuz. The remaining 13MB is the initramfs, which is
>> actually something that composes on the system during install and not
>> something we can shrink from a packaging standpoint.
>
> It also helps with monitoring. 3-4 years from now after all the chopping,
> these pacakges bloat right back up and everyone forgets why we chopped in
> the first place. Hard requirements help keep everything in check and
> forces people to request more space which the cloud team can evaluate
> properly and still control their enviroment.
Well, if we can remember why we put up a fixed size requirement, why
can't we remember why we did the chopping? ;) Anyway, I think it's
Heh. Ever work with open source projects? The turnover and lost
knowledge is one thing. However, to me the biggets problem would be the
'lack of caring'.
Trust me you can convince everyone to chop MBs out of their packages. You
might even get something really small for your cloud deployment.
Fast forward a year. Do you think package maintainers are going to
remember to go through the painful exercise of chopping again when they
rebase their packages or add new features? Probably not. Bloat will
shoot right back up.
Heck look at Fedora in the last few years. It went from running fine on a
laptop with 2GB of memory to now needing a laptop with 4GB of memory and
lots of disk space. Why? Because everyone thinks memory and disk is
cheap.
However, cloud folks have a different attitude. Memory and disk is
_not_ cheap and can impact their business.
Without disk and memory budgets how are you going to manage the bloat and
make sure the cloud deployments are competitive and provide the host
providers the most bang for the resource buck?
And I get that it is hard to come up with numbers like this. So I would
start with an image and see how it compares. I think Cloud already has an
image. Is it to big? What makes you believe that? Are competitors that
much smaller?
Then start chopping say 5% off at time to see how low you can bring it
before everyone screams too much.
fair to define a "kernel-core should be smaller than X MB"
requirement
but I don't think it's fair to say Y MB because I like the number Y. I
also don't like that we might throw out e.g. NFS just because we're
1MB over the limit.
Why not? If the host provider can buy a 1 TB disk and carve out X number
of cloud deployments on there, then if Fedora Cloud bloats up 10%, that
means less cloud deployments and less revenue. So yes, I would be concerned
over the 1MB. In fact, I would come back and ask the kernel team to find
something else to chop to bring it down in size. Or look to see if some
other package can trade the size.
But if it helps the kernel team to have a fixed number, someone tell
me what we roughly save by throwing out the stuff we discussed and we
can discuss what number would long-termish make sense, I guess. Also,
I'm not sure whether we should measure the extracted files, the
compressed RPM or both.
It should be the opposite. Tell us what number makes long term sense to
you and we can figure out what to throw out. I would calculate the number
based on deployment size (that is ultimately what you need to manage).
That includes the kernel binary, the initrd and the /lib/modules (though
the initrd image is thrown out once the disk is mounted).
I can also tell you how much memory that will take up too (as I did a
similar analysis to reduce the kdump memory usage on RHEL).
Based on Josh's rough number of ~143MB, I would selfishly propose 150 MB
disk space requirement. :-) The memory requirement for that is going to
be about 300MB I believe (I would have to re-run the numbers and figure
out what modules are loaded [probably a _lot_ less than that though]).
Cheers,
Don
9:33 a.m.
On Thu, Mar 6, 2014 at 9:57 AM, Don Zickus <dzickus(a)redhat.com> wrote:
On Thu, Mar 06, 2014 at 08:16:00AM +0900, Sandro "red"
Mathys wrote:
> That's the point, we want a reasonably small package while still
> providing the required functionality. Not sure how providing a fixed
> size number is helping in this. But most of all, I didn't throw in a
> number because I have no idea what is reasonably possible. I really
> only just said "every MB counts" because the question came up before
> (in Josh's old thread) and I hoped I could stop this discussion from
> happening again before we have any numbers for this.
Ever work in the embedded space? Every MB counts there too. :-) This was
solved by creating budgets for size and memory requirements. This helped
control bloat, which is going to be your biggest problem with cloud
deployment.
What concerns me is that you don't know what size your cloud deployment is
but expect everyone to just chop chop chop. How do we know if the kernel
is already at the right size?
There is s a huge difference between re-architecting the kernel packaging
to save 1 or 2 MB (off ~143 MB size currently) vs. re-architecting to save
50 MB. The former is really a wasted exercise in the bigger picture,
wherease the latter (if proven needed) accomplishes something.
But again it comes down to understanding your environment. Understanding
your environment revovles around control. I get the impression you are
not sure what size your environment should be.
So I was proposing the kernel stay put or maybe create _one_ extras
package that gets installed in addition to the bzImage. But from the
Right. When I said I had kernel-core and kernel-drivers, I wasn't
being theoretical. I already did the work in the spec file to split
it into kernel-core and kernel-drivers. The kernel package becomes a
metapackage that requires the other two, so that existing installs and
anaconda don't have to change (assuming I did thinks correctly).
Cloud can just specify kernel-core in the kickstart or whatever.
sound of it, the chopping is really going to get you savings of
about
~30MB or so.
I spent some time yesterday hacking around on an existing VM and just
removing stuff from /lib/modules/ for an installed kernel. I was able
to get it down from 123MB to 58MB by axing entire subsystems that
clearly didn't apply and running depmod on the results to make sure
there weren't missing dependencies. Some stuff had to be added back
(want virtio_scsi? you need target and libfc), but a lot could be
removed. That brings the total to about 81MB for vmlinuz, initramfs,
and /lib/modules for that particular kernel.
In those 81MB, I still had all of the main GPU drivers, all of the
intel and a few other ethernet drivers, ext4, xfs, btrfs, nfs, the
vast majority of the networking modules (so iptables and netfilter),
scsi, acpi, block, char, etc. The major things missing were bluetooth
and wireless, infiniband, some firewire stuff. Basically it resulted
in a system that boots perfectly fine in a VM for a variety of
different use cases.
I think that's a reasonable start, and it's a significant reduction.
Beyond that, we get into much less reduced savings and having to move
stuff around on a finer level. For the curious, I uploaded the module
list here:
http://jwboyer.fedorapeople.org/pub/modules.small
Again, this was just hacking around on an installed system. Still
work to do at a packaging level, but this is as good as anything to
start from.
josh
10:13 a.m.
On Thu, Mar 06, 2014 at 10:33:47AM -0500, Josh Boyer wrote:
On Thu, Mar 6, 2014 at 9:57 AM, Don Zickus <dzickus(a)redhat.com>
wrote:
> On Thu, Mar 06, 2014 at 08:16:00AM +0900, Sandro "red" Mathys wrote:
>> That's the point, we want a reasonably small package while still
>> providing the required functionality. Not sure how providing a fixed
>> size number is helping in this. But most of all, I didn't throw in a
>> number because I have no idea what is reasonably possible. I really
>> only just said "every MB counts" because the question came up before
>> (in Josh's old thread) and I hoped I could stop this discussion from
>> happening again before we have any numbers for this.
>
> Ever work in the embedded space? Every MB counts there too. :-) This was
> solved by creating budgets for size and memory requirements. This helped
> control bloat, which is going to be your biggest problem with cloud
> deployment.
>
> What concerns me is that you don't know what size your cloud deployment is
> but expect everyone to just chop chop chop. How do we know if the kernel
> is already at the right size?
>
> There is s a huge difference between re-architecting the kernel packaging
> to save 1 or 2 MB (off ~143 MB size currently) vs. re-architecting to save
> 50 MB. The former is really a wasted exercise in the bigger picture,
> wherease the latter (if proven needed) accomplishes something.
>
> But again it comes down to understanding your environment. Understanding
> your environment revovles around control. I get the impression you are
> not sure what size your environment should be.
>
> So I was proposing the kernel stay put or maybe create _one_ extras
> package that gets installed in addition to the bzImage. But from the
Right. When I said I had kernel-core and kernel-drivers, I wasn't
being theoretical. I already did the work in the spec file to split
it into kernel-core and kernel-drivers. The kernel package becomes a
metapackage that requires the other two, so that existing installs and
anaconda don't have to change (assuming I did thinks correctly).
Cloud can just specify kernel-core in the kickstart or whatever.
> sound of it, the chopping is really going to get you savings of about
> ~30MB or so.
I spent some time yesterday hacking around on an existing VM and just
removing stuff from /lib/modules/ for an installed kernel. I was able
to get it down from 123MB to 58MB by axing entire subsystems that
clearly didn't apply and running depmod on the results to make sure
there weren't missing dependencies. Some stuff had to be added back
(want virtio_scsi? you need target and libfc), but a lot could be
removed. That brings the total to about 81MB for vmlinuz, initramfs,
and /lib/modules for that particular kernel.
Ok. That is pretty damn good. Good job! Again the question is, is that
enough?
In those 81MB, I still had all of the main GPU drivers, all of the
intel and a few other ethernet drivers, ext4, xfs, btrfs, nfs, the
vast majority of the networking modules (so iptables and netfilter),
scsi, acpi, block, char, etc. The major things missing were bluetooth
and wireless, infiniband, some firewire stuff. Basically it resulted
in a system that boots perfectly fine in a VM for a variety of
different use cases.
I think that's a reasonable start, and it's a significant reduction.
Beyond that, we get into much less reduced savings and having to move
stuff around on a finer level. For the curious, I uploaded the module
list here:
http://jwboyer.fedorapeople.org/pub/modules.small
You might be able to chop sound/ too (I know we could probably go crazy
and chop lots of other things too., just thought sound/ would be a big easy
chop).
Does this approach affect other SIGs too? Most of the others don't care
about disk and memory space, right?
Cheers,
Don
10:18 a.m.
On Thu, Mar 06, 2014 at 10:33:47AM -0500, Josh Boyer wrote:
Right. When I said I had kernel-core and kernel-drivers, I
wasn't
being theoretical. I already did the work in the spec file to split
it into kernel-core and kernel-drivers. The kernel package becomes a
metapackage that requires the other two, so that existing installs and
anaconda don't have to change (assuming I did thinks correctly).
Cloud can just specify kernel-core in the kickstart or whatever.
Does yum's kernel-handling magic need to change to handle this? Probably you
have already thought of that.
I think that's a reasonable start, and it's a significant
reduction.
Beyond that, we get into much less reduced savings and having to move
stuff around on a finer level. For the curious, I uploaded the module
list here:
+1 to getting the low-hanging fruit and not worrying too much about the
stuff up on the hard to reach branches.
--
Matthew Miller -- Fedora Project -- <mattdm(a)fedoraproject.org>
12:14 p.m.
On Thu, Mar 6, 2014 at 11:18 AM, Matthew Miller
<mattdm(a)fedoraproject.org> wrote:
On Thu, Mar 06, 2014 at 10:33:47AM -0500, Josh Boyer wrote:
> Right. When I said I had kernel-core and kernel-drivers, I wasn't
> being theoretical. I already did the work in the spec file to split
> it into kernel-core and kernel-drivers. The kernel package becomes a
> metapackage that requires the other two, so that existing installs and
> anaconda don't have to change (assuming I did thinks correctly).
> Cloud can just specify kernel-core in the kickstart or whatever.
Does yum's kernel-handling magic need to change to handle this? Probably you
have already thought of that.
I gave it some thought. I haven't tested anything. Existing magic
should work for most installs since there's still a "kernel" package
and that's what yum keys off of. If/when we do this, I'll certainly
test more carefully to make sure.
That brings up a question though, how often would cloud expect to do a
yum update of individual packages as opposed to just updating the
entire image? If we expect the 3 kernel magic to work there, then
some changes to yum/dnf may be required given that Cloud would be
explicitly specifying "kernel-core" and not "kernel".
josh
12:34 p.m.
Am 06.03.2014 19:14, schrieb Josh Boyer:
That brings up a question though, how often would cloud expect to do
a
yum update of individual packages as opposed to just updating the
entire image?
in the best case you do the same as i do on our "private cloud"
install once and then upgrade for 5,6...10 years
i consider anything which forces me to a blank re-install as broken
12:48 p.m.
On Thu, Mar 06, 2014 at 01:14:54PM -0500, Josh Boyer wrote:
That brings up a question though, how often would cloud expect to do
a
yum update of individual packages as opposed to just updating the
entire image? If we expect the 3 kernel magic to work there, then
Unless we jump into Colin's rpm ostree future very quickly, we need to
support this.
some changes to yum/dnf may be required given that Cloud would be
explicitly specifying "kernel-core" and not "kernel".
There is a hardcoded default list for install-instead-of-upgrade packages --
I think we'd just ask for kernel-core to be added to that. Additonally, the
protected packages functionality (which keeps you from removing the running
kernel) may need a tweak.
--
Matthew Miller -- Fedora Project -- <mattdm(a)fedoraproject.org>
12:53 p.m.
On Thu, Mar 6, 2014 at 1:48 PM, Matthew Miller <mattdm(a)fedoraproject.org> wrote:
On Thu, Mar 06, 2014 at 01:14:54PM -0500, Josh Boyer wrote:
> That brings up a question though, how often would cloud expect to do a
> yum update of individual packages as opposed to just updating the
> entire image? If we expect the 3 kernel magic to work there, then
Unless we jump into Colin's rpm ostree future very quickly, we need to
support this.
OK.
> some changes to yum/dnf may be required given that Cloud would
be
> explicitly specifying "kernel-core" and not "kernel".
There is a hardcoded default list for install-instead-of-upgrade packages --
I think we'd just ask for kernel-core to be added to that. Additonally, the
protected packages functionality (which keeps you from removing the running
kernel) may need a tweak.
Sigh. I was trying to keep it as self-contained to the kernel package
as possible. Having to modify a bunch of other packages increases the
test matrix and the hassle factor. Oh well.
josh
1:28 a.m.
On Fri, Mar 7, 2014 at 3:53 AM, Josh Boyer <jwboyer(a)fedoraproject.org> wrote:
On Thu, Mar 6, 2014 at 1:48 PM, Matthew Miller
<mattdm(a)fedoraproject.org> wrote:
> On Thu, Mar 06, 2014 at 01:14:54PM -0500, Josh Boyer wrote:
>> That brings up a question though, how often would cloud expect to do a
>> yum update of individual packages as opposed to just updating the
>> entire image? If we expect the 3 kernel magic to work there, then
>
> Unless we jump into Colin's rpm ostree future very quickly, we need to
> support this.
OK.
>> some changes to yum/dnf may be required given that Cloud would be
>> explicitly specifying "kernel-core" and not "kernel".
>
> There is a hardcoded default list for install-instead-of-upgrade packages --
> I think we'd just ask for kernel-core to be added to that. Additonally, the
> protected packages functionality (which keeps you from removing the running
> kernel) may need a tweak.
Sigh. I was trying to keep it as self-contained to the kernel package
as possible. Having to modify a bunch of other packages increases the
test matrix and the hassle factor. Oh well.
What about Anaconda? I guess it does have its own mechanic to
guarantee a kernel is installed, right? Probably hardcoded as well.
Since we're going to build future images with ImageFactory/Anaconda,
it must be possible to install either kernel-core or kernel (when
installing with a kickstart file, that is). I think putting "-kernel"
in %packages doesn't currently have much effect, does it?
So if I'm right and Anaconda needs to be adapted for this, I'm happy
to reach out to the Anaconda team on our behalf, if necessary.
-- Sandro
7:14 a.m.
On Fri, Mar 07, 2014 at 04:28:56PM +0900, Sandro red Mathys wrote:
What about Anaconda? I guess it does have its own mechanic to
guarantee a kernel is installed, right? Probably hardcoded as well.
Since we're going to build future images with ImageFactory/Anaconda,
it must be possible to install either kernel-core or kernel (when
installing with a kickstart file, that is). I think putting "-kernel"
in %packages doesn't currently have much effect, does it?
So if I'm right and Anaconda needs to be adapted for this, I'm happy
to reach out to the Anaconda team on our behalf, if necessary.
Yeah. We also need to have it not install the kernel (and there's no need to
mess with a bootloader) for docker images, so that all kind of comes
together.
--
Matthew Miller -- Fedora Project -- <mattdm(a)fedoraproject.org>
7:23 a.m.
On Fri, Mar 7, 2014 at 8:14 AM, Matthew Miller <mattdm(a)fedoraproject.org> wrote:
On Fri, Mar 07, 2014 at 04:28:56PM +0900, Sandro red Mathys wrote:
> What about Anaconda? I guess it does have its own mechanic to
> guarantee a kernel is installed, right? Probably hardcoded as well.
> Since we're going to build future images with ImageFactory/Anaconda,
> it must be possible to install either kernel-core or kernel (when
> installing with a kickstart file, that is). I think putting "-kernel"
> in %packages doesn't currently have much effect, does it?
>
> So if I'm right and Anaconda needs to be adapted for this, I'm happy
> to reach out to the Anaconda team on our behalf, if necessary.
Yeah. We also need to have it not install the kernel (and there's no need to
mess with a bootloader) for docker images, so that all kind of comes
together.
I'm beginning to like this whole idea less and less.
josh
8:33 a.m.
On Fri, Mar 07, 2014 at 08:23:56AM -0500, Josh Boyer wrote:
>> So if I'm right and Anaconda needs to be adapted for
this, I'm happy
>> to reach out to the Anaconda team on our behalf, if necessary.
> Yeah. We also need to have it not install the kernel (and there's no need to
> mess with a bootloader) for docker images, so that all kind of comes
> together.
I'm beginning to like this whole idea less and less.
Actually, the part I said should make you less worried about that, I hope.
We _already_ need that change completely independently of the work you are
doing here, so there is no additional cost.
--
Matthew Miller -- Fedora Project -- <mattdm(a)fedoraproject.org>
1:46 a.m.
On Fri, Mar 7, 2014 at 3:14 AM, Josh Boyer <jwboyer(a)fedoraproject.org> wrote:
On Thu, Mar 6, 2014 at 11:18 AM, Matthew Miller
<mattdm(a)fedoraproject.org> wrote:
> On Thu, Mar 06, 2014 at 10:33:47AM -0500, Josh Boyer wrote:
>> Right. When I said I had kernel-core and kernel-drivers, I wasn't
>> being theoretical. I already did the work in the spec file to split
>> it into kernel-core and kernel-drivers. The kernel package becomes a
>> metapackage that requires the other two, so that existing installs and
>> anaconda don't have to change (assuming I did thinks correctly).
>> Cloud can just specify kernel-core in the kickstart or whatever.
>
> Does yum's kernel-handling magic need to change to handle this? Probably you
> have already thought of that.
I gave it some thought. I haven't tested anything. Existing magic
should work for most installs since there's still a "kernel" package
and that's what yum keys off of. If/when we do this, I'll certainly
test more carefully to make sure.
That brings up a question though, how often would cloud expect to do a
yum update of individual packages as opposed to just updating the
entire image? If we expect the 3 kernel magic to work there, then
some changes to yum/dnf may be required given that Cloud would be
explicitly specifying "kernel-core" and not "kernel".
Yesterday, I updated to Josh's 2.5 kernel, then I removed kernel and
kernel-drivers, only leaving kernel-core. Today, I again updates, this
time 3.8 being available. Now guess what? The "install instead of
update" magic worked. I know have both kernel-core packages. So I
figure yum works this magic with any kernel* package.
So it would seem no change to yum is necessary after all. Going to
reach out to the yum developers to find out the details. Also, whether
dnf does the exact same or not.
-- Sandro
2:42 a.m.
On Fri, Mar 14, 2014 at 3:46 PM, Sandro "red" Mathys
<red(a)fedoraproject.org> wrote:
On Fri, Mar 7, 2014 at 3:14 AM, Josh Boyer
<jwboyer(a)fedoraproject.org> wrote:
> On Thu, Mar 6, 2014 at 11:18 AM, Matthew Miller
> <mattdm(a)fedoraproject.org> wrote:
>> On Thu, Mar 06, 2014 at 10:33:47AM -0500, Josh Boyer wrote:
>>> Right. When I said I had kernel-core and kernel-drivers, I wasn't
>>> being theoretical. I already did the work in the spec file to split
>>> it into kernel-core and kernel-drivers. The kernel package becomes a
>>> metapackage that requires the other two, so that existing installs and
>>> anaconda don't have to change (assuming I did thinks correctly).
>>> Cloud can just specify kernel-core in the kickstart or whatever.
>>
>> Does yum's kernel-handling magic need to change to handle this? Probably you
>> have already thought of that.
>
> I gave it some thought. I haven't tested anything. Existing magic
> should work for most installs since there's still a "kernel" package
> and that's what yum keys off of. If/when we do this, I'll certainly
> test more carefully to make sure.
>
>
> That brings up a question though, how often would cloud expect to do a
> yum update of individual packages as opposed to just updating the
> entire image? If we expect the 3 kernel magic to work there, then
> some changes to yum/dnf may be required given that Cloud would be
> explicitly specifying "kernel-core" and not "kernel".
Yesterday, I updated to Josh's 2.5 kernel, then I removed kernel and
kernel-drivers, only leaving kernel-core. Today, I again updates, this
time 3.8 being available. Now guess what? The "install instead of
update" magic worked. I know have both kernel-core packages. So I
figure yum works this magic with any kernel* package.
So it would seem no change to yum is necessary after all. Going to
reach out to the yum developers to find out the details. Also, whether
dnf does the exact same or not.
Okay, I poked around in the yum source, yum docs and kernel packages a
bit. So yum (and some testing confirms, dnf too) does not check the
package names but the provides (obviously, thinking about it). The
actual magic being that both kernel and kernel-core provide "kernel".
If that is still the case once the patch is merged from copr to
Fedora, no changes to yum or dnf should become necessary. Which would
probably leave us with "anaconda must allow for installing only just
kernel-core instead of kernel (when kickstarting)" as the only
necessary change to Fedora. I guess.
-- Sandro
9 a.m.
On Fri, Mar 14, 2014 at 16:42:11 +0900,
"Sandro \"red\" Mathys" <red(a)fedoraproject.org> wrote:
Okay, I poked around in the yum source, yum docs and kernel packages a
bit. So yum (and some testing confirms, dnf too) does not check the
package names but the provides (obviously, thinking about it). The
actual magic being that both kernel and kernel-core provide "kernel".
If that is still the case once the patch is merged from copr to
Fedora, no changes to yum or dnf should become necessary. Which would
probably leave us with "anaconda must allow for installing only just
kernel-core instead of kernel (when kickstarting)" as the only
necessary change to Fedora. I guess.
The behavior is controlled by the installonlypkgs setting in yum.conf.
From the man page:
installonlypkgs List of package provides that should only ever
be installed, never updated. Kernels in particular fall into
this category. Defaults to kernel, kernel-bigmem, kernel-enter‐
prise, kernel-smp, kernel-modules, kernel-debug, kernel-unsup‐
ported, kernel-source, kernel-devel, kernel-PAE, kernel-PAE-
debug.
I believe kernel-modules-extra also got added to the default relatively
recently. But that change doesn't seem to have made it to the documentation.
(That package provides installonlypkgs(kernel-module), so there might
be some extra magic related to that provide.)
9:47 a.m.
On Fri, Mar 14, 2014 at 11:00 PM, Bruno Wolff III <bruno(a)wolff.to> wrote:
On Fri, Mar 14, 2014 at 16:42:11 +0900,
"Sandro \"red\" Mathys" <red(a)fedoraproject.org> wrote:
>
>
> Okay, I poked around in the yum source, yum docs and kernel packages a
> bit. So yum (and some testing confirms, dnf too) does not check the
> package names but the provides (obviously, thinking about it). The
> actual magic being that both kernel and kernel-core provide "kernel".
> If that is still the case once the patch is merged from copr to
> Fedora, no changes to yum or dnf should become necessary. Which would
> probably leave us with "anaconda must allow for installing only just
> kernel-core instead of kernel (when kickstarting)" as the only
> necessary change to Fedora. I guess.
The behavior is controlled by the installonlypkgs setting in yum.conf. From
the man page:
installonlypkgs List of package provides that should only ever
be installed, never updated. Kernels in particular fall
into
this category. Defaults to kernel, kernel-bigmem,
kernel-enter‐
prise, kernel-smp, kernel-modules, kernel-debug,
kernel-unsup‐
ported, kernel-source, kernel-devel, kernel-PAE,
kernel-PAE-
debug.
I believe kernel-modules-extra also got added to the default relatively
recently. But that change doesn't seem to have made it to the documentation.
(That package provides installonlypkgs(kernel-module), so there might be
some extra magic related to that provide.)
kernel-modules-extra hasn't been added, but
installonlypkgs(kernel-module) has been. Well, at least that's what I
see in the current code on git. And kernel-drivers also provides
installonlypkgs(kernel-module).
-- Sandro
10:06 a.m.
On Fri, Mar 14, 2014 at 03:46:02PM +0900, Sandro red Mathys wrote:
Yesterday, I updated to Josh's 2.5 kernel, then I removed kernel
and
kernel-drivers, only leaving kernel-core. Today, I again updates, this
time 3.8 being available. Now guess what? The "install instead of
update" magic worked. I know have both kernel-core packages. So I
figure yum works this magic with any kernel* package.
And this is why I should read all the mail before responding. :)
How about the removal protection?
--
Matthew Miller -- Fedora Project -- <mattdm(a)fedoraproject.org>
10:25 a.m.
On Sat, Mar 15, 2014 at 12:06 AM, Matthew Miller
<mattdm(a)fedoraproject.org> wrote:
On Fri, Mar 14, 2014 at 03:46:02PM +0900, Sandro red Mathys wrote:
> Yesterday, I updated to Josh's 2.5 kernel, then I removed kernel and
> kernel-drivers, only leaving kernel-core. Today, I again updates, this
> time 3.8 being available. Now guess what? The "install instead of
> update" magic worked. I know have both kernel-core packages. So I
> figure yum works this magic with any kernel* package.
And this is why I should read all the mail before responding. :)
How about the removal protection?
Oh, good catch. Also, that's a funny one.
So, you can't remove the currently running kernel-core. As expected.
BUT, you can remove the currently running kernel/kernel-drivers.
That's good because we want those packages to be optionally
(un)installable but bad because if you actually require
kernel-drivers, well, it's obviously bad.
Not sure if there's a good fix to this. We'd either limit people to
only being able to get rid of kernel-drivers with some dancing around
with different kernel versions and some rebooting, etc. Or we allow
people to remove the kernel-drivers of the running kernel which
defeats the purpose of the protection. #leSigh
Oh (#2), and here, dnf actually differs from yum. dnf protects *none*
of the packages. So that's definitely a bug and I'll report it once we
know exactly what behavior we want (so that yum and dnf will do the
same thing).
-- Sandro
10:43 a.m.
Am 14.03.2014 16:25, schrieb Sandro red Mathys:
Oh (#2), and here, dnf actually differs from yum. dnf protects
*none*
of the packages. So that's definitely a bug and I'll report it once we
know exactly what behavior we want (so that yum and dnf will do the
same thing)
be careful, on the devel-list that leaded to a flamewar without intention
https://lists.fedoraproject.org/pipermail/devel/2014-January/193310.html
10:46 a.m.
On Fri, Mar 14, 2014 at 11:25 AM, Sandro "red" Mathys
<red(a)fedoraproject.org> wrote:
On Sat, Mar 15, 2014 at 12:06 AM, Matthew Miller
<mattdm(a)fedoraproject.org> wrote:
> On Fri, Mar 14, 2014 at 03:46:02PM +0900, Sandro red Mathys wrote:
>> Yesterday, I updated to Josh's 2.5 kernel, then I removed kernel and
>> kernel-drivers, only leaving kernel-core. Today, I again updates, this
>> time 3.8 being available. Now guess what? The "install instead of
>> update" magic worked. I know have both kernel-core packages. So I
>> figure yum works this magic with any kernel* package.
>
> And this is why I should read all the mail before responding. :)
>
> How about the removal protection?
Oh, good catch. Also, that's a funny one.
So, you can't remove the currently running kernel-core. As expected.
BUT, you can remove the currently running kernel/kernel-drivers.
That's good because we want those packages to be optionally
(un)installable but bad because if you actually require
kernel-drivers, well, it's obviously bad.
"Require" is the word that needs further definition.
Not sure if there's a good fix to this. We'd either limit
people to
only being able to get rid of kernel-drivers with some dancing around
with different kernel versions and some rebooting, etc. Or we allow
people to remove the kernel-drivers of the running kernel which
defeats the purpose of the protection. #leSigh
I don't think we want to over-complicate things here. Removing
-drivers from a machine, whether it's running the corresponding kernel
or not, shouldn't be a huge deal. They might not have support for
some hotplugged device in that case. Also, it's not really going to
be a common case for people to do.
Oh (#2), and here, dnf actually differs from yum. dnf protects
*none*
of the packages. So that's definitely a bug and I'll report it once we
know exactly what behavior we want (so that yum and dnf will do the
same thing).
That's not a bug. They did that on purpose. There was a big thread
about it on the devel list a while ago.
josh
10:59 a.m.
On Sat, Mar 15, 2014 at 12:25:48AM +0900, Sandro red Mathys wrote:
Not sure if there's a good fix to this. We'd either limit
people to
only being able to get rid of kernel-drivers with some dancing around
with different kernel versions and some rebooting, etc. Or we allow
people to remove the kernel-drivers of the running kernel which
defeats the purpose of the protection. #leSigh
I think the first is preferable; the main case where you get a kernel
without the drivers package is when you're building something intended to be
small, and going downwards isn't the way to really do that.
Oh (#2), and here, dnf actually differs from yum. dnf protects
*none*
of the packages. So that's definitely a bug and I'll report it once we
know exactly what behavior we want (so that yum and dnf will do the
same thing).
Apparently this is by design in dnf; it's one of the features they didn't
see as valuable. Which is funny to me because that was the case with yum
initially too (we wrote it as a plugin for that reason) but over time it
became a core feature).
--
Matthew Miller -- Fedora Project -- <mattdm(a)fedoraproject.org>
11:06 a.m.
On Fri, Mar 14, 2014 at 11:59 AM, Matthew Miller
<mattdm(a)fedoraproject.org> wrote:
On Sat, Mar 15, 2014 at 12:25:48AM +0900, Sandro red Mathys wrote:
> Not sure if there's a good fix to this. We'd either limit people to
> only being able to get rid of kernel-drivers with some dancing around
> with different kernel versions and some rebooting, etc. Or we allow
> people to remove the kernel-drivers of the running kernel which
> defeats the purpose of the protection. #leSigh
I think the first is preferable; the main case where you get a kernel
without the drivers package is when you're building something intended to be
small, and going downwards isn't the way to really do that.
That protection needs to be done in yum/dnf then. I don't see a way
to do it cleanly in the kernel packages themselves.
josh
12:38 p.m.
On Fri, Mar 14, 2014 at 12:06:21PM -0400, Josh Boyer wrote:
> I think the first is preferable; the main case where you get a
kernel
> without the drivers package is when you're building something intended
> to be small, and going downwards isn't the way to really do that.
That protection needs to be done in yum/dnf then. I don't see a way
to do it cleanly in the kernel packages themselves.
Yeah, I agree.
--
Matthew Miller -- Fedora Project -- <mattdm(a)fedoraproject.org>
10:01 p.m.
On Sat, Mar 15, 2014 at 2:38 AM, Matthew Miller
<mattdm(a)fedoraproject.org> wrote:
On Fri, Mar 14, 2014 at 12:06:21PM -0400, Josh Boyer wrote:
> > I think the first is preferable; the main case where you get a kernel
> > without the drivers package is when you're building something intended
> > to be small, and going downwards isn't the way to really do that.
> That protection needs to be done in yum/dnf then. I don't see a way
> to do it cleanly in the kernel packages themselves.
Yeah, I agree.
And so would I. But it seems the dnf folks made this decision very
clear several times over a longer time span (like closing a bug in
June 2013, stating the fact in January 2014 and it's even been written
down in the docs) so it's unlikely to be changed. And since dnf is
planned for F22, changing yum's behavior would be much work for only
just one release. Also, as Josh stated, removing kernel-drivers isn't
that critical. Therefore, I vote we don't touch yum.
-- Sandro
9:16 a.m.
On Wed, Mar 05, 2014 at 10:02:17AM -0500, Josh Boyer wrote:
On Wed, Mar 5, 2014 at 9:54 AM, Don Zickus <dzickus(a)redhat.com>
wrote:
> On Wed, Mar 05, 2014 at 08:25:12PM +0900, Sandro "red" Mathys wrote:
> For example, lets start with 100MB package requirement for the kernel (and
> say 2 GB for userspace). This way the kernel team can implement
> reasonable changes and monitor proper usage (because things grow over
> time).
>
> If later on you realize 100 MB is not competitive enough, come back and
> chop it down to say 50 MB and let the kernel team figure it out.
>
> But please do not come in here with a 'every MB counts' approach. It is
> not very sustainable for future growth nor really easy to implement from
> an engineering approach.
>
> Is that acceptable? The kernel team can start with a hard limit of 100MB
> package requirement (or something reasonably less)? Let's work off budget
> requirements please.
This is a fair point. To be honest, I've ignored the "every MB
counts" aspect entirely for now. I've instead been focusing on
required functionality, because that's likely going to be the main
driver of what the resulting size will be.
FWIW, the existing kernel package installed today (a debug kernel
even) is ~142 MB. 123MB of that is the /lib/modules content. ~6MB of
that is vmlinuz. The remaining 13MB is the initramfs, which is
actually something that composes on the system during install and not
something we can shrink from a packaging standpoint.
Also, I just arbitrarly threw out 100MB, if we should start higher, say
150MB, then it doesn't matter to me. :-)
Cheers,
Don
10:28 a.m.
On Wed, 5 Mar 2014 10:16:21 -0500
Don Zickus <dzickus(a)redhat.com> wrote:
Also, I just arbitrarly threw out 100MB, if we should start higher,
say 150MB, then it doesn't matter to me. :-)
This entire disk size optimization seems kind of weird to me.
I just booted a f20 offiical cloud image in our openstack cloud. I used
the m1.tiny (smallest size, no persistent storage):
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 20G 1.1G 19G 6% /
Is a few 10's of MB's really worth making our kernel a bunch more
complex? Is disk space the right thing to be trying to optimize?
Perhaps I am missing it, but are there cases where the current cloud
image is too large? what are they?
kevin
10:33 a.m.
On Wed, Mar 05, 2014 at 09:28:45AM -0700, Kevin Fenzi wrote:
On Wed, 5 Mar 2014 10:16:21 -0500
Don Zickus <dzickus(a)redhat.com> wrote:
> Also, I just arbitrarly threw out 100MB, if we should start higher,
> say 150MB, then it doesn't matter to me. :-)
This entire disk size optimization seems kind of weird to me.
I just booted a f20 offiical cloud image in our openstack cloud. I used
the m1.tiny (smallest size, no persistent storage):
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 20G 1.1G 19G 6% /
Is a few 10's of MB's really worth making our kernel a bunch more
complex? Is disk space the right thing to be trying to optimize?
Perhaps I am missing it, but are there cases where the current cloud
image is too large? what are they?
Haha. Great point!
Cheers,
Don
10:37 a.m.
Am 05.03.2014 17:28, schrieb Kevin Fenzi:
On Wed, 5 Mar 2014 10:16:21 -0500
Don Zickus <dzickus(a)redhat.com> wrote:
> Also, I just arbitrarly threw out 100MB, if we should start higher,
> say 150MB, then it doesn't matter to me. :-)
This entire disk size optimization seems kind of weird to me.
in case of kernel agreed
in general you need to multiply the wasted space for each instance
I just booted a f20 offiical cloud image in our openstack cloud. I
used
the m1.tiny (smallest size, no persistent storage):
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 20G 1.1G 19G 6% /
way too fat and i even regret the 5.8 GB rootfs on 30 instances
but was unsure in 2008 how much reservation would be needed
[root@proxy:~]$ df
Filesystem Type Size Used Avail Use% Mounted on
/dev/sdb1 ext4 5.8G 681M 5.1G 12% /
/dev/sda1 ext4 493M 34M 460M 7% /boot
10:45 a.m.
On Wed, 05 Mar 2014 17:37:42 +0100
Reindl Harald <h.reindl(a)thelounge.net> wrote:
Am 05.03.2014 17:28, schrieb Kevin Fenzi:
> On Wed, 5 Mar 2014 10:16:21 -0500
> Don Zickus <dzickus(a)redhat.com> wrote:
>
>> Also, I just arbitrarly threw out 100MB, if we should start
>> higher, say 150MB, then it doesn't matter to me. :-)
>
> This entire disk size optimization seems kind of weird to me.
in case of kernel agreed
in general you need to multiply the wasted space for each instance
At least for my uses, the amount of non persistent disk space isn't a
big deal. If I need disk space, I would attach a persistent volume...
I'd care much more about memory usage, boot time (you want to spin
things up fast), and cpu usage (cpus are likely the most scarce
resource for me at least).
kevin
10:49 a.m.
Am 05.03.2014 17:45, schrieb Kevin Fenzi:
On Wed, 05 Mar 2014 17:37:42 +0100
Reindl Harald <h.reindl(a)thelounge.net> wrote:
> Am 05.03.2014 17:28, schrieb Kevin Fenzi:
>> On Wed, 5 Mar 2014 10:16:21 -0500
>> Don Zickus <dzickus(a)redhat.com> wrote:
>>
>>> Also, I just arbitrarly threw out 100MB, if we should start
>>> higher, say 150MB, then it doesn't matter to me. :-)
>>
>> This entire disk size optimization seems kind of weird to me.
>
> in case of kernel agreed
> in general you need to multiply the wasted space for each instance
At least for my uses, the amount of non persistent disk space isn't a
big deal. If I need disk space, I would attach a persistent volume...
in my use cases the SAN storage has already 12 SAS 15k disks
and the additional enclosure as well as the disks are expensive
well, i have rotating backups of the whole setups too and so
at least i have to multiply the storage by 3 or 4
I'd care much more about memory usage, boot time (you want to
spin
things up fast), and cpu usage (cpus are likely the most scarce
resource for me at least)
30 Fedora VM's on a single HP DL 380 average 5% CPU usage
that different the usecases are
5:38 p.m.
On Thu, Mar 6, 2014 at 1:45 AM, Kevin Fenzi <kevin(a)scrye.com> wrote:
On Wed, 05 Mar 2014 17:37:42 +0100
Reindl Harald <h.reindl(a)thelounge.net> wrote:
> in general you need to multiply the wasted space for each instance
Exactly, you usually have hundreds or even thousands of instances
running. Sure, "every MB counts" isn't to be taken literal here, maybe
I should rather have said "every 10 MB count".
At least for my uses, the amount of non persistent disk space
isn't a
big deal. If I need disk space, I would attach a persistent volume...
Figure you get your additional persistent volumes for free somehow, so
all those Amazon AWS, HP Cloud, Rackspace, etc. users envy you. And
those admins that need to buy physical disks to put into their private
clouds, too.
Also, more data equals more network traffic and more time - both
things that matter in terms of costs, at least in public clouds.
-- Sandro
9 a.m.
On Thu, Mar 6, 2014 at 12:38 AM, Sandro "red" Mathys
<red(a)fedoraproject.org> wrote:
On Thu, Mar 6, 2014 at 1:45 AM, Kevin Fenzi <kevin(a)scrye.com>
wrote:
> On Wed, 05 Mar 2014 17:37:42 +0100
> Reindl Harald <h.reindl(a)thelounge.net> wrote:
>> in general you need to multiply the wasted space for each instance
Exactly, you usually have hundreds or even thousands of instances
running. Sure, "every MB counts" isn't to be taken literal here, maybe
I should rather have said "every 10 MB count".
I suggested this once before and got no real answer but if you are so
disk space constrained wouldn't file system compression
do what you want instead of trying to micoroptimize every package?
9:13 a.m.
Am 06.03.2014 16:00, schrieb drago01:
On Thu, Mar 6, 2014 at 12:38 AM, Sandro "red" Mathys
<red(a)fedoraproject.org> wrote:
> On Thu, Mar 6, 2014 at 1:45 AM, Kevin Fenzi <kevin(a)scrye.com> wrote:
>> On Wed, 05 Mar 2014 17:37:42 +0100
>> Reindl Harald <h.reindl(a)thelounge.net> wrote:
>>> in general you need to multiply the wasted space for each instance
>
> Exactly, you usually have hundreds or even thousands of instances
> running. Sure, "every MB counts" isn't to be taken literal here, maybe
> I should rather have said "every 10 MB count".
I suggested this once before and got no real answer but if you are so
disk space constrained wouldn't file system compression
do what you want instead of trying to micoroptimize every package?
since there is no transparent compression for the rootfs and
even after btrfs get stable you shouldn't just convert
existing installations from 2008 that's no option
reducing package sizes would affect 5 years old setups too
but as said - i see more sane potential to split the glibc
locales out to sub packages which are as big as the kernel
then risk breaking updates because needed modules are in
a now extra package
another thing would be split docs and manpages in general
in subpackages - only god knows if and when DNF supports
"tsflags=nodocs" which is driven by a yum plugin
another thing would be to split database support of libraries
in subpackages -> dbmail pulls libzdb which pulls mysql, postgresql..
and take a deeper look in circular dependencies at all, i had
several "nice to have" things on my servers and removing
some of them ended in a freed dependency chain uninstalling
400 MB on each machine and even more on some - over the years
i reduced my rootfs-usage down to 40-50% that way
some of these expensive dependencies i solved by maintain
such packages at my own and disable postgresql support
completly - but that's only a sane way in a defined
environment
so again: there are *many* places to gain more than
with the kernel itself and especially in case of C
software yum pulls such subpackages in most cases by
implicit dependencies
11:18 p.m.
On Fri, Mar 7, 2014 at 12:00 AM, drago01 <drago01(a)gmail.com> wrote:
On Thu, Mar 6, 2014 at 12:38 AM, Sandro "red" Mathys
<red(a)fedoraproject.org> wrote:
> On Thu, Mar 6, 2014 at 1:45 AM, Kevin Fenzi <kevin(a)scrye.com> wrote:
>> On Wed, 05 Mar 2014 17:37:42 +0100
>> Reindl Harald <h.reindl(a)thelounge.net> wrote:
>>> in general you need to multiply the wasted space for each instance
>
> Exactly, you usually have hundreds or even thousands of instances
> running. Sure, "every MB counts" isn't to be taken literal here, maybe
> I should rather have said "every 10 MB count".
I suggested this once before and got no real answer but if you are so
disk space constrained wouldn't file system compression
do what you want instead of trying to micoroptimize every package?
Because file system compression requires cpu cycles (i.e. cpu power
and time) and I think they are more valuable to most users than
storage. So, not optimizing the packages would probably be preferable
over compressing the file system. But trimmed down packages need less
space but no additional cpu cycles so it's a win-win from the user's
perspective.
-- Sandro
9:04 a.m.
On Thu, Mar 06, 2014 at 08:38:44AM +0900, Sandro "red" Mathys wrote:
On Thu, Mar 6, 2014 at 1:45 AM, Kevin Fenzi <kevin(a)scrye.com>
wrote:
> On Wed, 05 Mar 2014 17:37:42 +0100
> Reindl Harald <h.reindl(a)thelounge.net> wrote:
>> in general you need to multiply the wasted space for each instance
Exactly, you usually have hundreds or even thousands of instances
running. Sure, "every MB counts" isn't to be taken literal here, maybe
I should rather have said "every 10 MB count".
> At least for my uses, the amount of non persistent disk space isn't a
> big deal. If I need disk space, I would attach a persistent volume...
Figure you get your additional persistent volumes for free somehow, so
all those Amazon AWS, HP Cloud, Rackspace, etc. users envy you. And
those admins that need to buy physical disks to put into their private
clouds, too.
Also, more data equals more network traffic and more time - both
things that matter in terms of costs, at least in public clouds.
Sure, but what if the trade-off in size comes with a cost in speed? Is
cloud ok with the kernel taking twice as long to boot? Or maybe running
slower? Or maybe crashing more often (because we removed safety checks?).
I mean if Josh wanted to he could make everything modular and have a
really small kernel footprint (like 40MB or so) running in 50MB of memory
(I have done this with kdump). But it costs you speed in loading modules
(as opposed to built into the kernel). You may lose other optional
optimizations that help speed things up.
Other SIGs made not like it, but again it depends on how you frame your
environment. Maybe cloud really needs its own kernel. We don't know.
What is cloud willing to sacrafice to obtain smaller size?
Cheers,
Don
10:02 a.m.
On Thu, Mar 06, 2014 at 10:04:56AM -0500, Don Zickus wrote:
On Thu, Mar 06, 2014 at 08:38:44AM +0900, Sandro "red"
Mathys wrote:
> On Thu, Mar 6, 2014 at 1:45 AM, Kevin Fenzi <kevin(a)scrye.com> wrote:
> > On Wed, 05 Mar 2014 17:37:42 +0100
> > Reindl Harald <h.reindl(a)thelounge.net> wrote:
> >> in general you need to multiply the wasted space for each instance
>
> Exactly, you usually have hundreds or even thousands of instances
> running. Sure, "every MB counts" isn't to be taken literal here,
maybe
> I should rather have said "every 10 MB count".
>
> > At least for my uses, the amount of non persistent disk space isn't a
> > big deal. If I need disk space, I would attach a persistent volume...
>
> Figure you get your additional persistent volumes for free somehow, so
> all those Amazon AWS, HP Cloud, Rackspace, etc. users envy you. And
> those admins that need to buy physical disks to put into their private
> clouds, too.
>
> Also, more data equals more network traffic and more time - both
> things that matter in terms of costs, at least in public clouds.
Sure, but what if the trade-off in size comes with a cost in speed? Is
cloud ok with the kernel taking twice as long to boot? Or maybe running
slower? Or maybe crashing more often (because we removed safety checks?).
I mean if Josh wanted to he could make everything modular and have a
really small kernel footprint (like 40MB or so) running in 50MB of memory
(I have done this with kdump). But it costs you speed in loading modules
(as opposed to built into the kernel). You may lose other optional
optimizations that help speed things up.
Other SIGs made not like it, but again it depends on how you frame your
environment. Maybe cloud really needs its own kernel. We don't know.
To touch on this a bit, if splitting the kernel up becomes unwieldy we
could possibly just build kernel-cloud as another flavour in kernel.spec
(similar to PAE, debug, etc). I'm not much of a fan of this though, as
it can lead to a very divergent vmlinux over time.
If it's _necessary_, that's one thing. I've yet to really see any data
backing up necessity on any of this at all though. Right now it seems
to be sitting in the "nice to have" category.
Perhaps someone from the cloud team could look at existing images from
other distros and figure out kernel sizes there, and how it plays into
usage and cost in those environments?
josh
10:32 a.m.
On Thu, Mar 06, 2014 at 11:02:47AM -0500, Josh Boyer wrote:
If it's _necessary_, that's one thing. I've yet to
really see any data
backing up necessity on any of this at all though. Right now it seems
to be sitting in the "nice to have" category.
For the record, it is _literally_ sitting in our "nice to have" category.
See
https://fedoraproject.org/wiki/Cloud_Changelist#Change:_Cloud-Friendly_Ke...
:)
Perhaps someone from the cloud team could look at existing images
from
other distros and figure out kernel sizes there, and how it plays into
usage and cost in those environments?
On the ubuntu EC2 image, /lib/modules/$(uname -r) is 24M + 5.2M vmlinuz +
1.1M in /lib/firmware. Total package size is 32M on disk. And 5.9M initrd.
CoreOS is bigger, with 33M in /lib/modules and 5.2M in lib/firmware, and a
/19M vmlinuz.
Which may just go to show that _calling_ yourself ultra-minimal and focused
is actually more important than _being_ that.
--
Matthew Miller -- Fedora Project -- <mattdm(a)fedoraproject.org>
11:04 a.m.
On Thu, Mar 06, 2014 at 11:32:55AM -0500, Matthew Miller wrote:
On Thu, Mar 06, 2014 at 11:02:47AM -0500, Josh Boyer wrote:
> If it's _necessary_, that's one thing. I've yet to really see any data
> backing up necessity on any of this at all though. Right now it seems
> to be sitting in the "nice to have" category.
For the record, it is _literally_ sitting in our "nice to have" category.
See
https://fedoraproject.org/wiki/Cloud_Changelist#Change:_Cloud-Friendly_Ke...
:)
> Perhaps someone from the cloud team could look at existing images from
> other distros and figure out kernel sizes there, and how it plays into
> usage and cost in those environments?
On the ubuntu EC2 image, /lib/modules/$(uname -r) is 24M + 5.2M vmlinuz +
1.1M in /lib/firmware. Total package size is 32M on disk. And 5.9M initrd.
CoreOS is bigger, with 33M in /lib/modules and 5.2M in lib/firmware, and a
/19M vmlinuz.
Yeah, hard numbers to compete with! :-)
I think Josh is mostly there. He has 58MB + 5M vmlinuz + <similar?>
firmwre.
He just has to cut 35MB or so from /lib/modules/. We can probably nickel
and dime and review a lot of cruft to get there, but what is that 35MB
really doing to get us anything? I am sure half of that can be removed by
re-examining the minimal-list he sent (I can even help there).
Maybe impose only xfs as the fs of choice or some other restrictions and
chop it further, but then we lose flexibility.
Instead of competing with Ubuntu on minimalist can we compete on pretty
close but a lot more flexible? Do Ubuntu users have much choice on how
they configure their environment? Or is Fedora Cloud providing a generic
cookie cutter installation?
Cheers,
Don
12:03 p.m.
Am 06.03.2014 18:04, schrieb Don Zickus:
Maybe impose only xfs as the fs of choice or some other restrictions
and
chop it further, but then we lose flexibility
hopefully a joke :-)
* i use a ton of virtual Fedora instances
* any of them is using ext4
* some may benefit from XFS but not that much to have
different base environments and spread the test matrix
for updates
* even if BTRFS is the overall default in a few years
most of them are installed 2008 and dist-upgraded
over the years, tiny, clean and perfectly maintained
* i do not plan a from-scratch install as long as i live
filesystems may hahve their pros and cons but as admin the
most benefit is limit the things you are using and know
how to handle them in trouble, 1% performance is nice but
the price of a mistake because things are working not as
usual in border cases is way too high
disclaimer:
maybe not perfect english but i think the point is clear
12:10 p.m.
On Thu, Mar 6, 2014 at 12:04 PM, Don Zickus <dzickus(a)redhat.com> wrote:
On Thu, Mar 06, 2014 at 11:32:55AM -0500, Matthew Miller wrote:
> On Thu, Mar 06, 2014 at 11:02:47AM -0500, Josh Boyer wrote:
> > If it's _necessary_, that's one thing. I've yet to really see any
data
> > backing up necessity on any of this at all though. Right now it seems
> > to be sitting in the "nice to have" category.
>
> For the record, it is _literally_ sitting in our "nice to have" category.
> See
>
https://fedoraproject.org/wiki/Cloud_Changelist#Change:_Cloud-Friendly_Ke...
>
> :)
>
>
> > Perhaps someone from the cloud team could look at existing images from
> > other distros and figure out kernel sizes there, and how it plays into
> > usage and cost in those environments?
>
> On the ubuntu EC2 image, /lib/modules/$(uname -r) is 24M + 5.2M vmlinuz +
> 1.1M in /lib/firmware. Total package size is 32M on disk. And 5.9M initrd.
>
> CoreOS is bigger, with 33M in /lib/modules and 5.2M in lib/firmware, and a
> /19M vmlinuz.
Yeah, hard numbers to compete with! :-)
The only way to win is to not play at all? :)
Small note too, just because the vmlinuz are of a certain size does
not mean they contain similar content. Without really digging into
the config settings it's hard to do a true apples to apples
comparison. Still, having the overall sizes handy is helpful, thanks.
I think Josh is mostly there. He has 58MB + 5M vmlinuz +
<similar?>
firmwre.
Firmware is owned by linux-firmware, not the kernel package. I didn't
include it in my kernel numbers for that reason.
He just has to cut 35MB or so from /lib/modules/. We can probably
nickel
and dime and review a lot of cruft to get there, but what is that 35MB
really doing to get us anything? I am sure half of that can be removed by
re-examining the minimal-list he sent (I can even help there).
Right. Considering the bloat elsewhere in the distro, I think we can
start with what I have and work from there if needed.
Maybe impose only xfs as the fs of choice or some other restrictions
and
chop it further, but then we lose flexibility.
Oh dear. Please not another FS thread. So many emails from last week...
Instead of competing with Ubuntu on minimalist can we compete on
pretty
close but a lot more flexible? Do Ubuntu users have much choice on how
they configure their environment? Or is Fedora Cloud providing a generic
cookie cutter installation?
Right, I kind of like that we'd have a smaller core package that is
still broadly useful.
josh
12:43 p.m.
On Thu, Mar 06, 2014 at 01:10:46PM -0500, Josh Boyer wrote:
> I think Josh is mostly there. He has 58MB + 5M vmlinuz +
<similar?>
> firmwre.
Firmware is owned by linux-firmware, not the kernel package. I didn't
include it in my kernel numbers for that reason.
Currently, this is required by the kernel for pre but not at runtime. (I
guess /bin/kernel-install ends up putting things into the initramfs.) Is
there any easy way to do that differently and any win from it?
> Instead of competing with Ubuntu on minimalist can we compete on
pretty
> close but a lot more flexible? Do Ubuntu users have much choice on how
> they configure their environment? Or is Fedora Cloud providing a generic
> cookie cutter installation?
Right, I kind of like that we'd have a smaller core package that is
still broadly useful.
+1
--
Matthew Miller -- Fedora Project -- <mattdm(a)fedoraproject.org>
12:48 p.m.
On Thu, Mar 6, 2014 at 1:43 PM, Matthew Miller <mattdm(a)fedoraproject.org> wrote:
On Thu, Mar 06, 2014 at 01:10:46PM -0500, Josh Boyer wrote:
> > I think Josh is mostly there. He has 58MB + 5M vmlinuz + <similar?>
> > firmwre.
> Firmware is owned by linux-firmware, not the kernel package. I didn't
> include it in my kernel numbers for that reason.
Currently, this is required by the kernel for pre but not at runtime. (I
guess /bin/kernel-install ends up putting things into the initramfs.) Is
there any easy way to do that differently and any win from it?
Possibly. The majority of the firmware is needed by the drivers. If
those drivers aren't in kernel-core, the pre isn't needed on
kernel-core. It would remain on kernel and possibly be added to
kernel-drivers.
Aside from that (or in combination with it), it would basically be
splitting up linux-firmware into more subpackages or maybe putting
requires on specific files.
josh
1:16 a.m.
On Fri, Mar 7, 2014 at 3:10 AM, Josh Boyer <jwboyer(a)fedoraproject.org> wrote:
On Thu, Mar 6, 2014 at 12:04 PM, Don Zickus
<dzickus(a)redhat.com> wrote:
> On Thu, Mar 06, 2014 at 11:32:55AM -0500, Matthew Miller wrote:
>> On Thu, Mar 06, 2014 at 11:02:47AM -0500, Josh Boyer wrote:
>> > If it's _necessary_, that's one thing. I've yet to really see
any data
>> > backing up necessity on any of this at all though. Right now it seems
>> > to be sitting in the "nice to have" category.
>>
>> For the record, it is _literally_ sitting in our "nice to have"
category.
>> See
>>
https://fedoraproject.org/wiki/Cloud_Changelist#Change:_Cloud-Friendly_Ke...
>>
>> :)
>>
>>
>> > Perhaps someone from the cloud team could look at existing images from
>> > other distros and figure out kernel sizes there, and how it plays into
>> > usage and cost in those environments?
>>
>> On the ubuntu EC2 image, /lib/modules/$(uname -r) is 24M + 5.2M vmlinuz +
>> 1.1M in /lib/firmware. Total package size is 32M on disk. And 5.9M initrd.
>>
>> CoreOS is bigger, with 33M in /lib/modules and 5.2M in lib/firmware, and a
>> /19M vmlinuz.
>>
> Yeah, hard numbers to compete with! :-)
I really think comparing our cloud images to theirs is comparing
apples to !apples. The numbers are nice to know what we're competing
against, but don't think we need to have the smallest kernel.
I don't know this for a fact, but I think the Ubuntu EC2 image is
tailored very specifically to run in EC2 only. We don't do this kind
of tailoring in Fedora. One to rule them all.
And CoreOS is very tailored, and has different images for about
everything they run on so it's fair to assume they also have several
different kernel setups. We only want two: core/cloud and full.
The only way to win is to not play at all? :)
Small note too, just because the vmlinuz are of a certain size does
not mean they contain similar content. Without really digging into
the config settings it's hard to do a true apples to apples
comparison. Still, having the overall sizes handy is helpful, thanks.
> I think Josh is mostly there. He has 58MB + 5M vmlinuz + <similar?>
> firmwre.
Firmware is owned by linux-firmware, not the kernel package. I didn't
include it in my kernel numbers for that reason.
> He just has to cut 35MB or so from /lib/modules/. We can probably nickel
> and dime and review a lot of cruft to get there, but what is that 35MB
> really doing to get us anything? I am sure half of that can be removed by
> re-examining the minimal-list he sent (I can even help there).
Right. Considering the bloat elsewhere in the distro, I think we can
start with what I have and work from there if needed.
Excellent progress there. So 35MB are already gone and I figure
(re)moving graphics, sound and other obvious things will gain us quite
some more MB. Nice job.
> Maybe impose only xfs as the fs of choice or some other
restrictions and
> chop it further, but then we lose flexibility.
Oh dear. Please not another FS thread. So many emails from last week...
> Instead of competing with Ubuntu on minimalist can we compete on pretty
> close but a lot more flexible? Do Ubuntu users have much choice on how
> they configure their environment? Or is Fedora Cloud providing a generic
> cookie cutter installation?
Right, I kind of like that we'd have a smaller core package that is
still broadly useful.
Exactly. We only offer different images for different purposes
(software stacks), not for different platforms. So every image should
be able to run on all supported platforms.
-- Sandro
7:23 a.m.
On Fri, Mar 7, 2014 at 2:16 AM, Sandro "red" Mathys
<red(a)fedoraproject.org> wrote:
>> I think Josh is mostly there. He has 58MB + 5M vmlinuz +
<similar?>
>> firmwre.
>
> Firmware is owned by linux-firmware, not the kernel package. I didn't
> include it in my kernel numbers for that reason.
>
>> He just has to cut 35MB or so from /lib/modules/. We can probably nickel
>> and dime and review a lot of cruft to get there, but what is that 35MB
>> really doing to get us anything? I am sure half of that can be removed by
>> re-examining the minimal-list he sent (I can even help there).
>
> Right. Considering the bloat elsewhere in the distro, I think we can
> start with what I have and work from there if needed.
Excellent progress there. So 35MB are already gone and I figure
(re)moving graphics, sound and other obvious things will gain us quite
some more MB. Nice job.
No. 62MB are already "gone". The 35MB was what is needed
additionally to get to comparable numbers with ubuntu. Sound I could
see dropping. Graphics, not so much.
Really, I'm likely to start with what I have and if there are major
reasons that get brought up (with data) to trim further, we can look
at it.
josh
7:59 a.m.
On Fri, Mar 07, 2014 at 08:23:15AM -0500, Josh Boyer wrote:
On Fri, Mar 7, 2014 at 2:16 AM, Sandro "red" Mathys
<red(a)fedoraproject.org> wrote:
>>> I think Josh is mostly there. He has 58MB + 5M vmlinuz + <similar?>
>>> firmwre.
>>
>> Firmware is owned by linux-firmware, not the kernel package. I didn't
>> include it in my kernel numbers for that reason.
>>
>>> He just has to cut 35MB or so from /lib/modules/. We can probably nickel
>>> and dime and review a lot of cruft to get there, but what is that 35MB
>>> really doing to get us anything? I am sure half of that can be removed by
>>> re-examining the minimal-list he sent (I can even help there).
>>
>> Right. Considering the bloat elsewhere in the distro, I think we can
>> start with what I have and work from there if needed.
>
> Excellent progress there. So 35MB are already gone and I figure
> (re)moving graphics, sound and other obvious things will gain us quite
> some more MB. Nice job.
No. 62MB are already "gone". The 35MB was what is needed
additionally to get to comparable numbers with ubuntu. Sound I could
see dropping. Graphics, not so much.
Really, I'm likely to start with what I have and if there are major
reasons that get brought up (with data) to trim further, we can look
at it.
I agree too. I think you did an awesome job Josh. Most of the fat is
trimmed. :-) Anything more should really come with some numbers/data.
Cheers,
Don
8:50 a.m.
On Fri, Mar 7, 2014 at 8:59 AM, Don Zickus <dzickus(a)redhat.com> wrote:
On Fri, Mar 07, 2014 at 08:23:15AM -0500, Josh Boyer wrote:
> On Fri, Mar 7, 2014 at 2:16 AM, Sandro "red" Mathys
> <red(a)fedoraproject.org> wrote:
> >>> I think Josh is mostly there. He has 58MB + 5M vmlinuz +
<similar?>
> >>> firmwre.
> >>
> >> Firmware is owned by linux-firmware, not the kernel package. I didn't
> >> include it in my kernel numbers for that reason.
> >>
> >>> He just has to cut 35MB or so from /lib/modules/. We can probably
nickel
> >>> and dime and review a lot of cruft to get there, but what is that 35MB
> >>> really doing to get us anything? I am sure half of that can be removed
by
> >>> re-examining the minimal-list he sent (I can even help there).
> >>
> >> Right. Considering the bloat elsewhere in the distro, I think we can
> >> start with what I have and work from there if needed.
> >
> > Excellent progress there. So 35MB are already gone and I figure
> > (re)moving graphics, sound and other obvious things will gain us quite
> > some more MB. Nice job.
>
> No. 62MB are already "gone". The 35MB was what is needed
> additionally to get to comparable numbers with ubuntu. Sound I could
> see dropping. Graphics, not so much.
>
> Really, I'm likely to start with what I have and if there are major
> reasons that get brought up (with data) to trim further, we can look
> at it.
I agree too. I think you did an awesome job Josh. Most of the fat is
trimmed. :-) Anything more should really come with some numbers/data.
Don't get too heavy on the praise yet. This still needs to be done at
the packaging level. The changes to kernel.spec might make you want
to poke your eyes out or something ;).
josh
9:06 a.m.
On Fri, Mar 07, 2014 at 09:50:37AM -0500, Josh Boyer wrote:
On Fri, Mar 7, 2014 at 8:59 AM, Don Zickus <dzickus(a)redhat.com>
wrote:
> On Fri, Mar 07, 2014 at 08:23:15AM -0500, Josh Boyer wrote:
>> On Fri, Mar 7, 2014 at 2:16 AM, Sandro "red" Mathys
>> <red(a)fedoraproject.org> wrote:
>> >>> I think Josh is mostly there. He has 58MB + 5M vmlinuz +
<similar?>
>> >>> firmwre.
>> >>
>> >> Firmware is owned by linux-firmware, not the kernel package. I
didn't
>> >> include it in my kernel numbers for that reason.
>> >>
>> >>> He just has to cut 35MB or so from /lib/modules/. We can probably
nickel
>> >>> and dime and review a lot of cruft to get there, but what is that
35MB
>> >>> really doing to get us anything? I am sure half of that can be
removed by
>> >>> re-examining the minimal-list he sent (I can even help there).
>> >>
>> >> Right. Considering the bloat elsewhere in the distro, I think we can
>> >> start with what I have and work from there if needed.
>> >
>> > Excellent progress there. So 35MB are already gone and I figure
>> > (re)moving graphics, sound and other obvious things will gain us quite
>> > some more MB. Nice job.
>>
>> No. 62MB are already "gone". The 35MB was what is needed
>> additionally to get to comparable numbers with ubuntu. Sound I could
>> see dropping. Graphics, not so much.
>>
>> Really, I'm likely to start with what I have and if there are major
>> reasons that get brought up (with data) to trim further, we can look
>> at it.
>
> I agree too. I think you did an awesome job Josh. Most of the fat is
> trimmed. :-) Anything more should really come with some numbers/data.
Don't get too heavy on the praise yet. This still needs to be done at
the packaging level. The changes to kernel.spec might make you want
to poke your eyes out or something ;).
Oh, I thought you had the package split already into kernel-common and
kernel-drivers?
But yes I would be interesting in seeing those changes because that
filters to RHEL eventually. I can already see RHEL developers whining
about extra packages to install... ;-) Though all the kernel-tools stuff
seemed to have gone smoothly...
Cheers,
Don
9:13 a.m.
On Fri, Mar 7, 2014 at 10:06 AM, Don Zickus <dzickus(a)redhat.com> wrote:
On Fri, Mar 07, 2014 at 09:50:37AM -0500, Josh Boyer wrote:
> On Fri, Mar 7, 2014 at 8:59 AM, Don Zickus <dzickus(a)redhat.com> wrote:
> > On Fri, Mar 07, 2014 at 08:23:15AM -0500, Josh Boyer wrote:
> >> On Fri, Mar 7, 2014 at 2:16 AM, Sandro "red" Mathys
> >> <red(a)fedoraproject.org> wrote:
> >> >>> I think Josh is mostly there. He has 58MB + 5M vmlinuz +
<similar?>
> >> >>> firmwre.
> >> >>
> >> >> Firmware is owned by linux-firmware, not the kernel package. I
didn't
> >> >> include it in my kernel numbers for that reason.
> >> >>
> >> >>> He just has to cut 35MB or so from /lib/modules/. We can
probably nickel
> >> >>> and dime and review a lot of cruft to get there, but what is
that 35MB
> >> >>> really doing to get us anything? I am sure half of that can be
removed by
> >> >>> re-examining the minimal-list he sent (I can even help there).
> >> >>
> >> >> Right. Considering the bloat elsewhere in the distro, I think we
can
> >> >> start with what I have and work from there if needed.
> >> >
> >> > Excellent progress there. So 35MB are already gone and I figure
> >> > (re)moving graphics, sound and other obvious things will gain us quite
> >> > some more MB. Nice job.
> >>
> >> No. 62MB are already "gone". The 35MB was what is needed
> >> additionally to get to comparable numbers with ubuntu. Sound I could
> >> see dropping. Graphics, not so much.
> >>
> >> Really, I'm likely to start with what I have and if there are major
> >> reasons that get brought up (with data) to trim further, we can look
> >> at it.
> >
> > I agree too. I think you did an awesome job Josh. Most of the fat is
> > trimmed. :-) Anything more should really come with some numbers/data.
>
> Don't get too heavy on the praise yet. This still needs to be done at
> the packaging level. The changes to kernel.spec might make you want
> to poke your eyes out or something ;).
Oh, I thought you had the package split already into kernel-common and
kernel-drivers?
I have the basics to create those packages and the kernel metapackage
done. Right now they contain strictly /boot for kernel-core and
/lib/modules for kernel-drivers. What I don't have done is the
swizzling around of certain modules to be in kernel-core yet. The
numbers gathering was just hacking a local VM on the filesystem
directly.
So the next step is to translate that to the proper %files entries and
build/test.
But yes I would be interesting in seeing those changes because that
filters to RHEL eventually. I can already see RHEL developers whining
about extra packages to install... ;-) Though all the kernel-tools stuff
seemed to have gone smoothly...
Right. As soon as I get something that seems like it boots in a VM
when you install it, I'll post the changes here for review. The
current stuff I have done is in a COPR if you are _really_ itching to
see it:
http://copr-fe.cloud.fedoraproject.org/coprs/jwboyer/kernel-core/
DO NOT INSTALL THAT. It's stale, won't boot, hasn't been tested, etc.
I'll be building there for now as I work through things, but until
further notice it only exists to make sure things build.
josh
8:16 a.m.
On Wed, Mar 5, 2014 at 2:48 AM, Sandro "red" Mathys
<red(a)fedoraproject.org> wrote:
So, in our case hardware drivers are rather unnecessary and the
kernel
experts might know other ways to shrink the footprint for our limited
use cases. The kernel we require supports all primary architectures
(i686 and x86_64 right now, ARM likely later) and is able to run on a
hypervisor (primarily KVM and Xen but support for ESXi and Hyper-V are
becoming crucial in private clouds). Only. No bare-metal. We also
require support for Linux Containers (LXC) to enable the use of
Docker.
Now, I heard some people screaming when I said HW drivers are
unnecessary. Some people will want to make use of PCI passthrough and
while I think we don't want to ship the necessary modules by default,
they should be easily installable through a separate package. If some
more granularity is acceptable (and makes sense), I think one package
for SRIOV NICs, one for graphic cards (to enable mathematical
computation stuff) and one for everything else PCI passthrough (plus
one for all other HW drivers for the non-cloud products) would be
totally nice.
I'm not overly thrilled with having multi-tiered driver packages.
That leads to major headaches when we start shuffling things around
from one driver package to another. The current solution I have
prototyped is a kernel-core/kernel-drivers split. Here "core" could
be analogous to "cloud", but I imagine it will be useful for other
things. People wanting to do PCI passthrough can just install the
kernel-drivers package.
What does everybody think about this? Can it be done? How is it best
done? What's the timeframe (we'd really like to see this implemented
in F21 Beta but obviously the earlier modular kernels can be tested,
the better)? Do you require additional input?
Same questions I asked Sam and Matt:
- Do you need/want a firewall (requires iptables, etc)?
- Do you need/want NFS or other cloudy storage things (for gluster?)?
- Do you need/want openvswitch?
The list of modules I have in my local rawhide KVM guest is below.
The snd_* related drivers probably aren't necessary. The btrfs and
things it depends on can be ignored (unless you plan on switching from
ext4). Anything that has table or nf in the module name is for the
firewall.
Matt already provided a much smaller module list for openstack and
EC2, but I'm guessing we want to target the broadest usecase. Think
about it and let me know.
josh
[jwboyer@localhost ~]$ lsmod
Module Size Used by
nls_utf8 12557 1
isofs 39794 1
uinput 17708 1
bnep 19735 2
bluetooth 445507 5 bnep
6lowpan_iphc 18591 1 bluetooth
fuse 91190 3
ip6t_rpfilter 12546 1
ip6t_REJECT 12939 2
xt_conntrack 12760 9
cfg80211 583354 0
rfkill 22195 4 cfg80211,bluetooth
ebtable_nat 12807 0
ebtable_broute 12731 0
bridge 135391 1 ebtable_broute
stp 12946 1 bridge
llc 14092 2 stp,bridge
ebtable_filter 12827 0
ebtables 30833 3 ebtable_broute,ebtable_nat,ebtable_filter
ip6table_nat 13015 1
nf_conntrack_ipv6 18777 6
nf_defrag_ipv6 100248 1 nf_conntrack_ipv6
nf_nat_ipv6 13213 1 ip6table_nat
ip6table_mangle 12700 1
ip6table_security 12710 1
ip6table_raw 12683 1
ip6table_filter 12815 1
ip6_tables 26809 5
ip6table_filter,ip6table_mangle,ip6table_security,ip6table_nat,ip6table_raw
iptable_nat 13011 1
nf_conntrack_ipv4 18791 5
nf_defrag_ipv4 12702 1 nf_conntrack_ipv4
nf_nat_ipv4 13199 1 iptable_nat
nf_nat 25249 4 nf_nat_ipv4,nf_nat_ipv6,ip6table_nat,iptable_nat
nf_conntrack 110550 8
nf_nat,nf_nat_ipv4,nf_nat_ipv6,xt_conntrack,ip6table_nat,iptable_nat,nf_conntrack_ipv4,nf_conntrack_ipv6
iptable_mangle 12695 1
iptable_security 12705 1
iptable_raw 12678 1
snd_hda_codec_generic 66943 1
ppdev 17635 0
snd_hda_intel 56588 4
snd_hda_codec 133858 2 snd_hda_codec_generic,snd_hda_intel
snd_hwdep 17650 1 snd_hda_codec
snd_seq 65180 0
snd_seq_device 14136 1 snd_seq
crct10dif_pclmul 14250 0
crc32_pclmul 13113 0
crc32c_intel 22079 0
snd_pcm 103502 2 snd_hda_codec,snd_hda_intel
ghash_clmulni_intel 13259 0
microcode 216608 0
virtio_console 28109 1
serio_raw 13413 0
snd_timer 28806 2 snd_pcm,snd_seq
virtio_balloon 13530 0
snd 83790 16
snd_hwdep,snd_timer,snd_pcm,snd_seq,snd_hda_codec_generic,snd_hda_codec,snd_hda_intel,snd_seq_device
soundcore 14491 1 snd
parport_pc 28048 0
parport 40605 2 ppdev,parport_pc
pvpanic 12801 0
i2c_piix4 22155 0
btrfs 940276 1
xor 21366 1 btrfs
raid6_pq 101472 1 btrfs
qxl 74078 2
drm_kms_helper 50413 1 qxl
virtio_pci 17713 0
8139too 33711 0
virtio_ring 20004 3 virtio_pci,virtio_balloon,virtio_console
virtio 14172 3 virtio_pci,virtio_balloon,virtio_console
ttm 85373 1 qxl
8139cp 32036 0
mii 13527 2 8139cp,8139too
ata_generic 12910 0
drm 288814 4 qxl,ttm,drm_kms_helper
i2c_core 38734 3 drm,i2c_piix4,drm_kms_helper
pata_acpi 13038 0
8:59 a.m.
Hi,
I'm not overly thrilled with having multi-tiered driver
packages.
That leads to major headaches when we start shuffling things around
from one driver package to another. The current solution I have
prototyped is a kernel-core/kernel-drivers split. Here "core" could
be analogous to "cloud", but I imagine it will be useful for other
things. People wanting to do PCI passthrough can just install the
kernel-drivers package.
Agree, I don't think it makes much sense to split things into many small
pieces.
- Do you need/want a firewall (requires iptables, etc)?
I'd say yes by default, but being able to remove it might be useful
(kernel-netfilter subpackage)?
- Do you need/want NFS or other cloudy storage things (for gluster?)?
I'm personally using NFS for host/guest filesystem sharing, so I'd vote
yes.
- Do you need/want openvswitch?
I think no. That's a host side thing and this is about the guest
kernel, right?
The list of modules I have in my local rawhide KVM guest is below.
The snd_* related drivers probably aren't necessary. The btrfs and
things it depends on can be ignored (unless you plan on switching from
ext4).
Depends on what is targeted. Strictly cloud? Or also someone running
Fedora as a guest in virt-manager / boxes?
I'd tend to include drivers for pretty much everything qemu can emulate.
uinput 17708 1
spice guest agent uses this.
bluetooth 445507 5 bnep
cfg80211 583354 0
bluetooth+wireless are not needed, dunno why they are loaded.
snd_hda_codec_generic 66943 1
snd_hda_intel 56588 4
For the qemu HDA soundcard. Should be included if desktop usecase
should be covered too.
qxl 74078 2
gpu driver. There are also cirrus + bochs-drm for qemu.
vmware vga has a drm driver too. They should be included.
ata_generic 12910 0
pata_acpi 13038 0
IDE. Needed. Qemu can emulate AHCI too, should be included (but I
think that is =y anyway).
Qemu can emulate a bunch of SCSI HBAs. Don't think we need the drivers
for them (except virtio-scsi). vmware emulates SCSI HBAs too. pvscsi
should be included. Dunno about the older ones, there was for example
some buslogic emulation in older vmware workstation versions ...
USB: Qemu can also emulate uhci + ehci + xhci, should be included. Also
usb-storage and generic hid support (for the usb tablet). IIRC most of
this is =y anyway.
NIC: e1000, rtl8139 (qemu), tulip (hyperv), dunno about vmware. qemu
can also emulate a bunch of other nics such as ne2k, but I think those
are not relevant in practice and not needed.
Of course all paravirtual drivers should be included too, i.e.
* all virtio (for qemu)
* all xen frontend (for xen, the backend drivers are for the host
side and are not needed in the guest).
* all hyperv (for hyperv)
HTH,
Gerd
9:08 a.m.
On Wed, Mar 5, 2014 at 9:59 AM, Gerd Hoffmann <kraxel(a)redhat.com> wrote:
Hi,
> I'm not overly thrilled with having multi-tiered driver packages.
> That leads to major headaches when we start shuffling things around
> from one driver package to another. The current solution I have
> prototyped is a kernel-core/kernel-drivers split. Here "core" could
> be analogous to "cloud", but I imagine it will be useful for other
> things. People wanting to do PCI passthrough can just install the
> kernel-drivers package.
Agree, I don't think it makes much sense to split things into many small
pieces.
> - Do you need/want a firewall (requires iptables, etc)?
I'd say yes by default, but being able to remove it might be useful
(kernel-netfilter subpackage)?
So you agree multi-tiered subpackages is a bad idea, but then you
propose a netfilter specific subpackage? ... Probably not. They'll
likely just be in kernel-core.
> - Do you need/want NFS or other cloudy storage things (for
gluster?)?
I'm personally using NFS for host/guest filesystem sharing, so I'd vote
yes.
> - Do you need/want openvswitch?
I think no. That's a host side thing and this is about the guest
kernel, right?
Nested virt? I dunno, it was mentioned at one point so I thought I'd
bring it up for discussion again.
> The list of modules I have in my local rawhide KVM guest is
below.
> The snd_* related drivers probably aren't necessary. The btrfs and
> things it depends on can be ignored (unless you plan on switching from
> ext4).
Depends on what is targeted. Strictly cloud? Or also someone running
Fedora as a guest in virt-manager / boxes?
I'm of the opinion the latter is probably what we should shoot for.
It's going to be the broadest target that is still reasonably small.
I'd tend to include drivers for pretty much everything qemu can
emulate.
> uinput 17708 1
spice guest agent uses this.
> bluetooth 445507 5 bnep
> cfg80211 583354 0
bluetooth+wireless are not needed, dunno why they are loaded.
Me either. I'm guessing because systemd/NetworkManager on my KVM
guest auto-loads them.
> snd_hda_codec_generic 66943 1
> snd_hda_intel 56588 4
For the qemu HDA soundcard. Should be included if desktop usecase
should be covered too.
> qxl 74078 2
gpu driver. There are also cirrus + bochs-drm for qemu.
vmware vga has a drm driver too. They should be included.
> ata_generic 12910 0
> pata_acpi 13038 0
IDE. Needed. Qemu can emulate AHCI too, should be included (but I
think that is =y anyway).
Qemu can emulate a bunch of SCSI HBAs. Don't think we need the drivers
for them (except virtio-scsi). vmware emulates SCSI HBAs too. pvscsi
should be included. Dunno about the older ones, there was for example
some buslogic emulation in older vmware workstation versions ...
USB: Qemu can also emulate uhci + ehci + xhci, should be included. Also
usb-storage and generic hid support (for the usb tablet). IIRC most of
this is =y anyway.
NIC: e1000, rtl8139 (qemu), tulip (hyperv), dunno about vmware. qemu
can also emulate a bunch of other nics such as ne2k, but I think those
are not relevant in practice and not needed.
Of course all paravirtual drivers should be included too, i.e.
* all virtio (for qemu)
* all xen frontend (for xen, the backend drivers are for the host
side and are not needed in the guest).
* all hyperv (for hyperv)
OK. Was planning on that already.
josh
9:26 a.m.
Hi,
> Agree, I don't think it makes much sense to split things
into many small
> pieces.
>
>> - Do you need/want a firewall (requires iptables, etc)?
>
> I'd say yes by default, but being able to remove it might be useful
> (kernel-netfilter subpackage)?
So you agree multi-tiered subpackages is a bad idea,
On the hardware support side it is a bad idea I think. Too many
different use cases and lines between them are blurry, so I suspect it
becomes messy quickly if you try to split it fine-grained.
but then you
propose a netfilter specific subpackage? ... Probably not. They'll
likely just be in kernel-core.
"all netfilter modules" is pretty clear and so is the use case (=want
firewall). But maybe not worth the trouble, didn't check what size they
sum up to.
cheers,
Gerd
9:51 a.m.
On Wed, Mar 05, 2014 at 10:08:04 -0500,
Josh Boyer <jwboyer(a)fedoraproject.org> wrote:
So you agree multi-tiered subpackages is a bad idea, but then you
propose a netfilter specific subpackage? ... Probably not. They'll
likely just be in kernel-core.
Couldn't the planned module provides allow something like this to work
without worry much about which modules are in which packages? netfilter
could juts require the modules it needs and it would pull in and extra
kernel module packages that are needed.
9:59 a.m.
On Wed, Mar 5, 2014 at 10:51 AM, Bruno Wolff III <bruno(a)wolff.to> wrote:
On Wed, Mar 05, 2014 at 10:08:04 -0500,
Josh Boyer <jwboyer(a)fedoraproject.org> wrote:
>
>
> So you agree multi-tiered subpackages is a bad idea, but then you
> propose a netfilter specific subpackage? ... Probably not. They'll
> likely just be in kernel-core.
Couldn't the planned module provides allow something like this to work
without worry much about which modules are in which packages? netfilter
could juts require the modules it needs and it would pull in and extra
kernel module packages that are needed.
Possibly. But it's not just the installed system to be taken into
account. Every additional subpackage and list is something that needs
to be maintained and further complicates the kernel.spec file. It
adds to testing all the combinations of kernel packages and
subpackages being installed (or not) and upgraded, etc. For the sake
of the kernel maintainers, keeping it limited is probably a good idea.
And honestly, a firewall in today's world seems pretty darn "core" to me
anyway.
josh
11:33 a.m.
On Wed, 2014-03-05 at 10:59 -0500, Josh Boyer wrote:
On Wed, Mar 5, 2014 at 10:51 AM, Bruno Wolff III
<bruno(a)wolff.to> wrote:
> On Wed, Mar 05, 2014 at 10:08:04 -0500,
> Josh Boyer <jwboyer(a)fedoraproject.org> wrote:
>>
>>
>> So you agree multi-tiered subpackages is a bad idea, but then you
>> propose a netfilter specific subpackage? ... Probably not. They'll
>> likely just be in kernel-core.
>
>
> Couldn't the planned module provides allow something like this to work
> without worry much about which modules are in which packages? netfilter
> could juts require the modules it needs and it would pull in and extra
> kernel module packages that are needed.
Possibly. But it's not just the installed system to be taken into
account. Every additional subpackage and list is something that needs
to be maintained and further complicates the kernel.spec file. It
adds to testing all the combinations of kernel packages and
subpackages being installed (or not) and upgraded, etc. For the sake
of the kernel maintainers, keeping it limited is probably a good idea.
This can also have unforseen consequences. Coming from conary, where we
had rich dependencies stored in the repository, if you start putting
module provides and depends metadata into the repositories for every
single module, the CPU time and network traffic required to even do a
simple update can grow excessively. I am not saying that there is no
solution to the problem, but I wouldn't count on it near term.
Justin
11:56 a.m.
On Wed, Mar 05, 2014 at 11:33:57AM -0600, Justin M. Forbes wrote:
On Wed, 2014-03-05 at 10:59 -0500, Josh Boyer wrote:
> On Wed, Mar 5, 2014 at 10:51 AM, Bruno Wolff III <bruno(a)wolff.to> wrote:
> > On Wed, Mar 05, 2014 at 10:08:04 -0500,
> > Josh Boyer <jwboyer(a)fedoraproject.org> wrote:
> >>
> >>
> >> So you agree multi-tiered subpackages is a bad idea, but then you
> >> propose a netfilter specific subpackage? ... Probably not. They'll
> >> likely just be in kernel-core.
> >
> >
> > Couldn't the planned module provides allow something like this to work
> > without worry much about which modules are in which packages? netfilter
> > could juts require the modules it needs and it would pull in and extra
> > kernel module packages that are needed.
>
> Possibly. But it's not just the installed system to be taken into
> account. Every additional subpackage and list is something that needs
> to be maintained and further complicates the kernel.spec file. It
> adds to testing all the combinations of kernel packages and
> subpackages being installed (or not) and upgraded, etc. For the sake
> of the kernel maintainers, keeping it limited is probably a good idea.
This can also have unforseen consequences. Coming from conary, where we
had rich dependencies stored in the repository, if you start putting
module provides and depends metadata into the repositories for every
single module, the CPU time and network traffic required to even do a
simple update can grow excessively. I am not saying that there is no
solution to the problem, but I wouldn't count on it near term.
Yeah, this is the biggest downside to auto-generated Provides for
modules. I haven't had time to flip the switch and see how much it
impacts things yet. If it results in any form of noticable impact we
need to think long and hard about whether it's actually worth doing.
josh
1:21 p.m.
On Wed, Mar 05, 2014 at 10:08:04AM -0500, Josh Boyer wrote:
> Depends on what is targeted. Strictly cloud? Or also someone
running
> Fedora as a guest in virt-manager / boxes?
I'm of the opinion the latter is probably what we should shoot for.
It's going to be the broadest target that is still reasonably small.
I agree that this is valuable, because it lets the same image be used for
testing locally.
--
Matthew Miller -- Fedora Project -- <mattdm(a)fedoraproject.org>
4:59 p.m.
On Thu, Mar 6, 2014 at 12:08 AM, Josh Boyer <jwboyer(a)fedoraproject.org> wrote:
On Wed, Mar 5, 2014 at 9:59 AM, Gerd Hoffmann
<kraxel(a)redhat.com> wrote:
>
> Hi,
>
>> I'm not overly thrilled with having multi-tiered driver packages.
>> That leads to major headaches when we start shuffling things around
>> from one driver package to another. The current solution I have
>> prototyped is a kernel-core/kernel-drivers split. Here "core" could
>> be analogous to "cloud", but I imagine it will be useful for other
>> things. People wanting to do PCI passthrough can just install the
>> kernel-drivers package.
Fair enough, agreed. We might need to eventually revisit this decision
if SRIOV or graphic cards computing really take off and people
complain. But right now, it's edge cases.
> Agree, I don't think it makes much sense to split things into
many small
> pieces.
>
>> - Do you need/want a firewall (requires iptables, etc)?
While not strictly necessary (the IaaS usually puts a firewall right
in front of the instance / guest), I think we do want to allow each
and everyone to optimize their security needs. The local iptables also
has way more capabilities than the provided one offers.
>
> I'd say yes by default, but being able to remove it might be useful
> (kernel-netfilter subpackage)?
So you agree multi-tiered subpackages is a bad idea, but then you
propose a netfilter specific subpackage? ... Probably not. They'll
likely just be in kernel-core.
>> - Do you need/want NFS or other cloudy storage things (for gluster?)?
Usually, you just attach additional volumes when you need more space
but, particularly in private clouds, people do all kinds of weird
things to get their data from here to there so I think we want all
storage options. Also, support for every (major) filesystem that you
could put on additionally attached disk drives.
> I'm personally using NFS for host/guest filesystem sharing,
so I'd vote
> yes.
>
>> - Do you need/want openvswitch?
>
> I think no. That's a host side thing and this is about the guest
> kernel, right?
Nested virt? I dunno, it was mentioned at one point so I thought I'd
bring it up for discussion again.
There are use cases that use OVS but I'd say they are rare. And nested
virt is even rarer (particularly since Docker helped pushing lxc). So
I don't think we want OVS in the guest image.
>> The list of modules I have in my local rawhide KVM guest is
below.
>> The snd_* related drivers probably aren't necessary. The btrfs and
>> things it depends on can be ignored (unless you plan on switching from
>> ext4).
As mentioned above, btrfs could be used on user-attached drives so
keep that in there. But the root disk will be in line with the other
Fedora products.
> Depends on what is targeted. Strictly cloud? Or also someone
running
> Fedora as a guest in virt-manager / boxes?
I'm of the opinion the latter is probably what we should shoot for.
It's going to be the broadest target that is still reasonably small.
"Guest in virt-manager" is not part of our PRD, so it's not actually
required. But as Matt mentioned, it's incredibly valuable for testing
purposes. That should be a good enough reason to include something
reasonably small.
> I'd tend to include drivers for pretty much everything qemu can emulate.
>
> <snip-snap detailed module talk>
OK. Was planning on that already.
Make it so :)
-- Sandro
3692
days inactive
3702
days old
kernel@lists.fedoraproject.org
65 comments
10 participants
participants (10)
-
Bruno Wolff III -
Don Zickus -
drago01 -
Gerd Hoffmann -
Josh Boyer -
Justin Forbes -
Kevin Fenzi -
Matthew Miller -
Reindl Harald -
Sandro "red" Mathys