Re: Modular Kernel Packaging for Cloud
by Josh Boyer
On Wed, Mar 5, 2014 at 9:59 AM, Gerd Hoffmann <kraxel(a)redhat.com> wrote:
>
> Hi,
>
>> I'm not overly thrilled with having multi-tiered driver packages.
>> That leads to major headaches when we start shuffling things around
>> from one driver package to another. The current solution I have
>> prototyped is a kernel-core/kernel-drivers split. Here "core" could
>> be analogous to "cloud", but I imagine it will be useful for other
>> things. People wanting to do PCI passthrough can just install the
>> kernel-drivers package.
>
> Agree, I don't think it makes much sense to split things into many small
> pieces.
>
>> - Do you need/want a firewall (requires iptables, etc)?
>
> I'd say yes by default, but being able to remove it might be useful
> (kernel-netfilter subpackage)?
So you agree multi-tiered subpackages is a bad idea, but then you
propose a netfilter specific subpackage? ... Probably not. They'll
likely just be in kernel-core.
>> - Do you need/want NFS or other cloudy storage things (for gluster?)?
>
> I'm personally using NFS for host/guest filesystem sharing, so I'd vote
> yes.
>
>> - Do you need/want openvswitch?
>
> I think no. That's a host side thing and this is about the guest
> kernel, right?
Nested virt? I dunno, it was mentioned at one point so I thought I'd
bring it up for discussion again.
>> The list of modules I have in my local rawhide KVM guest is below.
>> The snd_* related drivers probably aren't necessary. The btrfs and
>> things it depends on can be ignored (unless you plan on switching from
>> ext4).
>
> Depends on what is targeted. Strictly cloud? Or also someone running
> Fedora as a guest in virt-manager / boxes?
I'm of the opinion the latter is probably what we should shoot for.
It's going to be the broadest target that is still reasonably small.
> I'd tend to include drivers for pretty much everything qemu can emulate.
>
>> uinput 17708 1
>
> spice guest agent uses this.
>
>> bluetooth 445507 5 bnep
>> cfg80211 583354 0
>
> bluetooth+wireless are not needed, dunno why they are loaded.
Me either. I'm guessing because systemd/NetworkManager on my KVM
guest auto-loads them.
>> snd_hda_codec_generic 66943 1
>> snd_hda_intel 56588 4
>
> For the qemu HDA soundcard. Should be included if desktop usecase
> should be covered too.
>
>> qxl 74078 2
>
> gpu driver. There are also cirrus + bochs-drm for qemu.
> vmware vga has a drm driver too. They should be included.
>
>> ata_generic 12910 0
>> pata_acpi 13038 0
>
> IDE. Needed. Qemu can emulate AHCI too, should be included (but I
> think that is =y anyway).
>
> Qemu can emulate a bunch of SCSI HBAs. Don't think we need the drivers
> for them (except virtio-scsi). vmware emulates SCSI HBAs too. pvscsi
> should be included. Dunno about the older ones, there was for example
> some buslogic emulation in older vmware workstation versions ...
>
> USB: Qemu can also emulate uhci + ehci + xhci, should be included. Also
> usb-storage and generic hid support (for the usb tablet). IIRC most of
> this is =y anyway.
>
> NIC: e1000, rtl8139 (qemu), tulip (hyperv), dunno about vmware. qemu
> can also emulate a bunch of other nics such as ne2k, but I think those
> are not relevant in practice and not needed.
>
> Of course all paravirtual drivers should be included too, i.e.
> * all virtio (for qemu)
> * all xen frontend (for xen, the backend drivers are for the host
> side and are not needed in the guest).
> * all hyperv (for hyperv)
OK. Was planning on that already.
josh
10 years, 1 month
Google Compute Engine Fedora Images (status update)
by Vaidas Jablonskis
Hi People,
I am pretty new to contributing to Fedora, especially the cloud group.
I have mentioned a few times on IRC (nickname: zooz) that I have started
working on GCE cloud image.
So I thought I will give you an update on my progress so other people can
jump in or help me to figure out the process.
So far so good, GCE image does not seem to look much different that already
existing OpenStack/EC2 images.
- kickstart files for appliance-creator:
https://github.com/vaijab/fedora-gce-image
- gcimagebundle spec/srpm/copr: https://github.com/vaijab/gcimagebundle-rpm
- google-compute-daemon spec/srpm/copr:
https://github.com/vaijab/google-compute-daemon-rpm
I understand that you guys might not want to use google provided daemons
for managing ssh keys, users and forwarded IP addresses, but I thought I
will package it up just in case.
Current blockers: https://bugzilla.redhat.com/show_bug.cgi?id=1055181 but
they accepted my patch and it's been submitted to updates-testing, so it
should be fine soon.
So with what I currently have, I am able to build an image without any
issues which runs on GCE perfectly.
If anyone could have a quick poke at my RPM spec files and let me know if
there are any major issues with them before I submit them for a review for
the first time :-) Also if anyone wants to sponsor my fedora packager
membership I'd really appreciate that.
So what are the next steps? How do we actually start a process of getting
appliance KS files upstream?
Any pointers and feedback is welcome. If anyone wants to jump in and help -
please do so!
Thanks,
Vaidas
--
Vaidas Jablonskis
10 years, 1 month
Re: Modular Kernel Packaging for Cloud
by Josh Boyer
On Wed, Mar 5, 2014 at 9:54 AM, Don Zickus <dzickus(a)redhat.com> wrote:
> On Wed, Mar 05, 2014 at 08:25:12PM +0900, Sandro "red" Mathys wrote:
> For example, lets start with 100MB package requirement for the kernel (and
> say 2 GB for userspace). This way the kernel team can implement
> reasonable changes and monitor proper usage (because things grow over
> time).
>
> If later on you realize 100 MB is not competitive enough, come back and
> chop it down to say 50 MB and let the kernel team figure it out.
>
> But please do not come in here with a 'every MB counts' approach. It is
> not very sustainable for future growth nor really easy to implement from
> an engineering approach.
>
> Is that acceptable? The kernel team can start with a hard limit of 100MB
> package requirement (or something reasonably less)? Let's work off budget
> requirements please.
This is a fair point. To be honest, I've ignored the "every MB
counts" aspect entirely for now. I've instead been focusing on
required functionality, because that's likely going to be the main
driver of what the resulting size will be.
FWIW, the existing kernel package installed today (a debug kernel
even) is ~142 MB. 123MB of that is the /lib/modules content. ~6MB of
that is vmlinuz. The remaining 13MB is the initramfs, which is
actually something that composes on the system during install and not
something we can shrink from a packaging standpoint.
josh
10 years, 1 month
Re: Modular Kernel Packaging for Cloud
by Sandro "red" Mathys
On Wed, Mar 5, 2014 at 6:58 PM, drago01 <drago01(a)gmail.com> wrote:
> On Wed, Mar 5, 2014 at 8:48 AM, Sandro "red" Mathys
> <red(a)fedoraproject.org> wrote:
>> Heads-up: I've taken ownership over the Cloud SIG's planned
>> "Cloud-Friendly Kernel Packaging" (aka "Modular Kernel Packaging for
>> Cloud") Change [0].
>>
>> (Sorry for cross-posting this on the kernel and cloud lists, but both
>> the kernel team and the cloud SIG need to be involved in this
>> discussion).
>>
>> Obviously, we wish for modular kernel packaging and Josh Boyer already
>> reached out to the cloud SIG in this regard a while ago when we didn't
>> really know what we want yet [1]. We do now know that somewhat better
>> [2] but details could still require additional discussion. Our
>> motivation is to save space, time and bandwidth (the latter two
>> referring to required updates) as in the cloud, all three equal to
>> costs. And yes, we also target things other than the kernel to save
>> space.
>>
>> So, in our case hardware drivers are rather unnecessary and the kernel
>> experts might know other ways to shrink the footprint for our limited
>> use cases. The kernel we require supports all primary architectures
>> (i686 and x86_64 right now, ARM likely later) and is able to run on a
>> hypervisor (primarily KVM and Xen but support for ESXi and Hyper-V are
>> becoming crucial in private clouds). Only. No bare-metal. We also
>> require support for Linux Containers (LXC) to enable the use of
>> Docker.
>>
>> Now, I heard some people screaming when I said HW drivers are
>> unnecessary. Some people will want to make use of PCI passthrough and
>> while I think we don't want to ship the necessary modules by default,
>> they should be easily installable through a separate package. If some
>> more granularity is acceptable (and makes sense), I think one package
>> for SRIOV NICs, one for graphic cards (to enable mathematical
>> computation stuff) and one for everything else PCI passthrough (plus
>> one for all other HW drivers for the non-cloud products) would be
>> totally nice.
>>
>> What does everybody think about this? Can it be done? How is it best
>> done? What's the timeframe (we'd really like to see this implemented
>> in F21 Beta but obviously the earlier modular kernels can be tested,
>> the better)? Do you require additional input?
>>
>> Please try to keep the discussion as simple as possible (in terms of
>> us non-kernel hackers should be able to follow it). Just like Josh
>> requested when reaching out to us: "Explain it to me like I'm a
>> child."
>
> Some drivers are built inside the kernel (for various reasons) so if
> we keep them in they can't really be modular.
If there are reasons they are built-in (and I think currently nothing
is being built-in without good reasons), that should not be changed.
Only just talking about what is currently a module (or otherwise a
separate file - if applicable). The vmlinuz should be kept as-is and
stay the same for all products (server, workstation and cloud).
> If we are going down this route (ex. kernel-hw-drivers sub package) we
> should have a way to handle upgrades.
> You don't want to upgrade and be left out without any hardware support
> because it is now a sub package that nothing pulls in.
Yes, that's only too true. But it should be possible to guarantee
proper upgrades through means of packaging. And of course this will be
tested.
> How much are you trying to save by this? All modules (excluding extra
> which is already a sub package) are ~115MB here on x86_64.
As much as is sensibly possible. Every MB counts but we don't want to
drive the kernel team members insane. :)
-- Sandro
10 years, 1 month
changes list for fesco (Cloud WG members -- action needed!)
by Matthew Miller
Hi everyone. Joe and I did a lot of work on the cloud changelist, starting
from the brainstorming list that Sandro put into place. Please take a look
at <https://fedoraproject.org/wiki/Cloud_Changelist>.
The next steps are:
1. Many of the changes and dependencies are missing owners. It would be
nice to fill that in, although we can leave some of them TBD if need be.
2. Please make sure that we aren't missing anything crucial. Discuss here
if it looks like something is missing.
3. There's some room to discuss the priorities, although I do think we have
it about right.
4. Do feel free to make any minor cleanup or enhancements. Even spelling
fixes and the like. If it seems big, mention it here, or otherwise just
do it.
5. I propose that we accept this list by lazy consensus -- that is, if we
don't get any -1s from WG members by 23:00 UTC on Monday, we'll consider
it accepted. I don't think there is anything controversial in here.
Thanks everyone!
--
Matthew Miller -- Fedora Project -- <mattdm(a)fedoraproject.org>
10 years, 1 month
