Hello infra!
Currently, in Copr team we are periodically preparing those kinds of images for our VM builders:
- x86_64 AMI image for AWS - aarch64 AMI - x86_64 image for hypervisors - ppc64le qcow2 image (uploaded to Power9 OpenStack and to our Power8 hypervisors) - s390x qcow2 (built in IBM Cloud) - internally we build also x86_64 images for OpenStack
The way we do this now is pretty complicated:
- we start with the officially provided Fedora images (AMI/QCOW2) - we modify and update them using ansible scripts and/or libguestfs - then the images are uploaded, and tested in our development stack - and if everything is OK, then these images are used in production
Some more info https://docs.pagure.org/copr.copr/how_to_upgrade_builders.html
This is a tedious and repetitive list of tasks, and I'm sure we should/could automatize some parts. At least it would be awesome if we could try to trigger the build(s) for all the images by "a single button", somehow, somewhere... So I am here to ask you how you would do this.
- I thought the answer is Image Builder, but there's a missing support for Fedora ppc64le and s390x targets for now (rhbz#2040685).
- livemedia-creator, seems to be a bit related to Image Builder but I was told some time ago that AMI support is experimental, the docs say "At this time I have not tested the image with EC2. Feedback would be welcome."
- There's a Packer software (packer.io) used by testing farm folks (the tooling is not available in Fedora yet).
- Koji is able to build some images using Kickstarts. This one looks like low-hanging fruit. Existing and working solution, we could built on top of of the maintained fedora-kickstarts files that we "use" anyway (through the pre-built official images). But I'm not sure if our team could be allowed to use Koji like this, and provide additional *ks files?
Are there any other possibilities? Opinions?
Thanks, Pavel
I have looked at Packer and it looks like it could be used for this. Not sure how koji works with kickstarts but would love to check it out! On the authorization logistics side, I'm not the right person to talk to, but if I could help in any way that would be great. Thanks, Leo
On Wed, Feb 23, 2022 at 9:19 AM Pavel Raiskup praiskup@redhat.com wrote:
Hello infra!
Currently, in Copr team we are periodically preparing those kinds of images for our VM builders:
- x86_64 AMI image for AWS
- aarch64 AMI
- x86_64 image for hypervisors
- ppc64le qcow2 image (uploaded to Power9 OpenStack and to our Power8
hypervisors)
- s390x qcow2 (built in IBM Cloud)
- internally we build also x86_64 images for OpenStack
The way we do this now is pretty complicated:
- we start with the officially provided Fedora images (AMI/QCOW2)
- we modify and update them using ansible scripts and/or libguestfs
- then the images are uploaded, and tested in our development stack
- and if everything is OK, then these images are used in production
Some more info https://docs.pagure.org/copr.copr/how_to_upgrade_builders.html
This is a tedious and repetitive list of tasks, and I'm sure we should/could automatize some parts. At least it would be awesome if we could try to trigger the build(s) for all the images by "a single button", somehow, somewhere... So I am here to ask you how you would do this.
I thought the answer is Image Builder, but there's a missing support for Fedora ppc64le and s390x targets for now (rhbz#2040685).
livemedia-creator, seems to be a bit related to Image Builder but I was
told some time ago that AMI support is experimental, the docs say "At this time I have not tested the image with EC2. Feedback would be welcome."
- There's a Packer software (packer.io) used by testing farm folks (the
tooling is not available in Fedora yet).
- Koji is able to build some images using Kickstarts. This one looks like low-hanging fruit. Existing and working solution, we could built on top
of of the maintained fedora-kickstarts files that we "use" anyway (through the pre-built official images). But I'm not sure if our team could be allowed to use Koji like this, and provide additional *ks files?
Are there any other possibilities? Opinions?
Thanks, Pavel
infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedorapro... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Pavel, I know my team uses packer.io to build our golden images for our infrastructure. I know we build qcow2 and AMI images and have them uploaded to our AWS, OpenStack, and RHV instances. I would be willing to sit down and help test this with you if you would like.
On Wed, Feb 23, 2022, 12:19 PM Pavel Raiskup praiskup@redhat.com wrote:
Hello infra!
Currently, in Copr team we are periodically preparing those kinds of images for our VM builders:
- x86_64 AMI image for AWS
- aarch64 AMI
- x86_64 image for hypervisors
- ppc64le qcow2 image (uploaded to Power9 OpenStack and to our Power8
hypervisors)
- s390x qcow2 (built in IBM Cloud)
- internally we build also x86_64 images for OpenStack
The way we do this now is pretty complicated:
- we start with the officially provided Fedora images (AMI/QCOW2)
- we modify and update them using ansible scripts and/or libguestfs
- then the images are uploaded, and tested in our development stack
- and if everything is OK, then these images are used in production
Some more info https://docs.pagure.org/copr.copr/how_to_upgrade_builders.html
This is a tedious and repetitive list of tasks, and I'm sure we should/could automatize some parts. At least it would be awesome if we could try to trigger the build(s) for all the images by "a single button", somehow, somewhere... So I am here to ask you how you would do this.
I thought the answer is Image Builder, but there's a missing support for Fedora ppc64le and s390x targets for now (rhbz#2040685).
livemedia-creator, seems to be a bit related to Image Builder but I was
told some time ago that AMI support is experimental, the docs say "At this time I have not tested the image with EC2. Feedback would be welcome."
- There's a Packer software (packer.io) used by testing farm folks (the
tooling is not available in Fedora yet).
- Koji is able to build some images using Kickstarts. This one looks like low-hanging fruit. Existing and working solution, we could built on top
of of the maintained fedora-kickstarts files that we "use" anyway (through the pre-built official images). But I'm not sure if our team could be allowed to use Koji like this, and provide additional *ks files?
Are there any other possibilities? Opinions?
Thanks, Pavel
infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedorapro... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
On Wed, Feb 23, 2022 at 12:19 PM Pavel Raiskup praiskup@redhat.com wrote:
Hello infra!
Currently, in Copr team we are periodically preparing those kinds of images for our VM builders:
- x86_64 AMI image for AWS
- aarch64 AMI
- x86_64 image for hypervisors
- ppc64le qcow2 image (uploaded to Power9 OpenStack and to our Power8 hypervisors)
- s390x qcow2 (built in IBM Cloud)
- internally we build also x86_64 images for OpenStack
The way we do this now is pretty complicated:
- we start with the officially provided Fedora images (AMI/QCOW2)
- we modify and update them using ansible scripts and/or libguestfs
- then the images are uploaded, and tested in our development stack
- and if everything is OK, then these images are used in production
Some more info https://docs.pagure.org/copr.copr/how_to_upgrade_builders.html
This is a tedious and repetitive list of tasks, and I'm sure we should/could automatize some parts. At least it would be awesome if we could try to trigger the build(s) for all the images by "a single button", somehow, somewhere... So I am here to ask you how you would do this.
I thought the answer is Image Builder, but there's a missing support for Fedora ppc64le and s390x targets for now (rhbz#2040685).
livemedia-creator, seems to be a bit related to Image Builder but I was told some time ago that AMI support is experimental, the docs say "At this time I have not tested the image with EC2. Feedback would be welcome."
There's a Packer software (packer.io) used by testing farm folks (the tooling is not available in Fedora yet).
Koji is able to build some images using Kickstarts. This one looks like low-hanging fruit. Existing and working solution, we could built on top of of the maintained fedora-kickstarts files that we "use" anyway (through the pre-built official images). But I'm not sure if our team could be allowed to use Koji like this, and provide additional *ks files?
Are there any other possibilities? Opinions?
The kiwi image builder would be another option. The Fedora Cloud and CentOS Hyperscale teams are exploring using it for images, and we've been collaborating with the Koji folks on kiwi support. Also, at work I'm doing stuff with kiwi for our cloud images, and I'd be happy to help teach y'all how to use kiwi for building images for all of your platforms.
It's packaged in Fedora and EPEL, so it's easy to get and use.
-- 真実はいつも一つ!/ Always, there's only one truth!
On Wed, Feb 23, 2022 at 06:18:24PM +0100, Pavel Raiskup wrote:
Hello infra!
Currently, in Copr team we are periodically preparing those kinds of images for our VM builders:
...snip...
- I thought the answer is Image Builder, but there's a missing support for Fedora ppc64le and s390x targets for now (rhbz#2040685).
Well, it could still be. My understanding of things is that we could run ImageBuilder plugins in our koji and when we request builds it uses our koji to build them instead of internal as a service builders. Of course thats a ways out so, not sure it helps you now.
livemedia-creator, seems to be a bit related to Image Builder but I was told some time ago that AMI support is experimental, the docs say "At this time I have not tested the image with EC2. Feedback would be welcome."
There's a Packer software (packer.io) used by testing farm folks (the tooling is not available in Fedora yet).
Koji is able to build some images using Kickstarts. This one looks like low-hanging fruit. Existing and working solution, we could built on top of of the maintained fedora-kickstarts files that we "use" anyway (through the pre-built official images). But I'm not sure if our team could be allowed to use Koji like this, and provide additional *ks files?
We could make that work, it would need setup... (a repo for kickstarts, permissions and tags setup, etc), but it could work.
Are there any other possibilities? Opinions?
A related thought that occurs to me: Right now for koji builders we install them via virt-install and then run ansible to configure them. Since we only reinstall or upgrade them once a cycle it's not too much hassle, but I wonder if it wouldn't make sense to collaborate with you and come up with a base 'builder image' qcow or raw image and both copr and fedora builders just use the same image. It might be that the use cases diverge enough to not make that practical, but thought I would toss it out there... ;)
kevin
infrastructure@lists.fedoraproject.org