koji hubs and builders updated

Just thought I would share that (finally) we have completed upgrades on the koji hubs and builders. The hubs have been upgraded to Fedora 35 and the latest koji version (1.17.1). Builder hypervisors have been upgraded to RHEL8.5 and the latest advanced virt stack. Builders have been upgraded/reimaged to Fedora 35 and the latest packages. On armv7 builders: As you all know due to https://bugzilla.redhat.com/show_bug.cgi?id=1920183 ( Overeager OOM kills on 5.10 kernels on 32bit arm with lpae ) we had been running an old kernel on them, but after some testing in staging it seemed that the upcoming 5.16 kernel worked pretty well. So, All of them are running that now. There have been some OOM kills already, I'm playing around with options on them to try and get that number to 0. Hopefully they will be ok for the mass rebuild. On s390x builders: We got some more resources, so the kvm builders now have more memory and cpus. I've asked for some more disk space and once we have that I plan to spread those resources out to have 2x as many builders (ie, now: 4 cpus, 20gb mem x 10, after: 2 cpus, 10gb mem x 20) I don't know if this will be done in time for the mass rebuild, but hopefully so. On ppc64le: We have some more power9 hardware waiting to be put in service, so we should be adding some builders here, and/or spreading out resources. Thanks for everyone's patience with builds that have been taking a while or restarting. Do continue to report them and we will do the best we can to fix things. kevin _______________________________________________ devel-announce mailing list -- devel-announce(a)lists.fedoraproject.org To unsubscribe send an email to devel-announce-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedora... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure