On 5/8/23 19:09, Neal Gompa wrote:
> On Mon, May 8, 2023 at 7:05 PM Demi Marie Obenour <demiobenour@gmail.com> wrote:
>>
>> On 5/8/23 18:49, Kevin Fenzi wrote:
>>> On Mon, May 08, 2023 at 09:29:02PM +0100, Sebastian Crane wrote:
>>>> Dear Kevin,
>>>>
>>>>>> Hmm, quoting from https://pagure.io/releng/issue/11092:
>>>>>>>> Also the aarch64 cluster is running on Fedora 33 boxes, so we
>>>>>>>> should probably try to do a full redeploy :-(
>>>>>>> We can't upgrade it from f33 because docker is no longer in f34+ and
>>>>>>> openshift origin / 3.11 doesn't support any newer either.
>>>>>>
>>>>>> Is this still true? I don't think we want to make the Fedora release
>>>>>> process contingent on something that requires F33.
>>>>>
>>>>> yes, it's still true. Note thats the aarch64 osbs cluster.
>>>>> The x86_64 one is rhel7.
>>>>
>>>> Might it be possible to replace Docker with CRI-O on the OpenShift
>>>> cluster?
>>>
>>> Nope. openshift 3 / osbs1 uses docker. :(
>>>
>>> kevin
>>
>> alias docker=podman
>
> Using Podman with it through Podman's implementation of the Docker
> Host protocol *may* work, but a version of Podman that has it is not
> currently available for RHEL 7 (which is what OpenShift 3.11 runs on).
Time for an OpenShift upgrade?
I realize people are trying to help with suggestions, but the problem is that this is not a simple solution or it would have been done years ago.
The Fedora Build System is a very complicated set of applications all tied together with custom scripting, schema, and various updates over time. There are somewhere around 40 core services and 30 more subsidiary ones which need to be continually working to allow for the daily builds for multiple releases. Many of those services have been added in at different times with different operating systems and rules. In order to work with each other a lot of 'glue' scripting, libraries and such are needed.
You upgrade podman/docker in one section and you find that the message bus isn't sending in another because an API call isn't there anymore. You try putting in something which says it has a koji plugin and you find it doesn't work with our koji and then you need to extend the plugin 40 ways to work with bodhi, pdc, fedmsg and fedora-messaging, and a dozen other things which all are the actual build system. Once those get fixed, you find that it is now causing other glue parts to fail in odd ways no one remembers why without some archaeological coding.
When dealing with the Build system side of things there is currently 1 senior system administrator and 1 senior release engineer. There are some volunteers who can work on subparts but mainly have day jobs doing other things. Looking at IRC, most of the 10-12 hour work day is dealing with compose problems, build issues, hardware failures, and similar things. Most of the 40 component subsystems like the Open Shift Build System (OSBS) are brought in by a dedicated team who have a budget time to get it working and into place. Once that time is done, any fixes are on the Fedora staff or working with available time from whatever team.
tl;dr: The Fedora Build System is in total a very complicated set of tools where changing or upgrading any one part tends to cascade to fixing lots of glue throughout the system. Instead of looking to upgrade things to add more deliverables, it may be better to look at other ways to get those deliverables built.