infrastructure March 2022

infrastructure@lists.fedoraproject.org

18 participants
19 discussions

Bodhi client calls in the infra
by Aurelien Bompard 17 Mar '22

17 Mar '22

Hey folks! We are preparing for the deployment of the next major release of Bodhi (planned for a Friday evening on an Irish bank holiday during freeze in the Thanksgiving extended weekend), and the authentication has changed, which means automated calls of the bodhi client ("bodhi" command line) must be changed. I didn't see any such call in our ansible repo, but your apps may be calling the bodhi client themselves. Could you please tell me if you know of any app that does so? Thanks! Aurélien

2 3

[Fedocal] Reminder meeting : Websites & Apps Team Meeting
by t0xic0der＠fedoraproject.org 14 Mar '22

14 Mar '22

Dear all, You are kindly invited to the meeting: Websites & Apps Team Meeting on 2022-03-15 from 15:00:00 to 16:00:00 UTC At https://meet.jit.si/fedora-websites-apps-meeting The meeting will be about: Weekly team meeting for the Websites & Apps Team. This is part of the [Websites & Apps Community Revamp Objective](https://fedoraproject.org/wiki/Objectives/Websites_%26_Apps_Comm…. See [past discussion]( https://discussion.fedoraproject.org/t/planning-meeting-for-websites-apps-t…) to learn more about the Websites & Apps Team and how it came together. More information available at: [hackmd.io/Mxm2We3yTqKybLsdohadOA](https://hackmd.io/Mxm2We3yTqKybLsdohadOA?view) Source: https://calendar.fedoraproject.org//meeting/9990/

1 0

Freeze Break Request
by Mark O'Brien 14 Mar '22

14 Mar '22

Hi All, There are a few of our proxies in AWS that are giving us errors which I believe are related to low memory. I would like to reinstall a few of these. I would do one at a time so disruption should be minimal. There is a few benefits. 1. Increase server resources to allow for a bigger load 2. Stop using instances with ephemeral storage to avoid a need for a reinstall in future 3. Assign Elastic IPs to allow for zero downtime replacement in future Let me know what you think and add +1/-1 as you see fit. Thanks, Mark

3 2

FBR: OCP4 storage configuration
by Mark O'Brien 12 Mar '22

12 Mar '22

Hi All, I am asking for forgiveness here. We did this work earlier today and only realised afterward that a freeze break should have been requested. OCP worker node worker06.ocp.iad2.fedoraproject.org had its installed software raid removed. This node was then added to OCP as a local volume to allow OCP take control of the storage and manage it as part of its local volume set. Apologies, Mark

4 3

Freeze Break request: update mirrors.centos.org cert
by Kevin Fenzi 09 Mar '22

09 Mar '22

We host the mirros.centos.org site on our proxies (redirecting to mirrormanager). It has a letsencrypt cert and it expires on March 22nd. We have a new cert ready now, so I would like to just update it now so it's all in sync. Will require pushing the new cert to the ansible-private repo and running proxies playbook to deploy it. Can I get +1's for this from sysadmin-main/releng folks? kevin

3 2

FBR: Update oz on certain aarch64 builders
by Mohan Boddu 09 Mar '22

09 Mar '22

3 2

[Fedocal] Reminder meeting : Websites & Apps Team Meeting
by t0xic0der＠fedoraproject.org 07 Mar '22

07 Mar '22

Dear all, You are kindly invited to the meeting: Websites & Apps Team Meeting on 2022-03-08 from 15:00:00 to 16:00:00 UTC At https://meet.jit.si/fedora-websites-apps-meeting The meeting will be about: Weekly team meeting for the Websites & Apps Team. This is part of the [Websites & Apps Community Revamp Objective](https://fedoraproject.org/wiki/Objectives/Websites_%26_Apps_Comm…. See [past discussion]( https://discussion.fedoraproject.org/t/planning-meeting-for-websites-apps-t…) to learn more about the Websites & Apps Team and how it came together. More information available at: [hackmd.io/Mxm2We3yTqKybLsdohadOA](https://hackmd.io/Mxm2We3yTqKybLsdohadOA?view) Source: https://calendar.fedoraproject.org//meeting/9990/

1 0

CPE Weekly Update – Week of February 28th – March 3rd
by Michal Konecny 04 Mar '22

04 Mar '22

Hi everyone, This is a weekly report from the CPE (Community Platform Engineering) Team. If you have any questions or feedback, please respond to this report or contact us on #redhat-cpe channel on libera.chat (https://libera.chat/) If you wish to read this in form of a blog post, check the post on Fedora community blog: https://communityblog.fedoraproject.org/cpe-weekly-update-week-of-february-… # Highlights of the week ## Infrastructure & Release Engineering Goal of this Initiative ----------------------- Purpose of this team is to take care of day to day business regarding CentOS and Fedora Infrastructure and Fedora release engineering work. It’s responsible for services running in Fedora and CentOS infrastructure and preparing things for the new Fedora release (mirrors, mass branching, new namespaces etc.). The ARC (which is a subset of the team) investigates possible initiatives that CPE might take on. Update ------ ### Fedora Infra * One new power9 online in iad2, second one needs hands on today. * Almost all bugzilla auth changes made, still need some changes to toddlers (ongoing) * Container builds broken again in an odd way ( https://pagure.io/releng/issue/10658) * Discussion on fedoraplanet.org on infrastructure list, please chime in if you have thoughts about it. ### CentOS Infra including CentOS CI * Migration of Pagure to new CentOS CI in progress * Openssl3 [late change](https://gitlab.com/redhat/centos-stream/rpms/openssl/-/commit/78fb7… in EL9 impacting SIGs and gpg keys ### Release Engineering * Work on SCM request automation in progress - [PR](https://pagure.io/fedora-infra/toddlers/pull-request/93) * Container builds failing on armhfp * Bussiness as usual ## CentOS Stream Goal of this Initiative ----------------------- This initiative is working on CentOS Stream/Emerging RHEL to make this new distribution a reality. The goal of this initiative is to prepare the ecosystem for the new CentOS Stream. Updates ------- * Investigating compose QA improvements both in the t-functional suite and changes we may want to implement later in the project. * Developing sync2gitlab service for 8, 9 workflow alignment * Business as usual activities around CentOS Linux 7 * Work continuing on content resolvers maintainer pages also ## CentOS Duffy CI Goal of this Initiative ----------------------- Duffy is a system within CentOS CI Infra which allows tenants to provision and access bare metal resources of multiple architectures for the purposes of CI testing. We need to add the ability to checkout VMs in CentOS CI in Duffy. We have OpenNebula hypervisor available, and have started developing playbooks which can be used to create VMs using the OpenNebula API, but due to the current state of how Duffy is deployed, we are blocked with new dev work to add the VM checkout functionality. Updates ------- * Demo/Status call * Deployment to staging (ongoing) * Documentation (ongoing) * Expiring sessions (ongoing) ## Image builder for Fedora IoT Goal of this Initiative ----------------------- Integration of Image builder as a service with Fedora infra to allow Fedora IoT migrate their pipeline to Fedora infra. Updates ------- * No updates ## Bodhi Goal of this Initiative ----------------------- This initiative is to separate Bodhi into multiple sub packages, fix integration and unit tests in CI, fix dependency management and automate part of the release process. Read ARC team findings in detail at: https://fedora-arc.readthedocs.io/en/latest/bodhi/index.html Updates ------- * No updates ## EPEL Goal of this initiative ----------------------- Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest Group that creates, maintains, and manages a high quality set of additional packages for Enterprise Linux, including, but not limited to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL), Oracle Linux (OL). EPEL packages are usually based on their Fedora counterparts and will never conflict with or replace packages in the base Enterprise Linux distributions. EPEL uses much of the same infrastructure as Fedora, including buildsystem, bugzilla instance, updates manager, mirror manager and more. Updates ------- * EPEL9 up to 2059 source packages (increase of 71 from last week) * EPEL9 buildroot has switched to a snapshot of the CentOS Stream 9 mirror, to avoid building against any 9.1 changes prior to the RHEL 9.0 GA * [EPEL Office Hours](https://communityblog.fedoraproject.org/epel-office-hours/) were yestarday at 1700 UTC * epel-release (and epel-next-release) now available in CentOS Stream 9 Extras repo Kindest regards, CPE Team

1 0

3 small freeze breaks
by Kevin Fenzi 03 Mar '22

03 Mar '22

Greetings everyone. I have 3 PR's that I would like to get freeze breaks for (+1s from any sysadmin main or releng person). Of course comments from everyone are most welcome too. First: https://pagure.io/fedora-infra/ansible/pull-request/986 proxies: update *.id.fedoraproject.org wildcard to 2022 version This updates the wildcard cert for id.fedoraproject.org. If we don't do this in 4 days it will break. This will require running the proxies playbooks after merge. Second: https://pagure.io/fedora-infra/ansible/pull-request/987 buildvm-s390x kvm builders: increase memory from 10g to 13g Per the devel list some large packages like mariadb are failing to build due to lack of memory. We can easily just bump all the kvm builders up to 13G from 10G without much trouble. I think I can do this dynamically while they are up, but might need a reboot if something goes wrong. Third: https://pagure.io/fedora-infra/ansible/pull-request/988 proxies: open ocp4 api port in both stg and prod This opens port 6443/tcp on our iad2 proxies so we can reach the openshift 4 api port. We decided this would probibly be an ok risk, as you need auth for anything and this port is expected to be exposed, so it should be pretty secure. Note that there is still a self signed cert you would have to accept currently, but otherwise oc should work. Also, this adds some workers we have recently added to the clusters into haproxy. This will need a proxies playbook run after merged. Thanks and look forward to comments, +1s. ;) kevin

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure March 2022