Planned Outage - Updates / Reboots - 2022-05-18 20:00 UTC
by Mark O'Brien
Hi All,
There will be an outage starting at 2022-05-18 20:00 UTC
which will last approximately 4 hours.
To convert UTC to your local time, take a look at
http://fedoraproject.org/wiki/Infrastructure/UTCHowto
or run:
date -d '2022-05-18 20:00 UTC'
Reason for outage:
We will be updating all our instances and rebooting hardware nodes into new
kernels. This will catch us up on security updates after the finishing of
the Fedora 36 final freeze.
Affected Services:
All services may experience slowdowns or short outages as servers are
rebooted and return to service. We will try and minimize disruptions, but
users/maintainers are advised to avoid activity in the outage window.
Ticket Link:
https://pagure.io/fedora-infrastructure/issue/10692
Please join #fedora-admin or #fedora-noc on irc.libera.chat
or add comments to the ticket for this outage above.
Mark
2 days, 13 hours
CPE Weekly Update – Week 19 2022
by Michal Konecny
Hi everyone,
This is a weekly report from the CPE (Community Platform Engineering)
Team. If you have any questions or feedback, please respond to this
report or contact us on #redhat-cpe channel on libera.chat
(https://libera.chat/).
Week: 09th May - 13th May 2022
If you wish to read this in form of a blog post, check the post on
Fedora community blog:
https://communityblog.fedoraproject.org/cpe-weekly-update---week-19-2022/
# Highlights of the week
## Infrastructure & Release Engineering
Goal of this Initiative
-----------------------
Purpose of this team is to take care of day to day business regarding
CentOS and Fedora Infrastructure and Fedora release engineering work.
It’s responsible for services running in Fedora and CentOS
infrastructure and preparing things for the new Fedora release (mirrors,
mass branching, new namespaces etc.).
The ARC (which is a subset of the team) investigates possible
initiatives that CPE might take on.
Link to planning board: https://zlopez.fedorapeople.org/I&R-2022-05-11.pdf
Update
------
### Fedora Infra
* F34/F35 container builds failing due to 32bit arm (
https://bugzilla.redhat.com/show_bug.cgi?id=2077680 )
* git -core change broke koji. Downgraded git and upstream koji already
has a fix.
* Got a FMW macos build fully signed and notarized! Unfortunately, now
need to find out how to build it to be able to run on older macos. ;(
* Fedora 36 release went pretty smoothly, we are now out of Freeze
* Business as usual, misc tickets, etc.
### CentOS Infra including CentOS CI
* CentOS Stream storage migration spike (Netapp for nfs/iscsi)
* Duffy fixes and tests
* Investigating hardware issue on CI pool
* Investigating ci.centos.org decommission steps
* Git.centos.org pagure upgrade/migration (blocked, waiting on internal
Red Hat Team)
* Updated sshd host key signing (sha1 issue for el9 ssh clients)
* Bussiness as usual (mirrors, tags)
### Release Engineering
* F36 is out
* Firmware win binaries signed
* Bussiness as usual - stalled epel packages, package unretirements
## CentOS Stream
Goal of this Initiative
-----------------------
This initiative is working on CentOS Stream/Emerging RHEL to make this
new distribution a reality. The goal of this initiative is to prepare
the ecosystem for the new CentOS Stream.
Updates
-------
* Finished the RPM import for c8s to Stream Koji
* Business as usual otherwise
## CentOS Duffy CI
Goal of this Initiative
-----------------------
Duffy is a system within CentOS CI Infra which allows tenants to
provision and access bare metal resources of multiple architectures for
the purposes of CI testing.
We need to add the ability to checkout VMs in CentOS CI in Duffy. We
have OpenNebula hypervisor available, and have started developing
playbooks which can be used to create VMs using the OpenNebula API, but
due to the current state of how Duffy is deployed, we are blocked with
new dev work to add the VM checkout functionality.
Updates
-------
* More deployment tests
* Per tenant session lifetimes
* Some bug fixes
## Package Automation (Packit Service)
Goal of this initiative
-----------------------
Automate RPM packaging of infra apps/packages
Updates
-------
* The team is hitting lots of dependency and sub dependency issues,
working through them but its slow
* fasjson-client is our first package to be fully automated
* upstream release -> src.fp.o PR -> koji -> bodhi
* Thanks to Nils, Aurelien and Kevin for their help/advice
* fedora-messaging, datagrepper, fasjson currently being worked on (all
have deps issues)
* spec files will be staying downstream, packit has a way to facilitate this
## Flask-oidc: oauth2client replacement
Goal of this initiative
-----------------------
Flask-oidc is a library used across the Fedora infrastructure and is the
client for ipsilon for its authentication. flask-oidc uses oauth2client.
This library is now deprecated and no longer maintained. This will need
to be replaced with authlib.
Updates:
--------
* Setup dev environment (Work In Progress!)
* Starting to implement flask-oidc api using authlib.
## EPEL
Goal of this initiative
-----------------------
Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special
Interest Group that creates, maintains, and manages a high quality set
of additional packages for Enterprise Linux, including, but not limited
to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL),
Oracle Linux (OL).
EPEL packages are usually based on their Fedora counterparts and will
never conflict with or replace packages in the base Enterprise Linux
distributions. EPEL uses much of the same infrastructure as Fedora,
including buildsystem, bugzilla instance, updates manager, mirror
manager and more.
Updates
-------
* epel9 up to 2568 source packages (increase of 113 from last week).
* Added rhel+epel-9 mock configs to mock-core-configs.
* Updated slurm in epel7 and epel8 to fix CVE-2022-29500 and CVE-2022-29501.
* Retired swtpm and libtpms from epel8 because they were added to RHEL8.6.
* Added python-texttable to epel9 to allow c8s maintainers (Johnny) to
run c9s as a workstation.
* Added missing devel packages for cogl, clutter, and clutter-gtk to
epel9 to unblock other epel9 requests.
Kindest regards,
CPE Team
5 days, 18 hours
attention: resultsdb changes
by Kevin Fenzi
Greetings everyone.
For a while now we have been wanting to make some changes to resultsdb
and these changes are now going to happen soon!
Basically we are moving it into openshift, adding password protection
for posting to it (before it was just a simple network check) and some
other minor changes like adding some new states. We also want to move it
to it's own url instead of leaving it under the taskotron virtual host
(since taskotron is no more. RIP).
We already have the new versions all setup in staging (along with the
old one still running).
old url: https://taskotron.stg.fedoraproject.org/resultsdb
new url: https://resultsdb.stg.fedoraproject.org/
So, what does this mean?
As far as I am aware, only 2 things post results to resultsdb.
1. resultsdb-ci-listener (listens for ci fedora-messages and posts
results). We control this application and it will be moving into
openshift as well.
2. openqa. Which AdamW can adjust when we move prod, but we have tested
and stg can post fine.
Many other things query/read/display links to results and will need to
be updated after we switch prod:
bodhi
pagure/distgit
greenwave
monitor_gating
waiverdb
If app owners could ready PR's for these changes that would be great,
otherwise we can make them before switching.
So, did we forget anything? Any issues or problems anyone can see?
We would ideally like to move into prod monday, but can delay if there's
issues or concerns.
Thanks!
kevin
6 days, 5 hours
About JS framework
by Pierre-Yves Chibon
Good Morning Everyone,
Our infrastructure is mostly a python store, meaning almost all our apps are
written in python and most using wsgi.
However in python we are using a number of framework:
* flask for most
* pyramid for some of the biggest (bodhi, FAS3)
* Django (askbot, Hyperkitty)
* TurboGears2 (fedora-packages)
* aiohttp (python3, async app: mdapi)
While this makes sometime things difficult, these are fairly standard framework
and most of our developers are able to help on all.
However, as I see us starting to look at JS for some of our apps (fedora-hubs,
wartaa...), I wonder if we could start the discussion early about the different
framework and eventually see if we can unify around one.
This would also allow those of us not familiar with any JS framework to look at
the recommended one instead of picking one up semi-randomly.
So has anyone experience with one or more JS framework? Do you have one that
would you recommend? Why?
Thanks for your inputs,
Pierre
1 week
CPE Weekly Update – Week 18 2022
by Lenka Segura
Hi everyone,
This is a weekly report from the CPE (Community Platform Engineering) Team.
If you have any questions or feedback, please respond to this report or
contact us on #redhat-cpe channel on libera.chat (https://libera.chat/).
Week: 2nd - 6th May 2022
If you wish to read this in form of a blog post, check the post on Fedora
community blog:
https://communityblog.fedoraproject.org/cpe-weekly-update-week-18-2022/
<https://communityblog.fedoraproject.org/?p=10998>
# Highlights of the week
## Infrastructure & Release Engineering
Goal of this Initiative
-----------------------
Purpose of this team is to take care of day to day business regarding
CentOS and Fedora Infrastructure and Fedora release engineering work.
It’s responsible for services running in Fedora and CentOS infrastructure
and preparing things for the new Fedora release (mirrors, mass branching,
new namespaces etc.).
The ARC (which is a subset of the team) investigates possible initiatives
that CPE might take on.
Link to planning board: (https://zlopez.fedorapeople.org/I&R-2022-05-04.pdf)
Update
------
### Fedora Infra
* Resultsdb now entirely working in stg ocp4 (Many thanks Leo!)
* leaned up pagure.io ssl cert issues
* Fixed an issue on ipsilon02 that was preventing bugzilla.redhat.com logins
* Business as usual tickets (lists, groups, etc)
### CentOS Infra including CentOS CI
* Duffy tests for CI infra (no prod deployment scheduled yet)(
https://pagure.io/centos-infra/issue/712)
* Pagure on rhel8 upgrade tests (https://git.dev.centos.org is now up with
* pagure 5.13 on rhel8)(https://pagure.io/centos-infra/issue/456)
* BAU (new koji/cbs tags, mirrors proposal)
### Release Engineering
* F36 RC-1.4 is out
* 1 proposed blocker, GO/NOGO is tomorrow
## CentOS Stream
Goal of this Initiative
-----------------------
This initiative is working on CentOS Stream/Emerging RHEL to make this new
distribution a reality. The goal of this initiative is to prepare the
ecosystem for the new CentOS Stream.
Updates
-------
* May planning meeting happening now - outlining goals to work towards for
the month
* Active discussion happening on RHEL and CentOS Stream module
synchronization. Planning the branch names, stream names and DistroBaker
rules.
## CentOS Duffy CI
Goal of this Initiative
-----------------------
Duffy is a system within CentOS CI Infra which allows tenants to provision
and access bare metal resources of multiple architectures for the purposes
of CI testing.
We need to add the ability to checkout VMs in CentOS CI in Duffy. We have
OpenNebula hypervisor available, and have started developing playbooks
which can be used to create VMs using the OpenNebula API, but due to the
current state of how Duffy is deployed, we are blocked with new dev work to
add the VM checkout functionality.
Updates
-------
* More deployment testing
* Reverse lookup IPs to get hostnames when provisioning
* Legacy API: map parameter combinations to pools in configuration rather
than hard-coded
* Node quotas (ongoing)
## Package Automation (Packit Service)
Goal of this initiative
-----------------------
Automate RPM packaging of infra apps/packages
Updates
-------
* Business as usual, working through errors with packit configs
* Ready to try our first full release this week (hopefully) resulting in
koji builds and bodhi updates
## Flask-oidc: oauth2client replacement
Goal of this initiative
-----------------------
Flask-oidc is a library used across the Fedora infrastructure and is the
client for ipsilon for its authentication. flask-oidc uses oauth2client.
This library is now deprecated and no longer maintained. This will need to
be replaced with authlib.
Updates:
--------
* Investigating where oauth2client appears in the code, possible
replacement functions in authlib (
https://github.com/fedora-infra/flask-oidc/issues/5)
* Met with Aurelien this afternoon to gain some of his knowledge
## EPEL
Goal of this initiative
-----------------------
Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest
Group that creates, maintains, and manages a high quality set of additional
packages for Enterprise Linux, including, but not limited to, Red Hat
Enterprise Linux (RHEL), CentOS and Scientific Linux (SL), Oracle Linux
(OL).
EPEL packages are usually based on their Fedora counterparts and will never
conflict with or replace packages in the base Enterprise Linux
distributions. EPEL uses much of the same infrastructure as Fedora,
including buildsystem, bugzilla instance, updates manager, mirror manager
and more.
Updates
-------
* epel9 up to 2455 source packages (increase of 39 from last week).
* Qt5 rebuild issue from last week has been resolved in epel9-next, but
epel8-next rebuilds are blocked by a CentOS Stream 8 module bug.
* Partially unblocked azure-cli addition to epel9 by adding python-jwt.
* Bootstrapped the “testing-cabal” suite in epel9 (python-extras,
python-fixtures, python-testresources, python-testscenarios, and
python-testtools) (update:
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-c4b352d61a). This
potentially unblocks many new epel9 packages, notably several Openstack
client tools and libraries.
* ImageMagick incompatible upgrade and related rebuilt packages are
available in epel8-testing. (Fixes 81 bugs, 69 of them CVE bugs)(
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-62b1a9e158)
Kindest regards,
CPE Team
1 week, 5 days
