CPE Weekly Update - Week of October 04th 2021
by Akashdeep Dhar
Hi everyone,
This is a weekly report from the CPE (Community Platform Engineering) Team.
If you have any questions or feedback, please respond to this report or
contact us on `#redhat-cpe` channel on libera.chat.
- If you wish to read this in rendered markdown, check the post on
discussion link
https://discussion.fedoraproject.org/t/cpe-weekly-update-week-of-october-...
.
- *As October is the new quarter, we are about to overtake new projects,
there are no new updates on non-rolling initiatives[0].*
- *CentOS Stream team is continuing with their work:*
- The work on the Mirror Manager has been completed
- The work on the Automated Signing has been completed
- Automatic Signing was broken on Monday night
- Working on batch resigning packages still
- Completed our October planning - aiming to work on
- Content Resolver Docs
- Compose Reporting - content changes between RHEL9/Stream 9
- Forming a plan of record for the work needed to align Stream
8 and Stream 9 workflows in the future
- Technical debt movement - Jenkins updates, old image
cleanups, etc.
- *Infra and releng continues to take care of day to day business
as initiatives team members are working with them to handover SOPs and
maintenance tasks:*
- *Fedora Infra*
- 23 issue tickets were closed this week
- Unfroze after beta and then froze again for F35 final
- Merged a ton of pull requests that were pending
- We have a possible fix to the sssd auth issues, many thanks
sgallagh!
- Updated all the proxies, including for the latest httpd CVE
- Tracked down openQA worker instability to a kernel bug
- We now have backups locally on netapp for Weblate content
- *CentOS Infra*
- Collaboration with Artwork SIG for new CentOS Stream 9 theme
that will be pushed at release day. Preview at
https://www.dev.centos.org and https://lists.dev.centos.org
- CentOS Plus repo being worked on as a SIG (same process) through
Core SIG (new GPG public key) and so building for Stream 8
(and eventually
later Stream 9) on cbs.centos.org
- Business as usual:
- Sponsors leaving (decommissioning nodes in infra)
- Relocate some services
- Progress on the blocker for Stream 9 tests in CI
- *Release Engineering*
- Staging Bodhi deployment 5.7.1
- F35 final freeze
- Removal of bot form `#releng` channel will happen after the
freeze
[0] With the rolling initiative, we mean the initiatives that will take
longer than one quarter and they are not affected by the quarter cycle.
That’s all of this week :)
Kindest regards & on behalf of the CPE team,
Thanks and regards,
Akashdeep Dhar
t0xic0der(a)fedoraproject.org
akashdeep(a)redhat.com
2 years, 6 months
Retroactive freeze break: httpd update
by Kevin Fenzi
I've updated all our fedora 34 hosts to httpd-2.4.50-1.fc34
in order to protect us from CVE-2021-41773.
Since this was a security issue, I just went ahead and did it.
kevin
2 years, 6 months
CPE Weekly Update - Week of Sept 27th
by Michal Konecny
Hi everyone,
This is a weekly report from the CPE (Community Platform Engineering)
Team. If you have any questions or feedback, please respond to this
report or contact us on #redhat-cpe channel on libera.chat
(https://libera.chat/).
There was a hiatus for some time, expect this weekly from now on!
If you wish to read this in rendered markdown, check the post on
discussion.fedoraproject.org:
https://discussion.fedoraproject.org/t/cpe-weekly-update-week-of-sept-27t...
# Highlights of the week
## Infrastructure & Release Engineering
Goal of this Initiative
-----------------------
Purpose of this team is to take care of day to day business regarding
CentOS and Fedora Infrastructure and Fedora release engineering work.
It’s responsible for services running in Fedora and CentOS
infrastructure and preparing things for the new Fedora release
(mirrors, mass branching, new namespaces etc.). The ARC (which is a
subset of the team) investigates possible initiatives that CPE might
take on.
Update
------
### Hardware Updates
* Warranty renewal season for both Fedora and CentOS Infra, which
means we are looking at the hardware that needs attention.
### Fedora Infra
* Infra and Releng team is working with other initiatives team members
to handover maintenance tasks
* Started moving forward again on updating our dnssec keys
* Issues with openqa-x86-worker04 machine ongoing
* Planning a mass update/reboot cycle next week tentatively
* 66 issues open (many to close as the freeze lifted)
### CentOS Infra
*https://sigs.centos.org announced on the centos-devel list
* Updated ci.centos.org Jenkins to comply with security compliance
* Collaboration with Stream team for infra tasks (including for beaker/ftp)
* Business as usual
* New tags in cbs/koji (automotive, cloud)
* New projects on git.centos.org
### Release Engineering
* Fedora 35 Beta out the door! Freeze ended
## CentOS Stream
Goal of this Initiative
-----------------------
This initiative is working on CentOS Stream/Emerging RHEL to make this
new distribution a reality. The goal of this initiative is to prepare
the ecosystem for the new CentOS Stream.
Updates
-------
* Working on Content Resolver buildroot logic integration
* Metalinks for CentOS Stream 9 mirrors works now
* An updated centos-release package exercising the new mirror infra is
built, we expect it to be composed in the next couple of days
* Starting some investigations to help maintainers work with GitLab
* Reworking some of the protocols that we announce internally for our
composes (Beaker needs to pull the composes via FTP or tftp)
## Datanommer/Datagrepper
Goal of this Initiative
-----------------------
These apps are currently used to retrieve historical information about
messages on the fedmsg bus and add them to a Postgres database.
Datanommer reads-in messages from the bus and stores them in the
database and Datagrepper exposes the messages in the database via an
API with different filtering capacities. We want to upgrade these
applications to use fedora-messaging and increase the performance of
the applications for users.
Updates
-------
* Import script is still running, the ETA of ~2 months seems to be confirmed
* We’re doing the finishing touches to code
## Metrics for Apps on OpenShift
Goal of this Initiative
-----------------------
The project team will deliver on the installation of newest OpenShift
in Fedora infra to be then configured with prometheus so applications
can be hooked into this service and can be monitored and metricized
through this tech stack for more comprehensive understanding of app
behaviour, performance and troubleshooting.
Updates
-------
* A lot of work in docs - SOP docs merged
https://pagure.io/infra-docs-fpo/pull-request/8
* We’ve enabled the user-workload-monitoring stack on the staging
cluster, and began documenting it
* We’ve disabled the default permissions that users get when they
login on the staging cluster and started documenting that too
* A few things are not quite right like network interfaces, but we
have the basic production cluster up and running
* Working to get the web console working, but in the meantime can use
the CLI on the os-control01 node:
https://console-openshift-console.apps.ocp.fedoraproject.org/
## DNF Counting
Goal of this Initiative
-----------------------
The DNF Counting project will enhance the currently existing program
that captures a ‘countme’ value from computers running Fedora weekly.
There are multiple scripts running from a server which parses the data
into a csv file for graphing and displaying trends that had been
misbehaving a lot. This program gives a good indication for how many
computers use Fedora so the project team will work on improving these
scripts and the program overall to make it a more reliable and
maintainable solution.
Updates
-------
* Finishing off last tests
* More deployment cleanup
* Investigate why our cronjobs are so noisy (ongoing)
* Ongoing work on documentation:
https://docs.fedoraproject.org/en-US/infra/sysadmin_guide/dnf-counting/
Kindest regards & on behalf of the CPE team,
Michal
2 years, 6 months
Fedora 35 Final freeze now in effect!
by Kevin Fenzi
Greetings.
we are now in the infrastructure freeze leading up to the Fedora 35
Final release. This is a final release freeze.
We do this to ensure that our infrastructure is stable and ready to
release Fedora 35 when it's available.
You can see a list of hosts that do not freeze by checking out the
ansible repo and running the freezelist script:
git clone
https://infrastructure.fedoraproject.org/infra/ansible.git
ansible/scripts/freezelist -i inventory
Any hosts listed as freezes is frozen until 2021-10-19 (or later if
release slips). Frozen hosts should have no changes made to them without
a sign-off on the change from at least 2 sysadmin-main or rel-eng
members, along with (in most cases) a patch of the exact change to be
made to this list.
Thanks,
kevin
2 years, 6 months