Meeting Agenda Item: Introduction Orion Poplawski
by Orion Poplawski
IRC handle - orionp
I have been a professional unix/linux sysadmin for over 20 years. I use
ansible extensively in my job to configure a variety of systems. I have
broad experience in many scripting and programming languages, though I
would not consider myself an expert except for a few. I have just a bit
of web programming experience including Django, but it is not my strong
point.
The main motivation for my wanting to join the infra team is to help
with repository branching and creating, but hopefully I can find the
time to help out in other areas.
--
Orion Poplawski
Manager of NWRA Technical Systems 720-772-5637
NWRA, Boulder/CoRA Office FAX: 303-415-9702
3380 Mitchell Lane orion(a)nwra.com
Boulder, CO 80301 https://www.nwra.com/
4 years, 6 months
CPE Team Weekly Update: 2019-10-04
by Aoife Moloney
Hi Everyone,
Welcome to the CPE team weekly project update mail!
Background: The Community Platform Engineering group is the Red Hat team
combining IT and release engineering from Fedora and CentOS. Our goal is to
keep core servers and services running and maintained, build releases, and
other strategic tasks that need more dedicated time than volunteers can
give.
For better communication, we are giving weekly reports to the CentOS and
Fedora communities about the general tasks and work being done.
Fedora:
Rawhide Gating: <https://github.com/fedora-infra/bodhi/projects/3>
-
Moving towards testing in staging with a beta release being built to
deploy in staging.
-
Robosignatory using fedora-messaging deployed in staging.
-
There is an issue on documentation here that is also being addressed
-
Rawhide compose now fixed & some broken kde deps rebuilt
-
Work on staging environment set to start this week
-
This broke when we branched F31 in production & ticket tracked in:
https://pagure.io/releng/issue/8838
-
Koji storage issue was fixed and closed
-
Overview page of the remaining blockers and dependencies organized at:
-
https://hackmd.io/Gbuu9JOPR--Y2yNCBEYI5A?view
-
Input welcome for anything that would be missing
-
https://github.com/fedora-infra/bodhi/projects/3
-
High priority items in the “Ready” column are all hard-dependency
for pushing multi-builds to production
repoSpanner <https://github.com/repoSpanner/repoSpanner>
-
(Another!) Race condition discovered has been fixed and tested
-
Test suite is more stable, but is still an ongoing work for full
reliability
-
Performance testing is underway this week
Application Handover to Community
-
Badges: Thread has been started to get a decision with new maintainers
-
Packagedb-cli: Retired this week
-
Pastebin: Ongoing conversations with future maintainer, expecting an
update in the next 2 weeks
-
Elections: Move to CommuniShift underway with two blockers to fix before
complete.
-
Fedocal: If no maintainer is found by October 18th, it will be
decommissioned.
-
Please reach out to any of the CPE team if you want to take over this
project.
-
Nuancier: Identified a maintainer and the team are engaging in a
conversation
-
Whatcanidoforfedora.org: Moved to Communishift
-
DNS needs to be directed and a new owner is needed for it so this is
open for volunteers
-
Documentation for onboarding contributors to Community OpenShift was
started with a good mail thread on Fedora Devel
Misc highlights from various parts of the ecosystem:
-
FPDC: No Update
-
Fedora Container base image released.
-
Merged Fedora image PR on Dockerhub
https://github.com/docker-library/official-images/pull/6702,
-
Rawhide compose failures, due to podman gating, filed
https://github.com/fedora-infra/bodhi/issues/3512 are still being worked
on
-
F31 Final freeze will start next Tuesday, 8th October
-
Fedora 26 & 27 packages were archived in koji.
-
A discussion to define what Minimum Viable Fedora
<https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedora...>
looks like was started.
-
Please get involved! :)
-
Bug in Koji plugin was resolved so sending signing messages has resumed.
https://pagure.io/fedora-infrastructure/issue/8158
-
cpe/infra-docs repo is being removed from pagure and we are now going to
use infra-docs-pagure <https://pagure.io/infra-docs> instead. Working
with an active contributor in the repo to make sure there are no problems.
CentOS:
-
The team is working with CERN about koji upgrade process for
cbs.centos.org
-
CentOS CI
-
SSL Authentication issue with Fedora Messaging pluginI now solved.
-
Shared library being developed
-
8.0.1905 docs now published on docs.centos.org.
Comments & feedback are always welcome & have a great weekend!
Aoife
--
Aoife Moloney
Feature Driver
Community Platform Engineering Team
Red Hat EMEA <https://www.redhat.com>
Communications House
Cork Road
Waterford
<https://www.redhat.com>
4 years, 6 months
Meeting Agenda Item: Introduction Stephen Coady
by Stephen Coady
Hi everyone.
My name is Stephen Coady and I have just joined the community platform
engineering team.
I have been using Fedora for about 2 years now on and off. I am looking
forward to contributing to the platform in any way I can. I am familiar
with Javascript, GraphQL and Docker and I have done quite a bit of hybrid
mobile application development. I used Python a few years ago and I am
looking forward to getting back into it.
My irc nick is scoady. Looking forward to chatting to you all soon.
Stephen
--
Stephen Coady
Software Engineer
Red Hat <https://www.redhat.com/>
<https://red.ht/sig>
4 years, 6 months
Starting to address backlog
by Kevin Fenzi
Greetings.
A few weeks ago I went and tagged a bunch of old releng and
infrastructure tickets with the 'backlog' tag. Clement added some more
the other day.
In our last meeting we did some simple voting on the list to determine
priority.
The idea is that we will take the top 1-3 per week and get small groups
of people to work on them (at the very least one person can do the work
and explain it to another to document/provide feedback on). Then at each
meeting we look at the list, confirm what we did and take some more.
Of course we have other priorities to deal with too, but focusing on a
few tasks and trying to spread knowledge around them will hopefully get
our backlog down over time.
As a side note we are trying also with this copying these to an internal
jira instance to see if we can track backlog flow better and figure out
workflows, but this is purely a copy and pagure instances are where all
the work and comments are done, so anyone not on the CPE team can ignore
this for now. :)
So, for this week we decided to work on:
8178 provision new aarch64 builders xxxxxx
I have this one, it takes access to get them setup, but I am happy to
explain on IRC what I am doing and how the setup works. Assistance could
be used to add them to ansible, as well as documentation if there's
anything special about them. I intend to work on this tomorrow morning.
I'll ping everyone in #fedora-admin on this who is interested.
8157 ansible: enable ansible-report as a hook xxxxxx
I'd love someone else to take this one. Any takers?
I can provide pointers...
8065 Move older koji builds to archive volumes xxxxxx
This is already in progress and has been for some time. I'd love to talk
to others and explain how it's being done and get some documentation
written up on it and others to know how to do it if I am not around.
We could also use some discussion about how to split the koji volume a
bit more. I might be able to work on this wed morning? Anyone interested
in helping?
Thanks and hopefully we can start cranking out some of this backlog
now...
kevin
4 years, 6 months
2020 Datacenter Move: Request for comments
by Kevin Fenzi
Greetings,
Fedora Infrastructure currently has the majority of its hardware in a
datacenter in Arizona, USA. Red Hat leases this space for use by a number of
teams, including Fedora. However, they've been seeking a more modern and cost
effective location for some time and have decided on one:
So, we will be migrating to a new datacenter located in Ashburn, Virginia
in 2020.
FESCo has approved a 2 week window for the actual move to take place
( https://pagure.io/fesco/issue/2221 ): 2020-06-01 to 2020-06-15.
This window is after Fedora 32 is released, but before any major
Fedora 33 Milestones.
At a high level, our current plan is:
* Setup the new datacenter with networking/storage/management
* Populate the new datacenter with new hardware to replace old hardware that
either wouldn’t survive the shipping or is due to be refreshed
* Ship some small shipment of hardware from the old datacenter to the new
that are not easily duplicated like signing hardware,
alternative arch builders, etc.
* Setup and have by the early part of the outage window a
Minimum Viable Fedora Infrastructure (see below) using new hardware
and some old.
* Function in this minimal state as all the rest of the hardware is
shipped to the new datacenter.
* Re-add hardware to return to normal state.
We want to maintain continuity of service as best we can,
so we have defined a Minimal Viable Fedora which will move in advance
of the main hardware. Our intention is to reroute traffic to this setup
before moving the bulk of our hardware.
Our current list of what a Minimum Viable Fedora Infrastructure is:
* Mirroring fully functional. Users get metalinks, mirrors are crawled, etc
* The complete package lifecycle must work.
From commit to update installed on users machines.
We need this to push security and important bugfixes as well as to allow
maintainers to work toward Fedora 33.
* Our production openshift cluster must be up and running normally.
(This cluster has fas, bodhi and other important items in it)
* Builders will likely be constrained.
Ie, less of most arches.
Capacity will be re-added as soon as the hardware for it arrives.
* Rawhide composes take place as normal.
* Nameservers functional
* rabbitmq/fedora-messaging should be up and functional.
* Internal proxies must be functional (used by builders and other internal items)
* Mailing lists must be functional
* Backups must be functional
* OpenQA must be available to test updates/rawhide composes
* Wiki must be available for common bugs / qa
Other services not listed may or may not be up depending on capacity
and issues with more important services.
And explicitly some things will NOT be available during that window:
* Staging. There will be no staging, so no rolling out new services.
* Full capacity/number of builders
* External proxies in the new datacenter
* HA for some services.
We are sending this announcement not only to let you all be aware of this move,
but to help us plan. If you see some service that you think is critical
to Fedora and cannot be down for 2 weeks, and isn't listed above
please let us know so we can adjust our plans.
We want to make sure things that are critical keep running
smoothly for the Fedora community.
Feedback by next friday (2019-10-04) would be welcome.
Thanks,
Kevin for CPE and the Fedora Infrastructure team.
4 years, 6 months
