infrastructure July 2020

infrastructure@lists.fedoraproject.org

28 participants
18 discussions

About JS framework
by Pierre-Yves Chibon 17 Oct '22

17 Oct '22

Good Morning Everyone, Our infrastructure is mostly a python store, meaning almost all our apps are written in python and most using wsgi. However in python we are using a number of framework: * flask for most * pyramid for some of the biggest (bodhi, FAS3) * Django (askbot, Hyperkitty) * TurboGears2 (fedora-packages) * aiohttp (python3, async app: mdapi) While this makes sometime things difficult, these are fairly standard framework and most of our developers are able to help on all. However, as I see us starting to look at JS for some of our apps (fedora-hubs, wartaa...), I wonder if we could start the discussion early about the different framework and eventually see if we can unify around one. This would also allow those of us not familiar with any JS framework to look at the recommended one instead of picking one up semi-randomly. So has anyone experience with one or more JS framework? Do you have one that would you recommend? Why? Thanks for your inputs, Pierre

10 12

Another Rust MirrorManager experiment
by Adrian Reber 03 Nov '20

03 Nov '20

Our MirrorManager setup exports the current state of all mirrors every hour at :30 to a protobuf based file which is then used by the mirrorlist servers to answer the requests from yum and dnf. The Python script requires up to 10GB of memory and takes between 35 and 50 minutes. The script does a lot of SQL queries and also some really big SQL queries joining up to 6 large MirrorManager tables. I have rewritten this Python script in Rust and now it only needs around 1 minute instead of 35 to 50 minutes and only 600MB instead of 10GB. I think the biggest difference is that I am almost not doing any joins in my SQL request. I download all the tables once and then I do a lot of loops over the downloaded tables and this seems to be massively faster. As the mirrorlist-server in Rust has proven to be extremely stable over the last months we have been using it I would also like to replace the mirrorlist protbuf input generation with my new Rust based code. I am planing to try out the new protobuf file in staging in the next days and would then try to get my new protobuf generation program into Fedora. Once it is packaged I would discuss here how and if we want to deploy in Fedora's infrastructure. Having the possibility to generate the mirrorlist input data in about a minute would significantly reduce the load on the database server and enable us to react much faster if broken protobuf data has been synced to the mirrorlist servers on the proxies. Adrian

4 17

What is our technical debt?
by Pierre-Yves Chibon 21 Sep '20

21 Sep '20

Good Morning Everyone, Just like every team we have technical debt in our work. I would like your help to try to define what it is for us. So far, I've come up with the following: - python3 support/migration - fedora-messaging - fedora-messaging schema - documentation - (unit-)tests - OpenID Connect What else would we want in there? Looking forward to your thoughts, Pierre

22 55

Tech debt analysis tool
by James Richardson 08 Aug '20

08 Aug '20

Hi All, Vipul and I are looking into several different tools that will allow us to better analyze our tech debt with any new code that is merged into apps in http://github.com/fedora-infra. Currently, we have looked at the tools below, but we would love any and all input from the team and community on this. SonarCloud LGTM ShiftLeft Our goal is to find an open source tool that is easy to integrate as well as providing useful and timely feedback. So far, SonarCloud has proved to be the one that looks best, but again, we are very open to any and all suggestions, and at this early stage, a good conversation to arrive at the best solution. Regards, James -- James Richardson Engineering Intern He | Him | His Red Hat Waterford <https://www.redhat.com/> Communications House Cork Road, Waterford City jamricha(a)redhat.com <lgriffin(a)redhat.com> M: +353851970521 <+353877545162> IM: jamricha @redhatjobs <https://twitter.com/redhatjobs> redhatjobs <https://www.facebook.com/redhatjobs> @redhatjobs <https://instagram.com/redhatjobs> <https://red.ht/sig>

8 9

[Fedocal] Reminder meeting : Fedora Infrastructure
by smooge＠gmail.com 29 Jul '20

29 Jul '20

Dear all, You are kindly invited to the meeting: Fedora Infrastructure on 2020-07-30 from 15:00:00 to 16:00:00 UTC At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Weekly Fedora Infrastructure meeting. See infrastructure list for agenda a day before. Source: https://apps.fedoraproject.org/calendar/meeting/9675/

1 0

s390x for EPEL
by Miroslav Suchý 29 Jul '20

29 Jul '20

What is using Koji to build EPEL builds for s390x. We want to enable s390x in Copr. It is without problem for Fedora, but we cannot do that for EPEL, because there is no CentOS for s390x. Do we have entitlements for RHEL I can use? Or we have some local repo? Something completely different? -- Miroslav Suchy, RHCA Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys

4 7

packages app
by Kevin Fenzi 28 Jul '20

28 Jul '20

So, I think we have two groups approaching a replacement app for packages? Can we perhaps decide which way we want to go and look at deploying something soon? This has been more important since we didn't move the old broken packages app from the old datacenter and people keep asking about it. We could wait until we have staging back, but I don't think we need to wait for that to decide what we want to do here? Thoughts? kevin

3 8

CPE Weekly: 2020-07-25
by Aoife Moloney 26 Jul '20

26 Jul '20

--- title: CPE Weekly status email tags: CPE Weekly, email --- # CPE Weekly: 2020-07-25 Background: The Community Platform Engineering group is the Red Hat team combining IT and release engineering from Fedora and CentOS. Our goal is to keep core servers and services running and maintained, build releases, and other strategic tasks that need more dedicated time than volunteers can give. See our wiki page here for more information:https://docs.fedoraproject.org/en-US/cpe/ ## General Project Updates The CPE team are working on the following projects for Quarter 3, which is the months of July, August & September: * Data Centre Move - Final Works * CentOS Stream Phase 3 * Noggin Phase 3 * Packager Workflow Healthcare * Fedora Messaging Schemas Details of the above projects, and of projects currently in progress, done and what projects are in our backlog, can be found on our taiga board per project card: https://tree.taiga.io/project/amoloney1-cpe-team-projects/kanban?epic=null We also have an updated initiative timetable for briefing in new projects to our team & key dates here:https://docs.fedoraproject.org/en-US/cpe/time_tables/ *Note: Initiatives are large pieces of work that require a team of people and weeks/months to complete. Please continue to open tickets in the normal way for bugs, issues, etc. ### CPE Product Owner Office Hours #### #fedora-meeting-1 * Weekly on Thursdays @ 1300 UTC on #fedora-meeting-1 * Next Meeting: 2020-07-30 #### #centos-meeting * Every second Tuesday @ 1500 UTC on #centos-meeting * Next Meeting: 2020-08-04 ### Misc * CFP for Nest with Fedora submissions are closing on Monday 27th July - dont forget to submit your ideas/talks here https://pagure.io/flock * CPE Engagement email was sent a few weeks ago, please read if you have not already and feel free to send me your feedback on or off list before 31st July. A follow up email will be sent with feedback received and actions taken where it makes sense then afterwards. Thanks for your engagement!https://lists.fedoraproject.org/archives/list/devel-announce@lis… ## Fedora Updates ### Data Centre Move * The team have been working incredibly hard over the last few weeks and are now working on reinstalling the staging environment. * We estimate this work to be completed by July 31st, however things happen so we thank you in advance for your patience * The project card on taiga for builder - staging - remaining app bringup dates here https://tree.taiga.io/project/amoloney1-cpe-team-projects/us/42?kanban-stat… * A list of affected services is available here https://hackmd.io/hpYYJQRjQy-oHxUS7IonIA?view * Details on what this move may mean for you can be found here https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedorapr… * If an application is not working correctly at all, please check this list https://hackmd.io/hpYYJQRjQy-oHxUS7IonIA?view before opening a ticket to make sure its not listed as being moved. If it is being moved, please wait a day or two, then try again. * And finally a reminder to please be patient when opening tickets for service issues in general as we are in the critical point in this move and all of our sys-admins and wider teams are assisting in the successful bringup of the reduced Fedora service and facilitation of the final hardware shipment and move. ### AAA Replacement * The team are focusing work on this ticket and collaborating with the Sustaining team for the next sprint, with the goal to have noggin installed & deployed to staging https://pagure.io/fedora-infrastructure/issue/9152 * The have also been addressing how to curtail spam email and more information on that effort can be found here https://github.com/fedora-infra/noggin/issues/27 * And have added the feature to allow groups with a prefix for groups to import to avoid overlaps between CentOS and Fedora https://github.com/fedora-infra/fas2ipa/issues/5 * Please feel free to check out the team kanban board for more information on the features the team are working on and have already completed here https://github.com/orgs/fedora-infra/projects/6 ### Fedora Messaging Schemas * The team are using a kanban approach for this project as Noggin takes priority until fully complete * The team have already built a list of applications that require messaging schemas, list can be found here https://hackmd.io/@nilsph/H1i8CAbkP/edit * They also have completed a readme which contains documentation on messaging schemas, a cookie-cutter template to create the schema and a definition of Done for writing a schemas https://github.com/fedora-infra/fedora-messaging-schemas-issues * The board they are working from can be viewed here https://github.com/orgs/fedora-infra/projects/7 ### Packager Workflow Healthcare * Project information: This is an investigative project that aims to look at the entire packager workflow as a single piece of tooling to identify where failures happen, and try to identify when and why packages fail at different points within the workflow. We hope to have two possible outcomes from this project at the end of the quarter (September): * The workflow breaks at X point and we will work on a solution to fix * OR * The workflow works fine, but we will need better monitoring on the pipeline so we will work on a solution for this * The team are using this Monitor-Script https://pagure.io/fedora-ci/monitor-gating/ * Current status: failing consistently because of CI test timeouts * The jenkins instance being used is under-resourced, jobs waiting up to 40 minutes for a node * They are using a workable dump of historical datanommer/datagrepper data for queries to pull metrics on the historical/aggregate Packager Workflow from as a baseline * And are working on an outline of the workflow steps (from packager PoV) and systems involved (CPE team PoV), identifying metrics to be measured * The teams work is being tracked here https://teams.fedoraproject.org/project/cpe-cicd/kanban ## CentOS Updates ### CentOS * CentOS-infra project created https://pagure.io/centos-infra/ * Day to Day working with CentOS-infra being documented at: https://pagure.io/cpe/docs/pull-request/10 * New website is live https://www.centos.org ### CentOS Stream * Business as usual, the team are churning through building packages in Stream * They are also working on tooling to bring back some rpkg features As always, feedback is welcome, and we will continue to look at ways to improve the delivery and readability of this weekly report. Have a great week! Aoife Source: https://hackmd.io/8iV7PilARSG68Tqv8CzKOQ?view -- Aoife Moloney Product Owner Community Platform Engineering Team Red Hat EMEA Communications House Cork Road Waterford

1 0

[Fedocal] Reminder meeting : Fedora Infrastructure
by smooge＠gmail.com 22 Jul '20

22 Jul '20

Dear all, You are kindly invited to the meeting: Fedora Infrastructure on 2020-07-23 from 15:00:00 to 16:00:00 UTC At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Weekly Fedora Infrastructure meeting. See infrastructure list for agenda a day before. Source: https://apps.fedoraproject.org/calendar/meeting/9675/

1 0

The state of the planet (the app)
by Pierre-Yves Chibon 20 Jul '20

20 Jul '20

Good Morning Everyone, Running a personal planet and having upgraded the box running it to a recent OS (Fedora in this case), I have found out that the underlying application does not support python3. To be precise, the main script says: Requires Python 2.1, recommends 2.3. Needless to say, this doesn't work anymore with python3. So I went looking a bit around to see what is available. There is Venus which we run, which has this problem ^ and hasn't been touched in 9 years: https://github.com/rubys/venus There are a few java-based, php-based, or ruby-based, with different level of activity (though to be fair, RSS and planets have been there for a while now, so a working application likely does not need much maintenance anymore). However, nothing jumped at me as a clear successor for our setup. Do people have some experience with planet? Would you recommend us something? At this stage, it is very unclear to me if the best approach is to switch application (new stack, new UI to do, new configuration file to do/adjust...) or port venus to python3 (potentially a 1 time effort but what about the long term maintenance?). Looking forward to hear your thoughts, Pierre

4 4

[Fedocal] Reminder meeting : Fedora Infrastructure
by smooge＠gmail.com 15 Jul '20

15 Jul '20

Dear all, You are kindly invited to the meeting: Fedora Infrastructure on 2020-07-16 from 15:00:00 to 16:00:00 UTC At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Weekly Fedora Infrastructure meeting. See infrastructure list for agenda a day before. Source: https://apps.fedoraproject.org/calendar/meeting/9675/

1 0

[Fedocal] Reminder meeting : Fedora Infrastructure
by smooge＠gmail.com 09 Jul '20

09 Jul '20

Dear all, You are kindly invited to the meeting: Fedora Infrastructure on 2020-07-09 from 15:00:00 to 16:00:00 UTC At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Weekly Fedora Infrastructure meeting. See infrastructure list for agenda a day before. Source: https://apps.fedoraproject.org/calendar/meeting/9675/

3 3

Meeting Agenda Item: Introduction Mohammed Tayeh
by Mohammed Tayeh 09 Jul '20

09 Jul '20

Hi All, I'm a Systems Administrator, I have more than 4 years Experiences at Orange Palestine co. (ISP Company) I'm using Fedora on my main laptop over 5 years and I'm looking to contribute to the Fedora community my IRC nickname is "Tayeh" I'm a certified Linux System Admin (LFCSA), and I'm worked on a lot of projects I want to join the fedora system admin team if possible and I need to learn more from the fedora team and improve my skills I need some help from the team to help me to get started with fedora sysadmin team Best Regards

2 1

Testing new implementation of OIDC when updating app
by Jonathan Trossbach 08 Jul '20

08 Jul '20

Hi all, I am working on updating nuancier to work with the OIDC authentication library. I believe I have made all the necessary changes but am not sure about the best way to go about testing the changes I have made, which can be found here: https://github.com/jontrossbach/nuancier/tree/update-to-oidc Other applications like fedora elections have already been updated to OIDC, can someone tell me how they tested the new authentication process in their development environments for those applications and tell me how I can best test the changes for the OIDC in nuancier? I know at least one other person is wondering the same thing. Thank You, Jon Trossbach

2 3

Vagrant (Was: Re: What is our technical debt?)
by Pavel Valena 07 Jul '20

07 Jul '20

> -------- Přeposlaná zpráva -------- Předmět: Re: What is our technical debt? > Datum: Thu, 25 Jun 2020 21:59:37 +0200 > Od: Pierre-Yves Chibon <pingou(a)pingoured.fr> > Adresa pro odpověď: Fedora Infrastructure > <infrastructure(a)lists.fedoraproject.org> > Komu: Fedora Infrastructure <infrastructure(a)lists.fedoraproject.org> > On Thu, Jun 25, 2020 at 03:51:42PM -0400, Neal Gompa wrote: > > On Thu, Jun 25, 2020 at 3:27 PM Pierre-Yves Chibon <pingou(a)pingoured.fr> > > wrote: > > > > > > > > Good Morning Everyone, > > > > > > > > Just like every team we have technical debt in our work. > > > > I would like your help to try to define what it is for us. > > > > > > > > So far, I've come up with the following: > > > > - python3 support/migration > > > > - fedora-messaging > > > > - fedora-messaging schema > > > > - documentation > > > > - (unit-)tests > > > > - OpenID Connect > > > > > > > > What else would we want in there? > > > > > > > These are all good things, especially the documentation one. I'd like > > > to zero in on a particular aspect of documentation, though: getting to > > > hack on it. A lot of our projects are surprisingly difficult to get up > > > and running for someone to play with and hack on, and this is > > > increasingly true as we adopt OpenShift-style deployments. One way we > > > solved this in Pagure is by providing some quick start processes in > > > the documentation and a fully working Vagrant based process to boot up > > > and have a working environment to hack on the code. > > > I'm not necessarily going to specify it needs to be Vagrant for > > > everything, but I think this is something we should have for all of > > > our projects, so that people *can* easily get going to use and > > > contribute. > > I've recently had quite some pain with vagrant (just today, I've tried > several > time to start my bodhi vagrant box and lost my morning w/o success). Hello, sorry I'm late to the party, I've heard of your Vagrant issues- I'm a Vagrant maintainer, and I'll gladly help if you encounter any Vagrant issues, feel free to ping me on IRC (`pvalena`). I can also do some review of your Vagrantfiles, and run some tests, if you like (to prevent instabilities etc.). One good thing I always advise to try are the latest stable (=rawhide) Vagrant builds from my COPR: https://copr.fedorainfracloud.org/coprs/pvalena/vagrant/ (Built for all Fedoras and CentOS 7+8, although CentOS 7 currently needs manual workarounds.) Pavel > I guess it may be nice to see if there is something else out there that we > could > leverage. > If we could adopt one and try to get have it on most of our apps this may be > a > nice goal for us to work towards. > Pierre > _______________________________________________ > infrastructure mailing list -- infrastructure(a)lists.fedoraproject.org > To unsubscribe send an email to infrastructure-leave(a)lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedorapr…

1 0

CPE Weekly: 2020-07-05
by Aoife Moloney 05 Jul '20

05 Jul '20

# CPE Weekly: 2020-06-218 Background: The Community Platform Engineering group is the Red Hat team combining IT and release engineering from Fedora and CentOS. Our goal is to keep core servers and services running and maintained, build releases, and other strategic tasks that need more dedicated time than volunteers can give. See our wiki page here for more information:https://docs.fedoraproject.org/en-US/cpe/ ## General Project Updates The CPE team have finished our Quarterly Planning for Q3, July - September, and will begin work on the following projects starting from Monday: * Data Centre Move - Final Works * CentOS Stream Phase 3 * Noggin Phase 3 * Packager Workflow Healthcare * Fedora Messaging Schemas Details of the above projects, and of projects currently in progress, done and what projects are in our backlog, can be found on our taiga board per project card: https://tree.taiga.io/project/amoloney1-cpe-team-projects/kanban?epic=null We also have an updated initiative timetable for briefing in new projects to our team & key dates here:https://docs.fedoraproject.org/en-US/cpe/time_tables/ *Note: Initiatives are large pieces of work that require a team of people and weeks/months to complete. Please continue to open tickets in the normal way for bugs, issues, etc. Look out for our Q2 project achievements blog post coming soon! ### CPE Product Owner Office Hours #### #fedora-meeting-1 * Weekly on Thursdays @ 1300 UTC on #fedora-meeting-1 * Next Meeting: 2020-07-09 #### #centos-meeting * Every second Tuesday @ 1500 UTC on #centos-meeting * Next Meeting: 2020-07-07 ## Fedora Updates ### Data Centre Move * We are now officially operating under reduced Fedora services until est 28th July to facilitate the final shipment of hardware to the new data centre. * Please View project card on taiga for builder - staging - remaining app bringup dates here https://tree.taiga.io/project/amoloney1-cpe-team-projects/us/42?kanban-stat… * A list of affected services is available here https://hackmd.io/hpYYJQRjQy-oHxUS7IonIA?view * Details on what this move may mean for you can be found here https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedorapr… * If an application is not working correctly at all, please check this list https://hackmd.io/hpYYJQRjQy-oHxUS7IonIA?view before opening a ticket to make sure its not listed as being moved. If it is being moved, please wait a day or two, then try again. * Similarly, please be patient when opening tickets for service issues in general as we have now reached the critical point in this move and all of our sys-admins and wider teams will be assisting in the successful bringup of the reduced Fedora service and facilitation of the final hardware shipment and move. * Most recent update to devel-announce is here https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedorapr… ### AAA Replacement * User agreements for both CentOS and Fedora accounts are nearly complete. We will be able to test this in more detail once staging has been reinstalled (est July 25th due to colo move) * Fasjson now supports search feature * App migreations work still ongoing * Spam detection service is being investigated and scoped for Q3 work * Please feel free to check out the team kanban board for more information on the features the team are working on and have already completed here https://github.com/orgs/fedora-infra/projects/6 ### Mbbox * Project Dashboard here https://github.com/fedora-infra/mbbox/projects/1 * Tasks completed in the project currently * MBBox handover to CentOS Stream * The only thing that remains is putting the operator in the public operator registry which is pending Stream team feedback * MBox shared CRD is done * Deployment guide - https://mbbox-operator.readthedocs.io/en/latest/ * Blog post published - check it out here https://communityblog.fedoraproject.org/mbbox-module-building-in-a-box/ ### Gitforge We are still discussing technical aspects of the project and these are tracked here: https://gitlab.com/gitlab-org/gitlab/-/issues/217350 Planning a date in late August for an AMA session with GitLab that will be run through IRC for any questions the Fedora and CentOS communities may have to ask direct. We will keep you up to date with the developments as and when we have information to share and thank you again for your patience. ## CentOS Updates ### CentOS * New website went live (https://www.centos.org) * A new AWS CDN setup for Stream composes (https://composes.centos.org) is being worked on with a goal to have the latest stream artifacts to be publicly available and at faster speed than through normal mirror.centos.org cdn (and/or external mirrors) ### Centos Other * CentOS CI began onboarding Fedora CI, CoreOS CI and Fedora CoreOS as tenants this week * Work going well and stable! ### CentOS Stream * RealTime for Stream is released! * The team are also working on more module builds for Stream As always, feedback is welcome, and we will continue to look at ways to improve the delivery and readability of this weekly report. Have a great week ahead! Aoife Source: https://hackmd.io/8iV7PilARSG68Tqv8CzKOQ?view

1 0

Introduce my self [futur member]
by seddik alaoui ismaili 04 Jul '20

04 Jul '20

Hello team I am very pleased to write you this email, to let you know my motivation and my situation. - Name : alaoui ismaili seddik - Time zone / country : GMT +2 / France I will start with an introduction of my experiences and skills, So I was a GNU/LINUX administrator for 2 years, the first year was with a software editor (competitive intelligence), during this period I was able to develop my skills on different subjects including system administration (Centos os ), maintenance in operational condition, the implementation of basic tools/services, scripting (python/bash), db (mysql), C compilation. I also managed to develop the sense of sharing and collaborative work. Also for the second job at the company (numberly) as administrator/devops, I took care of the DNS platform (powerdns), automatic deployment of the conf via ansible, mainly the industrialization of the IT. And then on different projects, for example on the cloud (hosting dns on route_53 AWS), centreon, grafana, elk stack, also managing the resolution of incidents raised from the production servers. I was system operator at capgemini for 1 year and 9 months more precisely, the production piloting for two linux/unix (AIX, HP-UX, REDHAT) and Mainframe (MVS) spells. Currently I am an IT consultant at orange (telecom operator), I work in an application environment (tomcat), os linux (REDHAT), and my main missions are, the maintenance of the ELK stack, system administration, installation of application instances in automatic way (ansible), continuous deployment, migration of code on GITLAB. I'm always interested in the linux operating system because it's an opportunity to be closer to the machine, with it we can understand how a system works, and react in case of bugs, it also allows to have a support through a community that contributes behind it. For this I will be happy to contribute on projects/tasks related to GNU/LINUX system administration, bug analysis. I will be able to give my time to the community, about 10 hours a week. I hope I wasn't too long on my presentation. Kind regards Seddik

3 4

[Fedocal] Reminder meeting : Fedora Infrastructure
by smooge＠gmail.com 01 Jul '20

01 Jul '20

Dear all, You are kindly invited to the meeting: Fedora Infrastructure on 2020-07-02 from 15:00:00 to 16:00:00 UTC At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Weekly Fedora Infrastructure meeting. See infrastructure list for agenda a day before. Source: https://apps.fedoraproject.org/calendar/meeting/9675/

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure July 2020