About JS framework
by Pierre-Yves Chibon
Good Morning Everyone,
Our infrastructure is mostly a python store, meaning almost all our apps are
written in python and most using wsgi.
However in python we are using a number of framework:
* flask for most
* pyramid for some of the biggest (bodhi, FAS3)
* Django (askbot, Hyperkitty)
* TurboGears2 (fedora-packages)
* aiohttp (python3, async app: mdapi)
While this makes sometime things difficult, these are fairly standard framework
and most of our developers are able to help on all.
However, as I see us starting to look at JS for some of our apps (fedora-hubs,
wartaa...), I wonder if we could start the discussion early about the different
framework and eventually see if we can unify around one.
This would also allow those of us not familiar with any JS framework to look at
the recommended one instead of picking one up semi-randomly.
So has anyone experience with one or more JS framework? Do you have one that
would you recommend? Why?
Thanks for your inputs,
Pierre
1 month, 2 weeks
planet
by Kevin Fenzi
Hey folks. I thought I would open a discussion about fedoraplanet and
possibly some plans for it.
Right now:
fedoraplanet.org runs on people02.fedoraproject.org (aka fedorapeople).
To add a blog/rss feed you have to login there and edit your .planet
file, then scripting pulls all those .planet files and tries to fetch
all the feeds and then serves them up at http://fedoraplanet.org.
It uses a app called 'venus' to do this. venus is written in very old
python2 and very very dead upstream.
We run into the following problems with it:
* Sometimes it gets stuck and just stops processing until it's killed.
* It's serving on a http site, which causes people to ask us to make it
https, but that would just change the errors because many feeds it pulls
are still http since they were added back before letsencrypt existed.
* We have a handy 'website' field in our new account system, but aren't
using it at all.
* The .planet parsing is poor, any number of things can cause it to
break.
We have two open tickets on it:
- https://pagure.io/fedora-infrastructure/issue/10383 (upgrade to pluto,
a ruby based, but maintained thing)
- https://pagure.io/fedora-infrastructure/issue/10490
( planet not served via ssl) Which I am just going to close now.
So, I can think of a number of options and would love everyone who has
thoughts on it to chime in:
1. Do nothing. Venus "works" and .planet files are cool and retro.
2. Switch to pluto and use account system 'website' fields of
contributors. We could likely shove it in openshift and serve it
directly from there to avoid fedorapeople entirely.
(This would likely break anyone who has multiple feeds in there)
3. Switch to something better/bigger. I would think (although I don't
know) that there might be something that would not only aggregate rss
feeds for contributors, but perhaps mastodon/twitter/whatever also.
4. Planets are old and tired, just drop the entire thing. People can
maintain their own rss lists.
5. Planets are old and tired, just drop the entire thing.
But also, get our social media people to maintain contributor /
interesting lists. ie, the fedoraproject twitter account could maintain
a list of 'fedora contributors' and 'fedora packagers' or whatever.
6. Switch to pluto as in 2, but also setup some curators. Have a
'firehose' of all feeds, but the main fedora planet would be just
curated things that are known to be related to fedora and not off topic
or unrelated.
6. Get someones (not it!) to take in all the
twitter/facebook/mastodon/blog posts/rss feeds and post some kind of
curated round up every week or something.
7. Your brilliant idea here!
So, thoughts? this is not at all urgent, but we should end up doing
something with it sometime. :)
kevin
3 months, 1 week
old vm's
by Kevin Fenzi
Looking at our current hosts (ansible_distribution_major_version):
275 35
120 8
90 7
22 34
11 36
6 33
2 9
2 31
1 37
1 32
1 27
(The reason so many are f35 is because builders are f35)
Anyhow:
- 27 -> This is notifs-backend01. We are still trying to get the python3
FMN deployed and working in stg so we can repave this and move it to a
supported version.
- 31 -> These are resultsdb01 and resultsdb01.stg. We already have the
openshift replacement in staging working, we just need to cut over
prod. We should schedule a time/date to do that.
- 32 -> This is odcs-backend-releng01. I am just going to reinstall it
and hopefully odcs is happy on f36. :) I'll mail odcs folks to
confirm.
- 33 -> These are osbs-aarch64 prod and stg. I'm open to suggestion on
what to do here. I'd personally say we should leave them and hasten
our move to image builder and hopefully by then they have container
building and aarch64? Other ideas welcome.
- 34 -> These are still supported at least, but we need to move them
soon. I can do most of these, will be contacting people about them.
Finally after that we should try and start moving the 8's and 7's to
9's. ;)
kevin
5 months, 3 weeks
Upcoming MirrorManager changes (removal of Internet2 support)
by Adrian Reber
I will remove Internet2 support from MirrorManager in the next few
weeks.
The main reason is, that the source for the Internet2 routing tables
does not exist any more: http://routes.net.internet2.edu/
It changed a couple of years ago and it seems Fedora was one of the only
users of that data. They fixed it for us, but the latest ticket I opened
to access this information was closed with "This service was decomd".
I do not think that the removal of Internet2 support for our users will
be noticed at all. Removing Internet2 will remove some complexity from
the code and using ASN and GeoIP should still provide a reasonably close
mirror for most of out users.
I will start to update the different components to not expose and rely
on Internet2 information any more in the next couple of weeks.
Let me know if you see any major problems in removing Internet2 support
from MirrorManager.
Adrian
5 months, 3 weeks
on rpms
by Kevin Fenzi
So, in the past we have always had a policy to package as rpms and get
into fedora/epel applications we deploy and are upstream for.
There were a number of good reasons for this:
* We deployed everything on vm's using rpm.
* Other users that wanted to reproduce our infrastructure could use the
rpms.
* It made us sure that the thing passed review and built on various
Fedoras with the versions of things there it depended on.
However, now a days we have a number of new apps that are deployed in
openshift and aren't using rpms, but pip or s2i or other things. For
these packaging them up as rpms becomes a burden with not too many
gains. ;(
So, I was thinking we should codify a new policy.
(To avoid confusion for application authors and others).
Something like:
Applications in Fedora Infrastructure may be deployed via non rpm
methods (as long as they obey licensing guidelines (
https://fedoraproject.org/wiki/Infrastructure_Licensing )). For those
applications, creating and maintaining an rpm is optional.
Thoughts?
kevin
6 months
Announcing sqlalchemy-helpers
by Aurelien Bompard
Hey folks!
A few months ago I started a library to share some boilerplate code in our
applications when it comes to SQLAlchemy.
Remember the thread about Flask and SQLAlchemy
<https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedora...>?
It ended with us admitting that Flask-SQLAlchemy is pretty cool, but does
not cover our non-Flask needs, can introduce issues with scripts that do
multithreading manually, and ties us into Flask more than what we may feel
comfortable with.
But the problem remains the same: by doing SQLAlchemy integration manually
over the years, we've copied code from one project to the next, improving
it when we were working on it but not backporting those improvements to
older projects, and having rather sparse (I should say "distributed" ;-) )
knowledge of what the SQLAlchemy & Alembic best practices are (everybody
here knows about constraint naming conventions? Foreign key enforcement on
SQLite?)
In the spirit of avoiding code duplication and sharing maintenance, I've
started the sqlalchemy-helpers library
<https://github.com/fedora-infra/sqlalchemy-helpers>.
The idea is to standardize on best practices with SQLAlchemy & Alembic
usage without depending on Flask, but with an optional Flask integration.
The README has a list of features and usage examples, so I'll let you go
over there and skim read it.
Basically what I've done is go over all our Flask apps using SQLAlchemy,
look at what features they implemented, and bring them into
sqlalchemy-helpers. As a result if you decide to move to it you will only
gain features.
The only notable exception is the pagination system that Flask-SQLAlchemy
implements and that at least one project has copied. If it proves useful,
we can always bring it to sqlalchemy-helpers, but let's start small for now.
It targets SQLAlchemy 1.3+ & 2.0, Flask 2.0+ and Python 3.6+.
I've used it personally for a few months already and I'd say it works quite
well. I'm very interested in feedback. My next step is to improve user
documentation because right now it's only the README and the APIdocs.
I hope we can join forces and maintain this sort of boilerplate code in a
central place.
Cheers!
Aurélien
6 months
Renaming design-sw to design-suite
by Luya Tshimbalanga
Hello team,
Will it be possible to rename the group "design-sw" to "design-suite"
please? The goal is to keep consistancy with name. Thanks in advance.
--
Luya Tshimbalanga
Fedora Design Team
Fedora Design Suite maintainer
6 months
CPE Weekly Update – Week 21 2022
by Lenka Segura
Hi everyone,
This is a weekly report from the CPE (Community Platform Engineering) Team.
If you have any questions or feedback, please respond to this report or
contact us on #redhat-cpe channel on libera.chat (https://libera.chat/).
Week: 23rd May - 27th May 2022
If you wish to read this in form of a blog post, check the post on Fedora
community blog:
https://communityblog.fedoraproject.org/cpe-weekly-update---week-21-2022/
# Highlights of the week
## Infrastructure & Release Engineering
Goal of this Initiative
-----------------------
Purpose of this team is to take care of day to day business regarding
CentOS and Fedora Infrastructure and Fedora release engineering work.
It’s responsible for services running in Fedora and CentOS infrastructure
and preparing things for the new Fedora release (mirrors, mass branching,
new namespaces etc.).
The ARC (which is a subset of the team) investigates possible initiatives
that CPE might take on.
Link to planning board: https://zlopez.fedorapeople.org/I&R-2022-05-25.pdf
Update
------
* Infra and releng team was investigating Openshift in AWS for communishift
and CentOS CI
### Fedora Infra
* Ocp3 certs expired, had to renew/push new ones (~20min outage)
* Finished up a mass update/reboot. Things up on 8.6 now.
* Hit bug in new ansible-freeipa and filed it upstream
* Got first RHEL9 staging virthost installed
* Bunch of email issues to/from redhat.com, hopefully almost all solved now.
* OCP4 clusters moved to 4.10.
* Some compose machines moved to f36 already.
### CentOS Infra including CentOS CI
* CentOS Stream storage migration spike (Netapp for nfs/iscsi) (blocked :
no news)
* Duffy/CI progress:
* Site-to-site vpn tunnel between VPC (us-east-1) and CI infra
* delegated/hosted (dynamic) pool.ci.centos.org zone on route53
* Dns forwarding (both directions : vpc ⇔ ci infra)
* Ansible playbooks ready
* Git.centos.org pagure upgrade/migration (blocked, waiting on internal -
EXD)
* Business as usual (mirrors, tags)
### Release Engineering
* f32/33 archived. Old rc’s cleaned up
## CentOS Stream
Goal of this Initiative
-----------------------
This initiative is working on CentOS Stream/Emerging RHEL to make this new
distribution a reality. The goal of this initiative is to prepare the
ecosystem for the new CentOS Stream.
Updates
-------
* Working on first modules and software collections for CentOS Stream 9
* Changing the Fedora ELN Everything repository to only contain what's not
in AppStream, BaseOS, and CRB, and renaming it to Extras
* Getting CentOS Stream 8 workflow closer to 9:
* Currently testing "import latest modules", expect to have modules
imported early next week.
* Updating is currently manual for both modules and normal RPMs, also
working on an automated import script to run at regular intervals. Expect
to be finished next week.
## CentOS Duffy CI
Goal of this Initiative
-----------------------
Duffy is a system within CentOS CI Infra which allows tenants to provision
and access bare metal resources of multiple architectures for the purposes
of CI testing.
We need to add the ability to checkout VMs in CentOS CI in Duffy. We have
OpenNebula hypervisor available, and have started developing playbooks
which can be used to create VMs using the OpenNebula API, but due to the
current state of how Duffy is deployed, we are blocked with new dev work to
add the VM checkout functionality.
Updates
-------
* Skip beta and release 3.0.0
* Continue work on provisioning nodes for Duffy in EC2
* CLI for tenants (what was cicoclient, ongoing)
## Package Automation (Packit Service)
Goal of this initiative
-----------------------
Automate RPM packaging of infra apps/packages
Updates
-------
* Package proposal on fasjson created on bugzilla
* Release of fedora-messaging 3.0.2
* Packit added to noggin-messaging (noggin dependency)
* Packaging datanommer.models in progress (datagrepper dependency)
* Business as usual, a lot of manual packaging
## Flask-oidc: oauth2client replacement
Goal of this initiative
-----------------------
Flask-oidc is a library used across the Fedora infrastructure and is the
client for ipsilon for its authentication. flask-oidc uses oauth2client.
This library is now deprecated and no longer maintained. This will need to
be replaced with authlib.
Updates:
--------
* Test-auth app running, allows user to successfully login using Fedora
acct details (https://app-flask-oidc-dev.apps.ocp.stg.fedoraproject.org/)
* Implementing authlib function changes in codebase (
https://github.com/fedora-infra/test-auth/blob/authlib_dev/test_auth/flas...
)
* Trying to pull user info from login creds
## EPEL
Goal of this initiative
-----------------------
Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest
Group that creates, maintains, and manages a high quality set of additional
packages for Enterprise Linux, including, but not limited to, Red Hat
Enterprise Linux (RHEL), CentOS and Scientific Linux (SL), Oracle Linux
(OL).
EPEL packages are usually based on their Fedora counterparts and will never
conflict with or replace packages in the base Enterprise Linux
distributions. EPEL uses much of the same infrastructure as Fedora,
including buildsystem, bugzilla instance, updates manager, mirror manager
and more.
Updates
-------
* epel9 available at the launch of rhel9 with 5764 packages, from 2678
source packages, built by 249 different packagers
* epel9 update of python-jwt to fix CVE-2022-29217
* epel8 and epel9 update of epel-rpm-macros to require ansible-srpm-macros
(provides macros for packaging ansible collections)
* bdii installation from epel9 problem resolved for c9s, will be resolved
in RHEL 9.1
* Documentation improvements:
* added list of channels/repos epel9 and epel9-next are built against
* updated quickstart guide for rhel9 GA and alma9 beta
* removed mentions of CentOS Linux 8
Kindest regards,
CPE Team
6 months, 1 week
Upcoming Ipsilon update
by Aurelien Bompard
Hey folks!
I have recently been given the powers to make Ipsilon releases, so I'm
going to deploy the recent changes to production soonish (probably tomorrow
early morning UTC). We've been working with a snapshot so it's not as big
an update as you'd think when looking at the date of the last official
release (2017). Which means things should not break (staging works fine).
It should get us the *very much* requested feature of having a separate
field for the OTP token, among other things.
Also, the update is needed to update Bodhi later (something I still need to
organize, btw)
Cheers!
Aurélien
6 months, 1 week
