I'd like to let everyone know who uses it, that we are going to be
moving our ansible repo over to https://pagure.io/fedora-infra/ansible
(note the lower case fedora-infra there, we are going to move the
existing Fedora-Infra one and make sure there's a redirect from it).
This move will take place possibly this weekend if all testing
goes well. If not, we will be looking to do it next week on tuesday.
We will provide a heads up in this thread before the outage.
Now, some Q&A, since those are fun:
Question: How long is the outage going to be?
Answer: We don't have an exact number, but I think it should be pretty
short. An hour or two at most.
Question: what do I need to do to keep pushing my changes after this?
Answer: You will need to clone the git repo from
https://pagure.io/fedora-infra/ansible.git You should do this on your
day to day work machine (not batcave01).
Question: I used to push my changes in, but now it's denying me.
Answer: let us know in an infra ticket if you or others who need to
commit/merge PRs are unable to do so.
Question: why are you doing this?
Answer: we want PRs! This gives us:
* A easy way to review changes without sending patches to the list.
* A way to make sure changes in freeze are acked and pushed.
* Contributions from outside folks that just want to make specific
* ability to add CI/CD to changes and catch syntax errors early.
* More visibility as people can more easily 'see' the repo.
Question: why pagure? Why not gitlab? Or github? Or fossil?
Answer: we already have our tickets on pagure, so it makes sense to have
the ansible repo there too so we can refer to PR's/tickets easily, etc.
Someday we may move somewhere else, but today... is not that day.
Question: What happens if pagure wins the lotto and decides to quit
the source forge business? (Or is down for a long time, etc)
Answer: we are mirroring the ansible repo to batcave01. We have a
listener listening for fedora-messaging messages about commits or merges
and when they happen on pagure, our batcave01 repo updates right after.
So, the window is low for us missing changes, and we have a copy of the
repo there if we need to make changes when pagure is down.
Question: Why not put this in the same exact project as
fedora-infrastructure tickets are?
Answer: We already have a repo there, so it would mean pushing all the
commits over that. Also, we don't want to grant lots of people commit
there as that also allows them to see private tickets (which are usually
security or other sensitive issues), so a seperate repo seemed like an
easier thing to do.
Question: can I checkout on batcave01 and push my changes there like I
Answer: No. We don't want people forwarding their ssh agent into our
infra, so people push commits from your home machine.
Question: Do I still need to run ansible-playbook/rbac-playbook on
Answer: yes, batcave01 has the ansible ssh key in it's agent to allow it
to access all the machines, so you must continue to run that there.
If you have any other questions, ask away.
We hope this will help and make the team and community more productive,
happier, better looking and wise. All kudos to Pingou who has been
working on this and all flames my way. :)
Good Morning Everyone,
I have a few items in my backlog that I'd like to discuss priorities with you,
so here is the unsorted list, let me know how you would sort it :)
* Finish bringing bugzilla overrides to dist-git
- Deploy pagure 5.9.x to src.fp.o
- Migrate the data from the scm-requests repo to dist-git
- Adjust the README of the scm-requests repo
- Close off the scm-requests repo to pull-requests
- Announce & profit/watch the world burn
Blocked by the current freeze, unless a FBR is acceptable to upgrade pagure
to 5.9.x (knowing that 5.9.x does not bring any DB changes).
* Reworkd the packager sync from FAS to bugzilla
Currently, there is a cron job that adds bugzilla privileges to people that
are added to the packager group. That cron relies on a DB in FAS that tracks
people being added or removed from this group. This isn't quite the 2020 way
of doing things and this will not be portable to noggin (the next gen FAS).
Python-bugzilla also recently (it's merged but not released) gained support
for groups, so we can finally do something like: ask FAS for all the packagers
and their email, list all the people in the corresponding group in bugzilla,
do a diff and add/remove accordingly.
* Finish retiring in bugzilla packages that are retired in Fedora (ie: close
these components to new bugs in bugzilla).
This was blocked on a change in bugzilla which has been deployed in the last
release. So we should be able to proceed and adjust our bugzilla<->dist-git
sync script to do this.
* Mirror the ansible git repo on pagure.io
I'd like to set up a mirror on pagure.io that would pull from batcave01. It
would mean that PR can't really be merged in this mirror (unless we're fast
enough to pull from the mirror and push to the main repo right after the merge,
but there is a risk of a race-condition where the commit(s) just merged are
overridden by a push to the main repo).
It would expose a more up to date ansible repo to the public and we should be
able to wget the patch of the PRs, git am to apply them and git push to the
* Migrate stg.pagure.io and src.stg.fedoraproject.org to RHEL8.
While we're in freeze, I figure this is a good time to do this. We could do
pagure.io post-freeze and wait to do src.fp.o when it gets reinstalled in the
So here you go, let me know what you think :)
Hope youre doing well, My name is Keerthi, I am currently working as a Systems Tech, I am located in the US (CST). As part of my job I primarliy work with windows (both servers and desktops), I started learning Linux couple of years ago and now I use Linux (Fedora) as my daily driver.
What skills you have to offer and which you would like to learn :
I am familier with Python, HTML, CSS, Ansible, Linux administration, Basic knowledge on MariaDB, MySQL DB, and Containers, familier with AWS (Learning and preparing for certs), currently I hold AWS Could Practioner Cert. These are not something that I use everyday at my work but something I picked up and started learning on my down time, I've joined RedHat's #EnableSysadmin community, started wirting articles on various sysadmin topics.
I would like to lean more about the Fedora-Infrastructure team, get familier with tha workflow and pickup easyfixes and build upon as i go.
IRC handle : iamkc
Looking forward to Learn, contribute and share,
Thanks for your time,
my name is Jens and i'm from Germany. I am 47y old/young and I'm
working as an Bachelor Professional of Electrical Technology and
Management. In my sparetime i am currently learning for the LPIC
certification, why? Well... i'm working with Linux since 2000, beginning
with SuSE 6.4 and debian. Since 2009 i'm using Fedora.
I have a lot of fun with Linux, learning every day new things. So i
decided to take a exam for myself, not for my company.
On IRC in freenode i am now 15 years. Generally i want to learn more
about basic system-administration to get more experience to pass my
exam, and i would be happy to get the chance to to that :)
I'm not sure who is the correct person to direct this question at, so I'll
open it up to the group. In my previous job I worked daily with AWS and
have a decent grasp of the workings of some parts of it (I don't think
anyone knows all of AWS).
I think I could be of assistance in implementing best practises and also in
carrying out general day to day work. I have seen these 2 tickets
specifically on the fedora infrastructure issue tracker which I may be of
These involve IAM which I understand carries a bit of a security issue but
I could work with someone who has access or even with read only access and
I could offer some potential fixes. Let me know if I can help.
Good Morning Everyone,
Kevin and I had a discussion on IRC the other day about mirroring the ansible
The options we considered are:
- mirroring to pagure
- mirroring from pagure
- using pagure's mirroring feature
- using a different approach
- Mirroring to pagure
This meant, the canonical place for ansible remains in the batcave and pagure
only periodically pulls from there to update itself (or a hook updates pagure
This would work, but it basically make the pull-request workflow for reviewers
a little more complex, as you would have to download the patch of the PR,
apply it (potentially adding the: Merging <id> so it marks the PR as merged)
and then push to the batcave.
If you merged via the UI in pagure, you would then have to pull from pagure
and then push to the batcave, with the risk of a race condition if someone or
something updates pagure right after your merge and suddenly the commit(s) of
the PR disapear (since the content in pagure would be overriden by what is
coming from the batcave).
So, not an ideal workflow.
- Mirroring from pagure
This means, the canonical place for ansible is in pagure.io.
However, we need to still be able to run ansible when pagure.io goes down
(keeping in mind batcave and pagure.io are in two different data-center).
So we want to keep a mirror of the ansible repo in the batcave for emergencies
and that repo should be as up to date as possible.
With this approach, all the work happens on pagure.io, the PR workflow is the
usual, straight forward one and we have a mirror in the batcave that is mostly
up to date.
- Using pagure's mirroring feature
Pagure can mirror in and out, out is done right after a push, however, batcave
is not accessible from pagure.io, so we just can't use this.
- Using a different approach
We have two ways we could do this: a simple cron job, a message-based service.
For a first step I went with a third approach: a small python service that
runs every 3 minutes (configurable): git fetch && git fsck (to ensure the git
is in a correct state). It actually doesn't run every 3 minutes, it calls git,
then wait for 3 minutes then calls git again and so on. So if git was ever to
takes 4 minutes to run, there is no risk of the program stepping on its own
I would like to propose that install and configure this for a test.
I propose that we keep /git/ansible.git as is and just have a
/git/mirrors/ansible.git which will be the mirror from pagure.
We can migrate the hook from /git/ansible.git to /git/mirrors/ansible.git so
/srv/web/infra/ansible (ie: the exploded tree) is coming from the mirror.
I guess the lag time to get /git/mirrors updated will quickly be annoying (up to
3 minutes between when the PR is merged and when the playbook is updated), so
if we like the workflow we will likely want to switch pretty quickly to a
message-based service and then we could keep the current cron-like service to
run hourly and update /git/ansible.git which would then become some kind of
What do you think?
FBR worthy? (ie: if F32 goes out next week, I think we can wait, if it doesn't
do we want to try this sooner?)
If we like the idea, I'll start looking into the fedora-messaging based
consumer, using the approach from the current service, it should be pretty
straight forward (I'll re-use the same project for it).
Oh, and if you want to see the code of the current service:
The infrastructure team will be having its weekly meeting tomorrow,
2020-04-30 at 15:00 UTC in #fedora-meeting-1 on the freenode network.
We have a document at https://board.net/p/fedora-infra
Please try and review and edit that document before the meeting and we
will use it to have our agenda of things to discuss. A copy as of today
is included in this email.
If you have something to discuss, add the topic to the discussion area
with your name. If you would like to teach other folks about some
application or setup in our infrastructure, please add that topic and
your name to the learn about section.
We will use it over the week before the meeting to gather status and
info and discussion items and so forth, then use it in the irc meeting
to transfer information to the meetbot logs.
### Meeting start stuff
#startmeeting Infrastructure (2020-04-30)
#chair nirik pingou smooge cverna mizdebsk mkonecny abompard
#info Agenda is at: https://board.net/p/fedora-infra
#info About our team: https://docs.fedoraproject.org/en-US/cpe/
### Let new people say hello
#topic New folks introductions
#info This is a place where people who are interested in Fedora
Infrastructure can introduce themselves
#info Getting Started Guide:
### Determine who the next chair is
#topic Next chair
#info magic eight ball says:
#info 2020-04-30 - smooge
#info 2020-05-07 - cverna
#info 2020-05-14 - siddharthvipul
#info 2020-05-21 - ???
### Status / Information / Trivia / Announcements
(We put things here we want others on the team to know, but don't need
(Please use ```#info (the thing - your name)```
#topic announcements and information
#info CPE Sustaining EU-hours team has standups on Tuesday and
Thursday at 1400 UTC in #fedora-admin - please join
#info CPE Sustaining NA-hours team has a Monday through Friday 30
minute meeting going through tickets at 1800 UTC in #fedora-admin
#info Fedora Infrastructure will be moving in 2020-06 from its Phoenix
Az datacenter to one near Herndon Va. A lot of planning will be
involved on this. Please watch out for announcements on changes.
#info Fedora Communishift move has started but will take longer than
expected. Current estimate for bringing back into production is TBD
#info F32 final freeze is in effect!
#info Taskotron will EOL in 2020-05
### Things we should discuss
We use this section to bring up discussion topics. Things we want to talk about
as a group and come up with some consensus /suor decision or just brainstorm a
problem or issue. If there are none of these we skip this section.
(Use ```#topic your discussion topic - your username)```
#info siddharthvipul is oncall 2020-04-16 -> 2020-04-23
#info cverna is oncall 2020-04-23-> 2020-04-30
#info siddharthvipul is oncall 2020-04-30 -> 2020-05-07
#info ??? is oncall 2020-05-07 -> 2020-05-14
#info ??? is oncall 2020-05-14 -> 2020-05-21
## .oncalltakeeu .oncalltakeus
#info Summary of last week: (from current oncall )
#topic Monitoring discussion [nirik]
#info Go over existing out items and fix
### Put all topics for discussion under here
Here we will discuss any apprentice questions, try and match up people looking
for things to do with things to do, progress, testing anything like that.
### Learn about some application or setup in infrastructure
(This section, each week we get 1 person to talk about an application
or setup that we have. Just going over what it is, how to contribute,
ideas for improvement, etc. Whoever would like to do this, just add
the i/nfo in this section. In the event we don't find someone to teach
about something, we skip this section and just move on to open floor.)
### Meeting end stuff
#topic Open Floor
Stephen J Smoogen.