BYO Jenkins for Pagure CI?
by Justin W. Flory
Hi all,
I was looking into Pagure CI with Jenkins, but it wasn't clear if there
is a Fedora-provided Jenkins server or if someone needs to bring their
own Jenkins build infrastructure for Pagure CI.
If there is a Fedora-provided Jenkins, could someone point me towards
documentation on requesting an account / getting started?
Thanks!
--
Cheers,
Justin W. Flory
justinwflory.com
TZ=America/Chicago
Pronouns: he/him/his
4 years, 9 months
OSBS buildroot image + flatpak-module-tools-0.10.1
by Owen Taylor
[ Summary: if you are going to rebuild the OSBS buildroot image in the
near future, talk to me to make sure things don't break ]
I just submitted a new version of flatpak-module-tools to Bodhi. This
updates flatpak-module-tools to use the libmodulemd v2 API - this was
forced because module-build-service hsa switched to use the v2 API.
*However* this update is incompatible with the current version of
atomic-reactor - which has references to the libmodulemd v1 API. So,
if the OSBS buildroot image is rebuilt after
flatpak-module-tools-0.10.1 hits stable, but before atomic-reactor is
fixed, Flatpak builds are going to break.
Submitting the necessary atomic-reactor fixes upstream blocks on
getting a libmodulemd v2 package into EPEL-7:
https://bugzilla.redhat.com/show_bug.cgi?id=1724271 - but we can land
them as a local patch to the Fedora atomic-reactor package if
necessary.
Owen
4 years, 9 months
proposal: move pagure01 and pagure-stg01 to rdu2
by Kevin Fenzi
Greetings.
I'd like to propose we migrate pagure01 (which is pagure.io) and
pagure-stg01 (pagure-stg.io) from osuosl to the community cage in rdu2.
We have had odd networking problems between .cz and osuosl and at times
between phx2 and osuosl.
Moving things would allow us to:
drop the pagure-proxy in ibiblio (which currently proxies all traffic to
the osuosl instances to avoid the networking issues from cz).
move forward with adding repospanner to ansible and have it appear in
pagure.io so people could do PR's and CI and other fun things against
our ansible repo.
Why not move it to openshift? Well, we could, but thats going to take
cycles we don't have right now, so a simple migration seems much easier
and gets us a lot of good.
Why not move it to phx2? Well, we would need to get a bunch of ports
opened for it's use and we may well be moving resources out of phx2 at
some point, so we really don't want to add another thing to move out.
Why not make a new instance in rdu2 and migrate just data to it?
Again, we could, but at this point a brute force migration seems like a
better use of our cycles than figuring exactly what needs synced over.
If we really don't want the downtime we can look more into this, I just
worry that it will take a lot of time to make sure we have everything
synced and setup right.
If folks are ok with this, I'd like to consider taking it down friday
late afternoon and migrating it over this weekend. It would have
downtime for however long it takes to sync the disk over. I can do
pagure-stg01 as a test run later this week if we like.
Thoughts?
kevin
4 years, 9 months
Proposed Meeting Agenda for 2019-07-11
by Stephen John Smoogen
= Preamble =
The infrastructure team will be having its weekly meeting tomorrow,
2019-07-11 at 15:00 UTC in #fedora-meeting-1 on the freenode network.
We have a gobby document at
https://infinote.fedoraproject.org/cgit/infinote/tree/fedora-infrastructu...
which can be edited for the agenda (see:
https://fedoraproject.org/wiki/Gobby )
Please try and review and edit that document before the meeting and we
will use it to have our agenda of things to discuss. A copy as of today
is included in this email.
If you have something to discuss, add the topic to the discussion area
with your name. If you would like to teach other folks about some
application or setup in our infrastructure, please add that topic and
your name to the learn about section.
= Introduction =
We will use it over the week before the meeting to gather status and info
and
discussion items and so forth, then use it in the irc meeting to transfer
information to the meetbot logs.
= Meeting start stuff =
#startmeeting Infrastructure (2019-07-11)
#meetingname infrastructure
#topic aloha
#chair nirik pingou puiterwijk relrod smooge tflink cverna mizdebsk
mkonecny abompard bowlofeggs
= Let new people say hello =
#topic New folks introductions
#info This is a place where people who are interested in Fedora
Infrastructure can introduce themselves
#info Getting Started Guide:
https://fedoraproject.org/wiki/Infrastructure/GettingStarted
= Status / Information / Trivia / Announcements =
(We put things here we want others on the team to know, but don't need to
discuss)
(Please use #info <the thing> - your name)
#topic announcements and information
#info bowlofeggs is on extended leave
#info cverna will be going on extended leave
#info abombard will be going on extended leave
#info other vacations/leave/etc?
#info Flock2Fedora 2019-08-08 -> 2019-08-11
#info No site trip to PHX2
#info Red Hat is now owned by IBM.
= Things we should discuss =
We use this section to bring up discussion topics. Things we want to talk
about
as a group and come up with some consensus /suor decision or just
brainstorm a
problem or issue. If there are none of these we skip this section.
(Use #topic your discussion topic - your username)
#topic Oncall
#info https://fedoraproject.org/wiki/Infrastructure/Oncall
#info smooge is on call from 2019-07-04 -> 2019-07-11
#info ?????? is on call from 2019-07-11 -> 2019-07-18
#info Summary of last week: (from smooge )
#topic Monitoring discussion
#info https://nagios.fedoraproject.org/nagios
#info Go over existing out items and fix
#topic Tickets discussion
#info https://pagure.io/fedora-infrastructure/report/Meetings%20ticket
Go thru each ticket one by one
=====
Put all topics for discussion under here
=====
Here we will discuss any apprentice questions, try and match up people
looking
for things to do with things to do, progress, testing anything like that.
= Learn about some application or setup in infrastructure =
(This section, each week we get 1 person to talk about an application or
setup
that we have. Just going over what it is, how to contribute, ideas for
improvement,
etc. Whoever would like to do this, just add the i/nfo in this section. In
the
event we don't find someone to teach about something, we skip this section
and just move on to open floor.)
#info
= Meeting end stuff =
#topic Open Floor
#endmeeting
--
Stephen J Smoogen.
4 years, 9 months
compose connection issues
by Kevin Fenzi
Greetings everyone.
So, we have seen some very sporadic connection issues that have been
breaking composes. I'm at a loss how to debug it or figure out whats
going on, so I thought I would collect information here and see if
anyone had any ideas.
Of the last 11 doomed rawhide composes:
Fedora-Rawhide-20190622.n.0 - loop issue [0]
Fedora-Rawhide-20190622.n.1 - loop issue [0]
Fedora-Rawhide-20190623.n.0 - armv7 http2 framing error [1]
Fedora-Rawhide-20190624.n.0 - loop issue [0]
Fedora-Rawhide-20190626.n.0 - kde broken deps
Fedora-Rawhide-20190626.n.1 - loop issue [0]
Fedora-Rawhide-20190627.n.0 - loop issue [0]
Fedora-Rawhide-20190627.n.1 - pkg download issue [2]
Fedora-Rawhide-20190630.n.0 - armv7 http2 framing error[1]
Fedora-Rawhide-20190630.n.1 - a x86_64 live downloading issue [3]
Fedora-Rawhide-20190702.n.0 - pkg download issue [2]
[0] mock in old chroot has 5 loop devices available, if you try and use
more than that they just don't appear in the chroot. For some reason
loop devices aren't getting cleaned up as easily or we hit multiple
composes per builder and it goes over 5. I patched our mock to have 11
of them. :) See:
[1] This looks like:
DEBUG util.py:585: BUILDSTDERR: [MIRROR]
libavc1394-0.5.4-10.fc30.armv7hl.rpm: Curl error (16): Error in the
HTTP2 framing layer for
https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-2019062...
[]
DEBUG util.py:585: BUILDSTDERR: [FAILED]
libavc1394-0.5.4-10.fc30.armv7hl.rpm: No more mirrors to try - All
mirrors were already tried without success
DEBUG util.py:585: BUILDSTDERR: Unable to create appliance : Unable to
download from repo : Cannot download
Packages/l/libavc1394-0.5.4-10.fc30.armv7hl.rpm: All mirrors were tried
[2] This looks like:
(it's a screenshot, see:
https://kojipkgs.fedoraproject.org//work/tasks/3061/35873061/screenshot.ppm
) but basically:
Failed to download the following packages: Cannot download
Packages/s/systemd-bootchart-233-4.fc30.x86_64.rpm: All mirrors were tried.
[3] This looks like:
DEBUG util.py:585: BUILDSTDERR: 2019-07-01 03:02:48,081: Non
interactive installation failed: Failed to download the following
packages: Cannot download
Packages/i/iwl3945-firmware-15.32.2.9-97.fc31.noarch.rpm: All mirrors
were tried.
A short summary of our setup:
A builder, which might be running a mock chroot or a vm requests
something from kojipkgs.
kojipkgs resolves for them in dns to 2 ip's: proxy101 or proxy110.
Those run apache which proxies incoming requests for that host to
haproxy also running on those hosts.
haproxy checks for liveness of the two backend kojipkgs servers:
kojipkgs01 and kojipkgs02. The request will go to one of the two or
whichever is up.
kojipkgs01/02 have varnish listening on port 80, so if they have the
thing in cache is just replies with that. Otherwise it makes a query
against a locally running apache to fetch the item. That apache reads
the package from a nfs mount of all the koji data those machines have on
them.
So, lots of layers, but erverything should be pretty reliable.
And most of the time it is. The builders download tons and tons of
things just fine via this setup without problems usually.
At the point all these are failing, it would be running the rawhide
curl/dnf stack, not the native f29 one on the builders, so if it's a
curl problem I would expect it to happen more often.
So, how can we debug or mitigate this? any ideas?
kevin
4 years, 9 months
Retire keys.fedoraproject.org?
by Kevin Fenzi
Hey everyone,
As some of you may have read:
https://gist.github.com/rjhansen/67ab921ffb4084c865b3618d6955275f
and
https://dkg.fifthhorseman.net/blog/openpgp-certificate-flooding.html
or other media reports about vulnerabilities of the current gpg
keyserver software/network/policy.
TLDR: Someone can (and has been) flooding sks keyservers with poisoned
certs. Users that download from sks keyservers may well find gpg just
stops working, hangs, or breaks in terrible ways. The SKS software is no
longer maintained and because the policy is 'never delete anything'
there's likely no way to mitigate the attacks.
I've cc'ed nb here for his take on things, but as I read it, it might be
best to just retire the keys.fedoraproject.org service at least for now
to avoid breaking users or telling them we have a service they should
trust when they really... should not.
Thoughts?
kevin
4 years, 9 months
Re: Retire keys.fedoraproject.org?
by Julen Landa Alustiza
Clicking my gpg key on admin.fp.o ends on keys.fp.o . we should get rid of that link
Julen Landa Alustiza <jlanda(a)fedoraproject.org>
4 years, 9 months