[FBR] Make the CI hosts not frozen
by Pierre-Yves Chibon
Good Morning Everyone,
Here is a patch to mark the CI host as not frozen for now. There is currently
not release-sensitive application running there and I believe we should be able
to mess with it during freeze.
+1?
Thanks,
Pierre
6 years, 11 months
FBR: patched libgit2 on pagure.io
by Kevin Fenzi
Greetings.
A while back Patrick made a patch for libgit2 that worked around some of
its performance and locking issues. When we upgraded pagure at the
beginning of the month we upgraded to the epel version without the patch.
I'd like to apply the patch to the libgit2 on pagure.io and hopefully
this will fix the performance/lockups we have left.
note that Patrick is working on more libgit2 fixes as well as these to
try and upstream them, so hopefully we don't have to carry these forever.
+1s?
kevin
6 years, 11 months
Weekly Koji Infra Tag Report
by Nobody
This is a list of packages in the various infrastructure koji tags
Please check and make sure there are not any that can be removed/dropped
epel6-infra
(no matching packages)
epel7-infra
Package Tag Extra Arches Owner
----------------------- ----------------------- ---------------- ---------------
pkgdb2 epel7-infra pingou
freeipa-ktutils epel7-infra puiterwijk
compose-utils epel7-infra ausil
fedmsg-beaker-repoupdate epel7-infra tflink
anitya epel7-infra jcline
the-new-hotness epel7-infra jcline
fedocal epel7-infra pingou
python-IPy epel7-infra kevin
python-robosignatory epel7-infra puiterwijk
pdc-updater epel7-infra ralph
python-pdc epel7-infra ralph
glusterfs epel7-infra kevin
kerneltest epel7-infra pingou
mirrormanager2 epel7-infra puiterwijk
blockerbugs epel7-infra tflink
python-django-jsonfield epel7-infra ralph
f23-infra
Package Tag Extra Arches Owner
----------------------- ----------------------- ---------------- ---------------
libphutil, f23-infra tflink
arcanist, f23-infra tflink
phabricator f23-infra tflink
phabricator-extension-ipsilonauth f23-infra tflink
libphutil f23-infra tflink
arcanist f23-infra tflink
f24-infra
Package Tag Extra Arches Owner
----------------------- ----------------------- ---------------- ---------------
mediawiki-openid f24-infra kevin
phabricator-extension-oauth f24-infra tflink
python-twill f24-infra codeblock
stickynotes2modernpaste f24-infra codeblock
python-flask-testing f24-infra codeblock
modern-paste f24-infra codeblock
mediawiki-skin-fedora f24-infra puiterwijk
mediawiki-FedoraBadges f24-infra kevin
basset f24-infra puiterwijk
phabricator f24-infra tflink
mediawiki-Lockdown f24-infra kevin
libphutil f24-infra tflink
arcanist f24-infra tflink
mediawiki-RSS f24-infra kevin
mirrormanager2 f24-infra puiterwijk
f25-infra
Package Tag Extra Arches Owner
----------------------- ----------------------- ---------------- ---------------
python-flask-testing f25-infra codeblock
modern-paste f25-infra codeblock
python-coveralls f25-infra codeblock
mdapi f25-infra pingou
basset f25-infra puiterwijk
mediawiki-FedoraBadges f25-infra kevin
mediawiki-Lockdown f25-infra kevin
mediawiki-RSS f25-infra kevin
mediawiki-openid f25-infra kevin
plus-plus-service f25-infra pingou
python-pdc f25-infra ralph
python-django-cors-headers f25-infra ralph
python-django-rest-framework-composed-permissions f25-infra ralph
patternfly1 f25-infra ralph
fas f25-infra kevin
libphutil, f25-infra tflink
arcanist, f25-infra tflink
phabricator f25-infra tflink
phabricator-extension-ipsilonauth f25-infra tflink
libphutil f25-infra tflink
arcanist f25-infra tflink
python-twill f25-infra codeblock
stickynotes2modernpaste f25-infra codeblock
mediawiki-skin-fedora f25-infra kevin
f26-infra
(no matching packages)
f27-infra
(no matching packages)
6 years, 11 months
FBR: Add monitoring for Pagure's celery redis queue
by Ricky Elrod
This is an attempt to add some simple monitoring so if the Pagure task
queue starts backing up, we know about it.
---
commit f207778a0e5aceba6c18885be7478317aaeab4b0
Author: Ricky Elrod <codeblock(a)fedoraproject.org>
Date: Fri May 26 23:09:02 2017 +0000
add simple monitoring for pagure's celery redis queue
Signed-off-by: Ricky Elrod <codeblock(a)fedoraproject.org>
diff --git a/roles/nagios_client/files/scripts/check_redis_queue.sh
b/roles/nagios_client/files/scripts/check_redis_queue.sh
new file mode 100644
index 0000000..ca1f186
--- /dev/null
+++ b/roles/nagios_client/files/scripts/check_redis_queue.sh
@@ -0,0 +1,23 @@
+#!/usr/bin/env bash
+
+. /usr/lib64/nagios/plugins/utils.sh
+
+if [[ "$#" -ne 3 ]]; then
+ echo "Arguments: key warn crit"
+ exit $STATE_UNKNOWN
+fi
+
+tasks="$(redis-cli llen "$1" | awk '{print $1}')"
+
+check_range $tasks $2:$3
+status=$?
+
+if [[ "$status" == "$STATE_OK" ]]; then
+ echo "OK: $1 queue has $tasks tasks"
+elif [[ "$status" == "$STATE_WARNING" ]]; then
+ echo "WARNING: $1 queue has $tasks tasks"
+elif [[ "$status" == "$STATE_CRITICAL" ]]; then
+ echo "CRITICAL: $1 queue has $tasks tasks"
+fi
+
+exit $status
diff --git a/roles/nagios_client/tasks/main.yml
b/roles/nagios_client/tasks/main.yml
index 714be36..22ddd4f 100644
--- a/roles/nagios_client/tasks/main.yml
+++ b/roles/nagios_client/tasks/main.yml
@@ -73,6 +73,7 @@
- check_osbs_builds.py
- check_osbs_api.py
- check_ipa_replication
+ - check_redis_queue.sh
when: not inventory_hostname.startswith('noc')
tags:
- nagios_client
@@ -146,6 +147,7 @@
- check_koschei_watcher_proc.cfg
- check_testcloud.cfg
- check_mirrorlist_docker_proxy.cfg
+ - check_celery_redis_queue.cfg
notify:
- restart nrpe
tags:
diff --git a/roles/nagios_client/templates/check_celery_redis_queue.cfg.j2
b/roles/nagios_client/templates/check_celery_redis_queue.cfg.j2
new file mode 100644
index 0000000..56279f3
--- /dev/null
+++ b/roles/nagios_client/templates/check_celery_redis_queue.cfg.j2
@@ -0,0 +1 @@
+command[check_celery_redis_queue]=/usr/lib64/nagios/plugins/check_redis_queue.sh
celery 5 10
diff --git a/roles/nagios_server/files/nagios/services/pagure_redis.cfg
b/roles/nagios_server/files/nagios/services/pagure_redis.cfg
new file mode 100644
index 0000000..d5387d0
--- /dev/null
+++ b/roles/nagios_server/files/nagios/services/pagure_redis.cfg
@@ -0,0 +1,6 @@
+define service {
+ host_name pagure01.fedoraproject.org
+ service_description Redis/celery queue
+ check_command check_by_nrpe!check_celery_redis_queue
+ use defaulttemplate
+}
6 years, 11 months
fix easyfix
by Kevin Fenzi
Our easyfix page is broken. First because it's trying to query
fedorahosted.org still, and for some reason it's not answering. Secondly
because it's calling the python-bugzilla connection thing wrong.
+1s to apply the below and run the sundries playbook?
kevin
--
6 years, 11 months
FBR: Significantly bump v4 ping RTA alert threshold on builders
by Ricky Elrod
These keep alerting, but we can't do anything about it, so bump the RTA.
This should curtail some of the alerts.
+1s?
commit 03916461a88a8b0c995032d3dfe7e719e2a20875
Author: Ricky Elrod <codeblock(a)fedoraproject.org>
Date: Thu May 25 18:14:13 2017 +0000
Bump ping4 builder RTA, since it keeps alerting and we can't do
anything about it
Signed-off-by: Ricky Elrod <codeblock(a)fedoraproject.org>
diff --git a/roles/nagios_server/files/nagios/services/ping.cfg
b/roles/nagios_server/files/nagios/services/ping.cfg
index 368db10..6ba317e 100644
--- a/roles/nagios_server/files/nagios/services/ping.cfg
+++ b/roles/nagios_server/files/nagios/services/ping.cfg
@@ -8,7 +8,7 @@ define service {
define service {
hostgroup_name buildvm-armv7, buildvm-s390x, buildvm-s390
service_description ICMP-Ping4-vm-builders
- check_command check_ping4!350.0,20%!1000.0,80%
+ check_command check_ping4!1500.0,20%!2500.0,80%
use criticaltemplate
}
6 years, 11 months
Plan for tomorrow's Fedora Infrastructure meeting 2017-05-25
by Kevin Fenzi
The infrastructure team will be having it's weekly meeting tomorrow,
2017-05-25 at 18:00 UTC in #fedora-meeting on the freenode network.
We have a gobby document
(see: https://fedoraproject.org/wiki/Gobby )
fedora-infrastructure-meeting-next is the document.
Please try and review and edit that document before the meeting and we
will use it to have our agenda of things to discuss. A copy as of today
is included in this email.
If you have something to discuss, add the topic to the discussion area
with your name. If you would like to teach other folks about some
application or setup in our infrastructure, please add that topic and
your name to the learn about section.
--
> This shared document is for the next fedora infrastructure meeting.
> = Introduction =
> We will use it over the week before the meeting to gather status and info and
> discussion items and so forth, then use it in the irc meeting to transfer
> information to the meetbot logs.
>
> = Meeting start stuff =
>
> #startmeeting Infrastructure (2017-05-25)
> #meetingname infrastructure
> #topic aloha
> #chair smooge relrod nirik abadger1999 dgilmore threebean pingou puiterwijk pbrobinson
>
> = Let new people say hello =
>
> #topic New folks introductions
>
> = Status / information / Trivia / Announcements =
>
> (We put things here we want others on the team to know, but don't need to discuss)
> (Please use #info <the thing> - your name)
>
> #topic announcements and information
> #info we are now in f26 Beta freeze - everyone
> #info freeze rules are outlined here: https://docs.pagure.org/infra-docs/sysadmin-guide/sops/fedora-releases.html
> #info A number of new pagure releases, hopefully getting us to speedy and stable - patrick/pingou
> #info
>
> = Things we should discuss =
>
> We use this section to bring up discussion topics. Things we want to talk about
> as a group and come up with some consensus /suor decision or just brainstorm a
> problem or issue. If there are none of these we skip this section.
> (Use #topic your discussion topic - your username)
>
> #topic Apprentice work day recap - kevin
>
> #topic FLOCK! - kevin
>
> #topic
>
> #topic
>
> = Apprentice office hours =
>
> #topic Apprentice Open office hours
>
> Here we will discuss any apprentice questions, try and match up people looking
> for things to do with things to do, progress, testing anything like that.
>
> = Learn about some application or gsetup in infrastructure =
>
> (This section, each week we get 1 person to talk about an application or setup
> that we have. Just going over what it is, how to contribute, ideas for improvement,
> etc. Whoever would like to do this, just add the i/nfo in this section. In the
> event we don't find someone to teach about something, we skip this section
> and just move on to open floor.)
>
> #topic Learn about:
>
> = Meeting end stuff =
>
> #topic Open Floor
>
> #endmeeting
6 years, 11 months
Apprentice Work day tomorrow! (2017-05-23)
by Kevin Fenzi
Just a reminder that we will be having a apprentice work day tomorrow
(2017-05-23).
Folks are welcome to drop by #fedora-admin and we will look at finding
people things to work on, answer general quesions about accessing things
and how to contribute and hopefully get some things done. :)
* We may look at our existing apps where we are upstream and work on
triageing bugs/issues or just learning how they work.
* We may work on docs and getting things reviewed and merged or new
corrections landed.
I'm going to try and be around all day for me (14UTC to 23UTC).
Hope to see folks there!
kevin
6 years, 11 months
