infrastructure May 2015

infrastructure@lists.fedoraproject.org

55 participants
80 discussions

Freeze break request - change haproxy check URL for MirrorManager
by Adrian Reber 20 May '15

20 May '15

Can I get two +1 for this change? Switch to another URL for mirrormanager haproxy check The haproxy check URL for the MirrorManager web frontend was a URL which resulted in a large DB query. Every proxy, every minute. This resulted in two much memory and CPU consumption. This switches the check to a small static file to reduce the load on mm-frontend01. diff --git a/roles/haproxy/files/haproxy.cfg b/roles/haproxy/files/haproxy.cfg index 75bcf17..7a4b6cd 100644 --- a/roles/haproxy/files/haproxy.cfg +++ b/roles/haproxy/files/haproxy.cfg @@ -75,7 +75,7 @@ listen voting 0.0.0.0:10007 listen mirrormanager 0.0.0.0:10008 balance hdr(appserver) server mm-frontend01 mm-frontend01:80 check inter 60s rise 2 fall 3 - option httpchk GET /mirrormanager + option httpchk GET /mirrormanager/static/mirrormanager2.css listen bodhi 0.0.0.0:10009 balance hdr(appserver) diff --git a/roles/haproxy/files/haproxy.cfg.stg b/roles/haproxy/files/haproxy.cfg.stg index ef77c0d..6acc0fd 100644 --- a/roles/haproxy/files/haproxy.cfg.stg +++ b/roles/haproxy/files/haproxy.cfg.stg @@ -65,7 +65,7 @@ listen voting 0.0.0.0:10007 listen mirrormanager 0.0.0.0:10008 balance hdr(appserver) server mm-frontend01 mm-frontend01:80 check inter 60s rise 2 fall 3 - option httpchk GET /mirrormanager/ + option httpchk GET /mirrormanager/static/mirrormanager2.css listen bodhi 0.0.0.0:10009 balance hdr(appserver) Adrian

3 6

Freeze break request: push mirrormanager2-0.1.0-3
by Pierre-Yves Chibon 20 May '15

20 May '15

Hi all, Since MirrorManager2, the flask application incorporates the mirrorlist. We have seen this causing problem once in a while as the query that generates the list of mirror is pretty heavy. Patrick has been working on changing a little bit the layout of the mirrorlist so that we could cache it with varnish (the change is basically to remove the `login`/`logged in as XX | logout` from the top right corner of the templates). This way, people can login from the front page and will see all the pages as being logged in, or they can just see the cached pages or the mirrorlist. We were leaning to wait for after the freeze to push this, but Adrian Reber reported that the issue of mirrorlist being sometime un-available is causing problem with the report-mirror script. So I would like to ask for a freeze-break to push to MirrorManager2 the changes made by Patrick: https://github.com/fedora-infra/mirrormanager2/pull/80 While at it, I would like to push another fix, by Adrian, allowing to always mark as up to date, mirrors that are always up to date: https://github.com/fedora-infra/mirrormanager2/pull/67 These changes have been prepared via a 0.1.0-3 RPM release: http://koji.fedoraproject.org/koji/taskinfo?taskID=9804162 built from: https://github.com/fedora-infra/mirrormanager2/commit/b368e3aa8988367fcf101… Thanks, Pierre

4 4

Fed-clou02 migration
by Miroslav Suchý 20 May '15

20 May '15

Hi, as you know we have new Fedora Cloud instance. And we still have the *old* Fedora Cloud instance. I hereby declare fed-cloud02 a.k.a old Fedora Cloud as deprecated. There is currently 67 machines in running state. And bunch of VM in shutdown state. I would kindly ask all owners to: * not create new VM on fed-cloud02, but rather use fed-cloud09 * migrate your machines from fed-cloud02 to fed-cloud09 * terminate your machines on fed-cloud02, which you do not use (especially those under transient tenant). There is no hurry, we are under no press. However I would like to set up some dead line. Let say during June and July. During July I would like to gather list of remaining VMs and write personal email to its owners. In August - if there would be no reaction - I would suggest to power off (not terminate!) those remaining VMs and keep them for brief period. Sometime during fall terminate all machines and wipe old Fedora Cloud instance. Once again - this time-frame is just proposal as I would like to avoid having old Cloud instance running infinitely. If you have reason to have running it there and not migrating it, please raise your voice and we can alter the schedule. -- Miroslav Suchy, RHCA Red Hat, Senior Software Engineer, #brno, #devexp, #fedora-buildsys

3 2

Moving the codecs sticky post link on Ask Fedora to a sidebar to make it easier to find
by Ankur Sinha 20 May '15

20 May '15

Hi all, If you've been on the desktop ml, you'll know that we're trying to find ways to make installation of codecs and things easier. We already have a post on Ask Fedora that documents what needs to be done[1]. Unfortunately, askbot does not have a "pin question" or similar option and this very important question seems to get lost as a result. Is it OK if I add this question to the right sidebar under "Common Post installation tasks" or similar so that it jumps out to everyone that comes to the site? [1] https://ask.fedoraproject.org/en/question/9111/sticky-what-plugins -do-i-need-to-install-to-watch-movies-and-listen-to-music/ -- Thanks, Regards, Ankur Sinha "FranciscoD" http://fedoraproject.org/wiki/User:Ankursinha

2 2

Freeze break request: Use varnish for mirrormanager2 publiclist
by Patrick Uiterwijk 20 May '15

20 May '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Any +1s for the below patch? As noted in the commit message, this is mostly preferred after a new release is made (or the change in PR#80 is hotfixed), though it will work perfectly without, just confuse the user slightly, because whenever they're in /mirrors, they will always see like being logged out. This is currently already live in stg. commit e1a85426b83739e46c1e4f59e3b962738d0cbc24 Author: Patrick Uiterwijk <puiterwijk(a)redhat.com> Date: Tue May 19 16:32:38 2015 +0000 Use varnish for mirrormanager2 publiclist This change makes us use varnish for the mm2 publiclist and configures varnish to ignore cookies on the /mirrors subpath. The cookie ignore is only valid after the master-noauth Pull Request #80 is merged, as that hides all authed information from the publiclist pages. Signed-off-by: Patrick Uiterwijk <puiterwijk(a)redhat.com> diff --git a/playbooks/include/proxies-reverseproxy.yml b/playbooks/include/proxies-reverseproxy.yml index a55472f..904428a 100644 - --- a/playbooks/include/proxies-reverseproxy.yml +++ b/playbooks/include/proxies-reverseproxy.yml @@ -216,15 +216,6 @@ proxyurl: http://localhost:10009 - role: httpd/reverseproxy - - when: env != "staging" - - website: admin.fedoraproject.org - - destname: mirrormanager - - remotepath: /mirrormanager - - localpath: /mirrormanager - - proxyurl: http://localhost:10008 - - - - - role: httpd/reverseproxy - - when: env == "staging" website: admin.fedoraproject.org destname: mirrormanager remotepath: /mirrormanager diff --git a/roles/varnish/files/proxy.vcl b/roles/varnish/files/proxy.vcl index 37ca3da..ed8333b 100644 - --- a/roles/varnish/files/proxy.vcl +++ b/roles/varnish/files/proxy.vcl @@ -187,6 +187,10 @@ sub vcl_recv { unset req.http.cookie; set req.url = regsub(req.url, "\?.*", ""); } + if (req.url ~ "^/mirrormanager/mirrors") { + unset req.http.cookie; + set req.url = regsub(req.url, "\?.*", ""); + } } if (req.url ~ "^/mirrormanager2/") { set req.backend_hint = mirrormanager2; @@ -299,3 +303,13 @@ sub vcl_recv { # unset beresp.http.set-cookie; # } #} + + +# Make sure mirrormanager/mirrors doesn't set any cookies +# (Setting cookies would make varnish store a HIT-FOR-PASS +# making it always fetch from backend) +sub vcl_backend_response { + if (bereq.url ~ "^/mirrormanager/mirrors") { + unset beresp.http.set-cookie; + } +} -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBCgAGBQJVW2deAAoJEIZXmA2atR5QX58QAIyOKoRCasd7lWxkpHg8vlvr emasDLQ6+bG8UwERMU0Xbk85njzzAWBsUG6wbQKPkJRE+fHtoA2ZFOxrXD4o/ioF Uv+w6EwG8peltm3l/s9NOLxgzKOZoS/lULhfGshjxQVYXFcyexkNM5W82TyE7D9i 51wQ6pqcLiXeiWyzJGAA/MIA6zAm32bt+TAu7rq0uQSFcbKShhx9A6IHPlKRuELX U9ORY6Nh5HYqwE+FEkq5kQKkpiFMrohpffLBea4Q5urAxozb1uE1Nj4W5dP8b41H ZCPr3pbyFZEENJCC4MJ3wfhY7t4wMKUOEjyoEGte2l+rNT8mwchx/TMajOms803v /wTwG6Q4gbkaPbbxxEgjX9cNfzrvsahZE3n3oJm5ul55pVpMxBDqO6nBPfK1qqoi cVAurKTta6R6l7CIQC+t4BKFx3O6sfld/8eD4wiigHG0q7e61e5iZudGJkVaAy7m bz38ZgCHMwTzhjLAu4va0gNZZqhtJqpMnypR1ymbIrmsMi9/kMFh5QhnyI6CbR7r OY95yNxbAA2SyM7V+Ee9+L5FNd+6aX0jFG8bBZU06o4rZrZEg3BzdnLbNPApZoJs Q15RWRh9HWyZ8wMAg6u9PthPG4r3GrryksYSQ3s5SuArSvGWejZg9MX1r8wjbryP YetR6PlqZsGMs1/j55el =L8bm -----END PGP SIGNATURE-----

3 3

[release] pagure: 0.1.4, 0.1.5 and 0.1.6
by Pierre-Yves Chibon 20 May '15

20 May '15

Good morning everyone, Over the last two days I have been working on getting the milter up and running for pagure. This took a little more time than expected and a little more fixes as well. I had to change or add ome items in the configuration of pagure. While replying to new emails will work fine and add them to the ticket/PR as expected, replying to older emails won't work (as it did before). So from today on, you should be able to reply to a comment made on a ticket or a pull-request by directly replying to the email. If this does not work, a) ensure you have added the email your are using in pagure (cf your user settings page) and b) let me know! :) Here are the changelog for the three releases I made over the last two days to fix these issues and some other: * Wed May 20 2015 Pierre-Yves Chibon <pingou(a)pingoured.fr> - 0.1.6-1 - Update to 0.1.6 - Fix sending notification emails to multiple users, avoid sending private into to all of them * Tue May 19 2015 Pierre-Yves Chibon <pingou(a)pingoured.fr> - 0.1.5-1 - Update to 0.1.5 - Bug fix on the milter and the internal API endpoint * Tue May 19 2015 Pierre-Yves Chibon <pingou(a)pingoured.fr> - 0.1.4-1 - Update to 0.1.4 - Fix loading requests and tickets from git (allows syncing projects between pagure instances) - Add to the template .wsgi file a way to re-locate the tmp folder to work around a bug in libgit2 - Fix unit-tests suite - Adjust the spec file to install all the files required for the milters - Fix the `View` button on the pull-request pages Thanks, Pierre

1 0

Freeze break request: labs and arm sites for f22 release
by Kevin Fenzi 19 May '15

19 May '15

I'm sending this freeze break now, but likely won't apply it until later in the week. Also, there will be one more change before actual release when websites folks merge f22 branch to master (changing the puppet syncStatic.sh to pull labs and arm from master instead of f22 branch. They currently don't exist in master). This is needed to bring on line the labs.fedoraproject.org and arm.fedoraproject.org sites for f22 release. +1s? kevin -- In puppet: diff --git a/modules/fedora-web/files/syncStatic.sh b/modules/fedora-web/files/syncStatic.sh index de2845c..cba346c 100644 --- a/modules/fedora-web/files/syncStatic.sh +++ b/modules/fedora-web/files/syncStatic.sh @@ -45,11 +45,11 @@ cd /srv/web/fedora-web /usr/bin/git clean -q -fdx || exit 1 /usr/bin/git reset -q --hard || exit 1 -/usr/bin/git checkout -q master || exit 1 +/usr/bin/git checkout -q f22 || exit 1 /usr/bin/git pull -q --ff-only || exit 1 -build getfedora.org -build spins.fedoraproject.org +build labs.fedoraproject.org +build arm.fedoraproject.org pushd mirrors.fedoraproject.org > /dev/null rsync -qa --delete-after --delay-updates . /srv/web/mirrors.fedoraproject.org/ @@ -62,8 +62,9 @@ popd > /dev/null /usr/bin/git pull -q --ff-only || exit 1 +build getfedora.org +build spins.fedoraproject.org build boot.fedoraproject.org build fedoracommunity.org build fudcon.fedoraproject.org build start.fedoraproject.org in ansible: diff --git a/playbooks/include/proxies-fedora-web.yml b/playbooks/include/proxies-fedora-web.yml index d714a53..2170e42 100644 --- a/playbooks/include/proxies-fedora-web.yml +++ b/playbooks/include/proxies-fedora-web.yml @@ -33,10 +33,8 @@ website: getfedora.org - role: fedora-web/labs website: labs.fedoraproject.org - when: env == "staging" - role: fedora-web/arm website: arm.fedoraproject.org - when: env == "staging" # Some other static content, not strictly part of "fedora-web" goes below here - role: fedora-docs/proxy diff --git a/playbooks/include/proxies-websites.yml b/playbooks/include/proxies-websites.yml index 34fa6d6..c398a23 100644 --- a/playbooks/include/proxies-websites.yml +++ b/playbooks/include/proxies-websites.yml @@ -184,14 +184,12 @@ server_aliases: - labs.stg.fedoraproject.org cert_name: "{{wildcard_cert_name}}" - when: env == "staging" - role: httpd/website name: arm.fedoraproject.org server_aliases: - arm.stg.fedoraproject.org cert_name: "{{wildcard_cert_name}}" - when: env == "staging" - role: httpd/website name: boot.fedoraproject.org

3 2

Freeze Break Request: Whitelist copr-be for fedmsg
by Ralph Bean 19 May '15

19 May '15

copr-be.cloud.fedoraproject.org got a new IP in the upgrade going on today, so we need to change the whitelist listing for inbound fedmsg connections. I've already pushed the change out, but can I get two retroactive +1s? commit 30ab50c43b996952e191e9133c9f4ca0d0c5fac8 Author: Ralph Bean <rbean(a)redhat.com> Date: Tue May 19 15:15:36 2015 +0000 New IP for copr-be. diff --git a/inventory/group_vars/proxies b/inventory/group_vars/proxies index 23f8af3..bc0dfb8 100644 --- a/inventory/group_vars/proxies +++ b/inventory/group_vars/proxies @@ -47,7 +47,7 @@ custom_rules: [ # Allow jenkins.cloud to talk to the inbound fedmsg relay. '-A INPUT -p tcp -m tcp --dport 9941 -s 209.132.184.153 -j ACCEPT', # Allow copr-be.cloud to talk to the inbound fedmsg relay. - '-A INPUT -p tcp -m tcp --dport 9941 -s 209.132.184.131 -j ACCEPT', + '-A INPUT -p tcp -m tcp --dport 9941 -s 209.132.184.48 -j ACCEPT', # Also, ppc-composer.qa.fedoraproject.org (secondary arch) '-A INPUT -p tcp -m tcp --dport 9941 -s 209.132.181.33 -j ACCEPT', # Also, ppc-hub.qa.fedoraproject.org (secondary arch koji)

4 3

Freeze Break Request - fix redirect publiclist -> mirrormanager (part 2)
by Adrian Reber 19 May '15

19 May '15

I am not sure if this is necessary, but searching for 'publiclist fedora 20' returns following link: http://mirrors.fedoraproject.org/publiclist/Fedora/20/ which is currently not correctly rewritten/redirected to https://admin.fedoraproject.org/mirrormanager/mirrors/Fedora/20 The following patch fixes the rewrite (tested in staging). For full MM1 compatibility additional code changes are required: https://github.com/fedora-infra/mirrormanager2/pull/76 Without the above PR URLs like http://mirrors.fedoraproject.org/publiclist/Fedora http://mirrors.fedoraproject.org/publiclist/EPEL are not correctly rewritten/redirected. So I am not sure if this should also be applied as a hotfix. Can I get two +1 for the apache rewrite changes? And maybe some feedback if we need the changes from the PR mentioned above. Adrian commit fbe4525cf4b2589b3f44ad59429dd8625b4b5600 Author: Adrian Reber <adrian(a)lisas.de> Date: Fri May 15 21:06:18 2015 +0000 Add rewrite rule to access specific parts of the mirrorlist The MM1 publiclist links are supposed to still work. If an old publiclist link tries to directly access a product/version or product/version/arch combination an additional 'mirrors' needs to be inserted into the link. For full MM1 compatibility following PR is necessary: https://github.com/fedora-infra/mirrormanager2/pull/76 diff --git a/roles/mirrormanager/mirrorlist2/templates/mirrorlist-server.conf b/roles/mirrormanager/mirrorlist2/templates/mirrorlist-server.conf index 8ac75b2..5b877ae 100644 --- a/roles/mirrormanager/mirrorlist2/templates/mirrorlist-server.conf +++ b/roles/mirrormanager/mirrorlist2/templates/mirrorlist-server.conf @@ -3,9 +3,13 @@ # {% if env == "staging" %} Redirect /publiclist https://admin.stg.fedoraproject.org/mirrormanager +RewriteEngine On +RewriteRule ^/publiclist/(.+[^/])/?$ https://admin.stg.fedoraproject.org/mirrormanager/mirrors/$1 [R,L] {% else %} Redirect /publiclist https://admin.fedoraproject.org/mirrormanager Alias /static /var/lib/mirrormanager/mirrorlists/static/ +RewriteEngine On +RewriteRule ^/publiclist/(.+[^/])/?$ https://admin.fedoraproject.org/mirrormanager/mirrors/$1 [R,L] {% endif %} <Directory /var/lib/mirrormanager/mirrorlists>

3 5

Pagure prod (and stg) host key change
by Patrick Uiterwijk 19 May '15

19 May '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi all, I just found out that the SSH daemon for Pagure had not been restarted after the configuration file was put in place by Ansible (FBR for that will be sent soon). This means that when I just restarted the SSH Daemon on both prod and stg, it finally picked up the configured host key (and also PasswordAuthentication no, which was how we discovered about this). Because of this, the host keys that are being presented have changed. The new key fingerprints (RSA) are: Prod: 90:8e:7f:a3:f7:f1:70:cb:56:77:96:17:44:c4:fc:82 Stg: 69:50:46:24:c7:94:44:f8:8d:83:05:5c:eb:73:fb:c4 - -- With kind regards, Patrick Uiterwijk -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBCgAGBQJVWwKsAAoJEIZXmA2atR5QWiUQAJCpWUxbdrk9ivXFmjIcjl5J 39BwyM90Jw7LDT7s0PwoUeqXLlDU8iJBQym2iTdNVh6KfuQuxh292Sz/RutE9+HG KTi3Mld7OdbASQd943BEOX/PVjGZmxQDpXcBPWSo38jOgsZolP7+Ro4pvY466FE2 R/+RjhCfeIsn8I0BgPlYlVBDV1lJnY/B2YDsbwxh6PSEnOtU4WG1GrucHahQuEZb zawJ8xoDKVLInc42mf5lwrdg3vvkct+6tCJejO6JEFkjIIQmaf4BJNldP22/4lJ+ X4YiNvybLWdbW0KtR5TulxuHcxejD6kmA6nxeo0AVJ1jFMoBs2/JAmZ++el1t/YU 3QEYMw4TVIRRCrVcV/Divc4AJ4QR+8yiHNKA3x3Wevz0RVApmzbkVLXbvZbUmRRE sD3iSrI/oT9+jOQjT3cOSLRi7ZnpkV6s1V+c6mloCSWAQC55rl52ebD6uAydxDmn ZK9MhlCl4ZPV48u+Z+uYVB5KQphqRUscCsLzNxbfdaS07IobAk5+8iyEN4fWUd7C IwH129eWFSwUD0UDf6dTEePeKyS8M9k+XItDRYM0f6R9St8ZKFXmB+r86uVMpMes 3/MbFinZrAlAHd9EBlJSRnXHK/mQDOqWSCrFSUw5RkK6ccGIuO1hAdCWjfE60EKR o8bqo2TLMGvVsqbnu+ER =celu -----END PGP SIGNATURE-----

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure May 2015