infrastructure September 2018

infrastructure@lists.fedoraproject.org

35 participants
56 discussions

suggested patch for review - issue 7158 - corrections
by Zach Villers 26 Sep '18

26 Sep '18

Happy Monday all. Here's an updated set of patches for issue 7158, with some of Kevin's comments on my previous attempt for context; On 9/18/18 5:44 PM, Kevin Fenzi wrote: > > Confusingly, we have a 'certbot' rule, but thats old and we should nuke > it. The new one is the 'letencrypt' role. It handles requesting a cert > from letsencrypt for the site it's called with and putting certs on any > other machines. > > So, keep the planet role as it is. > > Instead add to the playbooks/groups/people.yml the letencrypt role with > the site_name as fedoraplanet.org. Take a look at the taskotron.yml > playbook, I added this setup for taskotron-dev eariler today. > New people.yml patch adding certgetter role - tried to follow the taskotron-dev example; diff --git a/playbooks/groups/people.yml b/playbooks/groups/people.yml index e7661b4b4..77b34cb23 100644 --- a/playbooks/groups/people.yml +++ b/playbooks/groups/people.yml @@ -69,6 +69,7 @@ - cgit/make_pkgs_list - clamav - planet + - { role: letsencrypt, site_name: 'fedoraplanet.org' } - fedmsg/base - git/server >> ^/\.well-known/(.*)/srv/web/acme-challenge/.well-known/$1 [L] >> + RewriteRule "^/?(.*)" "https://certgetter01/$1" [L,R=301,NE] > > Change 'certgetter01' here to 'fedoraproject.org'. That will hit our > proxies and get proxied into certgetter. >> > > kevin > New planet.conf patch with above change; diff --git a/roles/planet/templates/planet.conf b/roles/planet/templates/planet.conf index 319923d2a..0875e7aa4 100644 --- a/roles/planet/templates/planet.conf +++ b/roles/planet/templates/planet.conf @@ -14,6 +14,11 @@ ErrorLog logs/planet-error.log CustomLog logs/fedoraplanet.org-access.log common + + # let certbot get an answer from certgetter01 + RewriteEngine on + RewriteRule ^/\.well-known/(.*)/srv/web/acme-challenge/.well-known/$1 [L] + RewriteRule "^/?(.*)" "https://fedoraproject.org/$1" [L,R=301,NE] UserDir disable AddCharset UTF-8 .xml @@ -79,3 +84,32 @@ RedirectMatch permanent /(.*) http://fedoraplanet.org/$1 </VirtualHost> +<VirtualHost {{public_ip}}:443 _default_:443> + ## + # Domain: fedoraplanet.org + # Owner: admin(a)fedoraplanet.org + # + ServerName fedoraplanet.org + + SSLEngine on + SSLCertificateFile /etc/letsencrypt/live/fedoraplanet.org/cert.pem + SSLCertificateKeyFile /etc/letsencrypt/live/fedoraplanet.org/privkey.pem + SSLCertificateChainFile /etc/letsencrypt/live/fedoraplanet.org/fullchain.pem + SSLHonorCipherOrder On + SSLCipherSuite RC4-SHA:AES128-SHA:ALL:!ADH:!EXP:!LOW:!MD5:!SSLV2:!NULL + SSLProtocol ALL -SSLv2 + + ServerAdmin admin(a)fedoraplanet.org + ServerName fedoraplanet.org + + DocumentRoot "/srv/planet/site/" + + ErrorLog logs/planet-error.log + CustomLog logs/planet.fedoraproject.org-access.log common + + UserDir disable + AddCharset UTF-8 .xml + + RedirectMatch permanent /(.*) http://fedoraplanet.org/$1 + +</VirtualHost> Thanks! Zach

2 2

Infrastructure Weekly Meeting Agenda for 2018-09-26
by Stephen John Smoogen 26 Sep '18

26 Sep '18

This shared document is for the next fedora infrastructure meeting. = Preamble = The infrastructure team will be having its weekly meeting tomorrow, 2018-09-27 at 15:00 UTC in #fedora-meeting-1 on the freenode network. We have a gobby document (see: https://fedoraproject.org/wiki/Gobby ) fedora-infrastructure-meeting-next is the document. Please try and review and edit that document before the meeting and we will use it to have our agenda of things to discuss. A copy as of today is included in this email. If you have something to discuss, add the topic to the discussion area with your name. If you would like to teach other folks about some application or setup in our infrastructure, please add that topic and your name to the learn about section. = Introduction = We will use it over the week before the meeting to gather status and info and discussion items and so forth, then use it in the irc meeting to transfer information to the meetbot logs. = Meeting start stuff = #startmeeting Infrastructure (2018-09-27) #meetingname infrastructure #topic aloha #chair nirik pingou puiterwijk relrod smooge tflink threebean cverna = Let new people say hello = #topic New folks introductions #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves = Status / Information / Trivia / Announcements = (We put things here we want others on the team to know, but don't need to discuss) (Please use #info <the thing> - your name) #topic announcements and information #info tflink is on extended PTO #info smooge is on PTO 2018-10-05->2018-10-08 #info nirik is on PTO 2018-10-26->2018-11-?? #info Beta Freeze is OVER. #info Final Freeze will start 2018-10-09 #info REBOOT MANIA NEXT WEEK. WIN A FREE BROKEN KERNEL #info openshift prod will be redeployed 2018-09-27 at 21UTC = Things we should discuss = We use this section to bring up discussion topics. Things we want to talk about as a group and come up with some consensus /suor decision or just brainstorm a problem or issue. If there are none of these we skip this section. (Use #topic your discussion topic - your username) #topic Oncall #info Nirik is on call from 2018-09-20->2018-09-27 #info Puiterwijk is on call from 2018-09-27->2018-10-04 #info ??? is on call from 2018-10-04 -> 2018-10-18 #info smooge is on call from 2018-10-18 -> 2018-10-25 #info Summary of last week: (from Nirik) #topic container registry and Quay.io - cverna #topic torrents and their troubles - kevin #info the only good torrent is someone else's. #topic #topic Tickets discussion #info https://pagure.io/fedora-infrastructure/report/Meetings%20ticket Go thru each ticket one by one = Apprentice office hours = #topic Apprentice Open office minutes #info A time where apprentices may ask for help or look at problems. Here we will discuss any apprentice questions, try and match up people looking for things to do with things to do, progress, testing anything like that. = Learn about some application or setup in infrastructure = (This section, each week we get 1 person to talk about an application or setup that we have. Just going over what it is, how to contribute, ideas for improvement, etc. Whoever would like to do this, just add the i/nfo in this section. In the event we don't find someone to teach about something, we skip this section and just move on to open floor.) #info = Meeting end stuff = #topic Open Floor #endmeeting -- Stephen J Smoogen.

1 0

production openshift outage tomorrow at 21UTC
by Kevin Fenzi 26 Sep '18

26 Sep '18

Hey folks. I just filed https://pagure.io/fedora-infrastructure/issue/7264 for an outage tomorrow of production openshift. I want to redeploy it with more compute nodes and enable stuff we have been playing with in staging. Normally we could just add the nodes and make the changes and keep everything going, but one of the changes I want to do is to enable multitenant networking and in staging I could not get things to switch to that without redeploying. ;( I also scheduled a 4 hour window for this, but hopefully it will be less than 30min, but you never know. Anyhow, if this causes any problems, please let me know asap. kevin

1 0

Fedora 29 Beta Freeze now in effect
by Kevin Fenzi 26 Sep '18

26 Sep '18

(sorry this email was delayed, it didn't seem to send yesterday when I thought I sent it). Greetings. we are now in the infrastructure freeze leading up to the Fedora 29 Beta release. This is a pre release freeze. We do this to ensure that our infrastructure is stable and ready to release the Fedora 29 Beta when it's available. You can see a list of hosts that do not freeze by checking out the ansible repo and running the freezelist script: git clone https://infrastructure.fedoraproject.org/infra/ansible.git ansible/scripts/freezelist -i inventory Any hosts listed as freezes is frozen until 2018-09-18 (or later if release slips). Frozen hosts should have no changes made to them without a sign-off on the change from at least 2 sysadmin-main or rel-eng members, along with (in most cases) a patch of the exact change to be made to this list. Thanks, kevin

1 1

FBR: Only use proxy101 for internal candidate-regsitry for now to debug
by Patrick マルタインアンドレアス Uiterwijk 25 Sep '18

25 Sep '18

Hi all, Can I get +1s for the following DNS repo change? This will make it simpler to debug issues with the candidate-registry from OSBS. Patrick commit 103f8de1f81264568d74e256ea45588d553ddd31 Author: Patrick Uiterwijk <puiterwijk(a)redhat.com> Date: Tue Sep 25 18:52:47 2018 +0000 Only use proxy101 for internal candidate-regsitry for now to debug Signed-off-by: Patrick Uiterwijk <puiterwijk(a)redhat.com> diff --git a/fedoraproject.org.template b/fedoraproject.org.template index 25a090b..e99083c 100644 --- a/fedoraproject.org.template +++ b/fedoraproject.org.template @@ -499,7 +499,11 @@ cdn.registry IN CNAME 1723040894.rsc.cdn77.org. registry IN CNAME wildcard stable-registry IN CNAME wildcard registry.stg IN CNAME admin.stg.fedoraproject.org. +{% if region == "PHX2" %} +candidate-registry IN CNAME proxy101.phx2.fedoraproject.org. +{% else %} candidate-registry IN CNAME wildcard +{% endif %} candidate-registry.stg IN CNAME admin.stg.fedoraproject.org. retrace02 IN A 209.132.181.34 retrace01 IN A 209.132.181.28

3 4

FBR: Add proxy101/proxy110 to the OSBS whitelsit
by Patrick マルタインアンドレアス Uiterwijk 25 Sep '18

25 Sep '18

Hi all, Can I get +1s for the patch underneath? This adds proxy101/proxy110 to the whitelist for OSBS for candidate-registry pushing. Patrick commit ef9fa10389a6a60843c96e4855dbdcb604583baa (HEAD -> master) Author: Patrick Uiterwijk <patrick(a)puiterwijk.org> Date: Tue Sep 25 21:34:39 2018 +0200 Whitelist proxy101/proxy110 for OSBS It turns out that these were never added to the OSBS whitelist, breaking about 50% of all builds.... Signed-off-by: Patrick Uiterwijk <patrick(a)puiterwijk.org> diff --git a/files/osbs/fix-docker-iptables.production b/files/osbs/fix-docker-iptables.production index ee03776a5..f1a50781c 100644 --- a/files/osbs/fix-docker-iptables.production +++ b/files/osbs/fix-docker-iptables.production @@ -46,6 +46,8 @@ iptables -A FILTER_FORWARD -p udp -m udp -d 10.5.126.21 --dport 53 -j ACCEPT iptables -A FILTER_FORWARD -p udp -m udp -d 10.5.126.22 --dport 53 -j ACCEPT # mirrors.fp.o +iptables -A FILTER_FORWARD -p tcp -m tcp -d 10.5.126.8 --dport 443 -j ACCEPT +iptables -A FILTER_FORWARD -p tcp -m tcp -d 10.5.126.9 --dport 443 -j ACCEPT iptables -A FILTER_FORWARD -p tcp -m tcp -d 10.5.126.51 --dport 443 -j ACCEPT iptables -A FILTER_FORWARD -p tcp -m tcp -d 10.5.126.52 --dport 443 -j ACCEPT @@ -53,6 +55,8 @@ iptables -A FILTER_FORWARD -p tcp -m tcp -d 10.5.126.52 --dport 443 -j ACCEPT iptables -A FILTER_FORWARD -p tcp -m tcp -d 10.5.126.23 --dport 443 -j ACCEPT # Kerberos +iptables -A FILTER_FORWARD -p tcp -m tcp -d 10.5.126.8 --dport 1088 -j ACCEPT +iptables -A FILTER_FORWARD -p tcp -m tcp -d 10.5.126.9 --dport 1088 -j ACCEPT iptables -A FILTER_FORWARD -p tcp -m tcp -d 10.5.126.51 --dport 1088 -j ACCEPT iptables -A FILTER_FORWARD -p tcp -m tcp -d 10.5.126.52 --dport 1088 -j ACCEPT

3 2

FBR: Disable OCSP on non-proxies
by Patrick マルタインアンドレアス Uiterwijk 25 Sep '18

25 Sep '18

Can I get +1s for the following patch? Explanation as to why this specific patch is in the commit message. commit d2688610419973cd519ef8f8f581a03215aaf73c (HEAD -> master) Author: Patrick Uiterwijk <patrick(a)puiterwijk.org> Date: Tue Sep 25 21:25:00 2018 +0200 Only do OCSP stapling on the proxies The actual cache is only set in the proxy HTTP config. While we could set the cache path in the other servers' configs as well, that would be a significantly larger change. Signed-off-by: Patrick Uiterwijk <patrick(a)puiterwijk.org> diff --git a/roles/httpd/website/templates/website.conf b/roles/httpd/website/templates/website.conf index 8ff23bbd0..afedf27eb 100644 --- a/roles/httpd/website/templates/website.conf +++ b/roles/httpd/website/templates/website.conf @@ -57,7 +57,9 @@ {% endif %} SSLEngine on - SSLUseStapling on + {% if ansible_hostname.startswith('proxy') %} + SSLUseStapling on + {% endif %} {% if certbot %} SSLCertificateFile /etc/pki/tls/certs/{{ site_name }}.cert SSLCertificateKeyFile /etc/pki/tls/private/{{ site_name }}.key

3 2

Meeting Agenda Item: Introduction Oliver Leaver-Smith
by Oliver Leaver-Smith 25 Sep '18

25 Sep '18

Hi Infrastructure Team! My name is Oliver and I am looking forward to getting involved with infrastructure for the Fedora Project. To ensure I don't miss any information, I'll simply follow the headers from the GettingStarted page on the infra wiki My IRC nick is muggahtee, my FAS login is ols I work as a Platform Engineer for Sky Betting and Gaming, previously I have worked in Networks and Infrastructure at other companies and ISPs I primarily have experience of administering RHEL and CentOS servers I am finally getting around to working on my RHCSA I am comfortable with shell, Ruby, Python and Golang as languages In terms of what I want to work on, I'm quite easy to be honest. I suppose the sensible thing to do is to have a nosey round the docs and infrastructure, as well as everything you have in git to see where my efforts would be best utilised I can contribute a few hours a week normally, though this may fluctuate throughout the year My timezone is Europe/London If you would like to talk to me on the social places, I am muggahtee on twitter and github Many thanks, and hope to speak to you all on Thursday ols PS apologies if this comes through twice, seems my initial email to the list did not take for some reason

1 0

FBR: add lists.pagure.io to our mailing-lists
by Aurelien Bompard 24 Sep '18

24 Sep '18

Hey folks! Pingou would like to announce the availability of mailing-lists on lists.pagure.io with the 5.0 release. The following patch should add the new domain to our mailing list server. Affected services are the mailman server and the proxies. Can I get a couple +1s? A. commit 46c844f7ad9dfbd824c49fcbd95be3575345d2df Author: Aurélien Bompard <aurelien(a)bompard.org> Date: Thu Sep 20 08:28:47 2018 +0000 Add lists.pagure.org to Mailman Signed-off-by: Aurélien Bompard <aurelien(a)bompard.org> diff --git a/playbooks/include/proxies-reverseproxy.yml b/playbooks/include/proxies-reverseproxy.yml index 8d35a5f..8461162 100644 --- a/playbooks/include/proxies-reverseproxy.yml +++ b/playbooks/include/proxies-reverseproxy.yml @@ -71,6 +71,15 @@ keephost: true proxyurl: "{{ varnish_url }}" + - role: httpd/reverseproxy + website: lists.pagure.io + destname: mailman3 + localpath: / + remotepath: / + header_scheme: true + keephost: true + proxyurl: "{{ varnish_url }}" + # The place for the raw originals - role: httpd/reverseproxy website: meetbot-raw.fedoraproject.org diff --git a/playbooks/include/proxies-websites.yml b/playbooks/include/proxies-websites.yml index 9c0e173..65597cd 100644 --- a/playbooks/include/proxies-websites.yml +++ b/playbooks/include/proxies-websites.yml @@ -901,6 +901,12 @@ - release-monitoring.org when: env == "staging" + - role: httpd/website + site_name: lists.pagure.io + sslonly: true + cert_name: lists.pagure.io.cert + SSLCertificateChainFile: lists.pagure.io.intermediate.cert + # fedorahosted is retired. We have the site here so we can redirect it. - role: httpd/website diff --git a/roles/base/files/postfix/main.cf/main.cf.smtp-mm b/roles/base/files/postfix/main.cf/main.cf.smtp-mm index 65e3cf7..3130cd0 100644 --- a/roles/base/files/postfix/main.cf/main.cf.smtp-mm +++ b/roles/base/files/postfix/main.cf/main.cf.smtp-mm @@ -305,7 +305,7 @@ unknown_local_recipient_reject_code = 550 # #relay_domains = $mydestination -relay_domains = $mydestination lists.fedoraproject.org lists.fedorahosted.org fedorahosted.org +relay_domains = $mydestination lists.fedoraproject.org lists.fedorahosted.org fedorahosted.org lists.pagure.io # INTERNET OR INTRANET diff --git a/roles/base/files/postfix/transports.mm-smtp b/roles/base/files/postfix/transports.mm-smtp index 582d455..ace4660 100644 --- a/roles/base/files/postfix/transports.mm-smtp +++ b/roles/base/files/postfix/transports.mm-smtp @@ -2,4 +2,5 @@ lists.fedoraproject.org smtp:[mailman01.vpn.fedoraproject.org] lists.fedorahosted.org smtp:[mailman01.vpn.fedoraproject.org] redhat.com smtp:[mailman01.vpn.fedoraproject.org] lists2.fedoraproject.org smtp:[mailman01.vpn.fedoraproject.org] +lists.pagure.io smtp:[mailman01.vpn.fedoraproject.org] fedorahosted.org smtp:[bastion.vpn.fedoraproject.org] diff --git a/roles/mailman/tasks/main.yml b/roles/mailman/tasks/main.yml index 41e3ff6..a665bc1 100644 --- a/roles/mailman/tasks/main.yml +++ b/roles/mailman/tasks/main.yml @@ -519,6 +519,14 @@ - restart memcached +# SSL +- name: Letsencrypt for lists.pagure.org + include_role: name=letsencrypt + vars: + site_name: lists.pagure.io + when: env == 'production' + + # Start services - name: start services service: state=started enabled=yes name={{ item }}

4 5

[Fedocal] Reminder meeting : Infra Office Hours
by cverna＠fedoraproject.org 24 Sep '18

24 Sep '18

Dear all, You are kindly invited to the meeting: Infra Office Hours on 2018-09-25 from 18:00:00 to 19:00:00 UTC At fedora-admin(a)irc.freenode.net The meeting will be about: Weekly hour dedicated to answer questions or help people with fixing tickets or implementing features. Source: https://apps.fedoraproject.org/calendar/meeting/9255/

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure September 2018