The infrastructure team will be having it's weekly meeting tomorrow, 2013-04-18 at 19:00 UTC in #fedora-meeting on the freenode network.
Suggested topics:
#topic New folks introductions and Apprentice tasks.
If any new folks want to give a quick one line bio or any apprentices would like to ask general questions, they can do so in this part of the meeting. Don't be shy!
#topic Applications status / discussion
Check in on status of our applications: pkgdb, fas, bodhi, koji, community, voting, tagger, packager, dpsearch, etc. If there's new releases, bugs we need to work around or things to note.
#topic Sysadmin status / discussion
Here we talk about sysadmin related happenings from the previous week, or things that are upcoming.
#topic Private Cloud status update / discussion
#topic Upcoming Tasks/Items
#info 2013-04-02 to 2013-04-23 ALPHA infrastructure freeze #info 2013-04-22 - announce openid change on hosted. #info 2013-04-23 F19 alpha release #info 2013-04-24 - switch to new sigul servers. #info 2013-04-25 - switch hosted to openid #info 2013-05-01 nag fi-apprentices #info 2013-05-08 drop inactive apprentices #info 2013-05-14 to 2013-05-28 BETA infrastructure freeze #info 2013-05-28 F19 beta release #info 2013-05-31 end of 1st quarter #info 2013-06-01 nag fi-apprentices #info 2013-06-08 drop inactive apprentices #info 2013-06-18 to 2013-07-02 FINAL infrastructure freeze. #info 2013-07-01 nag fi-apprentices #info 2013-07-02 F19 FINAL release
#topic Open Floor
Submit your agenda items, as tickets in the trac instance and send a note replying to this thread.
More info here:
https://fedoraproject.org/wiki/Infrastructure/Meetings#Meetings
Thanks
kevin
On Wed, 17 Apr 2013 21:45:21 -0600 Kevin Fenzi kevin@scrye.com wrote:
The infrastructure team will be having it's weekly meeting tomorrow, 2013-04-18 at 19:00 UTC in #fedora-meeting on the freenode network.
Apologies, cannot make it.
============================================ #fedora-meeting: Infrastructure (2013-04-18) ============================================
Meeting started by nirik at 19:00:00 UTC. The full logs are available at http://meetbot.fedoraproject.org/fedora-meeting/2013-04-18/infrastructure.20... .
Meeting summary --------------- * welcome y'all (nirik, 19:00:01)
* New folks introductions and Apprentice tasks (nirik, 19:02:27)
* Applications status / discussion (nirik, 19:03:29) * threebean created a collectd thing for fedmsg: https://admin.fedoraproject.org/collectd/bin/index.cgi?hostname=busgateway01... (nirik, 19:06:52) * lots of work on datanommer this week as well upstream (nirik, 19:07:05) * Bugzilla upgrade is coming up very soon. We need to update stg to new python-bugzilla and test our apps asap. (nirik, 19:09:02)
* Sysadmin status / discussion (nirik, 19:14:06) * need to determine solution for username.id.stg.fedoraproject.org in stg, we have several possible, just need to see which is least bad. (nirik, 19:25:12) * we have reverse dns for cloud.fedoraproject.org (nirik, 19:25:25) * smooge to get new ppc box setup to replace ppc11/ppc12. (nirik, 19:25:43)
* Private Cloud status update / discussion (nirik, 19:27:08)
* Upcoming Tasks/Items (nirik, 19:30:32) * 2013-04-02 to 2013-04-23 ALPHA infrastructure freeze (nirik, 19:31:34) * 2013-04-22 - announce openid change on hosted. (nirik, 19:31:34) * 2013-04-23 F19 alpha release (nirik, 19:31:34) * 2013-04-24 - switch to new sigul servers. (nirik, 19:31:34) * 2013-04-25 - switch hosted to openid (nirik, 19:31:35) * 2013-05-01 nag fi-apprentices (nirik, 19:31:36) * 2013-05-08 drop inactive apprentices (nirik, 19:31:37) * 2013-05-14 to 2013-05-28 BETA infrastructure freeze (nirik, 19:31:40) * 2013-05-28 F19 beta release (nirik, 19:31:42)
* Open Floor (nirik, 19:32:31)
Meeting ended at 19:36:35 UTC.
Action Items ------------
Action Items, by person ----------------------- * **UNASSIGNED** * (none)
People Present (lines said) --------------------------- * nirik (87) * skvidal (39) * smooge (20) * puiterwijk (15) * pingou (12) * zodbot (4) * jsmith (4) * jerzyr (3) * cyberworm54 (2) * abadger1999 (2) * lmacken (2) * threebean (1) * relrod (1) * ricky (0) * mdomsch (0) * dgilmore (0) * CodeBlock (0) -- 19:00:00 <nirik> #startmeeting Infrastructure (2013-04-18) 19:00:00 <zodbot> Meeting started Thu Apr 18 19:00:00 2013 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:00:00 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 19:00:01 <nirik> #meetingname infrastructure 19:00:01 <zodbot> The meeting name has been set to 'infrastructure' 19:00:01 <nirik> #topic welcome y'all 19:00:01 <nirik> #chair smooge skvidal CodeBlock ricky nirik abadger1999 lmacken dgilmore mdomsch threebean 19:00:01 <zodbot> Current chairs: CodeBlock abadger1999 dgilmore lmacken mdomsch nirik ricky skvidal smooge threebean 19:00:11 * pingou here 19:00:12 * skvidal is here 19:00:13 * threebean here briefly 19:00:17 <jerzyr> here 19:00:19 * puiterwijk 19:00:21 * cyberworm54 is here 19:00:24 <skvidal> pingou: you beat me to the /me is here! 19:00:27 <skvidal> pingou: noooo! 19:00:36 <pingou> skvidal: I forgot the "is" :( 19:00:37 * skvidal will live in shame 19:00:41 <skvidal> SHAME 19:00:50 <nirik> :) 19:00:52 <jerzyr> present 19:00:53 * pingou learned from the best 19:00:57 * relrod here, but writing a paper 19:01:07 * abadger1999 here 19:01:29 <skvidal> pingou: to be fair I think lmacken was the best when he used to be able to make warren and katzj drop off the channel when he chimed in to rollcall 19:01:42 <pingou> skvidal: I can't compete with that! 19:01:44 <nirik> yeah, that was awesome. ;) 19:01:56 <skvidal> nirik: It took FOREVER to figure out how that was happening 19:01:59 <lmacken> haha 19:02:06 <nirik> good ol dircproxy bug. 19:02:07 <skvidal> lmacken: you remember that, right? 19:02:16 <lmacken> of course :) one of my best hacks 19:02:18 <smooge> here 19:02:20 <skvidal> hah 19:02:27 <skvidal> okay. down to business 19:02:27 <nirik> #topic New folks introductions and Apprentice tasks 19:02:37 <nirik> any new folks or apprentices with questions or comments? 19:02:57 <cyberworm54> >.> 19:03:21 * nirik listens to the sound of snow melting. 19:03:29 <nirik> #topic Applications status / discussion 19:03:40 <nirik> I had a few things here... 19:03:45 * pingou didn't do much this week 19:04:09 <nirik> unless threebean is still here and wants to mention his... 19:04:27 <nirik> pingou: is there still a mail issue on fedocal01.stg? or did we completely fix that now? 19:04:39 <pingou> nirik: I think there is 19:05:19 <smooge> so about summer of code stuff 19:05:33 <pingou> logs are empty but no emails received :( 19:05:42 <pingou> mailq is empty too :/ 19:05:56 <nirik> pingou: ok, I can try and help debug that at some point. 19:05:59 * nirik adds to his list. 19:06:16 <pingou> nirik: ah the cron just threw me an error 19:06:42 <pingou> nirik: something db related, let's check this out after the meeting 19:06:52 <nirik> #info threebean created a collectd thing for fedmsg: https://admin.fedoraproject.org/collectd/bin/index.cgi?hostname=busgateway01... 19:07:05 <nirik> #info lots of work on datanommer this week as well upstream 19:07:09 <nirik> pingou: ok. 19:07:13 <nirik> smooge: yeah? 19:08:36 <nirik> oh, I had another big one. ;( 19:08:56 <skvidal> ? 19:09:02 <nirik> #info Bugzilla upgrade is coming up very soon. We need to update stg to new python-bugzilla and test our apps asap. 19:09:20 <nirik> we should also point all stg stuff to parter-bugzilla if it's not already 19:10:39 * nirik listens to more snow melt. ;) 19:10:56 <nirik> hopefully we are already ready for the bz changes, but we should make sure in stg... 19:10:59 <abadger1999> grep -ri bugzilla in puppet or something. 19:11:05 <skvidal> nirik: we're having pollen-valanches here 19:11:05 <smooge> sorry I thought applications meant for application for projects versus the applications stuff 19:11:11 <skvidal> abadger1999: git grep 19:11:26 <puiterwijk> nirik: I have the commit for partner-bugzilla ready 19:11:52 <nirik> puiterwijk: cool. for only stg right? 19:12:04 <puiterwijk> I believe I sent it to you a while back to review, but didn't hear any of it afterwards 19:12:08 <puiterwijk> nirik: yeah, stg only 19:12:23 <nirik> ah, I must have dropped it somewhere, sorry. ;) Please do push it when you get a chance. ;) 19:12:42 <puiterwijk> sure, will do 19:13:23 <nirik> ok, any other applications news? 19:13:33 <nirik> note that we are still frozen until next tuesday... 19:13:48 * nirik needs to make sure to file the release tickets, I didn't get to it yet. 19:13:48 <puiterwijk> nirik: also the app/sys-admin stg stuff, I guess? 19:13:58 <nirik> yeah, was gonna do that in sysadmin 19:14:01 <puiterwijk> ah ok 19:14:06 <nirik> #topic Sysadmin status / discussion 19:14:23 <nirik> so, we have an issue with our stg setup... we want to move apps to use openid there. 19:14:37 <nirik> they reach out to 'username.id.stg.fedoraproject.org' 19:15:09 <nirik> but they cannot reach the outside ip of that due to firewall. 19:15:21 <nirik> we can't use /etc/hosts because we can't do wildcard's there. 19:16:09 <smooge> did you guys determine which firewall? 19:16:20 <nirik> smooge: yes, the rhit controlled one. 19:16:32 <nirik> to fix it we need to have them map every single stg host to a 1:1 nat 19:16:39 <nirik> which I really don't want to do 19:16:40 <puiterwijk> so my idea was to setup an ns01.stg which has its own database for stg.fp.o which the stg hosts use, and uses the normal paths for anything non-stg 19:16:59 <smooge> ugh 19:17:01 <puiterwijk> another idea I had was to add NAT entries to the stg boxes themselve 19:17:11 <skvidal> puiterwijk: split-brain dns aaaaaaaargh 19:17:14 <puiterwijk> smooge: yeah, I know it's ugly, but I'm just trying to come up with ideas... 19:17:17 <smooge> thinks about making id.stg.fedoraproject.org a box that sits at ibiblio 19:17:21 <puiterwijk> skvidal: no, no split-horizon DNS 19:17:38 <puiterwijk> skvidal: just another DNS server entirely 19:17:43 <skvidal> it's 2 different answers dependoing on who is asking 19:17:47 <nirik> I'm not sure about the local NAT... would have to play with that to see if it would really work, because not sure you can do wildcards there at all either. 19:17:53 <puiterwijk> skvidal: I know what split-horizon DNS is 19:18:03 <puiterwijk> nirik: you just redirect ip addresses 19:18:30 <puiterwijk> so it could translate 1.1.1.1 -> 192.168.1.1 (for example) before sending the packet to the NIC 19:19:02 <nirik> yeah, I suppose. 19:19:21 <nirik> ok, I'd like to ponder on this some more, but if someone has a better solution, feel free to suggest it anytime. ;) 19:20:14 <smooge> so why can't id.stg.fedoraproject.org sit at say ibiblio? 19:20:54 <nirik> well, several reasons: 19:21:09 <nirik> a) mixing stg in the same vhost as production stuff. 19:21:20 <nirik> b) would need proxy to talk to it and from it? 19:21:41 <nirik> it might work, but it seems messy to me 19:22:08 <skvidal> nirik: to be fair - what part of our staging environment IS NOT messy? 19:22:25 <nirik> well, yeah. 19:22:26 <skvidal> nirik: if we're going to have to do major forklifting to make this work - is there something else we could change to make this all simpler? 19:22:39 <nirik> smooge: it's a possibility, but I'd like to think about the implications of spreading that out 19:22:39 <smooge> move it all to the cloud? 19:22:44 <skvidal> can we make it all live outside of phx2? 19:23:01 <skvidal> can we stuff it in a cloud-in-a-box off of a spare system? 19:23:18 <smooge> well we don't have much in spare systems 19:23:20 <nirik> we could, but if this dns issue is a lot of work... you are asking for lots MORE work? ;) 19:23:41 <skvidal> nirik: well I was more thinking of longer-range but 19:23:42 <skvidal> yah 19:23:49 <skvidal> I understand where you're coming from 19:23:54 <smooge> I don't mind more lots work if it is lots work once :) 19:23:55 <nirik> yeah, understand... 19:23:57 <skvidal> so I will withdraw that sugestion 19:24:31 <smooge> now there is a GSOC project :).. build an implement a real dev and staging enviroment for us 19:24:31 <nirik> anyhow, we have options, but not clear on the least anoying one. ;) 19:24:41 <nirik> So, any other sysadmin news? 19:25:12 <nirik> #info need to determine solution for username.id.stg.fedoraproject.org in stg, we have several possible, just need to see which is least bad. 19:25:16 <nirik> oh, I have one: 19:25:20 <smooge> I am going to try and get a PPC working 19:25:25 <nirik> #info we have reverse dns for cloud.fedoraproject.org 19:25:29 <smooge> yay! 19:25:34 <pingou> cool :) 19:25:43 <nirik> #info smooge to get new ppc box setup to replace ppc11/ppc12. 19:25:47 <skvidal> nirik: we do? 19:26:02 <nirik> skvidal: we should? 19:26:07 * nirik meant to go back and check it. 19:26:08 <skvidal> dig isn't showing what I'd expect 19:26:16 <skvidal> which is to say - nothing at all 19:26:35 <nirik> hum. 19:26:56 <skvidal> but dig directly to our servers 19:26:57 <skvidal> and it works 19:26:57 <nirik> anyhow, we cna work on that out of band. ;( 19:27:00 <skvidal> yes 19:27:01 <skvidal> sorry 19:27:08 <nirik> #topic Private Cloud status update / discussion 19:27:11 <nirik> any cloud news? 19:27:26 <skvidal> nothing from me 19:28:01 <skvidal> we have 2 boxes ready to be added as compute nodes... just not done yet 19:28:11 <skvidal> smooge: any word on the disk for 03? 19:28:26 <smooge> O3? 19:28:33 <skvidal> fed-cloud03 19:28:40 <nirik> oh yeah, that one with the wacky disk. 19:28:40 <skvidal> the box with the wonky disk? 19:29:11 <smooge> ugh.. I don't know. I got focused on gnome last week and completely forgot where I left that 19:29:12 <smooge> sorry 19:29:24 <nirik> no hurry on it, if you could poke it again that would be good. 19:29:30 <smooge> I will do that before PPC 19:29:41 <nirik> once it's figured out we should reinstall it and 01 and setup 01/03 as a grizzley openstack instance. 19:30:24 <nirik> ok, moving on 19:30:27 <skvidal> smooge: no sweat - thx 19:30:32 <nirik> #topic Upcoming Tasks/Items 19:30:35 <skvidal> nirik: should we use rdo? 19:30:49 <nirik> skvidal: yeah, I think so. ;) 19:30:53 <skvidal> nirik: or should we do it by hand? 19:30:55 <skvidal> ok 19:31:07 <nirik> if it makes sense. I've not looked closely at rdo 19:31:34 <nirik> #info 2013-04-02 to 2013-04-23 ALPHA infrastructure freeze 19:31:34 <nirik> #info 2013-04-22 - announce openid change on hosted. 19:31:34 <nirik> #info 2013-04-23 F19 alpha release 19:31:34 <nirik> #info 2013-04-24 - switch to new sigul servers. 19:31:35 <nirik> #info 2013-04-25 - switch hosted to openid 19:31:36 <nirik> #info 2013-05-01 nag fi-apprentices 19:31:37 <nirik> #info 2013-05-08 drop inactive apprentices 19:31:40 <nirik> #info 2013-05-14 to 2013-05-28 BETA infrastructure freeze 19:31:42 <nirik> #info 2013-05-28 F19 beta release 19:31:44 <nirik> anything folks would like to note or schedule? 19:32:31 <nirik> #topic Open Floor 19:32:35 <nirik> any items for open floor? 19:33:48 <jsmith> nirik: Just a quick status update -- I should have a couple of dedicated servers for you guys in a the next month 19:33:57 <jsmith> nirik: Sorry it's taken so long :-( 19:33:57 <nirik> jsmith: oh, nice! 19:34:05 <pingou> cool :) 19:34:09 <smooge> what kind and where... and no problem 19:34:11 <nirik> no worries, those kinds of things are always welcome whenever they happen. 19:34:44 <jsmith> smooge: Quad-core Xeons, at least 4 gigs of ram (pushing for more), located in our data center in Utah 19:35:00 <jsmith> smooge: I can give you more specifics offline 19:35:04 <smooge> ah cool. thanks 19:35:17 <nirik> cool. I don't think we have anything in utah yet. 19:35:28 <smooge> road trip 19:35:49 <nirik> heh. 19:36:12 <jerzyr> utah jazz 19:36:28 <nirik> ok everyone. thanks for coming. Do continue in #fedora-admin, #fedora-noc and #fedora-apps. ;) 19:36:35 <nirik> #endmeeting
infrastructure@lists.fedoraproject.org