============================================ #fedora-meeting: Infrastructure (2012-03-29) ============================================

Meeting started by nirik at 20:00:06 UTC. The full logs are available at http://meetbot.fedoraproject.org/fedora-meeting/2012-03-29/infrastructure.20... .

Meeting summary --------------- * Robot Roll Call (nirik, 20:00:06)

* New folks introductions and Apprentice tasks. (nirik, 20:02:23)

* two factor auth status (nirik, 20:03:54) * narrowing in on deployable solution. (nirik, 20:07:18) * will not integrate with fas for first cut, but after proof of concept is working. (nirik, 20:10:28)

* Staging re-work status (nirik, 20:14:55)

* Applications status / discussion (nirik, 20:15:17) * LINK: http://mm3test.fedoraproject.org/ (abadger1999, 20:17:11)

* Upcoming Tasks/Items (nirik, 20:27:37) * 2012-03-20 to 2012-04-10 - F17 Beta Freeze (nirik, 20:28:03) * 2012-03-29 - take internetx01 out of rotation and power off (nirik, 20:28:03) * 2012-03-30 - 1:30am - run diag on internetx01. (nirik, 20:28:03) * 2012-04-01 - nag fi-apprentices. (nirik, 20:28:03) * 2011-04-03 - gitweb-cache removal day. (nirik, 20:28:03) * 2012-04-10 - drop inactive fi-apprentices (nirik, 20:28:03) * 2012-04-10 - F17Beta release day (nirik, 20:28:07) * 2012-05-01 to 2012-05-15 - F17 Final Freeze. (nirik, 20:28:09) * 2012-05-01 - nag fi-apprentices. (nirik, 20:28:11) * 2012-05-15 - F17 release (nirik, 20:28:15)

* Tickets from Ages past (nirik, 20:31:47) * LINK: https://fedorahosted.org/fedora-infrastructure/report/1?sort=created&asc... is our full list by filed time. (nirik, 20:33:02) * LINK: https://fedorahosted.org/fedora-infrastructure/ticket/1783 looks like something we could close. (nirik, 20:33:53)

* Open Floor (nirik, 20:35:01) * ACTION: nirik to clarify ticket and commits workflow for apprentices on wiki. (nirik, 20:43:08)

Meeting ended at 21:01:53 UTC.

Action Items ------------ * nirik to clarify ticket and commits workflow for apprentices on wiki.

Action Items, by person ----------------------- * nirik * nirik to clarify ticket and commits workflow for apprentices on wiki. * **UNASSIGNED** * (none)

People Present (lines said) --------------------------- * skvidal (104) * nirik (100) * mdomsch_ (19) * smooge (15) * kwame (15) * abadger1999 (11) * pingou (6) * wolfkit (5) * zodbot (5) * CodeBlock (2) * ianweller (1) * adrianhannah (1) * dgilmore (1) * lmacken (0) * ricky (0) * mdomsch (0) * Codeblock (0) -- 20:00:06 <nirik> #startmeeting Infrastructure (2012-03-29) 20:00:06 <nirik> #meetingname infrastructure 20:00:06 <nirik> #topic Robot Roll Call 20:00:06 <nirik> #chair smooge skvidal Codeblock ricky nirik abadger1999 lmacken dgilmore mdomsch 20:00:06 <zodbot> Meeting started Thu Mar 29 20:00:06 2012 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:00:06 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 20:00:06 <zodbot> The meeting name has been set to 'infrastructure' 20:00:06 <zodbot> Current chairs: Codeblock abadger1999 dgilmore lmacken mdomsch nirik ricky skvidal smooge 20:00:09 <CodeBlock> here 20:00:16 * wolfkit is here 20:00:17 * skvidal is here 20:00:18 <nirik> who all is around for a infra meeting? fun and exciting! :) 20:00:46 * kwame here 20:01:02 * mdomsch_ 20:01:07 * adrianhannah is here 20:01:23 <smooge> is here 20:02:12 <nirik> ok, lets dive on in... 20:02:23 <nirik> #topic New folks introductions and Apprentice tasks. 20:02:23 <nirik> If any new folks want to give a quick one line bio or any apprentices 20:02:23 <nirik> would like to ask general questions, they can do so now. Anyone? 20:02:33 <dgilmore> hey yall 20:02:54 <nirik> morning dgilmore 20:03:13 * nirik will wait a few on this then move on... 20:03:34 * abadger1999 here 20:03:34 * pingou here 20:03:54 <nirik> #topic two factor auth status 20:04:05 <nirik> anything new to report here? wolfkit ? skvidal ? 20:04:09 <skvidal> yes 20:04:17 <skvidal> bressers has passed me a sheaf of patches 20:04:22 <skvidal> fixing a number of things in pam_url 20:04:24 <nirik> awesome. 20:04:36 <wolfkit> skvidal: great, is there a place I can get these patches too? 20:04:37 <skvidal> I'm going to roll those in and let mr wolfkit take a look and mricon@kernel.org 20:04:42 <skvidal> wolfkit: there will be shortly :) 20:04:47 <wolfkit> cool :) 20:04:55 <skvidal> also I spoke with mricon today about the totp.cgi 20:05:07 <skvidal> he has some folks testing it out and pam_url out in "production" 20:05:15 <skvidal> so not _Really_ production but in real life 20:05:21 <nirik> nice. 20:05:24 <skvidal> and there are some issues - but perhaps solveable ones 20:05:37 <skvidal> I think we're moving in on a solution 20:05:43 <nirik> ok. If it would help we could setup a test instance somewhere too... 20:05:49 <skvidal> I think it will 20:05:58 <skvidal> but I am glad to have responses back from bressers 20:06:13 <skvidal> b/c it feels like we're doing the right "due diligence" on something so security sensitive as this 20:06:21 <skvidal> kwim? 20:06:29 <nirik> yes. We don't want to deploy something that messy/insecure for sure. 20:06:45 <skvidal> I have a test setup on my local system 20:06:50 <skvidal> that is just a couple of vm's 20:06:59 <skvidal> I may see about doing the same on a junk box 20:07:06 <nirik> sounds good. 20:07:10 <skvidal> also 20:07:18 <nirik> #info narrowing in on deployable solution. 20:07:19 <skvidal> icon has made the totp.cgi so it takes an optional pin 20:07:26 <skvidal> so instead of just being the otp 20:07:31 <skvidal> it can be pin + otp 20:07:49 <nirik> good. 20:07:57 <skvidal> so a couple of things 20:08:03 <skvidal> if we want to make 2fa for the world 20:08:10 <skvidal> then, obviously, we're going to need to tie this into fas 20:08:25 <skvidal> but as a deployable/functional prototype for sysadmin-* or whatnot 20:08:34 <skvidal> I think we can probably get a way without the fas integration 20:08:47 <skvidal> and I'm sure it won't break abadger1999's heart to not have to deal with a fas rfe :) 20:08:52 <nirik> yep. Just a lookup of pin and secret... 20:08:55 <abadger1999> <nod> 20:08:56 <skvidal> until we know it's what we want 20:09:06 <skvidal> does anyone object to that? 20:09:21 <nirik> I think thats fine for a first cut... 20:09:40 <nirik> if it all runs nicely and we are ready to expand, then we can look at how best to add to fas. 20:09:42 <wolfkit> yeah, no reason to bother integrating into FAS until it's ready for 'primetime' :) 20:10:04 <skvidal> nod 20:10:08 <skvidal> for fas 20:10:15 <skvidal> it should be relatively straightforward 20:10:21 <skvidal> except we'd be storing actual seeds _somewhere_ 20:10:25 <skvidal> which is the sensitive part 20:10:28 <nirik> #info will not integrate with fas for first cut, but after proof of concept is working. 20:10:41 <skvidal> abadger1999: hmmm 20:10:44 <nirik> yeah, can see how best to store those... 20:10:47 <skvidal> abadger1999: could we encode the seed in fas with the pin? 20:11:06 <skvidal> abadger1999: so if the pin is wrong or the otp is wrong you get rejected? 20:11:29 <skvidal> abadger1999: it wouldn't be A LOT of security if our db was compromised 20:11:32 <skvidal> but it might slow someone down 20:12:14 <skvidal> anyway 20:12:16 <skvidal> that's off in the weeds 20:12:33 <nirik> yeah, we can figure that out more when we get there. ;) 20:12:36 * smooge gets the weedwhacker 20:12:47 <nirik> thanks for the news and work on this skvidal and icon and wolfkit. 20:12:48 <skvidal> heh 20:12:55 <skvidal> nirik: and bressers 20:12:59 <skvidal> he's been very helpful 20:13:09 <nirik> yes. agreed. 20:13:12 <abadger1999> I guess we could but it would be trivial to reverse 20:14:30 <skvidal> well only if the pin is kept in the clear 20:14:37 <skvidal> anyway 20:14:42 <nirik> right, moving along. 20:14:55 <nirik> #topic Staging re-work status 20:15:11 <nirik> I've done some docs on this, still need to work on it more. and will do it after freeze is over. 20:15:17 <nirik> #topic Applications status / discussion 20:15:22 <nirik> any application news? 20:15:36 <nirik> abadger1999 / lmacken / pingou / threebean / CodeBlock 20:16:14 <wolfkit> haven't been around much the past few days, did anyone figure out what happened with mirrormanager wednesday morning? 20:16:41 * nirik isn't sure. 20:16:42 <mdomsch_> wolfkit, logs were blissfully unenlightening 20:17:00 <abadger1999> pingou has been doing some *great* work on the mailman3 archiver instance 20:17:11 <abadger1999> http://mm3test.fedoraproject.org/ 20:17:27 <nirik> oh, I have one bit of news... I went and triaged all the bodhi bugs. Closed dups, closed things that were done, etc... cut them down some. I also added easyfix to some tickets that sound like they would be easy to fix for new folks. 20:17:43 <nirik> nice! 20:17:50 <CodeBlock> I have nothing new for dpsearch. I tried pinging damaestro the other day but he was at dayjob and I didn't have a chance to ping him later that evening. I will try to catch up with him either tonight after class or tomorrow evening (tomorrow is probably better). 20:17:52 <abadger1999> We may be getting a GSoC student to work on that -- I'm trying to start discussion with terri of the mailman devs about who's going to mentor and which org is going to sponsor and such 20:18:23 <pingou> I hope to be able to couple HK with mm3 this week-end (if abadger1999 has some time as well) 20:18:38 <abadger1999> pingou's latest addition to the archiver is to start implementing a REST API: http://mm3test.fedoraproject.org/api/ 20:19:22 <nirik> pingou: HK? 20:19:29 <pingou> HyperKitty :) 20:19:44 <nirik> ah, right 20:20:37 <skvidal> so 20:20:42 <skvidal> mm3test has been useful 20:20:51 <skvidal> do we want to leave it as is 20:20:56 <skvidal> or setup another instance 20:21:40 <skvidal> ? 20:21:49 <nirik> oh, also we have some smolt news: npmccallum has setup a new project called 'census' that will replace much of smolt. https://fedorahosted.org/census/browser 20:22:04 <abadger1999> skvidal: at some point I think we want another instance but -- things are still very very developy atm 20:22:07 * nirik is fine on mm3test with whatever works for the people working on it. 20:22:09 <skvidal> abadger1999: if you'd like to leave mm3test as is b/c it lets you move more fluid that's fine 20:22:15 <skvidal> abadger1999: gotcha 20:22:16 <skvidal> absolutely 20:22:23 <skvidal> the machine is automatically applying patches/rpms 20:22:29 <abadger1999> we're doing development in a virtualenv -- so some things aren't packaged and such. 20:22:31 <skvidal> and it is pretty isolated 20:22:37 <skvidal> abadger1999: I completely understand 20:22:42 <skvidal> and am supportive of that 20:22:47 <skvidal> I just wanted to help if you needed help 20:22:51 <abadger1999> <nod> 20:22:57 <abadger1999> Thanks 20:23:03 <skvidal> cool 20:23:56 <pingou> it don't think it will be too hard to get packaged, the dependency list isn't too high I think 20:23:57 <nirik> ok, any other appy news? 20:25:38 <smooge> hey I wonder if we could make the hosted mail mm3? 20:25:44 <smooge> it is the smaller of the listsets 20:25:50 <skvidal> smooge: I wouldn't want mm3 in 'production' yet 20:25:53 <nirik> once it's usable, yeah... 20:26:10 <smooge> i am not talking production.. I am talking hosted :) 20:26:21 <smooge> but yeah I understand 20:26:30 <pingou> smooge: mm3 core is just beta - the admin web ui alpha 20:26:55 <smooge> ah I thought it might be beta through and through 20:26:59 <smooge> okie dokie 20:27:25 <pingou> and the archiver pre-alpha-rc 20:27:28 <nirik> ok, moving along then... 20:27:37 <nirik> #topic Upcoming Tasks/Items 20:28:00 <nirik> ok, we have slipped a week on beta, which slips the entire schedule a week. 20:28:03 <nirik> #info 2012-03-20 to 2012-04-10 - F17 Beta Freeze 20:28:03 <nirik> #info 2012-03-29 - take internetx01 out of rotation and power off 20:28:03 <nirik> #info 2012-03-30 - 1:30am - run diag on internetx01. 20:28:03 <nirik> #info 2012-04-01 - nag fi-apprentices. 20:28:03 <nirik> #info 2011-04-03 - gitweb-cache removal day. 20:28:03 <nirik> #info 2012-04-10 - drop inactive fi-apprentices 20:28:07 <nirik> #info 2012-04-10 - F17Beta release day 20:28:09 <nirik> #info 2012-05-01 to 2012-05-15 - F17 Final Freeze. 20:28:11 <nirik> #info 2012-05-01 - nag fi-apprentices. 20:28:15 <nirik> #info 2012-05-15 - F17 release 20:28:15 <nirik> also, we have some upcoming maint on internetx01. 20:28:24 <nirik> It's been locking up, so we want to take it down and run a hardware test on it. 20:29:40 <nirik> I am going to take it's proxy out and power it off for that later tonight. 20:29:52 <nirik> anything else anyone would like to schedule or note? 20:30:02 <skvidal> nothing leaps to mind 20:30:15 <skvidal> I've been working more with ansible and its playbooks 20:30:26 <skvidal> and I have the basis for something pretty workable for the builders 20:30:32 <skvidal> if anyone wants to get involved and check it out 20:30:52 <skvidal> ping at me or check out ansible on github or #ansible 20:31:20 <nirik> sounds good. 20:31:47 <nirik> #topic Tickets from Ages past 20:32:04 <nirik> anyone want to bring up some ancient tickets we can discuss and dispatch? 20:33:02 <nirik> https://fedorahosted.org/fedora-infrastructure/report/1?sort=created&asc... is our full list by filed time. 20:33:08 * skvidal looks at the ground and whistles 20:33:09 <nirik> oh, I also filed the F17 beta tickets. 20:33:53 <nirik> https://fedorahosted.org/fedora-infrastructure/ticket/1783 looks like something we could close. 20:34:08 <skvidal> +1 20:34:13 <kwame> :fit 20:35:01 <nirik> #topic Open Floor 20:35:08 <nirik> anyone have any general items for open floor? 20:35:29 <kwame> o/ 20:35:44 <skvidal> kwame: is that your hand up? 20:35:50 <kwame> so, I've been in the #fedora-admin channel for some week snow 20:35:55 <kwame> skvidal: yes :) 20:35:58 <skvidal> :) 20:36:14 <kwame> and already hace access to some boxes 20:36:34 <kwame> last week I spoke with nirik about a ticket that I thought I could work on 20:36:40 <smooge> I changed people's email passwords and am working on a blog about 20:36:44 <smooge> its slow going. 20:36:56 <smooge> oh sorry. shuts up for kwame. 20:37:41 <kwame> sorry 20:37:59 <kwame> so, my question is basically, how can we know who are the new commers in here (like me) 20:38:29 <nirik> good question... we don't keep a very good list or whatnot of new folks... 20:38:59 <nirik> we do have the fi-apprentice group. You can mostly be sure anyone in the group as a user is new, if they are a sponsor or admin they have been around a while... 20:39:10 <nirik> but that doesn't cover everyone. 20:39:16 <nirik> .members fi-apprentice 20:39:16 <zodbot> nirik: Members of fi-apprentice: adrianhannah aphukan arielb +codeblock ctria davidvz fortu icon informatux jacibato jsh @kevin kubo mangas marcelk +nb @skvidal smillie @smooge +toshio 20:39:56 <nirik> Open to ideas on improving things... 20:40:14 <kwame> nirik: one more question, the ticket I tried to work on was very simple, just install nrpe and get a new nrpe.cfg in place 20:40:40 <kwame> now, who should I ask about access, and just for +1 or -1 on how to access and do that kind of work? is the mailing list the proper place to do it? 20:40:43 <kwame> or just irc? 20:41:36 <nirik> which ticket? you're welcome to ask on irc... usually for apprentices we ask that you attach to the ticket a patch against our puppet repo that makes the change or fix. Then someone will apply it if all looks well. 20:42:31 <nirik> we could definitely clarify that workflow tho. 20:42:39 <kwame> ok 20:43:08 <nirik> #action nirik to clarify ticket and commits workflow for apprentices on wiki. 20:44:14 <kwame> nirik: so there's already a wiki for commits workflow for apprentices ? 20:44:14 <nirik> kwame: let me make some changes and you can review them to see if they explain things or make sense. ;) 20:44:41 <mdomsch_> how much do we care about s3-mirror logs? 20:45:00 <mdomsch_> I've started the module to collect, process, and analyze them in awstats 20:45:08 <mdomsch_> on log02 20:45:25 <mdomsch_> but they're kind of large, and growing at a couple hundred MB a week 20:45:31 <nirik> cool. I'm wondering if we shouldn't pick something better than awstats... I'm not all that impressed with it. 20:45:54 <mdomsch_> open to suggestions 20:45:56 <nirik> mdomsch_: even compressed? 20:45:59 <smooge> what else is there these days? 20:46:02 <mdomsch_> a perfect apprentice project 20:46:13 <smooge> I can look at other stats issues or help an apprentice 20:46:14 <kwame> lol 20:46:18 <mdomsch_> nirik, it's thousands of very small files, with a few hits per file 20:46:55 <nirik> weird. can we just cat them and compress that. ;) 20:47:57 <mdomsch_> nirik, yes... 20:47:58 <nirik> ideally we just want the data in some nice format and can toss the little files after a while. 20:48:13 <mdomsch_> I have S3 automatically nuking its copy of them after 60 days 20:48:24 <nirik> smooge: might want to see if ianweller's stats thing could take over awstats functions too... 20:48:27 <mdomsch_> can adjust as necessary 20:48:38 <smooge> ok will do so 20:48:46 <ianweller> it could 20:48:48 <mdomsch_> the most interesting bit of the stats is "what content is being requested" 20:48:57 <mdomsch_> e.g. Fedora X, EPEL Y 20:49:02 <mdomsch_> the IPs aren't 20:49:04 <nirik> yeah. 20:49:18 <mdomsch_> so lightweight, lighter than awstats, would be fine 20:49:29 * mdomsch_ has been using cat/awk/sort/grep/uniq 20:49:47 <nirik> yeah. It would be nice to see things like which fedorapeople repos are most popular, etc too... 20:50:05 <skvidal> oh 20:50:07 <skvidal> speaking of that 20:50:07 <nirik> but I guess we can see whats out and easy to deploy 20:50:17 <skvidal> do we want to move the /srv/repos dir on fedorapeople over to /projects 20:50:23 <skvidal> so people aren't quota-limited there? 20:50:53 <skvidal> it was something I didn't think of during the people migration 20:50:58 <skvidal> but after looking at it 20:51:01 <skvidal> I realized it kinda makes sense 20:52:03 <nirik> how big is that? 20:52:06 <nirik> might be nice... 20:52:09 * skvidal looks 20:52:21 <skvidal> 46G 20:52:24 <skvidal> s... biggish 20:53:07 * skvidal looks 20:53:17 <skvidal> most of it is spot 20:53:19 <skvidal> :) 20:53:28 <nirik> :) 20:53:34 <skvidal> seriously 20:53:42 <skvidal> 19G of the 46G is his repos 20:53:52 <skvidal> b/c he's captain bad-ass builder of chromium, etc 20:54:01 * nirik keeps pruning his calibre repo or it would grow pretty badly. 20:54:12 <skvidal> well this would help remove that need 20:54:28 <nirik> yeah, I'd be ok with moving it... 20:54:38 <nirik> a question tho: what do we want to back up there/ 20:54:51 <skvidal> that is a good question 20:55:02 <skvidal> so wehave a fair amount of stuff on fedorapeople 20:55:08 <skvidal> b/c it can be very convenient 20:55:16 <nirik> yep. The first full backup hasn't finished since the move. ;( 20:55:24 <skvidal> nirik: I liked your idea of keeping a semi-hot backup somewhere 20:55:34 <skvidal> hmmm 20:55:39 <skvidal> so maybe an interesting test 20:55:45 <skvidal> use a junk box 20:55:54 <skvidal> and test rsync time to copy off all of people03 20:56:05 <skvidal> root+ssh+rsync 20:56:22 <skvidal> or use the vpn 20:57:02 * nirik isn't sure. wants to ponder on that some more. 20:57:59 <skvidal> nirik: it's a shame we don't have another place with enough disk at ibiblio or even at another i2 locaton 20:58:21 <nirik> yeah. it would be nice ot have a people spare thats somewhere where it could go live if needed. 20:58:52 <nirik> however, once we have cloud, we can move the dev/publictest stuff from osuosl and possibly do it there. 20:59:03 <skvidal> nod 20:59:05 <skvidal> I agree 20:59:22 <skvidal> I assume we've got nothing on an eta on anything cloud related or I would have already heard? 20:59:47 <mdomsch_> speaking of people and rsync, there was a request from someone who wanted to mirror fedorapeople.org 20:59:49 <mdomsch_> repos 20:59:56 <skvidal> mdomsch_: nirik replied iirc 21:00:01 <nirik> not really... need to finalize hardware and order... then wait for that to show up, then we need netapp stuff to happy to get us storage there... 21:00:12 <skvidal> mdomsch_: it's not a terrible idea to make it rsyncable - a little worried what that might mean load-wise 21:00:15 <nirik> mdomsch_: yeah, saw that. We could look at a rsync module I guess... 21:00:55 <nirik> ok, we are over time... shall we call it a meeting? 21:01:04 <smooge> okie dokie 21:01:51 <nirik> thanks for coming everyone. Continue on #fedora-admin and on the list! 21:01:53 <nirik> #endmeeting