[15:04] mmcgrath has set the subject to Fedora Infrastructure -- Role ca [15:04] mmcgrath: Who's here? [15:04] * skvidal is [15:04] jcollie: i am [15:04] * lmacken [15:04] jeremy has left (Remote closed the connection (i=katzj@nat/redhat/x-2faba177d7c1624f)) [15:04] warren has left (Remote closed the connection (i=warren@redhat/wombat/warren)) [15:04] marek: Marek Mahut [15:05] paulobanon: pong [15:05] * jbowes watches [15:05] mether_ has joined the group chat (n=ask(a)202.80.58.21 [15:05] * abadger1999 splits himself into two. [15:05] mmcgrath: Ok, we'll get started on the tickets. [15:05] * rayvd is he [15:05] mmcgrath: https://hosted.fedoraproject.org/projects/fedora-infrastructure/query?sta... [15:05] mmcgrath has set the subject to Fedora Infrastructure -- Tickets [15:05] mmcgrath: first ticket [15:06] jeremy has joined the group chat (i=katzj@nat/redhat/x-baaa30725c1d4645) [15:06] mmcgrath: #18. Scripts should produce no output. [15:06] MrBawb has joined the group chat (n=abob(a)guppy.drown.org) [15:06] paulobanon: rayvd: befora i forget request sysadmin-test group access for your Moin project (we continue in the meeting now) [15:06] mether_ has left (Read error: 104 (Connection reset by peer) (n=ask(a)202.80.58.211)) [15:06] mmcgrath: I've been working on #18, and aside from the mass mailing this morning its about done. [15:06] * rordway|m is here [15:06] rordway|m: well, somewhat [15:06] mmcgrath: Just make sure that when you guys automate something that you redirect any stdout to a log or /dev/null. [15:06] mmcgrath: rordway|m: yo [15:06] rayvd: paulobanon: thanks, will do [15:06] rordway|m: at OSCON, so I'm here as much as the wifi allows me [15:06] mmcgrath: next ticket #52 [15:06] mdomsch has joined the group chat (n=mdomsch(a)cpe-70-124-62-55.austin.res.rr.com) [15:06] mmcgrath: paulobanon: Proxy Caching [15:07] * vpv is here too [15:07] mmcgrath: what did you want to kno [15:07] paulobanon: mmcgrath: k so i want to know what can we deploy in PT1 [15:07] paulobanon: squid or mod_cache [15:07] paulobanon: to start playing/testi [15:07] mmcgrath: we can do both actually. [15:08] mmcgrath: Pick one, then the other. [15:08] paulobanon: start with squid then [15:08] mmcgrath: I'd *prefer* mod_cache, unless there's some horrible reason we shouldn't. [15:08] paulobanon: k start with mod_cache [15:08] paulobanon: [15:08] paulobanon: _almost_ anything to make u happy [15:08] mmcgrath: paulobanon: its totally up to you though. My only concern with squid is making it work in our current balancer setup. [15:08] jcollie: i think squid has more options for managing cache disk spa [15:08] paulobanon: jcollie: +1 [15:09] jcollie: er spa=space [15:09] mmcgrath: jcollie: my issue is, for example, right now if one of the mirrormanager servers goes down. It fails over automatically. [15:09] mmcgrath: I'm not quite sure how to do that (haven't thought about it) so its still a grey spot in my head. [15:09] mmcgrath: paulobanon: anything elseith that ticket? [15:09] paulobanon: if the cache is independentu wont have probs [15:09] paulobanon: one fails u have the other [15:09] mmcgrath: paulobanon: but how does mod_proxy_balaner know that? [15:10] marek: good point [15:10] paulobanon: in theory u just need to redirect your requests like this [15:10] paulobanon: proxy > cache > app [15:10] warren has joined the group chat (i=warren@nat/redhat/x-c809280b07cd9d56) [15:10] paulobanon: so if 1 app is down, the request will still go to the other [15:11] mmcgrath: paulobanon: well, if you can set it up as a proof of concept with failover, I'll be happy [15:11] mmcgrath: anything els [15:11] paulobanon: nop nothing more [15:11] warren: I'm back, sorry lost connection [15:11] mmcgrath: cool [15:11] mmcgrath: warren: no worries. [15:11] mmcgrath: I think we can skip ticket #14, its got a whole meeting for it. [15:11] mmcgrath: #31 [15:11] mmcgrath: ricky: ping? [15:12] ricky: mmcgrath: pong [15:12] mmcgrath: ricky: Anything to comment on ticket #31 "New Wiki" [15:12] mmcgrath: its got the meeting keyword so we're talking about it (feel free to remove it if there's nothing to discuss) [15:12] ricky: mmcgrath: Have we looked at zwiki, actually, or are the memory requirements too high? [15:13] ricky: It'd integrate nicely with Plone, and seems to fit our requirements feature-wis [15:13] mmcgrath: ricky: I'm not familiar with its memory requirements. [15:14] mmcgrath: ricky: if you've got something you'dike to try send it to the list, we'll pick at it [15:14] ricky: Sure, I'll try to formalize wiki evaluations a bit more on the wiki. [15:14] mmcgrath: cool [15:15] mmcgrath: ricky: anything else on the ticket? If not I'll move on [15:15] ricky: Not yet. [15:15] mmcgrath: cool [15:15] mmcgrath: next ticket # [15:16] mmcgrath: paulobanon: whats the word "Moin wiki optimizations" [15:16] paulobanon: k, so the RFRs where sponsored [15:16] rayvd: paul just requested that i request sysadmin-test flag, which i will do... [15:16] paulobanon: so its time to start moving on with this [15:16] rayvd: vpv will also be using this instance correc [15:16] paulobanon: vpv already has access to PT1, and rayvd is requesting sysadmin-test now [15:16] paulobanon: rayvd: correct [15:17] paulobanon: the only concern right now its the python version [15:17] [splinux] has left ("Ex-Chat" (n=damien@fedora/splinux)) [15:17] rayvd: 2.4 is fine, i can use it. [15:17] paulobanon: perfect then [15:17] rayvd: let's make life easy [15:17] paulobanon: you require your moin1.6 branch or what you require _ [15:17] paulobanon: ? [15:17] rayvd: i'm going to use 1.5.8 as much as possible since that's what we are using currently. [15:18] paulobanon: perfect [15:18] paulobanon: mmcgrath: ill need your help here [15:18] mmcgrath: rayvd: is it your intention to get this stuff comitted upstream? [15:18] mmcgrath: paulobanon: sure thing. [15:18] rayvd: mmcgrath: maybe. they ahev a whole new storage back-end in the works [15:18] rayvd: which would be a _lot_ of work to backport. [15:18] hpachas-PE has joined the group chat (n=root(a)200.107.160.195) [15:18] rayvd: they sounded open to merging any fixes though from what we do. [15:19] mmcgrath: cool. [15:19] mether has left (Connection timed out (n=ask@fedora/mether)) [15:19] paulobanon: mmcgrath: to setup both instances on the same box; vpv needs his Moin1.7 branch, and we can use a stock Moin1.5.8 for rayvd [15:19] mmcgrath: ok, so do you two need anything else for now in the meeting? [15:19] mmcgrath: paulobanon: sure thing, ping me after the meeting and I'll show yo [15:19] paulobanon: mmcgrath: perfect [15:19] rayvd: nope. should be able to work with paul directly with any questions. [15:19] mmcgrath: excellent. [15:19] paulobanon: vpv: ping [15:19] vpv: I'm here [15:19] paulobanon: vpv: can you upload your Moin1.7 to your homedir please _ [15:19] paulobanon: ? [15:20] vpv: paulobanon: it should be there already [15:20] paulobanon: +1 [15:20] paulobanon: im done then [15:20] mmcgrath: ok. [15:20] mmcgrath: I'll move [15:20] hpachas-PE: the meeting is now? [15:20] mmcgrath has set the subject to Infrastructure -- Sponsorship [15:20] mmcgrath: hpachas-PE: yep, we're in it. [15:21] mmcgrath: Ok, so no one really responded to my notice about the pages being ready. [15:21] abadger1999: mmcgrath: Good docs. [15:21] mmcgrath: anyone have thoughts? [15:21] jcollie: yeah, i liked 'em [15:21] abadger1999: Should LDAP and DB go together on the FIG page? [15:21] mmcgrath: I'm talking about [15:21] mmcgrath: http://fedoraproject.org/wiki/Infrastructure/Sponsor [15:21] mmcgrath: http://fedoraproject.org/wiki/Infrastructure/FIGs [15:21] mmcgrath: http://fedoraproject.org/wiki/Infrastructure/GettingSponsored [15:21] mmcgrath: abadger1999: yeah, I'd think so. [15:22] jcollie: technically, my question this morning about what groups i'd need to be in was about them [15:22] ricky: What's sysadmin-general? I remember seeing that somewhere else befor [15:22] mmcgrath: ricky: yeah, sysadmin-general was a first (and poor) attempt at doing what we're doing for real this time. [15:22] mmcgrath: long story short, sysadmin-general grants access to bastio [15:22] mmcgrath: (this was before we had publictest servers I think) [15:22] mmcgrath: I'd imagine sysadmin-general is going to go away [15:22] paulobanon: mmcgrath: my question is who decides and how its decided who is a sponsor, and where [15:22] ricky: OK, so that's why it isn't on those pages. [15:23] mmcgrath: paulobanon: Good question, I'll document that. [15:23] mmcgrath: paulobanon: here's the steps, as I see them right now. [15:23] mmcgrath: 1) create the groups (some of those don't actually exist yet) [15:23] mmcgrath: 2) find some of the sysadmin-main guys to populate those groups. I'll be contacting people individually. [15:23] paulobanon: (me thinks sysadmin-main should decide; since its the _core_ group) [15:23] mmcgrath: paulobanon: but there's a catch as well. [15:24] mmcgrath: Hopefully, in the not too distant future, I'm hoping people will be able to be a sponsor in their group without being in sysadmin main. [15:24] bpepple|lt has left ("Ex-Chat" (n=bpepple|(a)adsl-75-42-213-28.dsl.wotnoh.sbcglobal.net)) [15:24] paulobanon: decide = who can sponsor in which group [15:24] paulobanon: thats what i meant [15:24] mmcgrath: got'cha. [15:24] paulobanon: so you guys, decide that john, can be in web and doe can be in cvs [15:24] paulobanon: if you know what i mean [15:25] mmcgrath: Yep, and if doe does a good job then someone already in web will upgrade doe so he can sponsor others. [15:25] paulobanon: +1 [15:25] mmcgrath: Thats the plan. [15:26] paulobanon: just carefull to dont have 'over' sponsors, too many sponsors may not be a good thing [15:26] mmcgrath: The plan is to be as open as possible with this, but not crazy. [15:26] paulobanon: since you start to loose track of stuff [15:26] mmcgrath: paulobanon: well, I'm hoping a strict removal policy will fix that. [15:26] paulobanon: +1 [15:26] mmcgrath: http://fedoraproject.org/wiki/Infrastructure/GettingSponsored#head-7ac849... [15:26] mmcgrath: we'll see. [15:26] paulobanon: k k [15:26] * paulobanon shutsup now [15:27] mmcgrath: I'm hoping some of our regulars that show up a lot but don't have much access will step up and ask for access to some things [15:27] mdomsch: that all looks quite sane to me [15:27] * jcollie blushes [15:27] mmcgrath: Does anyone know of any other communities with this sort of setup? [15:27] mdomsch: packagers? [15:27] mmcgrath: AFAIK once its implemented, it will be the most open, by far, sysadmin group around. [15:27] jcollie: can i have access to the fleet of fedora black helicopers? and the fedora orbital laser too? [15:28] ricky: Yay! [15:28] marek: [15:28] mdomsch: jcollie, notice that sysadmin-blackops isn't listed on the page [15:28] mmcgrath: [15:28] jcollie: mdomsch, uhm yeah that's why it's black i suppose [15:28] * paulobanon advises jcollie, that he is around, so he wants access too [15:28] paulobanon: [15:28] mmcgrath: There are two technical hurdles to overcome. [15:29] mmcgrath: the biggest is figuring out where to store puppet and what to do with it. [15:29] jcollie: i think we should do as much with puppet as we can [15:29] mmcgrath: I don't think this will all happen over night... But I do think we'll have something by next week, especially for the development and web teams (which is where most of the work is needed) [15:29] mmcgrath: jcollie: well we do everything with puppet now (except for the test servers) [15:30] paulobanon: mmcgrath: which is great [15:30] jcollie: other than being in CVS what's wrong with where it is now? [15:30] mmcgrath: the problem is opening up access but still keeping some of the passwords/keys secure. [15:30] mmcgrath: like the web guys don't need access to the buildserver keys. [15:30] mmcgrath: and the build guys don't need the fedoraproject.org ssl key. [15:30] mmcgrath: that sort of thing [15:30] * paulobanon remembers that we choose git to replace cvs [15:30] paulobanon: for our own confs [15:31] halfline_ has joined the group chat (i=rstrode@nat/redhat/x-a0ab6f0af8632421) [15:31] jcollie: are the keys and stuff in the puppet fileserver then? [15:31] mmcgrath: jcollie: some of them yes, some of them no. [15:31] paulobanon: and some that are, are only reachable by some ppl [15:32] skvidal: there's a good reason to keep stuff in cvs or svn for admin stuff for puppet [15:32] jcollie: hmm as nice as it would be to have everything in puppet maybe stuff like that we don't want to have there [15:32] skvidal: esp if we keep it on lockbox [15:32] skvidal: so that we don't end up with copies of our keys or pws out on someone's laptop [15:32] skvidal: in other words: if you want admin access you MUST be on lockbox or at least bastion to get to them [15:32] mmcgrath: jcollie: yep, those are the little things we have to keep an eye on. [15:33] mmcgrath: Ok, so I'll move on for now. Everyone seems ok with the docs. I'll get the groups set up and hopefully poulated in the next few days with sponsors. [15:33] jcollie: is there anything confidential in the puppet manifests? [15:34] nman64 has left (Remote closed the connection (n=n-man@fedora/nman64 [15:34] paulobanon: jcollie: in the manifests itself, i dont think so [15:34] mmcgrath: jcollie: there is, thats why in the GettingSponsored page I'm only allowing changes to files. [15:34] paulobanon: oh ok... didnt knew that [15:34] mmcgrath: unless your a sponso [15:34] mmcgrath: jcollie: any place we have a password stored, we're using a template. [15:35] mmcgrath: but thats mostly because of commit emails. [15:35] jcollie: hmm quite a conundrum [15:35] mmcgrath: jcollie: indeed. You're good with puppet so I'm totally open to ideas. [15:35] mmcgrath: its something to think about. [15:36] mmcgrath: even if for now we have to give access to the machines and have patches submitted to the sponsors. [15:36] jcollie: yeah, i have an idea but move on to something else for now [15:36] mmcgrath: <nod> jcollie email the list if you get somethign concrete. [15:36] jcollie: yep [15:36] mmcgrath has set the subject to Infrastructure -- SOP [15:37] ricky: It would feel kind of weird using sponsor status to control access to things. [15:37] mmcgrath: I really don't have much else to add on SOP's this week. I'm working on getting db1 and db2 finished. I'll have an SOP for failing over soo [15:37] skvidal: mmcgrath: do we have a ticket for reposync? [15:38] mmcgrath: ricky: it would be ideal to have a couple of test environments. [15:38] mmcgrath: skvidal: nope, want to open one or should I? [15:38] skvidal: the info I need from you is where I can put the files [15:38] skvidal: where has the space and where won't it suck? [15:38] mmcgrath: <nod> I'll get that to you. [15:39] skvidal: I think you told me before [15:39] skvidal: but I don't have it now [15:39] skvidal: b/c afaict, I suck. [15:39] skvidal: [15:39] mmcgrath: [15:39] mmcgrath: Ok, so thats all I've got for this week. [15:39] mmcgrath has set the subject to Infrastructure -- Open floor [15:39] rordway|m: that was quick [15:39] mmcgrath: who's got what they'd like to talk about? [15:39] rordway|m: [15:39] mmcgrath: rordway|m: yeah 40 min [15:39] paulobanon: lmacken: ping [15:39] jcollie: asterisk? [15:39] rordway|m: whee! [15:39] mmcgrath: ahh yes. [15:40] mmcgrath: asterisk. [15:40] mmcgrath: jcollie: I honestly don't know much about that. [15:40] mmcgrath: skvidal: do you? I think it was discussed in the board and dgilmore is on a plane. [15:40] skvidal: what do you want to know [15:40] jcollie: all i know is what was discussed in irc yesterday [15:40] lmacken: paulobanon: pong [15:40] mmcgrath: BTW al - http://fedoraproject.org/wiki/Infrastructure/RFR/AsteriskServerInstance [15:40] paulobanon: lmacken: lets wait for the asterisk stuff before fedora pastebin [15:41] mmcgrath: The idea is to get a asterisk server setup for the virtual FUDCon? [15:41] skvidal: yes [15:41] skvidal: so people can call in and talk [15:41] mmcgrath: k. [15:41] skvidal: we wanted to get it up and tested in the next week or so [15:42] skvidal: so we can know whether it is workable for the event [15:42] skvidal: if it is not workable then we won't announce it [15:42] mmcgrath: AFAIK this only needs to be done for the fudcon so we'll probably set up a temporary instance as a proof of concept. Then deploy the actual thing when we get omre hardware (which will be after FUDCon) [15:42] jcollie: my one concern with asterisk is i don't know how it'll behave on a xen guest [15:42] mmcgrath: jcollie: yeah thats the 'proof of concept' part. [15:42] mmcgrath: jcollie: is it in Fedora or EPEL yet? [15:42] jcollie: i have asterisk packages 99% ready to go so that's not a problem [15:43] mmcgrath: what will you be running it on for FudCON? FC6,F7 or EPEL5? [15:43] jcollie: no, it's not in yet but dgilmore promised to review [15:43] jcollie: mmcgrath, i don't care especially, i've never run it on RHEL [15:43] mmcgrath: k, stay on him till that gets done. [15:43] jcollie: EPEL might not have all of the dependencies yet [15:43] mmcgrath: jcollie: what would have the fewest question marks for you and dgilmore? [15:43] jcollie: FC6 or F7 [15:43] mmcgrath: k, we can do that. [15:44] mmcgrath: Also, I'm assuming this will require firewall changes? [15:44] jcollie: jared smith is going to be helping out [15:44] jcollie: mmcgrath, yeah big time [15:44] mmcgrath: I'm going to need those ASAP. [15:44] mmcgrath: stick them in the RFR [15:44] jcollie: sip is 5060 udp and rtp needs a lot of high UDP ports [15:44] * mmcgrath notes the network team probably won't be happy with 'big time' so the more time we give them the better. [15:45] jcollie: anyone have any estimates on how many people we could expect for the virtual fudcon? [15:45] mmcgrath: jcollie: be as specific as possible, I'd hate to have to ask pick to make all of these changes and have it not work or have him have to make more. [15:45] mmcgrath: we'll get it going though. [15:45] jcollie: mmcgrath, yeah we can control the ports pretty easy [15:45] mmcgrath: jcollie: none that I know of. [15:45] mmcgrath: what difference does it make? [15:45] mmcgrath: lets say I said 50 or 300, whats the difference for you guys? [15:45] jcollie: the number of udp ports is mostly dependent on how many simultaneous calls [15:46] skvidal: udp ports? [15:46] skvidal: mmcgrath: any fw issues? [15:46] jcollie: mmcgrath, actually we'd probably want 1000+ [15:46] jcollie: skvidal, RTP (audio transport) runs on UDP [15:46] jcollie: the number of udp ports is configurable though [15:46] mmcgrath: jcollie: so every caller gets assigned a special port? Is this something that we'll need to "fixup" ? [15:47] jcollie: mmcgrath, no turn off any fixups in the pix [15:47] mmcgrath: k. [15:47] jcollie: asterisk knows about nat and can deal with it without pix fixups [15:47] mmcgrath: jcollie: so the blockers as I see them are 1) server, 2) firewall and 3) install/test [15:48] mmcgrath: jcollie: k. [15:48] jcollie: yep [15:48] mmcgrath: jcollie: I'll get 1 ready sometime soon (unless dgilmore does it) [15:48] mmcgrath: get 2) on the RFR and 3) done as soon as you can and I think we'll be in good shape. [15:48] jcollie: ok, i'll get the list of ports and put them on the rfr [15:48] mmcgrath: excellent. [15:49] mmcgrath: so anything else? [15:49] paulobanon: Fedora PasteBin [15:49] paulobanon: [15:49] mmcgrath: Open floor and we've got 10 minutes left. [15:49] jcollie: what'll it take to get jared smith into sysadmin-test? he has a lot of asterisk experience too [15:49] mmcgrath: paulobanon: ahh, whats the latest on tha [15:49] mdomsch: paulobanon, I've got pastebin.domsch.com running for myself if you want that [15:49] mmcgrath: jcollie: he'll just need to apply we can add him. [15:49] jcollie: [15:49] mmcgrath: paulobanon: have you gotten an instance up anywhere? [15:49] mmcgrath: is gotten a word? [15:50] paulobanon: mdomsch: whats the difference between yours and lmacken [15:50] mclasen has left (Read error: 110 (Connection timed out) (i=mclasen@nat/redhat/x-e57cccf9577cb7d8)) [15:50] mdomsch: don't know [15:50] rayvd: jcollie: do you guys anticipate any issues with NAT? i have done lots with SER/OpenSER if so for NAT issues with SIP. [15:50] paulobanon: cause i only knew about lmacken one in pt2 [15:50] lmacken: I just did an `svn update` on our stickum instance on publictest2:/srv/stickum. There has yet to be any formal upstream releases, so we'll need to bash around with what we currently have to see if it'll suit our needs. [15:50] mdomsch: mine's on my own domain, not in fp.o [15:50] jcoie: rayvd, no it shouldn't be a problem i hope [15:50] lmacken: just tried to get it running, but encountered a sqlalchemy error.. I don't have any time this week to mess with this though [15:51] lmacken: the stickum code in /srv/stickum on pt2 should be hooked into FAS [15:51] mmcgrath: mdomsch: is yours in turbogears? [15:51] ricky: Looks like PHP to me. [15:51] ricky: lmacken: Curious: What cool things would we do with FAS integratio [15:51] mdomsch: mmcgrath, no, just from pastebin.com [15:51] paulobanon: PHP is *evil* [15:52] paulobanon: ricky: authentication [15:52] mmcgrath: ricky: the easiest way to do fas integration would just be to have apache do it. [15:52] lmacken: ricky: not sure.. afaik we just didn't want anyone to be able to add stuff to ou [15:52] paulobanon: so no anonymous spammage [15:52] abadger1999: fedora-db-access isn't available on pt2. [15:53] ricky: Pastebins really have spam problems? [15:53] mmcgrath: ricky: yeah [15:53] * abadger1999 copies that over to see if it'll work. [15:53] paulobanon: abadger1999: is it working in pt1 ? [15:53] lmacken: if it's a form, it'll get spammed [15:53] ricky: I was thinking that for example, random people on #fedora would be referred to it, etc. [15:53] ajax has left ( (i=ajackson@nat/redhat/x-1a0cf3da10d71e28)) [15:53] ricky: Oh, then who is this targeted at, actually? [15:53] mdomsch: gotta run, another I/T meeting [15:53] mmcgrath: mdomsch: later [15:54] paulobanon: lmacken: is there any voodoo required for me to get it working in pt1 [15:54] lmacken: paulobanon: shouldn't be.. [15:55] paulobanon: lmacken: k ill try setting that up in pt1 then, mmcgrath, do i have access to pt2 to pick that stuff or can u drop it in pt1 $homedir [15:55] mdomsch has left ("Leaving" (n=mdomsch(a)cpe-70-124-62-55.austin.res.rr.com)) [15:55] vpv: the pastebin could be protected with a password which would be put in the topic of #fedora. Might not protect from all spam, t some... [15:55] mmcgrath: paulobanon: paulobanon yep, looks like you've got access on pt2 [15:55] ricky: Then it don't see how it'd be worth using over all the open/convenient ones all over. [15:55] * mmcgrath thinks we should not do auth until the need proves itself. [15:55] paulobanon: mmcgrath: k k, ill work a bit on it tomorrow [15:56] mmcgrath: paulobanon: you're a busy guy these days [15:56] ricky: It would be cool to have the option of authenticated-view posts, though [15:56] rmarquez has joined the group chat (n=rmarquez(a)agssa.net) [15:56] paulobanon: everything is working in the world of warcraft domain, so i have time [15:56] paulobanon: [15:56] lmacken: paulobanon: ew [15:56] lmacken: paulobanon: my roomate lives in that game [15:56] mmcgrath: Ok, so paulobanon is going to setup a proof of concept for that for deployment soon. [15:56] paulobanon: lmacken: i work for that game [15:57] mmcgrath: Anyone have anything else? If not we'll close the meeting. [15:57] lmacken: paulobanon: nice! can you delete his accounts ? [15:57] blizzard: abadger1999: baby baby baby [15:57] lmacken: yeah [15:57] rmarquez: hello [15:57] paulobanon: blizzard: gratz !! [15:57] mmcgrath: 30 [15:57] lmacken: for our virtual fudcon I think we should have a sobby instance so we can all collaborate on notes and such [15:57] blizzard: paulobanon: thank you! [15:57] mmcgrath: 15 [15:57] jcollie: lmacken, inkscape whiteboard would be cool too [15:57] mmcgrath: 5 [15:58] * ricky re-mentions that ErrorDocument thing on download.fedora.redhat.com. [15:58] lmacken: jcollie: never played with it.. but the more real-time collaboration, the better [15:58] mmcgrath has set the subject to Infrastructure -- Meeting end