Apologies for the delayed response. On Fri, 2010-06-11 at 16:11 -0700, John Poelstra wrote: > I have created a detailed schedule for the QA team for Fedora 14 based > on the tasks from Fedora 13. The contents of this schedule will also > drive the weekly schedule reminders this list gets about testing tasks. > At one point there was discussion about adding specific test days to > the schedule. I'm glad to help with that too. I'm 50/50 on adding test days into the official schedule. Test day scheduling is still very dynamic in nature, so there could potentially be multiple schedule updates per week. Also, with exception of a handful of F13 events, my impression is that test day attendance was down. I'm not yet clear on the cause, but some feedback suggests we could improve communication of the events. I'm also curious whether we would benefit from focusing on fewer test days for Fedora 14. QA gang, any thoughts here?

On 06/18/2010 03:32 PM, Adam Williamson wrote: >> QA gang, any thoughts here? >> > I'm not big on putting the Test Days into that schedule, I think the > current system works fine from a scheduling POV. > > I don't think having fewer Test Days would improve attendance. I think > people just show up to Test Days they're really personally interested > in, no matter how many there are. I don't think many people are thinking > 'well, I'd come out to all the Test Day events, even ones I don't care > about, if there were fewer of them!' That's just my gut feeling, though. > Similar to my gut feeling.. Perhaps if we advertise it more and increase the test period from a day to a week we might get a better test coverage since testers might not be available exactly during the test day's time and think to themselves hey we just missed the test day so we wont perform those tests. We could group more tests into a test week and advertise as a Fedora test week and ask the testers to perform what ever test that interest them from the list of tests we provide for the test week and we would then gather the result in a weeks time instead of a day. This would give both maintainers and tester a bit more flexibility regarding the usage of their time instead of forcing them to bound to a single day. They might only be able to spare an hour per day which would only give us an hour for a test day but 7 hours of testing for a whole test week. If we want to improve test cover in general we might need to split tasks amongst people. John and Linda take test a - c, Betty and Bertha test y - z etc.

I want to keep to the 'Test Day' because the concurrency aspect - having everyone in IRC, including a developer to give a very fast feedback/fix loop - is one of the key features, and you can't maintain that over a period longer than a day. However, I do agree in one way: we can certainly make use of testing done outside of the exact 'day', and we should make sure to always communicate that. I try to do so when talking about Test Days and when making pages for them, but any ways we can be even more explicit about this would be great:)

> 3. Calling it a Test _Day_ sure makes it sound like it is limited to a > single 24hour period. Maybe this is a marketing problem, is there a > better name? If there is, I haven't been able to think of it :(. As I said, the problem is it cuts both ways; the 'day' concept has good connotations as well as bad ones. I haven't been able to come with anything that keeps the good while cutting the bad. Suggestions gratefully received!

On Mon, 2010-06-21 at 08:35 -0600, Eric Blake wrote: > On 06/18/2010 10:43 PM, Adam Williamson wrote: > >> 3. Calling it a Test _Day_ sure makes it sound like it is limited to a > >> single 24hour period. Maybe this is a marketing problem, is there a > >> better name? > > > > If there is, I haven't been able to think of it :(. As I said, the > > problem is it cuts both ways; the 'day' concept has good connotations as > > well as bad ones. I haven't been able to come with anything that keeps > > the good while cutting the bad. Suggestions gratefully received! > > What about "Test Blitz"? We document that developers will be online for > one particular day, but that testers can submit results for the duration > of the blitz, which can be longer than one day. It's a good shot. It feels a little fuzzy, though; the name alone doesn't convey much about the concept. I'm not sure. What do others think?

On Mon, 21 Jun 2010 13:44:47 -0400 James Laska<jlaska(a)redhat.com> wrote: > More food for thought ... > > I've heard thoughts from others about how the 'test day' name doesn't > accurately match the effort. One suggestion was to rebrand the effort > as 'bug day'. Similar to http://live.gnome.org/Bugsquad/BugDays > > Thanks, > James > I have only participated in a few test days however test day(s) are nothing like gnomes bug day from my experience so not a good name choice imho. It was my hope that bugzappers triage day would be an event similar to gnomes bug day but we (the bugzappers) have failed thus far to really get that going.

On Mon, 2010-06-21 at 18:31 +0000, "Jóhann B. Guðmundsson" wrote: > On 06/21/2010 06:22 PM, TK009 wrote: > > On Mon, 21 Jun 2010 13:44:47 -0400 > > James Laska<jlaska(a)redhat.com> wrote: > > > >> More food for thought ... > >> > >> I've heard thoughts from others about how the 'test day' name doesn't > >> accurately match the effort. One suggestion was to rebrand the effort > >> as 'bug day'. Similar to http://live.gnome.org/Bugsquad/BugDays > >> > >> Thanks, > >> James > >> > > I have only participated in a few test days however test day(s) are > > nothing like gnomes bug day from my experience so not a good name choice > > imho. > > > > It was my hope that bugzappers triage day would be an event similar to > > gnomes bug day but we (the bugzappers) have failed thus far to really > > get that going. > > > > Let's just call it Test sprint ;) Andre suggested 'Testathon' on IRC. Perhaps we can have a poll. =) It did make me have very rough thoughts about the possibility of doing a sort of 'rebranding exercise' (excuse me for a second while I shoot myself) around one of the newer names; change our conception of the event itself a little bit as well as the name, and maybe provide a longer time base for publicity. Ubuntu, for instance, has the old marketing trick of announcing everything at least five times down pat; we could do this with Test Sprints / Test Blitzes / Testathons / whatever. Very rough sketch - * we can do a shiny announcement when we've decided what events we'll have * we can do a shiny announcement when the test cases for a given event are done * we can declare week-long Focus Periods for each event and do shiny announcements at the start and end of each * we can do three shiny announcements of the 'Blitz Day' or whatever associated with each event; before, during, after (with the 'before' and 'after' announcements emphasizing that you can test already/still) * we can do a shiny wrap-up at the end of the cycle I dunno, it's just a thought. Very rough so far...

-- Adam Williamson Fedora QA Community Monkey IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org http://www.happyassassin.net -- test mailing list test(a)lists.fedoraproject.org To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test

On Fri, 2010-06-18 at 10:39 -0400, James Laska wrote: > Apologies for the delayed response. > > On Fri, 2010-06-11 at 16:11 -0700, John Poelstra wrote: > > I have created a detailed schedule for the QA team for Fedora 14 based > > on the tasks from Fedora 13. The contents of this schedule will also > > drive the weekly schedule reminders this list gets about testing tasks. > > At one point there was discussion about adding specific test days to > > the schedule. I'm glad to help with that too. > > I'm 50/50 on adding test days into the official schedule. Test day > scheduling is still very dynamic in nature, so there could potentially > be multiple schedule updates per week. > > Also, with exception of a handful of F13 events, my impression is that > test day attendance was down. I'm not yet clear on the cause, but some > feedback suggests we could improve communication of the events. I'm > also curious whether we would benefit from focusing on fewer test days > for Fedora 14. > > QA gang, any thoughts here? I'm not big on putting the Test Days into that schedule, I think the current system works fine from a scheduling POV. I don't think having fewer Test Days would improve attendance. I think people just show up to Test Days they're really personally interested in, no matter how many there are. I don't think many people are thinking 'well, I'd come out to all the Test Day events, even ones I don't care about, if there were fewer of them!' That's just my gut feeling, though.