From: dafrito(a)gmail.com Date: Mon, 5 Jul 2010 19:33:35 -0500 Subject: Proven tester wiki love To: test(a)lists.fedoraproject.org I went to work a bit on jdulaney's fork of the proven tester page to make the mentorship-merging stuff fit a little more smoothly, and I sort of got carried away. I'd normally just add this as a new revision for Proven_testers, but it's pretty substantially revised. It probably needs to get re-reviewed: https://fedoraproject.org/wiki/User:Dafrito/Proven_tester I can write up a summary of the major differences, but I figure I should only toss out one wall-of-text at a time. ;) Basically, I took the current one, and tried to expand on places that were shallow (like testing criteria) and made other sections flow a lot better (like the old Feedback section). I feel the need to mention I refer to critpath packages as just critical packages and critical updates, instead of critical path packages and critical path updates. This is purely a stylistic change: when I was reading it out loud, I trip over the alliteration of "path packages" a lot. Let me know if this (or anything else) is a mortal sin. -- Aaron Faanes -- test mailing list test(a)lists.fedoraproject.org To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test

I went to work a bit on jdulaney's fork of the proven tester page to make the mentorship-merging stuff fit a little more smoothly, and I sort of got carried away. I'd normally just add this as a new revision for Proven_testers, but it's pretty substantially revised. It probably needs to get re-reviewed: https://fedoraproject.org/wiki/User:Dafrito/Proven_tester I can write up a summary of the major differences, but I figure I should only toss out one wall-of-text at a time. ;) Basically, I took the current one, and tried to expand on places that were shallow (like testing criteria) and made other sections flow a lot better (like the old Feedback section). I feel the need to mention I refer to critpath packages as just critical packages and critical updates, instead of critical path packages and critical path updates. This is purely a stylistic change: when I was reading it out loud, I trip over the alliteration of "path packages" a lot. Let me know if this (or anything else) is a mortal sin.

On Wed, Jul 7, 2010 at 1:44 AM, Adam Williamson <awilliam(a)redhat.com&gt; wrote: > On Mon, 2010-07-05 at 19:33 -0500, Aaron Faanes wrote: >> I went to work a bit on jdulaney's fork of the proven tester page to >> make the mentorship-merging stuff fit a little more smoothly, and I >> sort of got carried away. I'd normally just add this as a new revision >> for Proven_testers, but it's pretty substantially revised. It probably >> needs to get re-reviewed: >> >> https://fedoraproject.org/wiki/User:Dafrito/Proven_tester >> >> I can write up a summary of the major differences, but I figure I >> should only toss out one wall-of-text at a time. ;) Basically, I took >> the current one, and tried to expand on places that were shallow (like >> testing criteria) and made other sections flow a lot better (like the >> old Feedback section). >> >> I feel the need to mention I refer to critpath packages as just >> critical packages and critical updates, instead of critical path >> packages and critical path updates. This is purely a stylistic change: >> when I was reading it out loud, I trip over the alliteration of "path >> packages" a lot. Let me know if this (or anything else) is a mortal >> sin. > > I understand - I've had the same feeling - but I think we may want to > stick with the 'critical path' naming, as it's what's used elsewhere in > Fedora-land. In general I am happy with the draft - however there is one area that I am a little uncertain about, and maybe others could clarify. This is the situation where for example there is currently a kernel in updates-testing for both f12 and f13, and each of these packages has a series of bug fixes. I am perfectly happy to install and test but I may not be able to test all the bugs reported as fixed. In this instance if the kernel installs without issue and the machine boots, with normal logins, normal networking for my particular wireless card on a laptop, and no issues with any tests that I conduct, should I report +ve karma if all the tests I can do pass without incident, or should I as proventester not add karma (i.e. neutral) since I can't check that all the bugs reported are not problematic. In the current case I see that for https://admin.fedoraproject.org/updates/kernel-2.6.32.16-141.fc12?_csrf_t... I can check 559153 but not 558002 for example. So should I add a report and have neutral karma or +ve? The instructions would indicate neutral karma is appropriate here. (Prior to becoming a proventester I would have given +ve karma in this situation. Similarly for https://admin.fedoraproject.org/updates/kernel-2.6.33.6-147.fc13?_csrf_to... where there is an even longer list of bugs fixed. Clearly I can only test on the hardware that I am installed on for either f12 or f13 .... Would be nice to have this clarified - I guess that reading the proventester instructions again this case should have neutral feedback. It seems that the only situation for giving positive karma as a proventester is if previously reported negative karma is demonstrated to have been wrong - please correct me if I am wrong.

On Wed, Jul 7, 2010 at 8:16 PM, Adam Williamson&lt;awilliam(a)redhat.com&gt; wrote: > Thinking about it, though, we could consider a slightly different > process for the kernel, as it's a component that's *extremely* subject > to different experiences for different users. I'm not sure the workflow > we've designed will work terribly well for kernels. I suspect it'll be > all too easy for a kernel which actually contains a major regression to > be approved; all it needs is for a proventester who doesn't happen to > own the hardware concerned to find it works fine on their system, and > file a +1, and anyone else to file a +1 too, and it'd be approved, even > though someone who does own the hardware might come by and test an hour > later and find the problem... > > we might want to design a system for the kernel where all proventesters > hold off posting positive feedback for a day or two, until several > proventesters and regular testers have had the chance to check for > regressions. That was exactly my thought too - I saw these kernel updates were there but thought that to satisfy the current criteria as best I could I would wait and see what comments that came in to bodhi over the next day or so looked like and then install and test. If I then saw no negatives, and my own tests found no problems then I felt +1 would be valid, but I wanted re-assurance from people here first. It would seem that in this situation neutral karma from a proventester would not be particularly useful as the package would not get the necessary push to stable unless a proventester gives +1. If this is acceptable as a way forward I would be happy with that but as you say for the kernel perhaps an additional paragraph in the draft would be useful.

On Wed, Jul 7, 2010 at 8:16 PM, Adam Williamson <awilliam(a)redhat.com&gt; wrote: > Thinking about it, though, we could consider a slightly different > process for the kernel, as it's a component that's *extremely* subject > to different experiences for different users.

test(a)lists.fedoraproject.org To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test