"Clyde E. Kunkel" <clydekunkel7734(a)cox.net> writes:
On 11/12/2010 02:32 PM, Tom Lane wrote:
> It's absolutely crystal clear to me that we don't have enough tester
> manpower to make the current policy workable; it's past time to stop
> denying that. I'd suggest narrowing the policy to a small number of
> critical packages, for which there might be some hope of it actually
> working as designed.
Test cases would help alleviate manpower issues. Many of the
security
updates and regular updates are outside my area and I feel some
frustration that I have to bypass providing karma; however, I am used to
doing QA work with test cases. Are they so hard to provide? Maybe
certain updates should have test cases, i.e., security updates and
critical path updates.
The major packages that I work with have regression test suites,
which in fact get run as part of the RPM build sequence. It's not
apparent to me that I should need to invent some more tests.
The likely failure cases that I can see are of two types:
1. Upstream screwed up and introduced a regression into what was
supposed to be a minor bug-fix or security update. This does happen,
for sure, but there's pretty much 0 chance that I as packager am going
to catch it if it gets past the built-in regression tests.
Unfortunately, there is also pretty much 0 chance that Fedora testers
are going to notice such a problem in the limited time window for sanity
testing. It hasn't ever happened for any of my packages that Fedora
testers caught such things in time.
2. I screwed up and introduced a packaging bug, for instance bad
dependencies or inability to "yum update". That's been known to happen
too. But I have a lot more faith in autoqa being able to catch that
kind of problem in a timely fashion than I do in manual testing catching
it.
I guess what this boils down to is that I'd be happier with the testing
process if it were actually successful at finding problems. In my
experience, it's a week's delay for exactly zero return.
regards, tom lane