On Sep 20, 2011, at 12:19 PM, Doug Ledford wrote:
----- Original Message -----
> This is essentially what we had a while ago, only with trac tickets
> instead of bodhi requests.
Bodhi is definitely a better place to track this stuff, regardless of how decisions are
made.
> There were a couple of problems with
> this.
>
> 1) Nowhere near enough releng folks to properly review all the
> tickets.
Fair enough.
> 2) 9 times out of 10 there was very little data put into the ticket.
Multiple options here. Kick back incomplete tickets, or the better option IMNSHO, run
rpmdiff runs between the package currently in the compose and the one in testing to check
for various failures and require the developer to justify failures.
There was supposed to be AutoQA filling this role here, catching obvious issues, sanity
checking, and letting things through that looked OK. We're still waiting for autoQA.
There certainly wasn't enough releng man power/volunteers to manually look through
this for every potential update.
> 3) releng folks were often not the best people to decide whether a
> change was "too risky"
The rpmdiff option above would help with this.
> 4) There was no easy way to get at the package and assess its
> stability.
Using bodhi instead of trac solves this, no?
No, what was missing was a repository with yum metadata to get at the package and any
sibling packages that needed to come with it for a comprehensive picture of what was going
on.
> I think there were more issues, but those come to mind first. We
> decided it was best instead to make a repository out of proposed
> changes,
But in practice that's not really what updates-testing on the early branched release
really is. It's a repo all right, but not of proposed changes, it's a repo of
packages, and getting to the actual changes versus the final package would require
installing the current source rpm, the new source rpm, then doing a manual inspection for
changes. An automated rpmdiff run would be a *far* superior means of presenting the
proposed changes to the community.
It depends on what you're after. If you just want a list of changes, sure, but if you
want some idea as to whether or not those changes introduce instability then you need to
look more comprehensively. It is rather difficult to look at a small list of changes and
gauge how well/ill it will effect the stability of the distribution going forward. Either
you're too liberal and we have rampant instability, or you're too draconian and we
have very strict barriers to entry, which are suddenly removed once a product goes GA.
Using the same criteria prior to and post GA to assess the validity of an update makes
sense to me.
> and let your packaging peers decide whether or not the
> update was safe enough to go into the release,
But they can't, not really. For instance, a proventester may install my mdadm
package, but if they don't have a raid array, they can't decide anything. Where
as if they had access to the code diffs and could tell that all I did was change a typo in
the udev rules file, and verify for themselves via code inspection that the code as it
stands in the repo can't work and the fix should work, then they could make an
educated contribution.
Sadly we have far far too few real developers who could do that comparison.
Because the testing repo doesn't really present changes, but
only a final product, there is no possibility for my peers to look at my changes and say
"Yeah, that's should be a safe change, let it in, and if it breaks then we'll
fix it". So the judgement call that I mentioned in my previous email is taken
entirely out of the loop, and there is no ability for my peers to make any contribution to
whether or not a package should be allowed in *except* unit testing, there is no ability
for them to easily do an actual analysis of the changes and ma
ke an engineering decision versus a QA decision.
I think that's largely because we don't have a community of engineers. We have a
community of /packagers/ who are able to cause packages to be built, and are able to do
some measure of QA to see if those builds work, but do not have the skill set to look at a
code diff and give a honest opinion as to whether or not the change is "safe".
If the makeup of our community were different, then you would have a case for your
proposal. I just don't believe we have the community it would take to accomplish it
on a large scale.
> thus we have bodhi
> and updates-testing as a gateway to get into the release.
It's a gateway, I just don't think it serves as useful a purpose as it was
intended to.
The question though really is whether or not it is more useful than a few (like 4) release
engineers looking at trac tickets and making best guesses depending on whatever the ticket
reporter put in the ticket about the change. We were trying to improve the workflow, not
perfect it. Has it been improved?
- jlk