With five Test Managers, become a Test Executive...
Sorry, Adam, just an attempt to give you opportunity to smile.
I am serious about the scalability issue, however. As you indicated, the
typical FOSS activity does not map readily into a classic business
hierarchical organization chart. I do not want that. It would likely be
difficult, unproductive, and even incite antagonism.
This does not mean all strategies will fail. I do not know any sure-fire
way to multiply the effectiveness of QA, but your efforts to document
what QA can do and what others might do seem headed in a useful
direction.
Test automation is useful, especially if developers can be engaged in its
creation. Abrt and other infrastructure is productive, and can be
improved. Documentation - how to test, how to collect and report
information about errors - is useful, especially when new testers are
sought. Bugzilla is not all that easy or intuitive to use. Some sort of
coverage map might help: what pieces are people currently testing or
planning to test; what parts have no people active. All depends on
cost/benefit estimation. Direct implementation is costly; leverage from
work done by others may be key.