2010-02-12 - AutoQA resultsdb design session

# AutoQA ResultsDB Discussion # Date: 2010-02-12 # Time: 15:00 UTC (10:00 EST, 16:00 CET) # Participants - kparal, wwoods, jskladan, jlaska = Introduction = Discussed how we would handle the meeting - http://en.wikipedia.org/wiki/Design_thinking = Define = * Decide what issue(s) we're trying to solve * Kamil described how AutoQA tests run now -- results currently sent to mailing list * Should this track general test results (from koji or other), or just AutoQA test results? -- focus on AutoQA test results * A common store that different views can draw from * What test results do we want to see? * Install test results (against trees/release candidates) * Package tests - store metadata to the tests [we should define required metadata] * Build tests * Not trying to write different views ... just focus on a common results aggregator * Provide storage and display of results for test writers * More useful, context-specific views of test result data * Agree on who the audience is * Focus on developers/testers * Possibly Fedora community as a view to the data * Establish a glossary of terms * State the obvious * we want to store only "our" data (~ test results & metadata), not including other tools data - other than that, we should store a link to brew/koji/... result along with the test results * Determine what success would look like * A results db holding results for multiple types of tests * API for storing and retrieving data * Different views to the same data - separate frontends (or one frontend with several views for like installation test/package test) = Research = * Review the history of the problem * israwhidebroken provides a db * results sent to the mailing list * Remember existing obstacles * RHTS/beaker does a *great* job gathering results and relevant log files, but * Collect examples of other attempts to solve the problem * israwhidebroken provides a db and a front-end to the data * spike source - defined data formats? * RHTS - dkovalsk/bpeck? * how does it store data * which "fields" should be required/optional for tests in general etc. * TCMS (nitrate) - how is it storing test results, what information is recorded * rpmdiff - dmalcolm? * TPS (Package sanity) - psplichal * Research questions for other tools * What data is required for generic test results? * What information is included (logs, timestamps?) * How is benchmark data recorded? * How is the data used? * What are the most important data to collect? * Note project supporters/investors/critics * QA needs this to aid developing front-ends for test groups (install, package) = Tasks = * [jlaska] - Setup a wiki page to store research * [kparal] - ask dkovalsky about RHTS, psplicha about TPS * [wwoods] - nitrate results research * [jlaska] - rpmdiff results research = Next meeting = * 2010-02-19 - E-checkin * 2010-02-26 - next gobby+ftalk session