# AutoQA ResultsDB Discussion
# Date: 2010-02-26
# Time: 15:00 UTC (10:00 EST, 16:00 CET)
# Participants - wwoods, jlaska, kparal, jskladan
= Review definition/goals =
https://fedorahosted.org/pipermail/autoqa-devel/2010-February/000201.html
Goals
* One central database to store test results (audience QA)
* Multiple front-ends to present stored results (audience QA/developers)
= Recap of research =
* Overview -
https://fedoraproject.org/wiki/AutoQA_resultsdb_approaches
Spikesource schema -
http://dev.spikesource.com/wiki/index.php/Trpi-schema_view
* No concept of WAIVE results
RHTS/Beaker -
http://beaker.fedorahosted.org/
* stores log files directly
* presents most important data on main results page (e.g. stdout)
* Hiearchy - Job contains RecipeSet(s), which contains Recipe(s), which
contains test(s), which contain phase(s)
Beakerlib
* Allows specification of test phases
* Provides common test writing format in bash
* beakerlib xml journal
http://jskladan.fedorapeople.org/beakerlib_format.xml
** test which produced the output:
http://git.fedorahosted.org/git/?p=autoqa.git;a=blob;f=tests/beakerlib_in...
RATS
* Test results linked by a unique key (timestamp of tree being tested)
* install.py test also uses concept of "phases" -
http://git.fedorahosted.org/git/?p=autoqa.git;a=blob;f=tests/rats_install...
* Multiple tests (2), each with multiple phases - Viewed from single
location -
http://jlaska.fedorapeople.org/rats.png
= Ideate =
* Define an output standard for each test type for easier inspection of
results (similar to beakerlib)
* TCMS - something we're gonna forget about for now (db scheme review in
'open topics')
= Terminology =
* test case - one unit with a single result (but may contain several
phases)
* test phase - parts of a test case, their results are not stored
separately in the results db, but define pass/fail result of the test
case itself (internally for the test)
* test plan - contains several test cases, provides meta data around
test cases (responsibility, workflow, high-level criteria etc...)
* mapping to the beakerlib_format.xml - depends on particular test, for
initscripts checking: <log> ~ test case, <phase> ~ test phase
= Open topics =
* How to handle policy (waive) in tools?
**
https://fedoraproject.org/wiki/AutoQA_resultsdb_approaches#DB_schema
- rpmdiff solution
* Is defining a common test writing and output format the same as
providing a results front-end/workflow?
* Examine nitrate/testopia resultsdb format?
* Test case results vs autotest job results (unhandled exception, etc)
= Tasks =
* [wwoods] - Examine beakerlib output format if it is sufficient for our
purposes
* [jlaska] - Examine nitrate/testopia DB schema
* [kparal + jskladan] - Define personas and write use cases
Example:
https://fedoraproject.org/wiki/No_frozen_rawhide_announce_plan#Use_Cases
Wiki page:
https://fedoraproject.org/wiki/AutoQA_resultsdb_use_cases
* [kparal + jskladan] - Propose some initial DB schema
= Next Meeting =
* 2010-03-05 - mailing list checkin (conference call if needed)