Greetings,
Now I am trying to integrate mediakit_sanity test into AutoQA, mediakit sanity test includes tests of media size, checksum, repoclosure, conflicts. I find it is trouble to deal with the test results, since it is impossible to specify which test is FAILED, we can only give a general result to all tests self.detail.result=FAILED even some tests passed.
So do you think it it OK that I split the test into small tests? apparently, there will be many test in the tests directory. Or do you have any suggestion to solve this? Thanks.
Hongqing
Greetings,
Now I am trying to integrate mediakit_sanity test into AutoQA, mediakit sanity test includes tests of media size, checksum, repoclosure, conflicts. I find it is trouble to deal with the test results, since it is impossible to specify which test is FAILED, we can only give a general result to all tests self.detail.result=FAILED even some tests passed.
So do you think it it OK that I split the test into small tests? apparently, there will be many test in the tests directory. Or do you have any suggestion to solve this? Thanks.
Hongqing
Hi,
you can report multiple results in a single test run. For example upgradepath test checks all proposed updates and reports PASS/FAIL for every one of them individually. The code is then a little more complicated. Basically you create a separate TestDetail instance for every "item" you want to have reported (in your case one for media size, one for checksum, ...): http://git.fedorahosted.org/git/?p=autoqa.git;a=blob;f=tests/upgradepath/upg... In the end you reports all those TestDetail results manually: http://git.fedorahosted.org/git/?p=autoqa.git;a=blob;f=tests/upgradepath/upg...
But I wonder, is it all really necessary? You can use Summary Details section to provide something like this: Media size: PASS Checksum: PASS Repoclosure: PASS Conflicts: FAIL and then set the overall result to PASS only if all the four subsections are PASS. This way if we encounter failed mediakit_sanity, it's easy to open the log and immediately see which parts failed.
Kamil
autoqa-devel@lists.fedorahosted.org