#298: test.py - split postprocess_iteration reporting into standalone methods ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: 0.5.0 Component: core | Keywords: ----------------------+----------------------------------------------------- At the moment, postprocess_iteration() handles the whole reporting/sending results. When the run_once ends, this method takes the content of self.{result, summary, highlights, outputs} and automagically sends an email to the mailing list, creates output.log file, etc.

This approach was very reasonable for tests which actually test just one thing (update/build/etc), because just a single result is to be sent.

With the new tests like Depcheck and Upgradepath, we'd love to be able to force-send several results as the test proceeds - e.g. to be able to send bodhi comment to every update, but send just one overall email...

So what I'd like to have is:

1) Take postprocess_iteration, and split it into standalone methods according to the 'destination' of the report (e.g. send_email(), create_output_log(), ...). These will be able to take {result, summary, highlights, outputs} parameters, which will override the 'automagicall' self.{result, summary, highlights, outputs}. I.e. if I set result parameter, summary, highlights and outputs will be filled using the 'self' variables, and so on.

2) Take the bodhi-reporting method (as used in depcheck and upgradepath), and move it to the AutoQATest class.

3) Add a method report_results(), again with the same parameters ({result, summary, highlights, outputs}) and one more (e.g. a dictionary) to control which reporting methods to call (i.e. to be able to say "send email, store in resultsdb, but do not send bodhi comment"). By default, this will call all the reporting routines.

4) In postprocess_iteration(), call the report_results(), so the 'automagical reporting' behaviour is not changed.

5) Add a variable, which will turn on/off the call of report_results() (True as default) in postprocess_iteration() (needs to be some attribute of AutoQATest class, since postprocess_iteration() does not take arguments). This is for the wrapper-writer, to be able to control if results are to be sent in the postprocess_iteration() or not (imagine Upgradepath - we will report all the results in the run_once(), and do not need postprocess iteration to actually send anything).

-------------------------------

This is also preparation for resultsdb, because it solves the problem with reporting multiple results from one test. Also turning the resultsdb reporting on is a matter of adding one method, and calling it from report_results() (and adding more types of reporting [I'm looking at you fedora message bus!] in the future will be also this simple.