Hello,
I'd like to get a few clarifications regarding the default test playbook
example here:
https://fedoraproject.org/wiki/CI/Tests#Wrapping_a_script_test
First, the example seems to suggest that all binaries should be placed in
/usr/local/bin:
- name: Create the folder where we will store the tests
action: file state=directory path={{ item }} owner=root group=root
with_items:
- /usr/local/bin
- name: Install the test files
copy: src={{ item.file }} dest=/usr/local/bin/{{ item.dest }} mode=0755
with_items:
- {file: test-simple, dest: test-simple }
But I don't see such a requirement in the SI. Why shouldn't I run the test
executable from its current location (i.e. the git checkout directory)? And
if they can, why make the example unnecessarily complex?
Second, I'm a bit confused by the exec command in the test execution line:
- name: Execute the tests
shell: exec > {{ artifacts }}/output && /usr/local/bin/test-simple
2>&1
I've never seen this usage so I had to study what this does exactly in the
bash man page. I believe it's the same thing as this:
shell: /usr/local/bin/test-simple >{{ artifacts }}/output 2>&1
So why exactly don't we use this more readable and more often used syntax
to redirect a process output? An example should be as simple as possible.
Third, SI says:
The playbook and its test suite or test framework:
MUST place the main readable output of the test suite into a test.log
file in the artifacts variable folder. This MUST happen even if some of the
test suites fail.
However, the default example doesn't contain any reference to *test.log*
and uses *output* file instead. Shouldn't those examples above redirect to
*test.log* instead? Or is *test.log* not supposed to be an stdout+stderr
output of the main test process? I don't exactly understand the definition
from SI, what *main readable output* is supposed to be. When the test suite
consists of X individual tests, are they supposed to concatenate their
output to *test.log*?
It would be great to have these clarified.
Thanks,
Kamil