On Wed, May 3, 2017 at 6:08 AM, Adam Williamson <adamwill@fedoraproject.org> wrote:
Kamil's suggestion was that rather than a result just providing a
single scenario value that's intended to be a not-very-human-readable
string that represents, in some sense, the entire 'scenario' for a
test, a result (let's say it's for the test update.base_selinux) would
provide something like this:

scenario_keys: [arch, machine, flavor, version, distri]
arch:          x86_64
machine:       uefi
flavor:        server
version:       26
distri:        fedora

Then if you're an rdb consumer and you wanted to be really sure you had
the entire 'description' of a given test you could represent it just by
displaying *all* those values, with or without the key name depending
on how it fit into your consumer's interface. But your consumer could
also think like this:

OK, I've got 50 results for this item. The arch, machine, version and
distri for *every* result are the same, so there's probably no need to
show those. But some of the results have different flavors. So I should
probably use 'testcase name - flavor' as the identifier, like so:
"update.base_selinux - server". Hey, that looks pretty good!


Thinking about this...

If two results have the same scenario-related extra args, but differ in "selinux: 0" vs "selinux: 1", the smart diff you proposed would just display "update.base_selinux - 0" and "update.base_selinux - 1". That's not very helpful.

Of course you can say you'll display the key name as well, so "update.base_selinux - selinux=0" and "update.base_selinux - selinux=1". That looks OK, but it assumes they key name is readable/understandable - it doesn't have to. Also, if the results differ in several fields, the whole line starts to be quite long, e.g.: "flavor=server;machine=uefi;arch=x86_64;selinux=1". Compare this with "server.uefi.x86_64.selinux-on".

What I'm trying to point out is that no matter what we do, this scenario description can very easily become unreadable or too long. OpenQA results scenario happens to be structured in some way, other systems will structure it in a different way. If we intend to display scenario strings to distinguish results in UI, we'll either need to live up with complex identifiers, or ask people to use readable strings and not overpopulate them with unnecessary data (or come up with a completely different solution). And assuming we'll ask people to make scenario short and to the point, it seems easier to me to use the scenario string in its current form ("server.uefi.x86_64.selinux-on") rather than display the full args ("flavor=server;machine=uefi;arch=x86_64;selinux=1").

But I don't have strong opinions here. All the approaches are usable, provided task creators collaborate in making it readable. I just slightly prefer the approach we already (at least partially) implemented rather than re-doing it again.