Hi folks!
A few of you have seen this already, but just wanted to send out a wider announcement: having been deservedly shamed by pbrobinson, I spent most of this week working on aarch64 testing in openQA, and am happy to announce that:
* Quite a few existing issues in tests have been fixed * Testing coverage has been enhanced a lot to cover the Minimal, Server and Workstation disk images, with the full set of Base tests run on each * aarch64 testing is now enabled on the production openQA instance!
We've had testing running on the lab (formerly known as staging) instance for a long time now, and I've been meaning to also enable it on production for over a year, but it kept getting delayed by this and that (most recently, the infra move). But now it's done.
The consequences of this - beyond, of course, that you can now see aarch64 tests in the production openQA web UI - should be that the aarch64 results show up in the "compose check report" emails, and will also show up in the validation wiki pages, just like x86_64 results. openQA should fill in quite a lot of boxes in the Installation, Base and Cloud pages for future validation events, taking some of the test load off of manual testers. Next week I intend to enable most of the Desktop tests on the Workstation disk image too.
I have not enabled updates testing yet, because we just don't have the capacity, unfortunately. If we can get more testing capacity we could do this.
On Fri, Oct 30, 2020 at 11:31 PM Adam Williamson adamwill@fedoraproject.org wrote:
Hi folks!
A few of you have seen this already, but just wanted to send out a wider announcement: having been deservedly shamed by pbrobinson, I spent most of this week working on aarch64 testing in openQA, and am happy to announce that:
- Quite a few existing issues in tests have been fixed
- Testing coverage has been enhanced a lot to cover the Minimal, Server
and Workstation disk images, with the full set of Base tests run on each
- aarch64 testing is now enabled on the production openQA instance!
We've had testing running on the lab (formerly known as staging) instance for a long time now, and I've been meaning to also enable it on production for over a year, but it kept getting delayed by this and that (most recently, the infra move). But now it's done.
The consequences of this - beyond, of course, that you can now see aarch64 tests in the production openQA web UI - should be that the aarch64 results show up in the "compose check report" emails, and will also show up in the validation wiki pages, just like x86_64 results. openQA should fill in quite a lot of boxes in the Installation, Base and Cloud pages for future validation events, taking some of the test load off of manual testers. Next week I intend to enable most of the Desktop tests on the Workstation disk image too.
Thanks for your work on this Adam, it's much appreciated, this is awesome news!
Hi Adam and Peter,
these are really great news!
Do you guys also have plans to enable some bare metal tests for some supported boards for aarch64?
Cheers,
Dan
Adam Williamson adamwill@fedoraproject.org writes:
Hi folks!
A few of you have seen this already, but just wanted to send out a wider announcement: having been deservedly shamed by pbrobinson, I spent most of this week working on aarch64 testing in openQA, and am happy to announce that:
- Quite a few existing issues in tests have been fixed
- Testing coverage has been enhanced a lot to cover the Minimal, Server
and Workstation disk images, with the full set of Base tests run on each
- aarch64 testing is now enabled on the production openQA instance!
We've had testing running on the lab (formerly known as staging) instance for a long time now, and I've been meaning to also enable it on production for over a year, but it kept getting delayed by this and that (most recently, the infra move). But now it's done.
The consequences of this - beyond, of course, that you can now see aarch64 tests in the production openQA web UI - should be that the aarch64 results show up in the "compose check report" emails, and will also show up in the validation wiki pages, just like x86_64 results. openQA should fill in quite a lot of boxes in the Installation, Base and Cloud pages for future validation events, taking some of the test load off of manual testers. Next week I intend to enable most of the Desktop tests on the Workstation disk image too.
I have not enabled updates testing yet, because we just don't have the capacity, unfortunately. If we can get more testing capacity we could do this. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net _______________________________________________ arm mailing list -- arm@lists.fedoraproject.org To unsubscribe send an email to arm-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/arm@lists.fedoraproject.org
On Sun, 2020-11-01 at 21:38 +0100, Dan Čermák wrote:
Hi Adam and Peter,
these are really great news!
Do you guys also have plans to enable some bare metal tests for some supported boards for aarch64?
Hi, Dan. If by "enable" you mean "run in openQA", then the answer is "not immediately, no". We don't do any bare metal testing in openQA on any arches, and we're not set up for it at all. openQA has some capacities in that direction but it requires hardware with specific capabilities (and/or a lot of work writing a backend around the capabilities that are present and adapting tests to it). Basically, at minimum we need a reliable way to get console input into and out from the device (usually a serial console), and that just lets us test stuff that can be input over a console line; to do any of the graphical interactive testing, we ideally need some kind of management interface which supports VNC, since VNC is the most mature and capable openQA graphical/input backend. And then we need to acquire some of that hardware and either deploy it in the Fedora infra data center (which has space issues, apparently) or try and make it work from outside, which should theoretically be possible but again is something we have not done before.
So: not right now. :)
We *do* already do bare metal testing manually as part of release validation, quite a lot of it.
-
Adam Williamson -
Dan Čermák -
Peter Robinson