test March 2023

test@lists.fedoraproject.org

29 participants
38 discussions

reversible dual-boot test station
by Chris Murphy 23 Feb '24

23 Feb '24

Hi, I mentioned this in a QA meeting, and have given it enough testing that I think it's broadly usable. If desired it can be copied out of my user account and put up somewhere where QA folks will see it and can modify it as issues or improvements are discovered. What is it? The idea is to produce a system that can confidently be used for baremetal testing, without risking the primary operating system. While VM's are a great way to test, it's also a really idealized environment that tends to not expose an assortment of bugs that affect particular hardware. But then quite a lot of folks reasonably don't want to upgrade their daily use hardware early on, because they don't want to always have to debug things, or have to figure out how to undo the upgrade if it really goes badly. Therefore, I present a dual-boot setup offering: * no re-partitioning; * no installation step, instead system upgrade is used; * reversibility, or undoability, i.e. with just a few steps you can delete the "test OS". https://fedoraproject.org/wiki/User:Chrismurphy/Draft/dualboot_teststation -- Chris Murphy

2 5

Re: Thoughts welcome: interface between automated test gating and the "critical path"
by Adam Williamson 16 Aug '23

16 Aug '23

On Fri, 2022-09-02 at 08:37 +0000, Zbigniew Jędrzejewski-Szmek wrote: > > > > Now, because I glued openQA to the critpath because it was handy, there > > are two sets of consequences to a package being in critical path: > > > > 1. Tighter Bodhi requirements > > 2. openQA tests are run, and results gate the update (except Rawhide) > > > > So, one of the implicit questions here is, is it OK to keep twinning > > these two sets of consequences, or should we split them up? Splitting > > them up kinda implies answer 2) from my original mail: "Keep the > > current "critical path" concept but define a broader group > > of "gated packages" somewhere". Because we would then need some new > > concept that isn't "critical path". As I said, that's more *work* - > > it'd require us to write new code in several places[0]. Even if we > > decide it'd be nice to do this, is it nice *enough* to be worth doing > > that work? > > I'd still vote for keeping a single critpath list and using it as > "the list of packages that require extra care and testing". > > As you describe, the original meaning of critpath has shifted, but > it's because the way we do updates and QA has also shifted. Doing > gating tests for a package seems much more useful than just keeping > it longer in 'updates-testing' in hope that somebody discovers an > important regresion in the second week. Well, there's a caveat there - openQA doesn't test everything. On the whole we cover quite a lot with the set of tests that gets run on updates, but there's certainly lots of potential for there to be important bugs it misses, that a human tester might catch. So I think there is still a case for the higher karma requirements too. > > So yeah, I don't think it makes sense to do the extra work to split > the concepts. Also because we have way too many concepts and processes > in Fedora already. On the whole, though, I agree with you. I just don't trust my own opinion because it's obviously biased by what's convenient for me. :D > > If we don't think it's worth doing that work, then we're kinda stuck > > with openQA glomming onto the critpath definition to decide which > > updates to test and gate, because I don't have any other current viable > > choices for that, really. And we'd have to figure out a critpath > > definition that's as viable as possible for both purposes. > > > > > > BTW, one other thought I've had in relation to all this is that we > > could enhance the current critpath definition somewhat. Right now, it's > > built out of package groups in comps which are kinda topic-separated: > > there's a critpath-kde, a critpath-gnome, a critpath-server, and so on. > > But the generated critical path package list is a monolith: it doesn't > > distinguish between a package that's on the GNOME critpath and a > > package that's on the KDE critpath, you just get a big list of all > > critpath packages. It might be nice if we actually did distinguish > > between those - the critpath definition could keep track of which > > critpath topic(s) a package is included in, and Bodhi could display > > that information in the web UI and provide it via the API. That way > > manual testers could get a bit more info on why a package is critpath > > and what areas to test, and openQA could potentially target its test > > runs to conserve resources a bit, though this might require a bit more > > coding work on the gating stuff now I think about it. > > That sounds useful. We only need a volunteer to figure out the details > and do the work ;) I actually did a huge rewrite of the thing that generates the critpath data this week, and it probably wouldn't be tooooo much work, honestly. The most annoying bit would be the Bodhi frontend stuff, but that's because I'm bad at frontend dev in general. :P But yeah, this is definitely off in sky-castle land. I'll add it to my ever-growing list of sky-castle projects to do when I get a couple of years of spare time... -- Adam Williamson Fedora QA IRC: adamw | Twitter: adamw_ha https://www.happyassassin.net

3 4

Plan / proposal: enable openQA update testing and potentially gating on Rawhide updates
by Adam Williamson 09 Jun '23

09 Jun '23

Hi folks! We've had openQA testing of updates for stable and branched releases, and gating based on those tests, enabled for a while now. I believe this is going quite well, and I think we addressed the issues reported when we first enabled gating - Bodhi's gating status updates work more smoothly now, and openQA respects Bodhi's "re-run tests" button so failed tests can be re-triggered. A few weeks ago, I enabled testing of Rawhide updates in the openQA lab/stg instance. This was to see how smoothly the tests run, how often we run into unexpected failures or problems, and whether the hardware resources we have are sufficient for the extra load. So far this has been going more smoothly than I anticipated, if anything. The workers seem to keep up with the test load, even though one out of three worker systems for the stg instance is currently out of commission (we're using it to investigate a bug). We do get occasional failures which seem to be related to Rawhide kernel slowness (e.g. operations timing out that usually don't otherwise time out), but on the whole, the level of false failures is (I would say) acceptably low, enough that my current regime of checking the test results daily and restarting failed ones that don't seem to indicate a real bug should be sufficient. So, I'd like to propose that we enable Rawhide update testing on the production openQA instance also. This would cause results to appear on the Automated Tests tab in Bodhi, but they would be only informational (and unless the update was gated by a CI test, or somehow otherwise configured not to be pushed automatically, updates would continue to be pushed 'stable' almost immediately on creation, regardless of the openQA results). More significantly, I'd also propose that we turn on gating on openQA results for Rawhide updates. This would mean Rawhide updates would be held from going 'stable' (and included in the next compose) until the gating openQA tests had run and passed. We may want to do this a bit after turning on the tests; perhaps Fedora 37 branch point would be a natural time to do it. Currently this would usually mean a wait from update submission to 'stable push' (which really means that the build goes into the buildroot, and will go into the next Rawhide compose when it happens) of somewhere between 45 minutes and a couple of hours. It would also mean that if Rawhide updates for inter-dependent packages are not correctly grouped, the dependent update(s) will fail testing and be gated until the update they depend on has passed testing and been pushed. The tests for the dependent update(s) would then need to be re- run, either by someone hitting the button in Bodhi or an openQA admin noticing and restarting them, before the dependent update(s) could be pushed. In the worst case, if updated packages A and B both need the other to work correctly but the updates are submitted separately, both updates may fail tests and be blocked. This could only be resolved by waiving the failures, or replacing the separate updates with an update containing both packages. All of those considerations are already true for stable and branched releases, but people are probably more used to grouping updates for stable and branched than doing it for Rawhide, and the typical flow of going from a build to an update provides more opportunity to create grouped updates for branched/stable. For Rawhide the easiest way to do it if you need to do it is to do the builds in a side tag and use Bodhi's ability to create updates from a side tag. As with branched/stable, only critical path updates would have the tests run and be gated on the results. Non-critpath updates would be unaffected. (There's a small allowlist of non-critpath packages for which the tests are also run, but they are not currently gated on the results). I think doing this could really help us keep Rawhide solid and avoid introducing major compose-breaking bugs, at minimal cost. But it's a significant change and I wanted to see what folks think. In particular, if you find the existing gating of updates for stable/branched releases to cause problems in any way, I'd love to hear about it. Thanks folks! -- Adam Williamson Fedora QA IRC: adamw | Twitter: adamw_ha https://www.happyassassin.net

3 4

Fedora Linux 38 blocker status summary
by Ben Cotton 07 Apr '23

07 Apr '23

We've branched F38 from Rawhide, so it's time to start everyone's favorite Friday email from Ben! The F38 Beta freeze begins on 21 February. The current target release date is the early target date (2023-03-14). Action summary ==================== Accepted blockers ----------------- 1–3. distribution — {Workstation,Everything,Server} boot x86_64 image exceeds maximum size — ASSIGNED ACTION: Relevant Teams to reduce image size or increase the limit Proposed blockers ----------------- 1. anaconda — installer fails to boot in text mode or rescue mode with systemd 253 — MODIFIED ACTION: Maintainers to build an update that includes upstream PR 2. grub2 — ext4 filessystem support missing — NEW ACTION: Maintainer to diagnose issue NEEDINFO: ausil 3. kwin — kwin_wayland often crashed when used as the sddm Wayland compositor and logging out of Plasma resulting in a black screen — NEW ACTION: Maintainer to diagnose issue 4. mesa — KDE crashes on start with mesa-23.0.0~rc3-3.fc38 — ASSIGNED ACTION: Maintainer to diagnose issue Bug-by-bug detail ============= Accepted blockers ----------------- 1–3. distribution — {Workstation,Everything,Server} boot x86_64 image exceeds maximum size — ASSIGNED https://bugzilla.redhat.com/show_bug.cgi?id=2149246 https://bugzilla.redhat.com/show_bug.cgi?id=2151495 https://bugzilla.redhat.com/show_bug.cgi?id=2151497 Image sizes exceed the specified limits. The choices are to either shrink the image by removing packages or to riase the limits. Proposed blockers ----------------- 1. anaconda — https://bugzilla.redhat.com/show_bug.cgi?id=2165433 — MODIFIED installer fails to boot in text mode or rescue mode with systemd 253 anaconda is writing systemd units to an unexpected area. Upstream PR 4534 contains a candidate fix: https://github.com/rhinstaller/anaconda/pull/4534 2. grub2 — https://bugzilla.redhat.com/show_bug.cgi?id=2166412 — NEW ext4 filessystem support missing Between grub2-2.06-76 and grub2-2.06-78, grub2 apparently lost the ability to detect ext4 filesystems. 3. kwin — https://bugzilla.redhat.com/show_bug.cgi?id=2168034 — NEW kwin_wayland often crashed when used as the sddm Wayland compositor and logging out of Plasma resulting in a black screen Logging out of Plasma sometimes triggers a kwin crash. This may be limited to running in a virtual machine. 4. mesa — https://bugzilla.redhat.com/show_bug.cgi?id=2164667 — ASSIGNED KDE crashes on start with mesa-23.0.0~rc3-3.fc38 A recent mesa update caused both GNOME and KDE to crash on start. Update FEDORA-2023-40b973fa06 fixed GNOME, but KDE is still seeing this issue. The root cause is still uncertain. -- Ben Cotton He / Him / His Fedora Program Manager Red Hat TZ=America/Indiana/Indianapolis

4 11

new criterion proposal: Window manager functionality
by Kamil Paral 06 Apr '23

06 Apr '23

Related to a recently proposed blocker [1][2], I propose a new release criterion which would cover such cases. Here it is: ~~~~~~~~~~~~ The desktop environment must work correctly when displaying common application content - that includes regular application windows, video output, games and possibly other common content. Regular operations like windows close/resize/maximize/minimize/fullscreen (when supported/applicable), windows switching, using windows on virtual workspaces, and similar common operations must behave as expected. Footnote - "What does this cover?": A small number of misbehaving applications is not a violation of this criterion. The goal of this criterion is to ensure general functionality for different types of applications and their operations. An example of a violation would be e.g. if a significant number of QT applications couldn't be resized (but they should be), or if all Java applications failed to display any content. ~~~~~~~~~~~~~ I think this criterion could target F38 Beta Release Criteria [3] or Basic Release Criteria [4]. Please tell me your thoughts, thanks! Kamil [1] https://pagure.io/fedora-qa/blocker-review/issue/1102 [2] https://bugzilla.redhat.com/show_bug.cgi?id=2178167 [3] https://fedoraproject.org/wiki/Fedora_38_Beta_Release_Criteria [4] https://fedoraproject.org/wiki/Basic_Release_Criteria

6 13

2023-04-03 @ 15:00 UTC - Fedora QA Meeting
by Adam Williamson 03 Apr '23

03 Apr '23

# Fedora Quality Assurance Meeting # Date: 2023-04-03 # Time: 15:00 UTC (https://fedoraproject.org/wiki/Infrastructure/UTCHowto) # Location: #fedora-meeting on irc.libera.chat Greetings testers! It's meeting time again! If anyone has any other items for the agenda, please reply to this email and suggest them! Thanks. == Proposed Agenda Topics == 1. Previous meeting follow-up 2. Window manager release criterion proposal 3. Fedora 38 status 4. Test Day / community event status 5. Open floor -- Adam Williamson (he/him/his) Fedora QA Fedora Chat: @adamwill:fedora.im | Mastodon: @adamw@fosstodon.org https://www.happyassassin.net

2 1

test kernels 6.2.x in F37 - an ultimate killer
by lejeczek 31 Mar '23

31 Mar '23

Hi guys. Just to let you know - all the kernels I tried, x.5 tin x.7 , kill the system... .. when some video in a browser - as I do not playback any videos allow cannot comment there - is played or paused and some resizing or switching or showing thumbnails/miniatures (tabs) is show/played/takes place, then Fedora starts blinking my both monitors, laptop's builtin & external Dell. Blinking slow, black & desktop view alternately, at ~1 sec intervals then about ~1 min later both monitors switch black & stay that way. From the very blink system is irresponsible - hard power-cycle is a must. Interestingly(?) audio continues - until of course, all goes black - to playback okey. Nothing else is from koji/test, up to date from repos OS. System - Thinkpad e14 g2 (AMD) regards, L.

3 4

Fedora 38 Upgrade Test Day 2023-03-28
by Sumantro Mukherjee 29 Mar '23

29 Mar '23

Hey All, As we come closer to Fedora 38 release dates, it's time to test upgrades. Fedora 38 has a lot of changesets and it becomes essential that we test the graphical upgrade methods as well as the command line. As a part of this test day[0], we will test upgrading from a full updated, F36 and F37 to the F38 for all architectures(x86_64,ARM,aarch 64) and variants(WS,cloud,server,silverblue,IoT). The results can be submitted here[1] As usual, we hang out on the #fedora-test-day, should you have questions. [0] https://fedoraproject.org/wiki/Test_Day:2023-03-28_F38_Upgrade_Test_Day [1] https://testdays.fedoraproject.org/events/154 -- //sumantro Fedora QE TRIED AND PERSONALLY TESTED, ERGO TRUSTED

1 0

Possible problem with texi2dvi on F38
by José Abílio Matos 28 Mar '23

28 Mar '23

Hi, while building from source the octave development (to become octave 9) I get a problem at the end: """ TEXI2DVI doc/interpreter/octave.dvi /usr/bin/texi2dvi: Your TeX installation appears to be broken - texi2dvi cannot determine auxiliary files output from a TeX run. You may need to install TeX, or change the values of your PATH, TEX or PDFTEX environment variables in order to proceed. """ In the end this is harmless since the only issue is that the documentation is not converted to pdf. In Fedora 38 texlive has been updated to texlive 2022 and I do not have any problem with any other application that uses texlive (e.g. when using latex). Does anyone has seen a problem like this before? If not I will fill a report in bugzila. Regards, -- José Abílio Matos

1 0

Fedora 38 CoreOS Test Week is underway
by Sumantro Mukherjee 27 Mar '23

27 Mar '23

Hey All, The Fedora 38 CoreOS Test Week focuses on testing FCOS based on Fedora 38. The FCOS next stream is already rebased on Fedora 38 content, which will be coming soon to testing and stable. To prepare for the content being promoted to other streams the Fedora CoreOS and QA teams have organized test days on Tues, March 28, 2023 (results accepted through Sun , 02 April). Refer to the wiki page[0] for links to the test cases and materials you’ll need to participate. The FCOS and QA team will meet and communicate with the community sync on a Google Meet[1] at the beginning of test week and async over multiple matrix/element channels. Read more about them in this announcement[2]. [0] http://fedoraproject.org/wiki/Test_Day:Fedora_38_CoreOS [1] https://meet.google.com/dsb-wixf-kqk [2] https://lists.fedoraproject.org/archives/list/coreos@lists.fedoraproject.or… -- //sumantro Fedora QE TRIED AND PERSONALLY TESTED, ERGO TRUSTED

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

test March 2023