Plan / proposal: enable openQA update testing and potentially
gating on Rawhide updates
by Adam Williamson
Hi folks!
We've had openQA testing of updates for stable and branched releases,
and gating based on those tests, enabled for a while now. I believe
this is going quite well, and I think we addressed the issues reported
when we first enabled gating - Bodhi's gating status updates work more
smoothly now, and openQA respects Bodhi's "re-run tests" button so
failed tests can be re-triggered.
A few weeks ago, I enabled testing of Rawhide updates in the openQA
lab/stg instance. This was to see how smoothly the tests run, how often
we run into unexpected failures or problems, and whether the hardware
resources we have are sufficient for the extra load.
So far this has been going more smoothly than I anticipated, if
anything. The workers seem to keep up with the test load, even though
one out of three worker systems for the stg instance is currently out
of commission (we're using it to investigate a bug). We do get
occasional failures which seem to be related to Rawhide kernel slowness
(e.g. operations timing out that usually don't otherwise time out), but
on the whole, the level of false failures is (I would say) acceptably
low, enough that my current regime of checking the test results daily
and restarting failed ones that don't seem to indicate a real bug
should be sufficient.
So, I'd like to propose that we enable Rawhide update testing on the
production openQA instance also. This would cause results to appear on
the Automated Tests tab in Bodhi, but they would be only informational
(and unless the update was gated by a CI test, or somehow otherwise
configured not to be pushed automatically, updates would continue to be
pushed 'stable' almost immediately on creation, regardless of the
openQA results).
More significantly, I'd also propose that we turn on gating on openQA
results for Rawhide updates. This would mean Rawhide updates would be
held from going 'stable' (and included in the next compose) until the
gating openQA tests had run and passed. We may want to do this a bit
after turning on the tests; perhaps Fedora 37 branch point would be a
natural time to do it.
Currently this would usually mean a wait from update submission to
'stable push' (which really means that the build goes into the
buildroot, and will go into the next Rawhide compose when it happens)
of somewhere between 45 minutes and a couple of hours. It would also
mean that if Rawhide updates for inter-dependent packages are not
correctly grouped, the dependent update(s) will fail testing and be
gated until the update they depend on has passed testing and been
pushed. The tests for the dependent update(s) would then need to be re-
run, either by someone hitting the button in Bodhi or an openQA admin
noticing and restarting them, before the dependent update(s) could be
pushed.
In the worst case, if updated packages A and B both need the other to
work correctly but the updates are submitted separately, both updates
may fail tests and be blocked. This could only be resolved by waiving
the failures, or replacing the separate updates with an update
containing both packages.
All of those considerations are already true for stable and branched
releases, but people are probably more used to grouping updates for
stable and branched than doing it for Rawhide, and the typical flow of
going from a build to an update provides more opportunity to create
grouped updates for branched/stable. For Rawhide the easiest way to do
it if you need to do it is to do the builds in a side tag and use
Bodhi's ability to create updates from a side tag.
As with branched/stable, only critical path updates would have the
tests run and be gated on the results. Non-critpath updates would be
unaffected. (There's a small allowlist of non-critpath packages for
which the tests are also run, but they are not currently gated on the
results).
I think doing this could really help us keep Rawhide solid and avoid
introducing major compose-breaking bugs, at minimal cost. But it's a
significant change and I wanted to see what folks think. In particular,
if you find the existing gating of updates for stable/branched releases
to cause problems in any way, I'd love to hear about it.
Thanks folks!
--
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net
10 months, 2 weeks
Request for testing: Fedora 37 pre-Beta validation tests
by Adam Williamson
Hey folks!
So we're in freeze for Fedora 37 Beta now, and the first go/no-go
meeting should be on September 8.
It would be really great if we can get the validation tests run now so
we can find any remaining blocker bugs in good time to get them fixed.
Right now the blocker list looks short, but there are definitely some
tests that have not been run.
You can use the testcase_stats view to find tests that need running:
https://openqa.fedoraproject.org/testcase_stats/37/
For each validation test set (Base, Desktop etc.) it shows when each
test was last performed. So you can easily look for Basic and Beta
tests that have not yet been run. We need to run all of these.
You can enter results using `relval report-results`, or edit the
summary results page at
https://fedoraproject.org/wiki/Test_Results:Current_Summary . That's a
redirect link which will always point to the validation results page
for the currently-nominated compose, which right now is 20220826.n.0.
Sumantro will be running a validation 'test week' starting on
Wednesday, so you can drop by the Fedora Test Day room on
chat.fedoraproject.org to hang out with other testers and get any help
you need in testing. See
https://lists.fedoraproject.org/archives/list/test-announce@lists.fedorap...
for that announcement.
Thanks folks!
--
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net
1 year, 7 months
Heads-up / for discussion: dnf not working with 1G of RAM or less
by Adam Williamson
Hey folks! I apologize for the wide distribution, but this seemed like
a bug it'd be appropriate to get a wide range of input on.
There's a bug that was proposed as an F37 Beta blocker:
https://bugzilla.redhat.com/show_bug.cgi?id=1907030
it's quite an old bug, but up until recently, the summary was
apparently accurate - dnf would run out of memory with 512M of RAM, but
was OK with 1G. However, as of quite recently, on F36 at least (not
sure if anyone's explicitly tested F37), dnf operations are commonly
failing on VMs/containers with 1G of RAM due to running out of RAM and
getting OOM-killed.
There's some discussion in the bug about what might be causing this and
potential ways to resolve it, and please do dig into/contribute to that
if you can, but the other question here I guess is: how much do we care
about this? How bad is it that you can't reliably run dnf operations on
top of a minimal Fedora environment with 1G of RAM?
This obviously has some overlap with our stated hardware requirements,
so here they are for the record:
https://docs.fedoraproject.org/en-US/fedora/latest/release-notes/welcome/...
that specifies 2GB as the minimum memory for "the default
installation", by which I think it's referring to a default Workstation
install, though this should be clarified. But then there's a "Low
memory installations" boxout, which suggests that "users with less than
768MB of system memory may have better results performing a minimal
install and adding to it afterward", which kinda is recommending that
people do exactly the thing that doesn't work (do a minimal install
then use dnf on it), and implying it'll work.
After some consideration I don't think it makes sense to take this bug
as an F37 blocker, since it already affects F36, and that's what I'll
be suggesting at the next blocker review meeting. However, it does seem
a perfect candidate for prioritized bug status, and I've nominated it
for that.
I guess if folks can chime in with thoughts here and/or in the bug
report, maybe a consensus will emerge on just how big of an issue this
is (and how likely it is to get fixed). There will presumably be a
FESCo ticket related to prioritized bug status too.
Thanks folks!
--
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net
1 year, 7 months
Ownership of KDE packages
by Daniel Vrátil
Hi all,
how's everyone in the SIG doing? :-) It's been a while since I posted to this list, and probably even more time since I contributed to Fedora KDE :-) I recently gave up the "packager" status, since, well, I'm no longer contributing to Fedora. Checking src.fpo I found I'm still in the role of Maintainer of many KDE packages (mostly Frameworks, which I submitted all those years ago), oftentimes with @kde-sig only having commit rights. Obviously, this is wrong, especially considering my recent level of involvement in the SIG ;-)
I would like to fix the current state of affairs, but I want to consult the SIG first - IIRC (and my memory is really cloudy on this one), only specific people from the SIG had admin rights for packages, but my memory might be wrong and/or things may have changed since my time, so - should I assign @kde-sig as the Admin of all those packages, or only specific people (e.g. Neal, Rex, Than, ...? I don't even know who's active in the SIG anymore).
Second, if I click the "Orphan" button, does it only remove me as a Maintainer, or does it completely orphan the package? If the latter, how do I remove myself from the package? At this point me having commit rights to all those packages is a security threat rather than anything else, so I would like to remove myself from all those packages. Especially since we can now do pull requests...
Cheers,
Dan
1 year, 8 months
2022-08-22 @ 16:00 UTC - Fedora 37 Blocker Review Meeting
by Adam Williamson
# F36 Blocker Review meeting
# Date: 2022-08-22
# Time: 16:00 UTC
# Location: #fedora-blocker-review on irc.libera.chat
Hi folks! We have 3 proposed Beta blockers, 1 proposed Beta freeze
exception issue, and 13 proposed Final blockers to review, so let's
have a review meeting. Note, I'll try and cut that number
of Final blockers down before the meeting - it'll help if folks can
vote ahead.
If you have time today, you can take a look at the proposed or
accepted blockers before the meeting - the full lists can be found
here: https://qa.fedoraproject.org/blockerbugs/ .
Remember, you can also now vote on bugs outside of review meetings! If
you look at the bug list in the blockerbugs app, you'll see links
labeled "Vote!" next to all proposed blockers and freeze exceptions.
Those links take you to tickets where you can vote.
https://pagure.io/fedora-qa/blocker-review has instructions on how
exactly you do it. We usually go through the tickets shortly before the
meeting and apply any clear votes, so the meeting will just cover bugs
where there wasn't a clear outcome in the ticket voting yet. **THIS
MEANS IF YOU VOTE NOW, THE MEETING WILL BE SHORTER!**
We'll be evaluating these bugs to see if they violate any of the
Release Criteria and warrant the blocking of a release if they're not
fixed. Information on the release criteria for F36 can be found on the
wiki [0].
For more information about the Blocker and Freeze exception process,
check out these links:
- https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process
- https://fedoraproject.org/wiki/QA:SOP_freeze_exception_bug_process
And for those of you who are curious how a Blocker Review Meeting
works - or how it's supposed to go and you want to run one - check out
the SOP on the wiki:
- https://fedoraproject.org/wiki/QA:SOP_Blocker_Bug_Meeting
Have a good night/day and see you tomorrow/later today!
[0] https://fedoraproject.org/wiki/Fedora_Release_Criteria
--
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net
1 year, 8 months
Plasma is not shown as an entry in switchdesk
by Robert McBroom
Fedora 36 booting mode 3 to a command line boot, the switchdesk entry
Plasma doesn't do anything. I can switch from the default Gnome to xfce
and back. Booting with a display manager has an entry for Plasma(X11)
that works. the gui for switchdesk does not show Plasma. What needs to
be done to get these functions back?
Also the desktop menu not longer has an entry for "leave" can get there
from the application launcher menu. Is there an edit to add it back to
the desktop menu?
1 year, 8 months
