Hey all, I would like for us to have some testing criteria around gaming and Steam so that we can ensure we're offering a working gaming experience in Fedora Linux releases. This is motivated by the issue we had in the F37 cycle where glibc broke popular multiplayer games[1]. I was reminded of this when I launched Steam today on F38 and zenity crashed[2]. I would like to propose the following criterion for Steam itself as a Beta Blocker bug: "Steam MUST be able to be installed and have its basic functionality work with no visible errors. Basic functionality for Steam includes: logging into a Steam account and installing a Windows/Proton game and a Linux/SteamOS native game." For gaming itself, I would like to propose the following criterion as a Final Blocker bug: "Steam games identified as Deck Verified by ProtonDB.com (see https://www.protondb.com/explore?selectedFilters=whitelisted) MUST launch and let the user play the game. This criterion is not intended to judge performance, merely accessibility. At least one Windows/Proton game and one Linux/SteamOS native game MUST be tested in this manner." Now, the tricky issue here is how to wordsmith the check for anti-cheat systems. I don't want to specifically call out just EAC, but I also don't know of a good mix of games with different anti-cheats. The important thing is to catch regressions and see if it's something we can resolve. In the EAC case from F37, it was easy for us to deal with, but if it's genuinely broken in a way we can't deal with it on the Fedora side, I don't know what we're supposed to do, so I'm wary of doing some kind of blocker criterion for that. I'd also like this to be imposed on both release-blocking desktops: GNOME and KDE Plasma. Any ideas welcome and appreciated!

On Thu, Apr 20, 2023 at 2:28 PM Adam Williamson <adamwill(a)fedoraproject.org&gt; wrote: > > On Thu, 2023-04-20 at 08:20 -0400, Neal Gompa wrote: > > Hey all, > > > > I would like for us to have some testing criteria around gaming and > > Steam so that we can ensure we're offering a working gaming experience > > in Fedora Linux releases. This is motivated by the issue we had in the > > F37 cycle where glibc broke popular multiplayer games[1]. I was > > reminded of this when I launched Steam today on F38 and zenity > > crashed[2]. > > > > I would like to propose the following criterion for Steam itself as a > > Beta Blocker bug: > > "Steam MUST be able to be installed and have its basic functionality > > work with no visible errors. Basic functionality for Steam includes: > > logging into a Steam account and installing a Windows/Proton game and > > a Linux/SteamOS native game." > > > > For gaming itself, I would like to propose the following criterion as > > a Final Blocker bug: > > "Steam games identified as Deck Verified by ProtonDB.com (see > > https://www.protondb.com/explore?selectedFilters=whitelisted) MUST > > launch and let the user play the game. This criterion is not intended > > to judge performance, merely accessibility. At least one > > Windows/Proton game and one Linux/SteamOS native game MUST be tested > > in this manner." > > > > Now, the tricky issue here is how to wordsmith the check for > > anti-cheat systems. I don't want to specifically call out just EAC, > > but I also don't know of a good mix of games with different > > anti-cheats. The important thing is to catch regressions and see if > > it's something we can resolve. In the EAC case from F37, it was easy > > for us to deal with, but if it's genuinely broken in a way we can't > > deal with it on the Fedora side, I don't know what we're supposed to > > do, so I'm wary of doing some kind of blocker criterion for that. > > > > I'd also like this to be imposed on both release-blocking desktops: > > GNOME and KDE Plasma. > > > > Any ideas welcome and appreciated! > > I'm against this. We have never blocked the OS on proprietary third- > party applications. I don't think it's a path we want to go down. > > I'm in favour of testing common third-party stuff before release and > fixing it if we can, but we have always said we will not block Fedora > on this, and I don't think we should change that. > > (I did actually test Steam, but I used the flatpak version, not RPM...) > > I'm fixing the zenity bug, BTW. Is there a way we can add regular testing for it without making it a blocker then?

On Thu, Apr 20, 2023 at 11:39 PM Neal Gompa <ngompa13(a)gmail.com&gt; wrote: > Is there a way we can add regular testing for it without making it a > blocker then? > For popular third-party software, I think this testing occurs quite naturally - people just use it. E.g. Steam, Chrome, VSCode, etc. I've been running Steam on F38 since Beta (and it helped me to discover an issue in mutter, which was later accepted as a blocker - but not because of Steam, but because of general issues). But if you want to have an explicit test case, Adam described how to do it. We could also have a test day for popular third-party software, if it makes sense.

On Tue, Apr 25, 2023 at 12:18 PM Adam Williamson <adamwill(a)fedoraproject.org&gt; wrote: > > On Tue, 2023-04-25 at 13:49 +0200, Kamil Paral wrote: > > On Thu, Apr 20, 2023 at 11:39 PM Neal Gompa <ngompa13(a)gmail.com&gt; wrote: > > > > > Is there a way we can add regular testing for it without making it a > > > blocker then? > > > > > > > For popular third-party software, I think this testing occurs quite > > naturally - people just use it. E.g. Steam, Chrome, VSCode, etc. I've been > > running Steam on F38 since Beta (and it helped me to discover an issue in > > mutter, which was later accepted as a blocker - but not because of Steam, > > but because of general issues). But if you want to have an explicit test > > case, Adam described how to do it. We could also have a test day for > > popular third-party software, if it makes sense. > > To be fair, we did miss > https://bugzilla.redhat.com/show_bug.cgi?id=2177287 (I missed it > because I run Steam via Flatpak, not RPM). So we do have scope to > improve there. I do think testing commonly used third-party stuff is a > good idea, to be clear, and I'm all in favor of adding optional test > cases for it. And I'm fine with us adding some optional test cases for this. Would running a test day each cycle also be possible?

On Thu, Apr 20, 2023 at 1:20 PM Neal Gompa <ngompa13(a)gmail.com&gt; wrote: > > Hey all, > > I would like for us to have some testing criteria around gaming and > Steam so that we can ensure we're offering a working gaming experience > in Fedora Linux releases. This is motivated by the issue we had in the > F37 cycle where glibc broke popular multiplayer games[1]. I was > reminded of this when I launched Steam today on F38 and zenity > crashed[2]. > > I would like to propose the following criterion for Steam itself as a > Beta Blocker bug: > "Steam MUST be able to be installed and have its basic functionality > work with no visible errors. Basic functionality for Steam includes: > logging into a Steam account and installing a Windows/Proton game and > a Linux/SteamOS native game." > > For gaming itself, I would like to propose the following criterion as > a Final Blocker bug: > "Steam games identified as Deck Verified by ProtonDB.com (see > https://www.protondb.com/explore?selectedFilters=whitelisted) MUST > launch and let the user play the game. This criterion is not intended > to judge performance, merely accessibility. At least one > Windows/Proton game and one Linux/SteamOS native game MUST be tested > in this manner." > > Now, the tricky issue here is how to wordsmith the check for > anti-cheat systems. I don't want to specifically call out just EAC, > but I also don't know of a good mix of games with different > anti-cheats. The important thing is to catch regressions and see if > it's something we can resolve. In the EAC case from F37, it was easy > for us to deal with, but if it's genuinely broken in a way we can't > deal with it on the Fedora side, I don't know what we're supposed to > do, so I'm wary of doing some kind of blocker criterion for that. > > I'd also like this to be imposed on both release-blocking desktops: > GNOME and KDE Plasma. > > Any ideas welcome and appreciated! So this is a proprietary platform, we don't even block on something like the binary nvidia drivers, so I'm not sure how we can block on something that is mostly out of our control. If there is a change in an upstream kernel that triggers a crash do we have to await a fix from them or roll back to an older kernel? Also are there free games that can be played for testing or reproducing bugs that don't require you to put in a credit card to be able to use them on Steam (sorry, I've never used it). Peter _______________________________________________ test mailing list -- test(a)lists.fedoraproject.org To unsubscribe send an email to test-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue