Re: Introducing Steam + video game testing criteria

On Thu, Apr 20, 2023 at 1:20 PM Neal Gompa <ngompa13(a)gmail.com&gt; wrote: > > Hey all, > > I would like for us to have some testing criteria around gaming and > Steam so that we can ensure we're offering a working gaming experience > in Fedora Linux releases. This is motivated by the issue we had in the > F37 cycle where glibc broke popular multiplayer games[1]. I was > reminded of this when I launched Steam today on F38 and zenity > crashed[2]. > > I would like to propose the following criterion for Steam itself as a > Beta Blocker bug: > "Steam MUST be able to be installed and have its basic functionality > work with no visible errors. Basic functionality for Steam includes: > logging into a Steam account and installing a Windows/Proton game and > a Linux/SteamOS native game." > > For gaming itself, I would like to propose the following criterion as > a Final Blocker bug: > "Steam games identified as Deck Verified by ProtonDB.com (see > https://www.protondb.com/explore?selectedFilters=whitelisted) MUST > launch and let the user play the game. This criterion is not intended > to judge performance, merely accessibility. At least one > Windows/Proton game and one Linux/SteamOS native game MUST be tested > in this manner." > > Now, the tricky issue here is how to wordsmith the check for > anti-cheat systems. I don't want to specifically call out just EAC, > but I also don't know of a good mix of games with different > anti-cheats. The important thing is to catch regressions and see if > it's something we can resolve. In the EAC case from F37, it was easy > for us to deal with, but if it's genuinely broken in a way we can't > deal with it on the Fedora side, I don't know what we're supposed to > do, so I'm wary of doing some kind of blocker criterion for that. > > I'd also like this to be imposed on both release-blocking desktops: > GNOME and KDE Plasma. > > Any ideas welcome and appreciated! So this is a proprietary platform, we don't even block on something like the binary nvidia drivers, so I'm not sure how we can block on something that is mostly out of our control. If there is a change in an upstream kernel that triggers a crash do we have to await a fix from them or roll back to an older kernel? Also are there free games that can be played for testing or reproducing bugs that don't require you to put in a credit card to be able to use them on Steam (sorry, I've never used it).