Re: Failing jobs for Rawhide (and F39?) in Fedora CI due to use of old mock configs

I've seen several folks report that jobs (usually scratch build tests) are failing in Fedora CI, e.g. this one: https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/dist-git-bu... the failures all seem to have a common cause. You'll note these lines: DEBUG: The GPG keys listed for the "fedora" repository are already installed but they are not correct for this package. DEBUG: Check that the correct key URLs are configured for this repository.. Failing package is: curl-8.2.1-2.fc40.x86_64 DEBUG: GPG Keys are configured as: file:///usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-39-primary, file:///usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-39-primary, file:///usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-38-primary DEBUG: Public key for fedora-gpg-keys-40-0.1.noarch.rpm is not installed. Failing package is: fedora-gpg-keys-40-0.1.noarch ...for several packages. If you look carefully, you'll see it's using the wrong GPG keys: it's using the keys for F38 and F39, trying to validate a package from Rawhide. Obviously that will fail. This is happening because CI is running EL 8, and the EPEL 8 mock/mock- core-configs update for F39 branching is not yet stable: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2023-a5c155a6c0 the current stable mock-core-configs for EPEL 8 still thinks Rawhide is F39, so that's why it tries to use the F39 and F38 GPG keys to validate Rawhide packages. It needs the newer version of mock-core-configs that knows Rawhide is now F40. I've tried pinging CI folks about this to see if they can pull mock in from updates-testing, but so far no response. I guess it would help if folks running EPEL 8 can test mock, confirm it works, and up-karma the update.