Hello! On Jun 10, 2020, a new Copr release landed production. Related client tooling updates are here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-390cce74d6 https://bodhi.fedoraproject.org/updates/FEDORA-2020-f90873700f https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-3b8c2778f1 https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-feca133c66 https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-8449e5d605 Here is the list of visible changes, and new features: - Increased build task throughput. The VM spawner is more flexible now. First, we don't allocate that many workers if they are not actually needed, but more importantly the upper limit for concurrently running workers has risen to 140 (including all architectures), previously we had ~70. We'll see how things go from the backend perspective, but it is expected that we'll go even higher (waiting for a new HW in the new Fedora lab, cheaper VM instances in the future, etc.). So the numbers will likely change; this is to make clear the current state of things. - Copr project "runtime" dependencies were implemented. Newly you can specify set of repositories your project depends on. Such repositories will be installed together with the copr project repo file (e.g., by 'dnf copr enable YOU/YOUR_PROEJCT'). Those repositories can be other project in Copr or some 3rd party repository. This is very similar to build-time dependencies implemented long time ago. Take a look at blog post: https://fedora-copr.github.io/posts/runtime-dependencies - There's now more fair build scheduler. Previously, no matter whether the "background" attribute was set or not for the build - once builder was allocated for a concrete copr user - copr never terminated the builder as long as the user kept filling the build queue with new tasks (and it blocked the quota for others). Newly, there's a limit of at most eight consecutive builds or 30 minutes for one user (sandbox) on one builder and the builder is immediately terminated - which gives a chance to assign new builders to others' tasks (which have a higher priority at that point). - Copr-cli supports batch build delete feature: $ copr-cli delete build_id [build_id ...] Please use this instead of requesting removal of each build_id separately - it will be much faster because you will save the copr backend useless createrepo_c runs after each request. - We also implemented a priority queue algorithm for Action tasks (non-build tasks, e.g., forking, build/project deleting, etc.). This should solve several issues when the action queue gets temporarily too large to process immediately; namely the delayed "initial createrepo" action problem that previously caused an ugly build failures in freshly created copr projects. - Cancel build feature was fixed, and the "cancel" request should reliably release all occupied builder machines (allowing them to be re-used, or terminated). - Users are not allowed to disable chroot in a project that has some running builds against the disabled chroot. The running builds need to be canceled first. This change as done to avoid inconsistencies between frontend/backend, and wasted builder quota on useless tasks. - More space for /var/cache/mock directory, so repeated builds on the same machine won't face the ENOSPC error on mock caches. Considering that each builder can only do at most eight builds (then terminated), this problem should be finally fixed. Happy building! Pavel