V Wed, May 04, 2022 at 03:14:24PM -0000, Artur Frenszek-Iwicki napsal(a): > For me personally, opt-out would be better, since the logic is right most of the time. > And doesn't break it, in the rare cases, building packages in Koji? When a source package is built in Koji, sources are downloaded from lookaside cache with "fedpkg sources".

Hi all, A few months ago fedpkg introduced a change which avoids downloading source files (from dist-git) that are not used in the specfile and therefore downloading them would be wasting of resources and time. The original request was opened here [1] and implemented here [2]. The logic is part of the command "fedpkg sources" and currently can't be disabled manually. The logic parses specfile, but doesn't do a deep analysis, so it is doesn't always right. Recently we got a request for opt-in implementation of this. It means you should actively use some argument (ie. --skip-unused) to avoid downloading unused sources. The requestor points out that it broke the original functionality and it is not possible to add any extra arguments into the complicated release process (RHEL kernel).

On Wed, 2022-05-04 at 21:45 +0300, Otto Urpelainen wrote: > > Author of the patch under discussion here. > > The premise was that "specfile sources" equal "sources file sources". > Since there is a request like this, that is apparently not always the > case. Hi, Again where is the request ? I'd like to see a bit better why it is needed and if we really need that. But if he want download sources that aren't use in spec file, I prefer opt-in in sources : fedpkg sources --force

On Wed, May 04, 2022 at 09:45:55PM +0300, Otto Urpelainen wrote: > Ondrej Nosek kirjoitti 4.5.2022 klo 18.01: > > Hi all, > > > > A few months ago fedpkg introduced a change which avoids downloading source > > files (from dist-git) that are not used in the specfile and therefore > > downloading them would be wasting of resources and time. > > The original request was opened here [1] and implemented here [2]. The > > logic is part of the command "fedpkg sources" and currently can't be > > disabled manually. The logic parses specfile, but doesn't do a deep > > analysis, so it is doesn't always right. > > > > Recently we got a request for opt-in implementation of this. It means you > > should actively use some argument (ie. --skip-unused) to avoid downloading > > unused sources. The requestor points out that it broke the original > > functionality and it is not possible to add any extra arguments into the > > complicated release process (RHEL kernel). > > Author of the patch under discussion here. > > The premise was that "specfile sources" equal "sources file sources". Since > there is a request like this, that is apparently not always the case. From > that perspective, the patch is wrong and opt-in would be the correct way. I think opt-in will be useless and make the entire option pointless. Most maintainers won't be aware it exists. Why would someone want to opt-out of this?

On Mon, May 09, 2022 at 01:21:53PM -0400, Neal Gompa wrote: > On Mon, May 9, 2022 at 1:13 PM Kevin Fenzi <kevin(a)scrye.com&gt; wrote: > > > > On Wed, May 04, 2022 at 09:45:55PM +0300, Otto Urpelainen wrote: > > > Ondrej Nosek kirjoitti 4.5.2022 klo 18.01: > > > > Hi all, > > > > > > > > A few months ago fedpkg introduced a change which avoids downloading source > > > > files (from dist-git) that are not used in the specfile and therefore > > > > downloading them would be wasting of resources and time. > > > > The original request was opened here [1] and implemented here [2]. The > > > > logic is part of the command "fedpkg sources" and currently can't be > > > > disabled manually. The logic parses specfile, but doesn't do a deep > > > > analysis, so it is doesn't always right. > > > > > > > > Recently we got a request for opt-in implementation of this. It means you > > > > should actively use some argument (ie. --skip-unused) to avoid downloading > > > > unused sources. The requestor points out that it broke the original > > > > functionality and it is not possible to add any extra arguments into the > > > > complicated release process (RHEL kernel). > > > > > > Author of the patch under discussion here. > > > > > > The premise was that "specfile sources" equal "sources file sources". Since > > > there is a request like this, that is apparently not always the case. From > > > that perspective, the patch is wrong and opt-in would be the correct way. > > > > I think opt-in will be useless and make the entire option pointless. > > Most maintainers won't be aware it exists. > > > > Why would someone want to opt-out of this? > > > > I need to when working on ffmpeg updates, since it clobbers my > regenerated tarballs when I'm working normally. I had no idea about > this until someone pointed it out to me. So you mean where you have modified the source, but the name is the same as in spec and it overwrites your local changes by downloading the lookaside one over it?

I can see that being an issue early on, but after initial packaging wouldn't changes always also include the version and thus be different from whats in the spec/sources?

I was pleasently surprised when it didn't uselessly download the old source after I locally updated a spec.

Hi all, A few months ago fedpkg introduced a change which avoids downloading source files (from dist-git) that are not used in the specfile and therefore downloading them would be wasting of resources and time. The original request was opened here [1] and implemented here [2]. The logic is part of the command "fedpkg sources" and currently can't be disabled manually. The logic parses specfile, but doesn't do a deep analysis, so it is doesn't always right. Recently we got a request for opt-in implementation of this. It means you should actively use some argument (ie. --skip-unused) to avoid downloading unused sources. The requestor points out that it broke the original functionality and it is not possible to add any extra arguments into the complicated release process (RHEL kernel). On the other hand, opt-out (--download-unused) has (I think) a significantly higher impact on saving resources (time and network capacity). Of course, it doesn't have to be implemented as an extra argument, there might be a different (maybe not so clean) solution for such projects. What do you think about it? I added into a loop (bcc) names who already were involved in this in a past (on the Fedora devel mailing list) Thanks, Ondrej [1] https://pagure.io/rpkg/issue/559 [2] https://pagure.io/rpkg/pull-request/564

Ok, now I see commits such as: https://src.fedoraproject.org/rpms/ffmpeg/c/70ecae14df6b89cbd269778fc6808... which is awful that we need something like this. But @Neal, wouldn't it be better if your `ffmpeg_gen_free_tarball.sh` simply updated the hashes in `sources` file? The `fedpkg new-sources --offline` could help with that I guess.

Vít Dne 10. 05. 22 v 10:10 Vít Ondruch napsal(a): > I somehow don't understand why there should be anything like > "unused > source files". Why is something like this even possible? It seems > strange that this was not questioned originally and it seems still > strange nobody questions this in this thread. > > > Vít > > > Dne 04. 05. 22 v 17:01 Ondrej Nosek napsal(a): > > Hi all, > > > > A few months ago fedpkg introduced a change which avoids > > downloading > > source files (from dist-git) that are not used in the specfile > > and > > therefore downloading them would be wasting of resources and > > time. > > The original request was opened here [1] and implemented here > > [2]. > > The logic is part of the command "fedpkg sources" and currently > > can't > > be disabled manually. The logic parses specfile, but doesn't do a > > deep analysis, so it is doesn't always right. > > > > Recently we got a request for opt-in implementation of this. It > > means > > you should actively use some argument (ie. --skip-unused) to > > avoid > > downloading unused sources. The requestor points out that it > > broke > > the original functionality and it is not possible to add any > > extra > > arguments into the complicated release process (RHEL kernel). > > > > On the other hand, opt-out (--download-unused) has (I think) a > > significantly higher impact on saving resources (time and network > > capacity). Of course, it doesn't have to be implemented as an > > extra > > argument, there might be a different (maybe not so clean) > > solution > > for such projects. > > > > What do you think about it? > > > > I added into a loop (bcc) names who already were involved in this > > in > > a past (on the Fedora devel mailing list) > > > > Thanks, Ondrej > > > > [1] https://pagure.io/rpkg/issue/559 > > [2] https://pagure.io/rpkg/pull-request/564 _______________________________________________ devel mailing list -- devel(a)lists.fedoraproject.org To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

> I somehow don't understand why there should be anything like "unused > source files". Why is something like this even possible? 1. Grab a package 2. Edit the spec file in a way that changes which sources are used (say, update to a new version) 3. Do not run "fedpkg new-sources" to upload the new tarballs 4. The "sources" file now lists files that are not actually used by the spec 5. Run "fedpkg mockbuild" 6. Mockbuild attempts to download package sources

A.FI. _______________________________________________ devel mailing list -- devel(a)lists.fedoraproject.org To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

> I somehow don't understand why there should be anything like "unused > source files". Why is something like this even possible? 1. Grab a package 2. Edit the spec file in a way that changes which sources are used (say, update to a new version) 3. Do not run "fedpkg new-sources" to upload the new tarballs 4. The "sources" file now lists files that are not actually used by the spec

5. Run "fedpkg mockbuild" 6. Mockbuild attempts to download package sources

Are you manually editing the "sources" file?