On Wed, Mar 11, 2020 at 12:41 PM Jeremy Cline <jeremy(a)jcline.org&gt; wrote: > Hi folks, > > This should come as no surprise to those who have been following > the > kernel list and/or saw Laura's Flock talk last summer, but there > are > some changes to the way the Fedora kernel is maintained coming in > the > next couple of weeks. > > For those folks who aren't committing directly to the kernel dist- > git, > this change won't really impact you, although contributing might be > easier. > > We're planning to switch from maintaining the kernel directly in > the > dist-git to using a kernel source tree along with a set of scripts > to > turn the source tree into something that can be automatically > checked > into the dist-git. This means anything committed directly to the > dist- > git repository will get overwritten on the next update. > > The git repository is currently hosted at > https://gitlab.com/cki-project/kernel-ark/. Documentation (still > very > rough) for common tasks is at > https://gitlab.com/cki-project/kernel-ark/-/wikis/home. There are a > few > outstanding merge requests to get the tree fully synced up with the > current state of the dist-git repository, so if you decide to jump > in > and try things out, be aware things might not work. This looks like it will completely remove the need for https://git.kernel.org/pub/scm/linux/kernel/git/jwboyer/fedora.git Is that correct? (Please let it be correct.)

josh > As part of this, we're going to start bridging merge requests to > this > list as emailed patch series for those who prefer email, and turn > any > emailed patches to the list into merge requests, so the list is > going > to get quite a bit more traffic. > > Regards, > Jeremy > > > > _______________________________________________ > kernel mailing list -- kernel(a)lists.fedoraproject.org > To unsubscribe send an email to > kernel-leave(a)lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: > https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/kernel@lists.fedoraproject.org

On Wed, Mar 11, 2020 at 1:21 PM Jeremy Cline <jeremy(a)jcline.org&gt; wrote: > > On Wed, 2020-03-11 at 12:58 -0400, Josh Boyer wrote: > > On Wed, Mar 11, 2020 at 12:41 PM Jeremy Cline <jeremy(a)jcline.org&gt; > > wrote: > > > Hi folks, > > > > > > This should come as no surprise to those who have been following > > > the > > > kernel list and/or saw Laura's Flock talk last summer, but there > > > are > > > some changes to the way the Fedora kernel is maintained coming in > > > the > > > next couple of weeks. > > > > > > For those folks who aren't committing directly to the kernel dist- > > > git, > > > this change won't really impact you, although contributing might be > > > easier. > > > > > > We're planning to switch from maintaining the kernel directly in > > > the > > > dist-git to using a kernel source tree along with a set of scripts > > > to > > > turn the source tree into something that can be automatically > > > checked > > > into the dist-git. This means anything committed directly to the > > > dist- > > > git repository will get overwritten on the next update. > > > > > > The git repository is currently hosted at > > > https://gitlab.com/cki-project/kernel-ark/. Documentation (still > > > very > > > rough) for common tasks is at > > > https://gitlab.com/cki-project/kernel-ark/-/wikis/home. There are a > > > few > > > outstanding merge requests to get the tree fully synced up with the > > > current state of the dist-git repository, so if you decide to jump > > > in > > > and try things out, be aware things might not work. > > > > This looks like it will completely remove the need for > > https://git.kernel.org/pub/scm/linux/kernel/git/jwboyer/fedora.git > > > > Is that correct? (Please let it be correct.) > > > > Eventually, yes. To start with we're just going to do Rawhide this > way, but assuming things go well handling stable releases the same way > should be straight-forward. Great. Please let me know when I can stop generating that tree.

On Wed, Mar 11, 2020 at 12:41 PM Jeremy Cline <jeremy(a)jcline.org&gt; wrote: > Hi folks, > > This should come as no surprise to those who have been following > the > kernel list and/or saw Laura's Flock talk last summer, but there > are > some changes to the way the Fedora kernel is maintained coming in > the > next couple of weeks. > > For those folks who aren't committing directly to the kernel dist- > git, > this change won't really impact you, although contributing might be > easier. > > We're planning to switch from maintaining the kernel directly in > the > dist-git to using a kernel source tree along with a set of scripts > to > turn the source tree into something that can be automatically > checked > into the dist-git. This means anything committed directly to the > dist- > git repository will get overwritten on the next update. > I don't think that's actually acceptable under Fedora guidelines, as that violates the canonicity rule on the sources and spec[1]. Have you already worked out how to deal with that yet?

Hi folks, This should come as no surprise to those who have been following the kernel list and/or saw Laura's Flock talk last summer, but there are some changes to the way the Fedora kernel is maintained coming in the next couple of weeks. For those folks who aren't committing directly to the kernel dist-git, this change won't really impact you, although contributing might be easier. We're planning to switch from maintaining the kernel directly in the dist-git to using a kernel source tree along with a set of scripts to turn the source tree into something that can be automatically checked into the dist-git. This means anything committed directly to the dist- git repository will get overwritten on the next update. The git repository is currently hosted at https://gitlab.com/cki-project/kernel-ark/. Documentation (still very rough) for common tasks is at https://gitlab.com/cki-project/kernel-ark/-/wikis/home. There are a few outstanding merge requests to get the tree fully synced up with the current state of the dist-git repository, so if you decide to jump in and try things out, be aware things might not work. As part of this, we're going to start bridging merge requests to this list as emailed patch series for those who prefer email, and turn any emailed patches to the list into merge requests, so the list is going to get quite a bit more traffic.

The git tags are still signed by Linus. Does that cover your concerns?

On 3/12/20 10:57 AM, Bastien Nocera wrote: > > > ----- Original Message ----- > <snip> >> The git tags are still signed by Linus. Does that cover your concerns? > > Not really, no. I think that multiplying the intermediaries between > kernel.org > and the Fedora repos by adding gitlab.com in the middle might not be the > best of ideas. > > If the Fedora security team is fine with it, I'm fine with it, and even if > I > understand the practical concerns (pagure not being up to par to deal with > repos that size, and without a mail gateway support), I find it slightly > concerning. > I think this boils down to how much do you trust the kernel maintainers. Keep in mind that the existing model requires the kernel maintainers to manually pull down a tree and extract the tarball and then upload. You can probably trust them to not do anything malicious but mistakes can happen (source: I screwed up many times). It's good to be concerned about provenance as a threat model but I consider maintainers screwing up manual tasks to be a bigger threat model to Fedora kernel security so anything that moves towards automation is a benefit in my eyes.

On Fri, Mar 13, 2020 at 7:02 AM Bastien Nocera <bnocera(a)redhat.com&gt; wrote: > > > > ----- Original Message ----- > > > > > > On 3/12/20 10:57 AM, Bastien Nocera wrote: > > > > > > > > > ----- Original Message ----- > > > <snip> > > >> The git tags are still signed by Linus. Does that cover your concerns? > > > > > > Not really, no. I think that multiplying the intermediaries between > > > kernel.org > > > and the Fedora repos by adding gitlab.com in the middle might not be the > > > best of ideas. > > > > > > If the Fedora security team is fine with it, I'm fine with it, and even if > > > I > > > understand the practical concerns (pagure not being up to par to deal with > > > repos that size, and without a mail gateway support), I find it slightly > > > concerning. > > > > > > > I think this boils down to how much do you trust the kernel maintainers. > > Keep in mind that the existing model requires the kernel maintainers > > to manually pull down a tree and extract the tarball and then upload. > > You can probably trust them to not do anything malicious but mistakes > > can happen (source: I screwed up many times). It's good to be concerned > > about provenance as a threat model but I consider maintainers screwing > > up manual tasks to be a bigger threat model to Fedora kernel security > > so anything that moves towards automation is a benefit in my eyes. > > For me, it's about how much we trust gitlab.com _in addition_ to trusting > kernel.org and fedoraproject.org. I wouldn't be concerned at all if > the new "in-between" tree was at either of those 2 locations. For what it's worth, while I agree, I doubt the kernel maintainers will care about that. They clearly haven't cared given that the CKI project does not run on what most in the project generally considers "trusted infrastructure".

On Fri, Mar 13, 2020 at 9:42 AM Josh Boyer <jwboyer(a)fedoraproject.org&gt; wrote: > > On Fri, Mar 13, 2020 at 7:08 AM Neal Gompa <ngompa13(a)gmail.com&gt; wrote: > > > > On Fri, Mar 13, 2020 at 7:02 AM Bastien Nocera <bnocera(a)redhat.com&gt; wrote: > > > > > > > > > > > > ----- Original Message ----- > > > > > > > > > > > > On 3/12/20 10:57 AM, Bastien Nocera wrote: > > > > > > > > > > > > > > > ----- Original Message ----- > > > > > <snip> > > > > >> The git tags are still signed by Linus. Does that cover your concerns? > > > > > > > > > > Not really, no. I think that multiplying the intermediaries between > > > > > kernel.org > > > > > and the Fedora repos by adding gitlab.com in the middle might not be the > > > > > best of ideas. > > > > > > > > > > If the Fedora security team is fine with it, I'm fine with it, and even if > > > > > I > > > > > understand the practical concerns (pagure not being up to par to deal with > > > > > repos that size, and without a mail gateway support), I find it slightly > > > > > concerning. > > > > > > > > > > > > > I think this boils down to how much do you trust the kernel maintainers. > > > > Keep in mind that the existing model requires the kernel maintainers > > > > to manually pull down a tree and extract the tarball and then upload. > > > > You can probably trust them to not do anything malicious but mistakes > > > > can happen (source: I screwed up many times). It's good to be concerned > > > > about provenance as a threat model but I consider maintainers screwing > > > > up manual tasks to be a bigger threat model to Fedora kernel security > > > > so anything that moves towards automation is a benefit in my eyes. > > > > > > For me, it's about how much we trust gitlab.com _in addition_ to trusting > > > kernel.org and fedoraproject.org. I wouldn't be concerned at all if > > > the new "in-between" tree was at either of those 2 locations. > > > > For what it's worth, while I agree, I doubt the kernel maintainers > > will care about that. They clearly haven't cared given that the CKI > > project does not run on what most in the project generally considers > > "trusted infrastructure". > > Fedora's "trusted infrastructure" can't scale to what CKI is doing. > One could argue about what trusted infrastructure means in general, > because in my opinion there is no such thing, but it would be entirely > irresponsible to overwhelm already limited capacity with something > that is done at the scale CKI runs. Figuring out how to get > comfortable with using cloud resources for workloads where that make > sense is critical to our long term success. > > (FWIW, I'm trying really hard not to read your comment as a slam on > the kernel team here. I also find it an interesting example of > cognitive dissonance that CKI running in AWS somehow triggers this > comment, when all of Fedora is dependent on the mirror network to > serve the actual binaries to users and *that* is far more risky than > doing build testing in the cloud that doesn't even impact end-users.) > I am not slamming the kernel maintainers. They're doing excellent work, and many of the efforts as part of the CKI project are to be lauded. I am also not saying cloud resources in themselves are a problem, but it's not like you're running on a git server hosted in Fedora's AWS/Azure/GCP account. My principal concern has always been that projects under the Fedora banner (or something like it) should be under infrastructure that the Fedora Project controls.

Honestly, I don't care if it's RHOSP, Fedora's AWS/Azure/GCP, or a server in a Red Hat office or datacenter. What I care about is that when push comes to shove, we're not screwed by an external force in an unexpected fashion in a way where we have no recovery plan.

Personally, I think it's cool to see that the Red Hat kernel might eventually be derived from the Fedora kernel as a consequence of this effort!

> > > >> The git tags are still signed by Linus. Does that cover your concerns? > > > > > > > > Not really, no. I think that multiplying the intermediaries between > > > > kernel.org > > > > and the Fedora repos by adding gitlab.com in the middle might not be the > > > > best of ideas. > > > > > > > > If the Fedora security team is fine with it, I'm fine with it, and even if > > > > I > > > > understand the practical concerns (pagure not being up to par to deal with > > > > repos that size, and without a mail gateway support), I find it slightly > > > > concerning. > > > > > > > > > > I think this boils down to how much do you trust the kernel maintainers. > > > Keep in mind that the existing model requires the kernel maintainers > > > to manually pull down a tree and extract the tarball and then upload. > > > You can probably trust them to not do anything malicious but mistakes > > > can happen (source: I screwed up many times). It's good to be concerned > > > about provenance as a threat model but I consider maintainers screwing > > > up manual tasks to be a bigger threat model to Fedora kernel security > > > so anything that moves towards automation is a benefit in my eyes. > > > > For me, it's about how much we trust gitlab.com _in addition_ to trusting > > kernel.org and fedoraproject.org. I wouldn't be concerned at all if > > the new "in-between" tree was at either of those 2 locations. > > For what it's worth, while I agree, I doubt the kernel maintainers > will care about that. They clearly haven't cared given that the CKI > project does not run on what most in the project generally considers > "trusted infrastructure". Fedora's "trusted infrastructure" can't scale to what CKI is doing. One could argue about what trusted infrastructure means in general, because in my opinion there is no such thing, but it would be entirely irresponsible to overwhelm already limited capacity with something that is done at the scale CKI runs. Figuring out how to get comfortable with using cloud resources for workloads where that make sense is critical to our long term success.

(FWIW, I'm trying really hard not to read your comment as a slam on the kernel team here. I also find it an interesting example of cognitive dissonance that CKI running in AWS somehow triggers this comment, when all of Fedora is dependent on the mirror network to serve the actual binaries to users and *that* is far more risky than doing build testing in the cloud that doesn't even impact end-users.)

Provenance isn't what I was phrasing as risky. We depend on the mirror network to scale, which is entirely outside of our control and if they decide it's no longer worth distributing Fedora binaries we'd be screwed. It's the exact concern Neal has :)

----- Original Message ----- <snip> > The git tags are still signed by Linus. Does that cover your concerns? Not really, no. I think that multiplying the intermediaries between kernel.org and the Fedora repos by adding gitlab.com in the middle might not be the best of ideas. If the Fedora security team is fine with it, I'm fine with it, and even if I understand the practical concerns (pagure not being up to par to deal with repos that size, and without a mail gateway support), I find it slightly concerning. I don't really see how this is relevant in regards to kernel.org.

Hi folks, This should come as no surprise to those who have been following the kernel list and/or saw Laura's Flock talk last summer, but there are some changes to the way the Fedora kernel is maintained coming in the next couple of weeks. For those folks who aren't committing directly to the kernel dist- git, this change won't really impact you, although contributing might be easier. We're planning to switch from maintaining the kernel directly in the dist-git to using a kernel source tree along with a set of scripts to turn the source tree into something that can be automatically checked into the dist-git. This means anything committed directly to the dist- git repository will get overwritten on the next update. The git repository is currently hosted at https://gitlab.com/cki-project/kernel-ark/. Documentation (still very rough) for common tasks is at https://gitlab.com/cki-project/kernel-ark/-/wikis/home. There are a few outstanding merge requests to get the tree fully synced up with the current state of the dist-git repository, so if you decide to jump in and try things out, be aware things might not work. As part of this, we're going to start bridging merge requests to this list as emailed patch series for those who prefer email, and turn any emailed patches to the list into merge requests, so the list is going to get quite a bit more traffic.

Okay, this is now done. You may notice a number of stale options made their way back into the config files, it's on my to-do list to clean this up assuming there aren't any larger fires this week. Please send any kernel changes as merge requests to the GitLab repository or as emails to this list. If you are one of the folks who has commit access to the dist-git and adds something there directly, I'll pull it into the source tree for you, but I *will* whine at you and I'm a world class whiner.

On Tue, 2020-04-14 at 21:13 -0400, Paul Moore wrote: > On Tue, Apr 14, 2020 at 6:37 PM Jeremy Cline <jeremy(a)jcline.org&gt; > wrote: > > Okay, this is now done. You may notice a number of stale options > > made > > their way back into the config files, it's on my to-do list to > > clean > > this up assuming there aren't any larger fires this week. > > > > Please send any kernel changes as merge requests to the GitLab > > repository or as emails to this list. If you are one of the folks > > who > > has commit access to the dist-git and adds something there > > directly, > > I'll pull it into the source tree for you, but I *will* whine at > > you > > and I'm a world class whiner. > > My apologies if I missed a discussion on this earlier, but what is > the > process for building the source tarball for the kernel-headers > package? The old process does not seem to apply to the new build > process ... > The script did indeed get nuked (although obviously it's in the history forever). I haven't made a particular plan for this yet, but the script could either move into the kernel-headers package or the source tree. I'm inclined to move it into the source tree so the kernel-headers (and kernel-tools) packages can be generated from it as well.

On Tue, Apr 14, 2020 at 9:52 PM Paul Moore <paul(a)paul-moore.com&gt; wrote: There is another change in the kernel's specfile relating to where the %buildid is inserted into the version string. Previously the kernel NVR would look like this: kernel-5.6.0-0.rc7.git1.1.BUILDID.fc33.src.rpm ... but now it looks like this: kernel-5.7.0-0.rc1.20200414git8632e9b5645b.1.fc33.BUILDID.src.rpm Any chance you can change the specfile so that the %buildid comes before %dist as it did in the past?

Am 15.04.20 um 00:37 schrieb Jeremy Cline: > On Tue, 2020-04-07 at 15:33 +0000, Jeremy Cline wrote: >> On Wed, 2020-03-11 at 16:40 +0000, Jeremy Cline wrote: >> >> Just a note folks, the plan is to do this starting next week after >> the close of the v5.7 merge window. > > Okay, this is now done. You may notice a number of stale options made > their way back into the config files, it's on my to-do list to clean > this up assuming there aren't any larger fires this week. There is one thing I really dislike about the scheme (one it didn't notice when I took a brief look at it weeks ago; sorry): There are no individual patches anymore in dist-git/the srpm and that afaics violates the packaging guidelines. https://docs.fedoraproject.org/en-US/packaging-guidelines/#_applying_patches ``` The files MUST then be checked into the Fedora Package revision control system […]. Storing the files in this way allows people to use standard tools to visualize the changes between revisions of the files and track additions and removals without a layer of indirection […]. ``` There are other rules in the patch section that afaics are violated. Were those violation discussed and blessed by the Fedora Packaging Committee or FESCo?

I for one would dislike such an exception, because I sometimes look at kernel source packages from other dists and it is always annoying when I can't easily see individual patches. It also makes it way harder for users to remove one certain patch that Fedora applied for testing or other reasons. So you you maybe change the scheme so individual patch files land in the src.rpm?

P.S.: The "--with-vanilla" build option afaics doesn't work anymore, as patch-%{rpmversion}-redhat.patch and linux-kernel-test.patch are always applied.

Le mer. 15 avr. 2020 à 13:34, Josh Boyer <jwboyer(a)fedoraproject.org&gt; a écrit : > On Wed, Apr 15, 2020 at 5:32 AM Thorsten Leemhuis < > fedora(a)leemhuis.info&gt; wrote: ... > > There is one thing I really dislike about the scheme (one it > > didn't > > notice when I took a brief look at it weeks ago; sorry): There > > are no > > individual patches anymore in dist-git/the srpm and that afaics > > violates > > the packaging guidelines. > > https://docs.fedoraproject.org/en-US/packaging-guidelines/#_applying_patches > > > > ``` > > The files MUST then be checked into the Fedora Package revision > > control > > system […]. Storing the files in this way allows people to use > > standard > > tools to visualize the changes between revisions of the files and > > track > > additions and removals without a layer of indirection […]. > > ``` > > There are other rules in the patch section that afaics are > > violated. > > Were those violation discussed and blessed by the > > Fedora Packaging Committee or FESCo? > > I don't think the split out patches "rule" is being violated here. > They changed the source tarball to one generated from the git tree > and > they don't have any patches at the dist-git level at all. Several > other packages in Fedora already do this, such as anaconda. So it means should one single line patch should be changed, you need to re-upload a 100M tarball to the fedora lookaside cache? I understand the benefit to have an upstream source tree as a primary origin for kernel patch development. But what is the reason behind this process in particular ? It would have been easier to only generate a full fedora diff against the original upstream tree (using a commit-id that can also be generated with gitlab with A PR). Like how I expect it's done with the patch-5.7.0-redhat.patch. Then keep using the original upstream tarballs. (linux-M.tar.xz and patch-M-m.xz) I'm sure that in some cases, (minor patch update) there is even not have to change the fedora diff.

On Wed, 2020-04-15 at 11:31 +0200, Thorsten Leemhuis wrote: > Am 15.04.20 um 00:37 schrieb Jeremy Cline: >> On Tue, 2020-04-07 at 15:33 +0000, Jeremy Cline wrote: >>> On Wed, 2020-03-11 at 16:40 +0000, Jeremy Cline wrote: > There is one thing I really dislike about the scheme (one it didn't > notice when I took a brief look at it weeks ago; sorry): There are no > individual patches anymore in dist-git/the srpm and that afaics > violates > the packaging guidelines. > https://docs.fedoraproject.org/en-US/packaging-guidelines/#_applying_patches > […] > So you you maybe change the scheme so individual patch files land in > the > src.rpm? I'll look into how simple it is to change.

It's easy to see in the source tree, though, just look at the "ark-patches" branch.

> P.S.: The "--with-vanilla" build option afaics doesn't work anymore, > as > patch-%{rpmversion}-redhat.patch and linux-kernel-test.patch are > always > applied. I'll see about that as well.

Note that if you want a vanilla SRPM it's easy from the source tree: $ git checkout master $ git merge internal $ sed -i 's/=13/=11/g' redhat/configs/fedora/generic/arm/aarch64/CONFIG_FORCE_MAX_ZONEORDER $ make rh-srpm

However, if you want to continue building from the dist-git, the patch is ignored if it's empty so doing $ truncate -s 0 patch-*-redhat.patch will also give you a vanilla build.

On Thu, 2020-04-16 at 08:34 +0200, Thorsten Leemhuis wrote: > Am 15.04.20 um 15:41 schrieb Jeremy Cline: >> On Wed, 2020-04-15 at 11:31 +0200, Thorsten Leemhuis wrote: […] I've proposed a readme[0]

and something to break the patches out for those who prefer dist-git[1].

>> However, if you want to continue building from the dist-git, the >> patch >> is ignored if it's empty so doing >> $ truncate -s 0 patch-*-redhat.patch >> will also give you a vanilla build. > Ahh, good to know. > Thx for replying and looking into this, Given this are do you still want a --with-vanilla option?