On Tue, Feb 8, 2022 at 2:24 PM Donald Zickus <dzickus(a)redhat.com&gt; wrote: I'm honestly not sure how useful it has been so far. I've been forced to push the kernel mail out of my inbox because the traffic is too high. In particular, rebases and force pushes are pretty spammy. As an observer and a relatively recent contributor, it's been pretty difficult to track things because of how much is pushed as emails. -- 真実はいつも一つ！/ Always, there's only one truth!

On Tue, 8 Feb 2022 14:24:22 -0500 Donald Zickus <dzickus(a)redhat.com&gt; wrote: I don't really have a dog in the fight, but I like reading the subject lines to see what is coming down the pipe for the kernel. The thing that I find annoying about the messages is how succinct they are. If someone is involved with the change, they probably just need the slight reminder. But for everyone else it would be good to have a little more flesh on the bones. Even in the body of the message there is no explanation of the purpose of the change other than the brief note. I see the same thing in rpm package descriptions (sometimes). e.g. the package description for libfoo says "this is a library for package foo." And in the kernel help messages during configuration (sometimes). e.g. the latest 5.17 kernel has something called straight line synchronization. But the description doesn't really explain what that is. I haven't got around to looking it up online yet. Should you change? Maybe not, maybe the time developers would spend upgrading the messages just isn't worth the hit to their productivity for the few consumers who aren't involved that read the messages. Anyhow, I'm fine with the level of messages in the kernel list.

[sending this a second time with a different "From" address to get it on the list; sorry!] Lo! On 08.02.22 20:24, Donald Zickus wrote: The mails are not a big problem for me, but they are related to a bigger question that again and again pops up in my head: Is it really a good idea to have all those totally RHEL specific patches in the Fedora rpm (and thus discussed here)? And does that approach mean that Red Hat is special, or are willing to include downstream patches for other distros as well? Say Amazon Linux? How is this handled in other packages? Or is it an issue that only happens with the kernel? Sure, Fedora is mainly sponsored by Red Hat and all Fedora kernel developers are working for Red Hat, but somehow it feels to me like a boundary was overstepped. Ciao, Thorsten

On Fri, Feb 11, 2022 at 11:25 AM Justin Forbes <jforbes(a)redhat.com&gt; wrote: On the other hand, the workflow changes have certainly made the Fedora kernel more difficult for outsiders to work with. I've somewhat adapted to it, but I'm still not a fan of the process for the Fedora kernel. I also don't like the implication that RHEL's needs are more special in the Fedora kernel sources than any other downstream. And with the configs for kernel-ark, we call the RHEL configs as "ark" instead of "rhel", which I think is weird and we should call it what it is. Admittedly, I appreciate being able to adapt config differences for them when working with ARK. There's a lot of other uncomfortable thoughts and questions that come up when I think more about the Fedora kernel wrt workflow and maintainer restrictions. For example, is it even possible anymore for non-RH Fedora kernel maintainers? Was it ever really possible? These are the kind of awkward questions that I don't know if I am going to like the answer I get if I ask. If we're going to have RHEL configs in kernel-ark (remember, "ark" is really "rhel"), would other downstream configs be allowed? Can the testing apparatus be extended to build and test them? How far can we go here? I have these questions as the CentOS Hyperscale SIG kernel maintainer and as a general Fedora community member. If I felt like the emails gave us something to do with it, I think I would be okay with the emails. But I don't know what I'm supposed to do with any of this. -- 真実はいつも一つ！/ Always, there's only one truth!

On Mon, Feb 14, 2022 at 5:15 AM Thorsten Leemhuis <fedora(a)leemhuis.info&gt; wrote: Patch count is really a marketing thing, that I am not particularly worried about. I am much more worried that the kernel we are building and running for Fedora behaves as similarly to upstream as possible. This is why having things like CONFIG_RHEL_DIFFERENCES does introduce additional code, but I think is an overall win because the Fedora kernel should be avoiding those codepaths all together. I can get problematic from the "apply this patch" to test a bug stance upstream, but given the locations for these, I expect actual conflict to be rare, and easy to fixup when it exists. This is possible, but it turns out to be a bit more of a nightmare to manage. The way it would be set up would be master is still the linus branch, a fedora-rawhide type branch would have to exist which merges in master regularly, and os-build would add the RHEL patches. Changing the spec to keep a linus tarball and separate patch sets for fedora and rhel, only applying the RHEL patch set if not fedora is trivial. What gets weird is from a maintainer or developer point of view. I would have to remember which branch to target my changes and it can get all very confusing, with different changes requiring different targets. I thought this had improved somewhat. and should be better. I am open to even more improvements as well. Right now, doing things like a stable rc build is fairly trivial provided you have the git foo to maintain various remotes. I keep a local branch which is a clone of fedora-5.1x and keep a local checkout of greg's stable rc repository. Because my fedora-5.16-rc branch has a Makefile.variables which points to the linux-5.16.y-rc branch as upstream, merging it in just works as expected. A lot can be done with Makefile.variables to make a custom branch behave as expected. The source matches upstream in that they are a tarball made from linus' branch (and nothing else). While we did previously download and verify the signature of a tarball from kernel.org for actual releases, the rc patches and such were generated from git before as well. As kernel.org does use signatures for releases, you can verify those signatures in our clone just the same. Using git log --show-signature' will show you the signatures on commits if they exist. This is somewhere that definitely needs improvement, but it has not made it up on my priority list. More importantly, changing a release should not change the sources required. This is an issue we are working to address. Right now, if things are building in rawhide, and not for you, the best bet is to look for merge requests labeled "Include in releases" The hope is to get these types of patches merged quickly now, though in the gcc12 case, there was some discussion upstream and both of those patches have gone through revisions, one as recently as yesterday. https://gitlab.com/cki-project/kernel-ark/-/merge_requests?scope=all&... This is the trade off, and I am okay with it. We have worked through some guidelines to make sure that RHEL specific downstream patches do not impact fedora kernel-builds. And there have been times when we just said no because the patch in question was not able to be isolated, and offered no value to Fedora. I do also remove the RHEL specific patches on Fedora rebases, so the stable branches do not include the patches. While the RHEL configs and files are still included in the Fedora stable branches, the only thing stopping me from removing them is having the time to do so. We already have some framework for it with "INCLUDE_RHEL_FILES" in Makefile.variables. It would just take some additional work to hide things like kabi sources and such behind that. There has been some discussion about this. It isn't that it was some trick to try to hide things. ARK has been used internally for quite some time, and has gone through many iterations before what we have now. The name doesn't particularly make sense anymore in the current context, but there is a bit of work required to change it. It is just as possible as it has ever been to have non-RH Fedora kernel maintainers, which is to say, not easy, but not impossible. It is a bit easier for a non-RH kernel maintainer to contribute though. Anyone can submit an MR, and MRs to fedora-5.x branches or to Fedora specific files (configs, filters, etc) in os-build bypass a lot of the RHEL process. The Fedora kernel is still the Fedora kernel. I am 100% dedicated to the Fedora kernel. While I do tend to make some changes to fix RHEL things that will not build, it is about as far as I go. Just as I don't really work with the RHEL infrastructure at all, most RHEL developers don't work with Fedora infrastructure, so I tend to be launching the eln builds. There are people from the RHEL side who track down acks, and manage the merge workflow for RHEL specific things. I can (and do occasionally) nack things that will end up in RHEL anyway, if they have a negative impact on the Fedora kernel. I believe there is an advantage to having someone dedicated to this, who is only concerned with what is best for Fedora, and I am that person. Once it leaves kernel-ark for CentOS stream or RHEL, I am not involved at all with that tree anymore. Every set of configs is another list of things that take time to process, require care and feeding to maintain, and must be reviewed and set. I don't see any value in adding too many additional downstream configs. The mechanism already exists with config-overrides to maintain separate configs in a branch, and it is not particularly difficult to add another set of configs without overrides. I am looking to drop configs (i686, armv7), not add more. There should be no fedora specific patches, the GCC bits apply to both. All gcc patches were delivered to this list at the time that the MRs were opened. Also, the "include in releases" tag is handy for figuring out what might fix a build. The only patches that get labeled as "include in releases" are those which we can't build without. Most of what Fedora is doing gets handled on upstream lists, as the primary concern is getting them fixed upstream. MRs do get opened here, but typically not a whole lot of discussion around them. The upstream first focus of Fedora hasn't changed here, and will not. Justin > Ciao, Thorsten > _______________________________________________ > kernel mailing list -- kernel(a)lists.fedoraproject.org > To unsubscribe send an email to kernel-leave(a)lists.fedoraproject.org > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedoraproject.org/archives/list/kernel@lists.fedoraproject.org > Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

Hi Don! On 08.02.22 20:24, Donald Zickus wrote: A mail Prarit Bhargava sent today ( https://lists.fedoraproject.org/archives/list/kernel@lists.fedoraproject.... ) to this list made me get back to this thread: I'm pretty sure Prarit didn't mean to, but both links in above mail are not accessible to outsiders. :-/ And that made me realize something: kernel-ark and its mails here often make me feel like I'm back in the days of Fedora Core and Extras, where Red Hat does one thing, while the community is doing a different thing and having a hard time influencing the thing done by Red Hat employees (back then that was because either community members had no access or because it required a great deal of effort to learn everything and stay on top of things). I known that this is unfair and inaccurate, as that's not how it is with kernel-ark. But nevertheless it often just feels like that to me. Sorry. Maybe that's just me. Anyway: One thing that IMHO could help a lot to avoid that feeling on my side: get rid of all the redhatisms. E.g. rename the redhat/ directory in kernel-ark to fcrh/ (short for Fedora, CentOS, Red Hat), rpmification/, a combination of those, or something like that -- just not "redhat" (and maybe rename kernel-ark at the same time as well). Same for makes targets. And make everything feel like kernel-ark is primary a Fedora/community thing where CentOS/Red Hat are downstream consumers that are closely involved because everyone benefits from it. Ciao, Thorsten

On 22.03.22 22:29, Justin Forbes wrote: Yeah, I know, that's why I mentioned that my comparison was unfair and inaccurate (I should have used the word "wrong" instead...), but nevertheless it for me kinda feels like that (which at the same time makes me feel bad, because I *know* that it's a unfair feeling). :-/ I know, I know, don't worry, as that is visible from the outside (at least for me); and I'm really grateful for that. IOW: many thx for this! Guess the thing is: for you it's the day job and you where kinda involved when kernel-ark was built. But as a outsider you needs some time to get a hang on things with it, as you needs to learn quite a few things if you want to use it and understand the mails that trickle in from it here. And I guess that leads to the feeling I mentioned, as it reminds me of the "required a great deal of effort to learn everything and stay on top of things" quoted above. I guess I had kinda hoped that kernel-ark would be a bit more like this: * you clone kernel-ark * you apply or revert patches or change a config * you run "make rpm", "make mockbuild" or something like that And thx to some magic after a while get a just a RPM with a vanilla kernel for your arch, the OS you are running (Fedora/CentOS/RHEL) in the kernel flavor you are running currently. Kernel-ark is not that far away from it if you look closer, but OTOH is sometimes feels a lot more complicated -- I for example more than once had to deal with some RHEL specific things I didn't care about when I tried to do something with kernel-ark (maybe that was just bad luck). Also see https://cki-project.gitlab.io/kernel-ark/, which as example uses "cp redhat/configs/<flavor_os>.config .config" – here RHEL is the only example and Fedora not mentioned at all. Small details like this are what contribute to the feeling mentioned above. And FWIW: Justin, I know that you have a lot on your plate already and doing a great job, I don't want to create extra load for you with this mail, but at the same time hope someone will improve these things. Heck, often I wish I had the time to do that, but right now I have other priorities. :-/ Great! Ciao, Thorsten