From: Don Zickus on gitlab.com Merge Request: https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2654
Compiling for clang is getting trickier to do because the configs are different than gcc in some cases. Mimic the kgcov framework to setup clang to be easier to update their configs.
This framework is designed to blindly merge.py with any configs fed to it. This is unlike the build_configs framework where it uses priority.rhel|fedora rules to merge things.
This leads to strange cases where clang debug configs need different settings than clang non-debug configs. For non-lto clang, due to clang bugs it is easy to disable the debug features and get away with this. But once resolved we will have to reinvestigate. For lto clang, I just hack out the debug and kgcov config files due to clashing Kconfig depend rules.
I don't expect this to be a long term solution. Perhaps when our config rework is fully realized this just melts away. TBD.
Signed-off-by: Don Zickus dzickus@redhat.com
--- redhat/configs/common/clang/debug/CONFIG_KASAN_STACK | 1 + redhat/configs/common/clang/debug/CONFIG_KMSAN | 1 + redhat/configs/common/clang/generic/CONFIG_KASAN_STACK | 1 + redhat/configs/common/clang/generic/CONFIG_KMSAN | 1 + redhat/configs/common/clang_lto/debug/README | 0 redhat/configs/common/clang_lto/generic/arm/CONFIG_LTO_CLANG_THIN | 1 + redhat/configs/common/clang_lto/generic/arm/CONFIG_LTO_NONE | 1 + redhat/configs/common/clang_lto/generic/x86/CONFIG_LTO_CLANG_THIN | 1 + redhat/configs/common/clang_lto/generic/x86/CONFIG_LTO_NONE | 1 + redhat/configs/common/clang_lto/generic/README | 0 redhat/configs/priority.fedora | 4 - redhat/configs/priority.rhel | 4 - redhat/configs/priority.snip | 9 + redhat/kernel.spec.template | 78 ++++++--- 14 files changed, 71 insertions(+), 32 deletions(-)
From: Don Zickus dzickus@redhat.com
Remove unpackaged kgcov config files
The kgcov option is applied as a partial snippet in the spec file. The generated kernel-*-kgcov.config are not packaged and therefore unused.
My motivation here is because they conflict with the CLANG options when I overlay the CLANG options on the gcov config file. Instead of special casing them, let's just drop them.
Signed-off-by: Don Zickus dzickus@redhat.com
diff --git a/redhat/configs/priority.fedora b/redhat/configs/priority.fedora index blahblah..blahblah 100644 --- a/redhat/configs/priority.fedora +++ b/redhat/configs/priority.fedora @@ -22,21 +22,17 @@ EMPTY=s390x-zfcpdump # x86_64 x86_64=generic:generic-x86 x86_64-debug=generic:generic-x86:debug:debug-x86 -x86_64-kgcov=generic:generic-x86:kgcov
# ppc64le ppc64le=generic:generic-powerpc ppc64le-debug=generic:generic-powerpc:debug:debug-powerpc -ppc64le-kgcov=generic:generic-powerpc:kgcov
# s390x s390x=generic:generic-s390x s390x-debug=generic:generic-s390x:debug:debug-s390x -s390x-kgcov=generic:generic-s390x:kgcov
# aarch64 aarch64=generic:generic-arm:generic-arm-aarch64 aarch64-debug=generic:generic-arm:generic-arm-aarch64:debug:debug-arm-aarch64 -aarch64-kgcov=generic:generic-arm:generic-arm-aarch64:kgcov aarch64-16k=generic:generic-arm:generic-arm-aarch64:generic-arm-aarch64-16k aarch64-16k-debug=generic:generic-arm:generic-arm-aarch64:generic-arm-aarch64-16k:debug:debug-arm-aarch64:debug-arm-aarch64-16k diff --git a/redhat/configs/priority.rhel b/redhat/configs/priority.rhel index blahblah..blahblah 100644 --- a/redhat/configs/priority.rhel +++ b/redhat/configs/priority.rhel @@ -21,25 +21,21 @@ ORDER=common rhel pending-rhel custom-overrides # x86_64 x86_64=generic:generic-x86 x86_64-debug=generic:generic-x86:debug:debug-x86 -x86_64-kgcov=generic:generic-x86:kgcov x86_64-rt=generic:generic-x86:rt-generic:rt-generic-x86 x86_64-rt-debug=generic:generic-x86:debug:debug-x86:rt-generic:rt-generic-x86:rt-debug:rt-debug-x86
# ppc64le ppc64le=generic:generic-powerpc ppc64le-debug=generic:generic-powerpc:debug:debug-powerpc -ppc64le-kgcov=generic:generic-powerpc:kgcov
# s390x s390x=generic:generic-s390x s390x-debug=generic:generic-s390x:debug:debug-s390x s390x-zfcpdump=generic:generic-s390x:generic-s390x-zfcpdump -s390x-kgcov=generic:generic-s390x:kgcov
# aarch64 aarch64=generic:generic-arm:generic-arm-aarch64 aarch64-debug=generic:generic-arm:generic-arm-aarch64:debug:debug-arm:debug-arm-aarch64 -aarch64-kgcov=generic:generic-arm:generic-arm-aarch64:kgcov aarch64-64k=generic:generic-arm:generic-arm-aarch64:generic-arm-aarch64-64k aarch64-64k-debug=generic:generic-arm:generic-arm-aarch64:generic-arm-aarch64-64k:debug:debug-arm-aarch64:debug-arm-aarch64-64k aarch64-rt=generic:generic-arm:generic-arm-aarch64:rt-generic:rt-generic-arm:rt-generic-arm-aarch64
-- https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2654
From: Don Zickus dzickus@redhat.com
Apply partial snippet configs to all configs
There is a macro all_arch_configs that tries to optimize what configs we apply the partial snippets to. While it speeds up things up a bit, it creates conflicts with CLANG options. The process_configs.sh script still processes _all_ the arches and expects the CLANG options to be set correctly.
I can either add in an arch specific option to process_configs.sh to only process the configs the correct CLANG snippet was applied to _or_ just apply the snippets to _all_ the configs.
The latter is easier so that is what this change does. The cost is an extra second to apply the snippets to more config files.
Signed-off-by: Don Zickus dzickus@redhat.com
diff --git a/redhat/kernel.spec.template b/redhat/kernel.spec.template index blahblah..blahblah 100644 --- a/redhat/kernel.spec.template +++ b/redhat/kernel.spec.template @@ -469,6 +469,8 @@ Summary: The Linux kernel %endif %endif
+%define all_configs %{name}-%{specrpmversion}-*.config + # don't build noarch kernels or headers (duh) %ifarch noarch %define with_up 0 @@ -480,7 +482,6 @@ Summary: The Linux kernel %define with_bpftool 0 %define with_selftests 0 %define with_debug 0 -%define all_arch_configs %{name}-%{specrpmversion}-*.config %endif
# sparse blows up on ppc @@ -509,13 +510,11 @@ Summary: The Linux kernel %ifarch i686 %define asmarch x86 %define hdrarch i386 -%define all_arch_configs %{name}-%{specrpmversion}-i?86*.config %define kernel_image arch/x86/boot/bzImage %endif
%ifarch x86_64 %define asmarch x86 -%define all_arch_configs %{name}-%{specrpmversion}-x86_64*.config %define kernel_image arch/x86/boot/bzImage %endif
@@ -526,19 +525,16 @@ Summary: The Linux kernel %define kernel_image vmlinux %define kernel_image_elf 1 %define use_vdso 0 -%define all_arch_configs %{name}-%{specrpmversion}-ppc64le*.config %endif
%ifarch s390x %define asmarch s390 %define hdrarch s390 -%define all_arch_configs %{name}-%{specrpmversion}-s390x.config %define kernel_image arch/s390/boot/bzImage %define vmlinux_decompressor arch/s390/boot/vmlinux %endif
%ifarch aarch64 -%define all_arch_configs %{name}-%{specrpmversion}-aarch64*.config %define asmarch arm64 %define hdrarch arm64 %define make_target vmlinuz.efi @@ -1781,7 +1777,7 @@ FLAVOR=%{primary_target} SPECPACKAGE_NAME=%{name} SPECVERSION=%{specversion} SPE
# Merge in any user-provided local config option changes %ifnarch %nobuildarches -for i in %{all_arch_configs} +for i in %{all_configs} do mv $i $i.tmp ./merge.py %{SOURCE3001} $i.tmp > $i
-- https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2654
From: Don Zickus dzickus@redhat.com
Add clang config framework
Compiling for clang is getting trickier to do because the configs are different than gcc in some cases. Mimic the kgcov framework (which uses priority files) to setup clang to be easier to update their configs.
This leads to strange cases where clang debug configs need different settings than clang non-debug configs. The priority.snip can handle the variety of partial snippets for variants and arches well, the kernel.spec needed to be updated to apply it reasonably.
I created a generic partial snippet loop that focuses on arch and non-debug vs debug. This required adding debug options to priority.snip. Once implemented it is a matter of collecting the snippets and applying them based on arch and debug. This should provide a variety of flexibility for teams like clang and make it easier to roll in config changes based on which compiler is used.
Signed-off-by: Don Zickus dzickus@redhat.com
diff --git a/redhat/configs/common/clang/debug/CONFIG_KASAN_STACK b/redhat/configs/common/clang/debug/CONFIG_KASAN_STACK new file mode 100644 index blahblah..blahblah 100644 --- /dev/null +++ b/redhat/configs/common/clang/debug/CONFIG_KASAN_STACK @@ -0,0 +1 @@ +CONFIG_KASAN_STACK=y diff --git a/redhat/configs/common/clang/debug/CONFIG_KMSAN b/redhat/configs/common/clang/debug/CONFIG_KMSAN new file mode 100644 index blahblah..blahblah 100644 --- /dev/null +++ b/redhat/configs/common/clang/debug/CONFIG_KMSAN @@ -0,0 +1 @@ +# CONFIG_KMSAN is not set diff --git a/redhat/configs/common/clang/generic/CONFIG_KASAN_STACK b/redhat/configs/common/clang/generic/CONFIG_KASAN_STACK new file mode 100644 index blahblah..blahblah 100644 --- /dev/null +++ b/redhat/configs/common/clang/generic/CONFIG_KASAN_STACK @@ -0,0 +1 @@ +# CONFIG_KASAN_STACK is not set diff --git a/redhat/configs/common/clang/generic/CONFIG_KMSAN b/redhat/configs/common/clang/generic/CONFIG_KMSAN new file mode 100644 index blahblah..blahblah 100644 --- /dev/null +++ b/redhat/configs/common/clang/generic/CONFIG_KMSAN @@ -0,0 +1 @@ +# CONFIG_KMSAN is not set diff --git a/redhat/configs/common/clang_lto/debug/README b/redhat/configs/common/clang_lto/debug/README new file mode 100644 index blahblah..blahblah 100644 --- /dev/null +++ b/redhat/configs/common/clang_lto/debug/README diff --git a/redhat/configs/common/clang_lto/generic/README b/redhat/configs/common/clang_lto/generic/README new file mode 100644 index blahblah..blahblah 100644 --- /dev/null +++ b/redhat/configs/common/clang_lto/generic/README diff --git a/redhat/configs/common/clang_lto/generic/arm/CONFIG_LTO_CLANG_THIN b/redhat/configs/common/clang_lto/generic/arm/CONFIG_LTO_CLANG_THIN new file mode 100644 index blahblah..blahblah 100644 --- /dev/null +++ b/redhat/configs/common/clang_lto/generic/arm/CONFIG_LTO_CLANG_THIN @@ -0,0 +1 @@ +CONFIG_LTO_CLANG_THIN=y diff --git a/redhat/configs/common/clang_lto/generic/arm/CONFIG_LTO_NONE b/redhat/configs/common/clang_lto/generic/arm/CONFIG_LTO_NONE new file mode 100644 index blahblah..blahblah 100644 --- /dev/null +++ b/redhat/configs/common/clang_lto/generic/arm/CONFIG_LTO_NONE @@ -0,0 +1 @@ +# CONFIG_LTO_NONE is not set diff --git a/redhat/configs/common/clang_lto/generic/x86/CONFIG_LTO_CLANG_THIN b/redhat/configs/common/clang_lto/generic/x86/CONFIG_LTO_CLANG_THIN new file mode 100644 index blahblah..blahblah 100644 --- /dev/null +++ b/redhat/configs/common/clang_lto/generic/x86/CONFIG_LTO_CLANG_THIN @@ -0,0 +1 @@ +CONFIG_LTO_CLANG_THIN=y diff --git a/redhat/configs/common/clang_lto/generic/x86/CONFIG_LTO_NONE b/redhat/configs/common/clang_lto/generic/x86/CONFIG_LTO_NONE new file mode 100644 index blahblah..blahblah 100644 --- /dev/null +++ b/redhat/configs/common/clang_lto/generic/x86/CONFIG_LTO_NONE @@ -0,0 +1 @@ +# CONFIG_LTO_NONE is not set diff --git a/redhat/configs/priority.snip b/redhat/configs/priority.snip index blahblah..blahblah 100644 --- a/redhat/configs/priority.snip +++ b/redhat/configs/priority.snip @@ -10,3 +10,12 @@ EMPTY=
# gcov options kgcov=kgcov +kgcov-debug=kgcov + +# clang options +clang=clang-generic +clang-debug=clang-debug +clang_lto-x86_64=clang-generic:clang_lto-generic:clang_lto-generic-x86 +clang_lto-x86_64-debug=clang-debug:clang_lto-debug +clang_lto-aarch64=clang-generic:clang_lto-generic:clang_lto-generic-arm +clang_lto-aarch64-debug=clang-debug:clang_lto-debug diff --git a/redhat/kernel.spec.template b/redhat/kernel.spec.template index blahblah..blahblah 100644 --- a/redhat/kernel.spec.template +++ b/redhat/kernel.spec.template @@ -908,7 +908,14 @@ Source65: filter-s390x.sh.fedora Source66: filter-modules.sh.fedora %endif
-Source75: partial-kgcov-snip.config +Source70: partial-kgcov-snip.config +Source71: partial-kgcov-debug-snip.config +Source72: partial-clang-snip.config +Source73: partial-clang-debug-snip.config +Source74: partial-clang_lto-x86_64-snip.config +Source75: partial-clang_lto-x86_64-debug-snip.config +Source76: partial-clang_lto-aarch64-snip.config +Source77: partial-clang_lto-aarch64-debug-snip.config Source80: generate_all_configs.sh Source81: process_configs.sh
@@ -1771,30 +1778,57 @@ cp $RPM_SOURCE_DIR/%{name}-*.config . cp %{SOURCE80} . # merge.py cp %{SOURCE3000} . -# kernel-local +# kernel-local - rename and copy for partial snippet config process cp %{SOURCE3001} . +cp kernel-local partial-kernel-local-snip.config +cp kernel-local partial-kernel-local-debug-snip.config FLAVOR=%{primary_target} SPECPACKAGE_NAME=%{name} SPECVERSION=%{specversion} SPECRPMVERSION=%{specrpmversion} ./generate_all_configs.sh %{debugbuildsenabled}
-# Merge in any user-provided local config option changes -%ifnarch %nobuildarches -for i in %{all_configs} -do - mv $i $i.tmp - ./merge.py %{SOURCE3001} $i.tmp > $i +# Collect custom defined config options +PARTIAL_CONFIGS="" %if %{with_gcov} - echo "Merging with gcov options" - cat %{SOURCE75} - mv $i $i.tmp - ./merge.py %{SOURCE75} $i.tmp > $i +PARTIAL_CONFIGS="$PARTIAL_CONFIGS %{SOURCE70} %{SOURCE71}" %endif - rm $i.tmp -done +%if %{with toolchain_clang} +PARTIAL_CONFIGS="$PARTIAL_CONFIGS %{SOURCE72} %{SOURCE73}" %endif - %if %{with clang_lto} -for i in *aarch64*.config *x86_64*.config; do - sed -i 's/# CONFIG_LTO_CLANG_THIN is not set/CONFIG_LTO_CLANG_THIN=y/' $i - sed -i 's/CONFIG_LTO_NONE=y/# CONFIG_LTO_NONE is not set/' $i +PARTIAL_CONFIGS="$PARTIAL_CONFIGS %{SOURCE74} %{SOURCE75} %{SOURCE76} %{SOURCE77}" +%endif +PARTIAL_CONFIGS="$PARTIAL_CONFIGS partial-kernel-local-snip.config partial-kernel-local-debug-snip.config" + +GetArch() +{ + case "$1" in + *aarch64*) echo "aarch64" ;; + *ppc64le*) echo "ppc64le" ;; + *s390x*) echo "s390x" ;; + *x86_64*) echo "x86_64" ;; + # no arch, apply everywhere + *) echo "" ;; + esac +} + +# Merge in any user-provided local config option changes +%ifnarch %nobuildarches +for i in %{all_configs} +do + kern_arch="$(GetArch $i)" + kern_debug="$(echo $i | grep -q debug && echo "debug" || echo "")" + + for j in $PARTIAL_CONFIGS + do + part_arch="$(GetArch $j)" + part_debug="$(echo $j | grep -q debug && echo "debug" || echo "")" + + # empty arch means apply to all arches + if [ "$part_arch" == "" -o "$part_arch" == "$kern_arch" ] && [ "$part_debug" == "$kern_debug" ] + then + mv $i $i.tmp + ./merge.py $j $i.tmp > $i + fi + done + rm -f $i.tmp done %endif
-- https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2654
From: Don Zickus on gitlab.com https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2654#note_1584576...
Yeah, I am scratching my head on that too. I got this working as a starter. Open to ideas.
kernel@lists.fedoraproject.org
-
Don Zickus (via Email Bridge)