kernel January 2024

kernel@lists.fedoraproject.org

5 participants
83 discussions

[OS-BUILD PATCH] Add wildcard macros -baseonly -gcov
by Don Zickus (via Email Bridge) 18 Sep '24

18 Sep '24

From: Don Zickus <dzickus(a)redhat.com> Add wildcard macros -baseonly -gcov There are multiple usecases for -baseonly and -gcov on various targets in the Makefile. Instead of calling those targets out repeatedly, lets generalize them a bit. Allow adding -baseonly and -gcov to: dist-rpm dist-srpm dist-brew dist-koji distg-brew distg-koji Gcov is useful for adding code coverage. Baseonly is useful to only build the kernel for debugging purposes without userspace tools. Signed-off-by: Don Zickus <dzickus(a)redhat.com> diff --git a/redhat/Makefile b/redhat/Makefile index blahblah..blahblah 100644 --- a/redhat/Makefile +++ b/redhat/Makefile @@ -297,6 +297,12 @@ dist-sources: dist-kabi dist-kabi-dup sources-rh dist-test-patch: generate-testpatch-tmp @mv $(TESTPATCH).tmp $(TESTPATCH); +%-baseonly: RPMBUILDOPTS=--target $(MACH) --without debug --without debuginfo --without vdso_install --without bpftool --without perf --without tools --without selftests -bb +%-baseonly: BUILDOPTS+=-debug -debuginfo -vdso_install -bpftool -perf -tools -selftests + +%-gcov: BUILDID=".gcov" +%-gcov: BUILDOPTS+="+gcov" + do-rpmbuild: dist-sources $(RPMBUILD) --define "_sourcedir $(SOURCES)" --define "_builddir $(RPM)/BUILD" --define "_srcrpmdir $(RPM)/SRPMS" --define "_rpmdir $(RPM)/RPMS" --define "_specdir $(RPM)/SPECS" --define "dist $(DIST)" $(RPMBUILDOPTS) $(RPM)/SOURCES/$(PACKAGE_NAME).spec @@ -306,9 +312,9 @@ dist-all-rpms: dist-sources do-rpmbuild dist-srpm: RPMBUILDOPTS=--nodeps -bs dist-srpm: dist-sources do-rpmbuild -dist-srpm-gcov: BUILDID=".gcov" -dist-srpm-gcov: BUILDOPTS+="+gcov" -dist-srpm-gcov: dist-srpm +dist-srpm-%: dist-srpm + @# phony command to force this to be a target and not variable + @FOO=BAR dist-rpms: RPMBUILDOPTS=--target $(MACH) -bb dist-rpms: dist-sources do-rpmbuild @@ -322,8 +328,7 @@ dist-prep: dist-sources do-rpmbuild dist-perf: RPMBUILDOPTS=--without up --without smp --without zfcpdump --without debug --without doc --without headers --without --without doc --without debuginfo --target $(MACH) -bb dist-perf: dist-sources do-rpmbuild -dist-rpm-baseonly: RPMBUILDOPTS=--target $(MACH) --without debug --without debuginfo --without vdso_install --without bpftool --without perf --without tools -bb -dist-rpm-baseonly: dist-sources do-rpmbuild +dist-rpm-%: dist-sources do-rpmbuild # unless you know what you're doing, you don't want to use the next four ones dist-release-finish: setup-source @@ -388,9 +393,25 @@ distg-koji: BUILD_FLAGS ?= $(KOJI_FLAGS) $(TEST_FLAGS) dist-brew dist-koji: dist-%: dist-srpm $* $(BUILD_PROFILE) build $(BUILD_FLAGS) $(BUILD_TARGET) $(SRPMS)/$(PACKAGE_NAME)-$(KVERSION)-$(PKGRELEASE)$(DIST).src.rpm $(OUTPUT_FILE) +dist-brew-%: dist-brew + @# phony command to force this to be a target and not variable + @FOO=BAR + +dist-koji-%: dist-koji + @# phony command to force this to be a target and not variable + @FOO=BAR + distg-brew distg-koji: distg-%: $* $(BUILD_PROFILE) build $(BUILD_FLAGS) $(BUILD_TARGET) "$(RHGITURL)?redhat/koji#$(RHGITCOMMIT)" +distg-brew-%: dist-brew + @# phony command to force this to be a target and not variable + @FOO=BAR + +distg-koji-%: dist-koji + @# phony command to force this to be a target and not variable + @FOO=BAR + .PHONY: $(REDHAT)/rpm/SOURCES/$(PACKAGE_NAME).spec $(REDHAT)/rpm/SOURCES/$(PACKAGE_NAME).spec: @echo "dist-sources" @@ -469,34 +490,30 @@ dist-full-help: @echo 'Building targets:' @echo ' All RPM/SRPM files will be put under the redhat/rpm/ directory.' @echo '' - @echo ' dist-srpm - Create a source RPM and put it into the redhat/rpm/SRPMS/' + @echo ' dist-srpm@ - Create a source RPM and put it into the redhat/rpm/SRPMS/' @echo ' directory. See the dist-brew target for available options.' - @echo ' dist-srpm-gcov - Create a source RPM with gcov enabled and put it into the' - @echo ' redhat/rpm/SRPMS/ directory.' - @echo ' dist-brew - Create a kernel SRPM and then call brew to build the' + @echo ' dist-brew@ - Create a kernel SRPM and then call brew to build the' @echo ' created SRPM. Add BUILDOPTS="+<opt> -<opt> [...]" to' @echo ' enable/disable build options.' @echo ' Available <opt>s and their default values:' \ $$(sed -n -e 's/^%define with_$[^ \t]*$.*\?_without_.*/+\1/p' \ -e 's/^%define with_$[^ \t]*$.*\?_with_.*/-\1/p' kernel.spec.template | \ grep -v 'only$$') | fmt -80 - @echo ' dist-koji - Create a kernel SRPM and then call koji to build the' + @echo ' dist-koji@ - Create a kernel SRPM and then call koji to build the' @echo ' created SRPM. See the dist-brew target for available' @echo ' options.' - @echo ' distg-brew - Pass HEAD of the current git branch to brew to build an' + @echo ' distg-brew@ - Pass HEAD of the current git branch to brew to build an' @echo ' RPM set. Do not forget to push to the remote repository' @echo ' first. Preceed make command by RHGITCOMMIT=<commitID>' @echo ' specify commit ID to use.' @echo ' To set the remote repo, invoke:' @echo ' git config rhg.url git://<repo_path>' - @echo ' distg-koji - Pass HEAD of the current git branch to koji to build an' + @echo ' distg-koji@ - Pass HEAD of the current git branch to koji to build an' @echo ' RPM set. Do not forget to push to the remote repository' @echo ' first. See the distg-brew target for options and' @echo ' configuration.' - @echo ' dist-rpms - Create the binary RPMS for the kernel.' + @echo ' dist-rpms@ - Create the binary RPMS for the kernel.' @echo ' See the dist-brew target for available options.' - @echo ' dist-rpm-baseonly - Create the binary RPMS for the kernel and modules' - @echo ' (no userspace tools or debuginfo).' @echo ' dist-kernel-<type> - Create binary RPMS for a particular kernel type.' @echo ' Available <type>s:'\ $$(sed -n 's/^%define with_$[^ ]*only$.*/\1/p' kernel.spec.template) @@ -524,6 +541,12 @@ dist-full-help: @echo ' dist-rhel-configs - build ELN configs' @echo ' dist-fedora-configs - build Fedora configs' + @echo '' + @echo 'Wildcard targets: [indicated by '@' above]' + @echo ' *-baseonly - builds only the kernel (disables userspace and debuginfo)' + @echo ' - Examples: dist-srpm-baseonly, dist-brew-baseonly' + @echo ' *-gcov - builds a kernel with gcov enabled' + @echo ' - Examples: dist-srpm-gcov, dist-brew-gcov' @echo '' @echo 'kABI targets:' @echo ' dist-kabi - Create kABI stablelist files in redhat/kabi/kabi-rhel*/' -- https://gitlab.com/cki-project/kernel-ark/-/merge_requests/1444

3 2

[OS-BUILD PATCHv2] Fix binutils breakage
by Don Zickus (via Email Bridge) 15 Aug '24

15 Aug '24

From: Don Zickus <dzickus(a)redhat.com> Fix binutils breakage The binutils package in Rawhide has stricter checks about PIE/PIC code and will start reporting errors if it detects mixing and matching no-PIE with PIE binaries (especially around bpf binaries). Example errors look like: /usr/bin/ld: /tmp/ccL7dkfR.o: relocation R_X86_64_32S against `.rodata' can not be used when making a PIE object; recompile with -fPIE /usr/bin/ld: failed to set dynamic section sizes: bad value collect2: error: ld returned 1 exit status Fix this by adding more CFLAGS to force PIE/PIC generated binaries to bpf releated selftests, tools and samples. This patch was provided by Carlos O'Donell and understands how it works. I am just the middle man. All -fPIE errors are resolved except for one about libbpf.a. The static version of the library doesn't take CFLAGS so it can't be forced to build with PIE/PIC. This error is ignored by the spec file, so it is ignored by this patch too. Signed-off-by: Don Zickus <dzickus(a)redhat.com> diff --git a/redhat/kernel.spec.template b/redhat/kernel.spec.template index blahblah..blahblah 100644 --- a/redhat/kernel.spec.template +++ b/redhat/kernel.spec.template @@ -2769,7 +2769,7 @@ if [ ! -f include/generated/autoconf.h ]; then %{make} %{?_smp_mflags} modules_prepare fi -%{make} %{?_smp_mflags} ARCH=$Arch V=1 M=samples/bpf/ VMLINUX_H="${RPM_VMLINUX_H}" || true +%{make} %{?_smp_mflags} TPROGS_USER_CFLAGS="%{?build_hostcflags}" TPROGS_USER_LDFLAGS="%{?build_hostldflags}" EXTRA_CFLAGS="%{?build_hostcflags}" EXTRA_LDFLAGS="%{?build_hostldflags}" ARCH=$Arch V=1 M=samples/bpf/ VMLINUX_H="${RPM_VMLINUX_H}" || true # Prevent bpf selftests to build bpftool repeatedly: export BPFTOOL=$(pwd)/tools/bpf/bpftool/bpftool @@ -2783,7 +2783,7 @@ pushd tools/testing/selftests force_targets="" %endif -%{make} %{?_smp_mflags} ARCH=$Arch V=1 TARGETS="bpf mm livepatch net net/forwarding net/mptcp netfilter tc-testing memfd drivers/net/bonding" SKIP_TARGETS="" $force_targets INSTALL_PATH=%{buildroot}%{_libexecdir}/kselftests VMLINUX_H="${RPM_VMLINUX_H}" install +%{make} %{?_smp_mflags} USERCFLAGS="%{?build_hostcflags}" USERLDFLAGS="%{?build_hostldflags}" ARCH=$Arch V=1 TARGETS="bpf mm livepatch net net/forwarding net/mptcp netfilter tc-testing memfd drivers/net/bonding" SKIP_TARGETS="" $force_targets INSTALL_PATH=%{buildroot}%{_libexecdir}/kselftests VMLINUX_H="${RPM_VMLINUX_H}" install # 'make install' for bpf is broken and upstream refuses to fix it. # Install the needed files manually. -- https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2749

4 6

[OS-BUILD PATCHv2 0/6] secure boot signing updates
by Jan Stancek (via Email Bridge) 02 Apr '24

02 Apr '24

From: Jan Stancek on gitlab.com Merge Request: https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2849 Forward port secure boot signing changes from c9s to ARK. eln scratch: https://koji.fedoraproject.org/koji/taskinfo?taskID=111509830 rawhide scratch: https://koji.fedoraproject.org/koji/taskinfo?taskID=111494146 Signed-off-by: Jan Stancek <jstancek(a)redhat.com> --- redhat/keys/redhatsecureboot003.cer | Bin redhat/keys/redhatsecureboot301.cer | Bin redhat/keys/redhatsecureboot401.cer | Bin redhat/keys/redhatsecurebootca1.cer | Bin redhat/keys/redhatsecurebootca2.cer | Bin redhat/keys/redhatsecurebootca4.cer | Bin redhat/keys/secureboot_ppc.cer | Bin redhat/keys/secureboot_s390.cer | Bin redhat/Makefile | 7 +- redhat/kernel.spec.template | 113 +++++++++++++++-------------------- 10 files changed, 51 insertions(+), 69 deletions(-)

2 9

[OS-BUILD PATCHv3 0/5] Add libperf packages and build kernel tools for Fedora
by Justin M. Forbes (via Email Bridge) 02 Feb '24

02 Feb '24

From: Justin M. Forbes on gitlab.com Merge Request: https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2907 When kernel-tools was split out to a separate package in Fedora, it was due to several factors. Most of those issues have been mitigated in other ways, and there are advantages to building tools with the kernel build. Let's turn on tools for Fedora. To do so, we need to add the libperf packages, but this enables them for Fedora only. It is worth noting that the Fedora package for bpftool will retain the upstream kernel versioning. This is because the bpftool versioning scheme is incompatible with the stable Fedora process. --- redhat/kernel.spec.template | 78 ++++++++++++++++++++++++++++++++++++++++++-- 1 files changed, 74 insertions(+), 4 deletions(-)

3 7

[OS-BUILD PATCHv5 0/2] spec: use jsut-built bpftool for vmlinux.h generation
by Artem Savkov (via Email Bridge) 02 Feb '24

02 Feb '24

From: Artem Savkov on gitlab.com Merge Request: https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2884 Forwardport from rhel9. Bugzilla: http://bugzilla.redhat.com/2120968 Upstream status: RHEL-only With the latest toolchain update build fails due to libbpf missing btf enum64 support. 5.19 update contains libbpf update but the spec uses buildroot's (old) bpftool. Switch to use the just-built bpftool. The snippet cannot be moved below bpftool build since vmlinux.h is needed to build_tools. Credits to Felix Maurer <fmaurer(a)redhat.com> Signed-off-by: Yauheni Kaliuta <ykaliuta(a)redhat.com> --- redhat/kernel.spec.template | 11 ++++++----- 1 files changed, 6 insertions(+), 5 deletions(-)

2 3

[OS-BUILD PATCH] configs: make CONFIG_EDAC_GHES a module
by Aristeu Rozanski (via Email Bridge) 31 Jan '24

31 Jan '24

From: Aristeu Rozanski <arozansk(a)redhat.com> configs: make CONFIG_EDAC_GHES a module Since 802e7f1dfed7cc7fb309995e0c4138f08977fdfc it's possible to build ghes_edac as module and this is useful for customers that want to unload it during runtime. Signed-off-by: Aristeu Rozanski <arozansk(a)redhat.com> diff --git a/redhat/configs/common/generic/CONFIG_EDAC_GHES b/redhat/configs/fedora/generic/CONFIG_EDAC_GHES rename from redhat/configs/common/generic/CONFIG_EDAC_GHES rename to redhat/configs/fedora/generic/CONFIG_EDAC_GHES index blahblah..blahblah 100644 --- a/redhat/configs/common/generic/CONFIG_EDAC_GHES +++ b/redhat/configs/fedora/generic/CONFIG_EDAC_GHES diff --git a/redhat/configs/rhel/generic/CONFIG_EDAC_GHES b/redhat/configs/rhel/generic/CONFIG_EDAC_GHES new file mode 100644 index blahblah..blahblah 100644 --- /dev/null +++ b/redhat/configs/rhel/generic/CONFIG_EDAC_GHES @@ -0,0 +1 @@ +CONFIG_EDAC_GHES=m -- https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2914

2 3

[OS-BUILD PATCH] gitlab-ci: enable native tools for Rawhide CI
by Michael Hofmann (via Email Bridge) 31 Jan '24

31 Jan '24

From: Michael Hofmann <mhofmann(a)redhat.com> gitlab-ci: enable native tools for Rawhide CI With [1], the kernel tools build are built from the kernel package. Enable them in the CI pipelines as well. [1] https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2907 Signed-off-by: Michael Hofmann <mhofmann(a)redhat.com> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index blahblah..blahblah 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -95,6 +95,7 @@ workflow: variables: name: rawhide-ark builder_image: quay.io/cki/builder-rawhide + native_tools: 'true' kpet_tree_name: rawhide .trigger_eln: -- https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2915

1 0

[OS-BUILD PATCHv2 0/4] Add libperf packages and build kernel tools for Fedora
by Justin M. Forbes (via Email Bridge) 30 Jan '24

30 Jan '24

From: Justin M. Forbes on gitlab.com Merge Request: https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2907 When kernel-tools was split out to a separate package in Fedora, it was due to several factors. Most of those issues have been mitigated in other ways, and there are advantages to building tools with the kernel build. Let's turn on tools for Fedora. To do so, we need to add the libperf packages, but this enables them for Fedora only. It is worth noting that the Fedora package for bpftool will retain the upstream kernel versioning. This is because the bpftool versioning scheme is incompatible with the stable Fedora process. --- redhat/kernel.spec.template | 81 +++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 78 insertions(+), 3 deletions(-)

3 45

[OS-BUILD PATCHv10] Add new os-build targets: rt-devel and automotive-devel
by Don Zickus (via Email Bridge) 29 Jan '24

29 Jan '24

From: Don Zickus <dzickus(a)redhat.com> Add new os-build targets: rt-devel and automotive-devel This is an attempt to automate the rt and automotive devel branches using the upstream linux-rt-devel tree as the base combined with os-build. The overall process isn't too complicated but there are enough steps to make it compilicated. Steps: * map upstream linux-rt-devel to master-rt branch * update os-build * grab linux version from master-rt and os-build * if version the same then merge os-build and master-rt to os-build-rt-automated * else merge tag kernel-N.V.0-0 and master-rt to os-build-rt-automated until master-rt is update to os-build * merge os-build-rt-automated into os-build-rt-devel * merge os-build-rt-automated into os-build-automotive-devel * run the generate pending-rhel config scripts on rt-devel and automotive-devel The script has beginning logic to handle rebasing if necessary when the rt-devel branch transitions from os-build-stable (linux-stable) to linux master again. NOTE: The script uses kernel-N.V.0-0 which is rebased after os-build linux-x.y GA is created but before linux-x.y+1 pre-rc1 merges happen. The reason for this is because linux-stable-x.y doesn't exist until linux-x.y+1-rc1 exists thus leaving linux-stable-x.y-1 in the meantime. An awkward gap exists, use kernel-N.V.0-0 as the base. The script has no effect on the day-to-day operations of os-build. They are designed to be run from a gitlab cron job and update in the background. Once they are deemed stable, adding ARK MRs that target either os-build-rt-devel or os-build-automotive-devel will be possible and those branches can start proper parallel developement with os-build. Cleanup namespace pollution because shellcheck doesn't like 'local'. Signed-off-by: Don Zickus <dzickus(a)redhat.com> diff --git a/redhat/scripts/ci/ark-ci-env.sh b/redhat/scripts/ci/ark-ci-env.sh index blahblah..blahblah 100644 --- a/redhat/scripts/ci/ark-ci-env.sh +++ b/redhat/scripts/ci/ark-ci-env.sh @@ -8,24 +8,188 @@ die() ci_pre_check() { - if test -n "${TO_PUSH}"; then - if test -z "${GITLAB_PROJECT_URL}" || test -z "$GITLAB_PROJECT_PUSHURL"; then - echo "To enable git-push, please run:" - echo "git remote add gitlab <url>" - echo "git remote set-url --push gitlab <pushurl>" + if test -z "${GITLAB_PROJECT_URL}" || test -z "$GITLAB_PROJECT_PUSHURL"; then + echo "To enable git-push, please run:" + echo "git remote add gitlab <url>" + echo "git remote set-url --push gitlab <pushurl>" + if test -n "${TO_PUSH}"; then die "Misconfigured 'gitlab' entry for git" fi fi git diff-index --quiet HEAD || die "Dirty tree, please clean before merging." } +# wrapper around branches that may not be exist yet +ark_git_branch() +{ + _target_branch="$1" + _source_branch="$2" + + # switch to branch if it exists otherwise create and set to source + # branch + git show-ref -q --heads "$_target_branch" || \ + git branch "$_target_branch" "$_source_branch" +} + +# GitLab can only mirror one project at a time. This wrapper function does +# the mirroring for any other branches. +ark_git_mirror() +{ + target_branch="$1" + upstream_tree="$2" + source_branch="$3" + reset_branch="$4" + + prev_branch="$(git rev-parse --abbrev-ref HEAD)" + remote_branch="$upstream_tree/$source_branch" + ark_git_branch "$target_branch" "$remote_branch" + git checkout "$target_branch" + git fetch "$upstream_tree" "$source_branch" + if test -z "$reset_branch"; then + git merge "$remote_branch" || die "git merge $remote_branch failed" + else + git reset --hard "$remote_branch" || die "git reset $remote_branch failed" + fi + git checkout "$prev_branch" +} + +# Merge wrapper in case issues arise +ark_git_merge() +{ + source_branch="$1" + target_branch="$2" + reset_branch="$3" + + prev_branch="$(git rev-parse --abbrev-ref HEAD)" + ark_git_branch "$target_branch" "$source_branch" + git checkout "$target_branch" + if test -n "$reset_branch"; then + # there are cases when the initial merge is a reset + git reset --hard "$source_branch" || die "git reset $source_branch failed" + elif ! git merge -m "Merge '$source_branch' into '$target_branch'" "$source_branch"; then + git merge --abort + printf "Merge conflict; halting!\n" + printf "To reproduce:\n" + printf "* git checkout %s\n" "${target_branch}" + printf "* git merge %s\n" "${source_branch}" + die "Merge conflicts" + fi + + git checkout "$prev_branch" + return 0 +} + +ark_git_rebase() +{ + rebase_branch="$1" + _upstream="$2" + _base="$3" + + prev_branch="$(git rev-parse --abbrev-ref HEAD)" + git checkout "${rebase_branch}" + if ! git rebase --onto "$_base" "$_upstream"; then + git rebase --abort + printf "Rebase conflict; halting!\n" + printf "To reproduce:\n" + printf "* git checkout %s\n" "${rebase_branch}" + printf "* git rebase --onto %s %s\n" "${_base}" "${_upstream}" + die "Rebase conflicts" + fi + git checkout "$prev_branch" + return 0 +} + +ark_update_configs() +{ + config_branch="$1" + skip_configs="$2" + + prev_branch="$(git rev-parse --abbrev-ref HEAD)" + git checkout "${config_branch}" + + # Generates and commits all the pending configs + make -j FLAVOR=fedora dist-configs-commit + # Skip executing gen_config_patches.sh for new Fedora configs + + old_head="$(git rev-parse HEAD)" + make -j FLAVOR=rhel dist-configs-commit + new_head="$(git rev-parse HEAD)" + + + # Converts each new pending config from above into its finalized git + # configs/<date>/<config> config_branch. These commits are used for Merge + # Requests. + [ "$old_head" != "$new_head" ] && CONFIGS_ADDED="1" || CONFIGS_ADDED="" + + if test "$CONFIGS_ADDED"; then + if test -z "$skip_configs"; then + git checkout "$prev_branch" + ./redhat/scripts/genspec/gen_config_patches.sh "$config_branch" + fi + else + printf "No new configuration values exposed from " + printf "merging %s into $BRANCH\n" "$UPSTREAM_REF" + fi + + git checkout "$prev_branch" + test -z "$CONFIGS_ADDED" && return 0 || return 1 +} + +ark_push_changes() +{ + push_branch="$1" + skip_configs="$2" + + prev_branch="$(git rev-parse --abbrev-ref HEAD)" + git checkout "${push_branch}" + + TMPFILE=".push-warnings" + touch "$TMPFILE" + + test "$TO_PUSH" && PUSH_VERB="Pushing" || PUSH_VERB="To push" + PUSH_STR="branch ${push_branch} to ${GITLAB_URL}" + PUSH_CMD="git push gitlab ${push_branch}" + PUSH_CONFIG_STR="config update branches" + PUSH_CONFIG_CMD="for conf_branch in \$(git branch | grep configs/${push_branch}/\"\$(date +%F)\"); do + git push \\ + -o merge_request.create \\ + -o merge_request.target=\"$push_branch\" \\ + -o merge_request.remove_source_branch \\ + gitlab \"\$conf_branch\" 2>&1 | tee -a $TMPFILE + done + " + + #Push push_branch + echo "# $PUSH_VERB $PUSH_STR" + echo "$PUSH_CMD" + test "$TO_PUSH" && eval "$PUSH_CMD" + + #Push config branches if created + if test -z "$skip_configs"; then + echo + echo "# $PUSH_VERB $PUSH_CONFIG_STR" + echo "$PUSH_CONFIG_CMD" + test "$TO_PUSH" && eval "$PUSH_CONFIG_CMD" + fi + + # GitLab server side warnings do not fail git-push but leave verbose + # WARNING messages. Grep for those and consider it a script + # failure. Make sure all push_branches are pushed first as follow up + # git-pushes may succeed. + grep -q "remote:[ ]* WARNINGS" "$TMPFILE" && die "Server side warnings" + + rm "$TMPFILE" + git checkout "$prev_branch" + return 0 +} + # Common variables for all CI scripts UPSTREAM_REF=${1:-"master"} BRANCH=${2:-"os-build"} PROJECT_ID=${PROJECT_ID:-"13604247"} TO_PUSH=${DIST_PUSH:-""} -GITLAB_PROJECT_URL="$(git remote get-url gitlab 2>/dev/null)" -GITLAB_PROJECT_PUSHURL="$(git config --get remote.gitlab.pushurl 2>/dev/null)" +GITLAB_PROJECT_URL="$(git remote get-url gitlab 2>/dev/null)" || true +GITLAB_PROJECT_PUSHURL="$(git config --get remote.gitlab.pushurl 2>/dev/null)" || true ci_pre_check diff --git a/redhat/scripts/ci/ark-merge-rt.sh b/redhat/scripts/ci/ark-merge-rt.sh new file mode 100755 index blahblah..blahblah 100755 --- /dev/null +++ b/redhat/scripts/ci/ark-merge-rt.sh @@ -0,0 +1,173 @@ +#!/bin/bash +# +# This script is intended to sync up the RT and automotive branch (derivative +# of RT). It adds the extra twist of detecting the right upstream rt branch +# to sync with depending on the existance of the next branch. Sometimes the +# rt-devel branch waits until -rc1/2 to create new branches. +# Finally the code handles the rebases in those cases where newer branches +# are available. +# +# Why the complexity? +# Development branches will need to be periodically rebased unfortunately. +# Using 'git rebase --onto <new head> <old_head>' only works with one common +# branch to rebase from not two. Meanwhile, the -devel branches are formed +# from two upstream branches, os-build and linux-rt-devel. The idea is +# to merge the two branches into a single throwaway branch that can be +# recreated from scratch anytime then use that as the base for -devel. + +set -e + +# source common CI functions and variables +# shellcheck disable=SC1091 +. "$(dirname "$0")"/ark-ci-env.sh + +#Upstream RT tree git://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-rt-devel.git +UPSTREAM_RT_TREE_URL="git://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-rt-devel.git" +UPSTREAM_RT_TREE_NAME="linux-rt-devel" +DOWNSTREAM_RT_BRANCH="master-rt-devel" +RT_AUTOMATED_BRANCH="os-build-rt-automated" +RT_DEVEL_BRANCH="os-build-rt-devel" +AUTOMOTIVE_DEVEL_BRANCH="os-build-automotive-devel" + +# verify git remote rt is setup +if ! git remote get-url "$UPSTREAM_RT_TREE_NAME" 2>/dev/null; then + die "Please 'git remote add linux-rt-devel $UPSTREAM_RT_TREE_URL'" +fi + +# grab the os-build base branches +ark_git_mirror "os-build" "origin" "os-build" +ark_git_mirror "master" "origin" "master" + +# make sure tags are available for git-describe to correctly work +git fetch -t origin + +# upstream -rt devel branches are aligned with version numbers and are not +# always up to date with master. Figure out which branch to mirror based on +# version number and existance. We may have to trigger a rebase. + +# what are the current versions of rt-devel and os-build (use 'master' to +# avoid fedora tagging of kernel-X.Y.0.0.....) +# use git tags which are always 'vX.Y-rcZ-aaa-gbbbbb' or 'vX.Y-aaa-gbbbbb' +# where X.Y is the version number that maps to linux-rt's branch of +# 'linux-X.Y.y' +get_upstream_version() +{ + branch=$1 + upstream="master" + + # Thanks to pre-rc1 merging, we have a 2 week window with an + # incorrect version number. Detect and correct. + mergebase="$(git merge-base "$branch" "$upstream")" + raw_version="$(git describe "$mergebase")" + version="$(git show "$branch":Makefile | sed -ne '/^VERSION\ =\ /{s///;p;q}')" + patchlevel="$(git show "$branch":Makefile | sed -ne '/^PATCHLEVEL\ =\ /{s///;p;q}')" + kver="${version}.${patchlevel}" + + #-rc indicates no tricks necessary, return version + if echo "${raw_version}" | grep -q -- "-rc"; then + echo "$kver" + return + fi + + #if -gXXX is _not_ there, must be a GA release, use version + if ! echo "${raw_version}" | grep -q -- "-g"; then + echo "$kver" + return + fi + + #must be a post tag release with -g but not -rcX, IOW an rc0. + #Add a 1 to the version number + echo "${version}.$((patchlevel + 1))" +} + +# To handle missing branches, precalculate previous kernel versions to fetch +get_prev_version() +{ + version_str=$1 + + version="$(echo "$version_str" | cut -c1)" + patchlevel="$(echo "$version_str" | cut -c3)" + + echo "${version}.$((patchlevel - 1))" +} + +OS_BUILD_VER="$(get_upstream_version os-build)" +OS_BUILD_VER_prev="$(get_prev_version "$OS_BUILD_VER")" + +# check latest upstream RT branch +if git fetch -q "$UPSTREAM_RT_TREE_NAME" "linux-${OS_BUILD_VER}.y-rt"; then + UPSTREAM_RT_DEVEL_VER="${OS_BUILD_VER}" +elif git fetch -q "$UPSTREAM_RT_TREE_NAME" "linux-${OS_BUILD_VER_prev}.y-rt"; then + UPSTREAM_RT_DEVEL_VER="${OS_BUILD_VER_prev}" +else + die "Neither version ($OS_BUILD_VER, $OS_BUILD_VER_prev) in upstream tree: $UPSTREAM_RT_TREE_NAME" +fi + +# verify the core branches exist or use provided defaults +UPSTREAM_RT_DEVEL_BRANCH="linux-${UPSTREAM_RT_DEVEL_VER}.y-rt" +ark_git_branch "$DOWNSTREAM_RT_BRANCH" "$UPSTREAM_RT_TREE_NAME/$UPSTREAM_RT_DEVEL_BRANCH" +ark_git_branch "$RT_AUTOMATED_BRANCH" "$UPSTREAM_RT_TREE_NAME/$UPSTREAM_RT_DEVEL_BRANCH" +ark_git_branch "$RT_DEVEL_BRANCH" "$UPSTREAM_RT_TREE_NAME/$UPSTREAM_RT_DEVEL_BRANCH" +ark_git_branch "$AUTOMOTIVE_DEVEL_BRANCH" "$UPSTREAM_RT_TREE_NAME/$UPSTREAM_RT_DEVEL_BRANCH" + +MASTER_RT_DEVEL_VER="$(get_upstream_version "$DOWNSTREAM_RT_BRANCH")" +RT_AUTOMATED_VER="$(get_upstream_version $RT_AUTOMATED_BRANCH)" +RT_DEVEL_VER="$(get_upstream_version $RT_DEVEL_BRANCH)" +AUTOMOTIVE_DEVEL_VER="$(get_upstream_version $AUTOMOTIVE_DEVEL_BRANCH)" + +OS_BUILD_BASE_BRANCH="os-build" +RT_REBASE="" + +if test "$UPSTREAM_RT_DEVEL_VER" != "$OS_BUILD_VER"; then + # no newer upstream branch to rebase onto, continue with an + # os-build stable tag + OS_BUILD_BASE_BRANCH="kernel-${MASTER_RT_DEVEL_VER}.0-0" +fi + +# sanity check, sometimes broken scripts leave a mess +if test "$MASTER_RT_DEVEL_VER" != "$UPSTREAM_RT_DEVEL_VER" -o \ + "$MASTER_RT_DEVEL_VER" != "$RT_AUTOMATED_VER" -o \ + "$MASTER_RT_DEVEL_VER" != "$RT_DEVEL_VER" -o \ + "$MASTER_RT_DEVEL_VER" != "$AUTOMOTIVE_DEVEL_VER"; then + # rebase time + RT_REBASE="yes" +fi + +## PREP the upstream branches +# on a rebase, propogate all the git resets +# fetch the determined rt-devel branch +ark_git_mirror "$DOWNSTREAM_RT_BRANCH" "$UPSTREAM_RT_TREE_NAME" "$UPSTREAM_RT_DEVEL_BRANCH" "$RT_REBASE" +# finally merge the two correct branches +ark_git_merge "$OS_BUILD_BASE_BRANCH" "$RT_AUTOMATED_BRANCH" "$RT_REBASE" +ark_git_merge "$DOWNSTREAM_RT_BRANCH" "$RT_AUTOMATED_BRANCH" + +## MERGE the upstream branches to the development branches +if test -n "$RT_REBASE"; then + # handle the rebase + # rebases usually go from prev version to new version + # rebuild the prev merge base in case the previous automated one is + # corrupted. + prev_branch="$(git rev-parse --abbrev-ref HEAD)" + temp_branch="_temp_rt_devel_$(date +%F)" + git branch -D "$temp_branch" 2>/dev/null + git checkout -b "$temp_branch" "kernel-${OS_BUILD_VER_prev}.0-0" + git merge "$UPSTREAM_RT_TREE_NAME/linux-${OS_BUILD_VER_prev}.y-rt" + git checkout "$prev_branch" + ark_git_rebase "$RT_DEVEL_BRANCH" "$temp_branch" "$RT_AUTOMATED_BRANCH" + ark_git_rebase "$AUTOMOTIVE_DEVEL_BRANCH" "$temp_branch" "$RT_AUTOMATED_BRANCH" + git branch -D "$temp_branch" +fi + +## Build -rt-devel branch, generate pending-rhel configs +ark_git_merge "$RT_AUTOMATED_BRANCH" "$RT_DEVEL_BRANCH" +# don't care if configs were added or not hence '|| true' +ark_update_configs "$RT_DEVEL_BRANCH" || true +# skip pushing config update MRs, keep them in pending-rhel +ark_push_changes "$RT_DEVEL_BRANCH" "skip" + +## Build -automotive-devel branch, generate pending-rhel configs +ark_git_merge "$RT_AUTOMATED_BRANCH" "$AUTOMOTIVE_DEVEL_BRANCH" +# don't care if configs were added or not hence '|| true' +ark_update_configs "$AUTOMOTIVE_DEVEL_BRANCH" || true +# skip pushing config update MRs, keep them in pending-rhel +ark_push_changes "$AUTOMOTIVE_DEVEL_BRANCH" "skip" -- https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2732

2 3

✅ PASS (MISSED 13 of 118): Test report for 6.7.2-200.fc39 (kernel-fedora)
by cki-project＠redhat.com 28 Jan '24

28 Jan '24

Hi, we tested your kernel and here are the results: Overall result: PASSED Merge: OK Compile: OK Test: OK Tested-by: CKI Project <cki-project(a)redhat.com> Kernel information: Brew / Koji Task ID: 112403029 You can find all the details about the test run at https://datawarehouse.cki-project.org/kcidb/checkouts/125969 Tests that were not ran because of internal issues: x86_64 - CKI/restraint x86_64 - SELinux Custom Module Setup x86_64 - machineinfo x86_64 - Boot test x86_64 - Hardware - CPU: Die Test x86_64 - Hardware - IPMI driver test x86_64 - Hardware - IPMItool loop stress test x86_64 - Storage - swraid scsi_raid x86_64 - stress: stress-ng - interrupt x86_64 - stress: stress-ng - cpu x86_64 - stress: stress-ng - cpu-cache x86_64 - stress: stress-ng - memory x86_64 - Reboot test If you find a failure unrelated to your changes, please ask the test maintainer to review it. This will prevent the failures from being incorrectly reported in the future. Please reply to this email if you have any questions about the tests that we ran or if you have any suggestions on how to make future tests more effective. ,-. ,-. ( C ) ( K ) Continuous `-',-.`-' Kernel ( I ) Integration `-' ______________________________________________________________________________

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

kernel January 2024