From: Clark Williams on gitlab.com Merge Request: https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2117
Replace the (still functioning) merge.pl in redhat/configs with a python version (merge.py) for ease of maintenance.
Signed-off-by: Clark Williams williams@redhat.com
--- redhat/configs/build_configs.sh | 3 +- redhat/configs/merge.py | 57 +++++++++++++++++++++++++++++++++++++++++ redhat/kernel.spec.template | 4 +- 3 files changed, 61 insertions(+), 3 deletions(-)
From: Clark Williams williams@redhat.com
redhat: python replacement for merge.pl
Replace the (still functioning) merge.pl in redhat/configs with a python version (merge.py) for ease of maintenance.
Signed-off-by: Clark Williams williams@redhat.com
diff --git a/redhat/configs/build_configs.sh b/redhat/configs/build_configs.sh index blahblah..blahblah 100755 --- a/redhat/configs/build_configs.sh +++ b/redhat/configs/build_configs.sh @@ -84,7 +84,8 @@ function merge_configs()
test -n "$skip_if_missing" && test ! -e "$cfile" && continue
- if ! perl merge.pl "$cfile" config-merging."$count" > config-merged."$count"; then + #if ! perl merge.pl "$cfile" config-merging."$count" > config-merged."$count"; then + if ! python merge.py "$cfile" config-merging."$count" > config-merged."$count"; then die "Failed to merge $cfile" fi mv config-merged."$count" config-merging."$count" diff --git a/redhat/configs/merge.py b/redhat/configs/merge.py new file mode 100644 index blahblah..blahblah 100644 --- /dev/null +++ b/redhat/configs/merge.py @@ -0,0 +1,57 @@ +# SPDX-License-Identifier: GPL-2.0 +# Author: Clark Williams williams@redhat.com +# Copyright (C) 2022 Red Hat, Inc. +# +# merge.py - a direct replacement for merge.pl in the redhat/configs directory +# +# invocation: python merge.py overrides baseconfig +# +# Both input files are kernel config files, where overides is config overides +# to the baseconfig file. Both are read into python dictionaries with the +# keys being the config name and the values being the config file text + +# The script iterates through the overrides keys adding/replacing the contents +# of the baseconfig values and then outputs the new baseconfig to stdout. +# + +import sys +import os.path + +def usage(msg): + print(msg) + print("usage: merge.py overrides baseconfig") + sys.exit(1) + +# read a config file and return a dictionary of the contents +def read_config_file(cfgfile): + configs = {} + for l in open(cfgfile).readlines(): + if len(l) == 0: continue + if l.startswith("# CONFIG_"): + configs[l.split()[1]] = l + continue + if l.startswith("CONFIG_"): + configs[l.split('=')[0]] = l + continue + return configs + + +if len(sys.argv) < 3: usage("must have two input files") + +# read in the overides file +if not os.path.exists(sys.argv[1]): + usage("overrides config file %s does not exist!" % sys.argv[1]) +overrides = read_config_file(sys.argv[1]) + +# read in the base config file +if not os.path.exists(sys.argv[2]): + usage("base config file %s does not exist!" % sys.argv[2]) +baseconfigs = read_config_file(sys.argv[2]) + +# iterate over the overrides, replacing values in the base config +for v in overrides.keys(): + baseconfigs[v] = overrides[v] + +# print the new config to stdout +for v in baseconfigs.keys(): + print(baseconfigs[v])
-- https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2117
From: Clark Williams williams@redhat.com
redhat: automatically strip newlines in merge.py
When reading configfiles call strip() to remove newlines, so that we can easily find blank lines
Signed-off-by: Clark Williams williams@redhat.com
diff --git a/redhat/configs/merge.py b/redhat/configs/merge.py index blahblah..blahblah 100644 --- a/redhat/configs/merge.py +++ b/redhat/configs/merge.py @@ -25,7 +25,7 @@ def usage(msg): # read a config file and return a dictionary of the contents def read_config_file(cfgfile): configs = {} - for l in open(cfgfile).readlines(): + for l in [ n.strip() for n in open(cfgfile).readlines()]: if len(l) == 0: continue if l.startswith("# CONFIG_"): configs[l.split()[1]] = l
-- https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2117
From: Patrick Talbert ptalbert@redhat.com
redhat: Use a context manager in merge.py for opening the config file for reading
Signed-off-by: Clark Williams williams@redhat.com
diff --git a/redhat/configs/merge.py b/redhat/configs/merge.py index blahblah..blahblah 100644 --- a/redhat/configs/merge.py +++ b/redhat/configs/merge.py @@ -25,14 +25,15 @@ def usage(msg): # read a config file and return a dictionary of the contents def read_config_file(cfgfile): configs = {} - for l in [ n.strip() for n in open(cfgfile).readlines()]: - if len(l) == 0: continue - if l.startswith("# CONFIG_"): - configs[l.split()[1]] = l - continue - if l.startswith("CONFIG_"): - configs[l.split('=')[0]] = l - continue + with open(cfgfile) as f: + for l in [n.strip() for n in f.readlines()]: + if not l: continue + if l.startswith("# CONFIG_"): + configs[l.split()[1]] = l + continue + if l.startswith("CONFIG_"): + configs[l.split('=')[0]] = l + continue return configs
-- https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2117
From: Patrick Talbert ptalbert@redhat.com
redhat: use 'update' method in merge.py
Use the update method to update our baseconfigs dictionary rather than iterating
Signed-off-by: Clark Williams williams@redhat.com
diff --git a/redhat/configs/merge.py b/redhat/configs/merge.py index blahblah..blahblah 100644 --- a/redhat/configs/merge.py +++ b/redhat/configs/merge.py @@ -49,9 +49,8 @@ if not os.path.exists(sys.argv[2]): usage("base config file %s does not exist!" % sys.argv[2]) baseconfigs = read_config_file(sys.argv[2])
-# iterate over the overrides, replacing values in the base config -for v in overrides.keys(): - baseconfigs[v] = overrides[v] +# update baseconfigs with the overrides values +baseconfigs.update(overrides)
# print the new config to stdout for v in baseconfigs.keys():
-- https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2117
From: Patrick Talbert ptalbert@redhat.com
redhat: method.py: change the output loop to use 'values' method
We don't need the key values when printing the output file so just use the values method to fetch them
Signed-off-by: Clark Williams williams@redhat.com
diff --git a/redhat/configs/merge.py b/redhat/configs/merge.py index blahblah..blahblah 100644 --- a/redhat/configs/merge.py +++ b/redhat/configs/merge.py @@ -53,5 +53,5 @@ baseconfigs = read_config_file(sys.argv[2]) baseconfigs.update(overrides)
# print the new config to stdout -for v in baseconfigs.keys(): - print(baseconfigs[v]) +for v in baseconfigs.values(): + print(v)
-- https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2117
From: Clark Williams williams@redhat.com
redhat: update specfile to require python3 for builds
Signed-off-by: Clark Williams williams@redhat.com
diff --git a/redhat/kernel.spec.template b/redhat/kernel.spec.template index blahblah..blahblah 100755 --- a/redhat/kernel.spec.template +++ b/redhat/kernel.spec.template @@ -579,7 +579,7 @@ BuildRequires: bzip2, xz, findutils, gzip, m4, perl-interpreter, perl-Carp, perl BuildRequires: gcc, binutils, redhat-rpm-config, hmaccalc, bison, flex, gcc-c++ BuildRequires: net-tools, hostname, bc, elfutils-devel BuildRequires: dwarves -BuildRequires: python3-devel +BuildRequires: python3 python3-devel python-unversioned-command BuildRequires: gcc-plugin-devel BuildRequires: kernel-rpm-macros # glibc-static is required for a consistent build environment (specifically @@ -1038,7 +1038,7 @@ This package provides debug information for package kernel-tools. %{expand:%%global _find_debuginfo_opts %{?_find_debuginfo_opts} -p '.*%%{_bindir}/centrino-decode(.debug)?|.*%%{_bindir}/powernow-k8-decode(.debug)?|.*%%{_bindir}/cpupower(.debug)?|.*%%{_libdir}/libcpupower.*|.*%%{_bindir}/turbostat(.debug)?|.*%%{_bindir}/x86_energy_perf_policy(.debug)?|.*%%{_bindir}/tmon(.debug)?|.*%%{_bindir}/lsgpio(.debug)?|.*%%{_bindir}/gpio-hammer(.debug)?|.*%%{_bindir}/gpio-event-mon(.debug)?|.*%%{_bindir}/gpio-watch(.debug)?|.*%%{_bindir}/iio_event_monitor(.debug)?|.*%%{_bindir}/iio_generic_buffer(.debug)?|.*%%{_bindir}/lsiio(.debug)?|.*%%{_bindir}/intel-speed-select(.debug)?|.*%%{_bindir}/page_owner_sort(.debug)?|.*%%{_bindir}/slabinfo(.debug)?|.*%%{_sbindir}/intel_sdsi(.debug)?|XXX' -o kernel-tools-debuginfo.list}
%package -n rtla -Summary: RTLA: Real-Time Linux Analysis tools +Summary: RTLA: Real-Time Linux Analysis tools %description -n rtla The rtla tool is a meta-tool that includes a set of commands that aims to analyze the real-time properties of Linux. But, instead of
-- https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2117
From: Clark Williams on gitlab.com https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2117#note_1151608...
Added a commit that BuildRequires: python3 and python-unversioned-command packages. Hopefully that will allow the pipeline to run correctly
From: Herton R. Krzesinski on gitlab.com https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2117#note_1151619...
Wouldn't it be better to just do what we already do eg. for redhat/genlog.py (where it's called directly from redhat/genlog.sh)?
``` Add #!/usr/bin/python3 to the first line of merge.py make it executable call it directly from buildconfigs.sh ```
From: Clark Williams on gitlab.com https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2117#note_1151620...
Are we guaranteed to actually *have* python3 in the buildroot?
From: Herton R. Krzesinski on gitlab.com https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2117#note_1151621...
Yes, python2 doesn't exist anymore, and python3 is the default. At least it hasn't been a problem.
From: Clark Williams on gitlab.com https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2117#note_1151622...
Ok, I can remove the buildrequires and I'll change how we call it
From: Justin M. Forbes on gitlab.com https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2117#note_1151622...
`BuildRequires: python3-devel`
This is already present.
kernel@lists.fedoraproject.org