1:55 p.m.
From: dzickusrh on gitlab.com
I am trying to modify the workflow to use a single branch for
development as opposed to 'os-build' and 'ark-patches'. The first step
I want to take is to split apart the merge-upstream step and release
step. This makes it easier to convert the merge-upstream step to a
single branch later.
This patchset prepares the scripts to separate the steps by making the
scripts less dependent on 'os-build' and more robust for repeated runs.
And importantly adds a Makefile command that the gitlab-ci.yml can use
too.
The next step is to update the kernel-ark-ci tree with gitlab-ci.yml
changes to utilize these changes. The third step is to actually split
them apart. Finally updating the release steps.
1:55 p.m.
New subject: [OS-BUILD PATCH 2/3] Add master merge check
From: Don Zickus <dzickus(a)redhat.com>
Avoid constantly running the config update scripts when the merge to
master did not update anything. This means there can be no new
configs to generate.
Same argument for ark-patches.
The only downside I see with this check is if the generate config
scripts fail and this needs to be re-run. But we should be able
to do that by hand.
Signed-off-by: Don Zickus <dzickus(a)redhat.com>
---
redhat/scripts/ci/ark-rebase-patches.sh | 3 +++
redhat/scripts/ci/ark-update-configs.sh | 5 +++++
2 files changed, 8 insertions(+)
diff --git a/redhat/scripts/ci/ark-rebase-patches.sh
b/redhat/scripts/ci/ark-rebase-patches.sh
index 2ac22bcba3bd..561a927d5946 100755
--- a/redhat/scripts/ci/ark-rebase-patches.sh
+++ b/redhat/scripts/ci/ark-rebase-patches.sh
@@ -46,6 +46,9 @@ else
GITLAB_CONFIG_OPT="-c $PYTHON_GITLAB_CONFIG"
fi
+# Check if ark-patches is already rebased to $UPSTREAM_REF
+test -n "$(git branch os-build --contains "$UPSTREAM_REF")"
&& exit 0
+
if git show "$UPSTREAM_REF" > /dev/null 2>&1; then
printf "Rebasing ark-patches onto %s...\n" "$UPSTREAM_REF"
else
diff --git a/redhat/scripts/ci/ark-update-configs.sh
b/redhat/scripts/ci/ark-update-configs.sh
index e0945d5ffbb4..5fb13eaf60d6 100755
--- a/redhat/scripts/ci/ark-update-configs.sh
+++ b/redhat/scripts/ci/ark-update-configs.sh
@@ -33,6 +33,7 @@ To resolve this, do the following:
git checkout os-build
BRANCH="$(git branch --show-current)"
+HEAD="$(git describe)"
if ! git merge -m "Merge '$UPSTREAM_REF' into '$BRANCH'"
"$UPSTREAM_REF"; then
git merge --abort
printf "Merge conflict; halting!\n"
@@ -47,6 +48,10 @@ if ! git merge -m "Merge '$UPSTREAM_REF' into
'$BRANCH'" "$UPSTREAM_REF"; then
fi
exit 1
fi
+NEW_HEAD="$(git describe)"
+
+# Merge is already up to date, nothing to do.
+test "$HEAD" != "$NEW_HEAD" || exit 0
# Generates and commits all the pending configs
make FLAVOR=fedora dist-configs-commit
--
GitLab
1:55 p.m.
New subject: [OS-BUILD PATCH 3/3] Add ability to sync upstream through Makefile
From: Don Zickus <dzickus(a)redhat.com>
Currently all the magic to sync upstream with os-build and ark-patches
is done through scripts in redhat/scripts/ci and .gitlab-ci.yml.
Make this easier by enabling this routine through a redhat/Makefile.
This allows:
* gitlab-ci.yml and maintainer to use same script
* allows maintainer to run local test runs before committing
* developers to hack on things a little more easily
This change also adds the ability to 'git push' the changes. This is
safe because os-build and ark-patches are considered 'protected'
branches meaning only the developers in the 'Maintainers' group can
push to those branches (which is a tiny group right now).
Because I can't atomically commit this change and the change to
kernel-ark-ci's gitlab-ci.yml file, I need this change to go in first.
Then I can modify the gitlab-ci.yml file.
Functionally, this change should be no different than what currently
runs today.
---
redhat/Makefile | 22 ++++++++++++++++++++
redhat/scripts/ci/ark-rebase-patches.sh | 16 ++++++++++++---
redhat/scripts/ci/ark-update-configs.sh | 27 ++++++++++++++++++++++++-
3 files changed, 61 insertions(+), 4 deletions(-)
diff --git a/redhat/Makefile b/redhat/Makefile
index 7cdf9e3bd0bf..61dcc522fbc6 100644
--- a/redhat/Makefile
+++ b/redhat/Makefile
@@ -319,6 +319,28 @@ dist-release: dist-clean-sources
dist-release-tag:
@git tag -a -m "$(PACKAGE_NAME)-$(STAMP_VERSION)-$(PKGRELEASE)"
$(PACKAGE_NAME)-$(STAMP_VERSION)-$(PKGRELEASE)
+DIST_BRANCH ?= "os-build"
+dist-merge-upstream:
+ @if test -n "$(DIST_PUSH)" && test -z "$(shell git remote get-url
gitlab 2>/dev/null)"; then \
+ echo -e "Please run 'git remote add gitlab <url>' to enable
git-push.\n"; \
+ exit 1; \
+ fi
+ @if test "$(shell git branch --show-current)" != "$(DIST_BRANCH)";
then \
+ echo -e "Please checkout $(DIST_BRANCH) branch before merging.\n"; \
+ exit 1; \
+ fi;
+ @(shell git --no-optional-locks status -uno --porcelain 2>/dev/null || \
+ { echo -e "Dirty tree, please clean before merging.\n"; exit 1; })
+
+ @# If TAG is empty, script defaults to master:HEAD
+ @$(REDHAT)/scripts/ci/ark-rebase-patches.sh $(TAG) || exit 1
+ @$(REDHAT)/scripts/ci/ark-update-configs.sh $(TAG)
+
+dist-merge-upstream-push: DIST_PUSH="1"
+dist-merge-upstream-push: dist-merge-upstream
+
+
+
.PHONY: dist-brew dist-koji
dist-brew : BUILD_FLAGS ?= $(BREW_FLAGS) $(TEST_FLAGS)
dist-koji : BUILD_FLAGS ?= $(KOJI_FLAGS) $(TEST_FLAGS)
diff --git a/redhat/scripts/ci/ark-rebase-patches.sh
b/redhat/scripts/ci/ark-rebase-patches.sh
index 561a927d5946..ef2183ecc799 100755
--- a/redhat/scripts/ci/ark-rebase-patches.sh
+++ b/redhat/scripts/ci/ark-rebase-patches.sh
@@ -21,7 +21,7 @@
set -e
UPSTREAM_REF=${1:-master}
-PROJECT_ID=${2:-13604247}
+test -n "$PROJECT_ID" || PROJECT_ID="${2:-13604247}"
ISSUE_TEMPLATE="During an automated rebase of ark-patches, commit %s failed to
rebase.
@@ -98,7 +98,12 @@ else
fi
if $CLEAN_REBASE; then
- printf "You can safely update ark-patches with 'git push -f <remote>
ark-patches'\n"
+ if test -n "$DIST_PUSH"; then
+ echo "Pushing branch ark-patches to $(get remote get-url gitlab)"
+ git push -f gitlab ark-patches
+ else
+ printf "You can safely update ark-patches with 'git push -f <remote>
ark-patches'\n"
+ fi
else
printf "Some patches could not be rebased, fix up ark-patches as necessary"
printf " before pushing the branch."
@@ -108,5 +113,10 @@ fi
if git tag -v "$UPSTREAM_REF" > /dev/null 2>&1; then
printf "Creating branch \"ark/patches/%s\"\n"
"$UPSTREAM_REF"
git branch ark/patches/"$UPSTREAM_REF"
- printf "Don't forget to run 'git push <remote>
ark/patches/%s'\n" "$UPSTREAM_REF"
+ if test -n "$DIST_PUSH"; then
+ echo "Pushing branch ark/patches to $(get remote get-url gitlab)"
+ git push gitlab ark/patches/"$UPSTREAM_REF"
+ else
+ printf "Don't forget to run 'git push <remote>
ark/patches/%s'\n" "$UPSTREAM_REF"
+ fi
fi
diff --git a/redhat/scripts/ci/ark-update-configs.sh
b/redhat/scripts/ci/ark-update-configs.sh
index 5fb13eaf60d6..22dcead23f88 100755
--- a/redhat/scripts/ci/ark-update-configs.sh
+++ b/redhat/scripts/ci/ark-update-configs.sh
@@ -19,7 +19,7 @@
set -e
UPSTREAM_REF=${1:-master}
-PROJECT_ID=${2:-13604247}
+test -n "$PROJECT_ID" || PROJECT_ID="${2:-13604247}"
ISSUE_DESCRIPTION="A merge conflict has occurred and must be resolved manually.
@@ -65,3 +65,28 @@ if git show -s --oneline HEAD | grep -q "AUTOMATIC: New
configs"; then
else
printf "No new configuration values exposed from merging %s into $BRANCH\n"
"$UPSTREAM_REF"
fi
+
+if test -n "$DIST_PUSH"; then
+ echo "Pushing branch $(git branch --show-current) to $(get remote get-url
gitlab)"
+ git push gitlab HEAD
+
+ echo "Pushing config update branches"
+ for branch in $(git branch | grep configs/"$(date +%F)"); do
+ git push \
+ -o merge_request.create \
+ -o merge_request.target="$BRANCH" \
+ -o merge_request.remove_source_branch \
+ gitlab "$branch"
+ done
+else
+ printf "
+To push all the release artifacts, run:
+
+git push gitlab HEAD
+for branch in \$(git branch | grep configs/\"\$(date +%%F)\"); do
+\tgit push -o merge_request.create -o merge_request.target=$BRANCH\
+ -o merge_request.remove_source_branch upstream gitlab \"\$branch\"
+done\n"
+
+fi
+
--
GitLab
1:55 p.m.
New subject: [OS-BUILD PATCH 1/3] Replace hardcoded values 'os-build' and project
id with
variables
From: Don Zickus <dzickus(a)redhat.com>
As a step towards switching away from the os-build branch start by
using more variables.
The patch does 2 things:
* adds a use of PROJECT_ID
* replace os-build with BRANCH in the update configs script
Signed-off-by: Don Zickus <dzickus(a)redhat.com>
---
redhat/scripts/ci/ark-create-release.sh | 2 +-
redhat/scripts/ci/ark-update-configs.sh | 17 +++++++++++------
2 files changed, 12 insertions(+), 7 deletions(-)
diff --git a/redhat/scripts/ci/ark-create-release.sh
b/redhat/scripts/ci/ark-create-release.sh
index 8b3b414be953..df501b0cce56 100755
--- a/redhat/scripts/ci/ark-create-release.sh
+++ b/redhat/scripts/ci/ark-create-release.sh
@@ -47,7 +47,7 @@ else
fi
git merge -m "Merge configuration and build scripts" os-build
-MR_PATCHES=$(gitlab project-merge-request list --project-id=13604247 \
+MR_PATCHES=$(gitlab project-merge-request list --project-id=${PROJECT_ID} \
--labels="Include in Releases" --state=opened | grep -v "^$" | sort
| \
awk '{ print "https://gitlab.com/cki-project/kernel-ark/-/merge_requests/"
$2 ".patch" }')
for patch_url in $MR_PATCHES; do
diff --git a/redhat/scripts/ci/ark-update-configs.sh
b/redhat/scripts/ci/ark-update-configs.sh
index 1669abe1ae89..e0945d5ffbb4 100755
--- a/redhat/scripts/ci/ark-update-configs.sh
+++ b/redhat/scripts/ci/ark-update-configs.sh
@@ -1,15 +1,15 @@
#!/bin/bash
#
-# This script is intended to regularly update the os-build branch with the latest
+# This script is intended to regularly update the $BRANCH branch with the latest
# configuration options from upstream. It merges the given reference into
-# os-build, adds all new configuration symbols to the pending/ config directory,
+# $BRANCH, adds all new configuration symbols to the pending/ config directory,
# and creates a branch for each new config group.
#
# If the upstream branch fails to merge and the REPORT_BUGS environment variable
# is set, an issue is filed against the provided project ID.
#
# Arguments:
-# 1) The git object to merge into os-build. This should be something from
+# 1) The git object to merge into $BRANCH. This should be something from
# Linus's master branch, usually either a tag such as v5.5-rc3 or just
# linus/master. The default is "master".
# 2) The Gitlab project ID to file issues against. See the project page on
@@ -32,14 +32,15 @@ To resolve this, do the following:
"
git checkout os-build
-if ! git merge -m "Merge '$UPSTREAM_REF' into 'os-build'"
"$UPSTREAM_REF"; then
+BRANCH="$(git branch --show-current)"
+if ! git merge -m "Merge '$UPSTREAM_REF' into '$BRANCH'"
"$UPSTREAM_REF"; then
git merge --abort
printf "Merge conflict; halting!\n"
if [ -n "$REPORT_BUGS" ]; then
ISSUES=$(gitlab project-issue list --state "opened" --labels
"Configuration Update" --project-id "$PROJECT_ID")
if [ -z "$ISSUES" ]; then
gitlab project-issue create --project-id "$PROJECT_ID" \
- --title "Merge conflict between '$UPSTREAM_REF' and
'os-build'" \
+ --title "Merge conflict between '$UPSTREAM_REF' and
'$BRANCH'" \
--labels "Configuration Update" \
--description "$ISSUE_DESCRIPTION"
fi
@@ -47,11 +48,15 @@ if ! git merge -m "Merge '$UPSTREAM_REF' into
'os-build'" "$UPSTREAM_REF"; then
exit 1
fi
+# Generates and commits all the pending configs
make FLAVOR=fedora dist-configs-commit
make FLAVOR=rhel dist-configs-commit
+# Converts each new pending config from above into its finalized git
+# configs/<date>/<config> branch. These commits are used for Merge
+# Requests.
if git show -s --oneline HEAD | grep -q "AUTOMATIC: New configs"; then
./redhat/gen_config_patches.sh
else
- printf "No new configuration values exposed from merging %s into os-build\n"
"$UPSTREAM_REF"
+ printf "No new configuration values exposed from merging %s into $BRANCH\n"
"$UPSTREAM_REF"
fi
--
GitLab
7:17 a.m.
On 7/31/20 2:55 PM, GitLab Bridge on behalf of dzickusrh wrote:
From: dzickusrh on gitlab.com
I am trying to modify the workflow to use a single branch for
development as opposed to 'os-build' and 'ark-patches'. The first step
I want to take is to split apart the merge-upstream step and release
step. This makes it easier to convert the merge-upstream step to a
single branch later.
This patchset prepares the scripts to separate the steps by making the
scripts less dependent on 'os-build' and more robust for repeated runs.
And importantly adds a Makefile command that the gitlab-ci.yml can use
too.
The next step is to update the kernel-ark-ci tree with gitlab-ci.yml
changes to utilize these changes. The third step is to actually split
them apart. Finally updating the release steps.
Acked-by: Prarit Bhargava <prarit(a)redhat.com>
P.
9:18 a.m.
On Fri, Jul 31, 2020 at 06:55:11PM -0000, GitLab Bridge on behalf of dzickusrh wrote:
From: dzickusrh on gitlab.com
I am trying to modify the workflow to use a single branch for
development as opposed to 'os-build' and 'ark-patches'. The first step
I want to take is to split apart the merge-upstream step and release
step. This makes it easier to convert the merge-upstream step to a
single branch later.
This patchset prepares the scripts to separate the steps by making the
scripts less dependent on 'os-build' and more robust for repeated runs.
And importantly adds a Makefile command that the gitlab-ci.yml can use
too.
The next step is to update the kernel-ark-ci tree with gitlab-ci.yml
changes to utilize these changes. The third step is to actually split
them apart. Finally updating the release steps.
Acked-by: Herton R. Krzesinski <herton(a)redhat.com>
--
[]'s
Herton
1360
days inactive
1364
days old
kernel@lists.fedoraproject.org
5 comments
3 participants
participants (3)
-
GitLab Bridge on behalf of dzickusrh -
Herton R. Krzesinski -
Prarit Bhargava