Hi everyone,
Please find attached the first version of the CI patch proposed for
submission.
Changes from the RFC version are numerous, so I'll just list some:
* added RHEL6 host support,
* added mock builds for fedora20 and epel6,
* added moderate test set,
* removed test code coverage from coverage data,
* removed unnecessary configure options,
* enabled optimization for DEBUG build.
I'm not sure if distcheck and mock builds should be in the moderate set and
static analyzer and coverage in the full set. I would swap them, personally.
See output examples of the full test set executing on CI servers:
Debian Testing
http://sssd-ci.idm.lab.eng.brq.redhat.com:8080/job/new_private_master_deb...
Fedora 20
http://sssd-ci.idm.lab.eng.brq.redhat.com:8080/job/new_private_master_fed...
RHEL6
http://sssd-ci.idm.lab.eng.brq.redhat.com:8080/job/new_private_master_rhe...
RHEL7
http://sssd-ci.idm.lab.eng.brq.redhat.com:8080/job/new_private_master_rhe...
Unfortunately, I won't be able to complete support for selecting which sets to
execute and which commits (all or the last) to verify when submitting to CI,
before going to my vacation on Friday. I will only be able to discuss this
patch to a limited extent before that as well.
I will pick up the work when I return on the 14th of July.
Nick
From 4d02183e74eaed5c8a64752a231348cc2af67bf5 Mon Sep 17 00:00:00
2001
From: Nikolai Kondrashov <Nikolai.Kondrashov(a)redhat.com>
Date: Tue, 25 Mar 2014 12:01:00 +0200
Subject: [PATCH 1/1] Add basic support for CI test execution
Add basic support for executing continuous integration (CI) tests on
RHEL6, RHEL7, Fedora 20 and Debian Testing.
This adds two front-end scripts which can be executed either locally by
developers, or on the CI server: contrib/ci/run and contrib/ci/clean.
The first one will run the tests and the second will wipe out the
artifacts. Note that contrib/ci/run will itself clean before running.
The contrib/ci/run script accepts options to choose from three test
sets: essential, moderate and full (-e/-m/-f), with the essential set
selected by default.
Essential tests include building everything and running tests under
Valgrind, completing in about 2.5 minutes. Valgrind failures are ignored
for now.
Moderate tests include essential tests, plus a distcheck target build
and mock builds for fedora20 and epel6, on distros where mock exists.
They complete in about 15 minutes.
Full test set includes the moderate set, plus a pass with Clang static
analyzer over the whole build and test execution with code coverage
collection, completing in about 20 minutes. Static analyzer failures are
ignored for now.
The first stage (install-deps) would invoke sudo to install dependencies
using the appropriate package manager. A relatively strict rule can be
added to sudoers to avoid password prompts. In case the user is not in
"mock" group when mock is executed, sudo will be invoked as well.
---
Makefile.am | 10 +-
contrib/ci/clean | 25 ++++
contrib/ci/configure.sh | 43 ++++++
contrib/ci/deps.sh | 157 +++++++++++++++++++++
contrib/ci/distro.sh | 72 ++++++++++
contrib/ci/make-check-wrap | 52 +++++++
contrib/ci/misc.sh | 116 ++++++++++++++++
contrib/ci/run | 322 +++++++++++++++++++++++++++++++++++++++++++
contrib/ci/valgrind-condense | 128 +++++++++++++++++
9 files changed, 924 insertions(+), 1 deletion(-)
create mode 100755 contrib/ci/clean
create mode 100644 contrib/ci/configure.sh
create mode 100644 contrib/ci/deps.sh
create mode 100644 contrib/ci/distro.sh
create mode 100755 contrib/ci/make-check-wrap
create mode 100644 contrib/ci/misc.sh
create mode 100755 contrib/ci/run
create mode 100755 contrib/ci/valgrind-condense
diff --git a/Makefile.am b/Makefile.am
index 83999f3..b903ea1 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -301,6 +301,10 @@ dist_noinst_SCRIPTS = \
contrib/rhel/update_debug_levels.py \
contrib/fedora/bashrc_sssd \
contrib/fedora/make_srpm.sh \
+ contrib/ci/clean \
+ contrib/ci/make-check-wrap \
+ contrib/ci/run \
+ contrib/ci/valgrind-condense \
src/tests/pyhbac-test.py \
src/tests/pysss_murmur-test.py
@@ -311,7 +315,11 @@ dist_noinst_DATA = \
src/config/testconfigs/sssd-badversion.conf \
src/config/testconfigs/sssd-invalid.conf \
src/config/testconfigs/sssd-invalid-badbool.conf \
- src/config/etc/sssd.api.d/crash_test_dummy
+ src/config/etc/sssd.api.d/crash_test_dummy \
+ contrib/ci/configure.sh \
+ contrib/ci/deps.sh \
+ contrib/ci/distro.sh \
+ contrib/ci/misc.sh
###############################
# Global compilation settings #
diff --git a/contrib/ci/clean b/contrib/ci/clean
new file mode 100755
index 0000000..ee18c10
--- /dev/null
+++ b/contrib/ci/clean
@@ -0,0 +1,25 @@
+#!/bin/bash
+#
+# Clean source tree after a run of integration tests.
+#
+# Copyright (C) 2014 Red Hat
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <
http://www.gnu.org/licenses/>.
+
+set -o nounset -o pipefail -o errexit
+export PATH=`dirname "\`readlink -f \"\$0\"\`"`:$PATH
+
+. misc.sh
+
+rm_rf_ro ci-*
diff --git a/contrib/ci/configure.sh b/contrib/ci/configure.sh
new file mode 100644
index 0000000..8af2730
--- /dev/null
+++ b/contrib/ci/configure.sh
@@ -0,0 +1,43 @@
+#
+# Configure argument management.
+#
+# Copyright (C) 2014 Red Hat
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <
http://www.gnu.org/licenses/>.
+
+if [ -z ${_CONFIGURE_SH+set} ]; then
+declare -r _CONFIGURE_SH=
+
+. distro.sh
+
+# List of "configure" arguments.
+declare -a CONFIGURE_ARG_LIST=(
+ "--disable-dependency-tracking"
+ "--disable-rpath"
+ "--disable-static"
+ "--enable-ldb-version-check"
+ "--with-syslog=journald"
+)
+
+
+if [[ "$DISTRO_BRANCH" == -redhat-redhatenterprise*-6.*- ]]; then
+ CONFIGURE_ARG_LIST+=(
+ "--disable-cifs-idmap-plugin"
+ "--with-syslog=syslog"
+ )
+fi
+
+declare -r -a CONFIGURE_ARG_LIST
+
+fi # _CONFIGURE_SH
diff --git a/contrib/ci/deps.sh b/contrib/ci/deps.sh
new file mode 100644
index 0000000..3761335
--- /dev/null
+++ b/contrib/ci/deps.sh
@@ -0,0 +1,157 @@
+#
+# Dependency management.
+#
+# Copyright (C) 2014 Red Hat
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <
http://www.gnu.org/licenses/>.
+
+if [ -z ${_DEPS_SH+set} ]; then
+declare -r _DEPS_SH=
+
+. distro.sh
+
+# Dependency list
+declare -a DEPS_LIST=(
+ autoconf
+ automake
+ lcov
+ libtool
+ valgrind
+ xqilla
+)
+
+# True, if all test dependencies are satisfied by the package list
+declare DEPS_TESTS_SATISFIED=true
+
+if [[ "$DISTRO_BRANCH" == -redhat-* ]]; then
+ DEPS_LIST+=(
+ augeas-devel
+ bind-utils
+ c-ares-devel
+ check-devel
+ clang-analyzer
+ dbus-devel
+ dbus-libs
+ docbook-style-xsl
+ doxygen
+ findutils
+ gettext-devel
+ glib2-devel
+ keyutils-libs-devel
+ krb5-devel
+ libcollection-devel
+ libdhash-devel
+ libini_config-devel
+ libldb-devel
+ libselinux-devel
+ libsemanage-devel
+ libsmbclient-devel
+ libtalloc-devel
+ libtdb-devel
+ libtevent-devel
+ libxml2
+ libxslt
+ mock
+ nspr-devel
+ nss-devel
+ openldap-devel
+ pam-devel
+ pcre-devel
+ pkgconfig
+ popt-devel
+ python-devel
+ rpm-build
+ samba4-devel
+ selinux-policy-targeted
+ )
+ if [[ "$DISTRO_BRANCH" == *-redhatenterprise*-6.*- ]]; then
+ DEPS_LIST+=(
+ libnl-devel
+ )
+ DEPS_TESTS_SATISFIED=false
+ else
+ DEPS_LIST+=(
+ cifs-utils-devel
+ libcmocka-devel
+ libnl3-devel
+ systemd-devel
+ )
+ fi
+fi
Sorry, it looks terible.
If you want to maintain dependencies yourself in CI script. Feel free to do in
extra git repository "sssd CI" on github.
If you want to have script in upstream sssd repo you will need to ger rid of
this duplication.
NACK
LS