LNST-developers February 2015

lnst-developers@lists.fedorahosted.org

6 participants
33 discussions

[PATCH v2] NetTestCommand: signal handling for bg cmds in netns

by Ondrej Lichtner

From: Ondrej Lichtner <olichtne(a)redhat.com> Background commands in a network namespace inherited signal handlers from the lnst-slave process that manages that namespace, this resulted in wrong handling of the interrupt command for the bg command. This commit fixes these issues by changing the signal handlers after the bg cmd fork to raise a KeyboardInterrupt. It also changes the default SIGINT handler (which was deprecated for a long time) to also raise a KeyboardInterrupt. Because of this the wait_on_interrupt method was modified and the read/write was removed from the TestGeneric class. Finally the commit also adds a new regression test for this bug. The test consists of running two recipes that differ in the type of the background command. The first one is a 'run command' that runs an infinite ping and the seconf one is a 'run module' that runs a modified version of the IcmpPing module for an infinite ping. The modification is simple - run the ping twice, this tests if we interrupt the bg cmd process as well as the exec-ed command. Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> --- lnst/Common/NetTestCommand.py | 8 ++- lnst/Common/TestsCommon.py | 19 ++---- regression-tests/tests/25/desc | 12 ++++ regression-tests/tests/25/lnst-ctl.conf | 2 + regression-tests/tests/25/recipe1.xml | 44 +++++++++++++ regression-tests/tests/25/recipe2.xml | 49 ++++++++++++++ regression-tests/tests/25/run.sh | 21 ++++++ regression-tests/tests/25/test_modules/IcmpPing.py | 75 ++++++++++++++++++++++ 8 files changed, 217 insertions(+), 13 deletions(-) create mode 100644 regression-tests/tests/25/desc create mode 100644 regression-tests/tests/25/lnst-ctl.conf create mode 100644 regression-tests/tests/25/recipe1.xml create mode 100644 regression-tests/tests/25/recipe2.xml create mode 100755 regression-tests/tests/25/run.sh create mode 100644 regression-tests/tests/25/test_modules/IcmpPing.py diff --git a/lnst/Common/NetTestCommand.py b/lnst/Common/NetTestCommand.py index 04d6e86..0f7c2a3 100644 --- a/lnst/Common/NetTestCommand.py +++ b/lnst/Common/NetTestCommand.py @@ -137,8 +137,14 @@ class NetTestCommand: "res_header": self._cmd_cls._format_cmd_res_header(), "msg": "Running in background."} + def _sig_handler(self, signum, frame): + raise KeyboardInterrupt() + def _run(self): os.setpgrp() + signal.signal(signal.SIGHUP, self._sig_handler) + signal.signal(signal.SIGINT, self._sig_handler) + signal.signal(signal.SIGTERM, self._sig_handler) self._cmd_cls.set_handle_intr() self._connection_pipe = self._write_pipe @@ -149,7 +155,7 @@ class NetTestCommand: result = {} try: self._cmd_cls.run() - except KeyboardInterrupt: + except (KeyboardInterrupt, SystemExit): pass except: type, value, tb = sys.exc_info() diff --git a/lnst/Common/TestsCommon.py b/lnst/Common/TestsCommon.py index 8086121..2504ecf 100644 --- a/lnst/Common/TestsCommon.py +++ b/lnst/Common/TestsCommon.py @@ -15,6 +15,7 @@ import logging import sys import os import signal +import time from lnst.Common.NetTestCommand import NetTestCommandGeneric class testLogger(logging.Logger): @@ -56,13 +57,8 @@ class TestOptionMissing(Exception): class TestGeneric(NetTestCommandGeneric): def __init__(self, command): self._testLogger = logging.getLogger("root.testLogger") - self._read_pipe, self._write_pipe = os.pipe() NetTestCommandGeneric.__init__(self, command) - def __del__(self): - os.close(self._read_pipe) - os.close(self._write_pipe) - def set_handle_intr(self): """ Call this if you need this class to handle SIGINT. For backgroud @@ -71,18 +67,17 @@ class TestGeneric(NetTestCommandGeneric): signal.signal(signal.SIGINT, self._signal_intr_handler) def _signal_intr_handler(self, signum, frame): - os.write(self._write_pipe, "a") + raise KeyboardInterrupt() def wait_on_interrupt(self): ''' Should be used by test implementation for waiting on SIGINT ''' - while True: - try: - os.read(self._read_pipe, 1) - except OSError: - continue - break + try: + while True: + time.sleep(1) + except KeyboardInterrupt: + pass def _get_val(self, value, opt_type, default): if opt_type == "addr": diff --git a/regression-tests/tests/25/desc b/regression-tests/tests/25/desc new file mode 100644 index 0000000..fd7c6b2 --- /dev/null +++ b/regression-tests/tests/25/desc @@ -0,0 +1,12 @@ +proper signal handling for bg processes in net namespaces + +When running a background process in a new network namespace signal handlers +were inherited from the namespace process lnst-slave which caused issues when +calling the interrupt command on the background process. + +This test uses two recipes - one for run command and one for run test module. +It also uses a custom IcmpPing test module that runs the ping twice this way we +can test that the whole background process is interrupted, not just the +currently running ping process. + +https://github.com/jpirko/lnst/issues/99 diff --git a/regression-tests/tests/25/lnst-ctl.conf b/regression-tests/tests/25/lnst-ctl.conf new file mode 100644 index 0000000..a4d654d --- /dev/null +++ b/regression-tests/tests/25/lnst-ctl.conf @@ -0,0 +1,2 @@ +[environment] +test_module_dirs = ./test_modules diff --git a/regression-tests/tests/25/recipe1.xml b/regression-tests/tests/25/recipe1.xml new file mode 100644 index 0000000..8b9aee3 --- /dev/null +++ b/regression-tests/tests/25/recipe1.xml @@ -0,0 +1,44 @@ +<lnstrecipe> + <network> + <host id="tm1"> + <interfaces> + <eth id="phy1" label="net1"/> + <veth_pair> + <veth id="in" netns="in"> + <addresses> + <address value="192.168.0.3/24"/> + </addresses> + </veth> + <veth id="out"/> + </veth_pair> + <bridge id="br"> + <slaves> + <slave id="out"/> + <slave id="phy1"/> + </slaves> + <addresses> + <address value="192.168.0.1/24"/> + </addresses> + </bridge> + </interfaces> + </host> + <host id="tm2"> + <interfaces> + <eth id="phy1" label="net1"> + <addresses> + <address value="192.168.0.10/24"/> + </addresses> + </eth> + </interfaces> + </host> + </network> + <task> + <run command="ping -c 4 {ip(tm2,phy1)}" host="tm1" netns="in"/> + <run command="ping {ip(tm2,phy1)}" host="tm1" netns="in" bg_id="on_bg"/> + <ctl_wait seconds="2"/> + <run command="pstree -p" host="tm1" netns="in"/> + <ctl_wait seconds="2"/> + <intr host="tm1" bg_id="on_bg"/> + </task> +</lnstrecipe> + diff --git a/regression-tests/tests/25/recipe2.xml b/regression-tests/tests/25/recipe2.xml new file mode 100644 index 0000000..fd8b3f6 --- /dev/null +++ b/regression-tests/tests/25/recipe2.xml @@ -0,0 +1,49 @@ +<lnstrecipe> + <network> + <host id="tm1"> + <interfaces> + <eth id="phy1" label="net1"/> + <veth_pair> + <veth id="in" netns="in"> + <addresses> + <address value="192.168.0.3/24"/> + </addresses> + </veth> + <veth id="out"/> + </veth_pair> + <bridge id="br"> + <slaves> + <slave id="out"/> + <slave id="phy1"/> + </slaves> + <addresses> + <address value="192.168.0.1/24"/> + </addresses> + </bridge> + </interfaces> + </host> + <host id="tm2"> + <interfaces> + <eth id="phy1" label="net1"> + <addresses> + <address value="192.168.0.10/24"/> + </addresses> + </eth> + </interfaces> + </host> + </network> + <task> + <run command="ping -c 4 {ip(tm2,phy1)}" host="tm1" netns="in"/> + <run host="tm1" module="IcmpPing" bg_id="on_bg" netns="in"> + <options> + <option name="addr" value="{ip(tm2,phy1)}"/> + <option name="interval" value="1"/> + </options> + </run> + <ctl_wait seconds="2"/> + <run command="pstree -p" host="tm1" netns="in"/> + <ctl_wait seconds="2"/> + <intr host="tm1" bg_id="on_bg"/> + </task> +</lnstrecipe> + diff --git a/regression-tests/tests/25/run.sh b/regression-tests/tests/25/run.sh new file mode 100755 index 0000000..3efc33a --- /dev/null +++ b/regression-tests/tests/25/run.sh @@ -0,0 +1,21 @@ +#!/bin/bash + +. ../lib.sh + +init_test + +lnst-ctl -c lnst-ctl.conf -d run recipe1.xml | tee test.log +rv1=${PIPESTATUS[0]} +log1=`cat test.log` + +lnst-ctl -c lnst-ctl.conf -d run recipe2.xml | tee test.log +rv2=${PIPESTATUS[0]} +log2=`cat test.log` + +print_separator +assert_status "pass" "$rv1" +assert_status "pass" "$rv2" + +rm -f test.log + +end_test diff --git a/regression-tests/tests/25/test_modules/IcmpPing.py b/regression-tests/tests/25/test_modules/IcmpPing.py new file mode 100644 index 0000000..0469687 --- /dev/null +++ b/regression-tests/tests/25/test_modules/IcmpPing.py @@ -0,0 +1,75 @@ +""" +This module defines icmp ping test + +Copyright 2011 Red Hat, Inc. +Licensed under the GNU General Public License, version 2 as +published by the Free Software Foundation; see COPYING for details. +""" + +__author__ = """ +jpirko(a)redhat.com (Jiri Pirko) +""" + +import logging +import re +from lnst.Common.TestsCommon import TestGeneric +from lnst.Common.ExecCmd import exec_cmd + +class IcmpPing(TestGeneric): + def _compose_cmd(self): + addr = self.get_mopt("addr", opt_type="addr") + cmd = "ping %s" % addr + count = self.get_opt("count") + if count: + cmd += " -c %s" % count + interval = self.get_opt("interval") + if interval: + cmd += " -i %s" % interval + iface = self.get_opt("iface") + if iface: + cmd += " -I %s" % iface + size = self.get_opt("size") + if size: + cmd += " -s %s" % size + return cmd + + def run(self): + cmd = self._compose_cmd() + limit_rate = self.get_opt("limit_rate", default=80) + + #run twice to test that we interrupt the entire bg command not just the + #currently running exec_cmd + data_stdout = exec_cmd(cmd, die_on_err=False)[0] + data_stdout = exec_cmd(cmd, die_on_err=False)[0] + + stat_pttr1 = r'(\d+) packets transmitted, (\d+) received' + stat_pttr2 = r'rtt min/avg/max/mdev = (\d+\.\d+)/(\d+\.\d+)/(\d+\.\d+)/(\d+\.\d+) ms' + + match = re.search(stat_pttr1, data_stdout) + if not match: + res_data = {"msg": "expected pattern not found"} + return self.set_fail(res_data) + + trans_pkts, recv_pkts = match.groups() + rate = int(round((float(recv_pkts) / float(trans_pkts)) * 100)) + logging.debug("Transmitted \"%s\", received \"%s\", " + "rate \"%d%%\", limit_rate \"%d%%\"" + % (trans_pkts, recv_pkts, rate, limit_rate)) + + res_data = {"rate": rate, + "limit_rate": limit_rate} + + match = re.search(stat_pttr2, data_stdout) + if match: + tmin, tavg, tmax, tmdev = [float(x) for x in match.groups()] + logging.debug("rtt min \"%.3f\", avg \"%.3f\", max \"%.3f\", " + "mdev \"%.3f\"" % (tmin, tavg, tmax, tmdev)) + + res_data["rtt_min"] = tmin + res_data["rtt_max"] = tmax + + if rate < limit_rate: + res_data["msg"] = "rate is lower than limit" + return self.set_fail(res_data) + + return self.set_pass(res_data) -- 2.1.0

9 years, 2 months

2
1
0 / 0

[PATCH] NetTestCommand: signal handling for bg cmds in netns

by Ondrej Lichtner

From: Ondrej Lichtner <olichtne(a)redhat.com> Background commands in a network namespace inherited signal handlers from the lnst-slave process that manages that namespace, this resulted in wrong handling of the interrupt command for the bg command. This commit fixes these issues by changing the signal handlers after the bg cmd fork to raise a KeyboardInterrupt. It also changes the default SIGINT handler (which was deprecated for a long time) to also raise a KeyboardInterrupt. Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> --- lnst/Common/NetTestCommand.py | 8 ++- lnst/Common/TestsCommon.py | 2 +- regression-tests/tests/25/desc | 12 ++++ regression-tests/tests/25/lnst-ctl.conf | 2 + regression-tests/tests/25/recipe1.xml | 44 +++++++++++++ regression-tests/tests/25/recipe2.xml | 49 ++++++++++++++ regression-tests/tests/25/run.sh | 21 ++++++ regression-tests/tests/25/test_modules/IcmpPing.py | 75 ++++++++++++++++++++++ 8 files changed, 211 insertions(+), 2 deletions(-) create mode 100644 regression-tests/tests/25/desc create mode 100644 regression-tests/tests/25/lnst-ctl.conf create mode 100644 regression-tests/tests/25/recipe1.xml create mode 100644 regression-tests/tests/25/recipe2.xml create mode 100755 regression-tests/tests/25/run.sh create mode 100644 regression-tests/tests/25/test_modules/IcmpPing.py diff --git a/lnst/Common/NetTestCommand.py b/lnst/Common/NetTestCommand.py index 04d6e86..0f7c2a3 100644 --- a/lnst/Common/NetTestCommand.py +++ b/lnst/Common/NetTestCommand.py @@ -137,8 +137,14 @@ class NetTestCommand: "res_header": self._cmd_cls._format_cmd_res_header(), "msg": "Running in background."} + def _sig_handler(self, signum, frame): + raise KeyboardInterrupt() + def _run(self): os.setpgrp() + signal.signal(signal.SIGHUP, self._sig_handler) + signal.signal(signal.SIGINT, self._sig_handler) + signal.signal(signal.SIGTERM, self._sig_handler) self._cmd_cls.set_handle_intr() self._connection_pipe = self._write_pipe @@ -149,7 +155,7 @@ class NetTestCommand: result = {} try: self._cmd_cls.run() - except KeyboardInterrupt: + except (KeyboardInterrupt, SystemExit): pass except: type, value, tb = sys.exc_info() diff --git a/lnst/Common/TestsCommon.py b/lnst/Common/TestsCommon.py index 8086121..a7a089a 100644 --- a/lnst/Common/TestsCommon.py +++ b/lnst/Common/TestsCommon.py @@ -71,7 +71,7 @@ class TestGeneric(NetTestCommandGeneric): signal.signal(signal.SIGINT, self._signal_intr_handler) def _signal_intr_handler(self, signum, frame): - os.write(self._write_pipe, "a") + raise KeyboardInterrupt() def wait_on_interrupt(self): ''' diff --git a/regression-tests/tests/25/desc b/regression-tests/tests/25/desc new file mode 100644 index 0000000..fd7c6b2 --- /dev/null +++ b/regression-tests/tests/25/desc @@ -0,0 +1,12 @@ +proper signal handling for bg processes in net namespaces + +When running a background process in a new network namespace signal handlers +were inherited from the namespace process lnst-slave which caused issues when +calling the interrupt command on the background process. + +This test uses two recipes - one for run command and one for run test module. +It also uses a custom IcmpPing test module that runs the ping twice this way we +can test that the whole background process is interrupted, not just the +currently running ping process. + +https://github.com/jpirko/lnst/issues/99 diff --git a/regression-tests/tests/25/lnst-ctl.conf b/regression-tests/tests/25/lnst-ctl.conf new file mode 100644 index 0000000..a4d654d --- /dev/null +++ b/regression-tests/tests/25/lnst-ctl.conf @@ -0,0 +1,2 @@ +[environment] +test_module_dirs = ./test_modules diff --git a/regression-tests/tests/25/recipe1.xml b/regression-tests/tests/25/recipe1.xml new file mode 100644 index 0000000..8b9aee3 --- /dev/null +++ b/regression-tests/tests/25/recipe1.xml @@ -0,0 +1,44 @@ +<lnstrecipe> + <network> + <host id="tm1"> + <interfaces> + <eth id="phy1" label="net1"/> + <veth_pair> + <veth id="in" netns="in"> + <addresses> + <address value="192.168.0.3/24"/> + </addresses> + </veth> + <veth id="out"/> + </veth_pair> + <bridge id="br"> + <slaves> + <slave id="out"/> + <slave id="phy1"/> + </slaves> + <addresses> + <address value="192.168.0.1/24"/> + </addresses> + </bridge> + </interfaces> + </host> + <host id="tm2"> + <interfaces> + <eth id="phy1" label="net1"> + <addresses> + <address value="192.168.0.10/24"/> + </addresses> + </eth> + </interfaces> + </host> + </network> + <task> + <run command="ping -c 4 {ip(tm2,phy1)}" host="tm1" netns="in"/> + <run command="ping {ip(tm2,phy1)}" host="tm1" netns="in" bg_id="on_bg"/> + <ctl_wait seconds="2"/> + <run command="pstree -p" host="tm1" netns="in"/> + <ctl_wait seconds="2"/> + <intr host="tm1" bg_id="on_bg"/> + </task> +</lnstrecipe> + diff --git a/regression-tests/tests/25/recipe2.xml b/regression-tests/tests/25/recipe2.xml new file mode 100644 index 0000000..fd8b3f6 --- /dev/null +++ b/regression-tests/tests/25/recipe2.xml @@ -0,0 +1,49 @@ +<lnstrecipe> + <network> + <host id="tm1"> + <interfaces> + <eth id="phy1" label="net1"/> + <veth_pair> + <veth id="in" netns="in"> + <addresses> + <address value="192.168.0.3/24"/> + </addresses> + </veth> + <veth id="out"/> + </veth_pair> + <bridge id="br"> + <slaves> + <slave id="out"/> + <slave id="phy1"/> + </slaves> + <addresses> + <address value="192.168.0.1/24"/> + </addresses> + </bridge> + </interfaces> + </host> + <host id="tm2"> + <interfaces> + <eth id="phy1" label="net1"> + <addresses> + <address value="192.168.0.10/24"/> + </addresses> + </eth> + </interfaces> + </host> + </network> + <task> + <run command="ping -c 4 {ip(tm2,phy1)}" host="tm1" netns="in"/> + <run host="tm1" module="IcmpPing" bg_id="on_bg" netns="in"> + <options> + <option name="addr" value="{ip(tm2,phy1)}"/> + <option name="interval" value="1"/> + </options> + </run> + <ctl_wait seconds="2"/> + <run command="pstree -p" host="tm1" netns="in"/> + <ctl_wait seconds="2"/> + <intr host="tm1" bg_id="on_bg"/> + </task> +</lnstrecipe> + diff --git a/regression-tests/tests/25/run.sh b/regression-tests/tests/25/run.sh new file mode 100755 index 0000000..3efc33a --- /dev/null +++ b/regression-tests/tests/25/run.sh @@ -0,0 +1,21 @@ +#!/bin/bash + +. ../lib.sh + +init_test + +lnst-ctl -c lnst-ctl.conf -d run recipe1.xml | tee test.log +rv1=${PIPESTATUS[0]} +log1=`cat test.log` + +lnst-ctl -c lnst-ctl.conf -d run recipe2.xml | tee test.log +rv2=${PIPESTATUS[0]} +log2=`cat test.log` + +print_separator +assert_status "pass" "$rv1" +assert_status "pass" "$rv2" + +rm -f test.log + +end_test diff --git a/regression-tests/tests/25/test_modules/IcmpPing.py b/regression-tests/tests/25/test_modules/IcmpPing.py new file mode 100644 index 0000000..0469687 --- /dev/null +++ b/regression-tests/tests/25/test_modules/IcmpPing.py @@ -0,0 +1,75 @@ +""" +This module defines icmp ping test + +Copyright 2011 Red Hat, Inc. +Licensed under the GNU General Public License, version 2 as +published by the Free Software Foundation; see COPYING for details. +""" + +__author__ = """ +jpirko(a)redhat.com (Jiri Pirko) +""" + +import logging +import re +from lnst.Common.TestsCommon import TestGeneric +from lnst.Common.ExecCmd import exec_cmd + +class IcmpPing(TestGeneric): + def _compose_cmd(self): + addr = self.get_mopt("addr", opt_type="addr") + cmd = "ping %s" % addr + count = self.get_opt("count") + if count: + cmd += " -c %s" % count + interval = self.get_opt("interval") + if interval: + cmd += " -i %s" % interval + iface = self.get_opt("iface") + if iface: + cmd += " -I %s" % iface + size = self.get_opt("size") + if size: + cmd += " -s %s" % size + return cmd + + def run(self): + cmd = self._compose_cmd() + limit_rate = self.get_opt("limit_rate", default=80) + + #run twice to test that we interrupt the entire bg command not just the + #currently running exec_cmd + data_stdout = exec_cmd(cmd, die_on_err=False)[0] + data_stdout = exec_cmd(cmd, die_on_err=False)[0] + + stat_pttr1 = r'(\d+) packets transmitted, (\d+) received' + stat_pttr2 = r'rtt min/avg/max/mdev = (\d+\.\d+)/(\d+\.\d+)/(\d+\.\d+)/(\d+\.\d+) ms' + + match = re.search(stat_pttr1, data_stdout) + if not match: + res_data = {"msg": "expected pattern not found"} + return self.set_fail(res_data) + + trans_pkts, recv_pkts = match.groups() + rate = int(round((float(recv_pkts) / float(trans_pkts)) * 100)) + logging.debug("Transmitted \"%s\", received \"%s\", " + "rate \"%d%%\", limit_rate \"%d%%\"" + % (trans_pkts, recv_pkts, rate, limit_rate)) + + res_data = {"rate": rate, + "limit_rate": limit_rate} + + match = re.search(stat_pttr2, data_stdout) + if match: + tmin, tavg, tmax, tmdev = [float(x) for x in match.groups()] + logging.debug("rtt min \"%.3f\", avg \"%.3f\", max \"%.3f\", " + "mdev \"%.3f\"" % (tmin, tavg, tmax, tmdev)) + + res_data["rtt_min"] = tmin + res_data["rtt_max"] = tmax + + if rate < limit_rate: + res_data["msg"] = "rate is lower than limit" + return self.set_fail(res_data) + + return self.set_pass(res_data) -- 2.1.0

9 years, 2 months

1
1
0 / 0

help: lnst-slave run fail on RHEL 7

by 李俊良

Hi all, Today I install and run LNST on RHEL 7, but fail when start lnst-slave. Got error output below: [root@PRP212 SPECS]# lnst-slave -d Loading config file '/etc/lnst-slave.conf' 2015-02-06 22:17:19 (localhost) - INFO: Started Traceback (most recent call last): File "/usr/bin/lnst-slave", line 98, in <module> main() File "/usr/bin/lnst-slave", line 90, in main nettestslave = NetTestSlave(log_ctl) File "/usr/lib/python2.7/site-packages/lnst/Slave/NetTestSlave.py", line 736, in __init__ self._if_manager = InterfaceManager(self._server_handler) File "/usr/lib/python2.7/site-packages/lnst/Slave/InterfaceManager.py", line 41, in __init__ self.rescan_devices() File "/usr/lib/python2.7/site-packages/lnst/Slave/InterfaceManager.py", line 83, in rescan_devices devs = scan_netdevs() File "/usr/lib/python2.7/site-packages/lnst/Common/NetUtils.py", line 28, in scan_netdevs ipr = IPRoute() File "/usr/lib/python2.7/site-packages/pyroute2/netlink/client.py", line 41, in __init__ host, key, cert, ca, addr) File "/usr/lib/python2.7/site-packages/pyroute2/iocore/iocore.py", line 91, in __init__ key, cert, ca) File "/usr/lib/python2.7/site-packages/pyroute2/iocore/iocore.py", line 285, in connect addr=addr) File "/usr/lib/python2.7/site-packages/pyroute2/iocore/iocore.py", line 228, in command assert rsp['cmd'] == IPRCMD_ACK AssertionError Maybe system config errors or something else? BTW, I used rpm packages on f21 success but fail on RHEL 7.

9 years, 2 months

2
1
0 / 0

← Newer
1
2
3
4
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

LNST-developers February 2015