[DISCUSSION] python version requirement
by Ondrej Lichtner
Hi all,
since we've moved to python3 that is actively developed and versions
move between various long term/short term support cycles, we should also
adapt LNST to this cycle of updating which minimal version of python
LNST requires.
TL;DR: The main questions I'm asking are:
* how do we _implement_ a python version requirement?
* how do we _upgrade_/_migrate_ in the future?
* how do we _document_ a python version requirement?
* which version do we want to use _now_?
More context:
I think at the moment we have a "soft" requirement for python3.6. Soft
because we:
* probably haven't tested on anything older
* it's not explicitly configured/documented anywhere
At the same time, there are now at least two reasons to start thinking
about moving to python3.8:
* I remember Perry asking about the f-string feature introduced in 3.8
* while working with Adrian on the TRex refactoring I started thinking
about a feature for the lnst.Tests package that I've had in mind for a
while, which requires a 3.7 feature
* python3.8 is the current version on Fedora32, is available in RHEL8
(via dnf install python38), and python3.7 was skipped
The lnst.Tests feature I'm thinking of is "lazy" and "dynamic" loading
of BaseTestModule derived modules - for example at the moment, if a
Recipe imports any module from lnst.Tests (e.g. lnst.Tests.Ping), the
entire package is parsed and "loaded", which means that the python
environment will also parse and load lnst.Tests.TRex. This means that a
basic hello world recipe that simply calls Ping, will in some way
require load time dependencies of TRex.
The "lazy" and "dynamic" loading of test modules would ensure that when
a recipe calls:
from lnst.Tests import Ping
Only the Ping module will be parsed, loaded and imported, and nothing
else. And the dynamicity here could mean that we could be able to extend
test modules exported by the lnst.Tests package via the lnst-ctl config
file, for example for user/tester implemented test modules that are not
tracked in the main lnst repository.
I wrote a rough patch to experiment with this:
---
diff --git a/lnst/Tests/__init__.py b/lnst/Tests/__init__.py
index f7c6c90..a39b6f4 100644
--- a/lnst/Tests/__init__.py
+++ b/lnst/Tests/__init__.py
@@ -12,8 +12,26 @@
olichtne(a)redhat.com (Ondrej Lichtner)
"""
-from lnst.Tests.Ping import Ping
-from lnst.Tests.PacketAssert import PacketAssert
-from lnst.Tests.Iperf import IperfClient, IperfServer
+# from lnst.Tests.Ping import Ping
+# from lnst.Tests.PacketAssert import PacketAssert
+# from lnst.Tests.Iperf import IperfClient, IperfServer
+import importlib
+
+lazy_load_modules = {
+ "Ping": "lnst.Tests.Ping",
+ "PacketAssert": "lnst.Tests.PacketAssert",
+ "IperfClient": "lnst.Tests.Iperf",
+ "IperfServer": "lnst.Tests.Iperf",
+}
+
+
+def __getattr__(name):
+ if name not in lazy_load_modules:
+ raise ImportError("Cannot import {}".format(name))
+ mod = importlib.import_module(lazy_load_modules[name])
+ globals()[name] = getattr(mod, name)
+ return globals()[name]
+
+
+# #TODO add support for test classes from lnst-ctl.conf
-#TODO add support for test classes from lnst-ctl.conf
---
However this requires the ability to define __getattr__ for a module,
which is introduced as a python3.7 feature via PEP562 [0].
-Ondrej
[0] https://www.python.org/dev/peps/pep-0562/
3 years
[PATCH] Recipes.ENRT.VlansOverBondRecipe: add PerfReversibleFlowMixin
by Jan Tluka
The test has asymmetric endpoints for the performance test so it's
useful to extend the recipe with the PerfReversibleFlowMixin to change
the direction of the performance test if needed.
Signed-off-by: Jan Tluka <jtluka(a)redhat.com>
---
lnst/Recipes/ENRT/VlansOverBondRecipe.py | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/lnst/Recipes/ENRT/VlansOverBondRecipe.py b/lnst/Recipes/ENRT/VlansOverBondRecipe.py
index f703bcf9..d4831616 100644
--- a/lnst/Recipes/ENRT/VlansOverBondRecipe.py
+++ b/lnst/Recipes/ENRT/VlansOverBondRecipe.py
@@ -6,13 +6,15 @@ from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import (
OffloadSubConfigMixin)
from lnst.Recipes.ENRT.ConfigMixins.CommonHWSubConfigMixin import (
CommonHWSubConfigMixin)
+from lnst.Recipes.ENRT.ConfigMixins.PerfReversibleFlowMixin import (
+ PerfReversibleFlowMixin)
from lnst.Devices import VlanDevice
from lnst.Devices.VlanDevice import VlanDevice as Vlan
from lnst.Devices import BondDevice
from lnst.Recipes.ENRT.PingMixins import VlanPingEvaluatorMixin
from lnst.RecipeCommon.Ping.PingEndpoints import PingEndpoints
-class VlansOverBondRecipe(VlanPingEvaluatorMixin,
+class VlansOverBondRecipe(PerfReversibleFlowMixin, VlanPingEvaluatorMixin,
CommonHWSubConfigMixin, OffloadSubConfigMixin,
BaseEnrtRecipe):
"""
--
2.21.3
3 years, 8 months
[PATCH v2 1/7] Recipes.ENRT.ConfigMixins: remove iptables rules for sctp in OffloadSubConfigMixin
by Jan Tluka
It does not make sense to have this setup just for offloaded tests.
A follow up patch will add this to proper place.
Signed-off-by: Jan Tluka <jtluka(a)redhat.com>
---
.../ENRT/ConfigMixins/OffloadSubConfigMixin.py | 12 ------------
1 file changed, 12 deletions(-)
diff --git a/lnst/Recipes/ENRT/ConfigMixins/OffloadSubConfigMixin.py b/lnst/Recipes/ENRT/ConfigMixins/OffloadSubConfigMixin.py
index 989743a8..3250a6af 100644
--- a/lnst/Recipes/ENRT/ConfigMixins/OffloadSubConfigMixin.py
+++ b/lnst/Recipes/ENRT/ConfigMixins/OffloadSubConfigMixin.py
@@ -45,12 +45,6 @@ class OffloadSubConfigMixin(BaseSubConfigMixin):
ethtool_offload_string += " %s %s" % (name, value)
for nic in self.offload_nics:
- if "sctp_stream" in self.params.perf_tests:
- nic.netns.run(
- "iptables -I OUTPUT ! -o %s -p sctp -j DROP" % nic.name,
- job_level=ResultLevel.NORMAL,
- )
-
nic.netns.run(
"ethtool -K {} {}".format(nic.name, ethtool_offload_string),
job_level=ResultLevel.NORMAL,
@@ -78,12 +72,6 @@ class OffloadSubConfigMixin(BaseSubConfigMixin):
ethtool_offload_string += " %s %s" % (name, "on")
for nic in self.offload_nics:
- if "sctp_stream" in self.params.perf_tests:
- nic.netns.run(
- "iptables -D OUTPUT ! -o %s -p sctp -j DROP" % nic.name,
- job_level=ResultLevel.NORMAL,
- )
-
# set all the offloads back to 'on' state
nic.netns.run(
"ethtool -K {} {}".format(nic.name, ethtool_offload_string),
--
2.21.3
3 years, 8 months
[PATCH 1/4] Recipes.ENRT.ConfigMixins: remove iptables rules for sctp in OffloadSubConfigMixin
by Jan Tluka
It does not make sense to have this setup just for offloaded tests.
A follow up patch will add this to proper place.
Signed-off-by: Jan Tluka <jtluka(a)redhat.com>
---
.../ENRT/ConfigMixins/OffloadSubConfigMixin.py | 12 ------------
1 file changed, 12 deletions(-)
diff --git a/lnst/Recipes/ENRT/ConfigMixins/OffloadSubConfigMixin.py b/lnst/Recipes/ENRT/ConfigMixins/OffloadSubConfigMixin.py
index 989743a8..3250a6af 100644
--- a/lnst/Recipes/ENRT/ConfigMixins/OffloadSubConfigMixin.py
+++ b/lnst/Recipes/ENRT/ConfigMixins/OffloadSubConfigMixin.py
@@ -45,12 +45,6 @@ class OffloadSubConfigMixin(BaseSubConfigMixin):
ethtool_offload_string += " %s %s" % (name, value)
for nic in self.offload_nics:
- if "sctp_stream" in self.params.perf_tests:
- nic.netns.run(
- "iptables -I OUTPUT ! -o %s -p sctp -j DROP" % nic.name,
- job_level=ResultLevel.NORMAL,
- )
-
nic.netns.run(
"ethtool -K {} {}".format(nic.name, ethtool_offload_string),
job_level=ResultLevel.NORMAL,
@@ -78,12 +72,6 @@ class OffloadSubConfigMixin(BaseSubConfigMixin):
ethtool_offload_string += " %s %s" % (name, "on")
for nic in self.offload_nics:
- if "sctp_stream" in self.params.perf_tests:
- nic.netns.run(
- "iptables -D OUTPUT ! -o %s -p sctp -j DROP" % nic.name,
- job_level=ResultLevel.NORMAL,
- )
-
# set all the offloads back to 'on' state
nic.netns.run(
"ethtool -K {} {}".format(nic.name, ethtool_offload_string),
--
2.21.3
3 years, 8 months
[PATCH] lnst.Slave.InterfaceManager: handle RTM_DELLINK with family
PF_BRIDGE properly
by Jan Tluka
If a bridge device (t_br0) is removed and another device (e.g. t_vlan0)
references it as IFLA_MASTER (ie. the device was previously added to the bridge),
an RTM_DELLINK is sent by kernel to userspace for t_vlan0 device before the
bridge device is removed.
The InterfaceManager handles all RTM_DELLINK as a notification of removal of
a device. In this case however, the RTM_DELLINK has a special meaning that the
link between t_vlan0 and t_br0 does not exist anymore, and is indicated by
PF_BRIDGE (value 7) family in the RTM_DELLINK message.
Without this patch, upon delivery of such message, the t_vlan0 device is
removed from the InterfaceManager's device list and marked as deleted. Later
on during the deconfiguration, the actual removal (ip link delete) of the
device is skipped because LNST thinks it has been already deleted.
Signed-off-by: Jan Tluka <jtluka(a)redhat.com>
---
lnst/Slave/InterfaceManager.py | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/lnst/Slave/InterfaceManager.py b/lnst/Slave/InterfaceManager.py
index 858c32f1..221d386f 100644
--- a/lnst/Slave/InterfaceManager.py
+++ b/lnst/Slave/InterfaceManager.py
@@ -38,6 +38,7 @@ from pyroute2.netlink.rtnl import RTM_GETADDR
from pyroute2.netlink.rtnl import RTM_DELADDR
NL_GROUPS = RTMGRP_IPV4_IFADDR | RTMGRP_IPV6_IFADDR | RTMGRP_LINK
+PF_BRIDGE = 7
class InterfaceManager(object):
def __init__(self, server_handler):
@@ -127,6 +128,9 @@ class InterfaceManager(object):
dev._disable()
elif msg['header']['type'] == RTM_DELLINK:
+ if msg['family'] == PF_BRIDGE:
+ return
+
if msg['index'] in self._devices:
dev = self._devices[msg['index']]
dev._deleted = True
--
2.21.3
3 years, 9 months
[PATCH v2 0/4] Make TRex profiles extensible
by Adrian Moreno
Currently the TRex Test has a hardcoded set of streams.
In order to make it more generic and extendable, this series first refactors
the client and server code into a library that does not depend on other
LNST code. That way, a small cli appliation can be implemented to inject
traffic using the same code as LNST would. This is useful for early prototyping.
An example of such as tool is introduced: test_tools/tperf. It basically runs the
client and server as LNST would and report the aggregated throughput.
Finally, the stream can be modularized into TRex compatible modules so that:
- New stream generators can be easily implemented
- TRex tools (e.g: stl-sym) can be used for stream generator development
The current stream generation is modularized as an example: UdpSimple
Finally a new stream generator is introduced: UdpMultiflow, that creates multiple
UDP streams by modifying the source and destination ports.
Adrian Moreno (4):
lnst.Tests.TRex Create lnst-independent library
test_tools: Add tperf
lnst.TRex Use stl compatible modules to generate streams
lnst.Tests.TRex: Add UDPMultiflow
lnst/External/TRex/TRexLib.py | 215 +++++++++++++++++++++++++++++
lnst/External/TRex/UDPMultiflow.py | 39 ++++++
lnst/External/TRex/UDPSimple.py | 38 +++++
lnst/External/TRex/__init__.py | 0
lnst/External/__init__.py | 0
lnst/Tests/TRex.py | 144 ++++---------------
test_tools/tperf/tperf | 201 +++++++++++++++++++++++++++
7 files changed, 517 insertions(+), 120 deletions(-)
create mode 100644 lnst/External/TRex/TRexLib.py
create mode 100644 lnst/External/TRex/UDPMultiflow.py
create mode 100644 lnst/External/TRex/UDPSimple.py
create mode 100644 lnst/External/TRex/__init__.py
create mode 100644 lnst/External/__init__.py
create mode 100755 test_tools/tperf/tperf
--
2.26.2
3 years, 9 months