On 6/23/20 11:31 AM, Ondrej Lichtner wrote: > Hi Adrian, > > adding Cc to lnst-developers as the discussed changes may be relevant to > a wider audience. > > replies inline. > > On Tue, Jun 23, 2020 at 10:38:20AM +0200, Adrian Moreno wrote: >> Hi Ondrej, >> >> I'm having a problem (which I think it's the second time I hit), when moving the >> TRex library files. >> >> My current file hierarchy >> >> lnst/Tests/ >> total 64K >> -rw-r--r--. 1 amorenoz adrian 3.8K Aug 29 2019 BaseTestModule.py >> -rw-r--r--. 1 amorenoz adrian 3.8K Aug 29 2019 CPUStatMonitor.py >> -rw-r--r-- 1 amorenoz adrian 636 Jun 23 10:32 __init__.py >> -rw-r--r--. 1 amorenoz adrian 4.4K Aug 29 2019 Iperf.py >> -rw-r--r--. 1 amorenoz adrian 23K Aug 29 2019 Netperf.py >> -rw-r--r--. 1 amorenoz adrian 3.3K Aug 29 2019 PacketAssert.py >> -rw-r--r-- 1 amorenoz adrian 3.8K Apr 16 18:04 Ping.py >> -rw-r--r--. 1 amorenoz adrian 1.8K Oct 23 2019 TestPMD.py >> drwxr-xr-x 3 amorenoz adrian 4.0K Jun 23 09:16 TRex >> >> lnst/Tests/TRex/ >> total 24K >> -rw-r--r-- 1 amorenoz adrian 0 Jun 23 08:46 __init__.py >> -rw-r--r-- 1 amorenoz adrian 1.8K Jun 23 08:33 TestModule.py <- The wrapper >> -rw-r--r-- 1 amorenoz adrian 7.1K Jun 23 09:07 TRexLib.py <- The library >> -rw-r--r-- 1 amorenoz adrian 1.4K Jun 23 09:07 UDPMultiflow.py >> -rw-r--r-- 1 amorenoz adrian 989 Jun 23 09:07 UDPSimple.py >> >> >> The problem is that when I: >> >> from lnst.Tests.TRex.TRexLib import TRexCli, TRexSrv, TRexParams >> >> I get: >> Traceback (most recent call last): >> File "/usr/lib64/python3.6/pdb.py", line 1667, in main >> pdb._runscript(mainpyfile) >> File "/usr/lib64/python3.6/pdb.py", line 1548, in _runscript >> self.run(statement) >> File "/usr/lib64/python3.6/bdb.py", line 434, in run >> exec(cmd, globals, locals) >> File "<string>", line 1, in <module> >> File "/root/lnst/test_tools/tperf/tperf", line 26, in <module> >> from lnst.Tests.TRex.TRexLib import TRexCli, TRexSrv, TRexParams >> File "/root/lnst/test_tools/tperf/../../lnst/Tests/__init__.py", line 15, in >> <module> >> from lnst.Tests.Ping import Ping >> File "/root/lnst/test_tools/tperf/../../lnst/Tests/Ping.py", line 8, in <module> >> from lnst.Tests.BaseTestModule import BaseTestModule, TestModuleError >> File "/root/lnst/test_tools/tperf/../../lnst/Tests/BaseTestModule.py", line >> 19, in <module> >> from lnst.Common.Logs import log_exc_traceback >> File "/root/lnst/test_tools/tperf/../../lnst/Common/Logs.py", line 16, in <module> >> from lnst.Common.LoggingHandler import TransmitHandler >> File "/root/lnst/test_tools/tperf/../../lnst/Common/LoggingHandler.py", line >> 20, in <module> >> from lnst.Common.ConnectionHandler import send_data >> File "/root/lnst/test_tools/tperf/../../lnst/Common/ConnectionHandler.py", >> line 19, in <module> >> from pyroute2 import IPRSocket >> ModuleNotFoundError: No module named 'pyroute2' > > I checked and we can at least remove the pyroute2 import from the > ConnectionHandler, that seems to be unused and probably left-over from > when I changed some stuff here... > >> >> The reason is that >> >> lnst.Tests.__init__.py contains: >> >> from lnst.Tests.Ping import Ping >> from lnst.Tests.PacketAssert import PacketAssert >> from lnst.Tests.Iperf import IperfClient, IperfServer >> >> >> Why is that needed and, can we remove them? > > The idea here was so that in a Recipe you can just do: > > from lnst.Tests import Ping > > Just removing the __init__.py contents will mean that all the recipes > that import this way also need to be updated so I'm not sure that's the > best approach. > > I have just now learned that what I attempted to do here is now called > "namespace package", I was also not aware that the __init__.py of a > package is called even if you're only importing a subpackage, that seems > rather unfortunate... but makes sense from what I see in python docs. > > Thinking about it more this looks like it was a bad idea because it > makes the test modules "depend on each other" - even if you only want to > run Ping, you still need to parse source of all the other modules, in > turn requiring all of their dependencies as well. > > I would still like to be able to do > > from lnst.Tests import Ping, TRexClient > > and have it in such a way that only doing > > from lnst.Tests import TRexClient > > will not even look at the ping source code... I think I'll have to look > a bit more into how to do this thing dynamically. > > So to improve the situation for now: > > * remove pyroute2 from ConnectionHandler.py > * maybe move log_exc_traceback to somewhere where it doesn't end up > importing anything wrt. lnst connections at all... maybe > lnst.Common.Utils > > > Would this improve the situation? Do you maybe have some better ideas? > Not really as the whole point of the work I'm doing (related to the RFC I sent recently) is to have lnst/Tests/Trex/TrexLib.py not depend on anything else. So as a workaround I would rather move the LNST-independent files out of lnst/Tests (maybe lnst/External/?). Other ways could be, have a separate namespace module: lnst/Tests/ __init__.py: from lnst.TestModules.Ping include Ping from lnst.TestModules.TRex include TRexClient lnst/TestModules/ Ping.py TRexClient.py Thanks -- Adrián Moreno