[PATCH] Recipes: Remove rootpass
by Radek Pazdera
Rootpass parameter is no longer used. This removes it from parser and
also from all example recipes.
Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com>
---
lnst/Controller/MachinePool.py | 2 +-
lnst/Controller/NetTestController.py | 1 -
lnst/Controller/NetTestParse.py | 3 ---
recipes/examples/quick_guides/ping_check.xml | 8 ++++----
recipes/multicast/multicast-all.xml | 6 ++----
recipes/team/machineconfig-test1.xml | 2 +-
recipes/team/machineconfig-test2.xml | 2 +-
7 files changed, 9 insertions(+), 15 deletions(-)
diff --git a/lnst/Controller/MachinePool.py b/lnst/Controller/MachinePool.py
index 6b7bf67..7ed6272 100644
--- a/lnst/Controller/MachinePool.py
+++ b/lnst/Controller/MachinePool.py
@@ -119,7 +119,7 @@ class MachinePool:
mcfg = dom.createElement("machineconfig")
info = dom.createElement("info")
- supported = ["hostname", "libvirt_domain", "rpcport", "rootpass"]
+ supported = ["hostname", "libvirt_domain", "rpcport"]
for attr_name, attr_val in self._pool[pm_id]["info"].iteritems():
if attr_name in supported:
info.setAttribute(attr_name, attr_val)
diff --git a/lnst/Controller/NetTestController.py b/lnst/Controller/NetTestController.py
index 469db04..b0ffd90 100644
--- a/lnst/Controller/NetTestController.py
+++ b/lnst/Controller/NetTestController.py
@@ -494,7 +494,6 @@ class NetTestController:
logging.info("Retrieving capture files from slaves")
for machine_id in self._recipe["machines"]:
hostname = self._recipe["machines"][machine_id]['info']['hostname']
- rootpass = self._recipe["machines"][machine_id]['info']['rootpass']
slave_logging_dir = os.path.join(logging_root, hostname)
try:
diff --git a/lnst/Controller/NetTestParse.py b/lnst/Controller/NetTestParse.py
index 40223f5..03fedd1 100644
--- a/lnst/Controller/NetTestParse.py
+++ b/lnst/Controller/NetTestParse.py
@@ -285,9 +285,6 @@ class MachineConfigParse(RecipeParser):
info["libvirt_domain"] = self._get_attribute(node,
"libvirt_domain")
- if self._has_attribute(node, "rootpass"):
- info["rootpass"] = self._get_attribute(node, "rootpass")
-
if self._has_attribute(node, "rpcport"):
info["rpcport"] = self._get_attribute(node, "rpcport", int)
diff --git a/recipes/examples/quick_guides/ping_check.xml b/recipes/examples/quick_guides/ping_check.xml
index 8960b63..9a0022c 100644
--- a/recipes/examples/quick_guides/ping_check.xml
+++ b/recipes/examples/quick_guides/ping_check.xml
@@ -8,8 +8,8 @@ is available on LNST wiki page:
<machines>
<machine id="testmachine1">
<machineconfig>
- <!-- Change hostname and rootpass bellow -->
- <info hostname="192.168.1.2" rootpass="r00t"/>
+ <!-- Change hostname bellow -->
+ <info hostname="192.168.1.2"/>
<netdevices>
<!-- Change hwaddr here -->
<netdevice type="eth" network="tnet" hwaddr="52:54:01:00:00:05"
@@ -27,8 +27,8 @@ is available on LNST wiki page:
<machine id="testmachine2">
<machineconfig>
- <!-- Change hostname and rootpass bellow -->
- <info hostname="192.168.1.3" rootpass="r00t"/>
+ <!-- Change hostname bellow -->
+ <info hostname="192.168.1.3"/>
<netdevices>
<!-- Change hwaddr here -->
<netdevice type="eth" network="tnet" hwaddr="52:54:01:00:00:06"
diff --git a/recipes/multicast/multicast-all.xml b/recipes/multicast/multicast-all.xml
index af9aae9..cd8da19 100644
--- a/recipes/multicast/multicast-all.xml
+++ b/recipes/multicast/multicast-all.xml
@@ -11,8 +11,7 @@
<machines>
<machine id="1">
<machineconfig>
- <info hostname="{$hostname1}" libvirt_domain="{$virtdomain1}"
- rootpass="redhat"/>
+ <info hostname="{$hostname1}" libvirt_domain="{$virtdomain1}"/>
<netdevices>
<libvirt_create>
<netdevice network="tnet" phys_id="1" type="eth"/>
@@ -35,8 +34,7 @@
</machine>
<machine id="2">
<machineconfig>
- <info hostname="{$hostname2}" libvirt_domain="{$virtdomain2}"
- rootpass="redhat"/>
+ <info hostname="{$hostname2}" libvirt_domain="{$virtdomain2}"/>
<netdevices>
<libvirt_create>
<netdevice network="tnet" phys_id="1" type="eth"/>
diff --git a/recipes/team/machineconfig-test1.xml b/recipes/team/machineconfig-test1.xml
index 1591a01..ec3062a 100644
--- a/recipes/team/machineconfig-test1.xml
+++ b/recipes/team/machineconfig-test1.xml
@@ -1,5 +1,5 @@
<machineconfig>
- <info hostname="192.168.122.182" rootpass="aaa"/>
+ <info hostname="192.168.122.182"/>
<netdevices>
<netdevice type="eth" phys_id="1" hwaddr="52:54:00:3d:c7:6d" network="testing"/>
<netdevice type="eth" phys_id="2" hwaddr="52:54:00:73:15:c2" network="testing"/>
diff --git a/recipes/team/machineconfig-test2.xml b/recipes/team/machineconfig-test2.xml
index 10a2cf7..4c57d2e 100644
--- a/recipes/team/machineconfig-test2.xml
+++ b/recipes/team/machineconfig-test2.xml
@@ -1,5 +1,5 @@
<machineconfig>
- <info hostname="192.168.122.223" rootpass="aaa"/>
+ <info hostname="192.168.122.223"/>
<netdevices>
<netdevice type="eth" phys_id="1" hwaddr="52:54:00:99:bb:27" network="testing"/>
<netdevice type="eth" phys_id="2" hwaddr="52:54:00:b7:cc:fb" network="testing"/>
--
1.7.7.6
11 years, 5 months
[PATCH] SshUtils: Removing unused module
by Radek Pazdera
This patch removes SshUtils module. It is no longer used nor necessary.
Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com>
---
lnst/Common/SshUtils.py | 421 -----------------------------------------------
1 files changed, 0 insertions(+), 421 deletions(-)
delete mode 100644 lnst/Common/SshUtils.py
diff --git a/lnst/Common/SshUtils.py b/lnst/Common/SshUtils.py
deleted file mode 100644
index 91d3386..0000000
--- a/lnst/Common/SshUtils.py
+++ /dev/null
@@ -1,421 +0,0 @@
-"""
-Ssh utils.
-
-Copyright 2011 Red Hat, Inc.
-Licensed under the GNU General Public License, version 2 as
-published by the Free Software Foundation; see COPYING for details.
-"""
-
-__autor__ = """
-jzupka(a)redhat.com (Jiri Zupka)
-"""
-
-import time, logging
-from lnst.Common.ShellProcess import ShellProcess
-
-class LoginError(Exception):
- def __init__(self, msg, output):
- Exception.__init__(self, msg, output)
- self.msg = msg
- self.output = output
-
- def __str__(self):
- return "%s (output: %r)" % (self.msg, self.output)
-
-
-class LoginAuthenticationError(LoginError):
- pass
-
-
-class LoginTimeoutError(LoginError):
- def __init__(self, output):
- LoginError.__init__(self, "Login timeout expired", output)
-
-
-class LoginProcessTerminatedError(LoginError):
- def __init__(self, status, output):
- LoginError.__init__(self, None, output)
- self.status = status
-
- def __str__(self):
- return ("Client process terminated (status: %s, output: %r)" %
- (self.status, self.output))
-
-
-class LoginBadClientError(LoginError):
- def __init__(self, client):
- LoginError.__init__(self, None, None)
- self.client = client
-
- def __str__(self):
- return "Unknown remote shell client: %r" % self.client
-
-
-class SCPError(Exception):
- def __init__(self, msg, output):
- Exception.__init__(self, msg, output)
- self.msg = msg
- self.output = output
-
- def __str__(self):
- indented = "\n".join((4 * " ") + i for i in self.output.splitlines())
- return "%s%s" % (self.msg, indented)
-
-
-class SCPAuthenticationError(SCPError):
- pass
-
-
-class SCPAuthenticationTimeoutError(SCPAuthenticationError):
- def __init__(self, output):
- SCPAuthenticationError.__init__(self, "Authentication timeout expired",
- output)
-
-
-class SCPTransferTimeoutError(SCPError):
- def __init__(self, output):
- SCPError.__init__(self, "Transfer timeout expired", output)
-
-
-class SCPTransferFailedError(SCPError):
- def __init__(self, status, output):
- SCPError.__init__(self, None, output)
- self.status = status
-
- def __str__(self):
- return ("SCP transfer failed (status: %s, output: %r)" %
- (self.status, self.output))
-
-
-def _remote_login(session, username, password, prompt, timeout=10):
- """
- Log into a remote host (guest) using SSH or Telnet. Wait for questions
- and provide answers. If timeout expires while waiting for output from the
- child (e.g. a password prompt or a shell prompt) -- fail.
-
- @brief: Log into a remote host (guest) using SSH or Telnet.
-
- @param session: An Expect or ShellSession instance to operate on
- @param username: The username to send in reply to a login prompt
- @param password: The password to send in reply to a password prompt
- @param prompt: The shell prompt that indicates a successful login
- @param timeout: The maximal time duration (in seconds) to wait for each
- step of the login procedure (i.e. the "Are you sure" prompt, the
- password prompt, the shell prompt, etc)
- @raise LoginTimeoutError: If timeout expires
- @raise LoginAuthenticationError: If authentication fails
- @raise LoginProcessTerminatedError: If the client terminates during login
- @raise LoginError: If some other error occurs
- """
- password_prompt_count = 0
- login_prompt_count = 0
-
- while True:
- try:
- match, text = session.read_until_output_matches(
- [r"[Aa]re you sure", r"[Pp]assword:\s*$", r"[Ll]ogin:\s*$",
- r"[Cc]onnection.*closed", r"[Cc]onnection.*refused",
- r"[Pp]lease wait", prompt],
- timeout=timeout, internal_timeout=2)
- if match == 0: # "Are you sure you want to continue connecting"
- logging.log(logging.DEBUG2, "Got 'Are you sure...'; sending 'yes'")
- session.sendline("yes")
- continue
- elif match == 1: # "password:"
- if password_prompt_count == 0:
- logging.log(logging.DEBUG2, "Got password prompt; sending '%s'", password)
- session.sendline(password)
- password_prompt_count += 1
- continue
- else:
- raise LoginAuthenticationError("Invalid password",
- text)
- elif match == 2: # "login:"
- if login_prompt_count == 0 and password_prompt_count == 0:
- logging.log(logging.DEBUG2, "Got username prompt; sending '%s'", username)
- session.sendline(username)
- login_prompt_count += 1
- continue
- else:
- if login_prompt_count > 0:
- msg = "Got username prompt twice"
- else:
- msg = "Got username prompt after password prompt"
- raise LoginAuthenticationError(msg, text)
- elif match == 3: # "Connection closed"
- logging.log(logging.INFO, "Remote command execution successful")
- break
- elif match == 4: # "Connection refused"
- raise LoginError("Client said 'connection refused'", text)
- elif match == 5: # "Please wait"
- logging.log(logging.DEBUG2, "Got 'Please wait'")
- timeout = 30
- continue
- elif match == 6: # prompt
- logging.log(logging.INFO, "Got shell prompt -- logged in")
- break
- except ShellProcess.ProcessTimeoutError as e:
- raise LoginTimeoutError(e.output)
- except ShellProcess.ProcessTerminatedError as e:
- raise LoginProcessTerminatedError(e.status, e.output)
-
-
-def remote_login(host, port, username, password, prompt, linesep="\n",
- log_filename=None, timeout=10, command=None):
- """
- Log into a remote host (guest) using SSH/Telnet/Netcat.
-
- @param client: The client to use ('ssh', 'telnet' or 'nc')
- @param host: Hostname or IP address
- @param port: Port to connect to
- @param username: Username (if required)
- @param password: Password (if required)
- @param prompt: Shell prompt (regular expression)
- @param linesep: The line separator to use when sending lines
- (e.g. '\\n' or '\\r\\n')
- @param log_filename: If specified, log all output to this file
- @param timeout: The maximal time duration (in seconds) to wait for
- each step of the login procedure (i.e. the "Are you sure" prompt
- or the password prompt)
- @raise LoginBadClientError: If an unknown client is requested
- @raise: Whatever _remote_login() raises
- @return: A ShellSession object.
- """
- if command is None:
- command = ""
- else:
- command = "-C '%s'" % (command)
- cmd = ("ssh -t -o ServerAliveInterval=5 -o ServerAliveCountMax=2 -o UserKnownHostsFile=/dev/null "
- "-p %s %s@%s %s" % #-o PreferredAuthentications=password
- (port, username, host, command))
-
- logging.log(logging.DEBUG2, "Trying to login with command '%s'", cmd)
- session = ShellProcess(cmd, linesep=linesep, debug_level=logging.DEBUG2,
- process_manager=True)
- try:
- _remote_login(session, username, password, prompt, timeout)
- except:
- raise
- if log_filename:
- pass
- #session.set_output_params((log_filename,))
- return session
-
-
-def wait_for_login(host, port, username, password, prompt, linesep="\n",
- log_filename=None, timeout=240, internal_timeout=10,
- command = ""):
- """
- Make multiple attempts to log into a remote host (guest) until one succeeds
- or timeout expires.
-
- @param timeout: Total time duration to wait for a successful login
- @param internal_timeout: The maximal time duration (in seconds) to wait for
- each step of the login procedure (e.g. the "Are you sure" prompt
- or the password prompt)
- @see: remote_login()
- @raise: Whatever remote_login() raises
- @return: A ShellSession object.
- """
- logging.debug("Attempting to log into %s:%s (timeout %ds)",
- host, port, timeout)
- end_time = time.time() + timeout
- while time.time() < end_time:
- try:
- return remote_login(host, port, username, password, prompt,
- linesep, log_filename, internal_timeout,
- command)
- except LoginError as e:
- logging.debug(e)
- time.sleep(2)
- # Timeout expired; try one more time but don't catch exceptions
- return remote_login(host, port, username, password, prompt,
- linesep, log_filename, internal_timeout, command)
-
-
-def _remote_scp(session, password_list, transfer_timeout=600, login_timeout=10):
- """
- Transfer file(s) to a remote host (guest) using SCP. Wait for questions
- and provide answers. If login_timeout expires while waiting for output
- from the child (e.g. a password prompt), fail. If transfer_timeout expires
- while waiting for the transfer to complete, fail.
-
- @brief: Transfer files using SCP, given a command line.
-
- @param session: An Expect or ShellSession instance to operate on
- @param password_list: Password list to send in reply to the password prompt
- @param transfer_timeout: The time duration (in seconds) to wait for the
- transfer to complete.
- @param login_timeout: The maximal time duration (in seconds) to wait for
- each step of the login procedure (i.e. the "Are you sure" prompt or
- the password prompt)
- @raise SCPAuthenticationError: If authentication fails
- @raise SCPTransferTimeoutError: If the transfer fails to complete in time
- @raise SCPTransferFailedError: If the process terminates with a nonzero
- exit code
- @raise SCPError: If some other error occurs
- """
- password_prompt_count = 0
- timeout = login_timeout
- authentication_done = False
-
- scp_type = len(password_list)
-
- while True:
- try:
- match, text = session.read_until_output_matches(
- [r"[Aa]re you sure", r"[Pp]assword:\s*$", r"lost connection",
- r"Exit status 0"],
- timeout=timeout, internal_timeout=5)
- if match == 0: # "Are you sure you want to continue connecting"
- logging.log(logging.DEBUG2, "Got 'Are you sure...'; sending 'yes'")
- session.sendline("yes")
- continue
- elif match == 1: # "password:"
- if password_prompt_count == 0:
- logging.log(logging.DEBUG2, "Got password prompt; sending '%s'" %
- password_list[password_prompt_count])
- try:
- session.sendline(password_list[password_prompt_count])
- except TypeError:
- logging.error("For logging is necessary set rootpass"+
- " in config.")
- raise
- password_prompt_count += 1
- timeout = transfer_timeout
- if scp_type == 1:
- authentication_done = True
- continue
- elif password_prompt_count == 1 and scp_type == 2:
- logging.log(logging.DEBUG2, "Got password prompt; sending '%s'" %
- password_list[password_prompt_count])
- session.sendline(password_list[password_prompt_count])
- password_prompt_count += 1
- timeout = transfer_timeout
- authentication_done = True
- continue
- else:
- raise SCPAuthenticationError("Invalid password",
- text)
- elif match == 2: # "lost connection"
- raise SCPError("SCP client said 'lost connection'", text)
- """elif match == 3: #Exit transfer correctly.
- logging.log(logging.DEBUG2, "SCP process terminated with status 0")
- break"""
- except ShellProcess.ProcessTimeoutError as e:
- if authentication_done:
- raise SCPTransferTimeoutError(e.output)
- else:
- raise SCPAuthenticationTimeoutError(e.output)
- except ShellProcess.ProcessTerminatedError as e:
- if e.status == 0:
- logging.log(logging.DEBUG2, "SCP process terminated with status 0")
- break
- else:
- raise SCPTransferFailedError(e.status, e.output)
-
-
-def remote_scp(command, password_list, log_filename=None, transfer_timeout=600,
- login_timeout=10):
- """
- Transfer file(s) to a remote host (guest) using SCP.
-
- @brief: Transfer files using SCP, given a command line.
-
- @param command: The command to execute
- (e.g. "scp -r foobar root@localhost:/tmp/").
- @param password_list: Password list to send in reply to a password prompt.
- @param log_filename: If specified, log all output to this file
- @param transfer_timeout: The time duration (in seconds) to wait for the
- transfer to complete.
- @param login_timeout: The maximal time duration (in seconds) to wait for
- each step of the login procedure (i.e. the "Are you sure" prompt
- or the password prompt)
- @raise: Whatever _remote_scp() raises
- """
- logging.debug("Trying to SCP with command '%s', timeout %ss",
- command, transfer_timeout)
- if log_filename:
- pass
- #output_func = log_line
- #output_params = (log_filename,)
- else:
- pass
- #output_func = None
- #output_params = ()
- session = ShellProcess(command, debug_level=logging.DEBUG2)
- _remote_scp(session, password_list, transfer_timeout, login_timeout)
-
-
-def scp_to_remote(host, port, username, password, local_path, remote_path,
- log_filename=None, timeout=600):
- """
- Copy files to a remote host (guest) through scp.
-
- @param host: Hostname or IP address
- @param username: Username (if required)
- @param password: Password (if required)
- @param local_path: Path on the local machine where we are copying from
- @param remote_path: Path on the remote machine where we are copying to
- @param log_filename: If specified, log all output to this file
- @param timeout: The time duration (in seconds) to wait for the transfer
- to complete.
- @raise: Whatever remote_scp() raises
- """
- command = ("scp -v -o UserKnownHostsFile=/dev/null "
- " -r -P %s '%s' '%s@%s:%s'" % #-o PreferredAuthentications=password
- (port, local_path, username, host, remote_path))
- password_list = []
- password_list.append(password)
- remote_scp(command, password_list, log_filename, timeout)
- logging.info("Copy to remote machine (%s) pass." % host)
-
-
-def scp_from_remote(host, port, username, password, remote_path, local_path,
- log_filename=None, timeout=600):
- """
- Copy files from a remote host (guest).
-
- @param host: Hostname or IP address
- @param username: Username (if required)
- @param password: Password (if required)
- @param local_path: Path on the local machine where we are copying from
- @param remote_path: Path on the remote machine where we are copying to
- @param log_filename: If specified, log all output to this file
- @param timeout: The time duration (in seconds) to wait for the transfer
- to complete.
- @raise: Whatever remote_scp() raises
- """
- command = ("scp -v -o UserKnownHostsFile=/dev/null "
- "-o PreferredAuthentications=password -r -P %s '%s@%s:%s' '%s'" %
- (port, username, host, remote_path, local_path))
- password_list = []
- password_list.append(password)
- remote_scp(command, password_list, log_filename, timeout)
- logging.info("Copy from remote machine (%s) pass." % host)
-
-
-def scp_between_remotes(src, dst, port, s_passwd, d_passwd, s_name, d_name,
- s_path, d_path, log_filename=None, timeout=600):
- """
- Copy files from a remote host (guest) to another remote host (guest).
-
- @param src/dst: Hostname or IP address of src and dst
- @param s_name/d_name: Username (if required)
- @param s_passwd/d_passwd: Password (if required)
- @param s_path/d_path: Path on the remote machine where we are copying
- from/to
- @param log_filename: If specified, log all output to this file
- @param timeout: The time duration (in seconds) to wait for the transfer
- to complete.
-
- @return: True on success and False on failure.
- """
- command = ("scp -v -o UserKnownHostsFile=/dev/null -o "
- "PreferredAuthentications=password -r -P %s '%s@%s:%s' '%s@%s:%s'" %
- (port, s_name, src, s_path, d_name, dst, d_path))
- password_list = []
- password_list.append(s_passwd)
- password_list.append(d_passwd)
- remote_scp(command, password_list, log_filename, timeout)
- logging.info("Copy betwen remote machines (%s) and (%s) pass." % (src, dst))
--
1.7.7.6
11 years, 5 months
[PATCH] XML-RPC: File transfers through RPC
by Radek Pazdera
This patch adds a XML API for backwards file transfers from
slaves to the controller.
Some changes to the previous ctl -> slave transfering API was made
so both the interfaces are similar to each other.
Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com>
---
lnst/Controller/NetTestController.py | 33 +++++++++---
lnst/Slave/NetTestSlave.py | 90 +++++++++++++++++++++++-----------
2 files changed, 86 insertions(+), 37 deletions(-)
diff --git a/lnst/Controller/NetTestController.py b/lnst/Controller/NetTestController.py
index 340cdea..469db04 100644
--- a/lnst/Controller/NetTestController.py
+++ b/lnst/Controller/NetTestController.py
@@ -20,7 +20,6 @@ import tempfile
from xmlrpclib import Binary
from pprint import pprint, pformat
from lnst.Common.Logs import Logs, log_exc_traceback
-from lnst.Common.SshUtils import scp_from_remote
from lnst.Common.XmlRpc import ServerProxy, ServerException
from lnst.Common.NetUtils import MacPool
from lnst.Common.VirtUtils import VirtNetCtl, VirtDomainCtl, BridgeCtl
@@ -490,7 +489,7 @@ class NetTestController:
self._rpc_call(machine_id, 'stop_packet_capture')
def _gather_capture_files(self):
- logging_root = Logs.get_logging_root_path()
+ logging_root = self._log_root_path
logging_root = os.path.abspath(logging_root)
logging.info("Retrieving capture files from slaves")
for machine_id in self._recipe["machines"]:
@@ -510,8 +509,7 @@ class NetTestController:
for remote_path in capture_files:
filename = os.path.basename(remote_path)
local_path = os.path.join(slave_logging_dir, filename)
- scp_from_remote(hostname, "22", "root", rootpass,
- remote_path, local_path)
+ self._copy_from_slave(machine_id, remote_path, local_path)
def _update_system_config(self, machine_id, res_data, persistent=False):
info = self._get_machineinfo(machine_id)
@@ -568,7 +566,7 @@ class NetTestController:
logger.handle(record)
def _copy_to_slave(self, local_path, machine_id, remote_path=None):
- self._rpc_call(machine_id, "start_copy", remote_path)
+ remote_path = self._rpc_call(machine_id, "start_copy_to", remote_path)
f = open(local_path, "r+b")
while True:
@@ -576,11 +574,28 @@ class NetTestController:
if len(data) == 0:
break
- self._rpc_call(machine_id, "copy_part", Binary(data))
+ self._rpc_call(machine_id, "copy_part_to",
+ remote_path, Binary(data))
- # return remote path
- rpath = self._rpc_call(machine_id, "finish_copy")
- return rpath
+ self._rpc_call(machine_id, "finish_copy_to", remote_path)
+ return remote_path
+
+ def _copy_from_slave(self, machine_id, remote_path, local_path):
+ status = self._rpc_call(machine_id, "start_copy_from", remote_path)
+ if not status:
+ raise NetTestError("The requested file cannot be transfered." \
+ "It file does not exist on machine %d" % machine_id)
+
+ local_file = open(local_path, "wb")
+
+ binary = "next"
+ while binary != "":
+ binary = self._rpc_call(machine_id, "copy_part_from",
+ remote_path, 1024*1024) # 1MB buffer
+ local_file.write(binary.data)
+
+ local_file.close()
+ self._rpc_call(machine_id, "finish_copy_from", remote_path)
def _load_test_modules(self, dirs):
modules = {}
diff --git a/lnst/Slave/NetTestSlave.py b/lnst/Slave/NetTestSlave.py
index 28cca22..4106f2d 100644
--- a/lnst/Slave/NetTestSlave.py
+++ b/lnst/Slave/NetTestSlave.py
@@ -14,6 +14,7 @@ jpirko(a)redhat.com (Jiri Pirko)
import signal
import select, logging
import os
+from xmlrpclib import Binary
from tempfile import NamedTemporaryFile
from SimpleXMLRPCServer import SimpleXMLRPCRequestHandler
from lnst.Common.Logs import Logs, log_exc_traceback
@@ -40,7 +41,8 @@ class NetTestSlaveXMLRPC:
self._netconfig = NetConfig()
self._command_context = command_context
- self._copy_target = None
+ self._copy_targets = {}
+ self._copy_sources = {}
self._cache = ResourceCache(config.get_option("cache", "dir"),
config.get_option("cache", "expiration_period"))
@@ -50,11 +52,13 @@ class NetTestSlaveXMLRPC:
def hello(self):
self.clear_resource_table()
self._cache.del_old_entries()
+ self.reset_file_transfers()
return "hello"
def bye(self):
self.clear_resource_table()
self._cache.del_old_entries()
+ self.reset_file_transfers()
return "bye"
def get_new_logs(self):
@@ -153,33 +157,6 @@ class NetTestSlaveXMLRPC:
self._cache.del_old_entries()
return True
- def start_copy(self, filename=None):
- if self._copy_target:
- return False
-
- if filename:
- self._copy_target = open(filename, "w+b")
- else:
- self._copy_target = NamedTemporaryFile("w+b", delete=False)
-
- return True
-
- def copy_part(self, binary_data):
- if self._copy_target:
- self._copy_target.write(binary_data.data)
- return True
-
- return False
-
- def finish_copy(self):
- if self._copy_target:
- name = self._copy_target.name
- del self._copy_target
- self._copy_target = None
- return name
-
- return ""
-
def clear_resource_table(self):
self._resource_table = {}
return True
@@ -206,6 +183,63 @@ class NetTestSlaveXMLRPC:
self._cache.add_cache_entry(file_hash, local_path, name, res_type)
return True
+ def start_copy_to(self, filepath=None):
+ if filepath in self._copy_targets:
+ return ""
+
+ if filepath:
+ self._copy_targets[filepath] = open(filepath, "w+b")
+ else:
+ tmpfile = NamedTemporaryFile("w+b", delete=False)
+ filepath = tmpfile.name
+ self._copy_targets[filepath] = tmpfile
+
+ return filepath
+
+ def copy_part_to(self, filepath, binary_data):
+ if self._copy_targets[filepath]:
+ self._copy_targets[filepath].write(binary_data.data)
+ return True
+
+ return False
+
+ def finish_copy_to(self, filepath):
+ if self._copy_targets[filepath]:
+ self._copy_targets[filepath].close()
+
+ del self._copy_targets[filepath]
+ return True
+
+ return False
+
+ def start_copy_from(self, filepath):
+ if filepath in self._copy_sources or not os.path.exists(filepath):
+ return False
+
+ self._copy_sources[filepath] = open(filepath, "rb")
+ return True
+
+ def copy_part_from(self, filepath, buffsize):
+ data = Binary(self._copy_sources[filepath].read(buffsize))
+ return data
+
+ def finish_copy_from(self, filepath):
+ if filepath in self._copy_sources:
+ self._copy_sources[filepath].close()
+ del self._copy_sources[filepath]
+ return True
+
+ return False
+
+ def reset_file_transfers(self):
+ for file_handle in self._copy_targets.itervalues():
+ file_handle.close()
+ self._copy_targets = {}
+
+ for file_handle in self._copy_sources.itervalues():
+ file_handle.close()
+ self._copy_sources = {}
+
class MySimpleXMLRPCServer(Server):
def __init__(self, command_context, *args, **kwargs):
self._finished = False
--
1.7.7.6
11 years, 5 months
[PATCH] PacketCapture: Suppress tcpdump's error output
by Radek Pazdera
Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com>
---
lnst/Common/PacketCapture.py | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/lnst/Common/PacketCapture.py b/lnst/Common/PacketCapture.py
index 8cfad17..69ad5cb 100644
--- a/lnst/Common/PacketCapture.py
+++ b/lnst/Common/PacketCapture.py
@@ -50,8 +50,8 @@ class PacketCapture:
output_file = self._file
pcap_filter = self._filter
- self._cmd = "tcpdump -p -i %s -w %s \"%s\"" % (interface, output_file,
- pcap_filter)
+ self._cmd = "tcpdump -p -i %s -w %s \"%s\" 2>/dev/null" % \
+ (interface, output_file, pcap_filter)
def _execute_tcpdump(self):
""" Start tcpdump in the background """
--
1.7.7.6
11 years, 5 months
[lnst] lnst-slave: Changing default pidfile
by Jiří Pírko
commit a18f3a8b9df30997cf9ab077a73c9a1614d3bdbf
Author: Radek Pazdera <rpazdera(a)redhat.com>
Date: Fri Nov 23 13:40:15 2012 +0100
lnst-slave: Changing default pidfile
Removing a reference to 'nettestslave'.
Default pidfile name is now lnst-slave.pid (was nettestslave.pid).
Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com>
lnst-slave | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
---
diff --git a/lnst-slave b/lnst-slave
index f0b959f..2181e42 100755
--- a/lnst-slave
+++ b/lnst-slave
@@ -58,7 +58,7 @@ def main():
debug = False
daemon = False
- pidfile = "nettestslave.pid"
+ pidfile = "lnst-slave.pid"
port = None
for opt, arg in opts:
if opt in ("-d", "--debug"):
11 years, 5 months
[lnst] Logs: multiple recipes bug fix
by Jiří Pírko
commit 78f3d68987d6ff934c2ed73426c775179f910ddb
Author: Ondrej Lichtner <olichtne(a)redhat.com>
Date: Fri Nov 23 13:13:18 2012 +0100
Logs: multiple recipes bug fix
Somwhere during the changes to the logging I created a bug that makes it
that multiple recipes are logged into incorrect directories, getting
further into the folder hierarchy. A related problem that occured is
that the logs specific to a recipe would also contain the logs of all
recipes executed after it.
This patch fixes that by saving the paths and then restoring them when
moving on to the next recipe.
The bug was created by moving code from the LoggingServer to the
NetTestController. The LoggingServer had one static variable as the base
for the paths, however after moving it to the controller the variable is
no longer static and is changed by every slave we are connecting to.
This results in the next recipe being logged in the directory for the
last connected slave of the previous recipe.
Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com>
lnst-ctl | 4 ++++
lnst/Common/Logs.py | 36 +++++++++++++++++++++++++++---------
2 files changed, 31 insertions(+), 9 deletions(-)
---
diff --git a/lnst-ctl b/lnst-ctl
index 2ce304f..8504ae1 100755
--- a/lnst-ctl
+++ b/lnst-ctl
@@ -140,6 +140,7 @@ def main():
summary = []
res_serializer = NetTestResultSerializer()
+ Logs.save_state()
for recipe_path in args:
if os.path.isdir(recipe_path):
all_files = []
@@ -151,6 +152,7 @@ def main():
for f in all_files:
recipe_file = os.path.join(recipe_path, f)
if re.match(r'^.*\.xml$', recipe_file):
+ Logs.reset_state()
logging.info("Processing recipe file \"%s\"" % recipe_file)
summary.append(get_recipe_result(action, recipe_file,
cleanup,
@@ -159,11 +161,13 @@ def main():
config))
Logs.set_logging_root_path(clean=False)
else:
+ Logs.reset_state()
summary.append(get_recipe_result(action, recipe_path,
cleanup, res_serializer,
packet_capture,
config))
+ Logs.reset_state()
Logs.set_logging_root_path(clean=False)
print_summary(summary)
diff --git a/lnst/Common/Logs.py b/lnst/Common/Logs.py
index a9910eb..e9c33ea 100644
--- a/lnst/Common/Logs.py
+++ b/lnst/Common/Logs.py
@@ -151,11 +151,12 @@ class Logs:
file_handlers = []
formatter = None
logFolder = None
- logger = None
+ loggers = []
root_path = None
debug = None
date = None
nameExtend = None
+ state = None
@classmethod
def __init__(cls,debug=0, waitForNet=False, logger=logging.getLogger(),
recipe_path=None, to_display=True, date=None,
@@ -175,7 +176,7 @@ class Logs:
cls.logFolder = log_folder
else:
cls.logFolder = os.path.join(os.path.dirname(sys.argv[0]), './Logs')
- cls.logger = logger
+ cls.loggers.append(logger)
cls.debug = debug
if date is None:
cls.date = datetime.datetime.now().strftime("%Y-%m-%d_%H:%M:%S")
@@ -186,6 +187,23 @@ class Logs:
recipe_path, to_display)
@classmethod
+ def save_state(cls):
+ cls.state = {"logFolder": cls.logFolder, "date": cls.date}
+
+ @classmethod
+ def reset_state(cls):
+ cls.logFolder = cls.state["logFolder"]
+ cls.date = cls.state["date"]
+
+ for logger in cls.loggers:
+ handlers = list(logger.handlers)
+ for handler in handlers:
+ if type(handler) == logging.FileHandler:
+ logger.removeHandler(handler)
+
+ cls.loggers = cls.loggers[:1]
+
+ @classmethod
def clean_root_log_folder(cls, logRootPath):
try:
shutil.rmtree(logRootPath)
@@ -204,8 +222,6 @@ class Logs:
file_info.setFormatter(cls.formatter)
file_info.setLevel(logging.INFO)
- cls.file_handlers.append(file_debug)
- cls.file_handlers.append(file_info)
return (file_debug, file_info)
@@ -217,15 +233,17 @@ class Logs:
if recipe_path is None:
recipe_path = ""
- root_logger = cls.logger
+ root_logger = cls.loggers[-1]
recipe_name = os.path.splitext(os.path.split(recipe_path)[1])[0]
cls.root_path = os.path.join(cls.logFolder, cls.date+cls.nameExtend,
recipe_name)
if (clean):
cls.clean_root_log_folder(cls.root_path)
- for fhandler in cls.file_handlers:
- root_logger.removeHandler(fhandler)
- del cls.file_handlers[:]
+
+ handlers = list(root_logger.handlers)
+ for handler in handlers:
+ if type(handler) == logging.FileHandler:
+ logger.removeHandler(handler)
(file_debug, file_info) = cls._create_file_handler(cls.root_path)
root_logger.addHandler(file_debug)
@@ -249,7 +267,7 @@ class Logs:
@param debug: If True print to terminal debug level of logging messages.
"""
- root_logger = cls.logger
+ root_logger = cls.loggers[-1]
if to_display:
display = logging.StreamHandler()
display.setFormatter(cls.formatter)
11 years, 5 months
Re: [lnst trac] #37: Remove Python 2.4 Specific Syntax
by fedora-badges
#37: Remove Python 2.4 Specific Syntax
-------------------------+----------------------
Reporter: rpazdera | Owner: somebody
Type: task | Status: new
Priority: minor | Milestone:
Component: component1 | Version:
Resolution: | Keywords:
Blocked By: | Blocking:
-------------------------+----------------------
Comment (by olichtne):
A patch concerning this ticket was posted [1] last month. It changes the
old exceptions syntax to the new one. New suggestions to what should be
changed:
* string formatting- at the moment we are using the syntax: "" % var ,we
should be using "".format()
* new style classes- I saw some classes that are not inheriting from
object
If anyone has other suggestions feel free to add them here.
[1] https://lists.fedorahosted.org/pipermail/lnst-
developers/2012-October/000695.html
--
Ticket URL: <https://fedorahosted.org/lnst/ticket/37#comment:1>
lnst <https://fedorahosted.org/lnst/>
Linux Network Stack Test
11 years, 5 months
[PATCH] lnst-slave: Changing default pidfile
by Radek Pazdera
Removing a reference to 'nettestslave'.
Default pidfile name is now lnst-slave.pid (was nettestslave.pid).
Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com>
---
lnst-slave | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/lnst-slave b/lnst-slave
index f0b959f..2181e42 100755
--- a/lnst-slave
+++ b/lnst-slave
@@ -58,7 +58,7 @@ def main():
debug = False
daemon = False
- pidfile = "nettestslave.pid"
+ pidfile = "lnst-slave.pid"
port = None
for opt, arg in opts:
if opt in ("-d", "--debug"):
--
1.7.7.6
11 years, 5 months
Re: [lnst trac] #27: update documentation
by fedora-badges
#27: update documentation
--------------------------+------------------------------------------------
Reporter: olichtne | Owner: somebody
Type: enhancement | Status: closed
Priority: major | Milestone: Push LNST as a package into Fedora
Component: component1 | Version:
Resolution: fixed | Keywords:
Blocked By: | Blocking:
--------------------------+------------------------------------------------
Changes (by olichtne):
* resolution: => fixed
* status: new => closed
Comment:
Documentation in git has been moved to the wiki pages. All further
documentation will be also located on the wiki pages. I am therefore
closing this ticket.
--
Ticket URL: <https://fedorahosted.org/lnst/ticket/27#comment:3>
lnst <https://fedorahosted.org/lnst/>
Linux Network Stack Test
11 years, 5 months