vdsm-patches December 2014

vdsm-patches@lists.fedorahosted.org

42 participants
734 discussions

Change in vdsm[master]: jsonrpc: Properly log and return an error if the threadFacto...
by smizrahi＠redhat.com 30 Mar '15

30 Mar '15

Saggi Mizrahi has uploaded a new change for review. Change subject: jsonrpc: Properly log and return an error if the threadFactory fails ...................................................................... jsonrpc: Properly log and return an error if the threadFactory fails Otherwise if scheduling fails there will be a thread error and no messages will be handled while VDSM still runs. Change-Id: I8b579fa0e95397f1dc3b3e939a9040dc5bdef36d Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M lib/yajsonrpc/__init__.py 1 file changed, 13 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/36/35936/1 diff --git a/lib/yajsonrpc/__init__.py b/lib/yajsonrpc/__init__.py index 8320cd6..8120c03 100644 --- a/lib/yajsonrpc/__init__.py +++ b/lib/yajsonrpc/__init__.py @@ -535,7 +535,19 @@ if self._threadFactory is None: self._serveRequest(ctx, request) else: - self._threadFactory(partial(self._serveRequest, ctx, request)) + try: + self._threadFactory(partial(self._serveRequest, ctx, request)) + except Exception as e: + self.log.exception("could not allocate request thread") + ctx.requestDone( + JsonRpcResponse( + None, + JsonRpcInternalError( + str(e) + ), + request.id + ) + ) def stop(self): self.log.info("Stopping JsonRPC Server") -- To view, visit http://gerrit.ovirt.org/35936 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8b579fa0e95397f1dc3b3e939a9040dc5bdef36d Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

8 39

Change in vdsm[master]: jsonrpc: Fix callMethod in JsonRpcClient
by smizrahi＠redhat.com 30 Mar '15

30 Mar '15

Saggi Mizrahi has uploaded a new change for review. Change subject: jsonrpc: Fix callMethod in JsonRpcClient ...................................................................... jsonrpc: Fix callMethod in JsonRpcClient Change-Id: I7cf78f3b7f5d32c10339bbe6b49ce9b93662b4bb Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M lib/yajsonrpc/__init__.py 1 file changed, 4 insertions(+), 7 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/35/35935/1 diff --git a/lib/yajsonrpc/__init__.py b/lib/yajsonrpc/__init__.py index b03151f..8320cd6 100644 --- a/lib/yajsonrpc/__init__.py +++ b/lib/yajsonrpc/__init__.py @@ -304,17 +304,14 @@ self._transport.connect() def callMethod(self, methodName, params=[], rid=None): - return self.call(JsonRpcRequest(methodName, params, rid)) - - def call(self, req): - resp = self.call_batch([req])[0] - if "error" in resp: + resp = self.call(JsonRpcRequest(methodName, params, rid))[0] + if resp.error: raise JsonRpcError(resp.error['code'], resp.error['message']) return resp.result - def call_batch(self, *reqs): - call = self.call_async(reqs) + def call(self, *reqs): + call = self.call_async(*reqs) call.wait() return call.responses -- To view, visit http://gerrit.ovirt.org/35935 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I7cf78f3b7f5d32c10339bbe6b49ce9b93662b4bb Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

8 47

Change in vdsm[master]: jsonrpc: Properly initialize RuntimeError for JsonRpcError
by smizrahi＠redhat.com 30 Mar '15

30 Mar '15

Saggi Mizrahi has uploaded a new change for review. Change subject: jsonrpc: Properly initialize RuntimeError for JsonRpcError ...................................................................... jsonrpc: Properly initialize RuntimeError for JsonRpcError Without this the error doesn't have a proper message when printed\logged Change-Id: Ic922a25e44ddc345d54dff19ffa6a93c461c1844 Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M lib/yajsonrpc/__init__.py 1 file changed, 3 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/34/35934/1 diff --git a/lib/yajsonrpc/__init__.py b/lib/yajsonrpc/__init__.py index dcfbbda..b03151f 100644 --- a/lib/yajsonrpc/__init__.py +++ b/lib/yajsonrpc/__init__.py @@ -33,6 +33,9 @@ def __init__(self, code, msg): self.code = code self.message = msg + super(RuntimeError, self).__init__( + "[%d] %s" % (self.code, self.message) + ) class JsonRpcParseError(JsonRpcError): -- To view, visit http://gerrit.ovirt.org/35934 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ic922a25e44ddc345d54dff19ffa6a93c461c1844 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

7 43

Change in vdsm[master]: gluster: command to create a public key file
by dnarayan＠redhat.com 26 Mar '15

26 Mar '15

Hello Bala.FA, I'd like you to do a code review. Please visit http://gerrit.ovirt.org/17644 to review the following change. Change subject: gluster: command to create a public key file ...................................................................... gluster: command to create a public key file This executes the command to create a public key file which will have public keys of all the hosts of source cluster. This is needed for password-less communication between slave cluster hosts during geo-replication Change-Id: If8c979a89ce11a1622819c474b59dcf088733594 Signed-off-by: ndarshan <dnarayan(a)redhat.com> --- M vdsm/gluster/cli.py M vdsm/gluster/exception.py 2 files changed, 20 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/44/17644/1 diff --git a/vdsm/gluster/cli.py b/vdsm/gluster/cli.py index bac6d1c..64529ae 100644 --- a/vdsm/gluster/cli.py +++ b/vdsm/gluster/cli.py @@ -897,3 +897,12 @@ return _parseVolumeProfileInfo(xmltree, nfs) except _etreeExceptions: raise ge.GlusterXmlErrorException(err=[etree.tostring(xmltree)]) + +@makePublic +def createPublicKeyFile(): + command = _getGlusterSystemCmd() + ["execute", "gsec_create"] + rc, out, err = _execGluster(command) + if rc: + raise ge.GlusterGeoRepPublicKeyFileCreationFailedException(rc, out, err) + else: + return True diff --git a/vdsm/gluster/exception.py b/vdsm/gluster/exception.py index c569a9e..1ee73bb 100644 --- a/vdsm/gluster/exception.py +++ b/vdsm/gluster/exception.py @@ -484,3 +484,14 @@ prefix = "%s: " % (action) self.message = prefix + "Service action is not supported" self.err = [self.message] + +#geo-replication +class GlusterGeoRepException(GlusterException): + code = 4560 + message = "Gluster Geo-Replication Exception" + + +class GlusterGeoRepPublicKeyFileCreationFailedException(GlusterGeoRepException): + code = 4561 + message = "Creation of public key file failed" + -- To view, visit http://gerrit.ovirt.org/17644 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: If8c979a89ce11a1622819c474b59dcf088733594 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: ndarshan <dnarayan(a)redhat.com> Gerrit-Reviewer: Bala.FA <barumuga(a)redhat.com>

9 105

Change in vdsm[master]: caps: Do not use lscpu on ppc64
by vdelima＠redhat.com 24 Mar '15

24 Mar '15

Vitor de Lima has uploaded a new change for review. Change subject: caps: Do not use lscpu on ppc64 ...................................................................... caps: Do not use lscpu on ppc64 Recent libvirt versions correctly show the topology of ppc64 hosts, so it is not required the use of 'lscpu' to obtain this information. Change-Id: I421c82f46a433387b5d7eb1e4624b5f166ee344c Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1148092 Signed-off-by: Vitor de Lima <vdelima(a)redhat.com> --- M vdsm/caps.py M vdsm/ppc64HardwareInfo.py 2 files changed, 1 insertion(+), 41 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/72/33872/1 diff --git a/vdsm/caps.py b/vdsm/caps.py index b8488b4..a594010 100644 --- a/vdsm/caps.py +++ b/vdsm/caps.py @@ -190,13 +190,7 @@ class CpuTopology(object): def __init__(self, capabilities=None): - - if platform.machine() == Architecture.PPC64: - from ppc64HardwareInfo import \ - getCpuTopology as getPPC64CpuTopology - self._topology = getPPC64CpuTopology(capabilities) - else: - self._topology = _getCpuTopology(capabilities) + self._topology = _getCpuTopology(capabilities) def threads(self): return self._topology['threads'] diff --git a/vdsm/ppc64HardwareInfo.py b/vdsm/ppc64HardwareInfo.py index 1a4b30e..b651139 100644 --- a/vdsm/ppc64HardwareInfo.py +++ b/vdsm/ppc64HardwareInfo.py @@ -49,37 +49,3 @@ infoStructure['systemUUID'] = vdsmId return infoStructure - - -(a)utils.memoized -def getCpuTopology(capabilities): - topology = {} - - if capabilities is None: - retcode, out, err = utils.execCmd(['lscpu'], raw=True) - capabilities = out - - corePS = None - threadsPC = None - sockets = None - - for line in capabilities.splitlines(): - if line.strip() == '': - continue - key, value = map(str.strip, line.split(':', 1)) - - if key == 'Socket(s)': - sockets = int(value) - elif key == 'Thread(s) per core': - threadsPC = int(value) - elif key == 'Core(s) per socket': - corePS = int(value) - - if corePS and threadsPC and sockets: - topology['sockets'] = sockets - topology['cores'] = corePS * sockets - topology['threads'] = threadsPC * corePS * sockets - else: - raise RuntimeError('Undefined topology') - - return topology -- To view, visit http://gerrit.ovirt.org/33872 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I421c82f46a433387b5d7eb1e4624b5f166ee344c Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Vitor de Lima <vdelima(a)redhat.com>

9 51

Change in vdsm[master]: virt: Add a function to execute a qemu monitor command
by alitke＠redhat.com 23 Mar '15

23 Mar '15

Adam Litke has uploaded a new change for review. Change subject: virt: Add a function to execute a qemu monitor command ...................................................................... virt: Add a function to execute a qemu monitor command Sometimes it is necessary to execute a monitor command directly (eg. if libvirt does not yet support a certain qemu feature). Libvirt will mark the VM as 'tainted' once this is done for the first time. The reason is that making modifications to a VM using this facility could cause a split-brain situation. It is generally harmless to execute monitor commands which act in a read-only fashion. This function is also useful as a utility for developers who are working on new features. Change-Id: I3e9e07ba0c236c0938b129ae90af825f18f0e644 Signed-off-by: Adam Litke <alitke(a)redhat.com> --- M vdsm/virt/vm.py 1 file changed, 11 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/50/27950/1 diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py index 9cc9958..de27a53 100644 --- a/vdsm/virt/vm.py +++ b/vdsm/virt/vm.py @@ -30,9 +30,11 @@ import threading import time import xml.dom.minidom +import json # 3rd party libs imports import libvirt +import libvirt_qemu # vdsm imports from vdsm import constants @@ -4948,6 +4950,15 @@ if 'tlsPort' in dev: self.conf['displaySecurePort'] = dev['tlsPort'] + def _internalQMPMonitorCommand(self, cmdDict): + """ + Execute a qemu monitor command directly. + WARNING: This will cause libvirt to mark the VM as tainted. + """ + jsonCmd = json.dumps(cmdDict) + ret = libvirt_qemu.qemuMonitorCommand(self._dom, jsonCmd, 0) + return json.loads(ret) + def _getNetworkIp(network): try: -- To view, visit http://gerrit.ovirt.org/27950 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I3e9e07ba0c236c0938b129ae90af825f18f0e644 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Adam Litke <alitke(a)redhat.com>

5 14

Change in vdsm[master]: split vdsm-network restoration service from vdsmd
by asegurap＠redhat.com 23 Mar '15

23 Mar '15

Antoni Segura Puimedon has uploaded a new change for review. Change subject: split vdsm-network restoration service from vdsmd ...................................................................... split vdsm-network restoration service from vdsmd Right now the task of doing network restoration is delegated to the vdsmd service initialization. This has the disadvantages of: - Disabling vdsmd can render the machine unaccessible as the vdsm networks will not be started upon boot. - Not allowing to run it before network.target, making boot slower (other services may configure the nets before the restoration). Change-Id: I70cdf1e8a1e7063ec850081ac6199728d4cde381 Signed-off-by: Antoni S. Puimedon <asegurap(a)redhat.com> --- M init/systemd/Makefile.am A init/systemd/vdsm-network.service.in M init/vdsmd_init_common.sh.in M vdsm/vdsm-restore-net-config 4 files changed, 50 insertions(+), 18 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/41/29441/1 diff --git a/init/systemd/Makefile.am b/init/systemd/Makefile.am index 4a871df..07c4ae8 100644 --- a/init/systemd/Makefile.am +++ b/init/systemd/Makefile.am @@ -23,8 +23,9 @@ nodist_noinst_DATA = \ supervdsmd.service \ systemd-vdsmd \ - vdsm-tmpfiles.d.conf \ vdsmd.service \ + vdsm-network.service \ + vdsm-tmpfiles.d.conf \ $(NULL) CLEANFILES = \ @@ -35,8 +36,9 @@ EXTRA_DIST = \ supervdsmd.service.in \ systemd-vdsmd.in \ - vdsm-tmpfiles.d.conf.in \ vdsmd.service.in \ + vdsm-network.service.in \ + vdsm-tmpfiles.d.conf.in \ $(NULL) all-local: \ diff --git a/init/systemd/vdsm-network.service.in b/init/systemd/vdsm-network.service.in new file mode 100644 index 0000000..97aa3e2 --- /dev/null +++ b/init/systemd/vdsm-network.service.in @@ -0,0 +1,20 @@ +[Unit] +Description=Virtual Desktop Server Manager network restoration +Wants=network.target +Before=network.target network.service NetworkManager.service + +[Service] +Type=oneshot +RemainAfterExit=yes +EnvironmentFile=-/etc/sysconfig/vdsm +ExecStartPre=@BINDIR@/vdsm-tool --vvverbose --append --logfile=@VDSMLOGDIR@/upgrade.log upgrade-unified-persistence +ExecStartPre=@BINDIR@/vdsm-tool --vvverbose --append --logfile=@VDSMLOGDIR@/upgrade.log upgrade-3.0.0-networks +ExecStart=@VDSMDIR@/vdsm-restore-net-config --conditional + +# With the following two directives, we make sure that using systemctl to +# restart will actually restore the networks +RemainAfterExit=yes +ExecStopPost=rm @VDSMRUN@/nets_restored + +[Install] +WantedBy=multi-user.target diff --git a/init/vdsmd_init_common.sh.in b/init/vdsmd_init_common.sh.in index e142470..327d2c3 100644 --- a/init/vdsmd_init_common.sh.in +++ b/init/vdsmd_init_common.sh.in @@ -189,15 +189,6 @@ } -task_restore_nets(){ - local restoration_file="@VDSMRUNDIR@/nets_restored" - if [ ! -f "${restoration_file}" ]; then - "@PYTHON@" "@VDSMDIR@/vdsm-restore-net-config" || return 1 - touch "${restoration_file}" - fi - return 0 -} - task_unified_network_persistence_upgrade(){ "$VDSM_TOOL" ${UPGRADE_LOGGING_PARAMS} upgrade-unified-persistence } @@ -253,7 +244,6 @@ test_space \ test_lo \ unified_network_persistence_upgrade \ - restore_nets \ upgrade_300_nets \ " ;; diff --git a/vdsm/vdsm-restore-net-config b/vdsm/vdsm-restore-net-config index 7adfcac..1c4a9eb 100755 --- a/vdsm/vdsm-restore-net-config +++ b/vdsm/vdsm-restore-net-config @@ -1,4 +1,4 @@ -#! /usr/bin/python +#! /usr/bin/env python2 # # Copyright 2011-2014 Red Hat, Inc. # @@ -18,10 +18,14 @@ # # Refer to the README and COPYING files for full details of the license # +import argparse import logging import logging.config +import os +import sys from vdsm.config import config +from vdsm import constants, utils # Ifcfg persistence restoration from network.configurators import ifcfg @@ -79,11 +83,20 @@ yield cls -def restore(): - if config.get('vars', 'net_persistence') == 'unified': - unified_restoration() +def restore(conditional=False): + if conditional: + restoration_file = constants.P_RUN + 'nets_restored' + if not os.path.exists(restoration_file): + _restore() + utils.touchFile(restoration_file) else: - ifcfg_restoration() + _restore() + + +if config.get('vars', 'net_persistence') == 'unified': + _restore = unified_restoration() +else: + _restore = ifcfg_restoration() if __name__ == '__main__': @@ -95,4 +108,11 @@ level=logging.DEBUG) logging.error('Could not init proper logging', exc_info=True) - restore() + parser = argparse.ArgumentParser( + description="Copies the specified file to /config and then bindmounts " + "it back at it's original location. The file is stored with its full " + "path detail.") + parser.add_argument('--conditional', action='store_true', default=False) + + arguments = parser.parse_args(sys.argv[1:]) + restore(conditional=arguments.conditional) -- To view, visit http://gerrit.ovirt.org/29441 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I70cdf1e8a1e7063ec850081ac6199728d4cde381 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Antoni Segura Puimedon <asegurap(a)redhat.com>

7 74

Change in vdsm[master]: jsonrpc: add ability to get dictionary directly from JsonRpc...
by smizrahi＠redhat.com 20 Mar '15

20 Mar '15

Saggi Mizrahi has uploaded a new change for review. Change subject: jsonrpc: add ability to get dictionary directly from JsonRpcRequest ...................................................................... jsonrpc: add ability to get dictionary directly from JsonRpcRequest Change-Id: Ieb7edc48aa7d773b54db9d43e9f5460920e8ea50 Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M lib/yajsonrpc/__init__.py 1 file changed, 9 insertions(+), 5 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/54/35254/1 diff --git a/lib/yajsonrpc/__init__.py b/lib/yajsonrpc/__init__.py index 69be3c4..cb04e13 100644 --- a/lib/yajsonrpc/__init__.py +++ b/lib/yajsonrpc/__init__.py @@ -105,12 +105,16 @@ return JsonRpcRequest(method, params, reqId) - def encode(self): - res = {'jsonrpc': '2.0', - 'method': self.method, - 'params': self.params, - 'id': self.id} + def toDict(self): + return { + 'jsonrpc': '2.0', + 'method': self.method, + 'params': self.params, + 'id': self.id + } + def encode(self): + res = self.toDict() return json.dumps(res, 'utf-8') def isNotification(self): -- To view, visit http://gerrit.ovirt.org/35254 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ieb7edc48aa7d773b54db9d43e9f5460920e8ea50 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

8 36

Change in vdsm[master]: jsonrpc: Remove JsonRpcClientPool
by smizrahi＠redhat.com 20 Mar '15

20 Mar '15

Saggi Mizrahi has uploaded a new change for review. Change subject: jsonrpc: Remove JsonRpcClientPool ...................................................................... jsonrpc: Remove JsonRpcClientPool It was created before the reactor was able to manage client connections as well. Change-Id: Ib6b67ee2ccb3acfac17a538ad2177f53c0007656 Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M lib/yajsonrpc/__init__.py M tests/jsonRpcHelper.py 2 files changed, 82 insertions(+), 106 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/53/35253/1 diff --git a/lib/yajsonrpc/__init__.py b/lib/yajsonrpc/__init__.py index b3fd590..69be3c4 100644 --- a/lib/yajsonrpc/__init__.py +++ b/lib/yajsonrpc/__init__.py @@ -259,96 +259,6 @@ self.sendReply() -class JsonRpcClientPool(object): - def __init__(self, reactor): - self.log = logging.getLogger("JsonRpcClientPool") - self._reactor = reactor - self._inbox = Queue() - self._clients = {} - self._eventcbs = [] - - def createClient(self, connected_socket): - transport = self._reactor.createClient(connected_socket) - transport.setMessageHandler(self._handleClientMessage) - client = JsonRpcClient(transport) - self._clients[transport] = client - return client - - def _handleClientMessage(self, req): - self._inbox.put_nowait(req) - - def registerEventCallback(self, eventcb): - self._eventcbs.append(ref(eventcb)) - - def unregisterEventCallback(self, eventcb): - for r in self._eventcbs[:]: - cb = r() - if cb is None or cb == eventcb: - try: - self._eventcbs.remove(r) - except ValueError: - # Double unregister, ignore. - pass - - def emit(self, client, event, params): - for r in self._eventcbs[:]: - cb = r() - if cb is None: - continue - - cb(client, event, params) - - def _processEvent(self, client, obj): - if isinstance(obj, list): - map(self._processEvent, obj) - return - - req = JsonRpcRequest.fromRawObject(obj) - if not req.isNotification(): - self.log.warning("Recieved non notification, ignoring") - - self.emit(client, req.method, req.params) - - def serve(self): - while True: - data = self._inbox.get() - if data is None: - return - - transport, message = data - client = self._clients[transport] - try: - mobj = json.loads(message) - isResponse = self._isResponse(mobj) - except: - self.log.exception("Problem parsing message from client") - transport.close() - del self._clients[transport] - continue - - if isResponse: - client._processIncomingResponse(mobj) - else: - self._processEvent(client, mobj) - - def _isResponse(self, obj): - if isinstance(obj, list): - v = None - for res in map(self._isResponse, obj): - if v is None: - v = res - - if v != res: - raise TypeError("batch is mixed") - - return v - else: - return ("result" in obj or "error" in obj) - - def close(self): - self._inbox.put(None) - - class JsonRpcCall(object): def __init__(self): self._ev = Event() @@ -372,9 +282,11 @@ class JsonRpcClient(object): def __init__(self, transport): self.log = logging.getLogger("jsonrpc.JsonRpcClient") + transport.setMessageHandler(self._handleMessage) self._transport = transport self._runningRequests = {} self._lock = Lock() + self._eventcbs = [] def setTimeout(self, timeout): self._transport.setTimeout(timeout) @@ -414,20 +326,20 @@ raise ValueError("Request id already in use %s", rid) self._runningRequests[rid] = ctx - self._transport.send(ctx.encode()) + + self._transport.send(ctx.encode()) # All notifications if ctx.isDone(): self._finalizeCtx(ctx) def _finalizeCtx(self, ctx): - with self._lock: - if not ctx.isDone(): - return + if not ctx.isDone(): + return - cb = ctx.callback - if cb is not None: - cb(self, ctx.getResponses()) + cb = ctx.callback + if cb is not None: + cb(self, ctx.getResponses()) def _processIncomingResponse(self, resp): if isinstance(resp, list): @@ -437,13 +349,73 @@ resp = JsonRpcResponse.fromRawObject(resp) with self._lock: ctx = self._runningRequests.pop(resp.id) - ctx.addResponse(resp) + + ctx.addResponse(resp) self._finalizeCtx(ctx) + def _isResponse(self, obj): + if isinstance(obj, list): + v = None + for res in map(self._isResponse, obj): + if v is None: + v = res + + if v != res: + raise TypeError("batch is mixed") + + return v + else: + return ("result" in obj or "error" in obj) + + def _handleMessage(self, req): + transport, message = req + try: + mobj = json.loads(message) + isResponse = self._isResponse(mobj) + except: + self.log.exception("Problem parsing message from client") + + if isResponse: + self._processIncomingResponse(mobj) + else: + self._processEvent(mobj) + + def _processEvent(self, obj): + if isinstance(obj, list): + map(self._processEvent, obj) + return + + req = JsonRpcRequest.fromRawObject(obj) + if not req.isNotification(): + self.log.warning("Recieved non notification, ignoring") + + self.emit(req.method, req.params) + def close(self): self._transport.close() + def registerEventCallback(self, eventcb): + self._eventcbs.append(ref(eventcb)) + + def unregisterEventCallback(self, eventcb): + for r in self._eventcbs[:]: + cb = r() + if cb is None or cb == eventcb: + try: + self._eventcbs.remove(r) + except ValueError: + # Double unregister, ignore. + pass + + def emit(self, event, params): + for r in self._eventcbs[:]: + cb = r() + if cb is None: + continue + + cb(self, event, params) + class JsonRpcServer(object): log = logging.getLogger("jsonrpc.JsonRpcServer") diff --git a/tests/jsonRpcHelper.py b/tests/jsonRpcHelper.py index 066eaeb..ed7c345 100644 --- a/tests/jsonRpcHelper.py +++ b/tests/jsonRpcHelper.py @@ -30,7 +30,7 @@ from rpc.BindingXMLRPC import BindingXMLRPC, XmlDetector from yajsonrpc.stompReactor import StompDetector from protocoldetector import MultiProtocolAcceptor -from yajsonrpc import JsonRpcClientPool +from yajsonrpc import JsonRpcClient from rpc.BindingJsonRpc import BindingJsonRpc from sslhelper import DEAFAULT_SSL_CONTEXT @@ -105,15 +105,19 @@ reactor = handler._reactor if not client: - cpool = JsonRpcClientPool(reactor) - t = threading.Thread(target=cpool.serve) - t.setDaemon(True) - t.start() - client = cpool.createClient + client = lambda client_socket: ( + JsonRpcClient(reactor.createClient(client_socket)) + ) _, port = acceptor._socket.getsockname() - clientFactory = partial(client, create_socket(sslctx, acceptor._host, - port)) + clientFactory = partial( + client, + create_socket( + sslctx, + acceptor._host, + port + ) + ) yield clientFactory -- To view, visit http://gerrit.ovirt.org/35253 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib6b67ee2ccb3acfac17a538ad2177f53c0007656 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

6 36

Change in vdsm[master]: profile: add trivial benchmark runner
by fromani＠redhat.com 20 Mar '15

20 Mar '15

Francesco Romani has uploaded a new change for review. Change subject: profile: add trivial benchmark runner ...................................................................... profile: add trivial benchmark runner add trivial test script to profile the execution of verbs outside Engine. Change-Id: I3c6918abf2f80c1169149a0c5c025b588012de36 Signed-off-by: Francesco Romani <fromani(a)redhat.com> --- M Makefile.am A contrib/bench.sh 2 files changed, 19 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/32/35132/1 diff --git a/Makefile.am b/Makefile.am index 88a99c8..ec9df9e 100644 --- a/Makefile.am +++ b/Makefile.am @@ -45,6 +45,7 @@ autogen.sh \ build-aux/pkg-version \ contrib/profile-stats \ + contrib/bench.sh \ pylintrc \ vdsm.spec \ vdsm.spec.in \ diff --git a/contrib/bench.sh b/contrib/bench.sh new file mode 100644 index 0000000..722beba --- /dev/null +++ b/contrib/bench.sh @@ -0,0 +1,18 @@ +#!/bin/bash + +if [ -z "$1" ]; then + echo "missing verb to profile" + exit 1 +fi + +STEPS=300 + +:> timings_$1.txt +vdsClient -s 0 startProfile +for i in `seq 1 $STEPS`; do + printf "step: %i/%i\n" $i $STEPS + /usr/bin/time -f '%e' vdsClient -s 0 "$@" 2>> timings_$1.txt > /dev/null +done +vdsClient -s 0 stopProfile +mv /var/run/vdsm/vdsmd.prof vdsmd_$1.prof + -- To view, visit http://gerrit.ovirt.org/35132 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I3c6918abf2f80c1169149a0c5c025b588012de36 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani <fromani(a)redhat.com>

3 7

← Newer
1
...
12
13
14
15
16
17
18
...
74
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

vdsm-patches December 2014