Yeela Kaplan has uploaded a new change for review.
Change subject: [WIP]configurators: move configurator functions into conf_utils
......................................................................
[WIP]configurators: move configurator functions into conf_utils
Change-Id: Id67a0f51adb2d543c0542a48c020d34a13d1f2ed
Signed-off-by: Yeela Kaplan <ykaplan(a)redhat.com>
---
M lib/vdsm/tool/Makefile.am
A lib/vdsm/tool/conf_utils.py
M vdsm.spec.in
3 files changed, 110 insertions(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/86/44286/1
diff --git a/lib/vdsm/tool/Makefile.am b/lib/vdsm/tool/Makefile.am
index c8979ca..c3ed0cc 100644
--- a/lib/vdsm/tool/Makefile.am
+++ b/lib/vdsm/tool/Makefile.am
@@ -35,7 +35,8 @@
dump_bonding_defaults.py \
dump_volume_chains.py \
nwfilter.py \
- configfile.py \
+ conf_utils.py \
+ configfile.py \
configurator.py \
register.py \
restore_nets.py \
diff --git a/lib/vdsm/tool/conf_utils.py b/lib/vdsm/tool/conf_utils.py
new file mode 100644
index 0000000..1308097
--- /dev/null
+++ b/lib/vdsm/tool/conf_utils.py
@@ -0,0 +1,107 @@
+# Copyright 2015 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+import os
+
+from .. configfile import (
+ ConfigFile,
+)
+
+from ... import utils
+
+if utils.isOvirtNode():
+ from ovirt.node.utils.fs import Config as NodeCfg
+
+
+def get_file(fname, files):
+ return files[fname]['path']
+
+
+def remove_conf(files, version):
+ for cfile, content in files.items():
+ content['removeConf'](content['path'], version)
+
+
+def add_section(content, version, vdsmConfiguration={}):
+ """
+ Add a 'configuration section by vdsm' part to a config file.
+ This section contains only keys not originally defined
+ The section headers will include the current configuration version.
+ """
+ configuration = {}
+ for fragment in content['fragments']:
+ if vdsmConfiguration:
+ if is_applicable(fragment, vdsmConfiguration):
+ configuration.update(fragment['content'])
+ else:
+ configuration.update(fragment['content'])
+ if configuration:
+ with open_config(content['path'], version) as conff:
+ for key, val in configuration.items():
+ conff.addEntry(key, val)
+
+
+def remove_section(path, version):
+ """
+ remove entire 'configuration section by vdsm' section.
+ section is removed regardless of it's version.
+ """
+ if os.path.exists(path):
+ with open_config(path, version) as conff:
+ conff.removeConf()
+
+
+def remove_file(content, vdsmConfiguration):
+ """
+ delete a file if it exists.
+ """
+ if utils.isOvirtNode():
+ NodeCfg().delete(content['path'])
+ else:
+ try:
+ os.unlink(content['path'])
+ except OSError as e:
+ if e.errno != errno.ENOENT:
+ raise
+
+
+def get_persisted_files(files):
+ """
+ get files where vdsm is expected to add a section.
+ """
+ return [
+ cfile['path'] for cfile in files.values()
+ if cfile['persisted']
+ ]
+
+
+def open_config(path, conf_version):
+ return ConfigFile(path, conf_version)
+
+
+def is_applicable(fragment, vdsmConfiguration):
+ """
+ Return true if 'fragment' should be included for current
+ configuration. An applicable fragment is a fragment who's list
+ of conditions are met according to vdsmConfiguration.
+ """
+ applyFragment = True
+ for key, booleanValue in fragment['conditions'].items():
+ if vdsmConfiguration[key] != booleanValue:
+ applyFragment = False
+ return applyFragment
diff --git a/vdsm.spec.in b/vdsm.spec.in
index 5020d8b..21e1b50 100644
--- a/vdsm.spec.in
+++ b/vdsm.spec.in
@@ -1094,6 +1094,7 @@
%{python_sitelib}/%{vdsm_name}/tool/dummybr.py*
%{python_sitelib}/%{vdsm_name}/tool/dump_bonding_defaults.py*
%{python_sitelib}/%{vdsm_name}/tool/nwfilter.py*
+%{python_sitelib}/%{vdsm_name}/tool/conf_utils.py*
%{python_sitelib}/%{vdsm_name}/tool/configurator.py*
%{python_sitelib}/%{vdsm_name}/tool/configurators/__init__*
%{python_sitelib}/%{vdsm_name}/tool/configurators/certificates.py*
--
To view, visit https://gerrit.ovirt.org/44286
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Id67a0f51adb2d543c0542a48c020d34a13d1f2ed
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Yeela Kaplan <ykaplan(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: contrib: Simple jsonrpc client
......................................................................
contrib: Simple jsonrpc client
This is a simple jsonrpc client for communicating with the jsonrpc
server from the command line.
Arguments
method one of the mehtods described in json schema
params optionl json object with message parameters
Examples
Calling method without arguements:
# jsonrpc Host.getVMList
{
"jsonrpc": "2.0",
"id": "0e043d83-294a-4d31-b1b6-6dc2f2747494",
"result": [
"b3f6fa00-b315-4ad4-8108-f73da817b5c5"
]
}
Calling method with arguements:
# jsonrpc VM.getStats '{"vmID": "b3f6fa00-b315-4ad4-8108-f73da817b5c5"}'
{
"jsonrpc": "2.0",
"id": "cefd25a3-6250-4123-8a56-d7047899e19e",
"result": [
{
"status": "Down",
"exitMessage": "Admin shut down from the engine",
"vmId": "b3f6fa00-b315-4ad4-8108-f73da817b5c5",
"exitReason": 6,
"timeOffset": "0",
"exitCode": 0
}
]
}
Requires stomp.py library:
https://pypi.python.org/pypi/stomp.py
Change-Id: Ia6273eabf6f3601602659d1e4e748d8025ae8084
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
A contrib/jsonrpc
1 file changed, 104 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/81/35181/1
diff --git a/contrib/jsonrpc b/contrib/jsonrpc
new file mode 100755
index 0000000..3080193
--- /dev/null
+++ b/contrib/jsonrpc
@@ -0,0 +1,104 @@
+#!/usr/bin/python
+#
+# Copyright 2014 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+"""
+jsonrpc-cli - Vdsm jsonrpc client
+"""
+
+import json
+import os
+import signal
+import sys
+import time
+import uuid
+import optparse
+
+import stomp
+
+# Copied from lib/vdsm/vdscli.py
+PKIDIR = '/etc/pki/vdsm'
+KEYFILE = os.path.join(PKIDIR, 'keys/vdsmkey.pem')
+CERTFILE = os.path.join(PKIDIR, 'certs/vdsmcert.pem')
+CACERT = os.path.join(PKIDIR, 'certs/cacert.pem')
+
+PORT = 54321
+DESTINATION = "/queue/_local/vdsm/requests"
+
+
+class Listener(stomp.ConnectionListener):
+
+ def on_error(self, headers, message):
+ print 'Error: %s' % message
+ terminate()
+
+ def on_message(self, headers, message):
+ msg = json.loads(message)
+ print json.dumps(msg, indent=4)
+ terminate()
+
+
+def main(args):
+ parser = option_parser()
+ options, args = parser.parse_args(args)
+ if not args:
+ parser.error("method required")
+
+ msg = {
+ "id": str(uuid.uuid4()),
+ "jsonrpc": "2.0",
+ "method": args[0]
+ }
+
+ if len(args) > 1:
+ msg["params"] = json.loads(args[1])
+
+ conn = stomp.Connection10(
+ host_and_ports=((options.host, PORT),),
+ use_ssl=True,
+ ssl_key_file=KEYFILE,
+ ssl_cert_file=CERTFILE,
+ ssl_ca_certs=CACERT)
+
+ conn.set_listener("", Listener())
+ conn.start()
+ conn.send(body=json.dumps(msg), destination=DESTINATION)
+
+ try:
+ signal.pause()
+ except KeyboardInterrupt:
+ pass
+
+ conn.disconnect()
+
+
+def option_parser():
+ parser = optparse.OptionParser(usage='%prog [options] method [params]')
+ parser.add_option("-a", "--host", dest="host",
+ help="host address (default localhost)")
+ parser.set_defaults(host="localhost")
+ return parser
+
+
+def terminate():
+ os.kill(os.getpid(), signal.SIGINT)
+
+
+if __name__ == "__main__":
+ main(sys.argv[1:])
--
To view, visit http://gerrit.ovirt.org/35181
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia6273eabf6f3601602659d1e4e748d8025ae8084
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Dan Kenigsberg has uploaded a new change for review.
Change subject: spbackends: simplify over-general evaluation
......................................................................
spbackends: simplify over-general evaluation
The removed code is too clever for the use case of only two values.
Change-Id: I775c4a6003a03c6b65bee3abc8559a88e3eaec01
Signed-off-by: Dan Kenigsberg <danken(a)redhat.com>
---
M vdsm/storage/spbackends.py
1 file changed, 5 insertions(+), 2 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/53/41253/1
diff --git a/vdsm/storage/spbackends.py b/vdsm/storage/spbackends.py
index 953e01e..f6ce02b 100644
--- a/vdsm/storage/spbackends.py
+++ b/vdsm/storage/spbackends.py
@@ -229,8 +229,11 @@
def setSpmStatus(self, lVer=None, spmId=None):
self.invalidateMetadata()
- metaParams = dict(filter(lambda kv: kv[1] is not None,
- ((PMDK_LVER, lVer), (PMDK_SPM_ID, spmId))))
+ metaParams = {}
+ if lVer is not None:
+ metaParams[PMDK_LVER] = lVer
+ if spmId is not None:
+ metaParams[PMDK_SPM_ID] = spmId
self._metadata.update(metaParams)
@unsecured
--
To view, visit https://gerrit.ovirt.org/41253
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I775c4a6003a03c6b65bee3abc8559a88e3eaec01
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: lvm: Fail loudly if called with unexptected input.
......................................................................
lvm: Fail loudly if called with unexptected input.
When creating pvs with the force option, we are very carefull to accept
only True. When using the jsonrpc transport, engine was sending "true"
and "false", causing the call to fail misteiously.
Now we are also carefull about rejecting invlid input, making debugging
easier.
Change-Id: If9e6754d4aa2efaf894a9309cfaa4595d710063b
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm/storage/lvm.py
1 file changed, 5 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/29/37329/1
diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py
index aa3c04b..549839a 100644
--- a/vdsm/storage/lvm.py
+++ b/vdsm/storage/lvm.py
@@ -724,6 +724,11 @@
else:
raise
+ # We must be very carefull here; any value execpt True or False is a user
+ # error.
+ if type(force) != bool:
+ raise ValueError("Invalid value for 'force': %r" % force)
+
if force is True:
options = ("-y", "-ff")
_initpvs_removeHolders()
--
To view, visit http://gerrit.ovirt.org/37329
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: If9e6754d4aa2efaf894a9309cfaa4595d710063b
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: hsm: Report vg name in getDeviceList
......................................................................
hsm: Report vg name in getDeviceList
Hosted engine needs the iscsi session info used by the hosted engine
storage domain. getDeviceList() seems to include the needed info, but it
does not report the vg name for each device, making it hard to match the
iscsi session info and the hosted engine storage domain.
We return now the vg name of each device, which seems to be useful info
regardless of hosted engine needs, and can be used on the engine side
for reconstructing host state or validating engine view vs host view.
Change-Id: I116714cb5143ea92f5cb54c3f80f895c07ada536
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm/rpc/vdsmapi-schema.json
M vdsm/storage/hsm.py
2 files changed, 11 insertions(+), 2 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/23/45823/1
diff --git a/vdsm/rpc/vdsmapi-schema.json b/vdsm/rpc/vdsmapi-schema.json
index e0e95b9..1b2a171 100644
--- a/vdsm/rpc/vdsmapi-schema.json
+++ b/vdsm/rpc/vdsmapi-schema.json
@@ -1473,6 +1473,10 @@
#
# @status: The device status (free/used/unknown)
#
+# @vgname: The LVM volume group name, if this device is used as
+# a physical volume. This is typically a storage domain
+# UUID.
+#
# Since: 4.10.0
#
# Notes: The value of @serial may be dependent on the current host so this
@@ -1490,7 +1494,8 @@
'pathstatus': ['BlockDevicePathInfo'],
'pathlist': ['IscsiSessionInfo'], 'logicalblocksize': 'uint',
'physicalblocksize': 'uint', 'partitioned': 'bool',
- 'pvsize': 'uint', 'status': 'BlockDeviceStatus'}}
+ 'pvsize': 'uint', 'status': 'BlockDeviceStatus',
+ 'vgname': 'str'}}
##
# @Host.getDeviceList:
diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py
index 1b8c064..32e16c3 100644
--- a/vdsm/storage/hsm.py
+++ b/vdsm/storage/hsm.py
@@ -2019,14 +2019,18 @@
pvuuid = pv.uuid
pvsize = pv.size
vguuid = pv.vg_uuid
+ vgname = pv.vg_name
else:
pvuuid = ""
pvsize = ""
vguuid = ""
+ vgname = ""
devInfo = {'GUID': dev.get("guid", ""), 'pvUUID': pvuuid,
'pvsize': str(pvsize),
- 'vgUUID': vguuid, 'vendorID': dev.get("vendor", ""),
+ 'vgUUID': vguuid,
+ 'vgname': vgname,
+ 'vendorID': dev.get("vendor", ""),
'productID': dev.get("product", ""),
'fwrev': dev.get("fwrev", ""),
"serial": dev.get("serial", ""),
--
To view, visit https://gerrit.ovirt.org/45823
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I116714cb5143ea92f5cb54c3f80f895c07ada536
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: utils: Consider sleep time in deadline calculation
......................................................................
utils: Consider sleep time in deadline calculation
This patches fixes 2 utils.retry() broken tests:
- The special case when deadline has reached when an operation was done.
Previously we use to sleep and perform another retry, now we bail out.
- The special case when deadline was not reached when an operation was
done, but we don't have time for sleep (sleeping will reach or exceed
the deadline). Previously we used to sleep and perform another retry,
now we bail out.
Change-Id: I7f41c6b21e3432159c13d46cfe75d1f6236cbb8c
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M lib/vdsm/utils.py
M tests/utilsTests.py
2 files changed, 3 insertions(+), 8 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/00/46400/1
diff --git a/lib/vdsm/utils.py b/lib/vdsm/utils.py
index caf4cc2..4a4b153 100644
--- a/lib/vdsm/utils.py
+++ b/lib/vdsm/utils.py
@@ -938,10 +938,7 @@
if tries in [0, None]:
tries = -1
- if timeout in [0, None]:
- timeout = -1
-
- startTime = monotonic_time()
+ deadline = monotonic_time() + timeout if timeout else None
while True:
tries -= 1
@@ -951,7 +948,7 @@
if tries == 0:
raise
- if (timeout > 0) and ((monotonic_time() - startTime) > timeout):
+ if deadline and monotonic_time() + sleep >= deadline:
raise
if stopCallback is not None and stopCallback():
diff --git a/tests/utilsTests.py b/tests/utilsTests.py
index d213de5..fd4f42e 100644
--- a/tests/utilsTests.py
+++ b/tests/utilsTests.py
@@ -44,7 +44,7 @@
from testlib import permutations, expandPermutations
from testlib import VdsmTestCase as TestCaseBase
from testValidation import checkSudo
-from testValidation import brokentest, stresstest
+from testValidation import stresstest
from multiprocessing import Process
EXT_SLEEP = "sleep"
@@ -86,7 +86,6 @@
# Make sure we had the proper amount of iterations before failing
self.assertEquals(counter[0], limit)
- @brokentest("deadline is not respected")
@MonkeyPatch(utils, 'monotonic_time', FakeTime(0))
@MonkeyPatch(time, 'sleep', fake_sleep)
def testTimeoutDeadlineReached(self):
@@ -104,7 +103,6 @@
timeout=3, sleep=1)
self.assertEqual(utils.monotonic_time.now, 3)
- @brokentest("sleep is not considered in deadline calculation")
@MonkeyPatch(utils, 'monotonic_time', FakeTime(0))
@MonkeyPatch(time, 'sleep', fake_sleep)
def testTimeoutNoTimeForSleep(self):
--
To view, visit https://gerrit.ovirt.org/46400
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7f41c6b21e3432159c13d46cfe75d1f6236cbb8c
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Adam Litke has uploaded a new change for review.
Change subject: Live Merge: Restore watermark tracking
......................................................................
Live Merge: Restore watermark tracking
Change-Id: I632f31e7795ec5d8c6f52a480116b14470c3163f
Signed-off-by: Adam Litke <alitke(a)redhat.com>
---
M vdsm/virt/vm.py
1 file changed, 108 insertions(+), 10 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/24/36924/1
diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py
index f22610d..09080b9 100644
--- a/vdsm/virt/vm.py
+++ b/vdsm/virt/vm.py
@@ -1512,12 +1512,94 @@
with self._confLock:
self.conf['timeOffset'] = newTimeOffset
+ def _getWriteWatermarks(self):
+ def pathToVolID(drive, path):
+ for vol in drive.volumeChain:
+ if os.path.realpath(vol['path']) == os.path.realpath(path):
+ return vol['volumeID']
+ raise LookupError("Unable to find VolumeID for path '%s'", path)
+
+ volAllocMap = {}
+ statsFlags = self._libvirtBackingChainStatsFlag()
+ conn = libvirtconnection.get()
+ blkStats = conn.domainListGetStats([self._dom._dom],
+ libvirt.VIR_DOMAIN_STATS_BLOCK,
+ statsFlags)[0][1]
+ for i in xrange(0, blkStats['block.count']):
+ name = blkStats['block.%i.name' % i]
+ try:
+ drive = self._findDriveByName(name)
+ except LookupError:
+ continue
+ if not drive.blockDev or drive.format != 'cow':
+ continue
+
+ try:
+ path = blkStats['block.%i.path' % i]
+ alloc = blkStats['block.%i.allocation' % i]
+ except KeyError as e:
+ self.log.debug("Block stats are missing expected key '%s', "
+ "skipping volume", e.args[0])
+ continue
+ volID = pathToVolID(drive, path)
+ volAllocMap[volID] = alloc
+ return volAllocMap
+
+ def _getLiveMergeExtendCandidates(self):
+ # The common case is that there are no active jobs.
+ if not self.conf['_blockJobs'].values():
+ return {}
+
+ candidates = {}
+ watermarks = self._getWriteWatermarks()
+ for job in self.conf['_blockJobs'].values():
+ try:
+ drive = self._findDriveByUUIDs(job['disk'])
+ except LookupError:
+ # After an active layer merge completes the vdsm metadata will
+ # be out of sync for a brief period. If we cannot find the old
+ # disk then it's safe to skip it.
+ continue
+
+ if not drive.blockDev:
+ continue
+
+ if job['strategy'] == 'commit':
+ volumeID = job['baseVolume']
+ else:
+ self.log.debug("Unrecognized merge strategy '%s'",
+ job['strategy'])
+ continue
+ res = self.cif.irs.getVolumeInfo(drive.domainID, drive.poolID,
+ drive.imageID, volumeID)
+ if res['status']['code'] != 0:
+ self.log.error("Unable to get the info of volume %s (domain: "
+ "%s image: %s)", volumeID, drive.domainID,
+ drive.imageID)
+ continue
+ volInfo = res['info']
+
+ if volInfo['format'].lower() != 'cow':
+ continue
+
+ if volumeID in watermarks:
+ self.log.debug("Adding live merge extension candidate: "
+ "volume=%s allocation=%i", volumeID,
+ watermarks[volumeID])
+ candidates[drive.imageID] = {
+ 'alloc': watermarks[volumeID],
+ 'physical': int(volInfo['truesize']),
+ 'capacity': int(volInfo['apparentsize']),
+ 'volumeID': volumeID}
+ else:
+ self.log.warning("No watermark info available for %s",
+ volumeID)
+ return candidates
+
def _getExtendCandidates(self):
ret = []
- # FIXME: mergeCandidates should be a dictionary of candidate volumes
- # once libvirt starts reporting watermark information for all volumes.
- mergeCandidates = {}
+ mergeCandidates = self._getLiveMergeExtendCandidates()
for drive in self._devices[hwclass.DISK]:
if not drive.blockDev or drive.format != 'cow':
continue
@@ -4771,6 +4853,14 @@
jobsRet[jobID] = entry
return jobsRet
+ def _libvirtBackingChainStatsFlag(self):
+ # Since libvirt 1.2.13, the virConnectGetAllDomainStats API will return
+ # block statistics for all volumes in the chain when using a new flag.
+ try:
+ return libvirt.VIR_CONNECT_GET_ALL_DOMAINS_STATS_BACKING
+ except AttributeError:
+ return 0
+
def merge(self, driveSpec, baseVolUUID, topVolUUID, bandwidth, jobUUID):
if not caps.getLiveMergeSupport():
self.log.error("Live merge is not supported on this host")
@@ -4815,6 +4905,8 @@
if res['info']['voltype'] == 'SHARED':
self.log.error("merge: Refusing to merge into a shared volume")
return errCode['mergeErr']
+ baseSize = int(res['info']['apparentsize'])
+ baseCow = bool(res['info']['format'].lower() == 'cow')
# Indicate that we expect libvirt to maintain the relative paths of
# backing files. This is necessary to ensure that a volume chain is
@@ -4865,13 +4957,19 @@
# blockCommit will cause data to be written into the base volume.
# Perform an initial extension to ensure there is enough space to
- # copy all the required data. Normally we'd use monitoring to extend
- # the volume on-demand but internal watermark information is not being
- # reported by libvirt so we must do the full extension up front. In
- # the worst case, we'll need to extend 'base' to the same size as 'top'
- # plus a bit more to accomodate additional writes to 'top' during the
- # live merge operation.
- self.extendDriveVolume(drive, baseVolUUID, topSize)
+ # copy all the required data. If libvirt supports monitoring of
+ # backing chain volumes, just extend by one chunk now and monitor
+ # during the rest of the operation. Otherwise, extend now to
+ # accomodate the worst case scenario: no intersection between the
+ # allocated blocks in the base volume and the top volume.
+ if drive.blockDev and baseCow:
+ if self._libvirtBackingChainStatsFlag():
+ self.extendDrivesIfNeeded()
+ else:
+ extendSize = baseSize + topSize
+ self.log.debug("Preemptively extending volume %s with size %i"
+ "(job: %s)", baseVolUUID, extendSize, jobUUID)
+ self.extendDriveVolume(drive, baseVolUUID, extendCurSize)
# Trigger the collection of stats before returning so that callers
# of getVmStats after this returns will see the new job
--
To view, visit http://gerrit.ovirt.org/36924
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I632f31e7795ec5d8c6f52a480116b14470c3163f
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: libvirtconnection: Replace assert with AssertionError
......................................................................
libvirtconnection: Replace assert with AssertionError
The code wrongly assumed that assert always exists. When running in
optimized mode, the check would be skipped, and instead of getting an
AssertionError, which is the expected error for programmer error
(starting the eventloop twice), we could get a confusing
RuntimeException or RuntimeError from Thread.start (depending on Python
version).
RuntimeError misused in the standard library for all kinds of errors
that do not have builtin errors. It is particularry bad option when used
for usage error.
Change-Id: Icf1564f81f4c1fbf77ccaff6d93c047a02d946da
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M lib/vdsm/libvirtconnection.py
1 file changed, 2 insertions(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/64/34364/1
diff --git a/lib/vdsm/libvirtconnection.py b/lib/vdsm/libvirtconnection.py
index 5430c82..009f8b7 100644
--- a/lib/vdsm/libvirtconnection.py
+++ b/lib/vdsm/libvirtconnection.py
@@ -37,7 +37,8 @@
self.__thread = None
def start(self):
- assert not self.run
+ if self.run:
+ raise AssertionError("EventLoop is running")
self.__thread = threading.Thread(target=self.__run,
name="libvirtEventLoop")
self.__thread.setDaemon(True)
--
To view, visit http://gerrit.ovirt.org/34364
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Icf1564f81f4c1fbf77ccaff6d93c047a02d946da
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: cache: Add caching decorator with invalidation
......................................................................
cache: Add caching decorator with invalidation
The new cache.memoized extends utils.memoized, adding invalidation
support.
Features added:
- An optional "validate" argument. This is a callable invoked each time
the memoized function is called. When the callable returns False, the
cache is invalidated.
- Memoized functions have an "invalidate" method, used to invalidate the
cache during testing.
- file_validator - invalidates the cache when a file changes.
Example usage:
from vdsm.cache import memoized, file_validator
@memoized(file_validator('/bigfile'))
def parse_bigfile():
# Expensive code processing '/bigfile' contents
Change-Id: I6dd8fb29d94286e3e3a3e29b8218501cbdc5c018
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M lib/vdsm/Makefile.am
A lib/vdsm/cache.py
M tests/Makefile.am
A tests/cacheTests.py
4 files changed, 366 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/09/34709/1
diff --git a/lib/vdsm/Makefile.am b/lib/vdsm/Makefile.am
index b862e71..6f0040d 100644
--- a/lib/vdsm/Makefile.am
+++ b/lib/vdsm/Makefile.am
@@ -23,6 +23,7 @@
dist_vdsmpylib_PYTHON = \
__init__.py \
+ cache.py \
compat.py \
define.py \
exception.py \
diff --git a/lib/vdsm/cache.py b/lib/vdsm/cache.py
new file mode 100644
index 0000000..9806e40
--- /dev/null
+++ b/lib/vdsm/cache.py
@@ -0,0 +1,98 @@
+#
+# Copyright 2014 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+
+import errno
+import os
+import functools
+
+
+def memoized(validate=None):
+ """
+ Return a caching decorator supporting invalidation.
+
+ The decorator accepts an optional validate callable, called each time the
+ memoized function is called. If the validate callable return True, the
+ memoized function will use the cache. If the validate callable return
+ False, the memoized cache is cleared.
+
+ The memoized function may accept multiple positional arguments. The
+ cache store the result for each combination of arguments. Functions with
+ kwargs are not supported.
+
+ Memoized functions have an "invalidate" method, used to invalidate the
+ memoized cache during testing.
+
+ To invalidate the cache when a file changes, use the file_validator from
+ this module.
+
+ Example usage:
+
+ from vdsm.cache import memoized, file_validator
+
+ @memoized(file_validator('/bigfile'))
+ def parse_bigfile():
+ # Expensive code processing '/bigfile' contents
+
+ """
+ def decorator(f):
+ cache = {}
+
+ @functools.wraps(f)
+ def wrapper(*args):
+ if validate is not None and not validate():
+ cache.clear()
+ try:
+ value = cache[args]
+ except KeyError:
+ value = cache[args] = f(*args)
+ return value
+
+ wrapper.invalidate = cache.clear
+ return wrapper
+
+ return decorator
+
+
+class file_validator(object):
+ """
+ I'm a validator returning False when a file has changed since the last
+ validation.
+ """
+
+ UNKNOWN = 0
+ MISSING = 1
+
+ def __init__(self, path):
+ self.path = path
+ self.stats = self.UNKNOWN
+
+ def __call__(self):
+ try:
+ stats = os.stat(self.path)
+ except OSError as e:
+ if e.errno != errno.ENOENT:
+ raise
+ stats = self.MISSING
+ else:
+ stats = stats.st_ino, stats.st_size, stats.st_mtime
+ if stats != self.stats:
+ self.stats = stats
+ return False
+ return True
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 36a1cdd..6fa7e64 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -26,6 +26,7 @@
alignmentScanTests.py \
blocksdTests.py \
bridgeTests.py \
+ cacheTests.py \
cPopenTests.py \
capsTests.py \
clientifTests.py \
diff --git a/tests/cacheTests.py b/tests/cacheTests.py
new file mode 100644
index 0000000..8927b39
--- /dev/null
+++ b/tests/cacheTests.py
@@ -0,0 +1,266 @@
+#
+# Copyright 2014 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+# 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+
+import os
+from vdsm.cache import memoized
+from vdsm.cache import file_validator
+from testlib import VdsmTestCase
+from testlib import namedTemporaryDir
+
+
+class Validator(object):
+ """ I'm a callable returning a boolean value (self.valid) """
+
+ def __init__(self):
+ self.valid = True
+ self.count = 0
+
+ def __call__(self):
+ self.count += 1
+ return self.valid
+
+
+class Accessor(object):
+ """ I'm recording how many times a dict was accessed. """
+
+ def __init__(self, d):
+ self.d = d
+ self.count = 0
+
+ def get(self, key):
+ self.count += 1
+ return self.d[key]
+
+
+class MemoizedTests(VdsmTestCase):
+
+ def setUp(self):
+ self.values = {'a': 0, 'b': 10, ('a',): 20, ('a', 'b'): 30}
+
+ def test_no_args(self):
+ accessor = Accessor(self.values)
+
+ @memoized()
+ def func(key):
+ return accessor.get(key)
+
+ # Fill the cache
+ self.assertEqual(func('a'), self.values['a'])
+ self.assertEqual(func('b'), self.values['b'])
+ self.assertEqual(accessor.count, 2)
+
+ # Values served now from the cache
+ self.assertEqual(func('a'), self.values['a'])
+ self.assertEqual(func('b'), self.values['b'])
+ self.assertEqual(accessor.count, 2)
+
+ def test_validation(self):
+ accessor = Accessor(self.values)
+ validator = Validator()
+
+ @memoized(validator)
+ def func(key):
+ return accessor.get(key)
+
+ # Fill the cache
+ self.assertEqual(func('a'), self.values['a'])
+ self.assertEqual(func('b'), self.values['b'])
+ self.assertEqual(accessor.count, 2)
+ self.assertEqual(validator.count, 2)
+
+ # Values served now from the cache
+ self.assertEqual(func('a'), self.values['a'])
+ self.assertEqual(func('b'), self.values['b'])
+ self.assertEqual(accessor.count, 2)
+ self.assertEqual(validator.count, 4)
+
+ # Values has changed
+ self.values['a'] += 1
+ self.values['b'] += 1
+
+ # Next call should clear the cache
+ validator.valid = False
+ self.assertEqual(func('a'), self.values['a'])
+ self.assertEqual(accessor.count, 3)
+ self.assertEqual(validator.count, 5)
+
+ # Next call should add next value to cache
+ validator.valid = True
+ self.assertEqual(func('b'), self.values['b'])
+ self.assertEqual(accessor.count, 4)
+ self.assertEqual(validator.count, 6)
+
+ # Values served now from the cache
+ self.assertEqual(func('a'), self.values['a'])
+ self.assertEqual(func('b'), self.values['b'])
+ self.assertEqual(accessor.count, 4)
+ self.assertEqual(validator.count, 8)
+
+ def test_raise_errors_in_memoized_func(self):
+ accessor = Accessor(self.values)
+ validator = Validator()
+
+ @memoized(validator)
+ def func(key):
+ return accessor.get(key)
+
+ # First run should fail, second shold fill the cache
+ self.assertRaises(KeyError, func, 'no such key')
+ self.assertEqual(func('a'), self.values['a'])
+ self.assertEqual(accessor.count, 2)
+ self.assertEqual(validator.count, 2)
+
+ def test_multiple_args(self):
+ accessor = Accessor(self.values)
+
+ @memoized()
+ def func(*args):
+ return accessor.get(args)
+
+ # Fill the cache
+ self.assertEqual(func('a'), self.values[('a',)])
+ self.assertEqual(func('a', 'b'), self.values[('a', 'b')])
+ self.assertEqual(accessor.count, 2)
+
+ # Values served now from the cache
+ self.assertEqual(func('a'), self.values[('a',)])
+ self.assertEqual(func('a', 'b'), self.values[('a', 'b')])
+ self.assertEqual(accessor.count, 2)
+
+ def test_kwargs_not_supported(self):
+ @memoized()
+ def func(a=None, b=None):
+ pass
+ self.assertRaises(TypeError, func, a=1, b=2)
+
+ def test_invalidate(self):
+ accessor = Accessor(self.values)
+
+ @memoized()
+ def func(key):
+ return accessor.get(key)
+
+ self.assertEqual(func('a'), self.values['a'])
+ self.assertEqual(accessor.count, 1)
+
+ func.invalidate()
+
+ self.assertEqual(func('a'), self.values['a'])
+ self.assertEqual(accessor.count, 2)
+
+
+class FileValidatorTests(VdsmTestCase):
+
+ def test_no_file(self):
+ with namedTemporaryDir() as tempdir:
+ path = os.path.join(tempdir, 'data')
+ validator = file_validator(path)
+
+ # Must be False so memoise call the decorated function
+ self.assertEqual(validator(), False)
+
+ # Since file state did not change, must remain True
+ self.assertEqual(validator(), True)
+
+ def test_file_created(self):
+ with namedTemporaryDir() as tempdir:
+ path = os.path.join(tempdir, 'data')
+ validator = file_validator(path)
+
+ self.assertEqual(validator(), False)
+ self.assertEqual(validator(), True)
+
+ with open(path, 'w') as f:
+ f.write('data')
+
+ self.assertEqual(validator(), False)
+ self.assertEqual(validator(), True)
+
+ def test_file_removed(self):
+ with namedTemporaryDir() as tempdir:
+ path = os.path.join(tempdir, 'data')
+ validator = file_validator(path)
+
+ with open(path, 'w') as f:
+ f.write('data')
+
+ self.assertEqual(validator(), False)
+ self.assertEqual(validator(), True)
+
+ os.unlink(path)
+
+ self.assertEqual(validator(), False)
+ self.assertEqual(validator(), True)
+
+ def test_size_changed(self):
+ with namedTemporaryDir() as tempdir:
+ path = os.path.join(tempdir, 'data')
+ validator = file_validator(path)
+ data = 'old data'
+ with open(path, 'w') as f:
+ f.write(data)
+
+ self.assertEqual(validator(), False)
+ self.assertEqual(validator(), True)
+
+ with open(path, 'w') as f:
+ f.write(data + ' new data')
+
+ self.assertEqual(validator(), False)
+ self.assertEqual(validator(), True)
+
+ def test_mtime_changed(self):
+ with namedTemporaryDir() as tempdir:
+ path = os.path.join(tempdir, 'data')
+ validator = file_validator(path)
+ data = 'old data'
+ with open(path, 'w') as f:
+ f.write(data)
+
+ self.assertEqual(validator(), False)
+ self.assertEqual(validator(), True)
+
+ # Fake timestamp change, as timestamp resolution may not be good
+ # enough when comparing changes during the test.
+ atime = mtime = os.path.getmtime(path) + 1
+ os.utime(path, (atime, mtime))
+
+ self.assertEqual(validator(), False)
+ self.assertEqual(validator(), True)
+
+ def test_ino_changed(self):
+ with namedTemporaryDir() as tempdir:
+ path = os.path.join(tempdir, 'data')
+ validator = file_validator(path)
+ data = 'old data'
+ with open(path, 'w') as f:
+ f.write(data)
+
+ self.assertEqual(validator(), False)
+ self.assertEqual(validator(), True)
+
+ tmp = path + '.tmp'
+ with open(tmp, 'w') as f:
+ f.write(data)
+ os.rename(tmp, path)
+
+ self.assertEqual(validator(), False)
+ self.assertEqual(validator(), True)
--
To view, visit http://gerrit.ovirt.org/34709
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I6dd8fb29d94286e3e3a3e29b8218501cbdc5c018
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: sdc: Rename method to make it less confusing
......................................................................
sdc: Rename method to make it less confusing
StorageCache.refresh() does not do any refresh, but clearing the domain
cahce and invlidating lvm cache (actually clearing it). Rename to
clear() to reflect what it does.
Change-Id: I2c67ae0ddc98857e406fec62be0cbcf817213236
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm/storage/sdc.py
M vdsm/storage/sp.py
2 files changed, 3 insertions(+), 3 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/16/47916/1
diff --git a/vdsm/storage/sdc.py b/vdsm/storage/sdc.py
index ecb9708..a26c05a 100644
--- a/vdsm/storage/sdc.py
+++ b/vdsm/storage/sdc.py
@@ -181,7 +181,7 @@
return uuids
- def refresh(self):
+ def clear(self):
with self._syncroot:
lvm.invalidateCache()
self.__domainCache.clear()
diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py
index b8fd8f3..6ecaf8d 100644
--- a/vdsm/storage/sp.py
+++ b/vdsm/storage/sp.py
@@ -627,7 +627,7 @@
self.id = hostID
# Make sure SDCache doesn't have stale data (it can be in case of FC)
sdCache.invalidateStorage()
- sdCache.refresh()
+ sdCache.clear()
# Rebuild whole Pool
self.__rebuild(msdUUID=msdUUID, masterVersion=masterVersion)
self.__createMailboxMonitor()
@@ -1245,7 +1245,7 @@
'msdUUID' - master storage domain UUID
"""
sdCache.invalidateStorage()
- sdCache.refresh()
+ sdCache.clear()
self.__rebuild(msdUUID=msdUUID, masterVersion=masterVersion)
def updateVM(self, vmList, sdUUID):
--
To view, visit https://gerrit.ovirt.org/47916
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2c67ae0ddc98857e406fec62be0cbcf817213236
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: scsi: Scan only the required domain type
......................................................................
scsi: Scan only the required domain type
We used to perform both iSCSI and FCP rescan when creating or editing a
storage domain, connecting to storage server, getting vg and storage
domain list and more.
The unneeded rescan is typically fast, but if a storage server or device
is not accessible, a SCSI rescan may block for couple of minutes,
leading to unwanted blocking of unrelated storage threads. This is
particularly bad when you are interested only in one domain type, but
the host get stuck scanning the other type.
To improve storage domain isolation, we use the specified storage type
to perform a rescan only of the relevant type. If storage type was not
specified, we scan both ISCSI and FCP keeping the old behavior.
Change-Id: Ic32cd683020e94df016dd77b19ae3eb7317c5554
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm/storage/hsm.py
M vdsm/storage/multipath.py
M vdsm/storage/sdc.py
3 files changed, 25 insertions(+), 17 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/24/45824/1
diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py
index 1b8c064..541b699 100644
--- a/vdsm/storage/hsm.py
+++ b/vdsm/storage/hsm.py
@@ -1984,15 +1984,16 @@
return dict(devList=devices)
def _getDeviceList(self, storageType=None, guids=(), checkStatus=True):
- sdCache.refreshStorage()
- typeFilter = lambda dev: True
- if storageType:
- if sd.storageType(storageType) == sd.type2name(sd.ISCSI_DOMAIN):
- typeFilter = \
- lambda dev: multipath.devIsiSCSI(dev.get("devtype"))
- elif sd.storageType(storageType) == sd.type2name(sd.FCP_DOMAIN):
- typeFilter = \
- lambda dev: multipath.devIsFCP(dev.get("devtype"))
+ domType = sd.storageType(storageType) if storageType else None
+
+ sdCache.refreshStorage(domType)
+
+ if domType == sd.ISCSI_DOMAIN:
+ typeFilter = lambda dev: multipath.devIsiSCSI(dev.get("devtype"))
+ elif domType == sd.FCP_DOMAIN:
+ typeFilter = lambda dev: multipath.devIsFCP(dev.get("devtype"))
+ else:
+ typeFilter = lambda dev: True
devices = []
pvs = {}
@@ -2470,7 +2471,7 @@
# while the VDSM was not connected, we need to
# call refreshStorage.
if domType in (sd.FCP_DOMAIN, sd.ISCSI_DOMAIN):
- sdCache.refreshStorage()
+ sdCache.refreshStorage(domType)
try:
doms = self.__prefetchDomains(domType, conObj)
except:
@@ -2864,7 +2865,8 @@
"""
vars.task.setDefaultException(
se.StorageDomainActionError("spUUID: %s" % spUUID))
- sdCache.refreshStorage()
+ domType = sd.storageType(storageType) if storageType else None
+ sdCache.refreshStorage(domType)
if spUUID and spUUID != volume.BLANK_UUID:
domList = self.getPool(spUUID).getDomains()
domains = domList.keys()
@@ -2925,7 +2927,8 @@
:rtype: dict
"""
vars.task.setDefaultException(se.VolumeGroupActionError())
- sdCache.refreshStorage()
+ domType = sd.storageType(storageType) if storageType else None
+ sdCache.refreshStorage(domType)
# getSharedLock(connectionsResource...)
vglist = []
vgs = self.__getVGsInfo()
diff --git a/vdsm/storage/multipath.py b/vdsm/storage/multipath.py
index ad81d2d..32deb98 100644
--- a/vdsm/storage/multipath.py
+++ b/vdsm/storage/multipath.py
@@ -39,6 +39,7 @@
import misc
import iscsi
import devicemapper
+import sd
DEV_ISCSI = "iSCSI"
DEV_FCP = "FCP"
@@ -61,7 +62,7 @@
""" multipath operation failed """
-def rescan():
+def rescan(domType=None):
"""
Forces multipath daemon to rescan the list of available devices and
refresh the mapping table. New devices can be found under /dev/mapper
@@ -70,8 +71,12 @@
"""
# First rescan iSCSI and FCP connections
- iscsi.rescan()
- hba.rescan()
+
+ if domType in (None, sd.ISCSI_DOMAIN):
+ iscsi.rescan()
+
+ if domType in (None, sd.FCP_DOMAIN):
+ hba.rescan()
# Now let multipath daemon pick up new devices
misc.execCmd([constants.EXT_MULTIPATH], sudo=True)
diff --git a/vdsm/storage/sdc.py b/vdsm/storage/sdc.py
index ecb9708..273c5c0 100644
--- a/vdsm/storage/sdc.py
+++ b/vdsm/storage/sdc.py
@@ -77,10 +77,10 @@
self.__staleStatus = self.STORAGE_STALE
@misc.samplingmethod
- def refreshStorage(self):
+ def refreshStorage(self, domType=None):
self.__staleStatus = self.STORAGE_REFRESHING
- multipath.rescan()
+ multipath.rescan(domType)
multipath.resize_devices()
lvm.invalidateCache()
--
To view, visit https://gerrit.ovirt.org/45824
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic32cd683020e94df016dd77b19ae3eb7317c5554
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Yeela Kaplan has uploaded a new change for review.
Change subject: [WIP]fencenode: split fenceNode into its own module
......................................................................
[WIP]fencenode: split fenceNode into its own module
Change-Id: I7bd5e7246cf6da21e355849014a8fc71d5edbde6
Signed-off-by: Yeela Kaplan <ykaplan(a)redhat.com>
---
M debian/vdsm-python.install
M lib/vdsm/Makefile.am
A lib/vdsm/fencenode.py
M vdsm.spec.in
M vdsm/API.py
5 files changed, 120 insertions(+), 80 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/97/43597/1
diff --git a/debian/vdsm-python.install b/debian/vdsm-python.install
index d9402c3..901edb6 100644
--- a/debian/vdsm-python.install
+++ b/debian/vdsm-python.install
@@ -10,6 +10,7 @@
./usr/lib/python2.7/dist-packages/vdsm/exception.py
./usr/lib/python2.7/dist-packages/vdsm/executor.py
./usr/lib/python2.7/dist-packages/vdsm/ipwrapper.py
+./usr/lib/python2.7/dist-packages/vdsm/fencenode.py
./usr/lib/python2.7/dist-packages/vdsm/jsonrpcvdscli.py
./usr/lib/python2.7/dist-packages/vdsm/libvirtconnection.py
./usr/lib/python2.7/dist-packages/vdsm/netconfpersistence.py
diff --git a/lib/vdsm/Makefile.am b/lib/vdsm/Makefile.am
index 95e236f..875507c 100644
--- a/lib/vdsm/Makefile.am
+++ b/lib/vdsm/Makefile.am
@@ -30,6 +30,7 @@
exception.py \
executor.py \
ipwrapper.py \
+ fencenode.py \
jsonrpcvdscli.py \
libvirtconnection.py \
netconfpersistence.py \
diff --git a/lib/vdsm/fencenode.py b/lib/vdsm/fencenode.py
new file mode 100644
index 0000000..3456b2f
--- /dev/null
+++ b/lib/vdsm/fencenode.py
@@ -0,0 +1,100 @@
+#
+# Copyright 2015 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+
+import logging
+import os
+import signal
+
+import utils
+import constants
+from define import doneCode, errCode
+
+
+(a)utils.traceback(on=logging.name)
+def fence(script, data):
+ # non-status actions are sent asyncronously. deathSignal is set to
+ # make sure that no stray fencing scripts are left behind if Vdsm
+ # crashes.
+ rc, out, err = utils.execCmd(
+ [script], deathSignal=signal.SIGTERM, data=data)
+ logging.debug('rc %s data %s out %s err %s', rc,
+ hidePasswd(data), out, err)
+ return rc, out, err
+
+
+def hidePasswd(text):
+ cleantext = ''
+ for line in text.splitlines(True):
+ if line.startswith('passwd='):
+ line = 'passwd=XXXX\n'
+ cleantext += line
+ return cleantext
+
+
+def fenceNode(addr, port, agent, username, password, action,
+ secure=False, options='', policy=None):
+
+ logging.debug('fenceNode(addr=%s,port=%s,agent=%s,user=%s,passwd=%s,'
+ 'action=%s,secure=%s,options=%s,policy=%s)',
+ addr, port, agent, username, password, action, secure,
+ options, policy)
+
+ if action not in ('status', 'on', 'off', 'reboot'):
+ raise ValueError('illegal action ' + action)
+
+ script = constants.EXT_FENCE_PREFIX + agent
+
+ data = ('agent=fence_%s\nipaddr=%s\nlogin=%s\naction=%s\n'
+ 'passwd=%s\n') % (agent, addr, username, action, password.value)
+ if port != '':
+ data += 'port=%s\n' % (port,)
+ if utils.tobool(secure):
+ data += 'secure=yes\n'
+ data += options
+
+ try:
+ rc, out, err = fence(script, data)
+ except OSError as e:
+ if e.errno == os.errno.ENOENT:
+ return errCode['fenceAgent']
+ raise
+
+ logging.debug('rc %s data %s out %s err %s',
+ rc, hidePasswd(data), out, err)
+
+ if not 0 <= rc <= 2:
+ return {'status': {'code': 1,
+ 'message': out + err}}
+
+ message = doneCode['message']
+ if action == 'status':
+ if rc == 0:
+ power = 'on'
+ elif rc == 2:
+ power = 'off'
+ else:
+ power = 'unknown'
+ message = out + err
+ return {'status': {'code': 0, 'message': message},
+ 'power': power}
+ if rc != 0:
+ message = out + err
+ return {'status': {'code': rc, 'message': message},
+ 'power': 'unknown', 'operationStatus': 'initiated'}
diff --git a/vdsm.spec.in b/vdsm.spec.in
index 966a974..9e69162 100644
--- a/vdsm.spec.in
+++ b/vdsm.spec.in
@@ -1071,6 +1071,7 @@
%{python_sitelib}/%{vdsm_name}/exception.py*
%{python_sitelib}/%{vdsm_name}/executor.py*
%{python_sitelib}/%{vdsm_name}/ipwrapper.py*
+%{python_sitelib}/%{vdsm_name}/fencenode.py*
%{python_sitelib}/%{vdsm_name}/jsonrpcvdscli.py*
%{python_sitelib}/%{vdsm_name}/libvirtconnection.py*
%{python_sitelib}/%{vdsm_name}/netinfo.py*
diff --git a/vdsm/API.py b/vdsm/API.py
index 0a93d2a..0719926 100644
--- a/vdsm/API.py
+++ b/vdsm/API.py
@@ -21,8 +21,6 @@
# pylint: disable=R0904
from contextlib import contextmanager
-import os
-import signal
import six
import sys
import time
@@ -38,6 +36,7 @@
from clientIF import clientIF
from vdsm import netinfo
from vdsm import constants
+from vdsm import fencenode
import storage.misc
import storage.clusterlock
import storage.volume
@@ -1187,6 +1186,11 @@
APIBase.__init__(self)
# General Host functions
+ def ping(self):
+ "Ping the server. Useful for tests"
+ updateTimestamp()
+ return {'status': doneCode}
+
def fenceNode(self, addr, port, agent, username, password, action,
secure=False, options='', policy=None):
"""Send a fencing command to a remote node.
@@ -1194,44 +1198,24 @@
agent is one of (rsa, ilo, drac5, ipmilan, etc)
action can be one of (status, on, off, reboot)."""
- @utils.traceback(on=self.log.name)
- def fence(script, inp):
- # non-status actions are sent asyncronously. deathSignal is set to
- # make sure that no stray fencing scripts are left behind if Vdsm
- # crashes.
- rc, out, err = utils.execCmd(
- [script], deathSignal=signal.SIGTERM,
- data=inp)
- self.log.debug('rc %s inp %s out %s err %s', rc,
- hidePasswd(inp), out, err)
- return rc, out, err
-
- def hidePasswd(text):
- cleantext = ''
- for line in text.splitlines(True):
- if line.startswith('passwd='):
- line = 'passwd=XXXX\n'
- cleantext += line
- return cleantext
-
def should_fence(policy):
# skip fence execution if map of storage domains with host id is
# entered and at least one storage domain connection from host is
# alive
if policy is None:
- self.log.debug('No policy specified')
+ logging.debug('No policy specified')
return True
hostIdMap = policy.get('storageDomainHostIdMap')
if not hostIdMap:
- self.log.warning('Invalid policy specified')
+ logging.warning('Invalid policy specified')
return True
result = self._irs.getHostLeaseStatus(hostIdMap)
- if result['status']['code'] != 0:
- self.log.error(
- "Error getting host lease status, error code '%s'",
- result['status']['code'])
+ rc = result['status']['code']
+ if rc != 0:
+ logging.error(
+ "Error getting host lease status, error code '%s'", rc)
return True
# HOST_STATUS_LIVE means that host renewed its lease in last 80
@@ -1239,65 +1223,18 @@
# fencing, even when it's unreachable from engine
for sd, status in result['domains'].iteritems():
if status == storage.clusterlock.HOST_STATUS_LIVE:
- self.log.debug("Host has live lease on '%s'", sd)
+ logging.debug("Host has live lease on '%s'", sd)
return False
- self.log.debug("Host doesn't have any live lease")
+ logging.debug("Host doesn't have any live lease")
return True
- self.log.debug('fenceNode(addr=%s,port=%s,agent=%s,user=%s,passwd=%s,'
- 'action=%s,secure=%s,options=%s,policy=%s)',
- addr, port, agent, username, password, action, secure,
- options, policy)
-
- if action not in ('status', 'on', 'off', 'reboot'):
- raise ValueError('illegal action ' + action)
-
if action != 'status' and not should_fence(policy):
- self.log.debug("Skipping execution of action '%s'", action)
+ logging.debug("Skipping execution of action '%s'", action)
return {'status': doneCode, 'operationStatus': 'skipped'}
- script = constants.EXT_FENCE_PREFIX + agent
-
- inp = ('agent=fence_%s\nipaddr=%s\nlogin=%s\naction=%s\n'
- 'passwd=%s\n') % (agent, addr, username, action, password.value)
- if port != '':
- inp += 'port=%s\n' % (port,)
- if utils.tobool(secure):
- inp += 'secure=yes\n'
- inp += options
-
- try:
- rc, out, err = fence(script, inp)
- except OSError as e:
- if e.errno == os.errno.ENOENT:
- return errCode['fenceAgent']
- raise
- self.log.debug('rc %s in %s out %s err %s', rc,
- hidePasswd(inp), out, err)
- if not 0 <= rc <= 2:
- return {'status': {'code': 1,
- 'message': out + err}}
- message = doneCode['message']
- if action == 'status':
- if rc == 0:
- power = 'on'
- elif rc == 2:
- power = 'off'
- else:
- power = 'unknown'
- message = out + err
- return {'status': {'code': 0, 'message': message},
- 'power': power}
- if rc != 0:
- message = out + err
- return {'status': {'code': rc, 'message': message},
- 'power': 'unknown', 'operationStatus': 'initiated'}
-
- def ping(self):
- "Ping the server. Useful for tests"
- updateTimestamp()
- return {'status': doneCode}
+ return fencenode.fenceNode(addr, port, agent, username, password,
+ action, secure, options, policy)
def getCapabilities(self):
"""
--
To view, visit https://gerrit.ovirt.org/43597
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7bd5e7246cf6da21e355849014a8fc71d5edbde6
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Yeela Kaplan <ykaplan(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: qemuimg: Memoize _supports_qcow2_compat
......................................................................
qemuimg: Memoize _supports_qcow2_compat
We used to run qemu-img twice when creating or converting qcow2 images.
The first run check if qemu-img supports the qcow2 "comapt" option, and
the second run uses the result to format the qemu-img command.
Now we run qemu-img once for "create" and "convert" to learn about its
capabilities, and use the cached value on the next runs. If qemu-img
executable is modified, we drop the cache, in case a new version was
installed with different capabilities.
Change-Id: Ic63f5e8c06993df8e4066bf7ac2dabfb4b4bdbfb
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M lib/vdsm/qemuimg.py
M tests/qemuimgTests.py
2 files changed, 8 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/11/34711/1
diff --git a/lib/vdsm/qemuimg.py b/lib/vdsm/qemuimg.py
index cf428b2..f295acc 100644
--- a/lib/vdsm/qemuimg.py
+++ b/lib/vdsm/qemuimg.py
@@ -23,6 +23,7 @@
import signal
from . import utils
+from . import cache
_qemuimg = utils.CommandPath("qemu-img",
"/usr/bin/qemu-img",) # Fedora, EL6
@@ -221,6 +222,7 @@
raise QImgError(rc, out, err)
+(a)cache.memoized(cache.file_validator(_qemuimg.cmd))
def _supports_qcow2_compat(command):
"""
qemu-img "create" and "convert" commands support a "compat" option in
diff --git a/tests/qemuimgTests.py b/tests/qemuimgTests.py
index 813a497..a642374 100644
--- a/tests/qemuimgTests.py
+++ b/tests/qemuimgTests.py
@@ -118,6 +118,9 @@
class CreateTests(TestCaseBase):
+ def setUp(self):
+ qemuimg._supports_qcow2_compat.invalidate()
+
def test_no_format(self):
def create(cmd, **kw):
expected = [QEMU_IMG, 'create', 'image']
@@ -161,6 +164,9 @@
class ConvertTests(TestCaseBase):
+ def setUp(self):
+ qemuimg._supports_qcow2_compat.invalidate()
+
def test_no_format(self):
def convert(cmd, **kw):
expected = [QEMU_IMG, 'convert', '-t', 'none', 'src', 'dst']
--
To view, visit http://gerrit.ovirt.org/34711
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic63f5e8c06993df8e4066bf7ac2dabfb4b4bdbfb
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Federico Simoncelli has uploaded a new change for review.
Change subject: vdsm: add eventfd and EventFile synchronization
......................................................................
vdsm: add eventfd and EventFile synchronization
Change-Id: I0d237f13c42b1f4505c90d30c6d3c3ecbd1e9fa7
Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com>
---
M lib/vdsm/Makefile.am
A lib/vdsm/eventfd.py
M tests/Makefile.am
A tests/eventfdTests.py
4 files changed, 251 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/87/33687/1
diff --git a/lib/vdsm/Makefile.am b/lib/vdsm/Makefile.am
index 4bebf28..e712cad 100644
--- a/lib/vdsm/Makefile.am
+++ b/lib/vdsm/Makefile.am
@@ -25,6 +25,7 @@
__init__.py \
compat.py \
define.py \
+ eventfd.py \
exception.py \
ipwrapper.py \
libvirtconnection.py \
diff --git a/lib/vdsm/eventfd.py b/lib/vdsm/eventfd.py
new file mode 100644
index 0000000..b2a7084
--- /dev/null
+++ b/lib/vdsm/eventfd.py
@@ -0,0 +1,140 @@
+#
+# Copyright 2014 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+
+"""\
+This module provides the support for eventfd(2).
+
+More information about eventfd and usage examples can be found in the
+eventfd(2) man page.
+
+The EventFile class provides a single synchronization object exposing
+the python Event interface and associated eventfds.
+
+The eventfd() context manager returns a file descriptor that can be
+used to provide the event notice to select, poll and epoll, e.g.
+
+ import os
+ import sys
+ import select
+ import threading
+ import time
+ from vdsm.eventfd import EventFile, DATASIZE
+
+ e = EventFile()
+ p = select.epoll()
+
+ threading.Timer(5, e.set).start()
+
+ with e.eventfd() as efd:
+ p.register(efd, select.EPOLLIN)
+ p.register(sys.stdin.fileno(), select.EPOLLIN)
+
+ print "Echoing lines until event is received"
+ event_received = False
+
+ while not event_received:
+ for fileno, event in p.poll():
+ if not event & select.EPOLLIN:
+ continue
+
+ if fileno == efd:
+ os.read(efd, DATASIZE)
+ event_received = True
+ elif fileno == sys.stdin.fileno():
+ print os.read(sys.stdin.fileno(), 1024),
+
+ print "Event received!"
+
+
+The Event set() semantic is preserved in the eventfd context manager:
+if the event is set then the eventfd already contains the notification.
+This is both to maintain the semantic and to avoid possible races as:
+
+ if not e.is_set():
+ with e.eventfd() as efd:
+ ...
+"""
+
+import os
+import ctypes
+import threading
+
+from contextlib import contextmanager
+
+_libc = ctypes.CDLL('libc.so.6', use_errno=True)
+
+EFD_NONBLOCK = os.O_NONBLOCK
+EFD_CLOEXEC = 02000000 # os.O_CLOEXEC in python 3.3
+EFD_SEMAPHORE = 00000001
+
+DATASIZE = ctypes.sizeof(ctypes.c_ulonglong)
+
+
+def eventfd(initval, flags):
+ return _libc.eventfd(initval, flags)
+
+
+class EventFile(object):
+ def __init__(self, event=None):
+ self.__lock = threading.Lock()
+ self.__fds = set()
+ self.__event = event or threading.Event()
+
+ @staticmethod
+ def __fire_event(fd):
+ os.write(fd, ctypes.c_ulonglong(1))
+
+ def open_eventfd(self):
+ with self.__lock:
+ fd = eventfd(0, 0)
+
+ self.__fds.add(fd)
+
+ if self.__event.is_set():
+ self.__fire_event(fd)
+
+ return fd
+
+ @contextmanager
+ def eventfd(self):
+ fd = self.open_eventfd()
+
+ yield fd
+
+ with self.__lock:
+ self.__fds.remove(fd)
+ os.close(fd)
+
+ def isSet(self):
+ return self.__event.isSet()
+
+ is_set = isSet
+
+ def set(self):
+ with self.__lock:
+ self.__event.set()
+ for fd in self.__fds:
+ self.__fire_event(fd)
+
+ def clear(self):
+ self.__event.clear()
+
+ def wait(self, timeout=None, balancing=True):
+ self.__event.wait(timeout)
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 449d7b1..120712e 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -31,6 +31,7 @@
clientifTests.py \
configNetworkTests.py \
domainMonitorTests.py \
+ eventfdTests.py \
fileVolumeTests.py \
fileUtilTests.py \
fuserTests.py \
diff --git a/tests/eventfdTests.py b/tests/eventfdTests.py
new file mode 100644
index 0000000..be15248
--- /dev/null
+++ b/tests/eventfdTests.py
@@ -0,0 +1,109 @@
+#
+# Copyright 2014 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+
+import os
+import select
+from vdsm.eventfd import EventFile, DATASIZE
+from nose.tools import timed, raises, TimeExpired
+
+TEST_TIMEOUT = 1
+WAIT_TIMEOUT = 2
+
+
+def test_set():
+ e = EventFile()
+ e.set()
+ assert e.is_set()
+ assert e.isSet()
+
+
+def text_clear():
+ e = EventFile()
+ e.set()
+ assert e.is_set()
+ e.clear()
+ assert not e.is_set()
+
+
+@timed(TEST_TIMEOUT)
+def test_wait_set():
+ e = EventFile()
+ e.set()
+ e.wait(WAIT_TIMEOUT)
+
+
+@raises(TimeExpired)
+@timed(TEST_TIMEOUT)
+def test_wait_noset():
+ e = EventFile()
+ e.wait(WAIT_TIMEOUT)
+
+
+@timed(TEST_TIMEOUT)
+def test_eventfd_earlyset():
+ e = EventFile()
+ e.set()
+ with e.eventfd() as fd:
+ assert len(__select_and_read(fd)) == DATASIZE
+
+
+@timed(TEST_TIMEOUT)
+def test_eventfd_lateset():
+ e = EventFile()
+ with e.eventfd() as fd:
+ e.set()
+ assert len(__select_and_read(fd)) == DATASIZE
+
+
+@raises(TimeExpired)
+@timed(TEST_TIMEOUT)
+def test_eventfd_noset():
+ e = EventFile()
+ with e.eventfd() as fd:
+ assert len(__select_and_read(fd)) != DATASIZE
+
+
+@timed(TEST_TIMEOUT)
+def test_eventfd_multiple():
+ e = EventFile()
+ e.set()
+ with e.eventfd() as fd1:
+ assert len(__select_and_read(fd1)) == DATASIZE
+ with e.eventfd() as fd2:
+ assert len(__select_and_read(fd2)) == DATASIZE
+ with e.eventfd() as fd3:
+ assert len(__select_and_read(fd3)) == DATASIZE
+
+
+@raises(TimeExpired)
+@timed(TEST_TIMEOUT)
+def test_eventfd_clear():
+ e = EventFile()
+ e.set()
+ e.clear()
+ with e.eventfd() as fd:
+ assert len(__select_and_read(fd)) != DATASIZE
+
+
+def __select_and_read(fd):
+ rd, wr, ex = select.select((fd,), (), (), WAIT_TIMEOUT)
+ if fd in rd:
+ return os.read(fd, DATASIZE)
+ return ''
--
To view, visit http://gerrit.ovirt.org/33687
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I0d237f13c42b1f4505c90d30c6d3c3ecbd1e9fa7
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>
Adam Litke has uploaded a new change for review.
Change subject: storage: Factor out getAllVolumes
......................................................................
storage: Factor out getAllVolumes
The SDM garbage collector needs to a mapping of volumes by imageID.
Factor out this logic from getAllVolumes so it can be reused later.
Change-Id: If8c55edf6416dddc4a4f30b91422b3e968df2b99
Signed-off-by: Adam Litke <alitke(a)redhat.com>
---
M vdsm/storage/fileSD.py
1 file changed, 17 insertions(+), 12 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/77/40377/1
diff --git a/vdsm/storage/fileSD.py b/vdsm/storage/fileSD.py
index 1220939..042f836 100644
--- a/vdsm/storage/fileSD.py
+++ b/vdsm/storage/fileSD.py
@@ -18,7 +18,6 @@
# Refer to the README and COPYING files for full details of the license
#
-import collections
import os
import errno
import logging
@@ -405,6 +404,21 @@
"""
pass
+ def getImageToVolumesMap(self):
+ volMetaPattern = os.path.join(self.mountpoint, self.sdUUID,
+ sd.DOMAIN_IMAGES, "*", "*.meta")
+ volMetaPaths = self.oop.glob.glob(volMetaPattern)
+ images = {}
+ for metaPath in volMetaPaths:
+ head, tail = os.path.split(metaPath)
+ volUUID, volExt = os.path.splitext(tail)
+ imgUUID = os.path.basename(head)
+ try:
+ images[imgUUID].append(volUUID)
+ except KeyError:
+ images[imgUUID] = [volUUID]
+ return images
+
def getAllVolumes(self):
"""
Return dict {volUUID: ((imgUUIDs,), parentUUID)} of the domain.
@@ -422,17 +436,8 @@
Template volumes have no parent, and thus we report BLANK_UUID as their
parentUUID.
"""
- volMetaPattern = os.path.join(self.mountpoint, self.sdUUID,
- sd.DOMAIN_IMAGES, "*", "*.meta")
- volMetaPaths = self.oop.glob.glob(volMetaPattern)
-
- # First create mapping from images to volumes
- images = collections.defaultdict(list)
- for metaPath in volMetaPaths:
- head, tail = os.path.split(metaPath)
- volUUID, volExt = os.path.splitext(tail)
- imgUUID = os.path.basename(head)
- images[imgUUID].append(volUUID)
+ # First get the mapping from images to volumes
+ images = self.getImageToVolumesMap()
# Using images to volumes mapping, we can create volumes to images
# mapping, detecting template volumes and template images, based on
--
To view, visit https://gerrit.ovirt.org/40377
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: If8c55edf6416dddc4a4f30b91422b3e968df2b99
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com>
Adam Litke has uploaded a new change for review.
Change subject: XXX: Insert sleep when creating block SD
......................................................................
XXX: Insert sleep when creating block SD
Change-Id: I10ed63d747f1353da824af8ec56ac8a6f66d666b
Signed-off-by: Adam Litke <alitke(a)redhat.com>
---
M vdsm/storage/blockSD.py
1 file changed, 4 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/53/43553/1
diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py
index 0b8bdc7..0d5e403 100644
--- a/vdsm/storage/blockSD.py
+++ b/vdsm/storage/blockSD.py
@@ -896,6 +896,10 @@
# Create VMS file system
_createVMSfs(os.path.join("/dev", vgName, MASTERLV))
+ # XXX: When running an iSCSI server and initiator on the same host we
+ # need to wait a bit for IO to settle before deactivating LVs
+ import time
+ time.sleep(10)
lvm.deactivateLVs(vgName, MASTERLV)
path = lvm.lvPath(vgName, sd.METADATA)
--
To view, visit https://gerrit.ovirt.org/43553
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I10ed63d747f1353da824af8ec56ac8a6f66d666b
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com>
Adam Litke has uploaded a new change for review.
Change subject: sdm: Add find_domain_manifest helpers
......................................................................
sdm: Add find_domain_manifest helpers
Change-Id: Ia5d139bde2349c8231141fe842d9ada96fa279fe
Signed-off-by: Adam Litke <alitke(a)redhat.com>
---
M tests/Makefile.am
A tests/domainmanifestfactory_tests.py
M tests/manifest_tests.py
M tests/storagefakelib.py
M tests/storagetestutils.py
M vdsm.spec.in
M vdsm/storage/Makefile.am
M vdsm/storage/blockSD.py
A vdsm/storage/domainManifestFactory.py
M vdsm/storage/glusterSD.py
M vdsm/storage/localFsSD.py
M vdsm/storage/nfsSD.py
12 files changed, 214 insertions(+), 3 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/55/43555/1
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 9cc3692..f116fc4 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -43,6 +43,7 @@
cpuProfileTests.py \
deviceTests.py \
domainDescriptorTests.py \
+ domainmanifestfactory_tests.py \
encodingTests.py \
executorTests.py \
fileSDTests.py \
diff --git a/tests/domainmanifestfactory_tests.py b/tests/domainmanifestfactory_tests.py
new file mode 100644
index 0000000..4c3f63e
--- /dev/null
+++ b/tests/domainmanifestfactory_tests.py
@@ -0,0 +1,124 @@
+# Copyright 2015 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+
+import os
+import uuid
+
+from testlib import VdsmTestCase, namedTemporaryDir
+from monkeypatch import MonkeyPatchScope
+from storagefakelib import FakeLVM
+from storagetestutils import make_blocksd_manifest, make_vg, \
+ make_filesd_manifest
+
+from storage import sd, blockSD, fileSD, localFsSD, mount
+from storage import storage_exception as se
+from storage import domainManifestFactory
+
+
+class DomainManifestFactoryTests(VdsmTestCase):
+ def get_patches(self, tmpdir, lvm):
+ return [(blockSD, 'lvm', lvm),
+ (blockSD, 'selectMetadata', lambda x: {}),
+ (sd.StorageDomain, 'storage_repository', tmpdir),
+ (mount, 'isMounted', lambda x: True)]
+
+ def make_mnt(self, tmpdir, link_name):
+ mntdir = os.path.join(tmpdir, sd.DOMAIN_MNT_POINT)
+ os.mkdir(mntdir)
+ os.symlink(tmpdir, os.path.join(mntdir, link_name))
+
+ def test_localfs_layout(self):
+ with namedTemporaryDir() as tmpdir:
+ self.make_mnt(tmpdir, '_localfssd')
+ lvm = FakeLVM(tmpdir)
+ with MonkeyPatchScope(self.get_patches(tmpdir, lvm)):
+ manifest = make_filesd_manifest(tmpdir)
+ found_manifest = domainManifestFactory.produce(
+ manifest.sdUUID, manifest._metadata)
+ self.assertEquals(manifest.sdUUID, found_manifest.sdUUID)
+ self.assertIsInstance(found_manifest,
+ fileSD.FileStorageDomainManifest)
+
+ def test_nfs_layout(self):
+ with namedTemporaryDir() as tmpdir:
+ self.make_mnt(tmpdir, '192.168.1.1:_localfssd')
+ lvm = FakeLVM(tmpdir)
+ with MonkeyPatchScope(self.get_patches(tmpdir, lvm)):
+ manifest = make_filesd_manifest(tmpdir)
+ found_manifest = domainManifestFactory.produce(
+ manifest.sdUUID, manifest._metadata)
+ self.assertEquals(manifest.sdUUID, found_manifest.sdUUID)
+ self.assertIsInstance(found_manifest,
+ fileSD.FileStorageDomainManifest)
+
+ def test_glusterfs_layout(self):
+ with namedTemporaryDir() as tmpdir:
+ mntdir = os.path.join(tmpdir, sd.DOMAIN_MNT_POINT)
+ glusterdir = os.path.join(mntdir, sd.GLUSTERSD_DIR)
+ os.makedirs(glusterdir)
+ os.symlink(tmpdir, os.path.join(glusterdir, 'host:_glustersd'))
+
+ lvm = FakeLVM(tmpdir)
+ with MonkeyPatchScope(self.get_patches(tmpdir, lvm)):
+ manifest = make_filesd_manifest(tmpdir)
+ found_manifest = domainManifestFactory.produce(
+ manifest.sdUUID, manifest._metadata)
+ self.assertEquals(manifest.sdUUID, found_manifest.sdUUID)
+ self.assertEquals(manifest.sdUUID, found_manifest.sdUUID)
+ self.assertIsInstance(found_manifest,
+ fileSD.FileStorageDomainManifest)
+
+ def test_block_layout(self):
+ with namedTemporaryDir() as tmpdir:
+ lvm = FakeLVM(tmpdir)
+ with MonkeyPatchScope(self.get_patches(tmpdir, lvm)):
+ manifest = make_blocksd_manifest()
+ vg_name = make_vg(lvm, manifest)
+ lvm.vgmd[vg_name]['tags'] = [blockSD.STORAGE_DOMAIN_TAG]
+ found_manifest = domainManifestFactory.produce(
+ manifest.sdUUID, manifest._metadata)
+ self.assertEquals(manifest.sdUUID, found_manifest.sdUUID)
+ self.assertIsInstance(found_manifest,
+ blockSD.BlockStorageDomainManifest)
+
+ def test_cached_finder(self):
+ def _trap(unused):
+ raise RuntimeError()
+
+ with namedTemporaryDir() as tmpdir:
+ self.make_mnt(tmpdir, '_localfssd')
+ lvm = FakeLVM(tmpdir)
+ patches = self.get_patches(tmpdir, lvm)
+ with MonkeyPatchScope(patches):
+ manifest = make_filesd_manifest(tmpdir)
+ res1 = domainManifestFactory.produce(
+ manifest.sdUUID, manifest._metadata)
+ patches.append((localFsSD, 'findDomainManifest', _trap))
+ with MonkeyPatchScope(patches):
+ res2 = domainManifestFactory.produce(
+ manifest.sdUUID, manifest._metadata)
+ self.assertEquals(res1.sdUUID, res2.sdUUID)
+
+ def test_not_found(self):
+ sdUUID = str(uuid.uuid4())
+ with namedTemporaryDir() as tmpdir:
+ lvm = FakeLVM(tmpdir)
+ with MonkeyPatchScope(self.get_patches(tmpdir, lvm)):
+ self.assertRaises(se.StorageDomainDoesNotExist,
+ domainManifestFactory.produce, sdUUID)
\ No newline at end of file
diff --git a/tests/manifest_tests.py b/tests/manifest_tests.py
index a22d7cf..967b663 100644
--- a/tests/manifest_tests.py
+++ b/tests/manifest_tests.py
@@ -161,7 +161,6 @@
imguuid = str(uuid.uuid4())
imagepath = manifest.getImagePath(imguuid)
- os.makedirs(os.path.dirname(imagepath))
with open(imagepath, 'w') as f:
f.truncate(0)
self.assertEquals(set(), manifest.getAllImages())
diff --git a/tests/storagefakelib.py b/tests/storagefakelib.py
index cb40fbb..bf79d91 100644
--- a/tests/storagefakelib.py
+++ b/tests/storagefakelib.py
@@ -23,6 +23,7 @@
from copy import deepcopy
from storage import lvm as real_lvm
+from storage import storage_exception as se
class FakeLVM(object):
@@ -154,7 +155,11 @@
return real_lvm.PV(**md)
def getVG(self, vgName):
- vg_md = deepcopy(self.vgmd[vgName])
+ try:
+ vg_md = deepcopy(self.vgmd[vgName])
+ except KeyError:
+ raise se.VolumeGroupDoesNotExist(vgName)
+
vg_attr = real_lvm.VG_ATTR(**vg_md['attr'])
vg_md['attr'] = vg_attr
return real_lvm.VG(**vg_md)
diff --git a/tests/storagetestutils.py b/tests/storagetestutils.py
index ed052af..eae803a 100644
--- a/tests/storagetestutils.py
+++ b/tests/storagetestutils.py
@@ -79,7 +79,7 @@
if metadata is None:
metadata = {sd.DMDK_VERSION: 3}
manifest = fileSD.FileStorageDomainManifest(domain_path, metadata)
- os.makedirs(os.path.join(manifest.domaindir, sduuid, sd.DOMAIN_IMAGES))
+ os.makedirs(os.path.join(manifest.domaindir, sd.DOMAIN_IMAGES))
return manifest
diff --git a/vdsm.spec.in b/vdsm.spec.in
index 4ba9812..57e673e 100644
--- a/vdsm.spec.in
+++ b/vdsm.spec.in
@@ -906,6 +906,7 @@
%{_datadir}/%{vdsm_name}/storage/curlImgWrap.py*
%{_datadir}/%{vdsm_name}/storage/devicemapper.py*
%{_datadir}/%{vdsm_name}/storage/dispatcher.py*
+%{_datadir}/%{vdsm_name}/storage/domainManifestFactory.py
%{_datadir}/%{vdsm_name}/storage/monitor.py*
%{_datadir}/%{vdsm_name}/storage/fileSD.py*
%{_datadir}/%{vdsm_name}/storage/fileUtils.py*
diff --git a/vdsm/storage/Makefile.am b/vdsm/storage/Makefile.am
index a4ec93a..e460518 100644
--- a/vdsm/storage/Makefile.am
+++ b/vdsm/storage/Makefile.am
@@ -31,6 +31,7 @@
curlImgWrap.py \
devicemapper.py \
dispatcher.py \
+ domainManifestFactory.py \
fileSD.py \
fileUtils.py \
fileVolume.py \
diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py
index 121cc6b..6de2737 100644
--- a/vdsm/storage/blockSD.py
+++ b/vdsm/storage/blockSD.py
@@ -1426,5 +1426,10 @@
return BlockStorageDomain(BlockStorageDomain.findDomainPath(sdUUID))
+def findDomainManifest(sdUUID, metadata=None):
+ return BlockStorageDomainManifest(
+ BlockStorageDomain.findDomainPath(sdUUID), metadata)
+
+
def getStorageDomainsList():
return [vg.name for vg in lvm.getAllVGs() if _isSD(vg)]
diff --git a/vdsm/storage/domainManifestFactory.py b/vdsm/storage/domainManifestFactory.py
new file mode 100644
index 0000000..fced9de
--- /dev/null
+++ b/vdsm/storage/domainManifestFactory.py
@@ -0,0 +1,60 @@
+#
+# Copyright 2015 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+
+import logging
+
+import blockSD, glusterSD, localFsSD, nfsSD
+import storage_exception as se
+
+
+log = logging.getLogger('Storage.domainManifestFactory')
+
+domain_manifest_producers = {}
+
+
+def produce(sdUUID, metadata=None):
+ log.error("looking for domain manifest %s", sdUUID)
+
+ # Check our cache of finder functions first. Since a domain manifest
+ # cannot change its type we don't have to worry about invalidation.
+ try:
+ producer = domain_manifest_producers[sdUUID]
+ except KeyError:
+ pass
+ else:
+ return producer(sdUUID, metadata)
+
+ # The order is somewhat important, it's ordered
+ # by how quickly get can find the domain. For instance
+ # if an nfs mount is unavailable we will get stuck
+ # until it times out, this should affect fetching
+ # of block\local domains. If for any case in the future
+ # this changes, please update the order.
+ for mod in (blockSD, glusterSD, localFsSD, nfsSD):
+ try:
+ manifest = mod.findDomainManifest(sdUUID, metadata)
+ domain_manifest_producers[sdUUID] = mod.findDomainManifest
+ return manifest
+ except se.StorageDomainDoesNotExist:
+ pass
+ except Exception:
+ log.exception("Error while looking for domain `%s`", sdUUID)
+
+ raise se.StorageDomainDoesNotExist(sdUUID)
diff --git a/vdsm/storage/glusterSD.py b/vdsm/storage/glusterSD.py
index 2931083..a62e242 100644
--- a/vdsm/storage/glusterSD.py
+++ b/vdsm/storage/glusterSD.py
@@ -30,3 +30,8 @@
def findDomain(sdUUID):
return GlusterStorageDomain(GlusterStorageDomain.findDomainPath(sdUUID))
+
+
+def findDomainManifest(sdUUID, metadata=None):
+ return fileSD.FileStorageDomainManifest(
+ GlusterStorageDomain.findDomainPath(sdUUID), metadata)
diff --git a/vdsm/storage/localFsSD.py b/vdsm/storage/localFsSD.py
index 7e11330..b4e0ce0 100644
--- a/vdsm/storage/localFsSD.py
+++ b/vdsm/storage/localFsSD.py
@@ -119,3 +119,8 @@
def findDomain(sdUUID):
return LocalFsStorageDomain(LocalFsStorageDomain.findDomainPath(sdUUID))
+
+
+def findDomainManifest(sdUUID, metadata=None):
+ return fileSD.FileStorageDomainManifest(
+ LocalFsStorageDomain.findDomainPath(sdUUID), metadata)
\ No newline at end of file
diff --git a/vdsm/storage/nfsSD.py b/vdsm/storage/nfsSD.py
index 1f94439..739efc8 100644
--- a/vdsm/storage/nfsSD.py
+++ b/vdsm/storage/nfsSD.py
@@ -120,3 +120,8 @@
def findDomain(sdUUID):
return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID))
+
+
+def findDomainManifest(sdUUID, metadata=None):
+ return fileSD.FileStorageDomainManifest(
+ NfsStorageDomain.findDomainPath(sdUUID), metadata)
--
To view, visit https://gerrit.ovirt.org/43555
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia5d139bde2349c8231141fe842d9ada96fa279fe
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com>
Adam Litke has uploaded a new change for review.
Change subject: sdm: Add decorators for sdm-only functions and their callers
......................................................................
sdm: Add decorators for sdm-only functions and their callers
Change-Id: Iaf736144e5640519851dc9175b5f17539d0ce23e
Signed-off-by: Adam Litke <alitke(a)redhat.com>
---
M tests/Makefile.am
A tests/sdm_tests.py
M vdsm.spec.in
M vdsm/storage/Makefile.am
A vdsm/storage/sdm.py
5 files changed, 102 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/56/43556/1
diff --git a/tests/Makefile.am b/tests/Makefile.am
index f116fc4..5907a0d 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -92,6 +92,7 @@
scheduleTests.py \
schemaTests.py \
schemaValidationTest.py \
+ sdm_tests.py \
securableTests.py \
sourceroutingTests.py \
sslhelper.py \
diff --git a/tests/sdm_tests.py b/tests/sdm_tests.py
new file mode 100644
index 0000000..95ae126
--- /dev/null
+++ b/tests/sdm_tests.py
@@ -0,0 +1,43 @@
+#
+# Copyright 2015 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+
+from testlib import VdsmTestCase
+
+from storage import sdm
+from storage.threadLocal import vars
+
+
+class RequireSDMTests(VdsmTestCase):
+ def setUp(self):
+ try:
+ delattr(vars, '__sdm__')
+ except AttributeError:
+ pass
+
+ @sdm.sdm_verb
+ def test_allowed(self):
+ self.assertTrue(self.sdm_fn())
+
+ def test_denied(self):
+ self.assertRaises(sdm.SDMFunctionNotCallable, self.sdm_fn)
+
+ @sdm.require_sdm
+ def sdm_fn(self):
+ return True
\ No newline at end of file
diff --git a/vdsm.spec.in b/vdsm.spec.in
index 57e673e..24e0ab7 100644
--- a/vdsm.spec.in
+++ b/vdsm.spec.in
@@ -936,6 +936,7 @@
%{_datadir}/%{vdsm_name}/storage/clusterlock.py*
%{_datadir}/%{vdsm_name}/storage/sdc.py*
%{_datadir}/%{vdsm_name}/storage/sd.py*
+%{_datadir}/%{vdsm_name}/storage/sdm.py*
%{_datadir}/%{vdsm_name}/storage/securable.py*
%{_datadir}/%{vdsm_name}/storage/sp.py*
%{_datadir}/%{vdsm_name}/storage/spbackends.py*
diff --git a/vdsm/storage/Makefile.am b/vdsm/storage/Makefile.am
index e460518..51768ce 100644
--- a/vdsm/storage/Makefile.am
+++ b/vdsm/storage/Makefile.am
@@ -59,6 +59,7 @@
resourceManager.py \
sdc.py \
sd.py \
+ sdm.py \
securable.py \
sp.py \
spbackends.py \
diff --git a/vdsm/storage/sdm.py b/vdsm/storage/sdm.py
new file mode 100644
index 0000000..97e599f
--- /dev/null
+++ b/vdsm/storage/sdm.py
@@ -0,0 +1,56 @@
+#
+# Copyright 2015 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+
+from functools import wraps
+from storage.threadLocal import vars
+
+
+class SDMFunctionNotCallable(Exception):
+ pass
+
+
+def require_sdm(f):
+ """
+ SDM method decorator.
+
+ This decorator is used to mark methods that can only be called when the
+ storage is operating in SDM mode (ie. without SPM).
+ """
+ @wraps(f)
+ def wrapper(*args, **kwds):
+ if getattr(vars, '__sdm__', False):
+ return f(*args, **kwds)
+ else:
+ raise SDMFunctionNotCallable(f.__name__)
+ return wrapper
+
+
+def sdm_verb(f):
+ """
+ SDM verb decorator
+
+ This decorator indicates that a function is designed to work without SPM
+ and is approved to access SDM-only functions.
+ """
+ @wraps(f)
+ def wrapper(*args, **kwds):
+ vars.__sdm__ = True
+ return f(*args, **kwds)
+ return wrapper
--
To view, visit https://gerrit.ovirt.org/43556
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iaf736144e5640519851dc9175b5f17539d0ce23e
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com>
Adam Litke has uploaded a new change for review.
Change subject: storage: Factor apparentsize adjustment into a reusable function
......................................................................
storage: Factor apparentsize adjustment into a reusable function
When creating a new volume sometimes the actual size is larger than the
requested size due to underlying storage limitations (such as physical
extent granularity in LVM). In these cases, the new volume's size must
be adjusted to match the actual size. The SDM volume creation code
would also like to use this function so extract it into the
VolumeMetadata class.
Change-Id: I7395850e127b941fc50ae8d5659868a2499b5076
Signed-off-by: Adam Litke <alitke(a)redhat.com>
---
M vdsm/storage/volume.py
1 file changed, 28 insertions(+), 18 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/44/44044/1
diff --git a/vdsm/storage/volume.py b/vdsm/storage/volume.py
index f7e5bc0..723b3d9 100644
--- a/vdsm/storage/volume.py
+++ b/vdsm/storage/volume.py
@@ -495,6 +495,31 @@
puuid)
return None
+ @classmethod
+ def adjust_new_volume_size(cls, dom_manifest, img_id, vol_id, req_size,
+ vol_format):
+ # When the volume format is raw what the guest sees is the apparent
+ # size of the file/device therefore if the requested size doesn't
+ # match the apparent size (eg: physical extent granularity in LVM)
+ # we need to update the size value so that the metadata reflects
+ # the correct state.
+ if vol_format == RAW_FORMAT:
+ apparentSize = int(dom_manifest.getVSize(img_id, vol_id) /
+ BLOCK_SIZE)
+ if apparentSize < req_size:
+ cls.log.error("The volume %s apparent size %s is smaller "
+ "than the requested size %s",
+ vol_id, apparentSize, req_size)
+ raise se.VolumeCreationError()
+ if apparentSize > req_size:
+ cls.log.info("The requested size for volume %s doesn't "
+ "match the granularity on domain %s, "
+ "updating the volume size from %s to %s",
+ vol_id, dom_manifest.sdUUID, req_size,
+ apparentSize)
+ return apparentSize
+ return req_size
+
class Volume(object):
log = logging.getLogger('Storage.Volume')
@@ -850,24 +875,9 @@
cls.log.error("Failed to create volume %s: %s", volPath, e)
vars.task.popRecovery()
raise
- # When the volume format is raw what the guest sees is the apparent
- # size of the file/device therefore if the requested size doesn't
- # match the apparent size (eg: physical extent granularity in LVM)
- # we need to update the size value so that the metadata reflects
- # the correct state.
- if volFormat == RAW_FORMAT:
- apparentSize = int(dom.getVSize(imgUUID, volUUID) / BLOCK_SIZE)
- if apparentSize < size:
- cls.log.error("The volume %s apparent size %s is smaller "
- "than the requested size %s",
- volUUID, apparentSize, size)
- raise se.VolumeCreationError()
- if apparentSize > size:
- cls.log.info("The requested size for volume %s doesn't "
- "match the granularity on domain %s, "
- "updating the volume size from %s to %s",
- volUUID, sdUUID, size, apparentSize)
- size = apparentSize
+
+ size = VolumeMetadata.adjust_new_volume_size(dom, imgUUID, volUUID,
+ size, volFormat)
vars.task.pushRecovery(
task.Recovery("Create volume metadata rollback", clsModule,
--
To view, visit https://gerrit.ovirt.org/44044
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7395850e127b941fc50ae8d5659868a2499b5076
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com>
Adam Litke has uploaded a new change for review.
Change subject: storage: Rename _share to share_to_image
......................................................................
storage: Rename _share to share_to_image
When a new volume is created whose parent is a template volume, that
template must be "shared" into the new volume's image since volume
chains only link volumes in the same image. This logic is reusable by
the SDM create volume flow but _share should be renamed to
share_to_image to remove the _ which designates this as a protected
method.
Change-Id: I44c7fe43ada38aa97bbd9c08438a0ee686680330
Signed-off-by: Adam Litke <alitke(a)redhat.com>
---
M vdsm/storage/blockVolume.py
M vdsm/storage/fileVolume.py
M vdsm/storage/volume.py
3 files changed, 3 insertions(+), 3 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/46/44046/1
diff --git a/vdsm/storage/blockVolume.py b/vdsm/storage/blockVolume.py
index 47336b9..23e636e 100644
--- a/vdsm/storage/blockVolume.py
+++ b/vdsm/storage/blockVolume.py
@@ -314,7 +314,7 @@
def refreshVolume(self):
lvm.refreshLVs(self.sdUUID, (self.volUUID,))
- def _share(self, dstImgPath):
+ def share_to_image(self, dstImgPath):
"""
Share this volume to dstImgPath
"""
diff --git a/vdsm/storage/fileVolume.py b/vdsm/storage/fileVolume.py
index 15ca05b..c5b8667 100644
--- a/vdsm/storage/fileVolume.py
+++ b/vdsm/storage/fileVolume.py
@@ -300,7 +300,7 @@
os.path.join(dstImgPath, self.volUUID))
self.oop.utils.forceLink(self._getLeaseVolumePath(), dstLeasePath)
- def _share(self, dstImgPath):
+ def share_to_image(self, dstImgPath):
"""
Share this volume to dstImgPath, including the metadata and the lease
"""
diff --git a/vdsm/storage/volume.py b/vdsm/storage/volume.py
index 0d61643..951efe0 100644
--- a/vdsm/storage/volume.py
+++ b/vdsm/storage/volume.py
@@ -598,7 +598,7 @@
self.md.removeMetadata()
def _share(self, dstImgPath):
- return self.md._share(dstImgPath)
+ return self.md.share_to_image(dstImgPath)
@classmethod
def _getModuleAndClass(cls):
--
To view, visit https://gerrit.ovirt.org/44046
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I44c7fe43ada38aa97bbd9c08438a0ee686680330
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com>
Adam Litke has uploaded a new change for review.
Change subject: ImageManifest: move getChain
......................................................................
ImageManifest: move getChain
The implementation of getChain instantiates storagedomain and volume
objects. This must be fixed. We must fix all callers of getChain to
work with VolumeMetadata objects instead of Volume objects.
Change-Id: Iffd24be3ba74ccdea8dfc744b579c0cf63771cdb
Signed-off-by: Adam Litke <alitke(a)redhat.com>
---
M vdsm/storage/image.py
1 file changed, 71 insertions(+), 66 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/49/44049/1
diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py
index bd79f6e..f852f3c 100644
--- a/vdsm/storage/image.py
+++ b/vdsm/storage/image.py
@@ -89,6 +89,8 @@
class ImageManifest(object):
+ log = logging.getLogger('Storage.ImageManifest')
+
def __init__(self, repoPath):
self.repoPath = repoPath
@@ -105,6 +107,74 @@
log.info("Create placeholder %s for image's volumes", img_dir)
os.mkdir(img_dir)
return img_dir
+
+ def getChain(self, sdUUID, imgUUID, volUUID=None):
+ """
+ Return the chain of volumes of image as a sorted list
+ (not including a shared base (template) if any)
+ """
+ chain = []
+ volclass = sdCache.produce(sdUUID).getVolumeClass()
+
+ # Use volUUID when provided
+ if volUUID:
+ srcVol = volclass(self.repoPath, sdUUID, imgUUID, volUUID)
+
+ # For template images include only one volume (the template itself)
+ # NOTE: this relies on the fact that in a template there is only
+ # one volume
+ if srcVol.isShared():
+ return [srcVol]
+
+ # Find all the volumes when volUUID is not provided
+ else:
+ # Find all volumes of image
+ uuidlist = volclass.getImageVolumes(self.repoPath, sdUUID, imgUUID)
+
+ if not uuidlist:
+ raise se.ImageDoesNotExistInSD(imgUUID, sdUUID)
+
+ srcVol = volclass(self.repoPath, sdUUID, imgUUID, uuidlist[0])
+
+ # For template images include only one volume (the template itself)
+ if len(uuidlist) == 1 and srcVol.isShared():
+ return [srcVol]
+
+ # Searching for the leaf
+ for vol in uuidlist:
+ srcVol = volclass(self.repoPath, sdUUID, imgUUID, vol)
+
+ if srcVol.isLeaf():
+ break
+
+ srcVol = None
+
+ if not srcVol:
+ self.log.error("There is no leaf in the image %s", imgUUID)
+ raise se.ImageIsNotLegalChain(imgUUID)
+
+ # We have seen corrupted chains that cause endless loops here.
+ # https://bugzilla.redhat.com/1125197
+ seen = set()
+
+ # Build up the sorted parent -> child chain
+ while not srcVol.isShared():
+ chain.insert(0, srcVol)
+ seen.add(srcVol.volUUID)
+
+ parentUUID = srcVol.getParentId()
+ if parentUUID == volume.BLANK_UUID:
+ break
+
+ if parentUUID in seen:
+ self.log.error("Image %s volume %s has invalid parent UUID %s",
+ imgUUID, srcVol.volUUID, parentUUID)
+ raise se.ImageIsNotLegalChain(imgUUID)
+
+ srcVol = srcVol.produceParent()
+
+ self.log.info("sdUUID=%s imgUUID=%s chain=%s ", sdUUID, imgUUID, chain)
+ return chain
class Image:
@@ -179,72 +249,7 @@
return newsize
def getChain(self, sdUUID, imgUUID, volUUID=None):
- """
- Return the chain of volumes of image as a sorted list
- (not including a shared base (template) if any)
- """
- chain = []
- volclass = sdCache.produce(sdUUID).getVolumeClass()
-
- # Use volUUID when provided
- if volUUID:
- srcVol = volclass(self.repoPath, sdUUID, imgUUID, volUUID)
-
- # For template images include only one volume (the template itself)
- # NOTE: this relies on the fact that in a template there is only
- # one volume
- if srcVol.isShared():
- return [srcVol]
-
- # Find all the volumes when volUUID is not provided
- else:
- # Find all volumes of image
- uuidlist = volclass.getImageVolumes(self.repoPath, sdUUID, imgUUID)
-
- if not uuidlist:
- raise se.ImageDoesNotExistInSD(imgUUID, sdUUID)
-
- srcVol = volclass(self.repoPath, sdUUID, imgUUID, uuidlist[0])
-
- # For template images include only one volume (the template itself)
- if len(uuidlist) == 1 and srcVol.isShared():
- return [srcVol]
-
- # Searching for the leaf
- for vol in uuidlist:
- srcVol = volclass(self.repoPath, sdUUID, imgUUID, vol)
-
- if srcVol.isLeaf():
- break
-
- srcVol = None
-
- if not srcVol:
- self.log.error("There is no leaf in the image %s", imgUUID)
- raise se.ImageIsNotLegalChain(imgUUID)
-
- # We have seen corrupted chains that cause endless loops here.
- # https://bugzilla.redhat.com/1125197
- seen = set()
-
- # Build up the sorted parent -> child chain
- while not srcVol.isShared():
- chain.insert(0, srcVol)
- seen.add(srcVol.volUUID)
-
- parentUUID = srcVol.getParentId()
- if parentUUID == volume.BLANK_UUID:
- break
-
- if parentUUID in seen:
- self.log.error("Image %s volume %s has invalid parent UUID %s",
- imgUUID, srcVol.volUUID, parentUUID)
- raise se.ImageIsNotLegalChain(imgUUID)
-
- srcVol = srcVol.produceParent()
-
- self.log.info("sdUUID=%s imgUUID=%s chain=%s ", sdUUID, imgUUID, chain)
- return chain
+ return self.manifest.getChain(sdUUID, imgUUID, volUUID)
def getTemplate(self, sdUUID, imgUUID):
"""
--
To view, visit https://gerrit.ovirt.org/44049
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iffd24be3ba74ccdea8dfc744b579c0cf63771cdb
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com>
Adam Litke has uploaded a new change for review.
Change subject: storage: block: getChildren: Do not count uninit LVs as children
......................................................................
storage: block: getChildren: Do not count uninit LVs as children
Any LV with the tag TAG_VOL_UNINIT is not a real volume yet. It is
either in the process of being created or deleted. It should not appear
in the children list of its parent until the tag is removed.
Change-Id: I03e489df8478de26b01ce34f41e87aecc04512ed
Signed-off-by: Adam Litke <alitke(a)redhat.com>
---
M vdsm/storage/blockVolume.py
1 file changed, 2 insertions(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/71/44571/1
diff --git a/vdsm/storage/blockVolume.py b/vdsm/storage/blockVolume.py
index ee531f3..51f60ea 100644
--- a/vdsm/storage/blockVolume.py
+++ b/vdsm/storage/blockVolume.py
@@ -183,7 +183,8 @@
"""
lvs = lvm.lvsByTag(self.sdUUID,
"%s%s" % (TAG_PREFIX_PARENT, self.volUUID))
- return tuple(lv.name for lv in lvs)
+
+ return tuple(lv.name for lv in lvs if TAG_VOL_UNINIT not in lv.tags)
def getImage(self):
"""
--
To view, visit https://gerrit.ovirt.org/44571
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I03e489df8478de26b01ce34f41e87aecc04512ed
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com>
Adam Litke has uploaded a new change for review.
Change subject: Garbage volumes are not part of an image
......................................................................
Garbage volumes are not part of an image
Change-Id: I8776a4b48cc8d16f420426fb28d4696b6f2bf357
Signed-off-by: Adam Litke <alitke(a)redhat.com>
---
M vdsm/storage/blockVolume.py
1 file changed, 1 insertion(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/30/44830/1
diff --git a/vdsm/storage/blockVolume.py b/vdsm/storage/blockVolume.py
index 9baa3d2..af11725 100644
--- a/vdsm/storage/blockVolume.py
+++ b/vdsm/storage/blockVolume.py
@@ -334,7 +334,7 @@
(template)
"""
lvs = lvm.lvsByTag(sdUUID, "%s%s" % (TAG_PREFIX_IMAGE, imgUUID))
- return [lv.name for lv in lvs]
+ return [lv.name for lv in lvs if TAG_VOL_GARBAGE not in lv.tags]
@logskip("ResourceManager")
def llPrepare(self, rw=False, setrw=False):
--
To view, visit https://gerrit.ovirt.org/44830
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I8776a4b48cc8d16f420426fb28d4696b6f2bf357
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: vm: Cleaner vm stats thread initialization
......................................................................
vm: Cleaner vm stats thread initialization
The vm stats thread was initialized too late, leading to unneeded checks
and unclear try except blocks when trying to stop the thread.
This patch changes AdvancedStatsThread so it keeps a thread instance
instead of inheriting from Thread, and initialize it in Vm.__init__, so
it is always available when we access it.
Change-Id: I2917de42b76ee3dc8b27bdc23b33f3c984a7cc68
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm/virt/sampling.py
M vdsm/virt/vm.py
2 files changed, 11 insertions(+), 18 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/99/39299/1
diff --git a/vdsm/virt/sampling.py b/vdsm/virt/sampling.py
index 3206b97..848bb36 100644
--- a/vdsm/virt/sampling.py
+++ b/vdsm/virt/sampling.py
@@ -401,7 +401,7 @@
return self._samples.last()
-class AdvancedStatsThread(threading.Thread):
+class AdvancedStatsThread(object):
"""
A thread that runs the registered AdvancedStatsFunction objects
for statistic and monitoring purpose.
@@ -412,9 +412,8 @@
"""
Initialize an AdvancedStatsThread object
"""
- threading.Thread.__init__(self)
self.daemon = daemon
-
+ self._thread = None
self._log = log
self._stopEvent = threading.Event()
self._contEvent = threading.Event()
@@ -426,7 +425,7 @@
"""
Register the functions listed as arguments
"""
- if self.isAlive():
+ if self._thread is not None:
raise RuntimeError("AdvancedStatsThread is started")
for statsFunction in args:
@@ -437,7 +436,9 @@
Start the execution of the thread and exit
"""
self._log.debug("Start statistics collection")
- threading.Thread.start(self)
+ self._thread = threading.Thread(target=self._run)
+ self._thread.daemon = self.daemon
+ self._thread.start()
def stop(self):
"""
@@ -464,7 +465,7 @@
def getLastSampleTime(self):
return self._statsTime
- def run(self):
+ def _run(self):
self._log.debug("Stats thread started")
self._contEvent.set()
diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py
index 28054bf..1bc04e7 100644
--- a/vdsm/virt/vm.py
+++ b/vdsm/virt/vm.py
@@ -779,7 +779,7 @@
self._initTimeRTC = long(self.conf.get('timeOffset', 0))
self._guestEvent = vmstatus.POWERING_UP
self._guestEventTime = 0
- self._vmStats = None
+ self._vmStats = VmStatsThread(self)
self._guestCpuRunning = False
self._guestCpuLock = threading.Lock()
self._startTime = time.time() - \
@@ -1269,7 +1269,7 @@
return
toSave = self.status()
toSave['startTime'] = self._startTime
- if self.lastStatus != vmstatus.DOWN and self._vmStats:
+ if self.lastStatus != vmstatus.DOWN:
guestInfo = self.guestAgent.getGuestInfo()
toSave['username'] = guestInfo['username']
toSave['guestIPs'] = guestInfo['guestIPs']
@@ -1758,7 +1758,7 @@
decStats = {}
try:
- if self._vmStats and self._vmStats.getLastSampleTime() is not None:
+ if self._vmStats.getLastSampleTime() is not None:
decStats = self._vmStats.get()
self._setUnresponsiveIfTimeout(
stats,
@@ -2001,19 +2001,11 @@
return domxml.toxml()
def startVmStats(self):
- self._vmStats = VmStatsThread(self)
self._vmStats.start()
self._guestEventTime = self._startTime
def stopVmStats(self):
- # this is less clean that it could be, but we can get here from
- # many flows and with various locks held
- # (_releaseLock, _shutdownLock)
- # _vmStats may be None already, and we're good with that.
- try:
- self._vmStats.stop()
- except AttributeError:
- pass
+ self._vmStats.stop()
@staticmethod
def _guestSockCleanup(sock):
--
To view, visit https://gerrit.ovirt.org/39299
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2917de42b76ee3dc8b27bdc23b33f3c984a7cc68
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Piotr Kliczewski has uploaded a new change for review.
Change subject: xmlrpc: retry when flushing socket
......................................................................
xmlrpc: retry when flushing socket
m2crypto requires retry of write and we need to make sure that we do it once
flushing data before closing a socket. In order to do so we need to move
original implementation to our code base.
Bug-Url: https://bugzilla.redhat.com/1261255
Change-Id: I52dd6ad304a82ff6c8d3dff12a38269684abf055
Signed-off-by: pkliczewski <piotr.kliczewski(a)gmail.com>
---
M vdsm/rpc/bindingxmlrpc.py
1 file changed, 19 insertions(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/37/47637/2
diff --git a/vdsm/rpc/bindingxmlrpc.py b/vdsm/rpc/bindingxmlrpc.py
index c521cd8..9f77457 100644
--- a/vdsm/rpc/bindingxmlrpc.py
+++ b/vdsm/rpc/bindingxmlrpc.py
@@ -25,6 +25,7 @@
import libvirt
import threading
import re
+import socket
import sys
from vdsm.password import (ProtectedPassword,
@@ -33,6 +34,7 @@
from vdsm import utils
from vdsm import xmlrpc
from vdsm.define import doneCode, errCode
+from vdsm.m2cutils import SSL
from vdsm.netinfo import getDeviceByIP
import API
from vdsm.exception import VdsmException
@@ -288,7 +290,23 @@
return r
def finish(self):
- xmlrpc.IPXMLRPCRequestHandler.finish(self)
+ if not self.wfile.closed:
+ while True:
+ try:
+ self.wfile.flush()
+ break
+ except socket.error:
+ # An final socket error may have occurred here,
+ # such as the local error ECONNABORTED.
+ break
+ except SSL.SSLError as e:
+ if e.message == 'bad write retry':
+ continue
+ else:
+ break
+
+ self.wfile.close()
+ self.rfile.close()
threadLocal.client = None
threadLocal.server = None
threadLocal.flowID = None
--
To view, visit https://gerrit.ovirt.org/47637
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I52dd6ad304a82ff6c8d3dff12a38269684abf055
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>
Gerrit-Reviewer: automation(a)ovirt.org
Adam Litke has uploaded a new change for review.
Change subject: fix remove: iterate while del
......................................................................
fix remove: iterate while del
Change-Id: I594c98cf54ed93d045b76df1c6faa5122277b19e
Signed-off-by: Adam Litke <alitke(a)redhat.com>
---
M vdsm/storage/blockSD.py
1 file changed, 2 insertions(+), 2 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/72/44572/1
diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py
index 74bf82d..832fb8b 100644
--- a/vdsm/storage/blockSD.py
+++ b/vdsm/storage/blockSD.py
@@ -201,8 +201,8 @@
vols = _getVolsTree(sdUUID)
# Remove volumes awaiting garbage collection from the result
- for vol_id in vols.iterkeys():
- if vols[vol_id].garbage_meta_id is not None:
+ for vol_id, vol_info in vols.items():
+ if vol_info.garbage_meta_id is not None:
del vols[vol_id]
res = {}
--
To view, visit https://gerrit.ovirt.org/44572
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I594c98cf54ed93d045b76df1c6faa5122277b19e
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com>
Adam Litke has uploaded a new change for review.
Change subject: storage: Do not count garbage volumes as children
......................................................................
storage: Do not count garbage volumes as children
Change-Id: Ice3c70249cbc3577b239bd11d224955f2e29b211
Signed-off-by: Adam Litke <alitke(a)redhat.com>
---
M vdsm/storage/blockVolume.py
1 file changed, 1 insertion(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/73/44573/1
diff --git a/vdsm/storage/blockVolume.py b/vdsm/storage/blockVolume.py
index 51f60ea..9baa3d2 100644
--- a/vdsm/storage/blockVolume.py
+++ b/vdsm/storage/blockVolume.py
@@ -184,7 +184,7 @@
lvs = lvm.lvsByTag(self.sdUUID,
"%s%s" % (TAG_PREFIX_PARENT, self.volUUID))
- return tuple(lv.name for lv in lvs if TAG_VOL_UNINIT not in lv.tags)
+ return tuple(lv.name for lv in lvs if TAG_VOL_GARBAGE not in lv.tags)
def getImage(self):
"""
--
To view, visit https://gerrit.ovirt.org/44573
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ice3c70249cbc3577b239bd11d224955f2e29b211
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: freeze: Freeze guest even when creating memory snapshot
......................................................................
freeze: Freeze guest even when creating memory snapshot
We used to skip freezing the guest if creating memory snapshot. This was
probably done because qemu is pausing the vm for creating memory
snapshot.
However, this is not consistent with snapshots of external disks such as
network disks, where we always freeze the vm before taking the snapshot.
Also, it is probably safer to freeze even when creating memory snapshot,
giving applications on the guest chance to pause in consistent state.
This patch removes the check for memory snapshot, and freeze the guest
unless it is already frozen.
Change-Id: I8aa7ac0dea8690ca33df8067f84734d788da8bf8
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm/virt/vm.py
1 file changed, 3 insertions(+), 6 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/99/43299/1
diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py
index 332281b..ecbbe1d 100644
--- a/vdsm/virt/vm.py
+++ b/vdsm/virt/vm.py
@@ -3093,9 +3093,6 @@
else:
snapFlags |= libvirt.VIR_DOMAIN_SNAPSHOT_CREATE_DISK_ONLY
- # When creating memory snapshot libvirt will pause the vm
- should_freeze = not (memoryParams or frozen)
-
snapxml = snap.toprettyxml()
# TODO: this is debug information. For 3.6.x we still need to
# see the XML even with 'info' as default level.
@@ -3109,7 +3106,7 @@
self.stopDisksStatsCollection()
try:
- if should_freeze:
+ if not frozen:
freezed = self.freeze()
try:
self._dom.snapshotCreateXML(snapxml, snapFlags)
@@ -3120,7 +3117,7 @@
# Must always thaw, even if freeze failed; in case the guest
# did freeze the filesystems, but failed to reply in time.
# Libvirt is using same logic (see src/qemu/qemu_driver.c).
- if should_freeze:
+ if not frozen:
self.thaw()
# We are padding the memory volume with block size of zeroes
@@ -3148,7 +3145,7 @@
# Returning quiesce to notify the manager whether the guest agent
# froze and flushed the filesystems or not.
return {'status': doneCode,
- 'quiesce': should_freeze and freezed["status"]["code"] == 0}
+ 'quiesce': not frozen and freezed["status"]["code"] == 0}
def diskReplicateStart(self, srcDisk, dstDisk):
try:
--
To view, visit https://gerrit.ovirt.org/43299
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I8aa7ac0dea8690ca33df8067f84734d788da8bf8
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: log: Use INFO log level as default
......................................................................
log: Use INFO log level as default
The current logs are much too verbose which cause trouble for users, and
make us look unprofessional. Mature project should not use debug log by
default.
To debug issues that are not clear enough using INFO logs, the relevant
logger level can be modified on a user machine as needed.
Change-Id: I767dcd9bad7b9fbeebb438e9ef13cb0ec3f042ee
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm/logger.conf.in
1 file changed, 4 insertions(+), 4 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/04/32504/1
diff --git a/vdsm/logger.conf.in b/vdsm/logger.conf.in
index 64b154f..8e963dd 100644
--- a/vdsm/logger.conf.in
+++ b/vdsm/logger.conf.in
@@ -8,18 +8,18 @@
keys=long,simple,none,sysform
[logger_root]
-level=DEBUG
+level=INFO
handlers=syslog,logfile
propagate=0
[logger_vds]
-level=DEBUG
+level=INFO
handlers=syslog,logfile
qualname=vds
propagate=0
[logger_Storage]
-level=DEBUG
+level=INFO
handlers=logfile
qualname=Storage
propagate=0
@@ -31,7 +31,7 @@
propagate=1
[logger_connectivity]
-level=DEBUG
+level=INFO
handlers=connlogfile
qualname=connectivity
propagate=0
--
To view, visit http://gerrit.ovirt.org/32504
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I767dcd9bad7b9fbeebb438e9ef13cb0ec3f042ee
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: vm: Cleanup waiting for xml update
......................................................................
vm: Cleanup waiting for xml update
This patch cleans up a bit the code for waiting until libvirt xml is
updated after pivot was completed.
- Clarify confusing log message claiming that pivot failed after it
completed successfully
- Cleanup creation of volumes lists using generator expression
- More clear logic for checking current volumes list
- Replace detailed log message and unhelpful exception with detailed
exception
- Move comment out of the loop to make the loop more clear
- Remove unneeded keys() calls when looking up alias in chains
This code was added as temporary solution until libvirt is fixed, but I
think we would like keep a simplified version of it even after libvirt
is fixed, verifying that the operation was successful.
Change-Id: I9fec5416a62736bad461ddd0b54093d23960b7a6
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm/virt/vm.py
1 file changed, 27 insertions(+), 24 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/38/39938/1
diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py
index efadbdb..8ece47b 100644
--- a/vdsm/virt/vm.py
+++ b/vdsm/virt/vm.py
@@ -5100,40 +5100,43 @@
# synchronized and we may start the vm with a stale volume in the
# future. See https://bugzilla.redhat.com/show_bug.cgi?id=1202719 for
# more details.
- # TODO: Remove once we depend on a libvirt with this bug fixed.
# We expect libvirt to show that the original leaf has been removed
# from the active volume chain.
origVols = sorted([x['volumeID'] for x in self.drive.volumeChain])
- expectedVols = origVols[:]
- expectedVols.remove(self.drive.volumeID)
+ expectedVols = [v for v in origVols if v != self.driveVolumeID]
alias = self.drive['alias']
self.vm.log.info("Waiting for libvirt to update the XML after pivot "
"of drive %s completed", alias)
- while True:
- # This operation should complete in either one or two iterations of
- # this loop. Until libvirt updates the XML there is nothing to do
- # but wait. While we wait we continue to tell engine that the job
- # is ongoing. If we are still in this loop when the VM is powered
- # off, the merge will be resolved manually by engine using the
- # reconcileVolumeChain verb.
- chains = self.vm._driveGetActualVolumeChain([self.drive])
- if alias not in chains.keys():
- raise RuntimeError("Failed to retrieve volume chain for "
- "drive %s. Pivot failed.", alias)
- curVols = sorted([entry.uuid for entry in chains[alias]])
- if curVols == origVols:
- time.sleep(1)
- elif curVols == expectedVols:
+ # This operation should complete in either one or two iterations of
+ # this loop. Until libvirt updates the XML there is nothing to do
+ # but wait. While we wait we continue to tell engine that the job
+ # is ongoing. If we are still in this loop when the VM is powered
+ # off, the merge will be resolved manually by engine using the
+ # reconcileVolumeChain verb.
+ # TODO: Check once when we depend on a libvirt with this bug fixed.
+
+ while True:
+ chains = self.vm._driveGetActualVolumeChain([self.drive])
+ if alias not in chains:
+ raise RuntimeError("Failed to retrieve volume chain for "
+ "drive %s after pivot completed", alias)
+
+ curVols = sorted(entry.uuid for entry in chains[alias])
+
+ if curVols == expectedVols:
self.vm.log.info("The XML update has been completed")
- break
- else:
- self.log.error("Bad volume chain found for drive %s. Previous "
- "chain: %s, Expected chain: %s, Actual chain: "
- "%s", alias, origVols, expectedVols, curVols)
- raise RuntimeError("Bad volume chain found")
+ return
+
+ if curVols != origVols:
+ raise RuntimeError(
+ "Bad volume chain after pivot for drive %s. Previous "
+ "chain: %s, Expected chain: %s, Actual chain: %s" %
+ (alias, origVols, expectedVols, curVols))
+
+ time.sleep(1)
def _devicesWithAlias(domXML):
--
To view, visit https://gerrit.ovirt.org/39938
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I9fec5416a62736bad461ddd0b54093d23960b7a6
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: lib: Revert and refine error handling in tmap()
......................................................................
lib: Revert and refine error handling in tmap()
In commit 2b7155b696 (lib: Simplify and generalize concurrent.tmap()),
we simplified error handling by returning a named tuple with function
results. This turned out less useful then the original error handling.
This patch returns the previous error handling:
- Functions passed to tmap() should not raise - if they raise, this is
considered a bug in the function.
- The last error is raised by tmap() instead of returning the result.
This make it easier to fail loudly for unexpected errors.
- The original exception is re-raised now with the original traceback.
- Error handling is documented properly now
Previously you had to make sure function raises to signal failures:
def func():
try:
code that should not fail...
code that may fail...
code that should not fail...
except ExpectedError:
log.error(...)
raise
except Exception:
log.exception(...)
raise
results = concurrent.tmap(func, values)
if not all(r.succeeded for r in results):
...
Returning the result as is lets us have nicer code:
def func():
code that should not fail...
try:
code that may fail...
except ExpectedError:
log.error(...)
return False
code that should not fail...
return True
succeeded = concurrent.tmap(func, values)
if not all(succeeded):
...
We can ignore unexpected errors, since tmap() will log them and fail
loudly. We can also minimize try except block for expected errors.
Change-Id: I0154b28ff7822c63e77181bbbf444c712bd0c31e
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M lib/vdsm/concurrent.py
M tests/concurrentTests.py
2 files changed, 45 insertions(+), 19 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/11/39211/1
diff --git a/lib/vdsm/concurrent.py b/lib/vdsm/concurrent.py
index 64e072d..5498052 100644
--- a/lib/vdsm/concurrent.py
+++ b/lib/vdsm/concurrent.py
@@ -18,22 +18,42 @@
# Refer to the README and COPYING files for full details of the license
#
+import logging
import threading
-from collections import namedtuple
-
-
-Result = namedtuple("Result", ["succeeded", "value"])
+import sys
def tmap(func, iterable):
+ """
+ Run func with arguments from iterable in multiple threads, a returning the
+ output in order of arguments.
+
+ func should not raise exceptions - we consider this a bug in func, and will
+ fail the call and re-raise the exception in the caller thread.
+
+ Expected exceptions should be handled in func. If the caller likes to
+ handle the error later, func should return it:
+
+ def func(value):
+ try:
+ return something(value)
+ except ExpectedError as e:
+ return e
+
+ Unexpected exceptions should not be handled, as they are logged in the
+ worker threads and re-raised in the caller thread. If multiple excpetions
+ raised, only the last one will be re-raised in the caller thread.
+ """
args = list(iterable)
results = [None] * len(args)
+ error = [None]
def worker(i, f, arg):
try:
- results[i] = Result(True, f(arg))
- except Exception as e:
- results[i] = Result(False, e)
+ results[i] = f(arg)
+ except Exception:
+ error[0] = sys.exc_info()
+ logging.exception("Unhandled exception in tmap worker thread")
threads = []
for i, arg in enumerate(args):
@@ -45,4 +65,8 @@
for t in threads:
t.join()
+ if error[0] is not None:
+ t, v, tb = error[0]
+ raise t, v, tb
+
return results
diff --git a/tests/concurrentTests.py b/tests/concurrentTests.py
index 307e397..5c0646b 100644
--- a/tests/concurrentTests.py
+++ b/tests/concurrentTests.py
@@ -26,13 +26,16 @@
from vdsm import concurrent
+class Error(Exception):
+ pass
+
+
class TMapTests(VdsmTestCase):
def test_results(self):
values = tuple(range(10))
results = concurrent.tmap(lambda x: x, values)
- expected = [concurrent.Result(True, x) for x in values]
- self.assertEqual(results, expected)
+ self.assertEqual(results, list(values))
def test_results_order(self):
def func(x):
@@ -40,8 +43,7 @@
return x
values = tuple(random.random() * 0.1 for x in range(10))
results = concurrent.tmap(func, values)
- expected = [concurrent.Result(True, x) for x in values]
- self.assertEqual(results, expected)
+ self.assertEqual(results, list(values))
def test_concurrency(self):
start = time.time()
@@ -49,12 +51,12 @@
elapsed = time.time() - start
self.assertTrue(0.1 < elapsed < 0.2)
- def test_error(self):
- error = RuntimeError("No result for you!")
-
+ def test_raise_last_error(self):
def func(x):
- raise error
-
- results = concurrent.tmap(func, range(10))
- expected = [concurrent.Result(False, error)] * 10
- self.assertEqual(results, expected)
+ raise Error(x)
+ try:
+ concurrent.tmap(func, (1, 2, 3))
+ except Error as e:
+ self.assertEqual(e.args, (3,))
+ else:
+ self.fail("Exception was not raised")
--
To view, visit https://gerrit.ovirt.org/39211
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I0154b28ff7822c63e77181bbbf444c712bd0c31e
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: misc: Safer and simpler itmap
......................................................................
misc: Safer and simpler itmap
The previous code had few issues:
- It used unlimited number of threads by default. This may lead to
creation of 100's of threads if you do not specify a value.
- It used non-daemon threads, which could lead to unwanted delay during
vdsm shutdown.
- It tried to yield results before all arguments were handled. This
could lead to unwanted delay in argument processing, if the caller
would block processing the results.
- It started one thread per value, even if maxthreads was smaller than
number of values.
- It was too complicated.
Changes:
- The caller must specify the maximum number of threads.
- Use daemon threads
- Queue all values before yielding results
- Start up to maxthreads worker threads, each processing multiple values
- Simplify the code
- Add test for error handling
Change-Id: Iba6116ac4003702c8e921cebaf494491a6f9afaf
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M tests/miscTests.py
M vdsm/storage/misc.py
2 files changed, 42 insertions(+), 42 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/19/39119/1
diff --git a/tests/miscTests.py b/tests/miscTests.py
index 31f64fa..4b3e3c3 100644
--- a/tests/miscTests.py
+++ b/tests/miscTests.py
@@ -196,7 +196,7 @@
# outOfProcess operation + 1. it let us know that oop and itmap operate
# properly with their limitations
data = frozenset(range(oop.HELPERS_PER_DOMAIN + 1))
- ret = frozenset(misc.itmap(dummy, data, misc.UNLIMITED_THREADS))
+ ret = frozenset(misc.itmap(dummy, data, len(data)))
self.assertEquals(ret, data)
def testMoreThreadsThanArgs(self):
@@ -207,6 +207,13 @@
data = 1
self.assertRaises(ValueError, misc.itmap(int, data, 0).next)
+ def testErrors(self):
+ err = Exception()
+ def dummy(arg):
+ raise err
+ data = [1, 2, 3]
+ self.assertEqual(list(misc.itmap(dummy, data, 4)), [err] * len(data))
+
class RotateFiles(TestCaseBase):
diff --git a/vdsm/storage/misc.py b/vdsm/storage/misc.py
index eb484c7..463fd04 100644
--- a/vdsm/storage/misc.py
+++ b/vdsm/storage/misc.py
@@ -58,7 +58,6 @@
STR_UUID_SIZE = 36
UUID_HYPHENS = [8, 13, 18, 23]
MEGA = 1 << 20
-UNLIMITED_THREADS = -1
log = logging.getLogger('Storage.Misc')
@@ -882,53 +881,47 @@
raise exception
-def itmap(func, iterable, maxthreads=UNLIMITED_THREADS):
+def itmap(func, iterable, maxthreads):
"""
- Make an iterator that computes the function using
- arguments from the iterable. It works similar to tmap
- by running each operation in a different thread, this
- causes the results not to return in any particular
- order so it's good if you don't care about the order
- of the results.
- maxthreads stands for maximum threads that we can initiate simultaneosly.
- If we reached to max threads the function waits for thread to
- finish before initiate the next one.
+ Return an iterator calling func with arguments from iterable in multiple threads.
+
+ Unlike tmap, the results are not returned in the original order of the
+ arguments, and number of threads is limited to maxthreads.
"""
- if maxthreads < 1 and maxthreads != UNLIMITED_THREADS:
- raise ValueError("Wrong input to function itmap: %s", maxthreads)
+ if maxthreads < 1:
+ raise ValueError("Invalid maxthreads value: %s" % maxthreads)
- respQueue = Queue.Queue()
+ DONE = object()
+ values = Queue.Queue()
+ results = Queue.Queue()
- def wrapper(value):
- try:
- respQueue.put(func(value))
- except Exception as e:
- respQueue.put(e)
+ def worker():
+ while True:
+ value = values.get()
+ if value is DONE:
+ return
+ try:
+ results.put(func(value))
+ except Exception as e:
+ results.put(e)
- threadsCount = 0
- for arg in iterable:
- if maxthreads != UNLIMITED_THREADS:
- if maxthreads == 0:
- # This not supposed to happened. If it does, it's a bug.
- # maxthreads should get to 0 only after threadsCount is
- # greater than 1
- if threadsCount < 1:
- raise RuntimeError("No thread initiated")
- else:
- yield respQueue.get()
- # if yield returns one thread stopped, so we can run
- # another thread in queue
- maxthreads += 1
- threadsCount -= 1
+ count = 0
+ threads = 0
- t = threading.Thread(target=wrapper, args=(arg,))
- t.start()
- threadsCount += 1
- maxthreads -= 1
+ for value in iterable:
+ values.put(value)
+ count += 1
+ if threads < maxthreads:
+ t = threading.Thread(target=worker)
+ t.daemon = True
+ t.start()
+ threads += 1
- # waiting for rest threads to end
- for i in xrange(threadsCount):
- yield respQueue.get()
+ for _ in range(threads):
+ values.put(DONE)
+
+ for _ in xrange(count):
+ yield results.get()
def isAscii(s):
--
To view, visit https://gerrit.ovirt.org/39119
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iba6116ac4003702c8e921cebaf494491a6f9afaf
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: udevadm: More precise error handling
......................................................................
udevadm: More precise error handling
udevadm provides a --timeout option, but there is no robust way to
detect a timeout in EL6, EL7, and Fedora 20. In Fedora 21 and upstream,
udevadm ignores the timeout option. This patch improves error handling
by using our own timeout.
udevadm.settle() raises now udevadm.Failure or udevadm.Timeout, and the
caller is responsible to handle the error.
In both multipath.rescan() and IscsiConnection.connect(), we warn about
timeout but do not handle other errors, so real errors in udevadm will
fail loudly.
Change-Id: Ia0a7380b1b181ec93399ea741122cfa2e98086fb
Relates-To: https://bugzilla.redhat.com/1209474
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
A tests/udevadmTests.py
M vdsm/storage/multipath.py
M vdsm/storage/storageServer.py
M vdsm/storage/udevadm.py
4 files changed, 106 insertions(+), 21 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/40/39740/1
diff --git a/tests/udevadmTests.py b/tests/udevadmTests.py
new file mode 100644
index 0000000..90841b2
--- /dev/null
+++ b/tests/udevadmTests.py
@@ -0,0 +1,52 @@
+#
+# Copyright 2015 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+
+
+from monkeypatch import MonkeyPatch
+from testlib import VdsmTestCase
+
+from vdsm import utils
+from storage import udevadm
+
+TRUE = utils.CommandPath("true", "/bin/true", "/usr/bin/true")
+FALSE = utils.CommandPath("false", "/bin/false", "/usr/bin/false")
+READ = utils.CommandPath("read", "/bin/read", "/usr/bin/read")
+
+
+class UdevadmSettleTests(VdsmTestCase):
+
+ @MonkeyPatch(udevadm, "_UDEVADM", TRUE)
+ def test_success(self):
+ udevadm.settle(5)
+
+ @MonkeyPatch(udevadm, "_UDEVADM", FALSE)
+ def test_error(self):
+ try:
+ udevadm.settle(5)
+ except udevadm.Failure as e:
+ self.assertEqual(e.rc, 1)
+ self.assertEqual(e.out, "")
+ self.assertEqual(e.err, "")
+ else:
+ self.fail("Failure not raised")
+
+ @MonkeyPatch(udevadm, "_UDEVADM", READ)
+ def test_timeout(self):
+ self.assertRaises(udevadm.Timeout, udevadm.settle, 1)
diff --git a/vdsm/storage/multipath.py b/vdsm/storage/multipath.py
index a1c42b3..925c411 100644
--- a/vdsm/storage/multipath.py
+++ b/vdsm/storage/multipath.py
@@ -73,7 +73,10 @@
# events are processed, ensuring detection of new devices and creation or
# update of multipath devices.
timeout = config.getint('irs', 'scsi_settle_timeout')
- udevadm.settle(timeout)
+ try:
+ udevadm.settle(timeout)
+ except udevadm.Timeout as e:
+ log.warning("Timeout waiting for udev events: %s", e)
def deduceType(a, b):
diff --git a/vdsm/storage/storageServer.py b/vdsm/storage/storageServer.py
index 22a90d1..c19fb8d 100644
--- a/vdsm/storage/storageServer.py
+++ b/vdsm/storage/storageServer.py
@@ -382,7 +382,10 @@
def connect(self):
iscsi.addIscsiNode(self._iface, self._target, self._cred)
timeout = config.getint("irs", "scsi_settle_timeout")
- udevadm.settle(timeout)
+ try:
+ udevadm.settle(timeout)
+ except udevadm.Timeout as e:
+ self.log.warning("Timeout waiting for udev events: %s", e)
def _match(self, session):
target = session.target
diff --git a/vdsm/storage/udevadm.py b/vdsm/storage/udevadm.py
index 4b4b54a..a2afd04 100644
--- a/vdsm/storage/udevadm.py
+++ b/vdsm/storage/udevadm.py
@@ -18,22 +18,39 @@
# Refer to the README and COPYING files for full details of the license
#
-import logging
+import errno
+import signal
+
from vdsm import utils
+from vdsm.infra import zombiereaper
_UDEVADM = utils.CommandPath("udevadm", "/sbin/udevadm", "/usr/sbin/udevadm")
class Error(Exception):
+ message = None
- def __init__(self, rc, out, err):
+ def __str__(self):
+ return self.message.format(self=self)
+
+
+class Failure(Error):
+ message = ("udevadm failed cmd={self.cmd} rc={self.rc} out={self.out!r} "
+ "err={self.err!r}")
+
+ def __init__(self, cmd, rc, out, err):
+ self.cmd = cmd
self.rc = rc
self.out = out
self.err = err
- def __str__(self):
- return "Process failed with rc=%d out=%r err=%r" % (
- self.rc, self.out, self.err)
+
+class Timeout(Error):
+ message = ("udevadm timed out cmd={self.cmd} timeout={self.timeout}")
+
+ def __init__(self, cmd, timeout):
+ self.cmd = cmd
+ self.timeout = timeout
def settle(timeout, exit_if_exists=None):
@@ -44,25 +61,35 @@
Arguments:
timeout Maximum number of seconds to wait for the event queue to
- become empty. A value of 0 will check if the queue is empty
- and always return immediately.
+ become empty.
exit_if_exists Stop waiting if file exists.
+
+ Raises Failure if udevadm failed, or Timeout if udevadm did not terminate
+ within the requested timeout.
"""
- args = ["settle", "--timeout=%s" % timeout]
+ cmd = [_UDEVADM.cmd, "settle"]
if exit_if_exists:
- args.append("--exit-if-exists=%s" % exit_if_exists)
+ cmd.append("--exit-if-exists=%s" % exit_if_exists)
- try:
- _run_command(args)
- except Error as e:
- logging.error("%s", e)
+ _run_command(cmd, timeout)
-def _run_command(args):
- cmd = [_UDEVADM.cmd]
- cmd.extend(args)
- rc, out, err = utils.execCmd(cmd, raw=True)
- if rc != 0:
- raise Error(rc, out, err)
+def _run_command(cmd, timeout=None):
+ proc = utils.execCmd(cmd, sync=False, deathSignal=signal.SIGKILL)
+
+ if not proc.wait(timeout):
+ try:
+ proc.kill()
+ except OSError as e:
+ if e.errno != errno.ESRCH:
+ raise
+ finally:
+ zombiereaper.autoReapPID(proc.pid)
+ raise Timeout(cmd, timeout)
+
+ if proc.returncode != 0:
+ out = "".join(proc.stdout)
+ err = "".join(proc.stderr)
+ raise Failure(cmd, proc.returncode, out, err)
--
To view, visit https://gerrit.ovirt.org/39740
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia0a7380b1b181ec93399ea741122cfa2e98086fb
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Roman Mohr has uploaded a new change for review.
Change subject: monitor: Add udev monitor
......................................................................
monitor: Add udev monitor
Allow easy monitoring of device hotplug and device state changes which
are covered by udev.
Change-Id: I4b91753424d83896fa538eb6b57f8653b6332fbb
Signed-off-by: Roman Mohr <rmohr(a)redhat.com>
---
A lib/vdsm/udev/monitor.py
M tests/Makefile.am
A tests/udevMonitorTests.py
M vdsm.spec.in
4 files changed, 272 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/29/47729/1
diff --git a/lib/vdsm/udev/monitor.py b/lib/vdsm/udev/monitor.py
new file mode 100644
index 0000000..8698bf3
--- /dev/null
+++ b/lib/vdsm/udev/monitor.py
@@ -0,0 +1,133 @@
+# Copyright 2015 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+from functools import partial
+import logging
+import threading
+from time import sleep
+
+from pyudev import Context, Monitor, MonitorObserver
+
+
+class UdevMonitor(object):
+
+ """
+ ``udev`` event monitor. Usage:
+
+ The monitor is a thread-save thin wrapper arount pyudev.MonitorObserver.
+ This allows multiple callbacks for the same netlink socket. To avoid
+ listening for udev events the application is not interested in, the
+ monitoring thread only starts listening on the socket when the monitor is
+ started and at least one subscriber is added.
+
+ The simplest way to use the monitor is to subscribe a callback to a
+ specific subsystem event and let the callback do the work:
+
+ dev listen_for_disabled_cpu_events(device):
+ if device.action == 'offline':
+ print('CPU {0.name} is now offline'.format(device))
+
+ monitor = UdevMonitor()
+ monitor.start()
+ monitor.subscribe(listen_for_disabled_cpu_events, subsystem='cpu')
+
+ Another approach would be to just enqueue the udev event in a queue and do
+ the actual work in another thread or a thread pool. This is one approach to
+ do this:
+
+ queue = Queue.Queue()
+ def new_device_listener(device):
+ if device.action == 'add':
+ queue.put(device)
+
+ monitor = UdevMonitor()
+ monitor.subscribe(new_device_listener, subsystem='usb',
+ device_type='usb_device')
+ monitor.start()
+ """
+
+ def __init__(self):
+ self._subsystems = {}
+ self._context = Context()
+ self._monitor = Monitor.from_netlink(self._context)
+ self._observer = MonitorObserver(self._monitor, callback=partial(
+ UdevMonitor._event_loop, self), name='udev-monitor')
+ self._filter_lock = threading.Lock()
+ self._is_started = False
+ self._can_start = False
+
+ def _event_loop(self, device):
+ subsystem = self._subsystems[device.subsystem]
+ for callback in subsystem.get(device.device_type, []):
+ _execute_callback(callback, device)
+ if device.device_type:
+ for callback in subsystem.get(None, []):
+ _execute_callback(callback, device)
+
+ def subscribe(self, callback, subsystem, device_type=None):
+ """
+ Raise :exc:`~exceptions.ValueError` if the callback is None
+
+ :param callback: function to invoke
+ :param subsystem: byte or unicode string representing the subsystem to
+ listen on (e.g. ``'cpu'``, ``'usb'``)
+ :param device_type: byte or unicode string representing the device type
+ to listen for changes (e.g. ``'usb_device'``,
+ ``'block'``)
+ :return: None
+ """
+ if callback is None:
+ raise ValueError('callback missing')
+ with self._filter_lock:
+ self._monitor.filter_by(subsystem, device_type)
+ device_types = self._subsystems.get(subsystem, {})
+ callbacks = device_types.get(device_type, [])
+ callbacks.append(callback)
+ device_types[device_type] = callbacks
+ self._subsystems[subsystem] = device_types
+ self._start_if_necessary()
+
+ def start(self):
+ self._can_start = True
+ with self._filter_lock:
+ if self._subsystems:
+ self._start_if_necessary()
+
+ def _start_if_necessary(self):
+ if self._can_start and not self._is_started:
+ self._observer.start()
+ self._is_started = True
+
+ def stop(self):
+ """
+ Stops the monitoring thread. It is guaranteed that callbacks are no
+ longer invoked after calling this method. The method can be called
+ multiple times.
+
+ Note that this only stops the monitoring thread and not the monitor
+ itself. The monitor stops listening when it is dereferenced.
+ """
+ self._observer.stop()
+
+
+def _execute_callback(callback, device):
+ try:
+ callback(device)
+ except Exception as callbackException:
+ logging.error(
+ 'Callback execution threw an exception: %s', callbackException)
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 1168862..7483d0c 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -110,6 +110,7 @@
testlibTests.py \
toolTests.py \
transportWrapperTests.py \
+ udevMonitorTests.py \
utilsTests.py \
vdscliTests.py \
vdsClientTests.py \
diff --git a/tests/udevMonitorTests.py b/tests/udevMonitorTests.py
new file mode 100644
index 0000000..2526459
--- /dev/null
+++ b/tests/udevMonitorTests.py
@@ -0,0 +1,137 @@
+# Copyright 2015 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+
+from functools import partial
+
+import Queue
+
+from nettestlib import Bridge
+from testValidation import ValidateRunningAsRoot
+from testlib import VdsmTestCase
+from vdsm.udev.monitor import UdevMonitor
+
+
+class UdevMonitorTest(VdsmTestCase):
+
+ def setUp(self):
+ self._queue = Queue.Queue()
+ self._monitor = UdevMonitor()
+ self._monitor.start()
+
+ def tearDown(self):
+ self._monitor.stop()
+
+ @ValidateRunningAsRoot
+ def testAddDevice(self):
+ self._monitor.subscribe(
+ partial(UdevMonitorTest._device_listener, self),
+ subsystem='net',
+ device_type='bridge')
+ bridge = Bridge()
+ try:
+ bridge.addDevice()
+ device = None
+ try:
+ device = self._queue.get(timeout=1)
+ except Queue.Empty:
+ pass
+ self.assertIsNotNone(device,
+ msg='Should have detected an event')
+ self.assertEqual(device.action, 'add')
+ finally:
+ bridge.delDevice()
+
+ @ValidateRunningAsRoot
+ def testAddDeviceForSubsystem(self):
+ self._monitor.subscribe(
+ partial(UdevMonitorTest._device_listener, self),
+ subsystem='net',
+ device_type=None)
+ bridge = Bridge()
+ try:
+ bridge.addDevice()
+ device = None
+ try:
+ device = self._queue.get(timeout=1)
+ except Queue.Empty:
+ pass
+ self.assertIsNotNone(device,
+ msg='Should have detected an event')
+ self.assertEqual(device.action, 'add')
+ finally:
+ bridge.delDevice()
+
+ @ValidateRunningAsRoot
+ def testRemoveDevice(self):
+
+ bridge = Bridge()
+ try:
+ bridge.addDevice()
+ self._monitor.subscribe(
+ partial(UdevMonitorTest._device_listener, self),
+ subsystem='net',
+ device_type='bridge')
+ bridge.delDevice()
+ device = None
+ try:
+ device = self._queue.get(timeout=1)
+ except Queue.Empty:
+ pass
+ self.assertIsNotNone(device,
+ msg='Should have detected an event')
+ self.assertEqual(device.action, 'remove')
+ except:
+ bridge.delDevice()
+
+ @ValidateRunningAsRoot
+ def testEventSeries(self):
+
+ bridge1 = Bridge()
+ bridge2 = Bridge()
+ try:
+ self._monitor.subscribe(
+ partial(UdevMonitorTest._device_listener, self),
+ subsystem='net',
+ device_type='bridge')
+ bridge1.addDevice()
+ bridge1.delDevice()
+ bridge2.addDevice()
+ bridge2.delDevice()
+ events = []
+ try:
+ events.append(self._queue.get(timeout=1))
+ events.append(self._queue.get(timeout=1))
+ events.append(self._queue.get(timeout=1))
+ events.append(self._queue.get(timeout=1))
+ except Queue.Empty:
+ pass
+ self.assertEqual(len(events), 4)
+ except:
+ _delBridge(bridge1)
+ _delBridge(bridge2)
+
+ def _device_listener(self, device):
+ self._queue.put(device)
+
+
+def _delBridge(bridge):
+ try:
+ bridge.delDevice()
+ except:
+ pass
diff --git a/vdsm.spec.in b/vdsm.spec.in
index 8f758ca..ae5e483 100644
--- a/vdsm.spec.in
+++ b/vdsm.spec.in
@@ -126,6 +126,7 @@
Requires: python-pthreading >= 0.1.3-3
Requires: python-six
Requires: python-requests
+Requires: python-udev >= 0.15
Requires: %{name}-infra = %{version}-%{release}
Requires: rpm-python
Requires: nfs-utils
--
To view, visit https://gerrit.ovirt.org/47729
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I4b91753424d83896fa538eb6b57f8653b6332fbb
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Roman Mohr <rmohr(a)redhat.com>
Adam Litke has uploaded a new change for review.
Change subject: VolumeMetadata: move prepare and teardown
......................................................................
VolumeMetadata: move prepare and teardown
Change-Id: Iba0954ace3b1da5ea9afc41aeb6ea69a729fe29c
Signed-off-by: Adam Litke <alitke(a)redhat.com>
---
M vdsm/storage/blockVolume.py
M vdsm/storage/fileVolume.py
M vdsm/storage/volume.py
3 files changed, 117 insertions(+), 109 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/50/44050/1
diff --git a/vdsm/storage/blockVolume.py b/vdsm/storage/blockVolume.py
index 0168d64..597e7ee 100644
--- a/vdsm/storage/blockVolume.py
+++ b/vdsm/storage/blockVolume.py
@@ -74,6 +74,8 @@
volume.VolumeMetadata.__init__(self, repoPath, sdUUID, imgUUID,
volUUID)
self.metaoff = None
+ self.lvmActivationNamespace = sd.getNamespace(self.sdUUID,
+ LVM_ACTIVATION_NAMESPACE)
def getMetadataId(self):
"""
@@ -332,6 +334,49 @@
lvs = lvm.lvsByTag(sdUUID, "%s%s" % (TAG_PREFIX_IMAGE, imgUUID))
return [lv.name for lv in lvs]
+ @logskip("ResourceManager")
+ def llPrepare(self, rw=False, setrw=False):
+ """
+ Perform low level volume use preparation
+
+ For the Block Volumes the actual LV activation is wrapped
+ into lvmActivation resource. It is being initialized by the
+ storage domain sitting on top of the encapsulating VG.
+ We just use it here.
+ """
+ if setrw:
+ self.setrw(rw=rw)
+ access = rm.LockType.exclusive if rw else rm.LockType.shared
+ activation = rmanager.acquireResource(self.lvmActivationNamespace,
+ self.volUUID, access)
+ activation.autoRelease = False
+
+ @classmethod
+ def teardown(cls, sdUUID, volUUID, justme=False):
+ """
+ Deactivate volume and release resources.
+ Volume deactivation occurs as part of resource releasing.
+ If justme is false, the entire COW chain should be torn down.
+ """
+ cls.log.info("Tearing down volume %s/%s justme %s"
+ % (sdUUID, volUUID, justme))
+ lvmActivationNamespace = sd.getNamespace(sdUUID,
+ LVM_ACTIVATION_NAMESPACE)
+ rmanager.releaseResource(lvmActivationNamespace, volUUID)
+ if not justme:
+ try:
+ pvolUUID = _getVolumeTag(sdUUID, volUUID, TAG_PREFIX_PARENT)
+ except Exception as e:
+ # If storage not accessible or lvm error occurred
+ # we will failure to get the parent volume.
+ # We can live with it and still succeed in volume's teardown.
+ pvolUUID = volume.BLANK_UUID
+ cls.log.warn("Failure to get parent of volume %s/%s (%s)"
+ % (sdUUID, volUUID, e))
+
+ if pvolUUID != volume.BLANK_UUID:
+ cls.teardown(sdUUID=sdUUID, volUUID=pvolUUID, justme=False)
+
class BlockVolume(volume.Volume):
""" Actually represents a single volume (i.e. part of virtual disk).
@@ -341,8 +386,6 @@
def __init__(self, repoPath, sdUUID, imgUUID, volUUID):
md = self.MetadataClass(repoPath, sdUUID, imgUUID, volUUID)
volume.Volume.__init__(self, md)
- self.lvmActivationNamespace = sd.getNamespace(self.sdUUID,
- LVM_ACTIVATION_NAMESPACE)
@property
def metaoff(self):
@@ -609,49 +652,6 @@
def shareVolumeRollback(cls, taskObj, volPath):
cls.log.info("Volume rollback for volPath=%s", volPath)
utils.rmFile(volPath)
-
- @logskip("ResourceManager")
- def llPrepare(self, rw=False, setrw=False):
- """
- Perform low level volume use preparation
-
- For the Block Volumes the actual LV activation is wrapped
- into lvmActivation resource. It is being initialized by the
- storage domain sitting on top of the encapsulating VG.
- We just use it here.
- """
- if setrw:
- self.setrw(rw=rw)
- access = rm.LockType.exclusive if rw else rm.LockType.shared
- activation = rmanager.acquireResource(self.lvmActivationNamespace,
- self.volUUID, access)
- activation.autoRelease = False
-
- @classmethod
- def teardown(cls, sdUUID, volUUID, justme=False):
- """
- Deactivate volume and release resources.
- Volume deactivation occurs as part of resource releasing.
- If justme is false, the entire COW chain should be torn down.
- """
- cls.log.info("Tearing down volume %s/%s justme %s"
- % (sdUUID, volUUID, justme))
- lvmActivationNamespace = sd.getNamespace(sdUUID,
- LVM_ACTIVATION_NAMESPACE)
- rmanager.releaseResource(lvmActivationNamespace, volUUID)
- if not justme:
- try:
- pvolUUID = _getVolumeTag(sdUUID, volUUID, TAG_PREFIX_PARENT)
- except Exception as e:
- # If storage not accessible or lvm error occurred
- # we will failure to get the parent volume.
- # We can live with it and still succeed in volume's teardown.
- pvolUUID = volume.BLANK_UUID
- cls.log.warn("Failure to get parent of volume %s/%s (%s)"
- % (sdUUID, volUUID, e))
-
- if pvolUUID != volume.BLANK_UUID:
- cls.teardown(sdUUID=sdUUID, volUUID=pvolUUID, justme=False)
def getVolumeTag(self, tagPrefix):
return self.md.getVolumeTag(tagPrefix)
diff --git a/vdsm/storage/fileVolume.py b/vdsm/storage/fileVolume.py
index 2dad91f..08e3f50 100644
--- a/vdsm/storage/fileVolume.py
+++ b/vdsm/storage/fileVolume.py
@@ -337,6 +337,26 @@
volList.append(volid)
return volList
+ def llPrepare(self, rw=False, setrw=False):
+ """
+ Make volume accessible as readonly (internal) or readwrite (leaf)
+ """
+ volPath = self.getVolumePath()
+
+ # Volumes leaves created in 2.2 did not have group writeable bit
+ # set. We have to set it here if we want qemu-kvm to write to old
+ # NFS volumes.
+ self.oop.fileUtils.copyUserModeToGroup(volPath)
+
+ if setrw:
+ self.setrw(rw=rw)
+ if rw:
+ if not self.oop.os.access(volPath, os.R_OK | os.W_OK):
+ raise se.VolumeAccessError(volPath)
+ else:
+ if not self.oop.os.access(volPath, os.R_OK):
+ raise se.VolumeAccessError(volPath)
+
class FileVolume(volume.Volume):
""" Actually represents a single volume (i.e. part of virtual disk).
@@ -488,26 +508,6 @@
procPool.utils.rmFile(volPath)
procPool.utils.rmFile(cls.__metaVolumePath(volPath))
procPool.utils.rmFile(cls.MetadataClass._leaseVolumePath(volPath))
-
- def llPrepare(self, rw=False, setrw=False):
- """
- Make volume accessible as readonly (internal) or readwrite (leaf)
- """
- volPath = self.getVolumePath()
-
- # Volumes leaves created in 2.2 did not have group writeable bit
- # set. We have to set it here if we want qemu-kvm to write to old
- # NFS volumes.
- self.oop.fileUtils.copyUserModeToGroup(volPath)
-
- if setrw:
- self.setrw(rw=rw)
- if rw:
- if not self.oop.os.access(volPath, os.R_OK | os.W_OK):
- raise se.VolumeAccessError(volPath)
- else:
- if not self.oop.os.access(volPath, os.R_OK):
- raise se.VolumeAccessError(volPath)
@classmethod
def __putMetadata(cls, metaId, meta):
diff --git a/vdsm/storage/volume.py b/vdsm/storage/volume.py
index a098eb7..22f7ead 100644
--- a/vdsm/storage/volume.py
+++ b/vdsm/storage/volume.py
@@ -525,6 +525,56 @@
return apparentSize
return req_size
+ def prepare(self, rw=True, justme=False,
+ chainrw=False, setrw=False, force=False):
+ """
+ Prepare volume for use by consumer.
+ If justme is false, the entire COW chain is prepared.
+ Note: setrw arg may be used only by SPM flows.
+ """
+ self.log.info("Volume: preparing volume %s/%s",
+ self.sdUUID, self.volUUID)
+
+ if not force:
+ # Cannot prepare ILLEGAL volume
+ if not self.isLegal():
+ raise se.prepareIllegalVolumeError(self.volUUID)
+
+ if rw and self.isShared():
+ if chainrw:
+ rw = False # Shared cannot be set RW
+ else:
+ raise se.SharedVolumeNonWritable(self)
+
+ if (not chainrw and rw and self.isInternal() and setrw and
+ not self.recheckIfLeaf()):
+ raise se.InternalVolumeNonWritable(self)
+
+ self.llPrepare(rw=rw, setrw=setrw)
+ self.updateInvalidatedSize()
+
+ try:
+ if justme:
+ return True
+ pvol = self.produceParent()
+ if pvol:
+ pvol.prepare(rw=chainrw, justme=False,
+ chainrw=chainrw, setrw=setrw)
+ except Exception:
+ self.log.error("Unexpected error", exc_info=True)
+ self.teardown(self.sdUUID, self.volUUID)
+ raise
+
+ return True
+
+ @classmethod
+ def teardown(cls, sdUUID, volUUID, justme=False):
+ """
+ Teardown volume.
+ If justme is false, the entire COW chain is teared down.
+ """
+ pass
+
class Volume(object):
log = logging.getLogger('Storage.Volume')
@@ -1091,53 +1141,11 @@
def prepare(self, rw=True, justme=False,
chainrw=False, setrw=False, force=False):
- """
- Prepare volume for use by consumer.
- If justme is false, the entire COW chain is prepared.
- Note: setrw arg may be used only by SPM flows.
- """
- self.log.info("Volume: preparing volume %s/%s",
- self.sdUUID, self.volUUID)
-
- if not force:
- # Cannot prepare ILLEGAL volume
- if not self.isLegal():
- raise se.prepareIllegalVolumeError(self.volUUID)
-
- if rw and self.isShared():
- if chainrw:
- rw = False # Shared cannot be set RW
- else:
- raise se.SharedVolumeNonWritable(self)
-
- if (not chainrw and rw and self.isInternal() and setrw and
- not self.recheckIfLeaf()):
- raise se.InternalVolumeNonWritable(self)
-
- self.llPrepare(rw=rw, setrw=setrw)
- self.updateInvalidatedSize()
-
- try:
- if justme:
- return True
- pvol = self.produceParent()
- if pvol:
- pvol.prepare(rw=chainrw, justme=False,
- chainrw=chainrw, setrw=setrw)
- except Exception:
- self.log.error("Unexpected error", exc_info=True)
- self.teardown(self.sdUUID, self.volUUID)
- raise
-
- return True
+ return self.md.prepare(rw, justme, chainrw, setrw, force)
@classmethod
def teardown(cls, sdUUID, volUUID, justme=False):
- """
- Teardown volume.
- If justme is false, the entire COW chain is teared down.
- """
- pass
+ cls.MetadataClass.teardown(sdUUID, volUUID, justme)
@classmethod
def newMetadata(cls, metaId, sdUUID, imgUUID, puuid, size, format, type,
--
To view, visit https://gerrit.ovirt.org/44050
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iba0954ace3b1da5ea9afc41aeb6ea69a729fe29c
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com>
Adam Litke has uploaded a new change for review.
Change subject: storage: Make Image.__chainSizeCalc public
......................................................................
storage: Make Image.__chainSizeCalc public
The new SDM copyVolumeData wants to make use of the same logic being
used by the classic copy flows to extend the size of the target volume
to the appropriate size. Make Image.__chainSizeCalc public so it can be
accessed from the SDM code.
Change-Id: Id079eb5067c16f934370e42b5f4e09bbcef1512b
Signed-off-by: Adam Litke <alitke(a)redhat.com>
---
M vdsm/storage/image.py
1 file changed, 2 insertions(+), 2 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/95/38995/1
diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py
index 791b48c..8d5e8c2 100644
--- a/vdsm/storage/image.py
+++ b/vdsm/storage/image.py
@@ -140,7 +140,7 @@
randomStr = misc.randomStr(RENAME_RANDOM_STRING_LEN)
return "%s%s_%s" % (sd.REMOVED_IMAGE_PREFIX, randomStr, uuid)
- def __chainSizeCalc(self, sdUUID, imgUUID, volUUID, size):
+ def chainSizeCalc(self, sdUUID, imgUUID, volUUID, size):
"""
Compute an estimate of the whole chain size
using the sum of the actual size of the chain's volumes
@@ -763,7 +763,7 @@
if volParams['volFormat'] != volume.COW_FORMAT or \
volParams['prealloc'] != volume.SPARSE_VOL:
raise se.IncorrectFormat(self)
- volParams['apparentsize'] = self.__chainSizeCalc(
+ volParams['apparentsize'] = self.chainSizeCalc(
sdUUID, srcImgUUID, srcVolUUID, volParams['size'])
# Find out dest volume parameters
--
To view, visit https://gerrit.ovirt.org/38995
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Id079eb5067c16f934370e42b5f4e09bbcef1512b
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: contrib: Add tool for getting hsm stats
......................................................................
contrib: Add tool for getting hsm stats
This is a very simple tool showing historgam of hsm calls in the given
log files.
Example:
$ hsmstat vdsm.log.1 vdsm.log.2
8515 getVolumeSize
2258 repoStats
51 getVolumeInfo
30 getVolumesList
19 teardownImage
...
We can get same output or better using logdb by writing some smart sql.
Change-Id: Ia6013da71ca96535b98edace8577a4d8bb529465
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
A contrib/hsmstat
1 file changed, 39 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/48/38748/1
diff --git a/contrib/hsmstat b/contrib/hsmstat
new file mode 100755
index 0000000..d3400d2
--- /dev/null
+++ b/contrib/hsmstat
@@ -0,0 +1,39 @@
+#!/usr/bin/env python
+#
+# Copyright 2015 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+
+from collections import defaultdict
+import fileinput
+import re
+
+verb_re = re.compile(r" Run and protect: (?P<name>.+)\(")
+
+verbs = defaultdict(int)
+
+for line in fileinput.input():
+ match = verb_re.search(line)
+ if match:
+ name = match.group('name')
+ verbs[name] += 1
+
+histogram = sorted(((count, verb) for verb, count in verbs.iteritems()),
+ reverse=True)
+for count, name in histogram:
+ print " %6d %s" % (count, name)
--
To view, visit https://gerrit.ovirt.org/38748
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia6013da71ca96535b98edace8577a4d8bb529465
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: contrib: Add tool for getting repoStats statistics
......................................................................
contrib: Add tool for getting repoStats statistics
This tool is useful when debugging storage domain monitoring issues.
Examples:
$ repostat vdsm.log.*
domain: ebcd25b1-1ab0-47e3-aa8b-83ee5f91b5bd
delay avg: 0.093941 min: 0.000430 max: 1.499200
last check avg: 5.117508 min: 0.000000 max: 12.300000
domain: 6d7f7a48-4511-43b8-8abf-d323f7118b19
delay avg: 0.001090 min: 0.000300 max: 0.030056
last check avg: 4.975313 min: 0.000000 max: 10.200000
Change-Id: Id5f7828216058bb401f8a472fa5601e79542def1
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M Makefile.am
A contrib/repostat
2 files changed, 55 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/49/38749/1
diff --git a/Makefile.am b/Makefile.am
index 34885a1..7407b22 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -49,6 +49,7 @@
autogen.sh \
build-aux/pkg-version \
contrib/profile-stats \
+ contrib/repostat \
pylintrc \
vdsm.spec \
vdsm.spec.in \
@@ -80,6 +81,7 @@
WHITELIST = \
contrib/profile-stats \
+ contrib/repostat \
init/daemonAdapter \
vdsm/get-conf-item \
vdsm/set-conf-item \
diff --git a/contrib/repostat b/contrib/repostat
new file mode 100755
index 0000000..c140b4c
--- /dev/null
+++ b/contrib/repostat
@@ -0,0 +1,53 @@
+#!/usr/bin/env python
+#
+# Copyright 2015 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+"""
+Parse repoStats log lines and calculate statistics.
+
+Usage: repostat vdsm.log [...]
+"""
+
+import fileinput
+import re
+
+repostat_re = re.compile(r' Run and protect: repoStats, Return response: ')
+
+stats = {}
+
+
+def liststat(a):
+ b = sorted(a)
+ return sum(b) / len(b), b[0], b[-1]
+
+
+for line in fileinput.input():
+ match = repostat_re.search(line)
+ if not match:
+ continue
+ response = eval(line[match.end():])
+ for uuid, info in response.items():
+ stats.setdefault(uuid, {'delays': [], 'checks': []})
+ stats[uuid]['delays'].append(float(info['delay']))
+ stats[uuid]['checks'].append(float(info['lastCheck']))
+
+for uuid, info in stats.iteritems():
+ print 'domain:', uuid
+ print ' delay avg: %f min: %f max: %f' % liststat(info['delays'])
+ print ' last check avg: %f min: %f max: %f' % liststat(info['checks'])
--
To view, visit https://gerrit.ovirt.org/38749
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Id5f7828216058bb401f8a472fa5601e79542def1
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: vm: Support for non-ascii vm name
......................................................................
vm: Support for non-ascii vm name
Libvirt does not support non-ascii vm name, failing creation of such vm.
Use generated vm name based on vm id, used when vm name is not provided.
For easier debugging, we log he non-ascii name when starting a vm:
Non-ASCII vm name 'מכונה-ויראטואלית'
And also the generaed name (same when vm name is not provided):
Using generated vm name 'nb938182d-7f19-435b-9499-4cb8f1578e1b'
Note: this change may break engine side, if it expects the original vm
name in vdsm response. Fixing this on the engine side will avoid this
issue and may be better idea.
Tested with both jsonrpc and xmlrpc:
- Creating vm with non-ascii name
- Staring vm
- Opening spice console
- Stopping vm
- Restarting vdsm while vm is running
- Migrating to another host (broken with jsonrpc see bz1282054)
Change-Id: I2386c0f98aeb8161feaf19c2862be229f73eb0df
Bug-Url: https://bugzilla.redhat.com/1260131
Relates-To: https://bugzilla.redhat.com/1062943
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm/virt/vm.py
1 file changed, 17 insertions(+), 2 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/70/48570/1
diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py
index f9bc25c..a6d2bc8 100644
--- a/vdsm/virt/vm.py
+++ b/vdsm/virt/vm.py
@@ -299,8 +299,7 @@
self._devices = self._emptyDevMap()
self._connection = libvirtconnection.get(cif)
- if 'vmName' not in self.conf:
- self.conf['vmName'] = 'n%s' % self.id
+ self.conf['vmName'] = self._normalizedVmName(params)
self._guestSocketFile = self._makeChannelPath(_VMCHANNEL_DEVICE_NAME)
self._qemuguestSocketFile = self._makeChannelPath(_QEMU_GA_DEVICE_NAME)
self.guestAgent = guestagent.GuestAgent(
@@ -380,6 +379,22 @@
break
return str(idx)
+ def _normalizedVmName(self, params):
+ name = params.get('vmName')
+ if isinstance(name, unicode):
+ # Libvirt does not support non-ascii vm name
+ # See https://bugzilla.redhat.com/1260131
+ try:
+ name = name.encode('ascii')
+ except UnicodeEncodeError:
+ self.log.debug("Non-ASCII vm name '%s'", name.encode('utf8'))
+ name = None
+ if name is None:
+ # Missing or non-ascii name
+ name = 'n' + params['vmId']
+ self.log.debug("Using generated vm name %r", name)
+ return name
+
def _normalizeVdsmImg(self, drv):
drv['reqsize'] = drv.get('reqsize', '0') # Backward compatible
if 'device' not in drv:
--
To view, visit https://gerrit.ovirt.org/48570
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2386c0f98aeb8161feaf19c2862be229f73eb0df
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: automation: Enable xunit report
......................................................................
automation: Enable xunit report
This used to segfault python in the past, but I think I fixed the root
cause long time ago.
Trying to use this again since it seems to give interesting output
like the time it took to run each test.
Change-Id: Ie8c09cf21e7cf4a97b7b006f2b3ebe4943ed72b9
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M automation/check-patch.sh
1 file changed, 1 insertion(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/40/46640/1
diff --git a/automation/check-patch.sh b/automation/check-patch.sh
index 15707e8..1dba426 100755
--- a/automation/check-patch.sh
+++ b/automation/check-patch.sh
@@ -14,7 +14,7 @@
"
./autogen.sh --system --enable-hooks
-make check
+make check NOSE_WITH_XUNIT=1
./automation/build-artifacts.sh
--
To view, visit https://gerrit.ovirt.org/46640
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie8c09cf21e7cf4a97b7b006f2b3ebe4943ed72b9
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: fileSD: Keep deleted images name unique
......................................................................
fileSD: Keep deleted images name unique
If an image delete fails, and same image is created again, the second
image cannot be deleted, since the first image occupy the deleted image
name.
We use now _remove_me_<imagid>_<taskid>. This make it easy to locate the
failed operation in vdsm or engine logs.
Change-Id: Ia0582896d7ede028e9096da1a688e3e449ab0258
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm/storage/fileSD.py
1 file changed, 3 insertions(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/32/48032/1
diff --git a/vdsm/storage/fileSD.py b/vdsm/storage/fileSD.py
index dc4a68e..7d8074a 100644
--- a/vdsm/storage/fileSD.py
+++ b/vdsm/storage/fileSD.py
@@ -34,6 +34,7 @@
import outOfProcess as oop
from remoteFileHandler import Timeout
from persistentDict import PersistentDict, DictValidator
+from threadLocal import vars
from vdsm import constants
from vdsm.utils import stripNewLines
from vdsm import supervdsm
@@ -204,7 +205,8 @@
def deleteImage(self, sdUUID, imgUUID, volsImgs):
currImgDir = self.getImagePath(imgUUID)
dirName, baseName = os.path.split(currImgDir)
- toDelDir = os.path.join(dirName, sd.REMOVED_IMAGE_PREFIX + baseName)
+ newName = "%s%s_%s" % (sd.REMOVED_IMAGE_PREFIX, baseName, vars.task.id)
+ toDelDir = os.path.join(dirName, newName)
self.log.debug("Renaming dir %s to %s", currImgDir, toDelDir)
try:
self.oop.os.rename(currImgDir, toDelDir)
--
To view, visit https://gerrit.ovirt.org/48032
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia0582896d7ede028e9096da1a688e3e449ab0258
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Ondřej Svoboda has uploaded a new change for review.
Change subject: ipv6 hook: support ipv6addrs
......................................................................
ipv6 hook: support ipv6addrs
Change-Id: I722d0007840060cdccdb12e4fba2a2066f33c62f
Signed-off-by: Ondřej Svoboda <osvoboda(a)redhat.com>
---
M vdsm_hooks/ipv6/README
M vdsm_hooks/ipv6/ipv6.py
2 files changed, 12 insertions(+), 5 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/59/39359/1
diff --git a/vdsm_hooks/ipv6/README b/vdsm_hooks/ipv6/README
index 05bd990..a96e2c2 100644
--- a/vdsm_hooks/ipv6/README
+++ b/vdsm_hooks/ipv6/README
@@ -5,13 +5,18 @@
Requirements:
* oVirt-3.5 (started supporting custom properties)
+New in version 4.18: (?)
+--------------------
+Multiple IPv6 addresses are now accepted, see Usage. Please use 'ipv6addrs'
+property instead of 'ipv6addr' which is deprecated.
+
Preparation on a host:
----------------------
yum install vdsm-hook-ipv6
Preparation on the engine side:
-------------------------------
-PROPERTIES='ipv6addr=.*;ipv6gateway=.*;ipv6autoconf=.*;dhcpv6=.*'
+PROPERTIES='ipv6addrs=.*;ipv6gateway=.*;ipv6autoconf=.*;dhcpv6=.*'
engine-config -s "UserDefinedNetworkCustomProperties=$PROPERTIES" --cver='3.5'
Don't forget to include the names of other custom network properties you may
@@ -20,9 +25,10 @@
Usage:
------
In the oVirt UI open the 'Setup Host Networks' dialog. Proceed to editing
-a desired logical network's properties. Among them you will find 'ipv6addr'
-and 'ipv6gateway', which accept custom IPv6 addresses, plus 'ipv6autoconf'
-and 'dhcpv6', which accept '0', '1' or 'false', 'true'.
+a desired logical network's properties. Among them you will find 'ipv6addrs'
+(which accepts a space-separated list of IPv6 addresses), 'ipv6gateway'
+(a single address), plus 'ipv6autoconf' and 'dhcpv6', which accept '0', '1' or
+'false', 'true'.
You may be warned that the network is in use when confirming the 'Setup Host
Networks' dialog. Make sure to stop any VMs that are using the network.
diff --git a/vdsm_hooks/ipv6/ipv6.py b/vdsm_hooks/ipv6/ipv6.py
index 6c00475..9385fab 100644
--- a/vdsm_hooks/ipv6/ipv6.py
+++ b/vdsm_hooks/ipv6/ipv6.py
@@ -34,7 +34,8 @@
def _process_network(attrs):
- for property_name in ('ipv6addr', 'ipv6gateway', 'ipv6autoconf', 'dhcpv6'):
+ for property_name in ('ipv6addrs', 'ipv6addr', 'ipv6gateway',
+ 'ipv6autoconf', 'dhcpv6'):
value = attrs['custom'].get(property_name)
if value is not None:
attrs[property_name] = value
--
To view, visit https://gerrit.ovirt.org/39359
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I722d0007840060cdccdb12e4fba2a2066f33c62f
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Ondřej Svoboda <osvoboda(a)redhat.com>
Francesco Romani has uploaded a new change for review.
Change subject: vm: do not call before_vm_create in dehibernation
......................................................................
vm: do not call before_vm_create in dehibernation
For reasons unknown, VDSM calls the before_vm_create hook
also in the dehibernation path.
This is wasteful and useless at best, and most likely misleading
and wrong, because VDSM logs the XML it produced, which is *not* what is
gonna be used in the dehibernation path. The hook is fed with
phony data, and the output of the hook is discarded.
This patch dispel the lie and avoid to call the before_vm_create
hook in the dehibernation path.
Change-Id: I4c1377346b1ae63a1feb15e1458b5141c255c849
X-Backport-To: 3.6
Signed-off-by: Francesco Romani <fromani(a)redhat.com>
---
M vdsm/virt/vm.py
1 file changed, 0 insertions(+), 2 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/74/45074/1
diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py
index c54853a..b77b9df 100644
--- a/vdsm/virt/vm.py
+++ b/vdsm/virt/vm.py
@@ -1887,8 +1887,6 @@
hooks.before_vm_dehibernate(srcDomXML, self.conf,
{'FROM_SNAPSHOT': str(fromSnapshot)})
- # TODO: this silliness is only to preserve the old behaviour
- hooks.before_vm_start(self._buildDomainXML(), self.conf)
# TODO: this is debug information. For 3.6.x we still need to
# see the XML even with 'info' as default level.
self.log.info(srcDomXML)
--
To view, visit https://gerrit.ovirt.org/45074
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I4c1377346b1ae63a1feb15e1458b5141c255c849
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Francesco Romani <fromani(a)redhat.com>
Francesco Romani has uploaded a new change for review.
Change subject: API: streamline and make setLogLevel correct
......................................................................
API: streamline and make setLogLevel correct
According to the schema, setLogLevel should accept
a log level string ('DEBUG', 'INFO',...). But the code
actually blindly casted the given value to int(), and this suggests
the code actually expected an integer value.
To make things a bit user friendlier and comformant to schema, change
the argument to actually be a log level in string format.
Along the way, this patch streamlines the implementation of setLogLevel
and makes it simpler.
Change-Id: Iaddbb12d13bdbaa7a02255ab209da11e42d2ea97
Signed-off-by: Francesco Romani <fromani(a)redhat.com>
---
M vdsm/API.py
1 file changed, 21 insertions(+), 6 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/25/38425/1
diff --git a/vdsm/API.py b/vdsm/API.py
index 5cbda50..85478b1 100644
--- a/vdsm/API.py
+++ b/vdsm/API.py
@@ -1369,13 +1369,22 @@
Doesn't survive a restart
"""
- logging.info('Setting loglevel to %s', level)
- handlers = logging.getLogger().handlers
- [fileHandler] = [h for h in handlers if
- isinstance(h, logging.FileHandler)]
- fileHandler.setLevel(int(level))
+ LEVELS = {
+ 'DEBUG': logging.DEBUG,
+ 'INFO': logging.INFO,
+ 'WARNING': logging.WARNING,
+ 'ERROR': logging.ERROR,
+ 'CRITICAL': logging.CRITICAL
+ }
- return {'status': doneCode}
+ try:
+ log_level = LEVELS[level]
+ except KeyError:
+ return errCode['unavail']
+ else:
+ logging.info('Setting loglevel to %s (%d)', level, log_level)
+ _set_log_level(logging.getLogger(), log_level)
+ return {'status': doneCode}
# VM-related functions
def getVMList(self, fullStatus=False, vmList=()):
@@ -1772,3 +1781,9 @@
logging.warn("options %s is deprecated. Use %s instead" %
(k, _translationMap[k]))
options[_translationMap[k]] = options.pop(k)
+
+
+def _set_log_level(logger, log_level):
+ for handler in logger.handlers:
+ if isinstance(handler, logging.FileHandler):
+ handler.setLevel(log_level)
--
To view, visit https://gerrit.ovirt.org/38425
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iaddbb12d13bdbaa7a02255ab209da11e42d2ea97
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Francesco Romani <fromani(a)redhat.com>
Shmuel Leib Melamud has uploaded a new change for review.
Change subject: virt: Correct VM state before vm.cont() in _recover()
......................................................................
virt: Correct VM state before vm.cont() in _recover()
In SourceThread._recover(), if error occured while hibernating a VM, set
VM status to PAUSED before calling vm.cont(). Otherwise, vm.cont() will
fail to run in SAVING_STATE state and the VM will be left paused.
Change-Id: I5b1c7b4eecacf87ece48dc563fd2da294af0510b
Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1238536
Signed-off-by: Shmuel Melamud <smelamud(a)redhat.com>
---
M vdsm/virt/migration.py
1 file changed, 1 insertion(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/27/47527/1
diff --git a/vdsm/virt/migration.py b/vdsm/virt/migration.py
index 32caccb..f950a6f 100644
--- a/vdsm/virt/migration.py
+++ b/vdsm/virt/migration.py
@@ -213,6 +213,7 @@
self.log.exception("Failed to destroy remote VM")
# if the guest was stopped before migration, we need to cont it
if self.hibernating:
+ self._vm.lastStatus = vmstatus.PAUSED
self._vm.cont()
# either way, migration has finished
self._vm.lastStatus = vmstatus.UP
--
To view, visit https://gerrit.ovirt.org/47527
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I5b1c7b4eecacf87ece48dc563fd2da294af0510b
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Shmuel Leib Melamud <smelamud(a)redhat.com>