December 2014 - vdsm-patches - Fedora Mailing-Lists

Change in vdsm[master]: attempt to make vdsm noarch

by Dan Kenigsberg

Dan Kenigsberg has uploaded a new change for review. Change subject: attempt to make vdsm noarch ...................................................................... attempt to make vdsm noarch I'd like to have a completely architecture-independent binary package. The only remaining bit that I can think of is the old "safelease" utility, which is dropped in this patch. I should be moved to a tiny helper package. However, this patch is not enough - when built on an x86_64 host, rpm still recognizes vdsm.rpm and vdsm-python.rpm as x86_64 instead of noarch. Dropping all mentions of x86_64 from the spec did not help, either. Change-Id: Ib858accfd9ff8c71c86ddc9e7a8628bb8630753e Signed-off-by: Dan Kenigsberg <danken(a)redhat.com> --- M vdsm.spec.in M vdsm/storage/protect/Makefile.am 2 files changed, 9 insertions(+), 15 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/78/30678/1 diff --git a/vdsm.spec.in b/vdsm.spec.in index f104ad9..91b13bd 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -83,9 +83,9 @@ BuildRequires: genisoimage BuildRequires: openssl BuildRequires: m2crypto -%ifarch x86_64 -BuildRequires: python-dmidecode -%endif +#%ifarch x86_64 +#BuildRequires: python-dmidecode +#%endif %if 0%{?rhel} == 6 BuildRequires: python-argparse BuildRequires: python-ordereddict @@ -133,10 +133,10 @@ Requires: mom >= 0.4.1 Requires: numactl -%ifarch x86_64 -Requires: python-dmidecode -Requires: dmidecode -%endif +#%ifarch x86_64 +#Requires: python-dmidecode +#Requires: dmidecode +#%endif %if 0%{?rhel} >= 7 || 0%{?fedora} >= 18 Requires: libvirt-daemon >= 1.0.2-1 @@ -169,7 +169,6 @@ %if 0%{?rhel} Requires: python-ethtool >= 0.6-3 %if 0%{?rhel} >= 7 -ExclusiveArch: x86_64 Requires: device-mapper-multipath Requires: e2fsprogs Requires: fence-agents-all @@ -1034,7 +1033,6 @@ %{_datadir}/%{vdsm_name}/storage/volume.py* %{_datadir}/%{vdsm_name}/storage/imageRepository/__init__.py* %{_datadir}/%{vdsm_name}/storage/imageRepository/formatConverter.py* -%{_libexecdir}/%{vdsm_name}/safelease %{_libexecdir}/%{vdsm_name}/spmprotect.sh %{_libexecdir}/%{vdsm_name}/spmstop.sh %dir %{_libexecdir}/%{vdsm_name}/hooks diff --git a/vdsm/storage/protect/Makefile.am b/vdsm/storage/protect/Makefile.am index feb40a9..1e9eb7c 100644 --- a/vdsm/storage/protect/Makefile.am +++ b/vdsm/storage/protect/Makefile.am @@ -6,11 +6,7 @@ # LICENSE_GPL_v2 which accompany this distribution. # -vdsmexec_PROGRAMS = safelease - dist_vdsmexec_SCRIPTS = \ spmprotect.sh \ - spmstop.sh - -safelease_SOURCES = \ - safelease.c + spmstop.sh \ + $(NULL) -- To view, visit http://gerrit.ovirt.org/30678 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib858accfd9ff8c71c86ddc9e7a8628bb8630753e Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>

9 years, 1 month

5
7
0 / 0

Change in vdsm[master]: block: simplify zeroImgVolumes using threads

by Federico Simoncelli

Federico Simoncelli has uploaded a new change for review. Change subject: block: simplify zeroImgVolumes using threads ...................................................................... block: simplify zeroImgVolumes using threads Change-Id: I665d863085842de8ff93c9aacf7c26277dfb031d Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com> --- M vdsm/storage/blockSD.py 1 file changed, 26 insertions(+), 72 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/30/35630/1 diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py index 36078d0..daaa94f 100644 --- a/vdsm/storage/blockSD.py +++ b/vdsm/storage/blockSD.py @@ -22,7 +22,6 @@ import threading import logging import signal -import select import errno import re from StringIO import StringIO @@ -50,6 +49,7 @@ import resourceManager as rm import mount import supervdsm as svdsm +from threadLocal import vars import volume STORAGE_DOMAIN_TAG = "RHAT_storage_domain" @@ -200,80 +200,34 @@ lvm.removeLVs(sdUUID, vols) -def _zeroVolume(sdUUID, volUUID): - """Fill a block volume. - - This function requires an active LV. - """ - dm = lvm.lvDmDev(sdUUID, volUUID) - size = multipath.getDeviceSize(dm) # Bytes - # TODO: Change for zero 128 M chuncks and log. - # 128 M is the vdsm extent size default - BS = constants.MEGAB # 1024 ** 2 = 1 MiB - count = size / BS - cmd = [constants.EXT_DD, "oflag=%s" % misc.DIRECTFLAG, "if=/dev/zero", - "of=%s" % lvm.lvPath(sdUUID, volUUID), "bs=%s" % BS, - "count=%s" % count] - p = misc.execCmd(cmd, sync=False, nice=utils.NICENESS.HIGH, - ioclass=utils.IOCLASS.IDLE, deathSignal=signal.SIGKILL) - return p - - def zeroImgVolumes(sdUUID, imgUUID, volUUIDs): - ProcVol = namedtuple("ProcVol", "proc, vol") - # Put a sensible value for dd zeroing a 128 M or 1 G chunk and lvremove - # spent time. - ZEROING_TIMEOUT = 60000 # [miliseconds] - log.debug("sd: %s, LVs: %s, img: %s", sdUUID, volUUIDs, imgUUID) - # Following call to changelv is separate since setting rw permission on an - # LV fails if the LV is already set to the same value, hence we would not - # be able to differentiate between a real failure of deltag/addtag and one - # we would like to ignore (permission is the same) - try: - lvm.changelv(sdUUID, volUUIDs, ("--permission", "rw")) - except se.StorageException as e: - # Hope this only means that some volumes were already writable. - log.debug("Ignoring failed permission change: %s", e) - # blank the volumes. - zerofds = {} - poller = select.poll() - for volUUID in volUUIDs: - proc = _zeroVolume(sdUUID, volUUID) - fd = proc.stdout.fileno() - zerofds[fd] = ProcVol(proc, volUUID) - poller.register(fd, select.EPOLLHUP) + taskid = vars.task.id + aborting = vars.task.aborting - # Wait until all the asyncs procs return - # Yes, this is a potentially infinite loop. Kill the vdsm task. - while zerofds: - fdevents = poller.poll(ZEROING_TIMEOUT) # [(fd, event)] - toDelete = [] - for fd, event in fdevents: - proc, vol = zerofds[fd] - if not proc.wait(0): - continue - else: - poller.unregister(fd) - zerofds.pop(fd) - if proc.returncode != 0: - log.error("zeroing %s/%s failed. Zero and remove this " - "volume manually. rc=%s %s", sdUUID, vol, - proc.returncode, proc.stderr.read(1000)) - else: - log.debug("%s/%s was zeroed and will be deleted", - sdUUID, volUUID) - toDelete.append(vol) - if toDelete: - try: - deleteVolumes(sdUUID, toDelete) - except se.CannotRemoveLogicalVolume: - # TODO: Add the list of removed fail volumes to the exception. - log.error("Remove failed for some of VG: %s zeroed volumes: " - "%s", sdUUID, toDelete, exc_info=True) + def zeroVolume(volUUID): + log.debug('starting to zero volume %s on domain %s for ' + 'task %s', volUUID, sdUUID, taskid) + try: + lvm.changelv(sdUUID, volUUIDs, ("--permission", "rw")) + except se.StorageException as e: + # We ignore the failure hoping that the volumes were + # already writable. + log.debug("Ignoring failed permission change: %s", e) - log.debug("finished with VG:%s LVs: %s, img: %s", sdUUID, volUUIDs, - imgUUID) - return + path = lvm.lvPath(sdUUID, volUUID) + size = multipath.getDeviceSize(lvm.lvDmDev(sdUUID, volUUID)) + + try: + misc.ddWatchCopy("/dev/zero", path, aborting, size) + except Exception as e: + log.exception('zeroing operation failed') + raise se.VolumesZeroingError(path) + + deleteVolumes(sdUUID, volUUID) + log.debug('zeroing of volume %s on domain %s for task %s ' + 'completed successfully', volUUID, sdUUID, taskid) + + misc.tmap(zeroVolume, volUUIDs) class VGTagMetadataRW(object): -- To view, visit http://gerrit.ovirt.org/35630 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I665d863085842de8ff93c9aacf7c26277dfb031d Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

9 years, 1 month

4
9
0 / 0

Change in vdsm[master]: Remove configure sanlock from %post installation

by ybronhei＠redhat.com

Yaniv Bronhaim has uploaded a new change for review. Change subject: Remove configure sanlock from %post installation ...................................................................... Remove configure sanlock from %post installation There is no need to run this verb during post as the user should run the full configure flow after installation. This is redundant call that might cause conflicts during upgrades. Change-Id: I6474b6fa6251fc79ecc04ca730b6dad93d2434d2 Signed-off-by: Yaniv Bronhaim <ybronhei(a)redhat.com> --- M vdsm.spec.in 1 file changed, 0 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/94/34394/1 diff --git a/vdsm.spec.in b/vdsm.spec.in index 64e03d0..3baac76 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -759,7 +759,6 @@ fi %post -%{_bindir}/vdsm-tool configure --module sanlock --force >/dev/null %{_bindir}/vdsm-tool sebool-config || : # set the vdsm "secret" password for libvirt %{_bindir}/vdsm-tool set-saslpasswd -- To view, visit http://gerrit.ovirt.org/34394 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I6474b6fa6251fc79ecc04ca730b6dad93d2434d2 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yaniv Bronhaim <ybronhei(a)redhat.com>

9 years, 1 month

7
16
0 / 0

Change in vdsm[master]: Adding --with-local-version flag to configure

by ybronhei＠redhat.com

Yaniv Bronhaim has uploaded a new change for review. Change subject: Adding --with-local-version flag to configure ...................................................................... Adding --with-local-version flag to configure Change-Id: I93dcd1abed31babceb3e36d07ac994765203a0be Signed-off-by: Yaniv Bronhaim <ybronhei(a)redhat.com> --- M configure.ac M vdsm.spec.in 2 files changed, 10 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/02/35202/1 diff --git a/configure.ac b/configure.ac index df133e6..dedad19 100644 --- a/configure.ac +++ b/configure.ac @@ -78,6 +78,13 @@ AC_SUBST([ENABLE_LIBVIRT_SELINUX], ["${enable_libvirt_selinux}"]) AC_ARG_WITH( + [local-version], + [AS_HELP_STRING([--with-local-version], [local version to use])], + [LOCAL_VERSION="${withval}"], + [LOCAL_VERSION=""] +) + +AC_ARG_WITH( [qemu-user], [AS_HELP_STRING( [--with-qemu-user=USER], diff --git a/vdsm.spec.in b/vdsm.spec.in index 4ac6747..2725e6d 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -672,7 +672,9 @@ %if 0%{?enable_autotools} autoreconf -if %endif -%configure %{?with_hooks:--enable-hooks} +%configure \ + %{?with_hooks:--enable-hooks} \ + --with-local-version="%{name}-%{version}-%{release}" make # Setting software_version and software_revision in dsaversion.py baserelease=`echo "%{release}" | sed 's/$[0-9]\+\(\.[0-9]\+$\?\).*/\1/'` -- To view, visit http://gerrit.ovirt.org/35202 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I93dcd1abed31babceb3e36d07ac994765203a0be Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yaniv Bronhaim <ybronhei(a)redhat.com>

9 years, 1 month

4
6
0 / 0

Change in vdsm[master]: utils: add CommandStream class

by Federico Simoncelli

Federico Simoncelli has uploaded a new change for review. Change subject: utils: add CommandStream class ...................................................................... utils: add CommandStream class Change-Id: Ie015368bb9c5992e5c73a149277c59fc4ffbd570 Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com> --- M lib/vdsm/utils.py M tests/utilsTests.py 2 files changed, 188 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/09/33909/1 diff --git a/lib/vdsm/utils.py b/lib/vdsm/utils.py index 244f0e2..4b0ccb7 100644 --- a/lib/vdsm/utils.py +++ b/lib/vdsm/utils.py @@ -313,6 +313,90 @@ timeout = max(0, endtime - time.time()) +class CommandStream(object): + def __init__(self, command, stdoutcb, stderrcb, cwd=None, + deathSignal=0): + self.command = command + self.returncode = None + + self.child = CPopen(self.command, cwd=cwd, close_fds=True, + deathSignal=deathSignal) + + self.epoll = select.epoll() + + self.iocb = { + self.child.stdout.fileno(): stdoutcb, + self.child.stderr.fileno(): stderrcb, + } + + for fd in self.iocb.keys(): + self.epoll.register(fd, select.EPOLLIN) + + def terminate(self): + self.child.terminate() + + def kill(self): + self.child.kill() + + def write(self, data): + self.child.stdin.write(data) + + def flush(self): + self.child.stdin.flush() + + def close(self): + self.child.stdin.close() + + def _epoll_input(self, fileno): + self.iocb[fileno](os.read(fileno, io.DEFAULT_BUFFER_SIZE)) + + def _epoll_event(self, fileno): + self.epoll.unregister(fileno) + del self.iocb[fileno] + + def _epoll_timeout(self, timeout): + fdevents = NoIntrPoll(self.epoll.poll, timeout) + + for fileno, event in fdevents: + if event & select.EPOLLIN: + self._epoll_input(fileno) + elif event & (select.EPOLLHUP | select.EPOLLERR): + self._epoll_event(fileno) + # Trying to collect the child status in case the + # file descriptor was closed because the process + # terminated. + self.returncode = self.child.poll() + + def wait(self, timeout=None): + if timeout is None: + epoll_remaining = -1 + else: + endtime = os.times()[4] + timeout + + while self.returncode is None: + if timeout is not None: + epoll_remaining = endtime - os.times()[4] + if epoll_remaining <= 0: + break + + if len(self.iocb): + self._epoll_timeout(epoll_remaining) + else: + # This is a busy-loop taken from issue5673, and + # python 3.4 still uses this implementation. + # A smarter solution would be using signalfd or + # sigtimedwait but they don't seem to mix well + # with multithreading (especially under heavy + # load: tens of children from tens of threads). + # Anyway we reach this only when both stdout and + # stderr are closed, which means that in most of + # the cases the child is about to die. + time.sleep(0.0005) + self.returncode = self.child.poll() + + return self.returncode + + class AsyncProc(object): """ AsyncProc is a funky class. It wraps a standard subprocess.Popen diff --git a/tests/utilsTests.py b/tests/utilsTests.py index 6e66c02..c279f04 100644 --- a/tests/utilsTests.py +++ b/tests/utilsTests.py @@ -22,8 +22,12 @@ import contextlib import errno import logging +import operator +import signal import sys import threading + +from contextlib import contextmanager from testlib import VdsmTestCase as TestCaseBase from testlib import permutations, expandPermutations @@ -634,3 +638,103 @@ def test_empty(self): self.assertEquals(utils._list2cmdline([]), "") + + +class CommandStreamTests(TestCaseBase): + + @contextmanager + def assertElapsed(self, limit): + start = os.times()[4] + + yield + + elapsed = os.times()[4] - start + + if elapsed < limit: + raise AssertionError("Operation time: %s" % elapsed) + + def assertNoOutput(self, data): + raise AssertionError("Unexpected data: " + repr(data)) + + def test_output(self): + text = "Hello World" + received = bytearray() + + def recv_stdout(buffer): + # cannot use received += buffer with a variable + # defined in the parent function. + operator.iadd(received, buffer) + + p = utils.CommandStream(["echo", "-n", text], + recv_stdout, + self.assertNoOutput) + + retcode = p.wait() + + self.assertEqual(retcode, 0) + self.assertEqual(text, str(received)) + + def test_write(self): + text = "Hello World" + received = bytearray() + + def recv_stdout(buffer): + # cannot use received += buffer with a variable + # defined in the parent function. + operator.iadd(received, buffer) + + p = utils.CommandStream(["cat"], recv_stdout, + self.assertNoOutput) + + p.write(text) + p.flush() + p.close() + + retcode = p.wait() + + self.assertEqual(retcode, 0) + self.assertEqual(text, str(received)) + + def test_timeout(self): + p = utils.CommandStream(["sleep", "3"], + self.assertNoOutput, + self.assertNoOutput) + + with self.assertElapsed(2): + retcode = p.wait(2) + + self.assertEqual(retcode, None) + + retcode = p.wait() + self.assertEqual(retcode, 0) + + def test_terminate(self): + p = utils.CommandStream(["sleep", "2"], + self.assertNoOutput, + self.assertNoOutput) + + p.terminate() + + retcode = p.wait() + self.assertEqual(retcode, -signal.SIGTERM) + + def test_kill(self): + p = utils.CommandStream(["sleep", "2"], + self.assertNoOutput, + self.assertNoOutput) + + p.kill() + + retcode = p.wait() + self.assertEqual(retcode, -signal.SIGKILL) + + def test_early_close(self): + p = utils.CommandStream(["bash", "-c", + "exec 1>&-; exec 2>&-; exec sleep 2"], + self.assertNoOutput, + self.assertNoOutput) + + with self.assertElapsed(2): + retcode = p.wait() + + self.assertEqual(retcode, 0) -- To view, visit http://gerrit.ovirt.org/33909 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ie015368bb9c5992e5c73a149277c59fc4ffbd570 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

9 years, 1 month

7
46
0 / 0

Change in vdsm[master]: block: use the contextmanager for mapMetaOffset

by Federico Simoncelli

Federico Simoncelli has uploaded a new change for review. Change subject: block: use the contextmanager for mapMetaOffset ...................................................................... block: use the contextmanager for mapMetaOffset Change-Id: Id1496238ea4f9a1902f816b20959ce3379e87871 Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com> --- M vdsm/storage/blockSD.py M vdsm/storage/blockVolume.py M vdsm/storage/imageRepository/formatConverter.py 3 files changed, 13 insertions(+), 10 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/88/25988/1 diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py index cac59fa..02c7dfd 100644 --- a/vdsm/storage/blockSD.py +++ b/vdsm/storage/blockSD.py @@ -29,6 +29,7 @@ import time import functools from collections import namedtuple +from contextlib import contextmanager from operator import itemgetter from vdsm.config import config @@ -732,11 +733,15 @@ newsize = self.metaSize(self.sdUUID) lvm.extendLV(self.sdUUID, sd.METADATA, newsize) + _mdSlotLock = threading.Lock() + + @contextmanager def mapMetaOffset(self, vol_name, slotSize): - if self.getVersion() in VERS_METADATA_LV: - return self.getVolumeMetadataOffsetFromPvMapping(vol_name) - else: - return self.getFreeMetadataSlot(slotSize) + with self._mdSlotLock: + if self.getVersion() in VERS_METADATA_LV: + yield self.getVolumeMetadataOffsetFromPvMapping(vol_name) + else: + yield self.getFreeMetadataSlot(slotSize) def _getOccupiedMetadataSlots(self): stripPrefix = lambda s, pfx: s[len(pfx):] diff --git a/vdsm/storage/blockVolume.py b/vdsm/storage/blockVolume.py index 36bfa1f..9a7fa81 100644 --- a/vdsm/storage/blockVolume.py +++ b/vdsm/storage/blockVolume.py @@ -73,7 +73,6 @@ class BlockVolume(volume.Volume): """ Actually represents a single volume (i.e. part of virtual disk). """ - _tagCreateLock = threading.Lock() def __init__(self, repoPath, sdUUID, imgUUID, volUUID): self.metaoff = None @@ -169,8 +168,7 @@ imgUUID, volUUID, srcImgUUID, srcVolUUID) volParent.clone(imgPath, volUUID, volFormat, preallocate) - with cls._tagCreateLock: - mdSlot = dom.mapMetaOffset(volUUID, VOLUME_MDNUMBLKS) + with dom.mapMetaOffset(volUUID, VOLUME_MDNUMBLKS) as mdSlot: mdTags = ["%s%s" % (TAG_PREFIX_MD, mdSlot), "%s%s" % (TAG_PREFIX_PARENT, srcVolUUID), "%s%s" % (TAG_PREFIX_IMAGE, imgUUID)] diff --git a/vdsm/storage/imageRepository/formatConverter.py b/vdsm/storage/imageRepository/formatConverter.py index c701163..11a0039 100644 --- a/vdsm/storage/imageRepository/formatConverter.py +++ b/vdsm/storage/imageRepository/formatConverter.py @@ -180,9 +180,9 @@ metaOffset, vol.volUUID) metaContent = vol.getMetadata() - with vol._tagCreateLock: - newMetaOffset = domain.mapMetaOffset( - vol.volUUID, blockVolume.VOLUME_MDNUMBLKS) + with domain.mapMetaOffset( + vol.volUUID, blockVolume.VOLUME_MDNUMBLKS) \ + as newMetaOffset: if newMetaOffset > metaMaxSlot: raise se.NoSpaceLeftOnDomain(domain.sdUUID) -- To view, visit http://gerrit.ovirt.org/25988 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id1496238ea4f9a1902f816b20959ce3379e87871 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

9 years, 1 month

7
61
0 / 0

Change in vdsm[master]: debug: Integrate manhole debugging service

by Nir Soffer

Nir Soffer has uploaded a new change for review. Change subject: debug: Integrate manhole debugging service ...................................................................... debug: Integrate manhole debugging service Manhole is in-process service that will accept unix domain socket connections and present the stacktraces for all threads and an interactive prompt. Usage: 0. Install it pip install manhole 1. Enable it in vdsm.conf: [vars] manhole_enable = true 2. Dive into vdsm: $ nc -U /tmp/manhole-1234 Python 2.7.5 (default, Jun 26 2014, 11:55:39) [GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> Change-Id: I8a6cdf97ddce446bea527e771eb641f969b1a532 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M lib/vdsm/config.py.in M vdsm/vdsm 2 files changed, 7 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/47/32147/1 diff --git a/lib/vdsm/config.py.in b/lib/vdsm/config.py.in index 69185e0..d085890 100644 --- a/lib/vdsm/config.py.in +++ b/lib/vdsm/config.py.in @@ -42,6 +42,9 @@ ('profile_clock', 'cpu', 'Sets the underlying clock type (cpu, wall)'), + ('manhole_enable', 'false', + 'Enable manhole debugging service (requires manhole package).'), + ('host_mem_reserve', '256', 'Reserves memory for the host to prevent VMs from using all the ' 'physical pages. The values are in Mbytes.'), diff --git a/vdsm/vdsm b/vdsm/vdsm index e575295..9fdf72f 100755 --- a/vdsm/vdsm +++ b/vdsm/vdsm @@ -70,6 +70,10 @@ profile.start() + if config.getboolean('vars', 'manhole_enable'): + import manhole + manhole.install() + libvirtconnection.start_event_loop() if config.getboolean('irs', 'irs_enable'): -- To view, visit http://gerrit.ovirt.org/32147 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8a6cdf97ddce446bea527e771eb641f969b1a532 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

9 years, 1 month

7
36
0 / 0

Change in vdsm[master]: tool: Remove leftover libvirtd.log rotating logic.

by mtayer＠redhat.com

mooli tayer has uploaded a new change for review. Change subject: tool: Remove leftover libvirtd.log rotating logic. ...................................................................... tool: Remove leftover libvirtd.log rotating logic. Vdsm libvirt logging definitions were recently dropped in favor of libvirt defaults[1]. It is now up to libvirt to manage logging, rotating included. [1] http://gerrit.ovirt.org/#/q/I251c2534d3284aa95bffa56143c449d5de887008,n,z Change-Id: I08e005c2fdd874941abd5dea7d30b3955cadde20 Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1149757 Signed-off-by: Mooli Tayer <mtayer(a)redhat.com> --- M debian/vdsm.prerm M lib/vdsm/tool/Makefile.am M lib/vdsm/tool/configurators/libvirt.py D lib/vdsm/tool/libvirtd.logrotate M tests/Makefile.am M tests/toolTests.py D tests/toolTests_libvirt_logrotate.conf M vdsm.spec.in M vdsm/Makefile.am D vdsm/vdsm-libvirt-logrotate 10 files changed, 0 insertions(+), 103 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/25/33925/1 diff --git a/debian/vdsm.prerm b/debian/vdsm.prerm index 596c5be..dfc54a6 100644 --- a/debian/vdsm.prerm +++ b/debian/vdsm.prerm @@ -31,12 +31,9 @@ /etc/default/libvirt-bin \ /etc/libvirt/libvirtd.conf \ /etc/libvirt/qemu.conf \ - /etc/logrotate.d/libvirtd \ /etc/sysctl.conf \ /etc/default/sanlock - # Restoring backuped logrotate.d/libvirtd - sed -i -e 's/# VDSM backup//' /etc/logrotate.d/libvirtd vdsm-tool sebool-unconfig || : saslpasswd2 -p -a libvirt -d vdsm@ovirt ;; diff --git a/lib/vdsm/tool/Makefile.am b/lib/vdsm/tool/Makefile.am index a181617..94059ce 100644 --- a/lib/vdsm/tool/Makefile.am +++ b/lib/vdsm/tool/Makefile.am @@ -21,12 +21,6 @@ include $(top_srcdir)/build-aux/Makefile.subs -toolfilesdir=$(pkgdatadir)/tool - -dist_toolfiles_DATA= \ - libvirtd.logrotate \ - $(NULL) - EXTRA_DIST = \ load_needed_modules.py.in \ validate_ovirt_certs.py.in \ diff --git a/lib/vdsm/tool/configurators/libvirt.py b/lib/vdsm/tool/configurators/libvirt.py index 5d15de2..63e5ff6 100644 --- a/lib/vdsm/tool/configurators/libvirt.py +++ b/lib/vdsm/tool/configurators/libvirt.py @@ -257,17 +257,6 @@ for key, val in configuration.items(): conff.addEntry(key, val) - def _prefixAndPrepend(self, content, vdsmConfiguration): - """ - Prefix each line with a comment and prepend a section - from file path defined by 'content["prependFile"]' - """ - with self._openConfig(content['path']) as conf: - conf.prefixLines() - - with open(self._getFile(content['prependFile'])) as src_conf: - conf.prependSection(src_conf.read()) - def _removeFile(self, content, vdsmConfiguration): """ delete a file if it exists. @@ -280,15 +269,6 @@ except OSError as e: if e.errno != errno.ENOENT: raise - - def _unprefixAndRemoveSection(self, path): - """ - undo changes done by _prefixAndPrepend. - """ - if os.path.exists(path): - with self._openConfig(path) as conff: - conff.removeConf() - conff.unprefixLines() def _removeSection(self, path): """ @@ -475,29 +455,6 @@ } ] }, - - 'LRCONF': { - 'path': os.path.join( - constants.SYSCONF_PATH, - 'logrotate.d/libvirtd', - ), - 'configure': _prefixAndPrepend, - 'prependFile': 'LRCONF_EXAMPLE', - 'removeConf': _unprefixAndRemoveSection, - 'persisted': True, - }, - - 'LRCONF_EXAMPLE': { - 'path': os.path.join( - constants.P_VDSM, - 'tool', - 'libvirtd.logrotate', - ), - 'configure': lambda x, y, z: True, - 'removeConf': lambda x, y: True, - 'persisted': False, - }, - 'QNETWORK': { 'path': os.path.join( diff --git a/lib/vdsm/tool/libvirtd.logrotate b/lib/vdsm/tool/libvirtd.logrotate deleted file mode 100644 index 8d9dfd8..0000000 --- a/lib/vdsm/tool/libvirtd.logrotate +++ /dev/null @@ -1,10 +0,0 @@ -/var/log/libvirt/libvirtd.log { - rotate 100 - missingok - copytruncate - size 15M - compress - compresscmd /usr/bin/xz - uncompresscmd /usr/bin/unxz - compressext .xz -} diff --git a/tests/Makefile.am b/tests/Makefile.am index 449d7b1..39299fb 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -128,7 +128,6 @@ tc_filter_show.out \ toolTests_empty.conf \ toolTests_lconf_ssl.conf \ - toolTests_libvirt_logrotate.conf \ toolTests_libvirtd.conf \ toolTests_qemu_sanlock.conf \ toolTests_qemu_ssl.conf \ diff --git a/tests/toolTests.py b/tests/toolTests.py index 67d9f6b..998df4b 100644 --- a/tests/toolTests.py +++ b/tests/toolTests.py @@ -256,12 +256,7 @@ self.test_env['QCONF'] = self._test_dir + '/qemu.conf' self.test_env['LDCONF'] = self._test_dir + '/qemu-sanlock.conf' self.test_env['QLCONF'] = self._test_dir + '/libvirtd' - self.test_env['LRCONF'] = self._test_dir + '/logrotate-libvirtd' self.test_env['QNETWORK'] = 'NON_EXISTENT' - self.test_env['LRCONF_EXAMPLE'] = os.path.join( - LibvirtModuleConfigureTests.srcPath, - 'lib/vdsm/tool/libvirtd.logrotate' - ) for key, val in self.test_env.items(): libvirt.Configurator.FILES[key]['path'] = val @@ -339,7 +334,6 @@ self._setConfig( ('LCONF', 'lconf_ssl'), ('QCONF', 'qemu_ssl'), - ('LRCONF', 'libvirt_logrotate') ) self.assertEqual( @@ -353,7 +347,6 @@ self._setConfig( ('LCONF', 'lconf_ssl'), ('QCONF', 'empty'), - ('LRCONF', 'empty'), ) self.assertEqual( libvirtConfigure.isconfigured(), @@ -367,7 +360,6 @@ 'LCONF', 'empty'), ('VDSM_CONF', 'vdsm_ssl'), ('QCONF', 'empty'), - ('LRCONF', 'empty'), ) self.assertEqual( @@ -388,7 +380,6 @@ ('LCONF', 'empty'), ('VDSM_CONF', 'vdsm_no_ssl'), ('QCONF', 'empty'), - ('LRCONF', 'empty'), ) self.assertEquals( libvirtConfigure.isconfigured(), diff --git a/tests/toolTests_libvirt_logrotate.conf b/tests/toolTests_libvirt_logrotate.conf deleted file mode 100644 index 7befe2d..0000000 --- a/tests/toolTests_libvirt_logrotate.conf +++ /dev/null @@ -1,21 +0,0 @@ -## beginning of configuration section by vdsm-%(LATEST_CONF_VERSION)s -/var/log/libvirt/libvirtd.log { - rotate 100 - missingok - copytruncate - size 15M - compress - compresscmd /usr/bin/xz - uncompresscmd /usr/bin/unxz - compressext .xz -} -## end of configuration section by vdsm-%(LATEST_CONF_VERSION)s -# VDSM backup/var/log/libvirt/libvirtd.log { -# VDSM backup weekly -# VDSM backup missingok -# VDSM backup rotate 4 -# VDSM backup compress -# VDSM backup delaycompress -# VDSM backup copytruncate -# VDSM backup minsize 100k -# VDSM backup} diff --git a/vdsm.spec.in b/vdsm.spec.in index c38d5bb..14d27e2 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -1014,7 +1014,6 @@ %{_datadir}/%{vdsm_name}/virt/vmxml.py* %{_datadir}/%{vdsm_name}/virt/sampling.py* %{_datadir}/%{vdsm_name}/virt/utils.py* -%{_datadir}/%{vdsm_name}/tool %config(noreplace) %{_sysconfdir}/%{vdsm_name}/vdsm.conf %config(noreplace) %{_sysconfdir}/%{vdsm_name}/logger.conf @@ -1033,7 +1032,6 @@ %{_sysconfdir}/dhcp/dhclient.d/sourceRoute.sh %{_sysconfdir}/sudoers.d/50_vdsm %{_sysconfdir}/cron.hourly/vdsm-logrotate -%{_sysconfdir}/cron.d/vdsm-libvirt-logrotate %if 0%{?fedora} || 0%{?rhel} >= 7 %{_sysconfdir}/libvirt/hooks/qemu %endif @@ -1299,7 +1297,6 @@ %{_datadir}/%{vdsm_name}/tests/tc_filter_show.out %{_datadir}/%{vdsm_name}/tests/toolTests_empty.conf %{_datadir}/%{vdsm_name}/tests/toolTests_lconf_ssl.conf -%{_datadir}/%{vdsm_name}/tests/toolTests_libvirt_logrotate.conf %{_datadir}/%{vdsm_name}/tests/toolTests_libvirtd.conf %{_datadir}/%{vdsm_name}/tests/toolTests_qemu_sanlock.conf %{_datadir}/%{vdsm_name}/tests/toolTests_qemu_ssl.conf diff --git a/vdsm/Makefile.am b/vdsm/Makefile.am index d40c7d0..f59b38d 100644 --- a/vdsm/Makefile.am +++ b/vdsm/Makefile.am @@ -110,7 +110,6 @@ vdsm-gencerts.sh.in \ vdsm-libvirt-access.pkla \ vdsm-libvirt-access.rules \ - vdsm-libvirt-logrotate \ vdsm-logrotate \ vdsm-logrotate.conf.in \ vdsm-modules-load.d.conf \ @@ -210,14 +209,10 @@ $(MKDIR_P) $(DESTDIR)$(sysconfdir)/cron.hourly $(INSTALL_SCRIPT) $(srcdir)/vdsm-logrotate \ $(DESTDIR)$(sysconfdir)/cron.hourly/vdsm-logrotate - $(MKDIR_P) $(DESTDIR)$(sysconfdir)/cron.d - $(INSTALL_DATA) $(srcdir)/vdsm-libvirt-logrotate \ - $(DESTDIR)$(sysconfdir)/cron.d/vdsm-libvirt-logrotate uninstall-data-logrotate: $(RM) $(DESTDIR)$(vdsmconfdir)/logrotate/vdsm $(RM) $(DESTDIR)$(sysconfdir)/cron.hourly/vdsm-logrotate - $(RM) $(DESTDIR)$(sysconfdir)/cron.d/vdsm-libvirt-logrotate install-data-mom: $(MKDIR_P) $(DESTDIR)$(vdsmconfdir)/mom.d diff --git a/vdsm/vdsm-libvirt-logrotate b/vdsm/vdsm-libvirt-logrotate deleted file mode 100644 index 5e70cde..0000000 --- a/vdsm/vdsm-libvirt-logrotate +++ /dev/null @@ -1,2 +0,0 @@ -10,25,40,55 * * * * root /usr/sbin/logrotate /etc/logrotate.d/libvirtd - -- To view, visit http://gerrit.ovirt.org/33925 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I08e005c2fdd874941abd5dea7d30b3955cadde20 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: mooli tayer <mtayer(a)redhat.com>

9 years, 2 months

5
47
0 / 0

Change in vdsm[master]: vm: leave VM responsive when sampling ends

by fromani＠redhat.com

Francesco Romani has uploaded a new change for review. Change subject: vm: leave VM responsive when sampling ends ...................................................................... vm: leave VM responsive when sampling ends There is a known and (usually) benign race between the sampling thread and the libvirt event handler. In extremely unlucky and unlikely cases, it may happen that the Vm is leaved marked as unresponsive by this race. This has been observed while reproducing bz#1142776. This patch adds more protection to this case by marking the Vm as responsive when sampling thread ends. Change-Id: I2ee3205da94152bfd39197dae6c06d00c06e1d58 Related-To: https://bugzilla.redhat.com/1142776 Signed-off-by: Francesco Romani <fromani(a)redhat.com> --- M vdsm/virt/sampling.py M vdsm/virt/vm.py 2 files changed, 11 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/02/35902/1 diff --git a/vdsm/virt/sampling.py b/vdsm/virt/sampling.py index eac4873..e6f0204 100644 --- a/vdsm/virt/sampling.py +++ b/vdsm/virt/sampling.py @@ -461,6 +461,7 @@ self._log.debug("Stats thread failed", exc_info=True) self._log.debug("Stats thread finished") + self._onShutdown(self) def handleStatsException(self, ex): """ @@ -497,6 +498,12 @@ self._stopEvent.wait(waitInterval) intervalAccum = (intervalAccum + waitInterval) % maxInterval + def _onShutdown(self): + """ + cleanup handler invoked when the stats thread is about to end + """ + pass + class HostStatsThread(threading.Thread): """ diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py index dd23db4..7c6bd40 100644 --- a/vdsm/virt/vm.py +++ b/vdsm/virt/vm.py @@ -590,6 +590,10 @@ return True + def _onShutdown(self): + # ensure the VM is not stuck on unresponsive state + self._vm._monitorResponse = 0 + def _calcDiskRate(vmDrive, sInfo, eInfo, sampleInterval): return { -- To view, visit http://gerrit.ovirt.org/35902 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I2ee3205da94152bfd39197dae6c06d00c06e1d58 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani <fromani(a)redhat.com>

9 years, 2 months

4
8
0 / 0

Change in vdsm[master]: stomp: Pop the frame even if a frame handler isn't defined

by smizrahi＠redhat.com

Saggi Mizrahi has uploaded a new change for review. Change subject: stomp: Pop the frame even if a frame handler isn't defined ...................................................................... stomp: Pop the frame even if a frame handler isn't defined Change-Id: I9cd3c5b9ad70cf49e1b105260548720d5b34ea9a Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M lib/yajsonrpc/stomp.py 1 file changed, 4 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/40/35940/1 diff --git a/lib/yajsonrpc/stomp.py b/lib/yajsonrpc/stomp.py index 279aa24..142b22a 100644 --- a/lib/yajsonrpc/stomp.py +++ b/lib/yajsonrpc/stomp.py @@ -445,9 +445,10 @@ parser.parse(data) frameHandler = self._frameHandler - if hasattr(frameHandler, "handle_frame"): - while parser.pending > 0: - frameHandler.handle_frame(self, parser.popFrame()) + while parser.pending > 0: + frame = parser.popFrame() + if hasattr(frameHandler, "handle_frame"): + frameHandler.handle_frame(self, frame) def popFrame(self): return self._parser.popFrame() -- To view, visit http://gerrit.ovirt.org/35940 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I9cd3c5b9ad70cf49e1b105260548720d5b34ea9a Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

9 years, 2 months

7
73
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

vdsm-patches December 2014