May 2012 - vdsm-devel - Fedora Mailing-Lists

by abaron＠redhat.com

Hi all, I would like to discuss the following on today's call: 1. Gerrit vs. mailing list 2. mandatory unittest per patch 3. pep8 4. ?? If you have anything else you'd like to discuss, please reply to this email. Regards, Ayal.

11 years, 8 months

7
26
0 / 0

RFC: Writeup on VDSM-libstoragemgmt integration

by deepakcs＠linux.vnet.ibm.com

Hello All, I have a draft write-up on the VDSM-libstoragemgmt integration. I wanted to run this thru' the mailing list(s) to help tune and crystallize it, before putting it on the ovirt wiki. I have run this once thru Ayal and Tony, so have some of their comments incorporated. I still have few doubts/questions, which I have posted below with lines ending with '?' Comments / Suggestions are welcome & appreciated. thanx, deepak [Ccing engine-devel and libstoragemgmt lists as this stuff is relevant to them too] -------------------------------------------------------------------------------------------------------------- 1) Background: VDSM provides high level API for node virtualization management. It acts in response to the requests sent by oVirt Engine, which uses VDSM to do all node virtualization related tasks, including but not limited to storage management. libstoragemgmt aims to provide vendor agnostic API for managing external storage array. It should help system administrators utilizing open source solutions have a way to programmatically manage their storage hardware in a vendor neutral way. It also aims to facilitate management automation, ease of use and take advantage of storage vendor supported features which improve storage performance and space utilization. Home Page: http://sourceforge.net/apps/trac/libstoragemgmt/ libstoragemgmt (LSM) today supports C and python plugins for talking to external storage array using SMI-S as well as native interfaces (eg: netapp plugin ) Plan is to grow the SMI-S interface as needed over time and add more vendor specific plugins for exploiting features not possible via SMI-S or have better alternatives than using SMI-S. For eg: Many of the copy offload features require to use vendor specific commands, which justifies the need for a vendor specific plugin. 2) Goals: 2a) Ability to plugin external storage array into oVirt/VDSM virtualization stack, in a vendor neutral way. 2b) Ability to list features/capabilities and other statistical info of the array 2c) Ability to utilize the storage array offload capabilities from oVirt/VDSM. 3) Details: LSM will sit as a new repository engine in VDSM. VDSM Repository Engine WIP @ http://gerrit.ovirt.org/#change,192 Current plan is to have LSM co-exist with VDSM on the virtualization nodes. *Note : 'storage' used below is generic. It can be a file/nfs-export for NAS targets and LUN/logical-drive for SAN targets. VDSM can use LSM and do the following... - Provision storage - Consume storage 3.1) Provisioning Storage using LSM Typically this will be done by a Storage administrator. oVirt/VDSM should provide storage admin the - ability to list the different storage arrays along with their types (NAS/SAN), capabilities, free/used space. - ability to provision storage using any of the array capabilities (eg: thin provisioned lun or new NFS export ) - ability to manage the provisioned storage (eg: resize/delete storage) Once the storage is provisioned by the storage admin, VDSM will have to refresh the host(s) for them to be able to see the newly provisioned storage. 3.1.1) Potential flows: Mgmt -> vdsm -> lsm: create LUN + LUN Mapping / Zoning / whatever is needed to make LUN available to list of hosts passed by mgmt Mgmt -> vdsm: getDeviceList (refreshes host and gets list of devices) Repeat above for all relevant hosts (depending on list passed earlier, mostly relevant when extending an existing VG) Mgmt -> use LUN in normal flows. 3.1.2) How oVirt Engine will know which LSM to use ? Normally the way this works today is that user can choose the host to use (default today is SPM), however there are a few flows where mgmt will know which host to use: 1. extend storage domain (add LUN to existing VG) - Use SPM and make sure *all* hosts that need access to this SD can see the new LUN 2. attach new LUN to a VM which is pinned to a specific host - use this host 3. attach new LUN to a VM which is not pinned - use a host from the cluster the VM belongs to and make sure all nodes in cluster can see the new LUN Flows for which there is no clear candidate (Maybe we can use the SPM host itself which is the default ?) 1. create a new disk without attaching it to any VM 2. create a LUN for a new storage domain 3.2) Consuming storage using LSM Typically this will be done by a virtualization administrator oVirt/VDSM should allow virtualization admin to - Create a new storage domain using the storage on the array. - Be able to specify whether VDSM should use the storage offload capability (default) or override it to use its own internal logic. 4) VDSM potential changes: 4.1) How to represent a VM disk, 1 LUN = 1 VMdisk or 1 LV = 1 VMdisk ? which bring another question...1 array == 1 storage domain OR 1 LUN/nfs-export on the array == 1 storage domain ? Pros & Cons of each... 1 array == 1 storage domain - Each new vmdisk (aka volume) will be a new lun/file on the array. - Easier to exploit offload capabilities, as they are available at the LUN/File granularity - Will there be any issues where there will be too many LUNs/Files ... any maxluns limit on linux hosts that we might hit ? -- VDSM has been tested with 1K LUNs and it worked fine - ayal - Storage array limitations on the number of LUNs can be a downside here. - Would it be ok to share the array for hosting another storage domain if need be ? -- Provided the existing domain is not utilising all of the free space -- We can create new LUNs and hand it over to anyone needed ? -- Changes needed in VDSM to work with raw LUNs, today it only has support for consuming LUNs via VG/LV. 1 LUN/nfs-export on the array == 1 storage domain - How to represent a new vmdisk (aka vdsm volume) if its a LUN provisioned using SAN target ? -- Will it be VG/LV as is done today for block domains ? -- If yes, then it will be difficult to exploit offload capabilities, as they are at LUN level, not at LV level. - Each new vmdisk will be a new file on the nfs-export, assuming offload capability is available at the file level, so this should work for NAS targets ? - Can use the storage array for hosting multiple storage domains. -- Provision one more LUN and use it for another storage domain if need be. - VDSM already supports this today, as part of block storage domains for LUNs case. Note that we will allow user to do either one of the two options above, depending on need. 4.2) Storage domain metadata will also include the features/capabilities of the storage array as reported by LSM. - Capabilities (taken via LSM) will be stored in the domain metadata during storage domain create flow. - Need changes in oVirt engine as well ( see 'oVirt Engine potential changes' section below ) 4.3) VDSM to poll LSM for array capabilities on a regular basis ? Per ayal: - If we have a 'storage array' entity in oVirt Engine (see 'oVirt Engine potential changes' section below ) then we can have a 'refresh capabilities' button/verb. - We can periodically query the storage array. - Query LSM before running operations (sounds redundant to me, but if it's cheap enough it could be simplest). Probably need a combination of 1+2 (query at very low frequency - 1/hour or 1/day + refresh button) 5) oVirt Engine potential changes - as described by ayal : - We will either need a new 'storage array' entity in engine to keep credentials, or, in case of storage array as storage domain, just keep this info as part of the domain at engine level. - Have a 'storage array' entity in oVirt Engine to support 'refresh capabilities' as a button/verb. - When user during storage provisioning, selects a LUN exported from a storage array (via LSM), the oVirt Engine would know from then onwards that this LUN is being served via LSM. It would then be able to query the capabilities of the LUN and show it to the virt admin during storage consumption flow. 6) Potential flows: - Create snapshot flow -- VDSM will check the snapshot offload capability in the domain metadata -- If available, and override is not configured, it will use LSM to offload LUN/File snapshot -- If override is configured or capability is not available, it will use its internal logic to create snapshot (qcow2). - Copy/Clone vmdisk flow -- VDSM will check the copy offload capability in the domain metadata -- If available, and override is not configured, it will use LSM to offload LUN/File copy -- If override is configured or capability is not available, it will use its internal logic to create snapshot (eg: dd cmd in case of LUN). 7) LSM potential changes: - list features/capabilities of the array. Eg: copy offload, thin prov. etc. - list containers (aka pools) (present in LSM today) - Ability to list different types of arrays being managed, their capabilities and used/free space - Ability to create/list/delete/resize volumes ( LUN or exports, available in LSM as of today) - Get monitoring info with object (LUN/snapshot/volume) as optional parameter for specific info. eg: container/pool free/used space, raid type etc. Need to make sure above info is listed in a coherent way across arrays (number of LUNs, raid type used? free/total per container/pool, per LUN?. Also need I/O statistics wherever possible.

11 years, 10 months

10
24
0 / 0

Re: [vdsm] [Users] How to configure sharedFS ?

by deepakcs＠linux.vnet.ibm.com

On 02/29/2012 08:06 PM, Dan Kenigsberg wrote: > On Wed, Feb 29, 2012 at 07:42:15PM +0530, Deepak C Shetty wrote: >> On 02/27/2012 04:55 AM, Ayal Baron wrote: >>> Any help on documenting this so people would not have to dig into the code would be greatly appreciated. >>> >>>> Great, let me start and get back if i land up into issues. >>>> thanks for the pointer. >>> Let us know if you need further assistance.. >> > If you have that python script working for LOCALFS, I'd suggest you try > making it work for SHAREDFS too. > > I was able to use the VDSM_Standalone example, and modify it to use SHAREDFS and export glusterfs as a DATA_DOMAIN and invoke VM backed by gluster storage. I edited http://www.ovirt.org/wiki/Vdsm_Standalone (scroll below) & added it as a SHAREDFS example.

11 years, 10 months

2
3
0 / 0

setgroup/setuid affects libvirt event loop results in connection broken

by Royce Lv

Guys, In debugging vdsm/supervdsm process I noticed that event loop in libvirtev.py will be affected by setgroups/setgid/setuid. Results in libvirtd close connection because the keepalive scheme fals.Follow up vdsm to libvirt calls fails because of broken pipe.(detail: https://bugzilla.redhat.com/show_bug.cgi?id=825634) I'm afraid this will make vdsm/libvirt connection vulnerable in some of the current use case,too. So I have plan: 1.we use default impl as https://bugzilla.redhat.com/show_bug.cgi?id=681885 mentioned 2.use the setKeepAlive api in libvirt (keepalive problem seems disappear with this) 3.start libvirt event loop in first time we get libvirt connection, not when the file first loaded . I'd appreciate ideas about this on how to address this.Thanks a lot!

11 years, 11 months

3
4
0 / 0

Re: [vdsm] [Users] glusterfs and ovirt

by deepakcs＠linux.vnet.ibm.com

On 05/17/2012 11:05 PM, Itamar Heim wrote: > On 05/17/2012 06:55 PM, Bharata B Rao wrote: >> On Wed, May 16, 2012 at 3:29 PM, Itamar Heim<iheim(a)redhat.com> wrote: >>> On 05/15/2012 07:35 PM, Andrei Vakhnin wrote: >>>> >>>> Yair >>>> >>>> Thanks for an update. Can I have KVM hypervisors also function as >>>> storage >>>> nodes for glusterfs? What is a release date for glusterfs support? >>>> We're >>>> looking for a production deployment in June. Thanks >>> >>> >>> current status is >>> 1. patches for provisioning gluster clusters and volumes via ovirt >>> are in >>> review, trying to cover this feature set [1]. >>> I'm not sure if all of them will make the ovirt 3.1 version which is >>> slated >>> to branch for stabilization June 1st, but i think "enough" is there. >>> so i'd start trying current upstream version to help find issues >>> blocking >>> you, and following on them during june as we stabilize ovirt 3.1 for >>> release >>> (planned for end of june). >>> >>> 2. you should be able to use same hosts for both gluster and virt, >>> but there >>> is no special logic/handling for this yet (i.e., trying and providing >>> feedback would help improve this mode). >>> I would suggest start from separate clusters though first, and only >>> later >>> trying the joint mode. >>> >>> 3. creating a storage domain on top of gluster: >>> - expose NFS on top of it, and consume as a normal nfs storage domain >>> - use posixfs storage domain with gluster mount semantics >>> - future: probably native gluster storage domain, up to native >>> integration with qemu >> >> I am looking at GlusterFS integration with QEMU which involves adding >> GlusterFS as block backend in QEMU. This will involve QEMU talking to >> gluster directly via libglusterfs bypassing FUSE. I could specify a >> volume file and the VM image directly on QEMU command line to boot >> from the VM image that resides on a gluster volume. >> >> Eg: qemu -drive file=client.vol:/Fedora.img,format=gluster >> >> In this example, Fedora.img is being served by gluster and client.vol >> would have client-side translators specified. >> >> I am not sure if this use case would be served if GlusterFS is >> integrated as posixfs storage domain in VDSM. Posixfs would involve >> normal FUSE mount and QEMU would be required to work with images from >> FUSE mount path ? >> >> With QEMU supporting GlusterFS backend natively, further optimizations >> are possible in case of gluster volume being local to the host node. >> In this case, one could provide QEMU with a simple volume file that >> would not contain client or server xlators, but instead just the posix >> xlator. This would lead to most optimal IO path that bypasses RPC >> calls. >> >> So do you think, this use case (QEMU supporting GlusterFS backend >> natively and using volume file to specify the needed translators) >> warrants a specialized storage domain type for GlusterFS in VDSM ? > > I'm not sure if a special storage domain, or a PosixFS based domain > with enhanced capabilities. > Ayal? Related Question: With QEMU using GlusterFS backend natively (as described above), it also means that it needs addnl options/parameters as part of qemu command line (as given above). How does VDSM today support generating a custom qemu cmdline. I know VDSM talks to libvirt, so is there a framework in VDSM to edit/modify the domxml based on some pre-conditions, and how / where one should hook up to do that modification ? I know of libvirt hooks framework in VDSM, but that was more for temporary/experimental needs, or am i completely wrong here ? Irrespective of whether GlusterFS integrates into VDSM as PosixFS or special storage domain it won't address the need to generate a custom qemu cmdline if a file/image was served by GlusterFS. Whats the way to address this issue in VDSM ? I am assuming here that special storage domain (aka repo engine) is only to manage image repository, and image related operations, won't help in modifying qemu cmd line being generated. [Ccing vdsm-devel also] thanx, deepak

11 years, 11 months

4
4
0 / 0

VDSM API/clientIF instance design issue

by wudxw＠linux.vnet.ibm.com

Hi Guys, Recently, I has been working on integrate MOM into VDSM. MOM needs to use VDSM API to interact with it. But currently, it requires the instance of clientIF to use vdsm API. Passing clientIF to MOM is not a good choice since it's a vdsm internal object. So I try to remove the parameter 'cif' from the interface definition and change to access the globally unique clientIF instance in API.py. To get the instance of clientIF, I add a decorator to clientIF to change it into singleton. Actually, clientIF has been working as a global single instance already. We just don't have an interface to get it and so passing it as parameter instead. I think using singleton to get the instance of clientIF is more clean. Dan and Saggi already gave some comments in http://gerrit.ovirt.org/#change,4839 Thanks for the reviewing! But I think we need more discussion on it, so I post it here because gerrit is not the appropriate to discuss a design issue. Thanks ! Mark.

11 years, 11 months

5
11
0 / 0

configure / autogen error on F16

by deepakcs＠linux.vnet.ibm.com

Hello, I have a lab machine that does not have internet access. So when i take the latest vdsm git src, tar it and put it on the lab machine and try to run ./configure, i get this... ./configure configure: error: package version not defined Note that all the dep packages etc have been resolved, and i first got this error when i ran 'make'. Somehow the PACKAGE_VERSION in ./configure is getting cleared off. I also see this... ./autogen.sh --prefix=/ aclocal.m4:16: warning: this file was generated for autoconf 2.67. You have another version of autoconf. It may work, but is not guaranteed to. If you have problems, you may need to regenerate the build system entirely. To do so, use the procedure documented by the package, typically `autoreconf'. aclocal.m4:16: warning: this file was generated for autoconf 2.67. You have another version of autoconf. It may work, but is not guaranteed to. If you have problems, you may need to regenerate the build system entirely. To do so, use the procedure documented by the package, typically `autoreconf'. Runnign autoreconf did not help. On another working system i compared the autoconf and other auto* packages with this system, and they are same, so not sure why I am seeing this issue on this particular system only. thanx, deepak

11 years, 11 months

2
1
0 / 0

Re: [vdsm] Change in vdsm[master]: tests: fix gluster storage exception test

by Ryan Harper

* danken(a)redhat.com <danken(a)redhat.com> [2012-05-29 09:29]: > Dan Kenigsberg has posted comments on this change. > > Change subject: tests: fix gluster storage exception test > ...................................................................... > > > Patch Set 1: Looks good to me, approved > > Thanks Ryan. Though Python has the lovely > > 4100 <= obj.code <= 4800 Thanks, but I'm well aware of the python syntax. I've been requested in the past[1] to be more verbose in the asserts so I figured I'd break it out into to separate asserts. Is there a general guideline we're to follow when it comes to the test cases? I'd be happy to follow. 1. http://gerrit.ovirt.org/#patch,unified,4323,3,tests/netinfoTests.py > > syntax. > > -- > To view, visit http://gerrit.ovirt.org/4822 > To unsubscribe, visit http://gerrit.ovirt.org/settings > > Gerrit-MessageType: comment > Gerrit-Change-Id: I94a4ac90cff064def5eeac30cbdab664c88b5253 > Gerrit-PatchSet: 1 > Gerrit-Project: vdsm > Gerrit-Branch: master > Gerrit-Owner: Ryan Harper <ryanh(a)us.ibm.com> > Gerrit-Reviewer: Bala.FA <barumuga(a)redhat.com> > Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> > Gerrit-Reviewer: Timothy Asir <tjeyasin(a)redhat.com> -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx ryanh(a)us.ibm.com

11 years, 11 months

2
3
0 / 0

About the work of "New repository system"

by shuming＠linux.vnet.ibm.com

Hi, I am noticing that Saggi post a set of patches to gerrit named "[WIP] New repository system". The latest update was post in Jan 9 and the latest update of depending patch was post in Feb 19. It seems that the patches are pending on "Refactor upgrade process to facilitate repository format conversions". I am wondering if there is any update about these patches and if there is any new patch based on the latest VDSM code. Recently, I am trying to test these code on the latest VDSM upstream. -- Shu Ming<shuming(a)linux.vnet.ibm.com> IBM China Systems and Technology Laboratory

11 years, 11 months

2
2
0 / 0

Move some of code from spec file into vdsm-tool function issue

by lilei＠linux.vnet.ibm.com

Hi guys, Adam point out a problem about my patch moving some of the post and preun section in vdsm spec file into vdsm-tool, and I have the same concern. After some discussion, I'd like to ask for your suggestion on the patch as link below. http://gerrit.ovirt.org/#patch,sidebyside,4528,3,vdsm.spec.in Please let me know your idea, thanks! -- Lei

11 years, 11 months

4
5
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

vdsm-devel May 2012