Currently most linux distributions that use dbus store a UUID in /var/lib/dbus/machine_id. In our pacemaker-cloud work test tools, we must manipulate this file via oz image creation to match a value we know about. Q1. Is this file freshly created on each image creation/cloning process? If not, it should be, because Matahari uses this information to uniquely identify a host. If it is copied exactly to each new image, that creates a problem (all hosts appear the same to matahari). Q2. If/when it is created by image factory, will it be stored in a database or other storage medium? In pacemaker-cloud we need to have a mapping from image->internal id id so that we know which VM maps to which deployable HA configuration. If we wait on this point until after our 1.0 release, we could end up with a bunch of images in the field that have either the same machine id or are not mapped in any way that allows us to provide HA functionality. Regards -steve

_______________________________________________ aeolus-devel mailing list aeolus-devel(a)lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/aeolus-devel

On 06/27/2011 10:59 PM, Steven Dake wrote: > Currently the Audrey script runs as a replacement to rc.local. Matahari > runs at S99. The general idea is for Audrey script to run as S98 > (before Matahari) and write a management-wide unique instance UUID to Relying on start priority levels seems fragile, esp. with systemd which "provides aggressive parallelization" > the file /etc/vm_machine_id (audrey has access to this information). > > Matahari could be changed to read /etc/vm_machine_id first. If that > file doesn't exist, /var/lib/dbus/machine_id would be read. > > This creates some difficulty in running the audrey script at a specific > runlevel (it requires some changes to oz to insert init scripts). To express such dependency you could use LSB initscript header Required-Start: in matahari initscript (which one, matahari-host I guess?) but such dependency in Matahari doesn't seem right. In systemd unit file you could specify Before and After, but that's only for Fedora >= 15. > Another option is for the current rc.local script that audrey replaces > to run the Matahari service starting giblits as its first action. That actually might be the most reliable and portable way.

> Also... I hate bunching up services towards the end of the list (98, 99, > etc). Is there any reason why these aren't slightly lower in the > runlevel numbering so that we aren't almost falling off the end? > I haven't made this decision - this is what is in upstream atm.

On Mon, 2011-06-27 at 13:59 -0700, Steven Dake wrote: > On 06/24/2011 04:16 PM, Steven Dake wrote: >> Currently most linux distributions that use dbus store a UUID in >> /var/lib/dbus/machine_id. In our pacemaker-cloud work test tools, we >> must manipulate this file via oz image creation to match a value we know >> about. >> >> Q1. Is this file freshly created on each image creation/cloning process? >> >> If not, it should be, because Matahari uses this information to uniquely >> identify a host. If it is copied exactly to each new image, that >> creates a problem (all hosts appear the same to matahari). >> >> Q2. If/when it is created by image factory, will it be stored in a >> database or other storage medium? >> >> In pacemaker-cloud we need to have a mapping from image->internal id id >> so that we know which VM maps to which deployable HA configuration. >> >> If we wait on this point until after our 1.0 release, we could end up >> with a bunch of images in the field that have either the same machine id >> or are not mapped in any way that allows us to provide HA functionality. >> >> Regards >> -steve >> > > After more investigation, Chris and I came up with a workable plan for > handling unique VM ids (see thread with subject How Audrey, Conductor, > and Audrey's config server interact and their relationship to a unique > vm instance id). > > Currently the Audrey script runs as a replacement to rc.local. Matahari > runs at S99. The general idea is for Audrey script to run as S98 > (before Matahari) and write a management-wide unique instance UUID to > the file /etc/vm_machine_id (audrey has access to this information). > > Matahari could be changed to read /etc/vm_machine_id first. If that > file doesn't exist, /var/lib/dbus/machine_id would be read. > > This creates some difficulty in running the audrey script at a specific > runlevel (it requires some changes to oz to insert init scripts). > > Another option is for the current rc.local script that audrey replaces > to run the Matahari service starting giblits as its first action. > > Comments welcome before I start writing code... > -steve > I think the solution of having Audrey store a launch time unique UUID in a file that Matahari can read will work. It may not be necessary to alter oz to insert init scripts to ensure Audrey runs before Matahari. Let me explain. Audrey does not replace /etc/rc.local When Image Factory builds the image it appends to the end of /etc/rc.local a line of code that will start Audrey. e.g.: [ -f /usr/bin/audrey ] && /usr/bin/audrey I propose having Image Factory append another line to /etc/rc.local below where it starts Audrey to start Matahari. e.g.: [ -f /usr/bin/audrey ] && /usr/bin/audrey [ -f <Matahari start> ] && <run Matahari start> We may need to manage timing to ensure /usr/bin/audrey does not return until it has stored the unique UUID in a file and have it return an error status if it is unable to. Thoughts?

Joe

On Tue, 2011-06-28 at 07:26 -0700, Steven Dake wrote: > On 06/28/2011 06:24 AM, Joseph VLcek wrote: >> On Mon, 2011-06-27 at 13:59 -0700, Steven Dake wrote: >>> On 06/24/2011 04:16 PM, Steven Dake wrote: >>>> Currently most linux distributions that use dbus store a UUID in >>>> /var/lib/dbus/machine_id. In our pacemaker-cloud work test tools, we >>>> must manipulate this file via oz image creation to match a value we know >>>> about. >>>> >>>> Q1. Is this file freshly created on each image creation/cloning process? >>>> >>>> If not, it should be, because Matahari uses this information to uniquely >>>> identify a host. If it is copied exactly to each new image, that >>>> creates a problem (all hosts appear the same to matahari). >>>> >>>> Q2. If/when it is created by image factory, will it be stored in a >>>> database or other storage medium? >>>> >>>> In pacemaker-cloud we need to have a mapping from image->internal id id >>>> so that we know which VM maps to which deployable HA configuration. >>>> >>>> If we wait on this point until after our 1.0 release, we could end up >>>> with a bunch of images in the field that have either the same machine id >>>> or are not mapped in any way that allows us to provide HA functionality. >>>> >>>> Regards >>>> -steve >>>> >>> >>> After more investigation, Chris and I came up with a workable plan for >>> handling unique VM ids (see thread with subject How Audrey, Conductor, >>> and Audrey's config server interact and their relationship to a unique >>> vm instance id). >>> >>> Currently the Audrey script runs as a replacement to rc.local. Matahari >>> runs at S99. The general idea is for Audrey script to run as S98 >>> (before Matahari) and write a management-wide unique instance UUID to >>> the file /etc/vm_machine_id (audrey has access to this information). >>> >>> Matahari could be changed to read /etc/vm_machine_id first. If that >>> file doesn't exist, /var/lib/dbus/machine_id would be read. >>> >>> This creates some difficulty in running the audrey script at a specific >>> runlevel (it requires some changes to oz to insert init scripts). >>> >>> Another option is for the current rc.local script that audrey replaces >>> to run the Matahari service starting giblits as its first action. >>> >>> Comments welcome before I start writing code... >>> -steve >>> >> >> I think the solution of having Audrey store a launch time unique UUID in >> a file that Matahari can read will work. >> >> It may not be necessary to alter oz to insert init scripts to ensure >> Audrey runs before Matahari. >> >> Let me explain. >> >> Audrey does not replace /etc/rc.local >> >> When Image Factory builds the image it appends to the end >> of /etc/rc.local a line of code that will start Audrey. >> >> e.g.: >> [ -f /usr/bin/audrey ] && /usr/bin/audrey >> >> >> I propose having Image Factory append another line to /etc/rc.local >> below where it starts Audrey to start Matahari. >> >> e.g.: >> >> [ -f /usr/bin/audrey ] && /usr/bin/audrey >> >> [ -f <Matahari start> ] && <run Matahari start> >> >> We may need to manage timing to ensure /usr/bin/audrey does not return >> until it has stored the unique UUID in a file and have it return an >> error status if it is unable to. >> >> Thoughts? >> > > This sounds fine to me, although the rc.local modification always > sounded a bit hacky (rather then using a proper init script). > One minor > issue is matahari expects to be started via service xxx start and each > agent has a separate init script. There are 5 or 6 agents. > > Regards > -steve >> Joe >> >> > Yes. I agree the rc.local modification could be viewed as a bit hacky however I don't think unacceptably so. If others disagree we could explore Chris's suggestion of altering oz to insert init scripts.

On 06/28/2011 10:26 AM, Steven Dake wrote: > On 06/28/2011 06:24 AM, Joseph VLcek wrote: >> On Mon, 2011-06-27 at 13:59 -0700, Steven Dake wrote: >>> On 06/24/2011 04:16 PM, Steven Dake wrote: >>>> Currently most linux distributions that use dbus store a UUID in >>>> /var/lib/dbus/machine_id. In our pacemaker-cloud work test tools, we >>>> must manipulate this file via oz image creation to match a value we know >>>> about. >>>> >>>> Q1. Is this file freshly created on each image creation/cloning process? >>>> >>>> If not, it should be, because Matahari uses this information to uniquely >>>> identify a host. If it is copied exactly to each new image, that >>>> creates a problem (all hosts appear the same to matahari). >>>> >>>> Q2. If/when it is created by image factory, will it be stored in a >>>> database or other storage medium? >>>> >>>> In pacemaker-cloud we need to have a mapping from image->internal id id >>>> so that we know which VM maps to which deployable HA configuration. >>>> >>>> If we wait on this point until after our 1.0 release, we could end up >>>> with a bunch of images in the field that have either the same machine id >>>> or are not mapped in any way that allows us to provide HA functionality. >>>> >>>> Regards >>>> -steve >>>> >>> >>> After more investigation, Chris and I came up with a workable plan for >>> handling unique VM ids (see thread with subject How Audrey, Conductor, >>> and Audrey's config server interact and their relationship to a unique >>> vm instance id). >>> >>> Currently the Audrey script runs as a replacement to rc.local. Matahari >>> runs at S99. The general idea is for Audrey script to run as S98 >>> (before Matahari) and write a management-wide unique instance UUID to >>> the file /etc/vm_machine_id (audrey has access to this information). >>> >>> Matahari could be changed to read /etc/vm_machine_id first. If that >>> file doesn't exist, /var/lib/dbus/machine_id would be read. >>> >>> This creates some difficulty in running the audrey script at a specific >>> runlevel (it requires some changes to oz to insert init scripts). >>> >>> Another option is for the current rc.local script that audrey replaces >>> to run the Matahari service starting giblits as its first action. >>> >>> Comments welcome before I start writing code... >>> -steve >>> >> >> I think the solution of having Audrey store a launch time unique UUID in >> a file that Matahari can read will work. >> >> It may not be necessary to alter oz to insert init scripts to ensure >> Audrey runs before Matahari. >> >> Let me explain. >> >> Audrey does not replace /etc/rc.local >> >> When Image Factory builds the image it appends to the end >> of /etc/rc.local a line of code that will start Audrey. >> >> e.g.: >> [ -f /usr/bin/audrey ] && /usr/bin/audrey >> >> >> I propose having Image Factory append another line to /etc/rc.local >> below where it starts Audrey to start Matahari. >> >> e.g.: >> >> [ -f /usr/bin/audrey ] && /usr/bin/audrey >> >> [ -f <Matahari start> ] && <run Matahari start> >> >> We may need to manage timing to ensure /usr/bin/audrey does not return >> until it has stored the unique UUID in a file and have it return an >> error status if it is unable to. >> >> Thoughts? >> > > This sounds fine to me, although the rc.local modification always > sounded a bit hacky (rather then using a proper init script). One minor > issue is matahari expects to be started via service xxx start and each > agent has a separate init script. There are 5 or 6 agents. Yes, I think having audrey 'start matahari' is very hackish, since in normal systems matahari would start via regular init scripts. So this means for cloud we'd need to disable the normal init scripts and then relegate control to audrey. I don't like that approach... Let's back up a bit... why does matahari start need to depend on audrey starting? The answer is that we need audrey to put in the /etc/machine-id file. Well, why not let matahari start at its normal runlevel, and respond to queries, etc, but if you call get-id API, then it should return something that indicates '/etc/machine-id not set yet, giving you dbus-id instead' Then it's just up to the person doing the querying to wait until the id returned is the /etc/machine-id. So remove the dep on service start and replace with intelligent application usage

Thoughts? Perry _______________________________________________ aeolus-devel mailing list aeolus-devel(a)lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/aeolus-devel

On 06/28/2011 10:26 AM, Steven Dake wrote: > On 06/28/2011 06:24 AM, Joseph VLcek wrote: >> On Mon, 2011-06-27 at 13:59 -0700, Steven Dake wrote: >>> On 06/24/2011 04:16 PM, Steven Dake wrote: >>>> Currently most linux distributions that use dbus store a UUID in >>>> /var/lib/dbus/machine_id. In our pacemaker-cloud work test tools, we >>>> must manipulate this file via oz image creation to match a value we know >>>> about. >>>> >>>> Q1. Is this file freshly created on each image creation/cloning process? >>>> >>>> If not, it should be, because Matahari uses this information to uniquely >>>> identify a host. If it is copied exactly to each new image, that >>>> creates a problem (all hosts appear the same to matahari). >>>> >>>> Q2. If/when it is created by image factory, will it be stored in a >>>> database or other storage medium? >>>> >>>> In pacemaker-cloud we need to have a mapping from image->internal id id >>>> so that we know which VM maps to which deployable HA configuration. >>>> >>>> If we wait on this point until after our 1.0 release, we could end up >>>> with a bunch of images in the field that have either the same machine id >>>> or are not mapped in any way that allows us to provide HA functionality. >>>> >>>> Regards >>>> -steve >>>> >>> >>> After more investigation, Chris and I came up with a workable plan for >>> handling unique VM ids (see thread with subject How Audrey, Conductor, >>> and Audrey's config server interact and their relationship to a unique >>> vm instance id). >>> >>> Currently the Audrey script runs as a replacement to rc.local. Matahari >>> runs at S99. The general idea is for Audrey script to run as S98 >>> (before Matahari) and write a management-wide unique instance UUID to >>> the file /etc/vm_machine_id (audrey has access to this information). >>> >>> Matahari could be changed to read /etc/vm_machine_id first. If that >>> file doesn't exist, /var/lib/dbus/machine_id would be read. >>> >>> This creates some difficulty in running the audrey script at a specific >>> runlevel (it requires some changes to oz to insert init scripts). >>> >>> Another option is for the current rc.local script that audrey replaces >>> to run the Matahari service starting giblits as its first action. >>> >>> Comments welcome before I start writing code... >>> -steve >>> >> >> I think the solution of having Audrey store a launch time unique UUID in >> a file that Matahari can read will work. >> >> It may not be necessary to alter oz to insert init scripts to ensure >> Audrey runs before Matahari. >> >> Let me explain. >> >> Audrey does not replace /etc/rc.local >> >> When Image Factory builds the image it appends to the end >> of /etc/rc.local a line of code that will start Audrey. >> >> e.g.: >> [ -f /usr/bin/audrey ] && /usr/bin/audrey >> >> >> I propose having Image Factory append another line to /etc/rc.local >> below where it starts Audrey to start Matahari. >> >> e.g.: >> >> [ -f /usr/bin/audrey ] && /usr/bin/audrey >> >> [ -f <Matahari start> ] && <run Matahari start> >> >> We may need to manage timing to ensure /usr/bin/audrey does not return >> until it has stored the unique UUID in a file and have it return an >> error status if it is unable to. >> >> Thoughts? >> > > This sounds fine to me, although the rc.local modification always > sounded a bit hacky (rather then using a proper init script). One minor > issue is matahari expects to be started via service xxx start and each > agent has a separate init script. There are 5 or 6 agents. Yes, I think having audrey 'start matahari' is very hackish, since in normal systems matahari would start via regular init scripts. So this means for cloud we'd need to disable the normal init scripts and then relegate control to audrey. I don't like that approach... Let's back up a bit... why does matahari start need to depend on audrey starting? The answer is that we need audrey to put in the /etc/machine-id file.

Well, why not let matahari start at its normal runlevel, and respond to queries, etc, but if you call get-id API, then it should return something that indicates '/etc/machine-id not set yet, giving you dbus-id instead' Then it's just up to the person doing the querying to wait until the id returned is the /etc/machine-id. So remove the dep on service start and replace with intelligent application usage Thoughts? Perry _______________________________________________ aeolus-devel mailing list aeolus-devel(a)lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/aeolus-devel

On 06/28/2011 10:47 AM, Greg Blomquist wrote: > On 06/28/2011 12:05 PM, Perry Myers wrote: >> On 06/28/2011 10:26 AM, Steven Dake wrote: >>> On 06/28/2011 06:24 AM, Joseph VLcek wrote: >>>> On Mon, 2011-06-27 at 13:59 -0700, Steven Dake wrote: >>>>> On 06/24/2011 04:16 PM, Steven Dake wrote: >>>>>> Currently most linux distributions that use dbus store a UUID in >>>>>> /var/lib/dbus/machine_id. In our pacemaker-cloud work test tools, we >>>>>> must manipulate this file via oz image creation to match a value we know >>>>>> about. >>>>>> >>>>>> Q1. Is this file freshly created on each image creation/cloning process? >>>>>> >>>>>> If not, it should be, because Matahari uses this information to uniquely >>>>>> identify a host. If it is copied exactly to each new image, that >>>>>> creates a problem (all hosts appear the same to matahari). >>>>>> >>>>>> Q2. If/when it is created by image factory, will it be stored in a >>>>>> database or other storage medium? >>>>>> >>>>>> In pacemaker-cloud we need to have a mapping from image->internal id id >>>>>> so that we know which VM maps to which deployable HA configuration. >>>>>> >>>>>> If we wait on this point until after our 1.0 release, we could end up >>>>>> with a bunch of images in the field that have either the same machine id >>>>>> or are not mapped in any way that allows us to provide HA functionality. >>>>>> >>>>>> Regards >>>>>> -steve >>>>>> >>>>> >>>>> After more investigation, Chris and I came up with a workable plan for >>>>> handling unique VM ids (see thread with subject How Audrey, Conductor, >>>>> and Audrey's config server interact and their relationship to a unique >>>>> vm instance id). >>>>> >>>>> Currently the Audrey script runs as a replacement to rc.local. Matahari >>>>> runs at S99. The general idea is for Audrey script to run as S98 >>>>> (before Matahari) and write a management-wide unique instance UUID to >>>>> the file /etc/vm_machine_id (audrey has access to this information). >>>>> >>>>> Matahari could be changed to read /etc/vm_machine_id first. If that >>>>> file doesn't exist, /var/lib/dbus/machine_id would be read. >>>>> >>>>> This creates some difficulty in running the audrey script at a specific >>>>> runlevel (it requires some changes to oz to insert init scripts). >>>>> >>>>> Another option is for the current rc.local script that audrey replaces >>>>> to run the Matahari service starting giblits as its first action. >>>>> >>>>> Comments welcome before I start writing code... >>>>> -steve >>>>> >>>> >>>> I think the solution of having Audrey store a launch time unique UUID in >>>> a file that Matahari can read will work. >>>> >>>> It may not be necessary to alter oz to insert init scripts to ensure >>>> Audrey runs before Matahari. >>>> >>>> Let me explain. >>>> >>>> Audrey does not replace /etc/rc.local >>>> >>>> When Image Factory builds the image it appends to the end >>>> of /etc/rc.local a line of code that will start Audrey. >>>> >>>> e.g.: >>>> [ -f /usr/bin/audrey ] && /usr/bin/audrey >>>> >>>> >>>> I propose having Image Factory append another line to /etc/rc.local >>>> below where it starts Audrey to start Matahari. >>>> >>>> e.g.: >>>> >>>> [ -f /usr/bin/audrey ] && /usr/bin/audrey >>>> >>>> [ -f <Matahari start> ] && <run Matahari start> >>>> >>>> We may need to manage timing to ensure /usr/bin/audrey does not return >>>> until it has stored the unique UUID in a file and have it return an >>>> error status if it is unable to. >>>> >>>> Thoughts? >>>> >>> >>> This sounds fine to me, although the rc.local modification always >>> sounded a bit hacky (rather then using a proper init script). One minor >>> issue is matahari expects to be started via service xxx start and each >>> agent has a separate init script. There are 5 or 6 agents. >> >> Yes, I think having audrey 'start matahari' is very hackish, since in >> normal systems matahari would start via regular init scripts. So this >> means for cloud we'd need to disable the normal init scripts and then >> relegate control to audrey. I don't like that approach... >> >> Let's back up a bit... why does matahari start need to depend on audrey >> starting? The answer is that we need audrey to put in the >> /etc/machine-id file. > > Hmm...I'm not sure I buy this. I don't think there needs to be even > this level of dependency. Why can't another mechanism be in charge of > putting the /etc/machine-id in place? > > It still looks like matahari and audrey both require a machine-id (and > even pacemaker?). If they could rely on the same machine id for their > individual purposes, then I think we're closer to a better answer. > > So, how do we get that machine-id in place? With ec2, it's "user data". > I honestly think with our own cloud offerings (rhev-m) we need a > solution for this, too. With rhev-m 3, it's supposed to be solved with > hooks (I don't claim to know how those work, just that they've been > proposed as the solution for injecting data into a launching instance). > > I really think that cloud engine should be responsible for creating this > machine-id and injecting it into the launching instance. This gives us > one place to generate this id and doesn't require mapping this id to > other ids in order to translate a single instance between services that > want to interact with the instance. > > This puts all the logic back into deltacloud. It becomes deltacloud's > responsibility to figure out how to interact with the various cloud > providers, so that no matter in what provider an instance is running, we > can always get to the machine-id for that instance. > > Am I seeing this completely upside down? Or, off in the trees by myself > on this?

> I had originally thought deltacloud might be involved here in some way to consistently load instance data into the instance (and then provide a consistent way of accessing it inside the vm that doesn't involve writing 10 different access mechanisms). I don't know enough about deltacloud, but if the insertion functionality is currently a gap in any virtual machine manager (such as paravirt Xen) or RHEVH, the entire Aeolus solution as well as cloud-ha becomes non-functional on those VMMs. The question for deltacloud evaluation of this functionality then boils down to: 1) is this functionality available on every deltacloud cloud provider implementation

2) is the functionality inside the vm consistent (does a user only call a "read_my_id" function or read a file inside the vm) - if library access what is the new dependency inside the vm?

Regards -steve >> >> Well, why not let matahari start at its normal runlevel, and respond to >> queries, etc, but if you call get-id API, then it should return >> something that indicates '/etc/machine-id not set yet, giving you >> dbus-id instead' >> >> Then it's just up to the person doing the querying to wait until the id >> returned is the /etc/machine-id. >> >> So remove the dep on service start and replace with intelligent >> application usage >> >> Thoughts? >> >> Perry >> _______________________________________________ >> aeolus-devel mailing list >> aeolus-devel(a)lists.fedorahosted.org >> https://fedorahosted.org/mailman/listinfo/aeolus-devel > > _______________________________________________ > aeolus-devel mailing list > aeolus-devel(a)lists.fedorahosted.org > https://fedorahosted.org/mailman/listinfo/aeolus-devel

On 06/28/11 - 03:12:57PM, Greg Blomquist wrote: > I talked to clalance for a moment on IRC to figure out what I wasn't > seeing. The main piece was the bit between networking and dbus. Since > dbus is likely to be up and configured by the time networking starts, > there are limitations to how we can get to the "user data" in some > cases. Specifically for ec2 (see comments below). > >>> >> >> I had originally thought deltacloud might be involved here in some way >> to consistently load instance data into the instance (and then provide a >> consistent way of accessing it inside the vm that doesn't involve >> writing 10 different access mechanisms). >> >> I don't know enough about deltacloud, but if the insertion functionality >> is currently a gap in any virtual machine manager (such as paravirt Xen) >> or RHEVH, the entire Aeolus solution as well as cloud-ha becomes >> non-functional on those VMMs. >> >> The question for deltacloud evaluation of this functionality then boils >> down to: >> 1) is this functionality available on every deltacloud cloud provider >> implementation > > Not readily or consistently. For instance, rhev-m has "hook" > functionality, but we haven't yet figured out what it means to use the > hooks to provide this functionality for rhev-m. > > VMWare has some form of data injection available, as long as you install > their guest agent, enabling certain api functionality that deltacloud > could talk to. > > It's significantly different for each provider, that it takes research > each time a new provider is taken on. > >> 2) is the functionality inside the vm consistent (does a user only call >> a "read_my_id" function or read a file inside the vm) - if library >> access what is the new dependency inside the vm? > > No. In fact, for the case of EC2, this particularly breaks down. > Because EC2 stores the "user data" in a place only accessible over the > network, the VM needs to ask a known ec2 location for the user data via > an HTTP GET. > > Since dbus is probably already firmly set in stone by the time > networking comes up, it's likely too late to rely on ec2's user data to > store to this information and have it impact dbus after retrieval. Just to be clear here, I don't know that this is the case. I haven't personally looked into dbus, and how to configure the UUID on-the-fly. This definitely bears investigation, as I feel that getting dbus to properly expose this information is our best path forward.

On 06/28/11 - 03:12:02PM, Steven Dake wrote: >>>> I had originally thought deltacloud might be involved here in some way >>>> to consistently load instance data into the instance (and then provide a >>>> consistent way of accessing it inside the vm that doesn't involve >>>> writing 10 different access mechanisms). >>>> >>>> I don't know enough about deltacloud, but if the insertion functionality >>>> is currently a gap in any virtual machine manager (such as paravirt Xen) >>>> or RHEVH, the entire Aeolus solution as well as cloud-ha becomes >>>> non-functional on those VMMs. >>>> >>>> The question for deltacloud evaluation of this functionality then boils >>>> down to: >>>> 1) is this functionality available on every deltacloud cloud provider >>>> implementation >>> >>> Not readily or consistently. For instance, rhev-m has "hook" >>> functionality, but we haven't yet figured out what it means to use the >>> hooks to provide this functionality for rhev-m. >>> >>> VMWare has some form of data injection available, as long as you install >>> their guest agent, enabling certain api functionality that deltacloud >>> could talk to. >>> >>> It's significantly different for each provider, that it takes research >>> each time a new provider is taken on. >>> >>>> 2) is the functionality inside the vm consistent (does a user only call >>>> a "read_my_id" function or read a file inside the vm) - if library >>>> access what is the new dependency inside the vm? >>> >>> No. In fact, for the case of EC2, this particularly breaks down. >>> Because EC2 stores the "user data" in a place only accessible over the >>> network, the VM needs to ask a known ec2 location for the user data via >>> an HTTP GET. >>> >>> Since dbus is probably already firmly set in stone by the time >>> networking comes up, it's likely too late to rely on ec2's user data to >>> store to this information and have it impact dbus after retrieval. >> >> Just to be clear here, I don't know that this is the case. I haven't >> personally looked into dbus, and how to configure the UUID on-the-fly. This >> definitely bears investigation, as I feel that getting dbus to properly >> expose this information is our best path forward. >> > > looks like upstream is taking a crack at moving this into systemd rather > then dbus........ > > http://lists.freedesktop.org/archives/dbus/2011-March/014187.html > > dbus does not have a set system id API. > > if we add a dbus set system id API and call it, say at the end of the > startup sequence, some apps will be aware of the old id and others will > be aware of the new id. I expect this would break things badly, but I > am not a dbus expert so can't comment here. > > Perhaps we should let dbus do its own thing and just create a new file. > Then we have two files > > /etc/machine_id (worthless in a cloud environment except for dbus operation) > /etc/vm_machine_id (written by audrey or a dbus/systemd api) which can > be set after the network is operational. Then the machine id logic can > be put in one place (audrey) rather then a bunch of separate apps. > > I still have a bit of confusion how current VMs get their unique id > inside a vm in the current model to communicate with Audrey's config server. > > more fuel on the fire: > http://www.freedesktop.org/wiki/Software/systemd/hostnamed > > Lets go back to basics, the requirements: > > R1: vm needs to have some mechanism of retrieving a globally unique UUID > across all VMs in a cloud enviornment > R2. whatever launches the vm needs to know how to map a UUID to an image > R3. To obtain the vm UUID inside a vm, the network must be operational > (is this true in atleast some cloud environments? If so, then it is a > requirement). Yes, this is all about right. Requirement 3 is the difficult one. But maybe we are looking at this the wrong way. If we start from the premise that dbus/systemd can generate it's own UUID at boot time (suitably configured), then maybe we should think about using *that* as the UUID for the machine. What this would then require is a mapping between the UUID the cloud machinery has (i.e. the one that was generated prior to launch) and the UUID that dbus generated. It's a little ugly, but it prevents us from having problems when we restart dbus/systemd on an already running system. Again, I'm just throwing out ideas here. I still think we need to look closer at dbus/systemd and see if we can possibly send some patches to those to support changing the UUID in a cleaner manner. Then we can determine what the best path forward is.

On 06/28/2011 10:26 AM, Steven Dake wrote: > On 06/28/2011 06:24 AM, Joseph VLcek wrote: >> On Mon, 2011-06-27 at 13:59 -0700, Steven Dake wrote: >>> On 06/24/2011 04:16 PM, Steven Dake wrote: >>>> Currently most linux distributions that use dbus store a UUID in >>>> /var/lib/dbus/machine_id.  In our pacemaker-cloud work test tools, we >>>> must manipulate this file via oz image creation to match a value we know >>>> about. >>>> >>>> Q1. Is this file freshly created on each image creation/cloning process? >>>> >>>> If not, it should be, because Matahari uses this information to uniquely >>>> identify a host.  If it is copied exactly to each new image, that >>>> creates a problem (all hosts appear the same to matahari). >>>> >>>> Q2. If/when it is created by image factory, will it be stored in a >>>> database or other storage medium? >>>> >>>> In pacemaker-cloud we need to have a mapping from image->internal id id >>>> so that we know which VM maps to which deployable HA configuration. >>>> >>>> If we wait on this point until after our 1.0 release, we could end up >>>> with a bunch of images in the field that have either the same machine id >>>> or are not mapped in any way that allows us to provide HA functionality. >>>> >>>> Regards >>>> -steve >>>> >>> >>> After more investigation, Chris and I came up with a workable plan for >>> handling unique VM ids (see thread with subject How Audrey, Conductor, >>> and Audrey's config server interact and their relationship to a unique >>> vm instance id). >>> >>> Currently the Audrey script runs as a replacement to rc.local.  Matahari >>> runs at S99.  The general idea is for Audrey script to run as S98 >>> (before Matahari) and write a management-wide unique instance UUID to >>> the file /etc/vm_machine_id (audrey has access to this information). >>> >>> Matahari could be changed to read /etc/vm_machine_id first.  If that >>> file doesn't exist, /var/lib/dbus/machine_id would be read. >>> >>> This creates some difficulty in running the audrey script at a specific >>> runlevel (it requires some changes to oz to insert init scripts). >>> >>> Another option is for the current rc.local script that audrey replaces >>> to run the Matahari service starting giblits as its first action. >>> >>> Comments welcome before I start writing code... >>> -steve >>> >> >> I think the solution of having Audrey store a launch time unique UUID in >> a file that Matahari can read will work. >> >> It may not be necessary to alter oz to insert init scripts to ensure >> Audrey runs before Matahari. >> >> Let me explain. >> >> Audrey does not replace /etc/rc.local >> >> When Image Factory builds the image it appends to the end >> of /etc/rc.local a line of code that will start  Audrey. >> >> e.g.: >> [ -f /usr/bin/audrey ] && /usr/bin/audrey >> >> >> I propose having Image Factory append another line to /etc/rc.local >> below where it starts Audrey to start Matahari. >> >> e.g.: >> >> [ -f /usr/bin/audrey ] && /usr/bin/audrey >> >> [ -f <Matahari start> ] && <run Matahari start> >> >> We may need to manage timing to ensure /usr/bin/audrey does not return >> until it has stored the unique UUID in a file and have it return an >> error status if it is unable to. >> >> Thoughts? >> > > This sounds fine to me, although the rc.local modification always > sounded a bit hacky (rather then using a proper init script).  One minor > issue is matahari expects to be started via service xxx start and each > agent has a separate init script.  There are 5 or 6 agents. Yes, I think having audrey 'start matahari' is very hackish, since in normal systems matahari would start via regular init scripts.  So this means for cloud we'd need to disable the normal init scripts and then relegate control to audrey.  I don't like that approach... Let's back up a bit... why does matahari start need to depend on audrey starting?  The answer is that we need audrey to put in the /etc/machine-id file. Well, why not let matahari start at its normal runlevel, and respond to queries, etc, but if you call get-id API, then it should return something that indicates '/etc/machine-id not set yet, giving you dbus-id instead' Then it's just up to the person doing the querying to wait until the id returned is the /etc/machine-id. So remove the dep on service start and replace with intelligent application usage Thoughts?

> TBH, I'm not a fan. > > Partly because that UUID is a property and attribute of every agent > and are initialised at startup - I don't like the idea of those being > volatile. ick, I forgot about that... > It also pre-supposes that you'd never want the dbus id if > /etc/machine-id is present. right, that is a bad assumption.  My original ideas around exposing UID in matahari APIs was to have _several_ api calls depending on what you considered to be the proper UUID So maybe we need to have calls like: get_dbus_uuid (/var/lib/dbus/machine_id) get_machine_uuid (/etc/machine-id) get_smbios_uuid And if we _really_ need a separate vm_machine_id get_vm_machine_uuid (/etc/vm_machine_id)

Yes, it's silly to have so many uuids, and I'd love to declare one of these to be canonical so we could deprecate the rest.  But until that happens, I fear we're going to need to at least provide support for exposing all of them... Then Aeolus and CloudHA can pick which ID they want to set and standardize on. The question specific to matahari would be: which uuid should we use consistently for each agent's core uuid?  Right now we use the dbus one right?  If no one is going to muck with that and reset it, I'm happy to continue using it for our purposes. > We talked in the past about providing access to both a hardware _and_ > a software uuid. > Which is  /etc/machine-id supposed to be?  If the former, then the > solution is easy - add an extra API call.

>>> It also pre-supposes that you'd never want the dbus id if >>> /etc/machine-id is present. >> >> right, that is a bad assumption.  My original ideas around exposing UID >> in >> matahari APIs was to have _several_ api calls depending on what you >> considered to be the proper UUID >> >> So maybe we need to have calls like: >> >> get_dbus_uuid (/var/lib/dbus/machine_id) >> get_machine_uuid (/etc/machine-id) >> get_smbios_uuid >> >> And if we _really_ need a separate vm_machine_id >> get_vm_machine_uuid (/etc/vm_machine_id) >> > > Slight variation on the theme... > I think we should focus on the properties of the uuids rather than > where they live and/or who generates them. > > So I'm thinking we want 5 uuids and probably one accessor, where each > uuid would have a different lifetime: > - externally configured So as you say below, this is set by postboot, current thinking is that postboot puts this in /etc/vm-machine-id, right?

> - generated once for the lifetime of the vm How is this different than the one above?

> - generated every time the hardware changes This is == to smbios uuid right?

> - generated every time the machine boots This is == DBus/systemd uuid right?

> - generated every time an agent starts What does this map to, and what's the value in it?  Does every Agent on a given guest get a different 'agent uuid'?

If so, I think I see why you'd want this. > With the accessor being: >    get_uuid(lifetime), and lifetime ::= > user|forever|hardware|boot|matahari I'd call the last one 'agent' not 'matahari' if it's unique to each agent.

> The externally configured one would/could be set as part of our "name > other than config or postboot" agent process. > People then pick whichever uuid has the properties they need. > > I imagine matahari would provide the first two as attributes/properties > also. No objections to the general concepts above, just the nitpicky questions I asked need to be clarified :) Perry

>>> - generated every time the machine boots >> >> This is == DBus/systemd uuid right? > > I believe not.  I believe the DBus uuid persists across reboots, but > potentially not across upgrades. Is that by design?  If not, then it seems like we could improve that particular uuid by submitting a bug to have dbus/systemd uuids persist across upgrades. ack to your other comments on this thread, can you write up an API proposal then with the new functions and properties that we'll be exposing for review on list?

On 07/21/2011 02:24 AM, Andrew Beekhof wrote: > > On Tue, Jul 5, 2011 at 11:58 PM, Perry Myers&lt;pmyers(a)redhat.com&gt;  wrote: >>>>> >>>>> - generated every time the machine boots >>>> >>>> This is == DBus/systemd uuid right? >>> >>> I believe not.  I believe the DBus uuid persists across reboots, but >>> potentially not across upgrades. >> >> Is that by design?  If not, then it seems like we could improve that >> particular uuid by submitting a bug to have dbus/systemd uuids persist >> across upgrades. >> >> ack to your other comments on this thread, can you write up an API >> proposal >> then with the new functions and properties that we'll be exposing for >> review >> on list? >> > > Sorry for the delay...  here are my proposed changes to the host api wrt. > uuids. > > > <!-- > Valid UUID lifetimes: >   Immutable    - Automatically configured by the system once if not >                  found and never reset.  May be pre-populated. >   Hardware     - Automatically (re)configured by the system whenever >                  the underlying hardware changes >   Reboot       - Automatically (re)configured by the system whenever >                  it boots >   Agent        - Automatically (re)configured by the system whenever >                  the agent starts >   User         - Manually configured by the admin/user. >                  May be pre-configured. > --> > > <method name="get_uuid"  desc="Obtain a UUID with the specified > lifetime from the machine"> >   <arg name="lifetime"   dir="I"  type="sstr" /> > </method> > > <!-- >   The only valid lifetime is 'User' >   Later implementations may support re-generating the 'Hardware' uuid > --> > <method name="set_uuid"  desc="Set a UUID with the specified lifetime"> >   <arg name="lifetime"   dir="I"  type="sstr" /> >   <arg name="uuid"       dir="I"  type="sstr" /> > </method> Thanks for writing this up.  My only question now is... what existing uuid implementations map here? i.e. for the implementation of hardware UUID we plan on using X, for the implementation of immutable UUID we will use /etc/machine-id

Stuff along those lines would complete the matrix needed here. Perry

Hi Andrew, On Fri, 2011-07-22 at 08:19 +1000, Andrew Beekhof wrote: > Are there any conditions under which /etc/machine-id would get > regenerated? If so that would rule it out. > > These were my initial thoughts: > > Immutable: /etc/machine-id (systemd) > Hardware: ? Something on top of the smbios API > Reboot: ? /var/lib/dbus/machine-id (dbus) > Agent: In memory using the libuuid API > User: /etc/custom-machine-id Okay, AIUI it (and I've take a quick look to confirm), both /etc/machine-id and /var/lib/dbus/machine-id are basically equivalent concepts They are both a UUID that should be generated when the machine is installed or booted for the first time and not change thereafter. Looking at the code, I think dbus and systemd make an effort to have these UUIDs be identical, but I'm not 100% sure. Hmm, I just checked an F-16 machine and the two UUIDs are different. The reference you've probably seen in the dbus docs to the UUID changing on reboot only applies to stateless systems - the UUID is stored on a part of the filesystem which may not persist across reboots. That could equally apply to /etc/machine-id too. In VM disk images, both of these UUIDs should be deleted so that they are generated when a new VM is booted from the image. Similar to what is done for host SSH keys. On the RHEV-M side, there's a UUID in SMBIOS that has nothing to do with dbus or systemd. This UUID corresponds to the instance ID you'd see in the deltacloud API. When you launch a deployable, you'll get the list of deltacloud instance IDs back. I'm guessing you want Matahari to reliably report this UUID back as a "system hardware UUID"? In that case, the answer when running under RHEV-M is to read the UUID from SMBIOS. Under EC2, you get the ID from: http://169.254.169.254/latest/meta-data/instance-id and this isn't a UUID at all. I'm not sure about other clouds. Now, config server seems to have its notion of a machine UUID. This is passed to the instance at launch time and the audrey startup script has logic to find it. The idea is that this is supplied to config server by whatever is launching the instance. I'm not sure why this wouldn't also be the deltacloud ID.

Summary: - For the HA stuff, I think you need Matahari to be able to reliably report the deltacloud instance ID a "system hardware ID" or similar - That's unrelated to the systemd and dbus IDs, but these IDs should be unique to each machine too. - You should be able to determine the deltacloud instance ID from inside each VM. - This deltacloud instance ID should be what Audrey is using when contacting config server. - Audrey and Matahari should have the same logic for figuring this ID out. Cheers, Mark. _______________________________________________ aeolus-devel mailing list aeolus-devel(a)lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/aeolus-devel

Hi Greg, On Fri, 2011-07-22 at 13:05 -0400, Greg Blomquist wrote: > If the deltacloud ID is the UUID presented to deltacloud by the cloud > provider for the launched instance, then the only reason not to use the > deltacloud ID for the config server and audrey startup script is that it > requires the guest to launch and report this ID before the > configurations can be handed to the config server. Yeah, it's not ideal. > That's not terrible, but just makes the launch sequence more serial > instead of parallelizing the launch and seeding the configs to the > config server. If it's necessary to reduce the number of IDs floating > around, we can do this. Well, AFAICT, the only ID that the cloud HA stuff will know about from conductor or deltacloud is the deltacloud instance ID. So, that's what we want the guest agent to use when reporting back.

If you choose to add another ID for config server purposes, that would mean the cloud HA stuff would need to find that out. And I think that's a step too far in terms of intertwining the architecture. The cloud HA stuff should work with standalone deltacloud, without conductor or config server IMHO.

Cheers, Mark.

On 07/22/2011 10:22 AM, Greg Blomquist wrote: > On 07/22/2011 01:16 PM, Mark McLoughlin wrote: >> Hi Greg, >> >> On Fri, 2011-07-22 at 13:05 -0400, Greg Blomquist wrote: >>> If the deltacloud ID is the UUID presented to deltacloud by the cloud >>> provider for the launched instance, then the only reason not to use the >>> deltacloud ID for the config server and audrey startup script is that it >>> requires the guest to launch and report this ID before the >>> configurations can be handed to the config server. >> >> Yeah, it's not ideal. >> >>> That's not terrible, but just makes the launch sequence more serial >>> instead of parallelizing the launch and seeding the configs to the >>> config server. If it's necessary to reduce the number of IDs floating >>> around, we can do this. >> >> Well, AFAICT, the only ID that the cloud HA stuff will know about from >> conductor or deltacloud is the deltacloud instance ID. So, that's what >> we want the guest agent to use when reporting back. > > Ah, good point. > >> >> If you choose to add another ID for config server purposes, that would >> mean the cloud HA stuff would need to find that out. And I think that's >> a step too far in terms of intertwining the architecture. The cloud HA >> stuff should work with standalone deltacloud, without conductor or >> config server IMHO. > > Yep, I'm sold. > > The first cut I'm working on will generate IDs for Audrey. But, we can > circle back on this and revamp to use the deltacloud ID. The two > changes this implies on the Audrey side are > 1) the audrey startup script should use the UUID API to acquire the > Immutable UUID (assuming this is what maps to Deltacloud ID) > 2) the conductor launch sequence will be modified to wait to push > configs to config server until the instance is launched and can report > back the deltacloud ID > > Sound about right? > This proposal sounds good but leaves out a bit more detail further down the bootstrap order. We also need this ID loaded into matahari as well as matahari qpidd authentication data. This is where the discussion around the machine-id came from. If it is a different machine-external API coming out of matahari but available as soon as matahari is started, this WFM. This could likely be done by audrey with following start order: audrey client starts audrey sets matahari authentication information on the filesystem audrey sets UUID information on the filesystem audrey starts matahari

The issue with feeding the UUID into matahari via a local api call to matahari is that once matahari is started it registers with QMF, which leaves racey conditions where matahari may be active but without an instance id. Regards -steve > >> >> Cheers, >> Mark. >> > > _______________________________________________ > aeolus-devel mailing list > aeolus-devel(a)lists.fedorahosted.org > https://fedorahosted.org/mailman/listinfo/aeolus-devel

On Fri, Jul 22, 2011 at 03:30:11PM -0400, Greg Blomquist wrote: > On 07/22/2011 02:34 PM, Steven Dake wrote: >> On 07/22/2011 10:22 AM, Greg Blomquist wrote: >>> On 07/22/2011 01:16 PM, Mark McLoughlin wrote: >>>> Hi Greg, >>>> >>>> On Fri, 2011-07-22 at 13:05 -0400, Greg Blomquist wrote: >>>>> If the deltacloud ID is the UUID presented to deltacloud by the cloud >>>>> provider for the launched instance, then the only reason not to use the >>>>> deltacloud ID for the config server and audrey startup script is that it >>>>> requires the guest to launch and report this ID before the >>>>> configurations can be handed to the config server. >>>> >>>> Yeah, it's not ideal. >>>> >>>>> That's not terrible, but just makes the launch sequence more serial >>>>> instead of parallelizing the launch and seeding the configs to the >>>>> config server. If it's necessary to reduce the number of IDs floating >>>>> around, we can do this. >>>> >>>> Well, AFAICT, the only ID that the cloud HA stuff will know about from >>>> conductor or deltacloud is the deltacloud instance ID. So, that's what >>>> we want the guest agent to use when reporting back. >>> >>> Ah, good point. >>> >>>> >>>> If you choose to add another ID for config server purposes, that would >>>> mean the cloud HA stuff would need to find that out. And I think that's >>>> a step too far in terms of intertwining the architecture. The cloud HA >>>> stuff should work with standalone deltacloud, without conductor or >>>> config server IMHO. >>> >>> Yep, I'm sold. >>> >>> The first cut I'm working on will generate IDs for Audrey. But, we can >>> circle back on this and revamp to use the deltacloud ID. The two >>> changes this implies on the Audrey side are >>> 1) the audrey startup script should use the UUID API to acquire the >>> Immutable UUID (assuming this is what maps to Deltacloud ID) >>> 2) the conductor launch sequence will be modified to wait to push >>> configs to config server until the instance is launched and can report >>> back the deltacloud ID >>> >>> Sound about right? >>> >> >> This proposal sounds good but leaves out a bit more detail further down >> the bootstrap order. >> >> We also need this ID loaded into matahari as well as matahari qpidd >> authentication data. This is where the discussion around the machine-id >> came from. If it is a different machine-external API coming out of >> matahari but available as soon as matahari is started, this WFM. >> >> This could likely be done by audrey with following start order: >> >> audrey client starts >> audrey sets matahari authentication information on the filesystem >> audrey sets UUID information on the filesystem >> audrey starts matahari > > There's still a bootstrapping issue here. Well, maybe...possibly > depending on who cares. > > The problem is that the matahari authentication information would likely > come from the instance configuration delivered by the config server. > The audrey client would not get configuration data from the config > server until the config server learns about the instance from conductor. > Conductor wouldn't alert the config server about the instance until it > learns the deltacloud ID from the deltacloud driver. > > So, it looks like this (I think--assuming we're talking about a cloud > engine environment): > > 1. conductor tells deltacloud to launch guest > 2. guest boots in cloud provider > 3. audrey client starts

> 4. audrey client contacts config server, but finds nothing there...yet > 5. deltacloud driver tells conductor deltacloud ID for guest > 6. conductor tells config server about instance configs > 7. audrey client contacts config server, gets configs > 8. audrey client sets matahari auth information on filesystem > 9. audrey sets UUID information on the filesystem > 10. audrey starts matahari (finally) > 11. matahari can auth against broker Hmm... Given that we now have a working userdata mechanism for EC2 and VMWare, and a soon-to-be-working one for RHEV-M... could we not leverage that to compress steps 3-7? Meaning, if we dump the instance configs into userdata, the audrey client could read them from there, then go straight to contacting config server to get its matahari auth information? Tell me if i'm out of my mind here.

--Hugh

On Sat, Jul 23, 2011 at 5:30 AM, Greg Blomquist <gblomqui(a)redhat.com&gt; wrote: > > So, it looks like this (I think--assuming we're talking about a cloud > engine environment): > > 1. conductor tells deltacloud to launch guest > 2. guest boots in cloud provider > 3. audrey client starts > 4. audrey client contacts config server, but finds nothing there...yet > 5. deltacloud driver tells conductor deltacloud ID for guest What about: 5. deltacloud calls into the cloud provider to obtain the/a hardware-based UUID for the guest and tells conductor

Hi Andrew, On Fri, 2011-07-22 at 08:19 +1000, Andrew Beekhof wrote: > Are there any conditions under which /etc/machine-id would get > regenerated? If so that would rule it out. > > These were my initial thoughts: > > Immutable:  /etc/machine-id  (systemd)

> Hardware:   ? Something on top of the smbios API > Reboot:      ? /var/lib/dbus/machine-id  (dbus) > Agent:        In memory using the libuuid API > User:          /etc/custom-machine-id Okay, AIUI it (and I've take a quick look to confirm), both /etc/machine-id and /var/lib/dbus/machine-id are basically equivalent concepts They are both a UUID that should be generated when the machine is installed or booted for the first time and not change thereafter.

Looking at the code, I think dbus and systemd make an effort to have these UUIDs be identical, but I'm not 100% sure. Hmm, I just checked an F-16 machine and the two UUIDs are different. The reference you've probably seen in the dbus docs to the UUID changing on reboot only applies to stateless systems - the UUID is stored on a part of the filesystem which may not persist across reboots. That could equally apply to /etc/machine-id too.

In VM disk images, both of these UUIDs should be deleted so that they are generated when a new VM is booted from the image. Similar to what is done for host SSH keys. On the RHEV-M side, there's a UUID in SMBIOS that has nothing to do with dbus or systemd. This UUID corresponds to the instance ID you'd see in the deltacloud API. When you launch a deployable, you'll get the list of deltacloud instance IDs back. I'm guessing you want Matahari to reliably report this UUID back as a "system hardware UUID"?

In that case, the answer when running under RHEV-M is to read the UUID from SMBIOS. Under EC2, you get the ID from:  http://169.254.169.254/latest/meta-data/instance-id and this isn't a UUID at all. I'm not sure about other clouds.

Now, config server seems to have its notion of a machine UUID. This is passed to the instance at launch time and the audrey startup script has logic to find it. The idea is that this is supplied to config server by whatever is launching the instance. I'm not sure why this wouldn't also be the deltacloud ID. Summary:  - For the HA stuff, I think you need Matahari to be able to reliably    report the deltacloud instance ID a "system hardware ID" or similar

 - That's unrelated to the systemd and dbus IDs, but these IDs should    be unique to each machine too.  - You should be able to determine the deltacloud instance ID from    inside each VM.  - This deltacloud instance ID should be what Audrey is using when    contacting config server.  - Audrey and Matahari should have the same logic for figuring this ID    out. Cheers, Mark.

On 07/25/2011 01:20 AM, Andrew Beekhof wrote: > On Sat, Jul 23, 2011 at 2:40 AM, Mark McLoughlin <markmc(a)redhat.com&gt; wrote:

> If people don't want to pre-configure something, then a hardware UUID > (either read from the hardware or generated based on what is > installed) is really the only option. > > I'm quite happy to have Matahari's 'Hardware' UUID use smbios if > available* and do something with MAC addresses (and/or some other > hardware identifier) if smbios is empty/unavailable. > > * From what I can tell, RHEV, VMware and EC2 all support smbios... > anyone who doesn't? Greg, do you know?

> I'm quite happy to have Matahari's 'Hardware' UUID use smbios if > available* and do something with MAC addresses (and/or some other > hardware identifier) if smbios is empty/unavailable. > > * From what I can tell, RHEV, VMware and EC2 all support smbios... > anyone who doesn't? Greg, do you know?

On Tue, 2011-07-26 at 03:06 -0400, Chris Lalancette wrote: > RHEV supports SMBIOS, but the propsed solution to > "injecting" a UUID from the RHEV guys is to use an attached floppy > disk (since RHN also uses SMBIOS, and since SMBIOS is limited to > 64-byte strings in the normal case, floppy was seen as larger, easier, > and less likely to collide with RHN). Not quite right - floppy or file injection VDSM hooks is what the RHEV team is proposing for user data. The deltacloud instance ID (i.e. the VM UUID) is available in SMBIOS by default in RHEV guests.

On 07/21/2011 04:55 PM, Perry Myers wrote: > On 07/21/2011 02:24 AM, Andrew Beekhof wrote: >> On Tue, Jul 5, 2011 at 11:58 PM, Perry Myers&lt;pmyers(a)redhat.com&gt;  wrote: >>>>>> - generated every time the machine boots >>>>> >>>>> This is == DBus/systemd uuid right? >>>> >>>> I believe not.  I believe the DBus uuid persists across reboots, but >>>> potentially not across upgrades. >>> >>> Is that by design?  If not, then it seems like we could improve that >>> particular uuid by submitting a bug to have dbus/systemd uuids persist >>> across upgrades. >>> >>> ack to your other comments on this thread, can you write up an API proposal >>> then with the new functions and properties that we'll be exposing for review >>> on list? >>> >> >> Sorry for the delay...  here are my proposed changes to the host api wrt. uuids. >> >> >> <!-- >> Valid UUID lifetimes: >>    Immutable - Automatically configured by the system once if not >>                found and never reset.  May be pre-populated. >>    Hardware  - Automatically (re)configured by the system whenever >>                the underlying hardware changes >>    Reboot    - Automatically (re)configured by the system whenever >>                it boots >>    Agent     - Automatically (re)configured by the system whenever >>                the agent starts >>    User              - Manually configured by the admin/user. >>                May be pre-configured. >> --> >> >> <method name="get_uuid"  desc="Obtain a UUID with the specified >> lifetime from the machine"> >>    <arg name="lifetime"   dir="I"  type="sstr" /> >> </method> >> >> <!-- >>    The only valid lifetime is 'User' >>    Later implementations may support re-generating the 'Hardware' uuid >> --> >> <method name="set_uuid"  desc="Set a UUID with the specified lifetime"> >>    <arg name="lifetime"   dir="I"  type="sstr" /> >>    <arg name="uuid"       dir="I"  type="sstr" /> >> </method> > > Thanks for writing this up.  My only question now is... what existing > uuid implementations map here? > > i.e. for the implementation of hardware UUID we plan on using X, for the > implementation of immutable UUID we will use /etc/machine-id > > Stuff along those lines would complete the matrix needed here. My understanding is that the UUID API being discussed here is largely for identifying guests and guest services.

 To that end, it seems like the objects in the warehouse (images and target images; and later, templates, assemblies, deployables, deployments, and services, more?) fall outside the scope of this discussion.  Lemme know if I'm wrong there... As for the Audrey Startup script (the only thing I immediately see as falling in scope here), it would likely rely on a Agent or User scoped UUID.  I balk at saying it relies on the Immutable scoped UUID, because there needs to be a reliable way to set the UUID used by the Audrey Startup script in the guest before it launches, so the same value can be shared with the config server. We could force the Audrey Startup script to rely on the Immutable UUID (if the idea is to reduce the number of UUIDs floating around).  That will turn the launching strategy around in Conductor, insofar as Conductor would have to wait until the guest launched and could report back the Immutable UUID (somehow) via deltacloud before it could publish configuration data to the config server for the guest.  In the current planned implementation, Conductor generates the Audrey UUID and tells both the Config Server and the guest the value at roughly the same time. > > Perry > _______________________________________________ > aeolus-devel mailing list > aeolus-devel(a)lists.fedorahosted.org > https://fedorahosted.org/mailman/listinfo/aeolus-devel _______________________________________________ aeolus-devel mailing list aeolus-devel(a)lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/aeolus-devel

On Fri, Jul 1, 2011 at 6:56 AM, Perry Myers <pmyers(a)redhat.com&gt; wrote: >> TBH, I'm not a fan. >> >> Partly because that UUID is a property and attribute of every agent >> and are initialised at startup - I don't like the idea of those being >> volatile. > > ick, I forgot about that... > >> It also pre-supposes that you'd never want the dbus id if >> /etc/machine-id is present. > > right, that is a bad assumption.  My original ideas around exposing UID in > matahari APIs was to have _several_ api calls depending on what you > considered to be the proper UUID > > So maybe we need to have calls like: > > get_dbus_uuid (/var/lib/dbus/machine_id) > get_machine_uuid (/etc/machine-id) > get_smbios_uuid > > And if we _really_ need a separate vm_machine_id > get_vm_machine_uuid (/etc/vm_machine_id) > Slight variation on the theme... I think we should focus on the properties of the uuids rather than where they live and/or who generates them. So I'm thinking we want 5 uuids and probably one accessor, where each uuid would have a different lifetime: - externally configured - generated once for the lifetime of the vm - generated every time the hardware changes - generated every time the machine boots - generated every time an agent starts With the accessor being: get_uuid(lifetime), and lifetime ::= user|forever|hardware|boot|matahari

> One of the things we were trying to do was to not add YAU (Yet Another > UUID). > That is, we were hoping to hook into one of the existing mechanisms > (either > dbus or systemd) so that we could re-use this infrastructure.  That way > existing software wouldn't have to learn about yet another place to fetch > a > UUID from. > > If it turns out that this is impossible, then so be it.  But I still think > we > should investigate the options for integrating with the existing > mechanisms. > > Or did I completely mis-understand your point? I don't think what you said conflicts with what Andrew said...  Matahari has other use cases than just cloud, so we'll provide a set of UUIDs. Some of them may be useful for Cloud, others would be ignored.

Make sense?