On Wednesday 23 February 2011 00:47:48 Perry Myers wrote: > cross-posting to Aeolus and Matahari upstream projects... > > We discussed sort of a pilot project for integration of Matahari into > Aeolus usage. The driver for this is that we need a way for folks > upstream to be able to deploy a single node Aeolus environment managing > 1 or more Fedora hosts using kvm/libvirt (or VMware) > > The problem is that we don't have a way in this environment to > bootstrap, specifically for Aeolus to get the ip address of the guests > as they are booting on the libvirt/kvm/VMware hosts. > > I thought that this would be a good simple first use case for running > Matahari in Aeolus managed guests. So here's what I propose: > > * Aeolus uses Image Factory to build images... When those images are > either Fedora, RHEL or Windows IF will include Matahari and puppet in > the core images > * Aeolus will inject basic bootstrapping info for Matahari like broker > ip address and broker auth info (this ignores the more complex post > boot config process Carl has for the time being, this process is > quick and dirty just to get stuff working and then we can expand to > use Carl's process as a second step) > * This injection could happen in a few ways: > - During image build > - During image deployment (using guestfish to inject a config file) > - Other? Could rely on DNS SRV records w/ no broker auth (again this > is meant for developer environments, not production usage) > * When an instance is booted, matahari comes up uses the injected config > and connects to the broker and a "hey I'm a new guest on the bus" > event should be sent out which includes the ip address of the guest > * Simple QMF console written that runs as part of the Conductor > infrastructure that monitors the bus for new 'I'm here' events and > when it sees one it grabs the ip address and stores it in the database > * It then calls a processConfig method (on which Agent?) which gives a > puppet file to a guest agent to process via a call to puppet tool. > Return from that method should be success or failure (with logging > info probably) This means puppet needs to be baked into the image as > well along with matahari. > This is good as long as the guest obtains network connectivity. When the guest comes up with no network nothing happens and Aeolus will not be able manage this guest.

On Wed, Feb 23, 2011 at 07:15:43AM -0500, Perry Myers wrote: > On 02/23/2011 04:27 AM, Barak Azulay wrote: > > On Wednesday 23 February 2011 00:47:48 Perry Myers wrote: > >> cross-posting to Aeolus and Matahari upstream projects... > >> > >> We discussed sort of a pilot project for integration of Matahari into > >> Aeolus usage. The driver for this is that we need a way for folks > >> upstream to be able to deploy a single node Aeolus environment > >> managing 1 or more Fedora hosts using kvm/libvirt (or VMware) > >> > >> The problem is that we don't have a way in this environment to > >> bootstrap, specifically for Aeolus to get the ip address of the guests > >> as they are booting on the libvirt/kvm/VMware hosts. > >> > >> I thought that this would be a good simple first use case for running > >> Matahari in Aeolus managed guests. So here's what I propose: > >> > >> * Aeolus uses Image Factory to build images... When those images are > >> > >> either Fedora, RHEL or Windows IF will include Matahari and puppet > >> in the core images > >> > >> * Aeolus will inject basic bootstrapping info for Matahari like broker > >> > >> ip address and broker auth info (this ignores the more complex post > >> boot config process Carl has for the time being, this process is > >> quick and dirty just to get stuff working and then we can expand to > >> use Carl's process as a second step) > >> > >> * This injection could happen in a few ways: > >> - During image build > >> - During image deployment (using guestfish to inject a config file) > >> - Other? Could rely on DNS SRV records w/ no broker auth (again > >> this > >> > >> is meant for developer environments, not production usage) > >> > >> * When an instance is booted, matahari comes up uses the injected > >> config > >> > >> and connects to the broker and a "hey I'm a new guest on the bus" > >> event should be sent out which includes the ip address of the guest > >> > >> * Simple QMF console written that runs as part of the Conductor > >> > >> infrastructure that monitors the bus for new 'I'm here' events and > >> when it sees one it grabs the ip address and stores it in the > >> database > >> > >> * It then calls a processConfig method (on which Agent?) which gives a > >> > >> puppet file to a guest agent to process via a call to puppet tool. > >> Return from that method should be success or failure (with logging > >> info probably) This means puppet needs to be baked into the image > >> as well along with matahari. > > > > This is good as long as the guest obtains network connectivity. > > When the guest comes up with no network nothing happens and Aeolus will > > not be able manage this guest. > > Yes, I think it's assumed that cloud guests require network > connectivity... If a guest comes up and cannot get an ip address then > this is pretty much a critical failure. > > Hugh/jrd, let me know if this assumption is incorrect. If a guest boots in the cloud but no network is there to connect to it, does it really exist?

--H

>> If a guest boots in the cloud but no network is there to connect to >> it, does it really exist? > > Yes it does and it consumes valubale CPU and memory. > The question what happens in such a case. Very simple answer... if you spin up a guest in a cloud, and it doesn't get a network interface/ip address, you wait a bit (configurable) and then you shoot it violently in the head. A guest in the cloud with no network is not worth keeping around :)

cross-posting to Aeolus and Matahari upstream projects... We discussed sort of a pilot project for integration of Matahari into Aeolus usage. The driver for this is that we need a way for folks upstream to be able to deploy a single node Aeolus environment managing 1 or more Fedora hosts using kvm/libvirt (or VMware) The problem is that we don't have a way in this environment to bootstrap, specifically for Aeolus to get the ip address of the guests as they are booting on the libvirt/kvm/VMware hosts. I thought that this would be a good simple first use case for running Matahari in Aeolus managed guests. So here's what I propose: * Aeolus uses Image Factory to build images... When those images are either Fedora, RHEL or Windows IF will include Matahari and puppet in the core images * Aeolus will inject basic bootstrapping info for Matahari like broker ip address and broker auth info (this ignores the more complex post boot config process Carl has for the time being, this process is quick and dirty just to get stuff working and then we can expand to use Carl's process as a second step) * This injection could happen in a few ways: - During image build - During image deployment (using guestfish to inject a config file) - Other? Could rely on DNS SRV records w/ no broker auth (again this is meant for developer environments, not production usage) * When an instance is booted, matahari comes up uses the injected config and connects to the broker and a "hey I'm a new guest on the bus" event should be sent out which includes the ip address of the guest * Simple QMF console written that runs as part of the Conductor infrastructure that monitors the bus for new 'I'm here' events and when it sees one it grabs the ip address and stores it in the database * It then calls a processConfig method (on which Agent?) which gives a puppet file to a guest agent to process via a call to puppet tool. Return from that method should be success or failure (with logging info probably) This means puppet needs to be baked into the image as well along with matahari. TODOs: Matahari Team: * Write host agent event for 'i'm here configure me, here's my ip address' * Write method (which agent?) for processConfig(puppet-conf) Aeolus Team: * Need to write a simple QMF Console app that is part of the Conductor that registers for 'new guest' events and when it sees these reports that to Conductor and calls the processConfig method with the right puppet script * Run a broker alongside the Conductor and other Aeolus core infrastructure. The ip address of the host running this is what needs to be pushed into images for bootstrapping The end goal here should be making it EASY for someone to grab Aeolus and set it up quickly to manage local resources (i.e. we don't want to _require_ people playing with Aeolus to need an EC2 account) Andrew/Adam: Random question, what links all of the top level objects on a single host together? Is there a common/shared UUID exposed by each Agent? If a guest comes up and the Host agent appears, I can query that Host object for UUID. But how do I know which Network Agent is the one on the same guest as the Host Agent with the UUID that I just got?

Perry