On Mon, 2011-05-23 at 15:36 -0400, jrd(a)redhat.com wrote:
[A couple of assorted questions and observations sprinkled below.
Leaving the whole text so far, for context]
> From: Mark McLoughlin <markmc(a)redhat.com>
> Date: Mon, 23 May 2011 07:09:29 +0100
>
> Hi Scott,
>
> On Mon, 2011-05-23 at 01:02 -0400, Scott Seago wrote:
> > On 05/20/2011 04:21 PM, Mark McLoughlin wrote:
> > > Finally, FWIW, I'm pretty excited that this will get us out of some
of
> > > the ratholes we've been in lately.
> > >
> > > Cheers,
> > > Mark.
> > >
> > > Changes From Before
> > > ===================
> > >
> > > - There are two XML document formats - deployables and templates -
and
> > > they are quite independent
> > >
> > > - Deployables (or rather, assemblies) contain image references
rather
> > > than template references
I think, after reading this, that this isn't really much of a change,
rather it's more terminology than anything. The way I've always
talked about this, the reference from an assemblu was a symbolic
reference to a specific version of a template/images/icicles. IOW,
there was that whole cluster of stuff out there, which was different
morphs of the same information.
Yep, it's not a massive change, but I hope it will clear up some
confusion.
e.g. currently when we import an image for EC2, we construct a template
for it ... that warped my brain a bit
Also, when conductor is launching a deployable, because it won't have
access to the template (one may not even exist) I hope we'll have a
clearer separation between the template and deployable/assembly problem
spaces.
In the model as previously discussed, we said that it was to a
specific version, though, as the idea was that you couldn't
legitimately construct assemblies and especially deployables without
some knowledge of the capabilities of the underlying images. And if
you changed the image (definition) you could invalidate the
assumptions embodied in the higher level constructs. I'm not sure
whether you're still trying to keep that sort of sematic connection.
Yep, I agree - the idea is that a deployable can have a versionless or
versioned image reference. So, it's up to the deployable author.
e.g. if a deployable author knew that an image was only going to be
rebuilt for safe updates, it might make sense to leave it versionless
Also, we hope to make the resolution of versionless references
environment specific, so e.g. it might resolve to different versions in
dev, test and prod.
> > >
> > > - An image reference maps to multiple versions of the image, where
> > > each version has multiple provider images
> > >
> > > - Images can be built from a template, or imported from elsewhere
> > >
> > Regarding the above three bullet points, I spoke with Hugh at length
> > about the model change here on Friday, and I should note that this is
> > both a model change _and_ a terminology change.
>
> Yes, absolutely
>
> > I'm fine with the
> > change, but I'd like to describe my understanding of both the old and
> > new models in more detail here to make sure that we're all clear on
> > what's happening. In addition, I think the above description misses one
> > level in the hierarchy, but it is one which is largely an implementation
> > detail (although a fairly important one)
> >
> > First, the old (current Conductor codebase) model:
> > - A Template is a target-and-provider-independent way of referencing
> > a Provider Image (i.e. the 'template' is what an end user selects in
the
> > UI that maps -- once Condor does its thing -- to a particular Image on a
> > particular provider.) In addition, a Template includes the XML
> > description which tells Image Factory how to build Images for the
> > various back end providers. Note that in the current model the Template
> > does double duty here -- it's the specification as to what to build
> > _and_ the object which references a series of _equivalent_ provider
> > images on various back ends. In the new model these two concepts get
> > split apart
>
> Right
>
> > - An Image is the output of the 'build' API call on Image Factory
> > which is the result of building a disk image for a single target which
> > is not yet associated with a particular provider. In other words, an ec2
> > image is the root FS image in the proper format for ec2 which has not
> > yet been bundled or registered as an AMI in a particular region with a
> > particular owner. For RHEV, this would be the KVM image which has not
> > yet been uploaded/enabled for a particular RHEV server. From an
> > implementation POV, this is an important intermediate step (which I
> > believe is missing in Mark's writeup), as we don't need to generate
a
> > new from-scratch build for each ec2 region or for each RHEV server we
> > want to upload to
> > - A Provider Image is an image uploaded (bundled/registered/etc) to a
> > particular provider (ec2-us-east,
my-rhev-instance1.redhat.com, etc)
>
> Right, I think some notion of Image vs Provider Image is there in the my
> writeup as the target-and-provider-agnostic disk image associated with
> an image version
>
> >From what you say, though, it sounds like we don't have the notion of a
> target-and-provider-agnostic disk image, only a provider-agnostic disk
> image. Hmm
If I'm reading it right, there's still room for that construct. I
believe it's needed, due to the characteristics of how we do image
registration on some backends (ec2).
I had assumed we would need it too, but now that I dig into image
factory I see that we don't actually use it anywhere - e.g. for EC2, we
build it for EC2 (i.e. a Target Image) and then push it to multiple
providers (giving provider images)
Where it might make sense is if factory is building for multiple
targets, it could store the targetless image and transform it for
multiple targets, rather than building for each target individually.
But yes, you're right - there's still room for the concept
> > So there are 2 main changes here in the new schema.
First of
all, the
> > Template concept is split into two separate (but related) resources. The
> > Template describes the specification for the series of images that will
> > be built from it (i.e. the input to Image Factory), and the Image
> > describes a particular build in a target-independent way -- i.e. an
> > Image reference maps to a series of related builds, for various targets
> > that are to be considered equivalent (i.e. built at roughly the same
> > time for ec2-us-east, ec2-us-west, various private cloud instances,
> > etc.). Adding versioning to this results in, for a given Template, a
> > version-independent Image reference (perhaps just a name) and a series
> > of Image Versions (i.e. builds), each of which points to one or more
> > provider images (via the yet-unnamed intermediary that we used to call
> > Image)
>
> Yes. And note that not all Images are associated with an origin Template
> - i.e. images that are imported are represented by an Image with no
> associated Template
But in that case, would you not want to have some kind of descriptor,
possibly fairly minimal, to describe the provenance of this image?
Yes, we'll have an image descriptor in the "image" object with
information supplied from the user when importing it
This is the information a deployable author would look at when choosing
an instance or, indeed, what a user would look at when launching an
instance directly from an image
Scanning this again, I don't see any mention of icicle, which
we've
been treating as the "fully-blown-out-template" output of image
factory, parallel to an image. I would think you'd at least want
something like that, to be the place where you hang any info about
where the image came from, and whatever you know about its
capabilities.
Yep, image factory would continue dumping the icicle in iwhd
(It looks like it doesn't actually fill it in with any information yet,
though)
> > The second change is one of nomenclature. Now that the
term "Image" is
> > used to describe a target-independent set of equivalent images -- the
> > thing referenced by assemblies -- rather than "Template", we need
a new
> > name for what was "Image" -- i.e. a build for a particular
> > target/provider type but not associated with a specific provider. This
> > is something that will be used primarily for the Image Factory API (i.e.
> > the output from the 'build' action and the input to the
'push' action.
> > Assemblies and Deployables won't need to worry about it -- they specify
> > an image, and condor will map this to a Provider Image. For the moment
> > I'm calling this a TargetImage -- it's a valid name, I'm open to
> > suggestions for a better name.
>
> Okay, makes sense.
>
> I think I'd like to try something like this:
>
> - A Provider Image has a pair of metadata tags - (Target, Provider) -
> and the Provider part of the pair is optional
>
> That way, we get the concept of a target specific image but not
> associated with a particular provider
>
> > Or, to use a concrete example, consider the following collection of
objects:
> > JBossAS-template (Template XML doc -- belongs_to image)
> > JBossAS (image) (has_one Template (optional) )
> > JBossAS 1.0 (Image version)
> > JBossAS 1.0-ec2 (Target Image)
> > JBossAS-ec2-us-east (Provider Image)
> > JBossAS-ec2-us-west (Provider Image)
> > JBossAS-rhev (Target Image)
> > JBossAS uploaded to
rhev1.redhat.com (Provider Image)
> > JBossAS uploaded to
rhev2.redhat.com (Provider Image)
>
> Yep, that sounds right
>
> > or to define rails-y associations (this isn't an implementation
> > suggestion, as for the moment _none_ of these objects are proposed to
> > belong in the conductor DB, but I just wanted to show the relationships
> > in another way to make sure it's clear):
> >
> > class Template
> > belongs_to :image
> > end
> > class Image
> > has_one :template
> > has_many :image_versions
> > end
> > class ImageVersion
> > belongs_to :image
> > has_many :target_images
> > end
> > class TargetImage
> > belongs_to :image_version
> > has_many :provider_images
> > end
> > class ProviderImage
> > belongs_to :target_image
> > end
>
> Yep, that's it.
>
> Also, I think a Template can be built multiple times, perhaps by
> different users, to give different Images. In the new model, a template
> is just referenced by a URL and supplied to image factory, and there's
> no reason why the same URL couldn't be used to build different Images,
> perhaps with a different set of package updates
>
Right. That's part of the reason why we had icicle in there, to hold
the actual output of what you got, as opposed to what you asked for.
> Given that and my other suggestion above (i.e. collapse TargetImage into
> ProviderImage) gives:
>
> class Template
> has_many :image
> end
> class Image
> belongs_to :template
> has_many :image_versions
> end
> class ImageVersion
> belongs_to :image
> has_many :provider_images
> end
> class ProviderImage
> belongs_to :image
> belongs_to :provider_type
> belongs_to :provider
> end
>
> > > - Services are at the deployable level, not the template level
> > >
> > Does this mean we won't be embedding the service scripts (puppet, etc)
> > into the template -- only the packages required by the service? If the
> > latter, will they be specified manually?
>
> Right, service scripts won't be included in the template
>
I'm not sure I buy that.
How do you represent the fact that image X has a particular
capability? It's more than just packages. I've read this a few
times, and feel like I'm missing something.
An image can have parameters, and those parameters are consumed by
scripts built into the image. At the image level, all you know is the
entire set of parameters available for the image. At the template level,
you'd know about the individual scripts too.
Services are a separate concept - scripts that are specified by the
assembly to be pulled from the config server at boot time. The also have
parameters.
So, there's pretty much no change here apart from a terminology change
(i.e. not using the service term for config scripts included in a
template, since the conflation was causing confusion) and a realization
that images just expose a set of parameters.
> Services may install packages. If the deployable author
wishes to avoid
> that, she needs to ensure the template has the required packages
>
> In future, we may try to have an administrator level policy to prevent
> services installing updates, but that's hand-waving territory for the
> moment
>
> >From the discussions during the week, it turns out that the constraint
> around preventing services from installing deployables isn't important.
> The most we need do is initially is to provide template authors with
> tools to make it easy to ensure that the packages required for services
> are installed
>
> > > - Templates expose parameters, but they're not described as
services
> > >
> > How are we exposing parameters without defining the scripts which use
> > these parameters? Or are we separating the scripts from the notion of
> > services?
>
> Templates may have scripts (or perhaps only a single script) with
> parameters, but they aren't services
>
I guess that's the part I'm not getting. You've got a body of code
which does something for you. It may or may not need to be
parameterized, in particular to work out how, at launch time, to get
it sorted out w/r/t the other members of the group being launched.
Why is that not a service? Is there some semantic difference there?
All we're really trying to do here is clear up confusion and concentrate
on what we support in the short term.
Supporting services at the template level is starting to make some sense
to me but, in terms of prioritization, I think the order would be:
1) A single template configuration script with parameters and returns,
similar to kickstart %post - people can run multiple scripts from
here if they like
2) A single service script per assembly
3) Multiple service scripts per assembly
4) Services scripts at the template level
So, I think I've just dropped (4) from my immediate thinking rather than
saying it will never make sense
[I'm assuming all the breaking out of parameters from service
constructs in the following verbiage is due to that point, so I won't
comment on those]
> > > - Use of ruby string interpolation as a more flexible and expressive
way
> > > to manipulate parameters
> > >
> > > Parameters and Returns
> > > ======================
> > >
> > > At each level of the model, the concept of parameters and returns are
> > > used.
> > >
> > > A parameter is an input and a return is an output. Parameters are
> > > typed, returns are strings. Both have names and human consumable
> > > descriptions.
> > >
> > > When declaring a parameter that is made available by a level of the
> > > model to the level above it, you do e.g.
> > >
> > > <param type="string">
> > > <name>admin_user</name>
> > > <description>Administrator's
username</description>
> > > <value>admin</value>
> > > </param>
> > >
> > > A default value may be supplied, essentially making the parameter
> > > optional.
> > >
> > > When supplying a parameter from a level of the model to the level
> > > below it you do e.g.
> > >
> > > <param>
> > > <name>admin_user</name>
> > > <value>admin</value>
> > > </param>
> > >
> > > i.e. the type attribute and description element is irrelevant.
> > >
> > > When declaring a return that is supplied from a level of the model to
> > > the level above it, you do e.g.
> > >
> > > <return>
> > > <name>http_port</name>
> > > <description>HTTP port number</description>
> > > <value>80</value>
> > > </return>
> > >
> > > A<value> element of a parameter or return may use Ruby's
string
> > > interpolation to reference other parameter or return values e.g.
> > >
> > > <return>
> > > <description>URL of the Redmine project management
system</description>
> > >
<value>http://#{frontend.ip_address}:#{frontend.http_port}/</value>
> > > </return>
> > >
> > > or:
> > >
> > > <params>
> > > <param>
> > > <name>url</name>
> > > <value>http://#{hostname}:#{http_port +
100}/redmine/</value>
> > > </param>
> > > </params>
> > >
> > > Services
> > > ========
> > >
> > > Services encapsulate a configuration script or recipe for a
configuration
> > > management tool.
> > >
> > > So, for example, a script referenced by a URI:
> > >
> > > <service>
> > > <script href="...."/>
> > > <params/>
> > > <returns/>
> > > <script>
> > > </service>
> > >
> > > or a script embedded in the document:
> > >
> > > <service>
> > > <script>
> > > <contents><![CDATA[
> > > #!/bin/bash
> > > ...
> > > ]]</contents>
> > > <params/>
> > > <returns/>
> > > </script>
> > > <service>
> > >
> > > Parameters are passed to the script on stdin using a<params/>
element
> > > and the script writes a<returns/> element to stdout.
> > >
> > > Puppet manifests and chef recipes are handled similarly
using<puppet/>
> > > and<chef/> elements with appropriate ways for params and returns
to be
> > > exchanged.
> > >
> > Just to clarify, as I'm a bit confused here. Above you said that we
> > define services only in assemblies/deployables (not templates), but that
> > templates will describe parameters. Are these params without scripts, or
> > something else?
>
> They are scripts (or it is a script) which takes parameters but we're
> not describing as a service
>
> > Also, is the intent to only load the puppet/chef/etc scripts on
> > firstboot (not in the image itself)?
>
> Yep
Yes. Thank you :-)
We have a huge amount to do. I expect that we will eventually get to
the point of richer build semantics, like running arbitrary
construction paradigms inside image factory, but I think it we get it
working for rev 1 with fairly explicit package contents and simple
init/service scripts, we'll be doing great.
Right
> > > Images
> > > ======
> > >
> > > The term "image" refers to an abstract concept here, as
opposed to a binary
> > > disk image.
> > >
> > > An image is created by importing as a disk image from elsewhere or by
building
> > > a template.
> > >
> > > An image has multiple versions. Each version has a set of provider
images and
> > > (optionally) a provider agnostic disk image.
> > >
> > OK, I think the provider-agnostic disk image is what I'm calling a
> > TargetImage above -- if by this you mean an image that's ec2-specific
> > but not registered/bundled as an AMI in a particular region.
>
> No, I was thinking this provider-agnostic disk image was
> provider-type-agnostic :)
>
> I'm pretty convinced now just by what you say that we don't have such a
> concept
>
I believe that's true.
We did flirt with the idea many months ago of a "vanilla" image which
would then be morphed (using v2v tools or similar) into all the
backend-specific ones. We abandoned it for various reasons, primarily
because we were pretty sure we couldn't guarantee that we could come
up with something that would really DTRT on any given backend.
> > I think we may be making this more complex if we make that intermediate
> > stage optional -- i.e. it may be simpler to always have this
> > image,although there may be provider types (presumably mostly non-ec2
> > public providers) for which the TargetImage is mainly a placeholder. In
> > any case,this is an impl detail that we can figure out the right answer
> > for as we get there, as I don't think it will have any significant API
> > implications (except possibly for Image Factory).
> >
> > Unless you're referring to the provider-and-target-agnostic image (i.e.
> > could point to _any_ provider, not just providers of a specific type) --
> > in which case the above comments about TargetImage being missing in this
> > description still apply.
>
> Yep, it was the latter and you're right
>
> Either of your Image/ImageVersion/ProviderImage/TargetImage or my
> Image/ProviderImage schemes makes sense to me
>
> > > The concept exists to allow deployables to refer to disk images
without
> > > referencing a template, a specific template build or provider image.
> > >
> > In fact, deployables will never reference a provider image directly
> > (unless we're changing this now too) -- the deployable should always
> > reference Images in a provider-agnostic way.
>
> I think deployables should be able to reference the Image but also
> optionally reference the ImageVersion
>
> i.e. a deployable author can choose whether to use the latest version
> available or use a specific version
>
> > > Assemblies
> > > ==========
> > >
> > > Assemblies encapsulate an image reference and a set of services.
> > >
> > > An image is referenced by ID. e.g.
> > >
> > > <assembly>
> > > <name>frontend</name>
> > > <image
id="11abf870-894c-4336-bc9f-37904c394924">
> > > ...
> > > </image>
> > > </assemly>
> > >
> > > Optionally, the image reference can include a version id:
> > >
> > > <assembly>
> > > <name>frontend</name>
> > > <image id="11abf870-894c-4336-bc9f-37904c394924"
version="be2a8e6">
> > > ...
> > > </image>
> > > </assemly>
> > >
> > > This ID and optional version can be resolved by conductor, via
metadata
> > > and tags IWHD, to the set of provider image builds associated with the
> > > image.
> > >
> > > The image reference also describes the parameters to pass to instance
> > > when it is started:
> > >
> > > <assembly>
> > > <name>frontend</name>
> > > <image
id="11abf870-894c-4336-bc9f-37904c394924">
> > > ...
> > > <params>
> > > <param>
> > > <name>http_port</name>
> > > <value>#{http_port}</value>
> > > </param>
> > > ...
> > > </params>
> > > </image>
> > > <assembly>
> > >
> > > When an instance is launched, it can return information and these are
> > > also described in the image reference:
> > >
> > > <assembly>
> > > <name>frontend</name>
> > > <image
id="11abf870-894c-4336-bc9f-37904c394924">
> > > ...
> > > <returns>
> > > <return>
> > > <name>http_port</name>
> > > </return>
> > > </returns>
> > > </image>
> > > <assembly>
> > >
> > > Like all other objects, assemblies have returns. However, they also
> > > implicitly have some returns which do not need to be explicitly
> > > defined, like ip_address.
> > >
> > > Finally, Assemblies include a list of services:
> > >
> > > <assembly>
> > > ...
> > > <services>
> > > <service/>
> > > <service/>
> > > ...
> > > </services>
> > > </assemblies>
> > >
> > > == Deployables ==
> > >
> > > A deployable encapsulates all the information required to launch a set
> > > of cooperating instances. It contains:
> > >
> > > - Its name and description:
> > >
> > > <deployable>
> > > <name>Redmine</name>
> > > <description>Redmine is a web-based project management
application</description>
> > > ...
> > > </deployable>
> > >
> > > - Some parameters that the user launching the deployable may
> > > specify:
> > >
> > > <deployable>
> > > ...
> > > <params>
> > > <param type="string">
> > > <name>admin_user</name>
> > > <description>Administrator's
username</description>
> > > <value>admin</value>
> > > </param>
> > > <param type="string">
> > > <name>admin_passwd</name>
> > > <description>Administrator's
password</description>
> > > <secret>true</secret>
> > > </param>
> > > <param type="int">
> > > <name>http_port</name>
> > > <description>HTTP port number</description>
> > > <value>80</value>
> > > </param>
> > > <param type="string">
> > > <name>timezone</name>
> > > <description>Timezone in which to run the
application>
> > > <values>
> > > <value>America/New_York</value>
> > > <value>Europe/Paris</value>
> > > </value>
> > > </param>
> > > ...
> > > </deployable>
> > >
> > > When a user launches a deployable, a form is displayed allowing
> > > the user to enter values for the parameters. The form is
> > > pre-populated with default values. Where multiple default values
> > > are specified, a drop-down box is used to allow the user choose
> > > one of the values.
> > >
> > > - Assemblies, which may be included inline in the deployable:
> > >
> > > <deployable>
> > > ...
> > > <assemblies>
> > > <assembly>
> > > <name>frontend</name>
> > > <image
id="11abf870-894c-4336-bc9f-37904c394924">
> > > <params>
> > > <param>
> > > <name>redmine_admin</name>
> > > <value>#{admin_user}</value>
> > > </param>
> > > <param>
> > > <name>redmine_passwd</name>
> > > <value>#{admin_passwd}</value>
> > > </param>
> > > </params>
> > > </image>
> > > <assembly>
> > > </assemblies>
> > > ...
> > > </deployable>
> > >
> > > Notice that because this assembly is inline, it can directly
> > > reference the deployables parameter values.
> > >
> > > Assemblies can also be included by reference:
> > >
> > > <deployable>
> > > ...
> > > <assemblies>
> > > <assembly
href="http://myserver/redmine/frontend.assy">
> > > <params>
> > > <param>
> > > <name>admin_user</name>
> > > <value>#{admin_user}</value>
> > > </param>
> > > <param>
> > > <name>admin_passwd</name>
> > > <value>#{admin_passwd}</value>
> > > </param>
> > > </params>
> > > </assembly>
> > > </assemblies>
> > > </deployable>
> > >
> > > Here, because the assembly is included by reference, the
> > > deployable must explicitly pass the parameters required by the
> > > assembly. In this example, it has two parameters and we pass the
> > > values of the equivalent deployable parameters straight
> > > through.
> > >
> > > It is common to use the return values of an assembly as the
> > > paramaters to another assembly e.g.
> > >
> > > <assembly
href="http://myserver/redmine/caching-proxy.assy">
> > > <param>
> > > <name>base_url</name>
> > >
<value>http://#{frontend.ip_address}:#{frontend.http_port}/</value>
> > > </param>
> > > </assembly>
> > >
> > > - Return values which may be displayed to the user e.g.
> > >
> > > <deployable>
> > > ...
> > > <returns>
> > > <return>
> > > <description>URL of the Redmine project management
system</description>
> > >
<value>http://#{frontend.ip_address}:#{frontend.http_port}/</value>
> > > </return>
> > > </returns>
> > > </deployable>
> > >
> > > In summary, a deployable has a description, a set of typed parameters
> > > with defaults, a set of (inline or included by reference) assemblies
> > > and their parameter values and a set of user consumable return
> > > values.
> > >
> > > In conductor, a user launches a deployable simply by supplying a URI
> > > to the deployable definition, or by selecting from a list of
> > > deployable URIs supplied by the administrator.
> > >
> > > Templates
> > > =========
> > >
> > > Templates are an XML document which describe how to build an image and
> > > what the parameters and returns of the image will be. For example:
> > >
> > > <template>
> > > <name>win2kjeos</name>
> > > <os>
> > > <name>Windows</name>
> > > <version>2008</version>
> > > <arch>x86_64</arch>
> > > <install type='iso'>
> > >
<iso>http://directory_path/windows2008x64.iso</iso>
> > > </install>
> > > <key></key>
> > > </os>
> > > <description>Windows 2008</description>
> > > <repos>
> > > <repo url="smb://domain\user:UserName@IP\share"
name="Default"/>
> > > <repo
url="smb://domain\user:UserName@IP\another_share"
name="Alternative"/>
> > > </repos>
> > > <packages>
> > > <package>
> > > <repo name="Default"/>
> > > <name>Dot Net 4.0</name>
> > > <file>dotNetFx40_Full_x86_x64.exe</file>
> > > <arguments>/passive</arguments>
> > > </package>
> > > <package>
> > > <repo name="Alternative"/>
> > > <name>Winrar</name>
> > > <file>winrarx64393.exe</file>
> > > <arguments>/s</arguments>
> > > </package>
> > > </packages>
> > > </template>
> > >
> > > (Yes, that's copied and pasted from imgfac(1) man page :-)
> > >
> > > Image Warehouse Metadata
> > > ========================
> > >
> > > Image warehouse will contain enough metadata for conductor to resolve
> > > an image reference to a set of provider images.
> > >
> > > One possible way of storing that metadata would be as follows:
> > >
> > > - An image descriptor XML document describing the image is stored as
> > > an object. The UUID of the object is the image ID.
> > >
> > > - Each provider agnostic disk image is also an object and represents
> > > a build of a template. So an image version/build is identified by
> > > this object's UUID. The other attributes of this object include
the
> > > image UUID and UUID of its parent version, if any.
> > >
> > One question though -- "provider agnostic disk image" is what
we've been
> > calling "Image" throught here, right? Or is there something else
I've
> > missed here?
>
> In your scheme, this would be ImageVersion. I realize now that there is
> no disk image associated with this concept
>
> > And we'd insert target image here once we all agree 1) that we need it
> > and 2) what it's called
> > > - Each provider image has the disk image UUID as an attribute.
> > >
> > > A special case we need to think about is where an image is imported
> > > from e.g. EC2. In that case, the provider agnostic disk image
doesn't
> > > exist, so perhaps we'd create an empty object with the metadata we
> > > need.
> > >
> > Yes -- I think we'd represent everything but the Template at least as
> > placeholders. So we'll have an Image object (for inclusion in
> > assemblies), and a single Provider Image (the one we imported) -- but no
> > Template (since we didn't build it).
> >
> > These aren't useless placholders either -- for example,what if I have
> > some completely separate way of getting images built (say pulled
> > directly from a vendor,etc) and I know that I have an ec2-us-east AMI,
> > an ec2-us-west AMI, and a RHEV image, all of which are equivalent -- I'd
> > want to import all 3 of them into IWHD but all pointing to the same
> > provider-agnostic disk image -- so any assemblies that use this image
> > can be instantiated in all 3 back ends.
>
> Exactly, yes!
>
> > > In order to find all the provider images associated with an image, you
> > > find all the disk images referencing the image UUID and find all the
> > > provider images referencing all the disk image UUIDs.
> > >
> > OK, I'm starting to think we're proposing the same thing here with
> > different words again. Image, Disk Image, Provider Image -- are we
> > saying Image can reference all providers, all types (ec2, RHEV,
> > etc),disk image applies to multiple providers but to one target type
> > (ec2, multiple regions _or_ RHEV, multiple servers), and of course
> > Provider Image is a particular image in a particular provider (i.e. a
> > single AMI, an image in one RHEV server, etc.) -- or is there a
> > different distinction between "image" and "disk image"
?
>
> Hopefully it's clear at this point that I was just confused in thinking
> that there was a provider-and-target-agnostic disk image somewhere
>
> If that doesn't exist, it is replaced by the metadata-only object called
> ImageVersion in your scheme
>
> Thanks for the feedback Scott. To be crystal clear, here's what I think
> needs to be fixed in the writeup:
>
> - Kill off the DiskImage concept and replace with ImageVersion
>
> - Either add TargetImage, or make the provider optional in
> ProviderImage's (target, provider) pair
>
> - Clarify that service scripts aren't included in templates, but that
> templates can include a script which takes parameters
>
>
> This morning, I'm going to look in detail at how this all maps to image
> warehouse metadata and the queries conductor will need etc. Hopefully
> that level of detail will help us be more confident that this all works
>
> Cheers,
> Mark.
>
> _______________________________________________
> aeolus-devel mailing list
> aeolus-devel(a)lists.fedorahosted.org
>
https://fedorahosted.org/mailman/listinfo/aeolus-devel
>
In general, I think you've done a good job of capturing the important
bits. And the main thing which is very encouraging is that it sounds
like this exercise is getting us moving, Yay!
Cool.
The thing that I'm not convinced on in here is the services vs
scripts
thing. Perhaps some more elucidation on why that distinction exists?
It basically comes down to:
- I think templates need the concept of a %post script, independent
of services
- Services make sense in deployables and justify the existence of
assemblies
- We can support services in templates down the road
And I still continue to believe that we're going to need more
comprehensive structured (yes, gui) ways of creating all these
provisioning definitions and structures and stuff. *But*, we have a
path that gets us to standing something up, which is even more
important, so I'm not going to make a lot of noise about that now. In
the fullness of time (one of my favorite phrases) we'll see where it
goes w/r/t all the tooling around these constructs.
Right, start small and iterate. Even our "start small" plan is more
ambitious than your average project
Thanks for pulling this all together and driving the discussion!
No problem, I'm glad we're making some progress
Cheers,
Mark.