FESCo Ticket #1427 https://fedorahosted.org/fesco/ticket/1427 formalizes the list of deliverables for each Fedora release. Cool.
Let's go a step beyond this and make a machine-readable list, at least of image-based deliverables. One approach would be to use using Ubuntu's simplestreams format, documented at http://bazaar.launchpad.net/~smoser/simplestreams/trunk/view/head:/doc/README.
A second option would be the index.asc as used by virt-builder: https://fedorahosted.org/rel-eng/ticket/5805
That way, we'd have a standardized way for applications to find and download/launch/whatever various Fedora images. This is a definite need (see https://fedorahosted.org/cloud/ticket/93) and rather than inventing a new wheel, maybe we could see if we could pound one of these into shape.
----- Original Message -----
FESCo Ticket #1427 https://fedorahosted.org/fesco/ticket/1427 formalizes the list of deliverables for each Fedora release. Cool.
Let's go a step beyond this and make a machine-readable list, at least of image-based deliverables. One approach would be to use using Ubuntu's simplestreams format, documented at http://bazaar.launchpad.net/~smoser/simplestreams/trunk/view/head:/doc/README.
Some pros/cons on this: + It has the concept of a stream (vendor, product, version) + It supports different hash types + It can be signed + Timestamp within the file + Can represent multiple versions of the same stream
- ftype is not very helpful (no precise format descirption) - size is very unspecific (compressed/uncompressed?) (gb vs mb vs mib) - The file slook quite complex
A second option would be the index.asc as used by virt-builder: https://fedorahosted.org/rel-eng/ticket/5805
Some pros/cons I see here: + I tincludes both, compressed + uncompressed, disk sizes + Tells the format of the image + I personally like the structure much more
+/- It includes some non-image stuff (i.e. expand=)
- The type of the checksum is not specified in the conf file - I'm missing a stream like scheme; a gloabl identifier to be able to sort images according to a version/build-date
That way, we'd have a standardized way for applications to find and download/launch/whatever various Fedora images. This is a definite need (see https://fedorahosted.org/cloud/ticket/93) and rather than inventing a new wheel, maybe we could see if we could pound one of these into shape.
After all IMO simplestreams gives us what we need.
- fabian
On Tue, May 19, 2015 at 04:38:21AM -0400, Fabian Deutsch wrote:
A second option would be the index.asc as used by virt-builder: https://fedorahosted.org/rel-eng/ticket/5805
Some pros/cons I see here:
- I tincludes both, compressed + uncompressed, disk sizes
- Tells the format of the image
- I personally like the structure much more
+/- It includes some non-image stuff (i.e. expand=)
'expand' is really the root filesystem. It's essential if you want to automatically resize the filesystem *offline* (ie. without using things like cloud-init). However you can omit it from the format and virt-builder just won't resize the root fs.
- The type of the checksum is not specified in the conf file
Actually it is:
http://libguestfs.org/virt-builder.1.html#creating-and-signing-the-index-fil...
It just defaults to sha512 unless specified otherwise.
- I'm missing a stream like scheme; a gloabl identifier to be able to sort images according to a version/build-date
You can add extra fields, they will be ignored by virt-builder.
Rich.
On Mon, 18.05.15 19:44, Matthew Miller (mattdm@fedoraproject.org) wrote:
FESCo Ticket #1427 https://fedorahosted.org/fesco/ticket/1427 formalizes the list of deliverables for each Fedora release. Cool.
Let's go a step beyond this and make a machine-readable list, at least of image-based deliverables. One approach would be to use using Ubuntu's simplestreams format, documented at http://bazaar.launchpad.net/~smoser/simplestreams/trunk/view/head:/doc/README.
A second option would be the index.asc as used by virt-builder: https://fedorahosted.org/rel-eng/ticket/5805
That way, we'd have a standardized way for applications to find and download/launch/whatever various Fedora images. This is a definite need (see https://fedorahosted.org/cloud/ticket/93) and rather than inventing a new wheel, maybe we could see if we could pound one of these into shape.
There's also the ACI discovery stuff:
https://github.com/appc/spec/blob/master/SPEC.md#app-container-image-discove...
Doesn't really apply to non-ACI images right now, but the concepts in it are a bit nicer I think then the alternatives (i.e. the fact that it renders the image names into URL via a fixed template, and that it uses <meta> tags for exposing this information is pretty nice).
Lennart
----- Original Message -----
On Mon, 18.05.15 19:44, Matthew Miller (mattdm@fedoraproject.org) wrote:
FESCo Ticket #1427 https://fedorahosted.org/fesco/ticket/1427 formalizes the list of deliverables for each Fedora release. Cool.
Let's go a step beyond this and make a machine-readable list, at least of image-based deliverables. One approach would be to use using Ubuntu's simplestreams format, documented at http://bazaar.launchpad.net/~smoser/simplestreams/trunk/view/head:/doc/README.
A second option would be the index.asc as used by virt-builder: https://fedorahosted.org/rel-eng/ticket/5805
That way, we'd have a standardized way for applications to find and download/launch/whatever various Fedora images. This is a definite need (see https://fedorahosted.org/cloud/ticket/93) and rather than inventing a new wheel, maybe we could see if we could pound one of these into shape.
There's also the ACI discovery stuff:
https://github.com/appc/spec/blob/master/SPEC.md#app-container-image-discove...
If you are just referring to this part of the spec …
Doesn't really apply to non-ACI images right now, but the concepts in it are a bit nicer I think then the alternatives (i.e. the fact that it renders the image names into URL via a fixed template, and that it uses <meta> tags for exposing this information is pretty nice).
… then I agree. The URL scheme based identifier is nice.
But it is lacking stuff like a description and the size of the image. And the method to retrieve metadata informations is somewhat cumbersome.
My 2ct.
- fabian
On 19/05/15 09:05 -0400, Fabian Deutsch wrote:
----- Original Message -----
On Mon, 18.05.15 19:44, Matthew Miller (mattdm@fedoraproject.org) wrote:
FESCo Ticket #1427 https://fedorahosted.org/fesco/ticket/1427 formalizes the list of deliverables for each Fedora release. Cool.
Let's go a step beyond this and make a machine-readable list, at least of image-based deliverables. One approach would be to use using Ubuntu's simplestreams format, documented at http://bazaar.launchpad.net/~smoser/simplestreams/trunk/view/head:/doc/README.
A second option would be the index.asc as used by virt-builder: https://fedorahosted.org/rel-eng/ticket/5805
That way, we'd have a standardized way for applications to find and download/launch/whatever various Fedora images. This is a definite need (see https://fedorahosted.org/cloud/ticket/93) and rather than inventing a new wheel, maybe we could see if we could pound one of these into shape.
There's also the ACI discovery stuff:
https://github.com/appc/spec/blob/master/SPEC.md#app-container-image-discove...
If you are just referring to this part of the spec …
Doesn't really apply to non-ACI images right now, but the concepts in it are a bit nicer I think then the alternatives (i.e. the fact that it renders the image names into URL via a fixed template, and that it uses <meta> tags for exposing this information is pretty nice).
… then I agree. The URL scheme based identifier is nice.
But it is lacking stuff like a description and the size of the image. And the method to retrieve metadata informations is somewhat cumbersome.
Description, maybe. But since it's just an HTML meta tag, then the page itself could be self serving of that information.
Size, it's just a flat file. An HTTP HEAD, an Content-Length header is typically sufficient.
As for the retrieval, I can't say it's entirely cumbersome. Parsing HTML is pretty straight forward. Serving this kind of information will eventually be usable as well, as Docker is looking to accommodate similar. https://github.com/docker/distribution/pull/303/files#diff-4846d2808cb787c14... Though in that case, the serving a flat ACI files is cleaner and nicer.
My 2ct.
- fabian
On 05/18/2015 07:44 PM, Matthew Miller wrote:
That way, we'd have a standardized way for applications to find and download/launch/whatever various Fedora images. This is a definite need (see https://fedorahosted.org/cloud/ticket/93) and rather than inventing a new wheel, maybe we could see if we could pound one of these into shape.
Thank you for that thought--I am so going to use it from now on:
Rather than reinventing a wheel, let's pound an existing one into a circular shape
This perfectly describes a situation I often find myself in. I didn't think of it this way, but there it is.
On Mon, May 18, 2015 at 07:44:37PM -0400, Matthew Miller wrote:
FESCo Ticket #1427 https://fedorahosted.org/fesco/ticket/1427 formalizes the list of deliverables for each Fedora release. Cool.
Let's go a step beyond this and make a machine-readable list, at least of image-based deliverables. One approach would be to use using Ubuntu's simplestreams format, documented at http://bazaar.launchpad.net/~smoser/simplestreams/trunk/view/head:/doc/README.
This could really do with an example. FWIW several can be found here:
https://cloud-images.ubuntu.com/releases/streams/v1/
A second option would be the index.asc as used by virt-builder: https://fedorahosted.org/rel-eng/ticket/5805
Example:
http://libguestfs.org/download/builder/index.asc
I agree it is long past time that we had a mechanical way to describe the various cloud and other images we are shipping.
Rich.
On 18/05/15 19:44 -0400, Matthew Miller wrote:
FESCo Ticket #1427 https://fedorahosted.org/fesco/ticket/1427 formalizes the list of deliverables for each Fedora release. Cool.
Let's go a step beyond this and make a machine-readable list, at least of image-based deliverables. One approach would be to use using Ubuntu's simplestreams format, documented at http://bazaar.launchpad.net/~smoser/simplestreams/trunk/view/head:/doc/README.
A second option would be the index.asc as used by virt-builder: https://fedorahosted.org/rel-eng/ticket/5805
That way, we'd have a standardized way for applications to find and download/launch/whatever various Fedora images. This is a definite need (see https://fedorahosted.org/cloud/ticket/93) and rather than inventing a new wheel, maybe we could see if we could pound one of these into shape.
Also, I'm wondering if this kind of image base deliveries could facilitate a more automated push to the docker-official[0] images (this is for `docker pull fedora` images). Presently this is an _overly_ manual process, which involves loosing git history, checking tarballs of rootfs into git, etc [1].
Most of this difficulty is imposed by the docker library team's process. Though surely they'll be amendable to opening the process to allow for pulling in an image produced by rel-eng (ideally automated per errata involved in the image's components).
Main concern from upstream is this automation still needs a bit of review, like what all is included by default (i.e. systemd) and that it still smoke tests.
Thoughts?
vb
[0] https://github.com/docker-library/official-images [1] https://github.com/fedora-cloud/docker-brew-fedora
-
Fabian Deutsch -
Lennart Poettering -
Matthew Miller -
Przemek Klosowski -
Richard W.M. Jones -
Vincent Batts