On Mon, Feb 13, 2012 at 04:20:46PM -0500, Tzu-Mainn Chen wrote: I definitely don't think we should *remove* any functionality; you can use realms to lock down where an instance can be launched and we should definitely preserve that. What I'm thinking is that, if I have an image that's built and pushed to EC2 and Rackspace and I go to launch it, we should do something more intelligent than just push it to the first one on our list. +1 billion (I'm being conservative) We have new users, new developers, and people who want to kick the tires... And all of them are going to want some sort of documentation. Assuming that new users are going to intuitively grasp what realms do or how to build and push an image is kind of silly, I think. I've tried to drops lots of links in the wiki pages I've put together so we can try to build some better documentation, but it's grossly inadequate right now. Yes, this would be great, too. I know Andy Fitzsimon has talked about integrating help into a new layout as well. Though on that note, "Work with the Red Hat UX team to make our app more usable" might be worthwhile, too. I have to admit that, even as someone who works full-time on developing this app, I sometimes find myself getting lost in our app, or I'll go to launch an instance without remembering to set up a provider, and then I have to jump to a totally separate part of the app. -- Matt

On 13/02/12 16:20 -0500, Tzu-Mainn Chen wrote: +1 to both of these ideas. I feel like I should throw out that many of the things mentioned in this thread are likely multi-sprint features, and unlikely to all be able to even be started in the next one, which means we are really building out a feature backlog here to some extent (and we should keep in mind we already have a huge list sitting in redmine) -j

On 13/02/12 19:33 -0500, Wesley Hayutin wrote: I have been thinking about something (I think) along similar lines. I have some beginnings of this in conductor/util (build-all and a git pull script that is not yet pushed), and martyn had a 'get all aeolus repos' script in his own repo, maybe he can respond here with the link again, so we have it together. My thought was to split out util into its own repo, and have it contain developer tooling, possibly in the form of a gem and optional rpm (that part highly tbd). A developer wanting to work on the project would clone that single repo. They could then call the developer tool (working name is 'ad' to keep it short, nemonic for 'aeolus developer') with something like: $ ad fetch This could then present the user with a list of aeolus repos (preferably with short description, retrieved from github description) and allow them to either get 'all', or check off/type numbers of the repos they want to have a copy of. As they develop, fetch moves to being used to keep their local copies of everything up to date (via the flags shown below). My initial idea is to keep the api for this tool similar to git and aeolus, so there will be an undetermined number of commands, and running ad [help,-h,--help] will return the list of commands. Then ad <command> [help,-h,--help] will return help for that specific command. An example of what fetch would tag for options is: $ ad fetch [all|list] [repolist] [tag] I suspect there are a number of things that could wind up in this repo, but the idea is to keep them all focused on aiding development. I think initial revs would focus on just 'fetch' and build' commands, then we can expand it over time. == Idea 2 == We have been using rake tasks of 'rpms' and 'yum' for configure, -cli, and -image. This code is currently duplicated in each project, which drives me insane. I'd like to get this extracted _somewhere_, whether it be some upstream project (which I think Mo mentioned a while back), or just a gem/rpm to make the task available system-wide. == Idea 3 == Francesco and I have discussed adding a peer-level project/user on github 'aeolus-incubator'. This would be a home for any projects to help around the aeolus ecosystem. It may be extra tooling, pluggable policy library, alternate expermiments for project management or bz integration. Basically anything that may be useful at some point, but either is not close to ready yet, or we are not sure if it will even be used. Similar to apache, once a project is mature enough and has community backing, it can be promoted to aeolusproject proper (and of course, this means it gets picked up in the ad tool, which may also get a flag to say 'get me incubator projects'). -j

On 14/02/12 10:36 -0500, Matt Wagner wrote: Good stuff. Funny enough, I used the same namespace for my (not yet sent) bundler/platform dev patches. I will note that these are kind of conductor specific, as are mine, so they may need to just end up there. -j

On 02/14/2012 04:21 PM, Jason Guiditta wrote: I am not 100% sure if this is relevant thing but anyway... I have bumped into Blueprint: "a much more thorough way to manage dependencies than Bundler-and-a-shell-script. It deals not only with RubyGems but with APT and Yum, so the entire dependency chain of your app can be automatically encapsulated in one place" which might be useful here. (I might be completely wrong though...) Anyway I think its definitelly worth checking out: https://github.com/devstructure/blueprint http://rcrowley.org/talks/rubyconf-2011/#1 Jirka

On 02/16/2012 11:53 AM, Wesley Hayutin wrote: And, related to this -- we need to merge the separate api controllers (for images, etc) back into the main controllers. We'll never have API parity if we have to duplicate the controller actions (possibly inconsistently) across the board. Scott

On 14/02/2012, at 8:06 AM, Matt Wagner wrote: c) Backup and restore scripting (or equivalent utils) We seem to have missed making this a core item. :( So, lets address it for the next sprint? :) + Justin -- Aeolus Community Manager http://www.aeolusproject.org

On 15/02/2012, at 1:25 AM, Matt Wagner wrote: Not the instances, as they'd be running on the various Clouds, but definitely all of the Aeolus controlled data. So, everything that's in Conductor, but also Image Factory, Image Warehouse, and anything else. Imagine an Aeolus server has a hard drive failure and needs to be recovered from the ground up. So, once the software's installed... everything else needs to be restorable from backup. + Justin -- Aeolus Community Manager http://www.aeolusproject.org

On 02/14/2012 01:36 PM, Mo Morsi wrote: Sorry forgot the link -Mo [1] https://github.com/movitto/snap

On Tue, Feb 14, 2012 at 02:18:11PM +0000, Martyn Taylor wrote: I think this is a good idea, both architecturally and as a way to get us rolling. Then we can potentially start the ball rolling with a few simple policies and expand from there. -- Matt

On 14/02/12 10:23 -0500, Tzu-Mainn Chen wrote: Agree, with the tweak that features and patched (or at least link to patch) go in redmine. We only track red hat bugs in BZ, upstream uses our redmine instance. Sending feature to list for review is something we have attempted with marginal success in the past, but I still like the idea, if only for visibility into what people are trying to add for a new feature. We should treat it as a patch review - if you send a feature to list and get no feedback, pester the heck out of the people you think should be replying until they do so. -j

On Tue, 14 Feb 2012 10:46:24 -0500, Jason Guiditta <jguiditt(a)redhat.com&gt; wrote: Yes, this. I'm curious, given the "attempt (again)" bit, what has been done in the past to try and correct this (and ultimately failed)?

On Tue, Feb 14, 2012 at 11:30:08AM -0500, Jason Guiditta wrote: As an avid *reader* of the patch flow and the review discussions around patches, I would only ask that the team preserve that aspect of the current setup -- meaning, please don't take away my ability to track everything that's going on with Aeolus code simply by reading the mailing list. Beyond that, as far as I'm concerned it's up to the team to decide. I tend to think that Github pull requests obscure the development discussion more than I would like by pulling the discussion off mail and onto web pages, but again that's just my opinion. Thanks, --Hugh -- == Hugh Brock, hbrock(a)redhat.com == == Engineering Manager, Cloud BU == == Aeolus Project: Manage virtual infrastructure across clouds. == == http://aeolusproject.org == "I know that you believe you understand what you think I said, but I’m not sure you realize that what you heard is not what I meant." --Robert McCloskey

On Tue, Feb 14, 2012 at 11:44:54AM -0500, Scott Seago wrote: Seems easily fixable by something like Trapaas or one of the other solutions. (/me suppresses urge to explain how easy it is to do above with mutt...) :) --Hugh -- == Hugh Brock, hbrock(a)redhat.com == == Engineering Manager, Cloud BU == == Aeolus Project: Manage virtual infrastructure across clouds. == == http://aeolusproject.org == "I know that you believe you understand what you think I said, but I’m not sure you realize that what you heard is not what I meant." --Robert McCloskey

On 02/14/2012 07:20 PM, Matt Wagner wrote: +1 for any of above solution - as Scott said, keeping track of what patches are pending is very difficult (almost impossible w/ current traffic). I was missing it especially now in BZ sprints. Last I heard, Trapaas was ready to use, also machine where it should run was ready, I wonder what happened then? Jan

On 02/14/2012 10:46 AM, Jason Guiditta wrote: IMO this should be one of the top tasks going forward. For various reasons including growing our user base, demonstrate the robustness of the app to our customers, and deploying a live public instance, we need to work on security. I'll look into starting breaking down the tasks on the wiki some more and analysing the app and exactly what needs TBD on this front and how long it will take. -Mo

On 02/13/2012 10:06 PM, Matt Wagner wrote: Thanks for getting the ball rolling, Matt. With all that's been happening today, I didn't get to actually reading all your suggestions, sorry. I shall remedy that tomorrow and add feedback if I have any. In a meeting with Angus here in Brno, two ideas popped up: 1) Display the list of importable instances. Instead of hunting in each provider's UI for the IDs of the images to import, the user would be able to see all the instances they could import, select them and import right from Conductor. There are things to consider here (mainly performance), but Deltacloud allows image listing and this would improve the import-image usability. 2) Option to import instances. This is a bit tricky and it would probably require some manual work. I think it's important to have, though. We would show the user (with appropriate privileges) all the instances that are running in a given provider that Conductor does not keep track of. They could then select a pool, assign owners and anything else that's necessary for Conductor to bring those in. Since these did not originate from Conductor, it's possible that some metadata will be missing -- but the same is true for imported images. And the benefit of using one tool to see the status of all your instances and being able to at least stop/restart them is huge. If we can include an automatic importing with reasonable defaults (one click imports all instances into the default pool under the Admin user), this will become immediately usable to people who have some things running and just want to try Conductor by themselves. 3) Embrace cross-cloud migration, adopt Mo's Snap! tool, integrate it within our existing tools and make it even more awesome. Thomas

On 02/17/2012 06:34 PM, Jason Guiditta wrote: Speaking of i18n files (I agree with both points you raised btw), they deserve a general cleanup anyway. There's at least a bunch of entries that we no longer use and others are horribly duplicated. Sorting this out would go along nicely with the reorganization effort since we'd be touching the entire structure anyway.

On 02/13/2012 10:06 PM, Matt Wagner wrote: Hooks support: there is now a basic Hook model, which should be used mainly for Pacemaker (according to https://www.aeolusproject.org/redmine/projects/aeolus/wiki/Pacemaker_Clou...) - it's not used ATM. It would be nice to give users a way how to hook their own actions to various events - and I don't mean only instance state changes, but also models: for example when a user creates new template, image build is finished, pool or other quota is reached, new user is added,... This is related to another idea - general plugin system. We can say "this is rails app, you can use rails plugins". Or we can explicitly define hook points - with this we can offer backward compatibility, so there is lesser chance that user plugins break w/ new conductor version. Jan

On 02/13/2012 04:06 PM, Matt Wagner wrote: What about additional reporting and monitoring? (didn't see this in the thread yet) Specifically, I'm thinking about how Audrey can provide configuration status of individual VMs. But, I think Conductor would have to grab that data and surface that data to the user. We're currently trying to record the data points we believe that Audrey can capture: https://www.aeolusproject.org/redmine/projects/audrey/wiki/Audrey_Reporting But, I'm sure there are tons of other data points to present. Are there rails reporting frameworks that anyone has worked with in the past? Have monitoring and reporting been discussed as features for Conductor?