Some notes on initial integration of Aeolus and Matahari
by Perry Myers
cross-posting to Aeolus and Matahari upstream projects...
We discussed sort of a pilot project for integration of Matahari into
Aeolus usage. The driver for this is that we need a way for folks
upstream to be able to deploy a single node Aeolus environment managing
1 or more Fedora hosts using kvm/libvirt (or VMware)
The problem is that we don't have a way in this environment to
bootstrap, specifically for Aeolus to get the ip address of the guests
as they are booting on the libvirt/kvm/VMware hosts.
I thought that this would be a good simple first use case for running
Matahari in Aeolus managed guests. So here's what I propose:
* Aeolus uses Image Factory to build images... When those images are
either Fedora, RHEL or Windows IF will include Matahari and puppet in
the core images
* Aeolus will inject basic bootstrapping info for Matahari like broker
ip address and broker auth info (this ignores the more complex post
boot config process Carl has for the time being, this process is
quick and dirty just to get stuff working and then we can expand to
use Carl's process as a second step)
* This injection could happen in a few ways:
- During image build
- During image deployment (using guestfish to inject a config file)
- Other? Could rely on DNS SRV records w/ no broker auth (again this
is meant for developer environments, not production usage)
* When an instance is booted, matahari comes up uses the injected config
and connects to the broker and a "hey I'm a new guest on the bus"
event should be sent out which includes the ip address of the guest
* Simple QMF console written that runs as part of the Conductor
infrastructure that monitors the bus for new 'I'm here' events and
when it sees one it grabs the ip address and stores it in the database
* It then calls a processConfig method (on which Agent?) which gives a
puppet file to a guest agent to process via a call to puppet tool.
Return from that method should be success or failure (with logging
info probably) This means puppet needs to be baked into the image as
well along with matahari.
TODOs:
Matahari Team:
* Write host agent event for 'i'm here configure me, here's my ip
address'
* Write method (which agent?) for processConfig(puppet-conf)
Aeolus Team:
* Need to write a simple QMF Console app that is part of the Conductor
that registers for 'new guest' events and when it sees these reports
that to Conductor and calls the processConfig method with the right
puppet script
* Run a broker alongside the Conductor and other Aeolus core
infrastructure. The ip address of the host running this is what
needs to be pushed into images for bootstrapping
The end goal here should be making it EASY for someone to grab Aeolus
and set it up quickly to manage local resources (i.e. we don't want to
_require_ people playing with Aeolus to need an EC2 account)
Andrew/Adam: Random question, what links all of the top level objects on
a single host together? Is there a common/shared UUID exposed by each
Agent? If a guest comes up and the Host agent appears, I can query that
Host object for UUID. But how do I know which Network Agent is the one
on the same guest as the Host Agent with the UUID that I just got?
Perry
13 years
[PATCH] Initial php for validating and emailing the web form.
by Justin Clift
This is PHP for validating the new Aeolus website's contact page
submission form (name, email, message), then emailing it to me
if it's ok.
Does anyone have the time/skills to look it over, to make sure
it's safe?
+ Justin
---
aeolusproject.org/content/process_form.php | 66 ++++++++++++++++++++++++++++
1 files changed, 66 insertions(+), 0 deletions(-)
create mode 100644 aeolusproject.org/content/process_form.php
diff --git a/aeolusproject.org/content/process_form.php b/aeolusproject.org/content/process_form.php
new file mode 100644
index 0000000..6811478
--- /dev/null
+++ b/aeolusproject.org/content/process_form.php
@@ -0,0 +1,66 @@
+<?
+// Include the mail validation and sending functions
+require_once '/usr/share/pear/Mail.php';
+require_once '/usr/share/pear/Mail/RFC822.php';
+
+// Static fields
+define(EMAIL_FROM, 'Aeolus Website <jclift(a)redhat.com>');
+define(EMAIL_TO, 'Justin Clift <jclift(a)redhat.com>');
+define(SUBJECT, 'Aeolus web form submission');
+define(MAIL_HOST, 'localhost');
+
+// Force UTF-8 locale, just to be safe
+setlocale(LC_CTYPE, 'en_US.UTF-8');
+
+// Check for missing input
+$missing_input = FALSE;
+if (empty($_POST['name'])) {
+ echo 'You forgot to fill out the Name field.' . '<br />';
+ $missing_input = TRUE;
+}
+if (empty($_POST['email'])) {
+ echo 'You forgot to fill out the Email field.' . '<br />';
+ $missing_input = TRUE;
+}
+if (empty($_POST['message'])) {
+ echo 'You forgot to fill out the Message field.' . '<br />';
+ $missing_input = TRUE;
+}
+if (TRUE == $missing_input) {
+ exit(1);
+}
+
+// Validate the email address against RFC822 spec
+$rfc822_check = Mail_RFC822::parseAddressList($_POST['email'], 'example.org', FALSE, TRUE);
+if (PEAR::isError($rfc822_check)) {
+ echo 'The email address you gave does not appear to be valid' . '<br /><br />';
+ echo '<b>Reason given:</b> ' . $rfc822_check->getMessage();
+ exit(2);
+}
+
+// Escape the web form input for safety
+$form_name = escapeshellarg($_POST['name']);
+$form_email = escapeshellarg($_POST['email']);
+$form_message = escapeshellarg($_POST['message']);
+
+// Construct the message body
+$body = 'A new submission has arrived through the Aeolus contact web form' . "\n\n";
+$body .= 'Name: ' . $form_name . "\n";
+$body .= 'Email: ' . $form_email . "\n\n";
+$body .= 'Message' . "\n" . '*******' . "\n\n" . $form_message . "\n\n";
+
+// Construct the mail message proper
+$headers = array ('From' => EMAIL_FROM, 'To' => EMAIL_TO, 'Subject' => SUBJECT);
+$smtp = Mail::factory('smtp', array ('host' => MAIL_HOST, 'auth' => FALSE));
+
+// Send the mail
+$mail = $smtp->send(EMAIL_TO, $headers, $body);
+
+// Check for errors
+if (PEAR::isError($mail)) {
+ echo('An error occurred when sending the message: ' . $mail->getMessage());
+} else {
+ echo('Message successfully sent!');
+}
+
+?>
--
1.7.4.3
13 years, 1 month
[PATCH aeolus] Fix attribute keypair in condormatic
by jzigmund＠redhat.com
From: Jozef Zigmund <jzigmund(a)redhat.com>
---
src/app/util/condormatic.rb | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/src/app/util/condormatic.rb b/src/app/util/condormatic.rb
index 6a28657..572f413 100644
--- a/src/app/util/condormatic.rb
+++ b/src/app/util/condormatic.rb
@@ -273,7 +273,7 @@ def condormatic_classads_sync
pipe.puts "username=\"#{account.credentials_hash['username']}\""
pipe.puts "password=\"#{account.credentials_hash['password']}\""
pipe.puts "provider_account_id=\"#{account.id}\""
- pipe.puts "keypair=\"#{account.instance_key.name}\""
+ pipe.puts "keypair=\"#{account.instance_key ? account.instance_key.name : ''}\""
rescue Exception => ex
Rails.logger.error "Error writing provider classad to condor."
Rails.logger.error ex.message
--
1.7.4
13 years, 1 month
Import image and start an instance on RHEV-M
by jzigmund＠redhat.com
Import image:
- It works through UI fine (Image factory->Image Imports)
Starting instance:
- Conductor is possible to run instance, but it has some issues:
1.) Bug was in condormatic (attribute keypair doesn't put in
classads, because it's instance_key is nil in current version of next).
So in "condor_q -better" you can see: "Request is held. Hold reason:
Cannot expand $$(keypair)." After fixing this issue condor matches
request and try to start instance.
2.) Name of instance - Condor's GlobalJobId is too long for name,
condor sent too long name to DC API.
3.) Hardware profile - memory - Using wrong units/value, sent value
is in MB but it's needed to be in bytes.
4.) After instance creation, start operation is performed and
passed to DC API, but Condor report "Hold reason: Action_Failure for
instance 5e7df1f8-a2b6-4d5d-92d4-2748e293ee68: (null)"
Result:
- Instance started and after then it's running, but in UI you can see
Status => error, because of point 4.
Fix for 1.) was sent to aeolus-devel mailing list.
--
Jozef Zigmund
13 years, 1 month
[PATCH imagefactory] Add a spec file to the imagefactory top-level.
by Chris Lalancette
Unfortunately, using the "bdist_rpm" target for setup.py
isn't really expressive enough to generate all of the things
we might need to do with the spec file for imagefactory.
As one example, we might need to add additional dependencies
to the spec beyond the ones that bdist_rpm detected. It would
also be nice to have a package description and a changelog.
To fix all of that, maintain our own spec at the top of
the repository hierarchy. That way we have full control
over it.
Signed-off-by: Chris Lalancette <clalance(a)redhat.com>
---
imagefactory.spec.in | 45 +++++++++++++++++++++++++++++++++++++++++++++
setup.py | 14 +++++++++++++-
2 files changed, 58 insertions(+), 1 deletions(-)
create mode 100644 imagefactory.spec.in
diff --git a/imagefactory.spec.in b/imagefactory.spec.in
new file mode 100644
index 0000000..f929183
--- /dev/null
+++ b/imagefactory.spec.in
@@ -0,0 +1,45 @@
+Summary: Image Factory system image generation tool
+Name: imagefactory
+Version: @VERSION@
+Release: 1%{?dist}
+Source0: %{name}-%{version}.tar.gz
+License: GPLv2
+Group: Development/Libraries
+BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-buildroot
+Url: http://www.aeolusproject.org/imagefactory.html
+BuildArch: noarch
+Requires: oz
+Requires: euca2ools
+Requires: python-qmf
+Requires: python-pycurl
+Requires: python-libguestfs
+Requires: python-zope-interface
+Requires: python-boto
+Requires: libxml2-python
+Requires: python-httplib2
+Requires: python-argparse
+BuildRequires: python
+
+%description
+imagefactory is a daemon that listens for build requests via QMF. When
+a build request is received, imagefactory will build the requested image
+for the requested target.
+
+%prep
+%setup -n %{name}-%{version}
+
+%build
+python setup.py build
+
+%install
+python setup.py install -O1 --root=$RPM_BUILD_ROOT --record=INSTALLED_FILES
+
+%clean
+rm -rf $RPM_BUILD_ROOT
+
+%files -f INSTALLED_FILES
+%defattr(-,root,root)
+
+%changelog
+* Mon Apr 04 2011 Chris Lalancette <clalance(a)redhat.com> - 0.1.6-1
+- Initial spec file.
diff --git a/setup.py b/setup.py
index 9e68f48..a6fcdb7 100644
--- a/setup.py
+++ b/setup.py
@@ -1,4 +1,5 @@
from distutils.core import setup, Extension
+from distutils.command.sdist import sdist as _sdist
import os
import os.path
import subprocess
@@ -6,7 +7,7 @@ import subprocess
version_file_name = "version.txt"
try:
if(not os.path.exists(version_file_name)):
- subprocess.call('/usr/bin/git describe > %s' % (version_file_name, ), shell=True)
+ subprocess.call('/usr/bin/git describe | tr - _ > %s' % (version_file_name, ), shell=True)
version_file = open(version_file_name, "r")
VERSION = version_file.read()[0:-1]
version_file.close()
@@ -15,6 +16,16 @@ except Exception, e:
datafiles=[('/etc', ['imagefactory.conf']), ('/etc/pki/imagefactory', ['cert-ec2.pem']), ('/etc/rc.d/init.d', ['imagefactory/scripts/imagefactory']), ('', ['version.txt'])]
+class sdist(_sdist):
+ """ custom sdist command to prepare imagefactory.spec file """
+
+ def run(self):
+ cmd = (""" sed -e "s/@VERSION@/%s/g" < imagefactory.spec.in """ %
+ VERSION) + " > imagefactory.spec"
+ os.system(cmd)
+
+ _sdist.run(self)
+
setup(name='imagefactory',
version=VERSION,
description='Image Factory system image generation tool',
@@ -25,4 +36,5 @@ setup(name='imagefactory',
packages=['imagefactory', 'imagefactory.builders', 'imagefactory.qmfagent'],
scripts=['imgfac.py'],
data_files = datafiles,
+ cmdclass = {'sdist': sdist}
)
--
1.7.4
13 years, 1 month
[PATCH conductor] Fix BZ #684185: Pool Family search doesn't work
by Tomas Sedovic
From: Tomas Sedovic <tsedovic(a)redhat.com>
https://bugzilla.redhat.com/show_bug.cgi?id=684185
Added search support to Pool Family page.
---
.../controllers/admin/pool_families_controller.rb | 20 ++++++++++++--
src/app/models/pool_family.rb | 5 +++
src/features/pool_family.feature | 28 +++++++++++++++++++-
3 files changed, 49 insertions(+), 4 deletions(-)
diff --git a/src/app/controllers/admin/pool_families_controller.rb b/src/app/controllers/admin/pool_families_controller.rb
index 29cbf59..b9069f6 100644
--- a/src/app/controllers/admin/pool_families_controller.rb
+++ b/src/app/controllers/admin/pool_families_controller.rb
@@ -1,8 +1,19 @@
class Admin::PoolFamiliesController < ApplicationController
before_filter :require_user
- before_filter :load_pool_families, :only =>[:index,:show]
+ before_filter :set_params_and_header, :only => [:index, :show]
+ before_filter :load_pool_families, :only =>[:show]
def index
+ @search_term = params[:q]
+ if @search_term.blank?
+ load_pool_families
+ return
+ end
+
+ search = PoolFamily.search() do
+ keywords(params[:q])
+ end
+ @pool_families = search.results
end
def new
@@ -73,15 +84,18 @@ class Admin::PoolFamiliesController < ApplicationController
protected
- def load_pool_families
+ def set_params_and_header
+ @url_params = params.clone
@header = [{ :name => "Name", :sort_attr => :name},
{ :name => "Quota limit", :sort_attr => :name},
{ :name => "Quota currently in use", :sort_attr => :name},
]
+ end
+
+ def load_pool_families
@pool_families = PoolFamily.paginate(:all,
:page => params[:page] || 1,
:order => ( params[:order_field] || 'name' ) + ' ' + (params[:order_dir] || 'asc')
)
- @url_params = params.clone
end
end
diff --git a/src/app/models/pool_family.rb b/src/app/models/pool_family.rb
index 7b8a679..2eee939 100644
--- a/src/app/models/pool_family.rb
+++ b/src/app/models/pool_family.rb
@@ -32,9 +32,14 @@
# Filters added to this controller apply to all controllers in the application.
# Likewise, all the methods added will be available for all controllers.
+require 'sunspot_rails'
class PoolFamily < ActiveRecord::Base
include PermissionedObject
DEFAULT_POOL_FAMILY_KEY = "default_pool_family"
+ searchable do
+ text :name, :as => :code_substring
+ text :description, :as => :code_substring
+ end
has_many :pools, :dependent => :destroy
has_and_belongs_to_many :provider_accounts
diff --git a/src/features/pool_family.feature b/src/features/pool_family.feature
index 2c0d753..6795d42 100644
--- a/src/features/pool_family.feature
+++ b/src/features/pool_family.feature
@@ -51,4 +51,30 @@ Feature: Pool Families
When I check "default" pool family
And I press "Delete"
Then I should see "Could not delete the following Pool Families: default."
- And I should see "default"
\ No newline at end of file
+ And I should see "default"
+
+ Scenario: Search for pool families
+ Given there are these pool families:
+ | name |
+ | first_family |
+ | second_family |
+ | third_family |
+ Given I am on the admin pool families page
+ Then I should see "first_family"
+ And I should see "second_family"
+ And I should see "third_family"
+ When I fill in "q" with "second"
+ And I press "Search"
+ Then I should see "second_family"
+ And I should not see "first_family"
+ And I should not see "third_family"
+ When I fill in "q" with "nomatch"
+ And I press "Search"
+ Then I should not see "first_family"
+ And I should not see "second_family"
+ And I should not see "third_family"
+ When I fill in "q" with ""
+ And I press "Search"
+ Then I should see "first_family"
+ And I should see "second_family"
+ And I should see "third_family"
--
1.7.4
13 years, 1 month
Need suggestions for an Aeolus website tagline
by Justin Clift
Hi all,
Does anyone have suggestions for a really good
tagline for the new Aeolus site?
Looking for a tagline that's:
+ A concise statement of Aeolus's purpose
+ Clear and informative
+ 6-8 words (general rule of thumb)
Placement of it on the website is in the top header,
where the initial design sent through yesterday says
"tag line goes here".
The things I keep thinking of are along the lines of:
"Cloud Management Software <???> <???> <???>"
But nothing so far feels really good. Too plain
maybe.
Any ideas? :)
Regards and best wishes,
Justin Clift
13 years, 1 month
[PATCH aeolus] BZ#690436 Fixed HardwareProfile sorting
by Martyn Taylor
From: Martyn Taylor <mtaylor(a)redhat.com>
---
.../admin/hardware_profiles_controller.rb | 8 +++++++-
src/app/models/hardware_profile.rb | 4 ++++
src/app/models/hardware_profile_property.rb | 13 +++++++++++++
3 files changed, 24 insertions(+), 1 deletions(-)
diff --git a/src/app/controllers/admin/hardware_profiles_controller.rb b/src/app/controllers/admin/hardware_profiles_controller.rb
index 783c0e8..a3ff77a 100644
--- a/src/app/controllers/admin/hardware_profiles_controller.rb
+++ b/src/app/controllers/admin/hardware_profiles_controller.rb
@@ -178,7 +178,13 @@ class Admin::HardwareProfilesController < ApplicationController
end
def load_hardware_profiles
- @hardware_profiles = HardwareProfile.all(:conditions => 'provider_id IS NULL')
+ sort_order = params['order_field'].nil? ? "name" : params['order_field']
+ if sort_order == "name"
+ @hardware_profiles = HardwareProfile.all(:order => sort_order, :conditions => 'provider_id IS NULL')
+ else
+ @hardware_profiles = HardwareProfile.all(:conditions => 'provider_id IS NULL')
+ @hardware_profiles.sort! {|x,y| x.get_property_map[sort_order].sort_value(true) <=> y.get_property_map[sort_order].sort_value(true)}
+ end
end
def load_hardware_profile
diff --git a/src/app/models/hardware_profile.rb b/src/app/models/hardware_profile.rb
index 9cfc770..836717e 100644
--- a/src/app/models/hardware_profile.rb
+++ b/src/app/models/hardware_profile.rb
@@ -83,6 +83,10 @@ class HardwareProfile < ActiveRecord::Base
validates_associated :cpu
validates_associated :architecture
+ def get_property_map
+ return {'memory' => memory, 'cpu' => cpu, 'architecture' => architecture, 'storage' => storage}
+ end
+
def provider_hardware_profile?
!provider.nil?
end
diff --git a/src/app/models/hardware_profile_property.rb b/src/app/models/hardware_profile_property.rb
index f38d394..41ad7aa 100644
--- a/src/app/models/hardware_profile_property.rb
+++ b/src/app/models/hardware_profile_property.rb
@@ -127,5 +127,18 @@ class HardwareProfileProperty < ActiveRecord::Base
"undefined"
end
end
+
+ def sort_value(ascending)
+ case kind
+ when "fixed"
+ return value
+ when range
+ return ascending ? range_from : range_to
+ when
+ entries = (property_enum_entries.map { |enum| enum.value }).sort!
+ return ascending ? entries.first : entries.last
+ end
+ end
+
end
--
1.7.4
13 years, 1 month
[PATCH] Make the conductor RPM depend on imagefactory.
by Chris Lalancette
Signed-off-by: Chris Lalancette <clalance(a)redhat.com>
---
aeolus-conductor.spec.in | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/aeolus-conductor.spec.in b/aeolus-conductor.spec.in
index f1b71b9..757dae3 100644
--- a/aeolus-conductor.spec.in
+++ b/aeolus-conductor.spec.in
@@ -48,6 +48,7 @@ Requires: rest >= 0.6
Requires: condor >= 7.5.6-2dcloud
Requires: iwhd
Requires: aeolus-configure
+Requires: imagefactory
# to ensure the service is actually started
# and is accessible in the init script
--
1.7.4
13 years, 1 month