[PATCH conductor] Initial version of qmf console for v2 factory.
Directions for install and test run are now in the README. Note there are also some directions there for the wrapper service that I have not sent yet, as it is still rather static and needs cleanup.
Also, at the current time, I expect one test to fail, due to a bug in the agent. Once that bug is fixed, test should passed here. Likely this will be Scott doing testing, as he has pre-reqs installed already and I woudl rather not see others broken until we can flip the switch over to the new qmf/factory.
For testing, the error you should expect to see is somthing like:
Spec::Mocks::MockExpectationError in 'ImageFactoryConsole needs a real agent running#push_image should return a build-adaptor with uuid' #BaseHandler:0x7f05feaa8830 expected :handle_failed with (#<Qmf2::Data:0x7f05fea5eb68 @impl=#Cqmf2::Data:0x7f05fea5ebe0, @schema=#<Qmf2::Schema:0x7f05fea5dd08 @impl=#Cqmf2::Schema:0x7f05fea5dd80>>) 0 times, but received it once ./spec/image_factory/console_spec.rb:41:
-j
This is the interface to the factory agent, the service, (which is what conductor will communicate with) will be in an upcoming patch. --- services/image_factory/README | 44 ++++++ services/image_factory/console/Rakefile | 67 +++++++++ .../image_factory/console/lib/image_factory.rb | 2 + .../console/lib/image_factory/base_handler.rb | 56 ++++++++ .../lib/image_factory/image_factory_console.rb | 141 ++++++++++++++++++++ .../console/spec/image_factory/console_spec.rb | 92 +++++++++++++ services/image_factory/console/spec/spec_helper.rb | 3 + 7 files changed, 405 insertions(+), 0 deletions(-) create mode 100644 services/image_factory/README create mode 100644 services/image_factory/console/Rakefile create mode 100644 services/image_factory/console/lib/image_factory.rb create mode 100644 services/image_factory/console/lib/image_factory/base_handler.rb create mode 100644 services/image_factory/console/lib/image_factory/image_factory_console.rb create mode 100644 services/image_factory/console/spec/image_factory/console_spec.rb create mode 100644 services/image_factory/console/spec/spec_helper.rb
diff --git a/services/image_factory/README b/services/image_factory/README new file mode 100644 index 0000000..7c70d9b --- /dev/null +++ b/services/image_factory/README @@ -0,0 +1,44 @@ +First, you'll need: +* rebuild the rpms for new qpid/qmf: + http://koji.fedoraproject.org/koji/packageinfo?packageID=10015 +* build rpm for oz (http://www.aeolusproject.org/oz-download.html) +* You can run imageFactory from src, or build the rpm, source is here: + git://github.com/aeolusproject/image_factory + +I have been developing for now using the following packages for qpid: +qpid-cpp-server-0.8-6.fc14.x86_64 +python-qpid-qmf-0.8-6.fc14.x86_64 +ruby-qpid-qmf-0.8-6.fc14.x86_64 +qpid-tools-0.8-6.fc14.noarch +qpid-qmf-0.8-6.fc14.x86_64 +qpid-cpp-client-0.8-6.fc14.x86_64 +qpid-cpp-server-store-0.8-6.fc14.x86_64 + +Note that you will need to rpm -e --nodeps all older qpid stuff +if you dont want to uninstall the entire conductor stack. + +Make sure /etc/qpidd.conf has the line: +auth=no +and restart qpidd service + +In the console dir, +run tests with: +[console]$ rake spec + +Build the console gem and install: +[console]$ rake gem +[console]$ sudo gem install pkg/image_factory_console-0.2.0.gem + +You can start the factory (from src) with: +[factory-checkout]$ sudo ./imgfac.py --qmf --foreground + +In the image_factory_connector/bin dir, run the app in any of the following ways: +* [image_factory_connector/bin]$ ./image_factory_connector -p 2003 start +* [image_factory_connector/lib]$ thin -R config.ru -p 2003 start +* [image_factory_connector/lib]$ rackup -p 2003 config.ru + +View some simple interactions (defined in image_factory_connector) from browser with: +http://localhost:2003/ +http://localhost:2003/do-something +http://localhost:2003/shutdown + diff --git a/services/image_factory/console/Rakefile b/services/image_factory/console/Rakefile new file mode 100644 index 0000000..fa07cb2 --- /dev/null +++ b/services/image_factory/console/Rakefile @@ -0,0 +1,67 @@ +# +# Copyright (C) 2011 Red Hat, Inc. +# Written by Jason Guiditta jguiditt@redhat.com +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; version 2 of the License. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, +# MA 02110-1301, USA. A copy of the GNU General Public License is +# also available at http://www.gnu.org/copyleft/gpl.html. + + +require 'rubygems' +require 'rake' +require 'rake/clean' +require 'rake/gempackagetask' +require 'rake/rdoctask' +require 'rake/testtask' +require 'spec/rake/spectask' + +spec = Gem::Specification.new do |s| + s.name = 'image_factory_console' + s.version = '0.2.0' + s.has_rdoc = true + #s.extra_rdoc_files = ['README', 'COPYING'] + s.summary = 'QMF Console for Aeolus Image Factory' + s.description = s.summary + s.author = 'Jason Guiditta' + s.email = 'jguiditt@redhat.com' + # s.executables = ['your_executable_here'] + s.files = %w(Rakefile) + Dir.glob("{bin,lib,spec}/**/*") + s.require_path = "lib" + s.bindir = "bin" +end + +Rake::GemPackageTask.new(spec) do |p| + p.gem_spec = spec + p.need_tar = true + p.need_zip = true +end + +Rake::RDocTask.new do |rdoc| + files =['lib/**/*.rb'] #'README', 'COPYING', + rdoc.rdoc_files.add(files) + #rdoc.main = "README" # page to start on + rdoc.title = "console Docs" + rdoc.rdoc_dir = 'doc/rdoc' # rdoc output folder + rdoc.options << '--line-numbers' +end + +Rake::TestTask.new do |t| + t.test_files = FileList['test/**/*.rb'] +end + +Spec::Rake::SpecTask.new do |t| + t.libs << 'lib' + t.spec_files = FileList['spec/**/*.rb'] + t.spec_opts = ['--color', '--format nested'] +end diff --git a/services/image_factory/console/lib/image_factory.rb b/services/image_factory/console/lib/image_factory.rb new file mode 100644 index 0000000..af8f3f9 --- /dev/null +++ b/services/image_factory/console/lib/image_factory.rb @@ -0,0 +1,2 @@ +require 'image_factory/image_factory_console' +require 'image_factory/base_handler' \ No newline at end of file diff --git a/services/image_factory/console/lib/image_factory/base_handler.rb b/services/image_factory/console/lib/image_factory/base_handler.rb new file mode 100644 index 0000000..7dcead8 --- /dev/null +++ b/services/image_factory/console/lib/image_factory/base_handler.rb @@ -0,0 +1,56 @@ +# +# Copyright (C) 2011 Red Hat, Inc. +# Written by Jason Guiditta jguiditt@redhat.com +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; version 2 of the License. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, +# MA 02110-1301, USA. A copy of the GNU General Public License is +# also available at http://www.gnu.org/copyleft/gpl.html. + +require 'logger' + +class BaseHandler + + def initialize(logger=nil) + logger(logger) + end + + def handle(data) + logger.debug "====== Type of event: #{data.event}" + #puts "should be calling the logger now..." + if data.event == 'STATUS' + handle_status(data) + elsif data.event == 'FAILURE' + handle_failed(data) + end + end + + def handle_status(data) + logger.debug "{data.event}, #{data.new_status}" + end + + def handle_failed(data) + logger.error "#{data.to_s}" + end + + private + def logger(logger=nil) + @logger ||= logger + unless @logger + @logger = Logger.new(STDOUT) + @logger.level = Logger::ERROR + @logger.datetime_format = "%Y-%m-%d %H:%M:%S" + end + return @logger + end +end diff --git a/services/image_factory/console/lib/image_factory/image_factory_console.rb b/services/image_factory/console/lib/image_factory/image_factory_console.rb new file mode 100644 index 0000000..6f9e11b --- /dev/null +++ b/services/image_factory/console/lib/image_factory/image_factory_console.rb @@ -0,0 +1,141 @@ +# +# Copyright (C) 2011 Red Hat, Inc. +# Written by Jason Guiditta jguiditt@redhat.com +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; version 2 of the License. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, +# MA 02110-1301, USA. A copy of the GNU General Public License is +# also available at http://www.gnu.org/copyleft/gpl.html. + +# TODO: figure out what I am doing wrong here that I need this line +$: << File.expand_path(File.join(File.dirname(__FILE__), ".")) +require 'cqpid' +require 'qmf2' +require 'logger' +require 'base_handler' + +class ImageFactoryConsole < Qmf2::ConsoleHandler + + attr_accessor :q, :handler + + def initialize(args={}) +# @retry_limit = args.include?(:retry_limit) ? args[:retry_limit] : 20 +# @delay = args.include?(:delay) ? args[:delay] : 15 + host = args.include?(:host) ? args[:host] : "localhost" + port = args.include?(:port) ? args[:port] : 5672 +# url = "amqp://#{host}:#{port}" <- the amqp part here doesnt work yet + url = "#{host}:#{port}" + conn_options = args.include?(:retry_limit)? "reconnect-limit:#{args[:retry_limit]}" : "" + @connection = Cqpid::Connection.new(url, conn_options) + @connection.open + @session = Qmf2::ConsoleSession.new(@connection) + @session.open + @session.set_agent_filter("[and, [eq, _vendor, [quote, 'redhat.com']], [eq, _product, [quote, 'imagefactory']]]") + + if args.include?(:logger) + @logger = args[:logger] + else + @logger = Logger.new(STDOUT) + @logger.level = Logger::ERROR + @logger.datetime_format = "%Y-%m-%d %H:%M:%S" + end + @handler = args.include?(:handler)? args[:handler]: BaseHandler.new + super(@session) + end + + # Call this method to initiate a build, and get back an + # BuildAdaptor object. + # * descriptor => String + # This can be either xml or a url pointing to the xml + # * target => String + # Represents the target provider type to build for (ec2, mock, etc) + # * Returns => a BuildAdaptor object + # + def build_image(descriptor, target) + # TODO: return error if there is a problem calling this method or getting + # a factory instance + response = factory.image(descriptor, target) + build_adaptor(response) + end + + # Call this method to push an image to a provider, and get back an + # BuildAdaptor object. + # * image_id => String (uuid) + # * provider => String + # Represents the target provider to build for (ec2-us-east, mock, etc) + # * credentials => String + # XML block to be used for registration, upload, etc + # * Returns => a BuildAdaptor object + # + def push_image(image_id, provider, credentials) + # TODO: return error if there is a problem calling this method or getting + # a factory instance + response = factory.provider_image(image_id, provider, credentials) + build_adaptor(response) + end + + #TODO: enhance both of these methods to handle multiple agents + def agent_added(agent) + @logger.debug "GOT AN AGENT: #{agent} at #{Time.now.utc}" + @q = agent if agent.product == "imagefactory" + end + + def agent_deleted(agent, reason) + @logger.debug "AGENT GONE: #{agent} at #{Time.now.utc}, because #{reason}" + unless @q==nil + @q = {} if @q.product == agent.product + end + end + + # TODO: handle agent restart events (this will be more useful when + # restarted agent can recover an in-process build + def agent_restarted(agent) + @logger.debug "AGENT RESTARTED: #{agent.product}" + end + + # TODO: handle schema updates. This will be more useful when/if + # we make this a more generic console to talk to different agents. + def agent_schema_updated(agent) + @logger.debug "AGENT SCHEMA UPDATED: #{agent.product}" + end + + def event_raised(agent, data, timestamp, severity) + @logger.debug "GOT AN EVENT: #{agent}, #{data} at #{timestamp}" + @handler.handle(data) + end + + def shutdown + @logger.debug "Closing connections.." + if @session + @session.close + end + @connection.close + self.cancel + end + + private + + def factory + @factory ||= @q.query("{class:ImageFactory, package:'com.redhat.imagefactory'}").first + end + + def build_adaptor(response) + imgfacaddr = Qmf2::DataAddr.new(response['build_adaptor']) + query = Qmf2::Query.new(imgfacaddr) + @q.query(query).first + end + +end + +#i = ImageBuilderConsole.new +#i.run diff --git a/services/image_factory/console/spec/image_factory/console_spec.rb b/services/image_factory/console/spec/image_factory/console_spec.rb new file mode 100644 index 0000000..a1aa4af --- /dev/null +++ b/services/image_factory/console/spec/image_factory/console_spec.rb @@ -0,0 +1,92 @@ +require 'spec_helper' + +module ImageFactory + describe ImageFactoryConsole do + context "needs a real agent running" do + # for now, these all require an actual agent on the bus + before(:each) do + @logger = Logger.new(STDOUT) + @logger.level = Logger::DEBUG + @logger.datetime_format = "%Y-%m-%d %H:%M:%S" + @i = ImageFactoryConsole.new() + @i.start + # Sadly, this is needed so we wait for an agent to appear. + # We might be able to extract this somewhere to hold off + # on tests until we get an agent_added notification + sleep(5) + end + + describe "#q" do + it "should get an agent set" do + @i.q.should_not be_nil + end + end + + describe "#build_image" do + it "should return a build-adaptor with uuid" do + @i.build_image("<template></template>", "mock").image_id.should_not be_nil + end + end + + describe "#push_image" do + it "should return a build-adaptor with uuid" do + @image = @i.build_image("<template></template>", "mock") + #@i.handler = BaseHandler.new(@logger) + @i.handler.should_not_receive(:handle_failed) + @i.push_image(@image.image_id, "mock", "some creds").image_id.should_not be_nil + end + end + + after(:each) do + @i.shutdown + end + end + + context "agent is stubbed" do + before(:each) do + @agent = mock('agent', :null_object => true) + @agent.stub(:product).and_return("imagefactory") + @output = double('output') + @i = ImageFactoryConsole.new(:logger => @output) + end + + describe "#agent_added" do + it "logs when a new agent appears" do + @output.should_receive(:debug).with(/GOT AN AGENT/).as_null_object + @i.agent_added(@agent) + end + + it "stores a reference to that agent if it is a factory" do + @output.should_receive(:debug).with(/GOT AN AGENT/) + @i.agent_added(@agent) + @i.q.should respond_to(:product) + end + end + + describe "#event_raised" do + before(:each) do + @data = mock('data', :null_object => true) + @data.stub(:event).and_return("STATUS") + end + + it "should log an event" do + @output.should_receive(:debug).with(/GOT AN EVENT/) + @i.event_raised(@agent, @data, Time.now, "horrid") + end + + it "should call handle on an event" do + @output.should_receive(:debug).with(/GOT AN EVENT/) + + @i.handler.should_receive(:handle).once + @i.event_raised(@agent, @data, Time.now, "horrid") + end + end + + after(:each) do + @output.should_receive(:debug).with(/Closing/) + @i.shutdown + end + end + + end +end diff --git a/services/image_factory/console/spec/spec_helper.rb b/services/image_factory/console/spec/spec_helper.rb new file mode 100644 index 0000000..56baf4a --- /dev/null +++ b/services/image_factory/console/spec/spec_helper.rb @@ -0,0 +1,3 @@ +$: << File.expand_path(File.join(File.dirname(__FILE__), "../lib")) +require 'rubygems' +require 'image_factory'
On Wed, 2011-03-02 at 22:13 -0500, Jason Guiditta wrote:
This is the interface to the factory agent, the service, (which is what conductor will communicate with) will be in an upcoming patch.
Not sure why, but my intro email for this didnt seem to come through (at least I don't see it). Scott is looking at this since he has the pre-reqs installed, so I would prefer no one else worry about it, as it will break your current end2end setups.
-j
aeolus-devel@lists.fedorahosted.org