[PATCH configure] changed cleanup script to log to aeolus-cleanup.log
by Martyn Taylor
From: Martyn Taylor <mtaylor(a)redhat.com>
---
bin/aeolus-cleanup | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/bin/aeolus-cleanup b/bin/aeolus-cleanup
index 22387a1..bdbbacd 100644
--- a/bin/aeolus-cleanup
+++ b/bin/aeolus-cleanup
@@ -36,6 +36,6 @@ export FACTER_AEOLUS_ENABLE_SECURITY=false
puppet /usr/share/aeolus-configure/modules/aeolus/manifests/defaults.pp \
--modulepath=/usr/share/aeolus-configure/modules/ \
--external_nodes '/usr/sbin/aeolus-node cleanup' --node_terminus exec \
- --logdest=/var/log/aeolus-configure/aeolus-configure.log \
+ --logdest=/var/log/aeolus-configure/aeolus-cleanup.log \
--logdest=console \
$LOGLEVEL
--
1.7.4
12 years, 11 months
[PATCH configure] Added options for verbose/debug logging in cleanup
by Martyn Taylor
From: Martyn Taylor <mtaylor(a)redhat.com>
---
bin/aeolus-cleanup | 27 ++++++++++++++++++++++++++-
1 files changed, 26 insertions(+), 1 deletions(-)
diff --git a/bin/aeolus-cleanup b/bin/aeolus-cleanup
index a2ff45f..552b9ed 100644
--- a/bin/aeolus-cleanup
+++ b/bin/aeolus-cleanup
@@ -1,4 +1,28 @@
#!/bin/sh
+usage()
+{
+cat << EOF
+
+USAGE:
+aeolus-cleanup [-d|--debug] [-h|--help] [-v|--verbose]
+
+OPTIONS:
+ -h | --help Show this message.
+ -d | --debug Debug logging mode.
+ -v | --verbose Verbose logging mode.
+EOF
+}
+
+eval set -- `getopt -o hdv --long help,debug,verbose -- "$@"`
+while true ; do
+ case "$1" in
+ -h|--help) usage ; exit 1 ; shift ;;
+ -d|--debug) LOGLEVEL="--debug" ; shift ;;
+ -v|--verbose) LOGLEVEL="--verbose" ; shift ;;
+ --) shift ; break ;;
+ *) usage ; exit 1 ;;
+ esac
+done
export FACTER_AEOLUS_ENABLE_HTTPS=true
export FACTER_AEOLUS_ENABLE_SECURITY=false
@@ -6,5 +30,6 @@ puppet /usr/share/aeolus-configure/modules/aeolus/manifests/defaults.pp \
--modulepath=/usr/share/aeolus-configure/modules/ \
--external_nodes '/usr/sbin/aeolus-node cleanup' --node_terminus exec \
--logdest=/var/log/aeolus-configure/aeolus-configure.log \
- --logdest=console
+ --logdest=console \
+ $LOGLEVEL
--
1.7.4
12 years, 11 months
[PATCH configure 1/2] Fix to aeolus-configure options
by Martyn Taylor
From: Martyn Taylor <mtaylor(a)redhat.com>
---
bin/aeolus-configure | 9 ++++++++-
1 files changed, 8 insertions(+), 1 deletions(-)
diff --git a/bin/aeolus-configure b/bin/aeolus-configure
index b4c1f39..310b76e 100644
--- a/bin/aeolus-configure
+++ b/bin/aeolus-configure
@@ -13,7 +13,14 @@ OPTIONS:
EOF
}
-eval set -- `getopt -o hdv --long help,debug,verbose -- "$@"`
+args=`getopt -o :hdv --long help,debug,verbose -- "$@"`
+if test $? != 0
+ then
+ usage
+ exit 1
+fi
+
+eval set -- $args
while true ; do
case "$1" in
-h|--help) usage ; exit 1 ; shift ;;
--
1.7.4
12 years, 11 months
#1451/1452 - RESTful controllers
by Matt Wagner
Hi all,
I'm attaching a patch that implements feature #1451 - RESTful
controllers. The scope here was just to update PoolsController,
InstancesController, and DeploymentsController.
There were some oddities along the process, since some of what we do
doesn't map up with the REST ideals. So please note (and feel free to
propose alternatives for) the following:
* I implemented destroy methods, but nothing calls them yet. The destroy
* methods will accept params[:id] or an array in params[:ids], so we
* could replace the multi_destroy methods with destroy. I have not done
* so yet because Rails won't generate an instance URL this way, so it
* would require some client-side wrangling. multi_destroy is a
* collection method in routes.rb, but destroy is expected to be an
* instance method and requires that the ID be passed in. In our case,
* the id is an array of whatever the user selects. For now I've just
* left multi_destroy, but this feels like needless duplication.
* I didn't link to the edit page for Instances or Deployments. Right
* now, the user is only permitted to edit the name attribute for these,
* since changing any other attributes would be harmful (or, at best,
* ineffective). I have what feels like a crude setup here to strip out
* non-permitted attributes. It seems like attr_protected would help
* here, but it's really going to get much more complicated -- other
* components of the app can use this method to, for example, mark the
* status of an instance as having changed, but a user shouldn't be able
* to. I'm all ears if someone sees a cleaner way to do this.
The short version is that there are currently no user-facing changes in
this patch, but it does afford us a RESTful interface for Pools,
Instances, and Deployments.
-- Matt
12 years, 11 months
[PATCH conductor 1/4] add extended dbomatic logging
by Mo Morsi
---
src/dbomatic/dbomatic | 57 ++++++++++++++++++++++++++++++++++++++----------
1 files changed, 45 insertions(+), 12 deletions(-)
diff --git a/src/dbomatic/dbomatic b/src/dbomatic/dbomatic
index e7f29b0..ff5b7fb 100755
--- a/src/dbomatic/dbomatic
+++ b/src/dbomatic/dbomatic
@@ -74,27 +74,34 @@ CONDOR_EVENT_LOG_FILE = "#{condor_event_log_dir}/EventLog"
CONDOR_EVENT_LOG_FILE_OLD = "#{condor_event_log_dir}/EventLog.old"
EVENT_LOG_POS_FILE = "#{dbomatic_run_dir}/event_log_position"
if dbomatic_log_dir == '-'
- DBOMATIC_LOG_FILE = STDOUT
+ DBOMATIC_LOG_FILE = STDOUT
+ DBOMATIC_PARSER_LOG_FILE = STDOUT
else
- DBOMATIC_LOG_FILE = "#{dbomatic_log_dir}/dbomatic.log"
+ DBOMATIC_LOG_FILE = "#{dbomatic_log_dir}/dbomatic.log"
+ DBOMATIC_PARSER_LOG_FILE = "#{dbomatic_log_dir}/dbomatic-parser.log"
end
-logger = Logger.new(DBOMATIC_LOG_FILE)
-logger.level = Logger::DEBUG
-logger.info "DBOmatic starting up"
-
# daemonize
if daemon
# note that this requires 'active_support', which we get for free from dutils
Process.daemon
end
+# Custom Log Format
+class DBomaticLogger < Logger
+ def format_message(severity, timestamp, progname, msg)
+ "#{timestamp.to_formatted_s(:db)} #{severity} #{msg}\n"
+ end
+end
+
# Handle the event log's xml
class CondorEventLog < Nokogiri::XML::SAX::Document
attr_accessor :tag, :event_type, :event_cmd, :event_time, :trigger_type, :grid_resource, :execute_host, :username, :hold_reason, :private_addresses, :public_addresses
- def initialize(logger)
- @logger = logger
+ def initialize
+ @logger = DBomaticLogger.new(DBOMATIC_PARSER_LOG_FILE)
+ @logger.level = Logger::DEBUG
+ @logger.info "DBOmatic parser starting up"
end
# Store the name of the event log attribute we're looking at
@@ -130,6 +137,8 @@ class CondorEventLog < Nokogiri::XML::SAX::Document
end
def update_instance_state_event(inst)
+ @logger.info "update_instance_state_event for #{inst}"
+
if @trigger_type == "ULOG_GRID_SUBMIT"
inst.state = Instance::STATE_PENDING
elsif @trigger_type == "ULOG_JOB_ABORTED" or @trigger_type == "ULOG_JOB_TERMINATED"
@@ -160,12 +169,14 @@ class CondorEventLog < Nokogiri::XML::SAX::Document
inst.last_error = @hold_reason
inst.state = Instance::STATE_ERROR
else
- @logger.info "Unexpected trigger type #{@trigger_type}, not updating instance state"
+ @logger.warn "Unexpected trigger type #{@trigger_type}, not updating instance state"
return
end
begin
+ @logger.info "update_instance_state_event saving instance #{inst}"
inst.save!
+ @logger.debug "updated_instance_state_event saved instance #{inst}, creating event for state #{inst.state}@#{@event_time}"
inst.events.create!(:status_code => inst.state,
:event_time => @event_time)
rescue => e
@@ -174,9 +185,13 @@ class CondorEventLog < Nokogiri::XML::SAX::Document
@logger.error "\tfrom #{step}"
end
end
+
+ @logger.info "update_instance_state_event completed fo #{inst}"
end
def update_instance_cloud_id(inst)
+ @logger.info "update_instance_cloud_id for #{inst}"
+
# The GridResource/ExecuteHost string looks like this:
# dcloud http://localhost:3001/api
@@ -185,7 +200,7 @@ class CondorEventLog < Nokogiri::XML::SAX::Document
elsif !(a)execute_host.nil?
resource = @execute_host
else
- @logger.info "Unexpected nil GridResource/ExecuteHost field, skipping cloud id update"
+ @logger.warn "Unexpected nil GridResource/ExecuteHost field, skipping cloud id update"
return
end
@@ -213,15 +228,22 @@ class CondorEventLog < Nokogiri::XML::SAX::Document
return
end
+ @logger.info "update_instance_cloud_id updating instance #{inst} to cloud provider #{provider}"
inst.provider_account_id = provider_account.id
inst.save!
+ @logger.info "update_instance_cloud_id completed for #{inst}"
end
def update_instance_addresses(inst)
+ @logger.info "update_instance_addresses for #{inst}, \
+ setting public addresses: #{@public_addresses} \
+ --- and private addresses #{private_addresses}"
+
inst.public_addresses = @public_addresses
inst.private_addresses = @private_addresses
inst.save!
+ @logger.info "update_instance_addresses completed for #{inst}"
end
# Create a new entry for events which we have all the neccessary data for
@@ -231,11 +253,13 @@ class CondorEventLog < Nokogiri::XML::SAX::Document
inst = Instance.find(:first, :conditions => ['condor_job_id = ?', @event_cmd])
if inst.nil?
- @logger.info "Unexpected nil instance, skipping..."
+ @logger.warn "Unexpected nil instance, skipping..."
else
+ @logger.info "Instance #{inst} found, running update events"
update_instance_state_event(inst)
update_instance_cloud_id(inst)
update_instance_addresses(inst)
+ @logger.info "Instance #{inst} update events completed"
end
@tag = @event_type = @event_cmd = @event_time = @trigger_type = @grid_resource = @execute_host = @hold_reason = @public_addresses = @private_addresses = nil
end
@@ -282,13 +306,18 @@ def parse_log_file(parser)
File.open(EVENT_LOG_POS_FILE, 'w') { |f| f.write log_file.pos.to_s }
end
+logger = DBomaticLogger.new(DBOMATIC_LOG_FILE)
+logger.level = Logger::DEBUG
+logger.datetime_format = "%Y-%m-%d %H:%M " # simplify time output
+logger.info "DBOmatic starting up"
+
begin
DBOMATIC_PID_FILE = "#{dbomatic_pid_dir}/dbomatic.pid"
FileUtils.mkdir_p File.dirname(DBOMATIC_PID_FILE)
open(DBOMATIC_PID_FILE, "w") {|f| f.write(Process.pid) }
File.chmod(0644, DBOMATIC_PID_FILE)
- parser = Nokogiri::XML::SAX::PushParser.new(CondorEventLog.new(logger))
+ parser = Nokogiri::XML::SAX::PushParser.new CondorEventLog.new
# XXX hack, condor event log doesn't seem to have a top level element
# enclosing everything else in the doc (as standards conforming xml must).
@@ -298,6 +327,7 @@ begin
notifier = INotify::Notifier.new
parse_log_file(parser) if File.exists? CONDOR_EVENT_LOG_FILE
+ logger.info "Parsed existing event log file - current postition: #{get_log_file_pos}"
# Setup inotify watch for condor event log changes
notifier.watch(condor_event_log_dir, :all_events){ |event|
@@ -306,6 +336,7 @@ begin
end
}
+ logger.info "Beginning main event loop"
while true
begin
notifier.run
@@ -314,8 +345,10 @@ begin
e.backtrace.each do |step|
logger.error "\tfrom #{step}"
end
+ logger.info "EventLog modification event trigger completed, parsing finished - current position #{get_log_file_pos}"
end
end
+ logger.info "Main event loop completed"
parser << "</events>"
parser.finish
--
1.7.2.3
12 years, 11 months
[PATCH configure] Added version dep 2.6.6 for puppet
by Martyn Taylor
From: Martyn Taylor <mtaylor(a)redhat.com>
---
contrib/aeolus-configure.spec | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/contrib/aeolus-configure.spec b/contrib/aeolus-configure.spec
index f578378..2cb6e47 100644
--- a/contrib/aeolus-configure.spec
+++ b/contrib/aeolus-configure.spec
@@ -12,7 +12,7 @@ URL: http://aeolusproject.org
Source0: %{name}-%{version}.tgz
BuildRoot: %{_tmppath}/%{name}-%{version}
BuildArch: noarch
-Requires: puppet
+Requires: puppet >= 2.6.6
Requires: rubygem(uuidtools)
# To send a request to iwhd rest interface to
# create buckets, eventually replace w/ an
--
1.7.4
12 years, 11 months
