FBR: Fix the metadata handling for Rawhide messages
by Sayan Chowdhury
Hi,
Here is a hotfix for fedimg. Right now Rawhide messages are not
getting processed. This hotfix will fix the issue.
PR for the same[1]
[1] https://github.com/fedora-infra/fedimg/pull/103
+1s
From b52a1dd1eda25bfbade026a1ebde93839fa3379b Mon Sep 17 00:00:00 2001
From: Sayan Chowdhury <sayan.chowdhury2012(a)gmail.com>
Date: Thu, 19 Apr 2018 22:05:27 +0530
Subject: [PATCH 1/2] fedimg: Add the intial files for PR#103
Signed-off-by: Sayan Chowdhury <sayan.chowdhury2012(a)gmail.com>
---
files/hotfix/fedimg/consumers.py | 160 +++++++++++++++++++++++++++++++++++++++
roles/fedimg/tasks/main.yml | 8 ++
2 files changed, 168 insertions(+)
create mode 100644 files/hotfix/fedimg/consumers.py
diff --git a/files/hotfix/fedimg/consumers.py b/files/hotfix/fedimg/consumers.py
new file mode 100644
index 0000000..ce4d662
--- /dev/null
+++ b/files/hotfix/fedimg/consumers.py
@@ -0,0 +1,160 @@
+# -*- coding: utf-8 -*-
+# This file is part of fedimg.
+# Copyright (C) 2014-2017 Red Hat, Inc.
+#
+# fedimg is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as
+# published by the Free Software Foundation, either version 3 of the
+# License, or (at your option) any later version.
+#
+# fedimg is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public
+# License along with fedimg; if not, see http://www.gnu.org/licenses,
+# or write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Authors: David Gay <dgay(a)redhat.com>
+# Sayan Chowdhury <sayanchowdhury(a)fedoraproject.org>
+"""
+This is the `fedmsg consumer`_ that subscribes to the topic emitted after the
+completion of the nightly and production compose. The consumer on receving the
+message uploads the image using the API of the cloud providers.
+"""
+
+import logging
+import multiprocessing.pool
+
+import fedmsg.consumers
+import fedmsg.encoding
+import fedfind.release
+import fedfind.exceptions
+
+import fedimg.uploader
+
+from fedimg.config import PROCESS_COUNT, STATUS_FILTER
+from fedimg.utils import get_rawxz_urls, get_value_from_dict
+
+_log = logging.getLogger(__name__)
+
+
+class FedimgConsumer(fedmsg.consumers.FedmsgConsumer):
+ """
+ A `fedmsg consumer`_ that listens to the pungi compose topics and kicks
+ of the process to upload the images to various cloud providers.
+
+ Attributes:
+ topic (str): The topics this consumer is subscribed to. Set to
+ ``org.fedoraproject.prod.pungi.compose.status.change``.
+ config_key (str): The key to set to ``True`` in the fedmsg config to
+ enable this consumer. The key is ``fedimgconsumer.prod.enabled``.
+ """
+ topic = ['org.fedoraproject.prod.pungi.compose.status.change']
+ config_key = "fedimgconsumer.prod.enabled"
+
+ def __init__(self, *args, **kwargs):
+ _log.info("FedimgConsumer initializing")
+ super(FedimgConsumer, self).__init__(*args, **kwargs)
+
+ # Threadpool for upload jobs
+ _log.info("Creating thread pool of %s process", PROCESS_COUNT)
+ self.upload_pool = multiprocessing.pool.ThreadPool(
+ processes=PROCESS_COUNT
+ )
+ _log.info("FedimgConsumer initialized")
+
+ def consume(self, msg):
+ """
+ This is called when we receive a message matching our topics.
+
+ Args:
+ msg (dict): The raw message from fedmsg.
+ """
+ _log.info('Received %r %r', msg['topic'], msg['body']['msg_id'])
+
+ msg_info = msg['body']['msg']
+ if msg_info['status'] not in STATUS_FILTER:
+ _log.debug('%s is not valid status' % msg_info['status'])
+ return
+
+ location = msg_info['location']
+ compose_id = msg_info['compose_id']
+ try:
+ compose_metadata =
fedfind.release.get_release(cid=compose_id).metadata
+ except fedfind.exceptions.UnsupportedComposeError:
+ LOG.debug("%r is unsupported compose" % compose_id)
+ return
+
+
+ # Till F27, both cloud-base and atomic images were available
+ # under variant CloudImages. With F28 and onward releases,
+ # cloud-base image compose moved to cloud variant and atomic images
+ # moved under atomic variant.
+ prev_rel = ['26', '27']
+ if msg_info['release_version'] in prev_rel:
+ images_meta = get_value_from_dict(
+ compose_metadata, 'images', 'payload', 'images', 'CloudImages',
+ 'x86_64')
+ else:
+ images_meta = get_value_from_dict(
+ compose_metadata, 'images', 'payload', 'images',
+ 'Cloud', 'x86_64')
+ images_meta.extend(get_value_from_dict(
+ compose_metadata, 'images', 'payload',
+ 'images', 'AtomicHost', 'x86_64'))
+
+ images_meta = get_value_from_dict(
+ compose_metadata,
+ 'images',
+ 'payload',
+ 'images',
+ 'CloudImages',
+ 'x86_64'
+ )
+
+ if images_meta is None:
+ _log.debug('No compatible image found to process')
+ return
+
+ upload_urls = get_rawxz_urls(location, images_meta)
+ if len(upload_urls) > 0:
+ _log.info("Start processing compose id: %s", compose_id)
+ fedimg.uploader.upload(
+ pool=self.upload_pool,
+ urls=upload_urls,
+ compose_id=compose_id,
+ push_notifications=True
+ )
+
+
+class FedimgStagingConsumer(FedimgConsumer):
+ """
+ A `fedmsg consumer`_ that listens to the staging pungi compose topics and
+ kicks of the process to upload the images to various cloud providers.
+
+ Attributes:
+ topic (str): The topics this consumer is subscribed to. Set to
+ ``org.fedoraproject.stg.pungi.compose.status.change``.
+ config_key (str): The key to set to ``True`` in the fedmsg config to
+ enable this consumer. The key is ``fedimgconsumer.stg.enabled``.
+ """
+ topic = ['org.fedoraproject.stg.pungi.compose.status.change']
+ config_key = "fedimgconsumer.stg.enabled"
+
+
+class FedimgDevConsumer(FedimgConsumer):
+ """
+ A `fedmsg consumer`_ that listens to the dev pungi compose topics and
+ kicks of the process to upload the images to various cloud providers.
+
+ Attributes:
+ topic (str): The topics this consumer is subscribed to. Set to
+ ``org.fedoraproject.dev.pungi.compose.status.change``.
+ config_key (str): The key to set to ``True`` in the fedmsg config to
+ enable this consumer. The key is ``fedimgconsumer.dev.enabled``.
+ """
+ topic = ['org.fedoraproject.dev.pungi.compose.status.change']
+ config_key = "fedimgconsumer.dev.enabled"
diff --git a/roles/fedimg/tasks/main.yml b/roles/fedimg/tasks/main.yml
index b0f18bf..2e35608 100644
--- a/roles/fedimg/tasks/main.yml
+++ b/roles/fedimg/tasks/main.yml
@@ -130,3 +130,11 @@
tags:
- fedimg
- hotfix
+
+- name: hotfix - copy the consumers.py over to the site-packages
+ copy: src="{{ files }}/hotfix/fedimg/consumers.py"
dest=/usr/lib/python2.7/site-packages/fedimg/consumers.py
+ notify:
+ - restart fedmsg-hub
+ tags:
+ - fedimg
+ - hotfix
--
2.9.4
From 1bf3db45c8154d0a6a0f29836ec14ce24fa404c1 Mon Sep 17 00:00:00 2001
From: Sayan Chowdhury <sayan.chowdhury2012(a)gmail.com>
Date: Thu, 19 Apr 2018 22:07:12 +0530
Subject: [PATCH 2/2] fedimg: Add the patch for the PR#103, fix processing
Rawhide messages
Signed-off-by: Sayan Chowdhury <sayan.chowdhury2012(a)gmail.com>
---
files/hotfix/fedimg/consumers.py | 11 +----------
1 file changed, 1 insertion(+), 10 deletions(-)
diff --git a/files/hotfix/fedimg/consumers.py b/files/hotfix/fedimg/consumers.py
index ce4d662..8195321 100644
--- a/files/hotfix/fedimg/consumers.py
+++ b/files/hotfix/fedimg/consumers.py
@@ -85,7 +85,7 @@ class FedimgConsumer(fedmsg.consumers.FedmsgConsumer):
try:
compose_metadata =
fedfind.release.get_release(cid=compose_id).metadata
except fedfind.exceptions.UnsupportedComposeError:
- LOG.debug("%r is unsupported compose" % compose_id)
+ _log.debug("%r is unsupported compose" % compose_id)
return
@@ -106,15 +106,6 @@ class FedimgConsumer(fedmsg.consumers.FedmsgConsumer):
compose_metadata, 'images', 'payload',
'images', 'AtomicHost', 'x86_64'))
- images_meta = get_value_from_dict(
- compose_metadata,
- 'images',
- 'payload',
- 'images',
- 'CloudImages',
- 'x86_64'
- )
-
if images_meta is None:
_log.debug('No compatible image found to process')
return
--
2.9.4
--
Sayan Chowdhury <https://sayanchowdhury.dgplug.org/>
Senior Software Engineer, Fedora Engineering - Emerging Platform
GPG Fingerprint : 0F16 E841 E517 225C 7D13 AB3C B023 9931 9CD0 5C8B
Proud to work at The Open Organization!
6 years
FBR MBS: Fix traceback when reusing components from module build with
different buildrequires.
by Jan Kaluža
Hi,
this is FBR to fix MBS traceback which prevents builds of modules in situation when new build-require is added to a module Foo. Currently, MBS tries to check what was the commit hash ("ref") of such buildrequired module in previous build of Foo, but it fails with KeyError, because the previous Foo module build did not build-required this newly added build-required module.
Full traceback is here:
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/module_build_service/scheduler/consumer.py", line 240, in process_message
further_work = handler(conf, session, msg) or []
File "/usr/lib/python2.7/site-packages/module_build_service/scheduler/handlers/modules.py", line 292, in wait
if attempt_to_reuse_all_components(builder, session, build):
File "/usr/lib/python2.7/site-packages/module_build_service/utils/reuse.py", line 140, in attempt_to_reuse_all_components
previous_module_build = _get_reusable_module(session, module)
File "/usr/lib/python2.7/site-packages/module_build_service/utils/reuse.py", line 123, in _get_reusable_module
ref2 = old_xmd['mbs']['buildrequires'][br_module_name].get('ref')
KeyError: 'platform'
Fixed patch [1] addresses this by catching the KeyError in this code block. It has been tested on staging and I verified it fixes this issue.
[1] https://src.fedoraproject.org/rpms/module-build-service/c/9d3a4923631efca...
Regards,
Jan Kaluza
6 years
Freeze Break Request: re-enable fedmsg hook on src
by Kevin Fenzi
Greetings.
Just before freeze in ansible commit
dc86b22f9054bdcefd51e46db90433c03d54410c pingou disabled the fedmsg hook
because there was a permissions problem reading the fedmsg key.
In ansible commit 9cbf120b54a79ecb07931f3a3a1b059525d4d766 patrick fixed
this perm issue via a facl.
So, I would like to re-enable the hook on normal projects and remove it
from forks. (I think we decided forks should only get the pagure hook,
not the main fedmsg one).
So, I'd like to apply:
> diff --git a/roles/git/hooks/files/post-receive-chained b/roles/git/hooks/files/post-receive-chained
> index 99b5a2a..5c74da4 100755
> --- a/roles/git/hooks/files/post-receive-chained
> +++ b/roles/git/hooks/files/post-receive-chained
> @@ -8,7 +8,7 @@ pee \
> /usr/share/git-core/post-receive-alternativearch \
> /usr/lib/python2.7/site-packages/pagure/hooks/files/default_hook.py \
> /usr/lib/python2.7/site-packages/pagure/hooks/files/pagure_hook.py \
> - /usr/lib/python2.7/site-packages/pagure/hooks/files/fedmsg_hook.py \
> +# /usr/lib/python2.7/site-packages/pagure/hooks/files/fedmsg_hook.py \
> "/usr/bin/grok-manifest -m /srv/git/grokmirror/manifest.js.gz -t /srv/git/repositories/ -n `pwd`"
>
> # We used to send emails directly from the git hook here, but now we send to
> diff --git a/roles/git/hooks/files/post-receive-chained-forks b/roles/git/hooks/files/post-receive-chained-forks
> index 5e0056d..67565e6 100755
> --- a/roles/git/hooks/files/post-receive-chained-forks
> +++ b/roles/git/hooks/files/post-receive-chained-forks
> @@ -6,7 +6,7 @@
> pee \
> /usr/lib/python2.7/site-packages/pagure/hooks/files/default_hook.py \
> /usr/lib/python2.7/site-packages/pagure/hooks/files/pagure_hook.py \
> - /usr/lib/python2.7/site-packages/pagure/hooks/files/fedmsg_hook.py \
> +# /usr/lib/python2.7/site-packages/pagure/hooks/files/fedmsg_hook.py \
> "/usr/bin/grok-manifest -m /srv/git/grokmirror/manifest.js.gz -t /srv/git/repositories/ -n `pwd`"
>
> # We used to send emails directly from the git hook here, but now we send to
Run the ansible playbook against pkgs,
And then run the script to make sure all projects are set right.
+1s?
kevin
6 years
Fedora Tagger - Future development meeting
by Elorm Buertey
Hi everyone,
You are kindly invited to the Fedora tagger meeting
Starting from Fri, April 20, 2018 - 14:00 UTC and ending on Fri, April 20, 2018 - 15:00:00 UTC
At fedora-meeting-1(a)irc.freenode.net
This meeting is for stakeholders and interested developers in the future of the Fedora Tagger application.
This meeting will focus on figuring out milestones to update the Tagger application and how to support it in Fedora's infrastructure going forward.
Anyone is welcome to participate. Please see the link below for more information:
https://github.com/fedora-infra/fedora-tagger/issues/200
Regards,
Elorm Buertey
6 years
FBR: Check if AMI process is complete before processing
by Sayan Chowdhury
Hi,
I want to push a hotfix for fedimg. This patch would check if the AMI
process is complete before proceeding forward.
Without this patch the AMIs are failed to copied to different region
after it is uploaded to the base region.
PR for the same[1]
[1] https://github.com/fedora-infra/fedimg/pull/100
+1s
From 191af6ec22d5977a432862aa08254f131cd6e9a1 Mon Sep 17 00:00:00 2001
From: Sayan Chowdhury <sayan.chowdhury2012(a)gmail.com>
Date: Wed, 18 Apr 2018 23:56:45 +0530
Subject: [PATCH 1/2] fedimg: Add the initial files for the hotfix, fix copied
amis
Signed-off-by: Sayan Chowdhury <sayan.chowdhury2012(a)gmail.com>
---
files/hotfix/fedimg/ec2imgpublisher.py | 300 +++++++++++++++++++++++++++++++++
roles/fedimg/tasks/main.yml | 7 +
2 files changed, 307 insertions(+)
create mode 100644 files/hotfix/fedimg/ec2imgpublisher.py
diff --git a/files/hotfix/fedimg/ec2imgpublisher.py
b/files/hotfix/fedimg/ec2imgpublisher.py
new file mode 100644
index 0000000..6e62e04
--- /dev/null
+++ b/files/hotfix/fedimg/ec2imgpublisher.py
@@ -0,0 +1,300 @@
+# This file is part of fedimg.
+# Copyright (C) 2014-2017 Red Hat, Inc.
+#
+# fedimg is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as
+# published by the Free Software Foundation, either version 3 of the
+# License, or (at your option) any later version.
+#
+# fedimg is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public
+# License along with fedimg; if not, see http://www.gnu.org/licenses,
+# or write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Authors: Sayan Chowdhury <sayanchowdhury(a)fedoraproject.org>
+#
+
+import logging
+_log = logging.getLogger(__name__)
+
+import re
+
+from time import sleep
+
+import fedimg.messenger
+
+from fedimg.utils import external_run_command, get_item_from_regex
+from fedimg.utils import get_image_name_from_ami_name
+from fedimg.utils import get_image_name_from_ami_name_for_fedmsg
+from fedimg.services.ec2.ec2base import EC2Base
+
+
+class EC2ImagePublisher(EC2Base):
+ """ Comment goes here """
+
+ def __init__(self, **kwargs):
+ defaults = {
+ 'access_key': None,
+ 'compose_id': None,
+ 'image_id': None,
+ 'image_name': 'Fedora-AMI',
+ 'image_url': None,
+ 'image_description': 'Fedora AMI Description',
+ 'service': 'EC2',
+ 'region': None,
+ 'secret_key': None,
+ 'visibility': 'all',
+ 'push_notifications': False,
+ }
+
+ for (prop, default) in defaults.iteritems():
+ setattr(self, prop, kwargs.get(prop, default))
+
+ def _retry_till_image_is_public(self, image):
+ """ Comment goes here """
+
+ driver = self._connect()
+
+ is_image_public = False
+ while True:
+ try:
+ is_image_public = driver.ex_modify_image_attribute(
+ image,
+ {'LaunchPermission.Add.1.Group': 'all'})
+ except Exception as e:
+ if 'InvalidAMIID.Unavailable' in str(e):
+ # The copy isn't completed yet, so wait for 20 seconds
+ # more.
+ sleep(20)
+ continue
+ break
+
+ return is_image_public
+
+ def _retry_till_snapshot_is_public(self, snapshot):
+
+ driver = self._connect()
+
+ while True:
+ is_snapshot_public = driver.ex_modify_snapshot_attribute(
+ snapshot,
+ {'CreateVolumePermission.Add.1.Group': 'all'})
+
+ if is_snapshot_public:
+ break
+
+ return is_snapshot_public
+
+
+ def _retry_till_snapshot_is_available(self, image):
+
+ driver = self._connect()
+ while True:
+ image = driver.get_image(image.id)
+ snapshot_id =
image.extra['block_device_mapping'][0]['ebs']['snapshot_id']
+
+ if snapshot_id:
+ break
+
+ return snapshot_id
+
+ def _generate_dummy_snapshot_object(self, snapshot_id):
+
+ driver = self._connect()
+
+ snapshot_obj = type('', (), {})()
+ snapshot_obj.id = snapshot_id
+ snapshot = driver.list_snapshots(snapshot=snapshot_obj)
+
+ return snapshot
+
+ def _retry_till_blk_mapping_is_available(self, image):
+
+ while True:
+ image = self._connect().get_image(image_id=image.id)
+ blk_mapping = image.extra['block_device_mapping']
+
+ if blk_mapping:
+ return blk_mapping
+
+ def get_snapshot_from_image(self, image):
+ """ Comment goes here """
+ if isinstance(image, str):
+ image_id = image
+ image = self._connect().get_image(image_id)
+
+ blk_mapping = image.extra['block_device_mapping']
+ if not blk_mapping:
+ blk_mapping = self._retry_till_blk_mapping_is_available(image)
+
+ snapshot_id = blk_mapping[0]['ebs']['snapshot_id']
+ if snapshot_id is None:
+ snapshot_id = self._retry_till_snapshot_is_available(image)
+
+ snapshot = self._generate_dummy_snapshot_object(snapshot_id)[0]
+
+ return snapshot
+
+ def get_volume_type_from_image(self, image):
+ if isinstance(image, str):
+ image_id = image
+ image = self._connect().get_image(image_id)
+
+ blk_mapping = image.extra['block_device_mapping']
+ if not blk_mapping:
+ blk_mapping = self._retry_till_blk_mapping_is_available(image)
+
+ return blk_mapping[0]['ebs']['volume_type']
+
+ def get_virt_type_from_image(self, image):
+ return 'hvm'
+
+ def publish_images(self, region_image_mapping=None):
+ """ Comment goes here """
+
+ published_images = []
+ if region_image_mapping is None:
+ return published_images
+
+ for region, image_id in region_image_mapping:
+ self.set_region(region)
+
+ _log.info('Publish image (%s) in %s started' % (image_id, region))
+ image = self._connect().get_image(image_id=image_id)
+ is_image_public = self._retry_till_image_is_public(image)
+ _log.info('Publish image (%s) in %s completed' %
(image_id, region))
+
+ _log.info('Publish snaphsot for image (%s) in %s started'
% (image_id, region))
+ snapshot = self.get_snapshot_from_image(image)
+ _log.info('Fetched snapshot for image (%s): %s' %
(image_id, snapshot.id))
+ is_snapshot_public = self._retry_till_snapshot_is_public(snapshot)
+ _log.info('Publish snaphsot for image (%s) in %s
completed' % (image_id, region))
+
+ volume_type = self.get_volume_type_from_image(image)
+ virt_type = self.get_virt_type_from_image(image)
+
+ if self.push_notifications:
+ fedimg.messenger.notify(
+ topic='image.publish',
+ msg=dict(
+
image_name=get_image_name_from_ami_name_for_fedmsg(image.name),
+ image_url=self.image_url,
+ destination=self.region,
+ service=self.service,
+ compose=self.compose_id,
+ extra=dict(
+ id=image.id,
+ virt_type=virt_type,
+ vol_type=volume_type
+ )
+ )
+ )
+
+ fedimg.messenger.notify(
+ topic='image.upload',
+ msg=dict(
+
image_name=get_image_name_from_ami_name_for_fedmsg(image.name),
+ image_url=self.image_url,
+ destination=self.region,
+ service=self.service,
+ status='completed',
+ compose=self.compose_id,
+ extra=dict(
+ id=image.id,
+ virt_type=virt_type,
+ vol_type=volume_type
+ )
+ )
+ )
+
+ published_images.append({
+ 'image_id': image.id,
+ 'is_image_public': is_image_public,
+ 'snapshot_id': snapshot.id,
+ 'is_snapshot_public': is_snapshot_public,
+ 'regions': self.region
+ })
+
+ return published_images
+
+ def copy_images_to_regions(self, image_id=None, base_region=None,
regions=None):
+ """ Comment goes here """
+
+ if (image_id is None) or (regions is None) or (base_region is None):
+ return
+
+ counter = 0
+ copied_images = []
+
+ self.set_region(base_region)
+ image = self._connect().get_image(image_id=image_id)
+ if not image:
+ return []
+
+ for region in regions:
+ _log.info('Copy %s to %s started' % (image_id, region))
+ self.set_region(region)
+ self.image_name = get_image_name_from_ami_name(image.name, region)
+
+ while True:
+ if counter > 0:
+ self.image_name = re.sub(
+ '\d(?!\d)',
+ lambda x: str(int(x.group(0))+1),
+ self.image_name
+ )
+ try:
+ copied_image = self._connect().copy_image(
+ source_region=base_region,
+ image=image,
+ name=self.image_name,
+ description=self.image_description)
+
+ virt_type = image.extra['virtualization_type']
+ volume_type =
image.extra['block_device_mapping'][0]['ebs']['volume_type']
+
+ if self.push_notifications:
+ fedimg.messenger.notify(
+ topic='image.copy',
+ msg=dict(
+
image_name=get_image_name_from_ami_name_for_fedmsg(copied_image.name),
+ destination=self.region,
+ service=self.service,
+ compose_id=self.compose_id,
+ extra=dict(
+ id=copied_image.id,
+ virt_type=virt_type,
+ vol_type=volume_type,
+ source_image_id=image.id
+ )
+ )
+ )
+
+ _log.info('Copy %s to %s is completed.' %
(image_id, region))
+ copied_images.append({
+ 'region': region,
+ 'copied_image_id': copied_image.id
+ })
+ break
+
+ except Exception as e:
+ _log.info('Could not register '
+ 'with name: %r' % self.image_name)
+ if 'InvalidAMIName.Duplicate' in str(e):
+ counter = counter + 1
+ else:
+ _log.info('Failed')
+ break
+
+ return copied_images
+
+ def deprecate_images(self, image_ids=None, snapshot_perm='all'):
+ raise NotImplementedError
+
+ def delete_images(self, image_ids=None, snapshot_perm='all'):
+ raise NotImplementedError
diff --git a/roles/fedimg/tasks/main.yml b/roles/fedimg/tasks/main.yml
index c21cb8f..b0f18bf 100644
--- a/roles/fedimg/tasks/main.yml
+++ b/roles/fedimg/tasks/main.yml
@@ -123,3 +123,10 @@
- fedimg
- hotfix
+- name: hotfix - copy the ec2imgpublisher.py over to the site-packages
+ copy: src="{{ files }}/hotfix/fedimg/ec2imgpublisher.py"
dest=/usr/lib/python2.7/site-packages/fedimg/services/ec2/ec2imgpublisher.py
+ notify:
+ - restart fedmsg-hub
+ tags:
+ - fedimg
+ - hotfix
--
2.9.4
From d35fa21e8a6c0fe1e07ed9b74b0f524bba3c6c84 Mon Sep 17 00:00:00 2001
From: Sayan Chowdhury <sayan.chowdhury2012(a)gmail.com>
Date: Wed, 18 Apr 2018 23:57:46 +0530
Subject: [PATCH 2/2] fedimg: Checkin the patch for PR#100. Check if the AMI is
complete before proceeding
Signed-off-by: Sayan Chowdhury <sayan.chowdhury2012(a)gmail.com>
---
files/hotfix/fedimg/ec2imgpublisher.py | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/files/hotfix/fedimg/ec2imgpublisher.py
b/files/hotfix/fedimg/ec2imgpublisher.py
index 6e62e04..b1eab48 100644
--- a/files/hotfix/fedimg/ec2imgpublisher.py
+++ b/files/hotfix/fedimg/ec2imgpublisher.py
@@ -90,6 +90,22 @@ class EC2ImagePublisher(EC2Base):
return is_snapshot_public
+ def _retry_till_image_is_available(self, image_id):
+ driver = self._connect()
+
+ while True:
+ try:
+ image = driver.get_image(image_id)
+ image_name = image.name
+ if image_name is None:
+ continue
+ return image
+ except Exception as e:
+ if 'InvalidAMIID.Unavailable' in str(e):
+ # The copy isn't completed yet, so wait for 20 seconds
+ # more.
+ sleep(20)
+ continue
def _retry_till_snapshot_is_available(self, image):
@@ -255,6 +271,8 @@ class EC2ImagePublisher(EC2Base):
name=self.image_name,
description=self.image_description)
+ copied_image =
self._retry_till_image_is_available(copied_image.id)
+
virt_type = image.extra['virtualization_type']
volume_type =
image.extra['block_device_mapping'][0]['ebs']['volume_type']
--
2.9.4
--
Sayan Chowdhury <https://sayanchowdhury.dgplug.org/>
Senior Software Engineer, Fedora Engineering - Emerging Platform
GPG Fingerprint : 0F16 E841 E517 225C 7D13 AB3C B023 9931 9CD0 5C8B
Proud to work at The Open Organization!
6 years
Re: How To Contact Folks Interested In Tagger
by Justin W. Flory
On 04/14/2018 10:15 PM, Elorm Buertey wrote:
> Hi. I have created the WhenIsGood survey but I'm not sure of the
> best way to send it to the others. I couldn't find their
> emails on their profiles. Would posting the link in the Github
> issue be okay?
>
Hi Elorm,
Sharing the WhenIsGood link on the GitHub issue for the Fedora Tagger
development meeting works; I would also consider sharing it on the
Fedora Infrastructure mailing list since there are other people there
who may be interested.
You can subscribe here:
https://lists.fedoraproject.org/admin/lists/infrastructure.lists.fedorapr...
I've included this response to the mailing list already, so you can
subscribe and then share the link on this thread for the Fedora Tagger
development meeting time.
Hope this helps!
--
Cheers,
Justin W. Flory
jflory7(a)gmail.com
6 years
Fwd: mirror failures
by Miroslav Suchý
Can someone comment on this?
I am not really sure if there are some issues on mirrors (not likely) or if this is some issue with Fedora Cloud when
network is overloaded (more likely).
Miroslav
-------- Přeposlaná zpráva --------
Předmět: mirror failures
Datum: Mon, 9 Apr 2018 09:38:04 +0200
Od: Michal Novotny <clime(a)redhat.com>
Adresa pro odpověď: Community Projects <copr-devel(a)lists.fedorahosted.org>
Komu: Cool Other Package Repositories <copr-devel(a)lists.fedorahosted.org>
Hello,
as of recent month or two I quite often encounter mirror sync failures
on COPR builders during the build setup. Could you, please, confirm
that this is really the case and link the build logs here if possible?
I will then setup some extended monitoring if this proves to be true.
Thank you
clime
_______________________________________________
copr-devel mailing list -- copr-devel(a)lists.fedorahosted.org
To unsubscribe send an email to copr-devel-leave(a)lists.fedorahosted.org
6 years
