[copr] master: [frontend] try again if whoosh does not get lock (28aced9)
by Miroslav Suchý
Repository : http://git.fedorahosted.org/cgit/copr.git
On branch : master
>---------------------------------------------------------------
commit 28aced9cb3e360c6cb83cce7f561d4e03afd900f
Author: Miroslav Suchý <msuchy(a)redhat.com>
Date: Mon Dec 16 14:51:11 2013 +0100
[frontend] try again if whoosh does not get lock
addressing:
Exception on /backend/update/ [POST]
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/flask/app.py", line 1687, in wsgi_app
response = self.full_dispatch_request()
File "/usr/lib/python2.7/site-packages/flask/app.py", line 1360, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/lib/python2.7/site-packages/flask/app.py", line 1358, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/lib/python2.7/site-packages/flask/app.py", line 1344, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/share/copr/coprs_frontend/coprs/views/misc.py", line 141, in decorated_function
return f(*args, **kwargs)
File "/usr/share/copr/coprs_frontend/coprs/views/backend_ns/backend_general.py", line 70, in update
db.session.commit()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/scoping.py", line 149, in do
return getattr(self.registry(), name)(*args, **kwargs)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 721, in commit
self.transaction.commit()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 361, in commit
self.session.dispatch.after_commit(self.session)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/event.py", line 409, in __call__
fn(*args, **kw)
File "/usr/lib/python2.7/site-packages/flask_sqlalchemy.py", line 185, in after_commit
models_committed.send(session.app, changes=d.values())
File "/usr/lib/python2.7/site-packages/blinker/base.py", line 267, in send
for receiver in self.receivers_for(sender)]
File "/usr/lib/python2.7/site-packages/flask_whooshee.py", line 174, in on_commit
writer = wh.index.writer()
File "/usr/lib/python2.7/site-packages/whoosh/index.py", line 464, in writer
return SegmentWriter(self, **kwargs)
File "/usr/lib/python2.7/site-packages/whoosh/writing.py", line 502, in __init__
raise LockError
LockError
This has been reported as https://github.com/bkabrda/flask-whooshee/issues/3
but in mean time do this hack.
>---------------------------------------------------------------
.../coprs/views/backend_ns/backend_general.py | 17 +++++++++++++++--
1 files changed, 15 insertions(+), 2 deletions(-)
diff --git a/coprs_frontend/coprs/views/backend_ns/backend_general.py b/coprs_frontend/coprs/views/backend_ns/backend_general.py
index c76c5fb..01f7fab 100644
--- a/coprs_frontend/coprs/views/backend_ns/backend_general.py
+++ b/coprs_frontend/coprs/views/backend_ns/backend_general.py
@@ -1,4 +1,6 @@
import flask
+import sys
+import time
from coprs import db
from coprs.logic import actions_logic
@@ -6,7 +8,7 @@ from coprs.logic import builds_logic
from coprs.views import misc
from coprs.views.backend_ns import backend_ns
-
+from whoosh.index import LockError
@backend_ns.route('/waiting/')
@misc.backend_authenticated
@@ -67,7 +69,18 @@ def update():
for i, obj in existing.items():
logic_cls.update_state_from_dict(obj, to_update[i])
- db.session.commit()
+ i = 5
+ exc_info = None
+ while i > 0:
+ try:
+ db.session.commit()
+ i = -100
+ except LockError:
+ i -= 1
+ exc_info = sys.exc_info()[2]
+ time.sleep(5)
+ if i == -100:
+ raise LockError, None, exc_info
result.update({'updated_{0}_ids'.format(typ): list(existing.keys()),
'non_existing_{0}_ids'.format(typ): non_existing_ids})
10 years, 4 months
[copr] master: [backend] if frontend does not respond, repeat (5ca9f74)
by Miroslav Suchý
Repository : http://git.fedorahosted.org/cgit/copr.git
On branch : master
>---------------------------------------------------------------
commit 5ca9f743ffc84ee243d53f6ca266f145f8946214
Author: Miroslav Suchý <msuchy(a)redhat.com>
Date: Mon Dec 16 14:36:25 2013 +0100
[backend] if frontend does not respond, repeat
addressing:
Traceback (most recent call last):
File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/share/copr/backend/dispatcher.py", line 320, in run
self.mark_started(job)
File "/usr/share/copr/backend/dispatcher.py", line 253, in mark_started
raise errors.CoprWorkerError, "Could not communicate to front end to submit status info"
CoprWorkerError: Could not communicate to front end to submit status info
>---------------------------------------------------------------
backend/dispatcher.py | 12 +++++++++---
1 files changed, 9 insertions(+), 3 deletions(-)
diff --git a/backend/dispatcher.py b/backend/dispatcher.py
index 83053d1..fee9f3d 100644
--- a/backend/dispatcher.py
+++ b/backend/dispatcher.py
@@ -232,9 +232,15 @@ class Worker(multiprocessing.Process):
# maybe we move this to the callback?
def post_to_frontend(self, data):
"""send data to frontend"""
- result = self.frontend_callback.post_to_frontend(data)
- if not result:
- self.callback.log(self.frontend_callback.msg)
+ i = 10
+ while i > 0:
+ result = self.frontend_callback.post_to_frontend(data)
+ if not result:
+ self.callback.log(self.frontend_callback.msg)
+ i -= 1
+ time.sleep(5)
+ else:
+ i = 0
return result
# maybe we move this to the callback?
10 years, 4 months
[copr] master: print yum repos nicely (d914a2f)
by Miroslav Suchý
Repository : http://git.fedorahosted.org/cgit/copr.git
On branch : master
>---------------------------------------------------------------
commit d914a2f3fd9cc76211ed6de42235d92999abaf9c
Author: Miroslav Suchý <msuchy(a)redhat.com>
Date: Fri Dec 13 19:12:32 2013 +0100
print yum repos nicely
>---------------------------------------------------------------
copr_cli/subcommands.py | 10 ++++++----
1 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/copr_cli/subcommands.py b/copr_cli/subcommands.py
index b727dad..eb11397 100644
--- a/copr_cli/subcommands.py
+++ b/copr_cli/subcommands.py
@@ -82,19 +82,21 @@ def listcoprs(username=None):
if 'description' in repo:
desc = repo['description']
- print ' ' * 5, 'Description: {0}'.format(desc)
+ print ' ' * 2, 'Description: {0}'.format(desc)
if 'yum_repos' in repo:
yum_repos = repo['yum_repos']
- print ' ' * 5, 'Yum repo: {0}'.format(yum_repos)
+ print ' ' * 2, 'Yum repo(s):'
+ for k in sorted(yum_repos.keys()):
+ print ' ' * 4, '{0}: {1}'.format(k, yum_repos[k])
if 'additional_repos' in repo:
add_repos = repo['additional_repos']
- print ' ' * 5, 'Additional repos: {0}'.format(add_repos)
+ print ' ' * 2, 'Additional repos: {0}'.format(add_repos)
if 'instructions' in repo:
instructions = repo['instructions']
- print ' ' * 5, 'Instructions: {0}'.format(instructions)
+ print ' ' * 2, 'Instructions: {0}'.format(instructions)
else:
print 'No copr retrieved for user: "{0}"'.format(
user['username'])
10 years, 4 months
[copr] master: Bump the copr-cli release to 0.2.0 with all the changes made (0f7c151)
by Miroslav Suchý
Repository : http://git.fedorahosted.org/cgit/copr.git
On branch : master
>---------------------------------------------------------------
commit 0f7c1515ef3f14b63511e207ca700868a6049437
Author: Pierre-Yves Chibon <pingou(a)pingoured.fr>
Date: Wed Nov 27 20:42:08 2013 +0100
Bump the copr-cli release to 0.2.0 with all the changes made
>---------------------------------------------------------------
copr_cli/main.py | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/copr_cli/main.py b/copr_cli/main.py
index 8983159..25727a7 100644
--- a/copr_cli/main.py
+++ b/copr_cli/main.py
@@ -6,7 +6,7 @@ import sys
import subcommands
import copr_exceptions
-__version__ = '0.1.0'
+__version__ = '0.2.0'
__description__ = "CLI tool to run copr"
10 years, 4 months
[copr] master: Refer to the man page for more information about the configuration file for copr-cli (d91e203)
by Miroslav Suchý
Repository : http://git.fedorahosted.org/cgit/copr.git
On branch : master
>---------------------------------------------------------------
commit d91e203676ef88507edca3a89988bda69a7a9261
Author: Pierre-Yves Chibon <pingou(a)pingoured.fr>
Date: Wed Nov 27 20:42:07 2013 +0100
Refer to the man page for more information about the configuration file for copr-cli
>---------------------------------------------------------------
copr_cli/subcommands.py | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/copr_cli/subcommands.py b/copr_cli/subcommands.py
index 7b3f8ad..b727dad 100644
--- a/copr_cli/subcommands.py
+++ b/copr_cli/subcommands.py
@@ -23,7 +23,8 @@ def get_user():
if not config.read(
os.path.join(os.path.expanduser('~'), '.config', 'copr')):
raise copr_exceptions.CoprCliNoConfException(
- 'No configuration file "~/.config/copr" found.')
+ 'No configuration file "~/.config/copr" found. '
+ 'See man copr-cli for more information')
try:
username = config.get('copr-cli', 'username', None)
login = config.get('copr-cli', 'login', None)
10 years, 4 months
[copr] master: Rework the layout of the list command (c81e6c2)
by Miroslav Suchý
Repository : http://git.fedorahosted.org/cgit/copr.git
On branch : master
>---------------------------------------------------------------
commit c81e6c2336e4a98d6e71a6397357235ed2d39f8e
Author: Miroslav Suchý <msuchy(a)redhat.com>
Date: Fri Dec 13 18:59:17 2013 +0100
Rework the layout of the list command
Instead of the table that was getting completely twisted by large description, use
a more simpler tree/list type of layout.
Contributed by pingou, but I had to rebase on current master.
>---------------------------------------------------------------
copr_cli/subcommands.py | 78 ++++++++++++++--------------------------------
1 files changed, 24 insertions(+), 54 deletions(-)
diff --git a/copr_cli/subcommands.py b/copr_cli/subcommands.py
index 5e3a7d4..7b3f8ad 100644
--- a/copr_cli/subcommands.py
+++ b/copr_cli/subcommands.py
@@ -20,8 +20,8 @@ import copr_exceptions
def get_user():
""" Retrieve the user information from the config file. """
config = ConfigParser.ConfigParser()
- if not config.read(os.path.join(os.path.expanduser('~'), '.config',
- 'copr')):
+ if not config.read(
+ os.path.join(os.path.expanduser('~'), '.config', 'copr')):
raise copr_exceptions.CoprCliNoConfException(
'No configuration file "~/.config/copr" found.')
try:
@@ -73,62 +73,32 @@ def listcoprs(username=None):
return
output = json.loads(req.text)
- columns = []
- values = []
+
if 'repos' in output:
if output['repos']:
- columns = ['name', 'description', 'instructions']
- values = []
- for entry in output['repos']:
- values.append([entry[key] for key in columns])
- else:
- columns = ['output']
- values = ['No copr retrieved for user: "{0}"'.format(
- user['username'])]
- else:
- columns = ['output']
- values = ['Wrong output format returned by the server']
-
- def _list_to_row(values, widths):
- ''' Return a print ready version of the provided list '''
- row = []
- cnt = 0
- for item in values:
- max_width = widths[cnt]
- cnt += 1
- if not item:
- item = ''
- if cnt < len(values):
- row.append(item.ljust(max_width + 1))
- else:
- row.append(item)
- return row
-
- if len(columns) > 1:
- widths = {}
- cnt = 0
- for item in columns:
- widths[cnt] = len(item)
- cnt += 1
- for row in values:
- cnt = 0
- for item in row:
- if not item:
- item = ''
- widths[cnt] = max(widths[cnt], len(item))
- cnt += 1
-
- headers = '|'.join(_list_to_row(columns, widths))
- print headers
- print '-' * len(headers)
- for row in values:
- print "|".join(_list_to_row(row, widths))
+ for repo in output['repos']:
+ print 'Name: {0}'.format(repo['name'])
+
+ if 'description' in repo:
+ desc = repo['description']
+ print ' ' * 5, 'Description: {0}'.format(desc)
+ if 'yum_repos' in repo:
+ yum_repos = repo['yum_repos']
+ print ' ' * 5, 'Yum repo: {0}'.format(yum_repos)
+
+ if 'additional_repos' in repo:
+ add_repos = repo['additional_repos']
+ print ' ' * 5, 'Additional repos: {0}'.format(add_repos)
+
+ if 'instructions' in repo:
+ instructions = repo['instructions']
+ print ' ' * 5, 'Instructions: {0}'.format(instructions)
+ else:
+ print 'No copr retrieved for user: "{0}"'.format(
+ user['username'])
else:
- headers = columns[0]
- print headers
- print "-"*len(headers)
- print values[0]
+ print 'Un-expected data returned, please report this issue'
def create(name, chroots=[], description=None, instructions=None,
10 years, 4 months
[copr] master: Fix parsing the copr_url from the configuration file (05be171)
by Miroslav Suchý
Repository : http://git.fedorahosted.org/cgit/copr.git
On branch : master
>---------------------------------------------------------------
commit 05be17196bf491facfe7a2f436b276d62c543fef
Author: Pierre-Yves Chibon <pingou(a)pingoured.fr>
Date: Wed Nov 27 20:42:04 2013 +0100
Fix parsing the copr_url from the configuration file
This was resulting in exceptions beeing thrown with a command as simple as:
`copr_cli list pingou`
>---------------------------------------------------------------
copr_cli/subcommands.py | 14 +++++++++-----
1 files changed, 9 insertions(+), 5 deletions(-)
diff --git a/copr_cli/subcommands.py b/copr_cli/subcommands.py
index 171e4f5..5e3a7d4 100644
--- a/copr_cli/subcommands.py
+++ b/copr_cli/subcommands.py
@@ -36,11 +36,15 @@ def get_user():
def get_api_url():
""" Retrieve the user information from the config file. """
- config = ConfigParser.ConfigParser(
- {'copr_url': 'http://copr-fe.cloud.fedoraproject.org'})
- config.read(os.path.join(os.path.expanduser('~'), '.config',
- 'copr'))
- copr_url = config.get('copr-cli', 'copr_url')
+ config = ConfigParser.ConfigParser()
+ config.read(
+ os.path.join(os.path.expanduser('~'), '.config', 'copr')
+ )
+
+ # Default copr_url:
+ copr_url = 'http://copr-fe.cloud.fedoraproject.org/'
+ if config.has_section('copr-cli'):
+ copr_url = config.get('copr-cli', 'copr_url')
return '%s/api' % copr_url
10 years, 4 months
[copr] master: [backend] run createrepo as copr user (c689109)
by Miroslav Suchý
Repository : http://git.fedorahosted.org/cgit/copr.git
On branch : master
>---------------------------------------------------------------
commit c68910906d7b7aa90abc152c623af6ec970af204
Author: Miroslav Suchý <msuchy(a)redhat.com>
Date: Thu Dec 12 16:28:16 2013 +0100
[backend] run createrepo as copr user
otherwise this update will be run as root and repodata will be owned by root
and next update of repodata (initiated by next build) will fail, because
copr user could not overwrite files of root user
>---------------------------------------------------------------
copr-prune-repo | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/copr-prune-repo b/copr-prune-repo
index 7d8a3e9..4db116c 100755
--- a/copr-prune-repo
+++ b/copr-prune-repo
@@ -44,7 +44,7 @@ for REPO in $( ls -d */*/* | grep -v '/repodata$' ); do
rm -f $LATEST_PKGS $ERR_LOG
# unless we previosly deleted the repo, regen metadata
if [ -d $(pwd) ]; then
- createrepo --update -q . &>/dev/null || createrepo --update -q .
+ runuser -c "createrepo --update -q $(pwd)" - copr &>/dev/null || runuser -c "createrepo --update -q $(pwd)" - copr
fi
popd >/dev/null
done
10 years, 4 months
[copr] master: 1040615 - wrap lines with long URL (a3ce565)
by Miroslav Suchý
Repository : http://git.fedorahosted.org/cgit/copr.git
On branch : master
>---------------------------------------------------------------
commit a3ce5656a69ea894dd5c293394969df0bb56a2bd
Author: Miroslav Suchý <msuchy(a)redhat.com>
Date: Wed Dec 11 18:42:08 2013 +0100
1040615 - wrap lines with long URL
>---------------------------------------------------------------
coprs_frontend/coprs/static/copr.css | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/coprs_frontend/coprs/static/copr.css b/coprs_frontend/coprs/static/copr.css
index a168439..9d7322c 100644
--- a/coprs_frontend/coprs/static/copr.css
+++ b/coprs_frontend/coprs/static/copr.css
@@ -208,7 +208,7 @@ div.horizontal-menu li.selected a, div.horizontal-menu li.hovered a {
}
div.pkg-url-list {
- white-space: pre;
+ white-space: pre-wrap;
background-color: #f9f9f9;
font-family: monospace;
padding: 1em;
10 years, 4 months