September 2015 - copr-commits - Fedora Mailing-Lists

[copr] master: try more fix for jenkins (d733375)

by vgologuz＠fedoraproject.org

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit d733375e55fc4d99c6c7c89d2e45aa57c424165b Author: Valentin Gologuzov <vgologuz(a)redhat.com> Date: Thu Sep 10 14:26:14 2015 +0200 try more fix for jenkins >--------------------------------------------------------------- frontend/coprs_frontend/coprs/rest_api/common.py | 12 ++++++------ frontend/coprs_frontend/manage.py | 2 +- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/frontend/coprs_frontend/coprs/rest_api/common.py b/frontend/coprs_frontend/coprs/rest_api/common.py index 1945ce8..061627d 100644 --- a/frontend/coprs_frontend/coprs/rest_api/common.py +++ b/frontend/coprs_frontend/coprs/rest_api/common.py @@ -7,7 +7,7 @@ from logging import getLogger from flask import url_for import flask -from .. import models +from ..models import User, Copr, BuildChroot, Build from ..logic.users_logic import UsersLogic from ..logic.builds_logic import BuildsLogic from ..logic.coprs_logic import CoprsLogic @@ -48,7 +48,7 @@ def render_build(build, self_params=None): def render_project(project, self_params=None): """ - :param models.Copr project: + :param Copr project: """ if self_params is None: self_params = {} @@ -66,7 +66,7 @@ def render_project(project, self_params=None): def render_build_task(chroot): """ - :type chroot: models.BuildChroot + :type chroot: BuildChroot """ return { "build_task": mm_serialize_one(BuildTaskSchema, chroot), @@ -118,7 +118,7 @@ def rest_api_auth_required(f): def get_project_safe(project_id): """ :param int project_id: - :rtype: models.Copr + :rtype: Copr """ return get_one_safe( CoprsLogic.get_by_id(project_id), @@ -130,7 +130,7 @@ def get_project_safe(project_id): def get_build_safe(build_id): """ :param int build_id: - :rtype: models.Build + :rtype: Build """ return get_one_safe( BuildsLogic.get(build_id), @@ -142,7 +142,7 @@ def get_build_safe(build_id): def get_user_safe(username): """ :param str username: - :rtype: models.User + :rtype: User """ return get_one_safe( UsersLogic.get(username), diff --git a/frontend/coprs_frontend/manage.py b/frontend/coprs_frontend/manage.py index 3ec8f66..abe3fb9 100755 --- a/frontend/coprs_frontend/manage.py +++ b/frontend/coprs_frontend/manage.py @@ -191,7 +191,7 @@ class AddDebugUserCommand(Command): """ def run(self, name, mail, **kwargs): - user = models.User(username=name, mail=mail) + user = User(username=name, mail=mail) if kwargs["admin"]: user.admin = True

8 years, 8 months

1
0
0 / 0

[copr] master: minor (e5f723c)

by vgologuz＠fedoraproject.org

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit e5f723c88762ad55c97ac341c236857cafcc7d86 Author: Valentin Gologuzov <vgologuz(a)redhat.com> Date: Thu Sep 10 14:19:14 2015 +0200 minor >--------------------------------------------------------------- frontend/docs/api_2/source/Resources/build.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/frontend/docs/api_2/source/Resources/build.rst b/frontend/docs/api_2/source/Resources/build.rst index 517df70..f90b5ef 100644 --- a/frontend/docs/api_2/source/Resources/build.rst +++ b/frontend/docs/api_2/source/Resources/build.rst @@ -8,8 +8,8 @@ is available through sub-resource :doc:`./build_task`. -Structure of the build ----------------------- +Structure of the build entity +----------------------------- .. code-block:: javascript

8 years, 8 months

1
0
0 / 0

[copr] master: fixes for jenkins tests (15a8986)

by vgologuz＠fedoraproject.org

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit 15a89862046844cbde00ab8ffeb1061da856d651 Author: Valentin Gologuzov <vgologuz(a)redhat.com> Date: Thu Sep 10 14:08:16 2015 +0200 fixes for jenkins tests >--------------------------------------------------------------- backend/requirements.txt | 1 + test_suite.sh | 2 +- 2 files changed, 2 insertions(+), 1 deletions(-) diff --git a/backend/requirements.txt b/backend/requirements.txt index 386fbc3..325fd08 100644 --- a/backend/requirements.txt +++ b/backend/requirements.txt @@ -11,3 +11,4 @@ sphinx sphinx-argparse netaddr psutil +munch diff --git a/test_suite.sh b/test_suite.sh index ca50848..6ec0ecb 100644 --- a/test_suite.sh +++ b/test_suite.sh @@ -33,7 +33,7 @@ python -m pytest python/copr/test --junitxml=_report/python-copr.junit.xml --co mv {,_report/python-copr.}coverage.xml COPR_CONFIG="$(pwd)/frontend/coprs_frontend/config/copr_unit_test.conf" \ - python -m pytest frontend/coprs_frontend/tests --junitxml=_report/frontend.junit.xml --cov-report xml --cov frontend/coprs_frontend/coprs $@ +PYTHONPATH=frontend/coprs_frontend/:$PYTHONPATH python -m pytest frontend/coprs_frontend/tests --junitxml=_report/frontend.junit.xml --cov-report xml --cov frontend/coprs_frontend/coprs $@ mv {,_report/frontend.}coverage.xml PYTHONPATH=backend/run:backend:python:$PYTHONPATH python -m pytest backend/tests --junitxml=_report/backend.junit.xml \

8 years, 8 months

1
0
0 / 0

[copr] master: [backend] fixed unittests (f468112)

by vgologuz＠fedoraproject.org

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit f468112ffc4ba5e57e262f63a9438c3a2921ef79 Author: Valentin Gologuzov <vgologuz(a)redhat.com> Date: Thu Sep 10 12:43:54 2015 +0200 [backend] fixed unittests >--------------------------------------------------------------- backend/backend/daemons/vm_master.py | 9 ++- backend/backend/vm_manage/manager.py | 3 - backend/tests/deamons/test_vm_master.py | 128 +++++++++++++++++------- backend/tests/vm_manager/test_event_handle.py | 22 ++--- backend/tests/vm_manager/test_manager.py | 63 +------------ 5 files changed, 110 insertions(+), 115 deletions(-) diff --git a/backend/backend/daemons/vm_master.py b/backend/backend/daemons/vm_master.py index c6a61c8..3d75413 100644 --- a/backend/backend/daemons/vm_master.py +++ b/backend/backend/daemons/vm_master.py @@ -11,6 +11,7 @@ import time from setproctitle import setproctitle import traceback import sys +import types import psutil from ..constants import JOB_GRAB_TASK_END_PUBSUB @@ -25,7 +26,9 @@ class VmMaster(Process): """ Spawns and terminate VM for builder process. - :type vm_manager: backend.vm_manage.manager.VmManager + :type vmm: backend.vm_manage.manager.VmManager + :type spawner: backend.vm_manage.spawn.Spawner + :type checker: backend.vm_manage.check.HealthChecker """ def __init__(self, opts, vmm, spawner, checker): super(VmMaster, self).__init__(name="vm_master") @@ -273,6 +276,10 @@ class VmMaster(Process): def terminate(self): self.kill_received = True + if self.spawner is not None: + self.spawner.terminate() + if self.checker is not None: + self.checker.terminate() def finalize_long_health_checks(self): """ diff --git a/backend/backend/vm_manage/manager.py b/backend/backend/vm_manage/manager.py index 11010b0..8787811 100644 --- a/backend/backend/vm_manage/manager.py +++ b/backend/backend/vm_manage/manager.py @@ -120,9 +120,6 @@ class VmManager(object): :type logger: logging.Logger :param logger: Logger instance to use inside of manager, if None manager would create new logger using helpers.get_redis_logger - - :param checker: object with method `check_health(ip) -> None or raise exception` - :param terminator: object with safe method `terminate(ip, vm_name)` """ def __init__(self, opts, logger=None): diff --git a/backend/tests/deamons/test_vm_master.py b/backend/tests/deamons/test_vm_master.py index df236d8..688b927 100644 --- a/backend/tests/deamons/test_vm_master.py +++ b/backend/tests/deamons/test_vm_master.py @@ -133,17 +133,22 @@ class TestVmMaster(object): self.terminator = MagicMock() self.mc_logger = MagicMock() - self.vmm = VmManager(self.opts, logger=self.mc_logger, - checker=self.checker, - spawner=self.spawner, - terminator=self.terminator) + self.vmm = VmManager(self.opts, logger=self.mc_logger) self.vmm.post_init() self.event_handler = MagicMock() - self.vm_master = VmMaster(self.vmm) + self.vm_master = VmMaster( + self.opts, + self.vmm, + self.spawner, + self.checker, + ) self.vm_master.event_handler = MagicMock() self.pid = 12345 + self.vm_ip = "127.0.0.1" + self.vm_name = "build 12345" + def clean_redis(self): keys = self.vmm.rc.keys("*") if keys: @@ -264,7 +269,7 @@ class TestVmMaster(object): # assert set(["2", "4"]) == set([json.loads(m["data"])["build_id"] for m in msg_list]) def test_check_vms_health(self, mc_time, add_vmd): - self.vmm.start_vm_check = types.MethodType(MagicMock(), self.vmm) + self.vm_master.start_vm_check = types.MethodType(MagicMock(), self.vmm) for vmd in [self.vmd_a1, self.vmd_a2, self.vmd_a3, self.vmd_b1, self.vmd_b2, self.vmd_b3]: vmd.store_field(self.rc, "last_health_check", 0) @@ -277,18 +282,19 @@ class TestVmMaster(object): mc_time.time.return_value = 1 self.vm_master.check_vms_health() - assert not self.vmm.start_vm_check.called + assert not self.vm_master.start_vm_check.called mc_time.time.return_value = 1 + self.opts.build_groups[0]["vm_health_check_period"] self.vm_master.check_vms_health() - to_check = set(call[0][1] for call in self.vmm.start_vm_check.call_args_list) + to_check = set(call[0][1] for call in self.vm_master.start_vm_check.call_args_list) assert set(['a1', 'a3', 'b1', 'b2']) == to_check - self.vmm.start_vm_check.reset_mock() + self.vm_master.start_vm_check.reset_mock() for vmd in [self.vmd_a1, self.vmd_a2, self.vmd_a3, self.vmd_b1, self.vmd_b2, self.vmd_b3]: - self.rc.hdel(self.vmd_a3.vm_key, "last_health_check") + self.rc.hdel(vmd.vm_key, "last_health_check") + self.vm_master.check_vms_health() - to_check = set(call[0][1] for call in self.vmm.start_vm_check.call_args_list) + to_check = set(call[0][1] for call in self.vm_master.start_vm_check.call_args_list) assert set(['a1', 'a3', 'b1', 'b2']) == to_check def test_finalize_long_health_checks(self, mc_time, add_vmd): @@ -362,15 +368,15 @@ class TestVmMaster(object): assert self.vmm.remove_vm_from_pool.call_args[0][0] == self.vmd_a1.vm_name assert not self.vmm.start_vm_termination.called - def test_run_undefined_helpers(self, mc_setproctitle): - for target in ["spawner", "terminator", "checker"]: - setattr(self.vmm, target, None) - with pytest.raises(RuntimeError): - self.vm_master.run() - - setattr(self.vmm, target, MagicMock()) - - assert not mc_setproctitle.called + # def test_run_undefined_helpers(self, mc_setproctitle): + # for target in ["spawner", "terminator", "checker"]: + # setattr(self.vmm, target, None) + # with pytest.raises(RuntimeError): + # self.vm_master.run() + # + # setattr(self.vmm, target, MagicMock()) + # + # assert not mc_setproctitle.called def test_dummy_run(self, mc_time, mc_setproctitle): mc_do_cycle = MagicMock() @@ -391,19 +397,13 @@ class TestVmMaster(object): self.vm_master.kill_received = True mc_time.sleep.side_effect = on_sleep - with mock.patch("{}.EventHandler".format(MODULE_REF)) as mc_event_handler: - - self.vm_master.run() - - assert mc_event_handler.called - assert mc_event_handler.return_value.start.called - - assert self.vmm.mark_server_start.called + self.vm_master.run() def test_dummy_terminate(self): self.vm_master.terminate() assert self.vm_master.kill_received - assert self.vm_master.event_handler.terminate.called + assert self.vm_master.checker.terminate.called + assert self.vm_master.spawner.terminate.called def test_dummy_do_cycle(self): self.vm_master.remove_old_dirty_vms = types.MethodType(MagicMock(), self.vm_master) @@ -416,8 +416,7 @@ class TestVmMaster(object): assert self.vm_master.remove_old_dirty_vms.called assert self.vm_master.check_vms_health.called assert self.vm_master.start_spawn_if_required.called - - assert self.vmm.spawner.recycle.called + assert self.vm_master.spawner.recycle.called def test_dummy_start_spawn_if_required(self): self.vm_master.try_spawn_one = MagicMock() @@ -443,7 +442,7 @@ class TestVmMaster(object): state = choice(active_vm_states) vmd_list[idx].store_field(self.rc, "state", state) - self.vmm.spawner.get_proc_num_per_group.return_value = spawn_procs_number + self.vm_master.spawner.get_proc_num_per_group.return_value = spawn_procs_number with pytest.raises(VmSpawnLimitReached): self.vm_master._check_total_running_vm_limit(0) self.vm_master.log.reset_mock() @@ -468,7 +467,7 @@ class TestVmMaster(object): state = choice(active_vm_states) vmd_list[idx].store_field(self.rc, "state", state) # self.vmm.spawner.children_number = spawn_procs_number - self.vmm.spawner.get_proc_num_per_group.return_value = spawn_procs_number + self.vm_master.spawner.get_proc_num_per_group.return_value = spawn_procs_number # doesn't raise exception self.vm_master._check_total_running_vm_limit(0) @@ -500,11 +499,11 @@ class TestVmMaster(object): def test__check_number_of_running_spawn_processes(self): for i in range(self.opts.build_groups[0]["max_spawn_processes"]): - self.vmm.spawner.get_proc_num_per_group.return_value = i + self.vm_master.spawner.get_proc_num_per_group.return_value = i self.vm_master._check_number_of_running_spawn_processes(0) for i in [0, 1, 2, 5, 100]: - self.vmm.spawner.get_proc_num_per_group.return_value = \ + self.vm_master.spawner.get_proc_num_per_group.return_value = \ self.opts.build_groups[0]["max_spawn_processes"] + i with pytest.raises(VmSpawnLimitReached): @@ -526,10 +525,10 @@ class TestVmMaster(object): mc_time.time.return_value = 0 self.vm_master.log = MagicMock() - self.vmm.spawner.start_spawn.side_effect = IOError() + self.vm_master.spawner.start_spawn.side_effect = IOError() self.vm_master.try_spawn_one(0) - assert self.vmm.spawner.start_spawn.called + assert self.vm_master.spawner.start_spawn.called def test_try_spawn_exit_on_check_fail(self): check_mocks = [] @@ -558,4 +557,57 @@ class TestVmMaster(object): self.vm_master.try_spawn_one(0) assert self.vm_master.vmm.write_vm_pool_info.called - assert self.vm_master.vmm.spawner.start_spawn.called + assert self.vm_master.spawner.start_spawn.called + + def test_start_vm_check_ok_ok(self): + self.vmm.start_vm_termination = types.MethodType(MagicMock(), self.vmm) + self.vmm.add_vm_to_pool(self.vm_ip, self.vm_name, self.group) + vmd = self.vmm.get_vm_by_name(self.vm_name) + # can start, no problem to start + # > can start IN_USE, don't change status + vmd.store_field(self.rc, "state", VmStates.IN_USE) + self.vm_master.start_vm_check(vm_name=self.vm_name) + + assert self.checker.run_check_health.called + self.checker.run_check_health.reset_mock() + assert vmd.get_field(self.rc, "state") == VmStates.IN_USE + + # > changes status to HEALTH_CHECK + states = [VmStates.GOT_IP, VmStates.CHECK_HEALTH_FAILED, VmStates.READY] + for state in states: + vmd.store_field(self.rc, "state", state) + self.vm_master.start_vm_check(vm_name=self.vm_name) + + assert self.checker.run_check_health.called + self.checker.run_check_health.reset_mock() + assert vmd.get_field(self.rc, "state") == VmStates.CHECK_HEALTH + + def test_start_vm_check_wrong_old_state(self): + self.vmm.start_vm_termination = types.MethodType(MagicMock(), self.vmm) + self.vmm.add_vm_to_pool(self.vm_ip, self.vm_name, self.group) + vmd = self.vmm.get_vm_by_name(self.vm_name) + + states = [VmStates.TERMINATING, VmStates.CHECK_HEALTH] + for state in states: + vmd.store_field(self.rc, "state", state) + assert not self.vm_master.start_vm_check(vm_name=self.vm_name) + + assert not self.checker.run_check_health.called + assert vmd.get_field(self.rc, "state") == state + + def test_start_vm_check_lua_ok_check_spawn_failed(self): + self.vmm.start_vm_termination = types.MethodType(MagicMock(), self.vmm) + self.vmm.add_vm_to_pool(self.vm_ip, self.vm_name, self.group) + vmd = self.vmm.get_vm_by_name(self.vm_name) + + self.vm_master.checker.run_check_health.side_effect = RuntimeError() + + # restore orig state + states = [VmStates.GOT_IP, VmStates.CHECK_HEALTH_FAILED, VmStates.READY, VmStates.IN_USE] + for state in states: + vmd.store_field(self.rc, "state", state) + self.vm_master.start_vm_check(vm_name=self.vm_name) + + assert self.checker.run_check_health.called + self.checker.run_check_health.reset_mock() + assert vmd.get_field(self.rc, "state") == state diff --git a/backend/tests/vm_manager/test_event_handle.py b/backend/tests/vm_manager/test_event_handle.py index 591bb8b..dc69bad 100644 --- a/backend/tests/vm_manager/test_event_handle.py +++ b/backend/tests/vm_manager/test_event_handle.py @@ -112,7 +112,9 @@ class TestEventHandle(object): self.grl_patcher = mock.patch("{}.get_redis_logger".format(MODULE_REF)) self.grl_patcher.start() - self.eh = EventHandler(self.opts, self.vmm) + self.eh = EventHandler(self.opts, + self.vmm, + self.terminator) self.eh.post_init() self.vm_ip = "127.0.0.1" @@ -134,14 +136,14 @@ class TestEventHandle(object): self.erase_redis() def test_post_init(self): - test_eh = EventHandler(self.opts, self.vmm) + test_eh = EventHandler(self.opts, self.vmm, self.terminator) assert "on_health_check_success" not in test_eh.lua_scripts test_eh.post_init() assert test_eh.lua_scripts["on_health_check_success"] assert isinstance(test_eh.lua_scripts["on_health_check_success"], Script) def test_recycle(self, mc_time): - self.recycle = Recycle(vmm=self.vmm, recycle_period=60) + self.recycle = Recycle(terminator=self.terminator, recycle_period=60) self.stage = 0 def incr(*args, **kwargs): @@ -150,10 +152,10 @@ class TestEventHandle(object): self.recycle.terminate() mc_time.sleep.side_effect = incr - assert not self.vmm.terminator.recycle.called + assert not self.terminator.recycle.called self.recycle.run() - assert self.vmm.terminator.recycle.called - assert len(self.vmm.terminator.recycle.call_args_list) == 3 + assert self.terminator.recycle.called + assert len(self.terminator.recycle.call_args_list) == 3 def test_on_vm_spawned(self): expected_call = mock.call(**self.msg) @@ -163,7 +165,7 @@ class TestEventHandle(object): def test_on_vm_termination_request(self): expected_call = mock.call(**self.msg) self.eh.on_vm_termination_request(self.msg) - assert self.vmm.terminator.terminate_vm.call_args == expected_call + assert self.terminator.terminate_vm.call_args == expected_call def test_health_check_result_no_vmd(self): self.vmm.get_vm_by_name.side_effect = VmDescriptorNotFound("foobar") @@ -281,14 +283,10 @@ class TestEventHandle(object): def test_dummy_run(self, mc_setproctitle, mc_recycle): # dummy test, mainly for perfect coverage - self.eh.post_init = types.MethodType(MagicMock(), self.eh) self.eh.start_listen = types.MethodType(MagicMock(), self.eh) - self.eh.run() - assert mc_recycle.call_args == mock.call(vmm=self.eh.vmm, recycle_period=self.eh.recycle_period) - - assert self.eh.post_init.called + assert mc_recycle.called assert self.eh.start_listen.called def test_dummy_terminate(self, mc_setproctitle, mc_recycle): diff --git a/backend/tests/vm_manager/test_manager.py b/backend/tests/vm_manager/test_manager.py index b8f60d2..5ddf248 100644 --- a/backend/tests/vm_manager/test_manager.py +++ b/backend/tests/vm_manager/test_manager.py @@ -81,16 +81,10 @@ class TestManager(object): self.log_msg_list = [] self.callback = TestCallback() - # checker = HealthChecker(self.opts, self.callback) - self.checker = MagicMock() - self.spawner = MagicMock() - self.terminator = MagicMock() self.queue = Queue() - self.vmm = VmManager(self.opts, self.queue, - checker=self.checker, - spawner=self.spawner, - terminator=self.terminator) + self.vmm = VmManager(self.opts) + self.vmm.post_init() self.vmm.log = MagicMock() self.pid = 12345 @@ -122,42 +116,6 @@ class TestManager(object): with pytest.raises(VmError): self.vmm.add_vm_to_pool(self.vm_ip, self.vm_name, self.group) - def test_start_vm_check_ok_ok(self): - self.vmm.start_vm_termination = types.MethodType(MagicMock(), self.vmm) - self.vmm.add_vm_to_pool(self.vm_ip, self.vm_name, self.group) - vmd = self.vmm.get_vm_by_name(self.vm_name) - # can start, no problem to start - # > can start IN_USE, don't change status - vmd.store_field(self.rc, "state", VmStates.IN_USE) - self.vmm.start_vm_check(vm_name=self.vm_name) - - assert self.checker.run_check_health.called - self.checker.run_check_health.reset_mock() - assert vmd.get_field(self.rc, "state") == VmStates.IN_USE - - # > changes status to HEALTH_CHECK - states = [VmStates.GOT_IP, VmStates.CHECK_HEALTH_FAILED, VmStates.READY] - for state in states: - vmd.store_field(self.rc, "state", state) - self.vmm.start_vm_check(vm_name=self.vm_name) - - assert self.checker.run_check_health.called - self.checker.run_check_health.reset_mock() - assert vmd.get_field(self.rc, "state") == VmStates.CHECK_HEALTH - - def test_start_vm_check_wrong_old_state(self): - self.vmm.start_vm_termination = types.MethodType(MagicMock(), self.vmm) - self.vmm.add_vm_to_pool(self.vm_ip, self.vm_name, self.group) - vmd = self.vmm.get_vm_by_name(self.vm_name) - - states = [VmStates.TERMINATING, VmStates.CHECK_HEALTH] - for state in states: - vmd.store_field(self.rc, "state", state) - assert not self.vmm.start_vm_check(vm_name=self.vm_name) - - assert not self.checker.run_check_health.called - assert vmd.get_field(self.rc, "state") == state - def test_mark_vm_check_failed(self, mc_time): self.vmm.start_vm_termination = types.MethodType(MagicMock(), self.vmm) self.vmm.add_vm_to_pool(self.vm_ip, self.vm_name, self.group) @@ -174,23 +132,6 @@ class TestManager(object): self.vmm.mark_vm_check_failed(self.vm_name) assert vmd.get_field(self.rc, "state") == state - def test_start_vm_check_lua_ok_check_spawn_failed(self): - self.vmm.start_vm_termination = types.MethodType(MagicMock(), self.vmm) - self.vmm.add_vm_to_pool(self.vm_ip, self.vm_name, self.group) - vmd = self.vmm.get_vm_by_name(self.vm_name) - - self.vmm.checker.run_check_health.side_effect = RuntimeError() - - # restore orig state - states = [VmStates.GOT_IP, VmStates.CHECK_HEALTH_FAILED, VmStates.READY, VmStates.IN_USE] - for state in states: - vmd.store_field(self.rc, "state", state) - self.vmm.start_vm_check(vm_name=self.vm_name) - - assert self.checker.run_check_health.called - self.checker.run_check_health.reset_mock() - assert vmd.get_field(self.rc, "state") == state - def test_acquire_vm_no_vm_after_server_restart(self, mc_time): vmd = self.vmm.add_vm_to_pool(self.vm_ip, self.vm_name, self.group) vmd.store_field(self.rc, "state", VmStates.READY)

8 years, 8 months

1
0
0 / 0

[copr] master: [docs][api 2] review of index.html (f9f7eac)

by asamalik＠fedoraproject.org

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit f9f7eac46232021b8f7e39f22bcc40f2de8e1970 Author: Adam Samalik <asamalik(a)redhat.com> Date: Wed Sep 9 16:26:08 2015 +0200 [docs][api 2] review of index.html >--------------------------------------------------------------- frontend/docs/api_2/source/index.rst | 26 ++++++++++++++------------ 1 files changed, 14 insertions(+), 12 deletions(-) diff --git a/frontend/docs/api_2/source/index.rst b/frontend/docs/api_2/source/index.rst index 6bf0743..70c614e 100644 --- a/frontend/docs/api_2/source/index.rst +++ b/frontend/docs/api_2/source/index.rst @@ -7,12 +7,12 @@ Welcome to Copr Api 2's documentation! ====================================== Welcome to the documentation of the new REST-like API for the Copr build service. -Almost all API calls is done using ``application/json`` ContentType. +Almost all API calls are done using ``application/json`` ContentType. Endpoint of the the API is ``/api_2``, public data is available without authorization. -To create new projects, submit builds and do other modification requests you will need to provide API token using -BasicAuth_ . Token can be obtained and renewed at the CoprAPI_ page. +To create new projects, submit builds and do other modification requests, you need to provide an API token using +BasicAuth_ . The token can be obtained and renewed at the CoprAPI_ page. Resources --------- @@ -30,8 +30,8 @@ HETEOAS ------- -This API implements HETEOAS_ in the very simple form: each entity is accompanied with set of relative links -to other related entities. HETEOAS makes API self discoverable, so you shouldn't learn how to access sub-resources. +This API implements HETEOAS_ in the very simple form: each entity is accompanied with a set of relative links +to other related entities. HETEOAS makes API self discoverable, so you don't need to learn how to access sub-resources. Here is a short example with the content of API root: **GET /api_2** @@ -60,13 +60,13 @@ Here is a short example with the content of API root: Response structure ------------------ -The API operates two kinds of resources: collections and individuals. - -Each entity is enveloped into json dict and accompanied with set of HETEOAS references. +Each entity is enveloped into a json dict and accompanied with set of HETEOAS references. GET requests would return the following structures: **Collection structure**: +A collection provides information about several objects. + .. code-block:: javascript { @@ -90,6 +90,8 @@ GET requests would return the following structures: **Individual structure**: +An individual provides information about one object. + .. code-block:: javascript { @@ -107,10 +109,10 @@ GET requests would return the following structures: Errors ______ -To distinguish errors we use standard HTTP codes: https://en.wikipedia.org/wiki/List_of_HTTP_status_codes. -Additional information may be contained in the response body, it SHOULD have `application/json` Content-Type. -Inside json object, there are MUST be key ``message`` with error description, -additional information MAY be present at key ``data`` +To distinguish errors, we use standard HTTP codes: https://en.wikipedia.org/wiki/List_of_HTTP_status_codes. +Additional information may be contained in the response body, which SHOULD have `application/json` Content-Type. +Inside json object, there will always be a key ``message`` with error description. In some cases, an +additional information would be present at key ``data``. **Example**

8 years, 8 months

1
0
0 / 0

[copr] master: [frontend] source info on the rebuild view (a53c553)

by asamalik＠fedoraproject.org

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit a53c55333a5100245061f0c89f55cea6372784f3 Author: Adam Samalik <asamalik(a)redhat.com> Date: Wed Sep 9 16:01:37 2015 +0200 [frontend] source info on the rebuild view >--------------------------------------------------------------- .../templates/coprs/detail/add_build/rebuild.html | 26 ++++++++++++++++++++ 1 files changed, 26 insertions(+), 0 deletions(-) diff --git a/frontend/coprs_frontend/coprs/templates/coprs/detail/add_build/rebuild.html b/frontend/coprs_frontend/coprs/templates/coprs/detail/add_build/rebuild.html index f42fcda..984a0d2 100644 --- a/frontend/coprs_frontend/coprs/templates/coprs/detail/add_build/rebuild.html +++ b/frontend/coprs_frontend/coprs/templates/coprs/detail/add_build/rebuild.html @@ -1,5 +1,6 @@ {% extends "coprs/detail.html" %} {% from "coprs/detail/_builds_forms.html" import copr_build_form_rebuild with context %} +{% from "coprs/detail/_describe_source.html" import describe_source %} {% block new_build_rebuild_selected %}active{% endblock %} @@ -23,6 +24,31 @@ <div class="col-sm-12"> <h2> Resubmit build {{ build.id }} </h2> <p> Resubmitting a build will rebuild the same sources again. </p> + + + <h3> Original Build Details </h3> + <dl class="dl-horizontal"> + <dt> Package:</dt> + <dd> + {% if build.package %} + <a href="{{url_for('coprs_ns.copr_package', username=build.copr.owner.name, package_name=build.package.name, coprname=copr.name)}}"> + {{ build.package.name }} + </a> + {% else %} + - + {% endif %} + </dd> + <dt> Version:</dt> + <dd> + {% if build.pkg_version %} + {{ build.pkg_version}} + {% else %} + - + {% endif %} + </dd> + {{ describe_source(build) }} + </dl> + <h3> New Build Options </h3> {{ copr_build_form_rebuild(form, 'coprs_ns.copr_new_build_rebuild', copr, build) }} </div> </div>

8 years, 8 months

1
0
0 / 0

[copr] master: [frontend][API] Added query parameter `is_finished` to the Builds collection; fixed parsing of the boolean query args (e768f7e)

by vgologuz＠fedoraproject.org

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit e768f7e720fd16a44e792164509f94d514ec2e59 Author: Valentin Gologuzov <vgologuz(a)redhat.com> Date: Wed Sep 9 15:27:57 2015 +0200 [frontend][API] Added query parameter `is_finished` to the Builds collection; fixed parsing of the boolean query args >--------------------------------------------------------------- .../coprs/rest_api/resources/build.py | 7 +- .../coprs/rest_api/resources/mock_chroot.py | 4 +- .../coprs/rest_api/resources/project.py | 6 +- .../coprs/rest_api/resources/project_chroot.py | 3 - frontend/coprs_frontend/coprs/rest_api/util.py | 21 +++++- frontend/coprs_frontend/tests/coprs_test_case.py | 7 ++- .../coprs_frontend/tests/test_api/test_build_r.py | 76 +++++++++++++------- .../tests/test_api/test_build_task_r.py | 78 +++++++++++++++++++- 8 files changed, 161 insertions(+), 41 deletions(-) diff --git a/frontend/coprs_frontend/coprs/rest_api/resources/build.py b/frontend/coprs_frontend/coprs/rest_api/resources/build.py index bc1605f..082bb25 100644 --- a/frontend/coprs_frontend/coprs/rest_api/resources/build.py +++ b/frontend/coprs_frontend/coprs/rest_api/resources/build.py @@ -12,7 +12,7 @@ from ..common import get_project_safe from ..exceptions import MalformedRequest, CannotProcessRequest, AccessForbidden from ..common import render_build, rest_api_auth_required, render_build_task, get_build_safe, get_user_safe from ..schemas import BuildSchema, BuildCreateSchema, BuildCreateFromUrlSchema -from ..util import mm_deserialize, get_request_parser +from ..util import mm_deserialize, get_request_parser, arg_bool class BuildListR(Resource): @@ -27,7 +27,7 @@ class BuildListR(Resource): parser.add_argument('limit', type=int) parser.add_argument('offset', type=int) - parser.add_argument('is_finished', type=bool) + parser.add_argument('is_finished', type=arg_bool) # parser.add_argument('package', type=str) req_args = parser.parse_args() @@ -44,7 +44,6 @@ class BuildListR(Resource): if req_args["is_finished"] is not None: is_finished = req_args["is_finished"] query = BuildsLogic.filter_is_finished(query, is_finished) - # TODO: add test! if req_args["limit"] is not None: limit = req_args["limit"] @@ -161,7 +160,7 @@ class BuildR(Resource): def get(self, build_id): parser = get_request_parser() - parser.add_argument('show_build_tasks', type=bool, default=False) + parser.add_argument('show_build_tasks', type=arg_bool, default=False) req_args = parser.parse_args() build = get_build_safe(build_id) diff --git a/frontend/coprs_frontend/coprs/rest_api/resources/mock_chroot.py b/frontend/coprs_frontend/coprs/rest_api/resources/mock_chroot.py index a81569c..262e57f 100644 --- a/frontend/coprs_frontend/coprs/rest_api/resources/mock_chroot.py +++ b/frontend/coprs_frontend/coprs/rest_api/resources/mock_chroot.py @@ -6,7 +6,7 @@ from flask_restful import Resource, reqparse from ...logic.coprs_logic import MockChrootsLogic from ..schemas import MockChrootSchema -from ..util import get_one_safe, get_request_parser +from ..util import get_one_safe, get_request_parser, arg_bool def render_mock_chroot(chroot): @@ -22,7 +22,7 @@ class MockChrootListR(Resource): def get(self): parser = get_request_parser() - parser.add_argument('active_only', type=bool) + parser.add_argument('active_only', type=arg_bool) req_args = parser.parse_args() active_only = False if req_args["active_only"]: diff --git a/frontend/coprs_frontend/coprs/rest_api/resources/project.py b/frontend/coprs_frontend/coprs/rest_api/resources/project.py index e3157b8..a9a84f0 100644 --- a/frontend/coprs_frontend/coprs/rest_api/resources/project.py +++ b/frontend/coprs_frontend/coprs/rest_api/resources/project.py @@ -16,7 +16,7 @@ from ...exceptions import DuplicateException from ..common import rest_api_auth_required, render_copr_chroot, render_build, render_project, get_project_safe from ..schemas import ProjectSchema, ProjectCreateSchema from ..exceptions import ObjectAlreadyExists, CannotProcessRequest, AccessForbidden -from ..util import mm_deserialize, get_request_parser +from ..util import mm_deserialize, get_request_parser, arg_bool log = getLogger(__name__) @@ -111,8 +111,8 @@ class ProjectR(Resource): def get(self, project_id): parser = get_request_parser() - parser.add_argument('show_builds', type=bool, default=False) - parser.add_argument('show_chroots', type=bool, default=False) + parser.add_argument('show_builds', type=arg_bool, default=False) + parser.add_argument('show_chroots', type=arg_bool, default=False) req_args = parser.parse_args() project = get_project_safe(project_id) diff --git a/frontend/coprs_frontend/coprs/rest_api/resources/project_chroot.py b/frontend/coprs_frontend/coprs/rest_api/resources/project_chroot.py index ff7a3b1..dfefbcf 100644 --- a/frontend/coprs_frontend/coprs/rest_api/resources/project_chroot.py +++ b/frontend/coprs_frontend/coprs/rest_api/resources/project_chroot.py @@ -60,9 +60,6 @@ class ProjectChrootListR(Resource): if get_one_safe(CoprChrootsLogic.get_by_name(project, name)) is not None: raise ObjectAlreadyExists("Copr {} already has chroot {} enabled" .format(project.full_name, name)) - else: - raise ServerError("Unexpected error, contact site administrator: {}" - .format(err)) resp = make_response("", 201) resp.headers["Location"] = url_for(".projectchrootr", diff --git a/frontend/coprs_frontend/coprs/rest_api/util.py b/frontend/coprs_frontend/coprs/rest_api/util.py index 402edd1..ef5a3e7 100644 --- a/frontend/coprs_frontend/coprs/rest_api/util.py +++ b/frontend/coprs_frontend/coprs/rest_api/util.py @@ -82,4 +82,23 @@ class MyArg(Argument): def get_request_parser(): return RequestParser(argument_class=MyArg) - #return RequestParser(bundle_errors=True) + + +def arg_bool(value): + """ + :param str value: value to convert + :rtype: bool + :raises ValueError: + """ + true_values = ["true", "yes", "1"] + false_values = ["false", "no", "0"] + value = str(value).strip().lower() + + if value in true_values: + return True + elif value in false_values: + return False + + raise ValueError("Value `{}` doesn't belong to either true of false sets. " + "Allowed values: {} and {}" + .format(value, true_values, false_values)) diff --git a/frontend/coprs_frontend/tests/coprs_test_case.py b/frontend/coprs_frontend/tests/coprs_test_case.py index a6dff96..6da78d1 100644 --- a/frontend/coprs_frontend/tests/coprs_test_case.py +++ b/frontend/coprs_frontend/tests/coprs_test_case.py @@ -218,6 +218,7 @@ class CoprsTestCase(object): [self.b1, self.b2, self.b3, self.b4], [self.b1_bc, self.b2_bc, self.b3_bc, self.b4_bc]): + # import ipdb; ipdb.set_trace() status = None if build is self.b1: # this build is going to be deleted status = StatusEnum("succeeded") @@ -229,11 +230,12 @@ class CoprsTestCase(object): git_hash="12345", ) - if build is [self.b1, self.b2]: + if build is self.b1 or build is self.b2: buildchroot.started_on = 139086644000 buildchroot.ended_on = 149086644000 build.ended_on = 149086644000 + build_chroots.append(buildchroot) self.db.session.add(buildchroot) @@ -275,6 +277,9 @@ class CoprsTestCase(object): @pytest.fixture def f_build_many_chroots(self): + """ + Requires: f_mock_chroots_many + """ self.b_many_chroots = models.Build( id=12347, copr=self.c1, user=self.u1, diff --git a/frontend/coprs_frontend/tests/test_api/test_build_r.py b/frontend/coprs_frontend/tests/test_api/test_build_r.py index bcc5cf7..ae5bee3 100644 --- a/frontend/coprs_frontend/tests/test_api/test_build_r.py +++ b/frontend/coprs_frontend/tests/test_api/test_build_r.py @@ -20,7 +20,7 @@ class TestBuildResource(CoprsTestCase): for b_dict in response_object["builds"] ]) - def test_collection_ok(self, f_users, f_coprs, f_builds, f_db, + def test_build_collection_ok(self, f_users, f_coprs, f_builds, f_db, f_users_api, f_mock_chroots): href = "/api_2/builds" @@ -35,7 +35,30 @@ class TestBuildResource(CoprsTestCase): assert expected_ids == self.extract_build_ids(obj) - def test_collection_by_owner(self, f_users, f_coprs, f_builds, f_db, + def test_build_collection_ok_finished( + self, f_users, f_coprs, f_mock_chroots, f_builds, f_db): + + self.db.session.commit() + + href_a = "/api_2/builds?is_finished=True" + href_b = "/api_2/builds?is_finished=False" + + r_a = self.tc.get(href_a) + r_b = self.tc.get(href_b) + + assert r_a.status_code == 200 + assert r_b.status_code == 200 + obj_a = json.loads(r_a.data) + obj_b = json.loads(r_b.data) + + builds = BuildsLogic.get_multiple().all() + expected_ids_a = set([b.id for b in builds if b.ended_on is not None]) + expected_ids_b = set([b.id for b in builds if b.ended_on is None]) + + assert expected_ids_a == self.extract_build_ids(obj_a) + assert expected_ids_b == self.extract_build_ids(obj_b) + + def test_build_collection_by_owner(self, f_users, f_coprs, f_builds, f_db, f_users_api, f_mock_chroots): names_list = [user.username for user in self.basic_user_list] @@ -54,8 +77,8 @@ class TestBuildResource(CoprsTestCase): expected_ids = set([b.id for b in builds]) assert expected_ids == self.extract_build_ids(obj) - def test_collection_by_project_id(self, f_users, f_coprs, f_builds, f_db, - f_users_api, f_mock_chroots): + def test_build_collection_by_project_id( + self, f_users, f_mock_chroots, f_coprs, f_builds, f_db): project_id_list = [copr.id for copr in self.basic_coprs_list] for id_ in project_id_list: @@ -73,8 +96,9 @@ class TestBuildResource(CoprsTestCase): expected_ids = set([b.id for b in builds]) assert expected_ids == self.extract_build_ids(obj) - def test_collection_limit_offset(self, f_users, f_coprs, f_builds, f_db, - f_users_api, f_mock_chroots): + def test_build_collection_limit_offset( + self, f_users, f_mock_chroots, f_coprs, f_builds, f_db): + self.db.session.commit() builds = BuildsLogic.get_multiple().all() total = len(builds) @@ -110,7 +134,7 @@ class TestBuildResource(CoprsTestCase): r = self.tc.get(href) assert r.status_code == 200 - def test_post_bad_content_type( + def test_build_post_bad_content_type( self, f_users, f_coprs, f_db, f_mock_chroots, f_mock_chroots_many, f_build_many_chroots, f_users_api): @@ -128,7 +152,7 @@ class TestBuildResource(CoprsTestCase): ) assert r0.status_code == 400 - def test_post_json_bad_url( + def test_build_post_json_bad_url( self, f_users, f_coprs, f_db, f_mock_chroots, f_mock_chroots_many, f_build_many_chroots, f_users_api): @@ -147,7 +171,7 @@ class TestBuildResource(CoprsTestCase): ) assert r0.status_code == 400 - def test_post_json( + def test_build_post_json( self, f_users, f_coprs, f_db, f_mock_chroots, f_mock_chroots_many, f_build_many_chroots, f_users_api): @@ -173,7 +197,7 @@ class TestBuildResource(CoprsTestCase): metadata["srpm_url"] assert build_dict["source_type"] == "srpm_link" - def test_post_json_on_wrong_user( + def test_build_post_json_on_wrong_user( self, f_users, f_coprs, f_db, f_mock_chroots, f_mock_chroots_many, f_build_many_chroots, f_users_api): @@ -196,7 +220,7 @@ class TestBuildResource(CoprsTestCase): ) assert r0.status_code == 403 - def test_post_json_on_project_during_action( + def test_build_post_json_on_project_during_action( self, f_users, f_coprs, f_db, f_mock_chroots, f_mock_chroots_many, f_build_many_chroots, f_users_api): @@ -216,7 +240,7 @@ class TestBuildResource(CoprsTestCase): ) assert r0.status_code == 400 - def test_post_multipart_wrong_user( + def test_build_post_multipart_wrong_user( self, f_users, f_coprs, f_builds, f_db, f_mock_chroots, f_mock_chroots_many, f_build_many_chroots, f_users_api): @@ -242,7 +266,7 @@ class TestBuildResource(CoprsTestCase): ) assert r0.status_code == 403 - def test_post_multipart_on_project_during_action( + def test_build_post_multipart_on_project_during_action( self, f_users, f_coprs, f_builds, f_db, f_mock_chroots, f_mock_chroots_many, f_build_many_chroots, f_users_api): @@ -267,7 +291,7 @@ class TestBuildResource(CoprsTestCase): ) assert r0.status_code == 400 - def test_post_multipart( + def test_build_post_multipart( self, f_users, f_coprs, f_builds, f_db, f_mock_chroots, f_mock_chroots_many, f_build_many_chroots, f_users_api): @@ -303,7 +327,7 @@ class TestBuildResource(CoprsTestCase): assert set(chroot_name_list) == build_chroots_names assert len(chroot_name_list) == len(build_chroots_obj["build_tasks"]) - def test_post_multipart_missing_file( + def test_build_post_multipart_missing_file( self,f_users, f_coprs, f_builds, f_db, f_users_api, f_mock_chroots): @@ -322,7 +346,7 @@ class TestBuildResource(CoprsTestCase): ) assert r0.status_code == 400 - def test_post_multipart_missing_metadata( + def test_build_post_multipart_missing_metadata( self,f_users, f_coprs, f_builds, f_db, f_users_api, f_mock_chroots): data = { @@ -337,7 +361,7 @@ class TestBuildResource(CoprsTestCase): ) assert r0.status_code == 400 - def test_get_one_build(self, f_users, f_coprs, f_builds, f_db, + def test_build_get_one(self, f_users, f_coprs, f_builds, f_db, f_users_api, f_mock_chroots): build_id_list = [b.id for b in self.basic_builds] @@ -351,7 +375,7 @@ class TestBuildResource(CoprsTestCase): assert obj["build"]["id"] == b_id assert obj["_links"]["self"]["href"] == href - def test_get_one_build_with_tasks(self, f_users, f_coprs, f_builds, f_db, + def test_build_get_one_with_tasks(self, f_users, f_coprs, f_builds, f_db, f_users_api, f_mock_chroots): build_id_list = [b.id for b in self.basic_builds] @@ -366,7 +390,7 @@ class TestBuildResource(CoprsTestCase): assert obj["_links"]["self"]["href"] == href assert "build_tasks" in obj - def test_get_one_build_not_found(self, f_users, f_coprs, f_builds, f_db, + def test_build_get_one_not_found(self, f_users, f_coprs, f_builds, f_db, f_users_api, f_mock_chroots): build_id_list = [b.id for b in self.basic_builds] max_id = max(build_id_list) + 1 @@ -379,7 +403,7 @@ class TestBuildResource(CoprsTestCase): r = self.tc.get(href) assert r.status_code == 404 - def test_delete_build_ok(self, f_users, f_coprs, + def test_build_delete_ok(self, f_users, f_coprs, f_mock_chroots, f_builds,f_users_api, ): self.db.session.commit() @@ -394,7 +418,7 @@ class TestBuildResource(CoprsTestCase): r2 = self.tc.get(href) assert r2.status_code == 404 - def test_delete_build_wrong_user(self, f_users, f_coprs, + def test_build_delete_wrong_user(self, f_users, f_coprs, f_mock_chroots, f_builds,f_users_api, ): login = self.u2.api_login @@ -409,7 +433,7 @@ class TestBuildResource(CoprsTestCase): ) assert r.status_code == 403 - def test_delete_build_in_progress(self, f_users, f_coprs, + def test_build_delete_in_progress(self, f_users, f_coprs, f_mock_chroots, f_builds,f_users_api, ): login = self.u2.api_login @@ -424,7 +448,7 @@ class TestBuildResource(CoprsTestCase): ) assert r.status_code == 400 - def test_put_build_wrong_user( + def test_build_put_wrong_user( self, f_users, f_coprs, f_mock_chroots, f_builds,f_users_api, ): @@ -453,7 +477,7 @@ class TestBuildResource(CoprsTestCase): ) assert r.status_code == 403 - def test_put_build_not_found( + def test_build_put_not_found( self, f_users, f_coprs, f_mock_chroots, f_builds,f_users_api, ): @@ -469,7 +493,7 @@ class TestBuildResource(CoprsTestCase): ) assert r.status_code == 404 - def test_put_cancel_build( + def test_build_put_cancel( self, f_users, f_coprs, f_mock_chroots, f_builds,f_users_api, ): @@ -499,7 +523,7 @@ class TestBuildResource(CoprsTestCase): obj = json.loads(r2.data) assert obj["build"]["state"] == "canceled" - def test_put_cancel_build_wrong_state( + def test_build_put_cancel_wrong_state( self, f_users, f_coprs, f_mock_chroots, f_builds,f_users_api, ): diff --git a/frontend/coprs_frontend/tests/test_api/test_build_task_r.py b/frontend/coprs_frontend/tests/test_api/test_build_task_r.py index 253ed95..1ffc37e 100644 --- a/frontend/coprs_frontend/tests/test_api/test_build_task_r.py +++ b/frontend/coprs_frontend/tests/test_api/test_build_task_r.py @@ -6,6 +6,7 @@ from marshmallow import pprint import pytest import sqlalchemy +from coprs.helpers import StatusEnum from coprs.logic.users_logic import UsersLogic from coprs.logic.coprs_logic import CoprsLogic @@ -18,7 +19,7 @@ class TestBuildTaskResource(CoprsTestCase): def test_collection_ok(self, f_users, f_coprs, f_mock_chroots, f_builds, f_db, f_users_api): - href = "/api_2/build_tasks?build_id=1&limit=5" + href = "/api_2/build_tasks?build_id=1" bc_list = copy.deepcopy(self.b1_bc) self.db.session.commit() @@ -27,6 +28,81 @@ class TestBuildTaskResource(CoprsTestCase): assert r0.status_code == 200 obj = json.loads(r0.data) assert len(obj["build_tasks"]) == len(bc_list) + + def test_collection_ok_default_limit(self, f_users, f_coprs, f_mock_chroots, f_builds, f_db, + f_users_api): + + self.db.session.commit() + href_tpl = "/api_2/build_tasks?limit={}" + expected = href_tpl.format(100) + + for val in [-1, 0, 100, 105, 1000]: + href = href_tpl.format(val) + + r0 = self.tc.get(href) + assert r0.status_code == 200 + obj = json.loads(r0.data) + assert obj["_links"]["self"]["href"] == expected + + def test_collection_ok_by_state( + self, f_users, f_coprs, + f_mock_chroots_many, + f_build_many_chroots, + f_db, + f_users_api): + + self.db.session.commit() + for status in StatusEnum.vals.values(): + expected_chroots = set([ + name + for name, chroot_status in + self.status_by_chroot.items() + if chroot_status == status + ]) + + href = "/api_2/build_tasks?state={}&limit=50".format(StatusEnum(status)) + + r0 = self.tc.get(href) + assert r0.status_code == 200 + obj = json.loads(r0.data) + assert len(obj["build_tasks"]) == len(expected_chroots) + assert set(bt["build_task"]["chroot_name"] + for bt in obj["build_tasks"]) == expected_chroots + assert obj["_links"]["self"]["href"] == href + + def test_collection_ok_by_project( + self, f_users, f_coprs, f_mock_chroots, f_builds, + f_users_api, f_db): + + href = "/api_2/build_tasks?project_id=1&limit=50" + bc_list = copy.deepcopy(self.b1_bc) + bc_list.extend(copy.deepcopy(self.b2_bc)) + + self.db.session.commit() + + r0 = self.tc.get(href) + assert r0.status_code == 200 + obj = json.loads(r0.data) + assert len(obj["build_tasks"]) == len(bc_list) + assert obj["_links"]["self"]["href"] == href + + def test_collection_ok_by_owner( + self, f_users, f_coprs, f_mock_chroots, f_builds, + f_users_api, f_db): + + href = "/api_2/build_tasks?owner={}&limit=50".format(self.u2.username) + bc_list_len = sum( + len(b.build_chroots) + for b in self.basic_builds + if b.copr.owner == self.u2 + ) + + self.db.session.commit() + + r0 = self.tc.get(href) + assert r0.status_code == 200 + obj = json.loads(r0.data) + assert len(obj["build_tasks"]) == bc_list_len assert obj["_links"]["self"]["href"] == href def test_post_not_allowed(self, f_users, f_coprs, f_mock_chroots, f_builds, f_db,

8 years, 8 months

1
0
0 / 0

[copr] master: [frontend][API] fixed tests and updated doc to reflect recent changes (3ca8a07)

by vgologuz＠fedoraproject.org

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit 3ca8a07387d89a930c67a7d53e4faa74e1d303ab Author: Valentin Gologuzov <vgologuz(a)redhat.com> Date: Wed Sep 9 12:14:44 2015 +0200 [frontend][API] fixed tests and updated doc to reflect recent changes >--------------------------------------------------------------- .../coprs_frontend/coprs/logic/builds_logic.py | 10 +++++- frontend/coprs_frontend/coprs/rest_api/common.py | 31 ++++++++++------- .../coprs/rest_api/resources/build.py | 17 ++++++--- .../coprs/rest_api/resources/build_task.py | 7 +--- .../coprs_frontend/tests/test_api/test_build_r.py | 16 ++++---- ...test_build_chroot_r.py => test_build_task_r.py} | 32 +++++++++--------- frontend/docs/api_2/source/Resources/build.rst | 29 ++++++++-------- .../docs/api_2/source/Resources/build_task.rst | 35 ++++++++++--------- frontend/docs/api_2/source/index.rst | 3 ++ 9 files changed, 99 insertions(+), 81 deletions(-) diff --git a/frontend/coprs_frontend/coprs/logic/builds_logic.py b/frontend/coprs_frontend/coprs/logic/builds_logic.py index 3992ed0..3082b5c 100644 --- a/frontend/coprs_frontend/coprs/logic/builds_logic.py +++ b/frontend/coprs_frontend/coprs/logic/builds_logic.py @@ -511,6 +511,15 @@ class BuildsLogic(object): else: return None + @classmethod + def filter_is_finished(cls, query, is_finished): + # todo: check that ended_on is set correctly for all cases + # e.g.: failed dist-git import, cancellation + if is_finished: + return query.filter(models.Build.ended_on.isnot(None)) + else: + return query.filter(models.Build.ended_on.is_(None)) + class BuildChrootsLogic(object): @classmethod @@ -531,7 +540,6 @@ class BuildChrootsLogic(object): .join(models.BuildChroot.mock_chroot) .join(models.Build.copr) .join(models.Copr.owner) - ) return query diff --git a/frontend/coprs_frontend/coprs/rest_api/common.py b/frontend/coprs_frontend/coprs/rest_api/common.py index 7bc9c81..1945ce8 100644 --- a/frontend/coprs_frontend/coprs/rest_api/common.py +++ b/frontend/coprs_frontend/coprs/rest_api/common.py @@ -3,23 +3,23 @@ import base64 import datetime import functools from logging import getLogger -from coprs.logic.builds_logic import BuildsLogic -from coprs.logic.coprs_logic import CoprsLogic -from coprs.rest_api.schemas import BuildTaskSchema -from coprs.rest_api.util import mm_serialize_one, get_one_safe - -log = getLogger(__name__) from flask import url_for import flask from .. import models from ..logic.users_logic import UsersLogic +from ..logic.builds_logic import BuildsLogic +from ..logic.coprs_logic import CoprsLogic +from ..rest_api.schemas import BuildTaskSchema +from ..rest_api.util import mm_serialize_one, get_one_safe from .exceptions import AuthFailed, ObjectNotFoundError from .schemas import CoprChrootSchema, BuildSchema, ProjectSchema from .util import mm_serialize_one +log = getLogger(__name__) + def render_copr_chroot(chroot): return { @@ -41,21 +41,25 @@ def render_build(build, self_params=None): "_links": { "self": {"href": url_for(".buildr", build_id=build.id, **self_params)}, "project": {"href": url_for(".projectr", project_id=build.copr_id)}, - "tasks": {"href": url_for(".buildtasklistr", build_id=build.id)} + "build_tasks": {"href": url_for(".buildtasklistr", build_id=build.id)} } } -def render_project(copr, self_params=None): +def render_project(project, self_params=None): + """ + :param models.Copr project: + """ if self_params is None: self_params = {} return { - "project": mm_serialize_one(ProjectSchema, copr), + "project": mm_serialize_one(ProjectSchema, project), "_links": { - "self": {"href": url_for(".projectr", project_id=copr.id, **self_params)}, - "builds": {"href": url_for(".buildlistr", project_id=copr.id)}, - "chroots": {"href": url_for(".projectchrootlistr", project_id=copr.id)} + "self": {"href": url_for(".projectr", project_id=project.id, **self_params)}, + "builds": {"href": url_for(".buildlistr", project_id=project.id)}, + "chroots": {"href": url_for(".projectchrootlistr", project_id=project.id)}, + "build_tasks": {"href":url_for(".buildtasklistr", project_id=project.id) } }, } @@ -65,9 +69,10 @@ def render_build_task(chroot): :type chroot: models.BuildChroot """ return { - "task": mm_serialize_one(BuildTaskSchema, chroot), + "build_task": mm_serialize_one(BuildTaskSchema, chroot), "_links": { "project": {"href": url_for(".projectr", project_id=chroot.build.copr_id)}, + "build": {"href": url_for(".buildr", build_id=chroot.build_id)}, "self": {"href": url_for(".buildtaskr", build_id=chroot.build.id, name=chroot.name)}, diff --git a/frontend/coprs_frontend/coprs/rest_api/resources/build.py b/frontend/coprs_frontend/coprs/rest_api/resources/build.py index 7d31412..bc1605f 100644 --- a/frontend/coprs_frontend/coprs/rest_api/resources/build.py +++ b/frontend/coprs_frontend/coprs/rest_api/resources/build.py @@ -27,7 +27,7 @@ class BuildListR(Resource): parser.add_argument('limit', type=int) parser.add_argument('offset', type=int) - parser.add_argument('state', type=str) + parser.add_argument('is_finished', type=bool) # parser.add_argument('package', type=str) req_args = parser.parse_args() @@ -41,6 +41,11 @@ class BuildListR(Resource): else: query = BuildsLogic.get_multiple() + if req_args["is_finished"] is not None: + is_finished = req_args["is_finished"] + query = BuildsLogic.filter_is_finished(query, is_finished) + # TODO: add test! + if req_args["limit"] is not None: limit = req_args["limit"] if limit <= 0 or limit > 100: @@ -156,18 +161,18 @@ class BuildR(Resource): def get(self, build_id): parser = get_request_parser() - parser.add_argument('show_chroots', type=bool, default=False) + parser.add_argument('show_build_tasks', type=bool, default=False) req_args = parser.parse_args() build = get_build_safe(build_id) self_params = {} - if req_args["show_chroots"]: - self_params["show_chroots"] = req_args["show_chroots"] + if req_args["show_build_tasks"]: + self_params["show_build_tasks"] = req_args["show_build_tasks"] result = render_build(build, self_params) - if req_args["show_chroots"]: - result["build_chroots"] = [ + if req_args["show_build_tasks"]: + result["build_tasks"] = [ render_build_task(chroot) for chroot in build.build_chroots ] diff --git a/frontend/coprs_frontend/coprs/rest_api/resources/build_task.py b/frontend/coprs_frontend/coprs/rest_api/resources/build_task.py index c17a1d1..4c2109c 100644 --- a/frontend/coprs_frontend/coprs/rest_api/resources/build_task.py +++ b/frontend/coprs_frontend/coprs/rest_api/resources/build_task.py @@ -24,11 +24,6 @@ from ..util import get_one_safe, get_request_parser # resp.headers["Location"] == url_for(".buildtasklistr", build_id=build_id) -class BuildTaskListQuerySchema(Schema): - - build_id = fields.Int() - - class BuildTaskListR(Resource): state_choices = StatusEnum.vals.keys() @@ -80,7 +75,7 @@ class BuildTaskListR(Resource): build_chroots = query.all() return { - "tasks": [ + "build_tasks": [ render_build_task(chroot) for chroot in build_chroots ], diff --git a/frontend/coprs_frontend/tests/test_api/test_build_r.py b/frontend/coprs_frontend/tests/test_api/test_build_r.py index 1f0075d..bcc5cf7 100644 --- a/frontend/coprs_frontend/tests/test_api/test_build_r.py +++ b/frontend/coprs_frontend/tests/test_api/test_build_r.py @@ -295,13 +295,13 @@ class TestBuildResource(CoprsTestCase): assert build_obj["build"]["source_type"] == "srpm_upload" - chroots_href = build_obj["_links"]["chroots"]["href"] - r2 = self.tc.get(chroots_href) + tasks_href = build_obj["_links"]["build_tasks"]["href"] + r2 = self.tc.get(tasks_href) build_chroots_obj = json.loads(r2.data) - build_chroots_names = set([bc["chroot"]["name"] for bc in - build_chroots_obj["chroots"]]) + build_chroots_names = set([bc["build_task"]["chroot_name"] for bc in + build_chroots_obj["build_tasks"]]) assert set(chroot_name_list) == build_chroots_names - assert len(chroot_name_list) == len(build_chroots_obj["chroots"]) + assert len(chroot_name_list) == len(build_chroots_obj["build_tasks"]) def test_post_multipart_missing_file( self,f_users, f_coprs, f_builds, f_db, @@ -351,20 +351,20 @@ class TestBuildResource(CoprsTestCase): assert obj["build"]["id"] == b_id assert obj["_links"]["self"]["href"] == href - def test_get_one_build_with_chroots(self, f_users, f_coprs, f_builds, f_db, + def test_get_one_build_with_tasks(self, f_users, f_coprs, f_builds, f_db, f_users_api, f_mock_chroots): build_id_list = [b.id for b in self.basic_builds] self.db.session.commit() for b_id in build_id_list: - href = "/api_2/builds/{}?show_chroots=True".format(b_id) + href = "/api_2/builds/{}?show_build_tasks=True".format(b_id) r = self.tc.get(href) assert r.status_code == 200 obj = json.loads(r.data) assert obj["build"]["id"] == b_id assert obj["_links"]["self"]["href"] == href - assert "build_chroots" in obj + assert "build_tasks" in obj def test_get_one_build_not_found(self, f_users, f_coprs, f_builds, f_db, f_users_api, f_mock_chroots): diff --git a/frontend/coprs_frontend/tests/test_api/test_build_chroot_r.py b/frontend/coprs_frontend/tests/test_api/test_build_task_r.py similarity index 75% rename from frontend/coprs_frontend/tests/test_api/test_build_chroot_r.py rename to frontend/coprs_frontend/tests/test_api/test_build_task_r.py index 8b1b40a..253ed95 100644 --- a/frontend/coprs_frontend/tests/test_api/test_build_chroot_r.py +++ b/frontend/coprs_frontend/tests/test_api/test_build_task_r.py @@ -13,14 +13,12 @@ from coprs.logic.coprs_logic import CoprsLogic from tests.coprs_test_case import CoprsTestCase, TransactionDecorator -class TestBuildChrootResource(CoprsTestCase): +class TestBuildTaskResource(CoprsTestCase): def test_collection_ok(self, f_users, f_coprs, f_mock_chroots, f_builds, f_db, f_users_api): - # project_id = self.c1.id - # import ipdb; ipdb.set_trace() - href = "/api_2/builds/1/chroots" + href = "/api_2/build_tasks?build_id=1&limit=5" bc_list = copy.deepcopy(self.b1_bc) self.db.session.commit() @@ -28,14 +26,14 @@ class TestBuildChrootResource(CoprsTestCase): r0 = self.tc.get(href) assert r0.status_code == 200 obj = json.loads(r0.data) - assert len(obj["chroots"]) == len(bc_list) + assert len(obj["build_tasks"]) == len(bc_list) assert obj["_links"]["self"]["href"] == href def test_post_not_allowed(self, f_users, f_coprs, f_mock_chroots, f_builds, f_db, f_users_api): self.db.session.commit() r0 = self.request_rest_api_with_auth( - "/api_2/builds/1/chroots", + "/api_2/build_tasks", method="post", content={}, ) @@ -44,30 +42,32 @@ class TestBuildChrootResource(CoprsTestCase): def test_get_one(self, f_users, f_coprs, f_mock_chroots, f_builds, f_db, f_users_api): expected_fields = [ - "state", - "git_hash", - "started_on", - "ended_on", - "name" + ("state", None), + ("git_hash", None), + ("started_on", None), + ("ended_on", None), + ("name", "chroot_name"), ] bc = self.b1_bc[0] expected_dict = { f: getattr(bc, f) - for f in expected_fields + for f, _ in expected_fields } - href = "/api_2/builds/1/chroots/{}".format(bc.name) + href = "/api_2/build_tasks/1/{}".format(bc.name) self.db.session.commit() r0 = self.tc.get(href) assert r0.status_code == 200 obj = json.loads(r0.data) - for k in expected_fields: - assert obj["chroot"][k] == expected_dict[k] + for k, res_k in expected_fields: + if res_k is None: + res_k = k + assert obj["build_task"][res_k] == expected_dict[k] def test_get_one_bad_name(self, f_users, f_coprs, f_mock_chroots, f_builds, f_db, f_users_api): - href = "/api_2/builds/1/chroots/surely_bad_chroot_name" + href = "/api_2/build_tasks/1/surely_bad_chroot_name" self.db.session.commit() r0 = self.tc.get(href) diff --git a/frontend/docs/api_2/source/Resources/build.rst b/frontend/docs/api_2/source/Resources/build.rst index 3769808..517df70 100644 --- a/frontend/docs/api_2/source/Resources/build.rst +++ b/frontend/docs/api_2/source/Resources/build.rst @@ -4,7 +4,7 @@ Build Build resources allows to submit new builds and access current build progress. Build in fact consists of the few tasks, one per chroot, and detail information -is available through sub-resource :doc:`./build_chroot`. +is available through sub-resource :doc:`./build_task`. @@ -47,10 +47,10 @@ Build fields Field Type Description ================== ==================== =============== id int unique build identifier -state string current state of the build, value is aggregated from build chroots +state string current state of the build, value is aggregated from build tasks submitted_on int(unixtime UTC) time of the build submission -started_on int(unixtime UTC) time when the first build chroot started, otherwise ``null`` -ended_on int(unixtime UTC) time when the last build chroot ended, otherwise ``null`` +started_on int(unixtime UTC) time when the first build task started, otherwise ``null`` +ended_on int(unixtime UTC) time when the last build task ended, otherwise ``null`` source_type string method used for build creation source_metadata json object build source information package_version string version of the source package @@ -108,8 +108,8 @@ List builds "self": { "href": "/api_2/builds/106897" }, - "chroots": { - "href": "/api_2/builds/106897/chroots" + "build_tasks": { + "href": "/api_2/build_tasks?build_id=106897" } }, "build": { @@ -269,7 +269,7 @@ Get build details Returns details about build :param int build_id: a unique identifier of the build - :query bool show_chroots: embed :doc:`./build_chroot` sub-resources into the result, default is False + :query bool show_build_tasks: embed :doc:`./build_task` sub-resources into the result, default is False :statuscode 200: no error :statuscode 404: build not found @@ -278,7 +278,7 @@ Get build details .. sourcecode:: http - GET /api_2/builds/106897 HTTP/1.1 + GET /api_2/builds/106897?show_build_tasks=True HTTP/1.1 Host: copr.fedoraproject.org **Response** @@ -289,10 +289,11 @@ Get build details Content-Type: application/json { - "build_chroots": [ + "build_tasks": [ { - "chroot": { - "name": "fedora-21-i386", + "tasks": { + "build_id": 3985, + "chroot_name": "fedora-21-i386", "started_on": 1441366860, "state": "succeeded", "ended_on": 1441366969, @@ -304,7 +305,7 @@ Get build details "href": "/api_2/projects/3985" }, "self": { - "href": "/api_2/builds/106897/chroots/fedora-21-i386" + "href": "/api_2/build_tasks/106897/fedora-21-i386" } } } @@ -316,8 +317,8 @@ Get build details "self": { "href": "/api_2/builds/106897?show_chroots=True" }, - "chroots": { - "href": "/api_2/builds/106897/chroots" + "build_tasks": { + "href": "/api_2/build_tasks/?build_id=3985" } }, "build": { diff --git a/frontend/docs/api_2/source/Resources/build_task.rst b/frontend/docs/api_2/source/Resources/build_task.rst index c5871a4..2212475 100644 --- a/frontend/docs/api_2/source/Resources/build_task.rst +++ b/frontend/docs/api_2/source/Resources/build_task.rst @@ -1,7 +1,7 @@ Build Task ========== -Build task represents information about individual build tasks per each chroot. +Build task represents information about individual build tasks. One task is responsible for one chroot. Structure of the build task entity ---------------------------------- @@ -9,7 +9,7 @@ Structure of the build task entity .. code-block:: javascript { - "name": "fedora-rawhide-x86_64", + "chroot_name": "fedora-rawhide-x86_64", "build_id": 12345, "started_on": 1440753865, "ended_on": 1440753919, @@ -18,14 +18,14 @@ Structure of the build task entity "git_hash": "d241064b14f9dcd5d9032d0aca3b4e78fbd1aafd" } -Build chroot fields -~~~~~~~~~~~~~~~~~~~ +Build tasks fields +~~~~~~~~~~~~~~~~~~ ================== ==================== =============== Field Type Description ================== ==================== =============== chroot_name str chroot name build_id int unique build identifier -state str current build state +state str current build task state started_on int(unixtime UTC) time when the build chroot started ended_on int(unixtime UTC) time when the build chroot ended git_hash str hash of the git commit in dist-git used for the build @@ -37,8 +37,8 @@ result_dir_url str(URL) location of the build results Build Chroot doesn't currently support any modifications, so all fields are read-only. -List build chroots ------------------- +List build tasks +---------------- .. http:get:: /api_2/builds_tasks @@ -77,10 +77,10 @@ List build chroots Content-Type: application/json { - "chroots": [ + "build_tasks": [ { - "chroot": { - "name": "fedora-rawhide-x86_64", + "build_task": { + "chroot_name": "fedora-rawhide-x86_64", "started_on": 1440753865, "state": "succeeded", "ended_on": 1440753919, @@ -92,14 +92,14 @@ List build chroots "href": "/api_2/projects/3985" }, "self": { - "href": "/api_2/builds/106882/chroots/fedora-rawhide-x86_64" + "href": "/api_2/build_tasks/106882/fedora-rawhide-x86_64" } } } ], "_links": { "self": { - "href": "/api_2/builds/106882/chroots" + "href": "/api_2/build_tasks?build_id=106882" } } } @@ -109,7 +109,7 @@ List build chroots Get build task details ---------------------- -.. http:get:: /api_2/builds/(int:build_id)/chroots/(str:name) +.. http:get:: /api_2/build_tasks/(int:build_id)/(str:name) Returns details about one build task @@ -123,7 +123,7 @@ Get build task details .. sourcecode:: http - GET /api_2/builds/106882/chroots/fedora-rawhide-x86_64 HTTP/1.1 + GET /api_2/build_tasks/106882/fedora-rawhide-x86_64 HTTP/1.1 Host: copr.fedoraproject.org **Response** @@ -134,8 +134,9 @@ Get build task details Content-Type: application/json { - "chroot": { - "name": "fedora-rawhide-x86_64", + "build_task": { + "chroot_name": "fedora-rawhide-x86_64", + "build_id": 3985, "started_on": 1440753865, "state": "succeeded", "ended_on": 1440753919, @@ -147,7 +148,7 @@ Get build task details "href": "/api_2/projects/3985" }, "self": { - "href": "/api_2/builds/106882/chroots/fedora-rawhide-x86_64" + "href": "/api_2/build_tasks/106882/fedora-rawhide-x86_64" } } } diff --git a/frontend/docs/api_2/source/index.rst b/frontend/docs/api_2/source/index.rst index 3ed6a68..6bf0743 100644 --- a/frontend/docs/api_2/source/index.rst +++ b/frontend/docs/api_2/source/index.rst @@ -52,6 +52,9 @@ Here is a short example with the content of API root: "builds": { "href": "/api_2/builds" } + "build_tasks": { + "href": "/api_2/build_tasks" + } } Response structure

8 years, 8 months

1
0
0 / 0

[copr] master: [frontend][API] replaced BuildChroot with BuildTask, tests are broken and documentation should reflect this changes (a61ba32)

by vgologuz＠fedoraproject.org

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit a61ba325517ffe0a15b4b72a387b1e48a16a113d Author: Valentin Gologuzov <vgologuz(a)redhat.com> Date: Tue Sep 8 18:15:14 2015 +0200 [frontend][API] replaced BuildChroot with BuildTask, tests are broken and documentation should reflect this changes >--------------------------------------------------------------- .../coprs_frontend/coprs/logic/builds_logic.py | 30 ++++++ frontend/coprs_frontend/coprs/rest_api/__init__.py | 38 ++++--- frontend/coprs_frontend/coprs/rest_api/common.py | 10 +- .../coprs/rest_api/resources/build.py | 31 +++--- .../coprs/rest_api/resources/build_chroot.py | 47 -------- .../coprs/rest_api/resources/build_task.py | 111 ++++++++++++++++++++ .../coprs/rest_api/resources/mock_chroot.py | 4 +- .../coprs/rest_api/resources/project.py | 11 +- frontend/coprs_frontend/coprs/rest_api/schemas.py | 5 +- frontend/coprs_frontend/coprs/rest_api/util.py | 25 ++++- .../Resources/{build_chroot.rst => build_task.rst} | 43 +++++--- frontend/docs/api_2/source/index.rst | 2 +- 12 files changed, 247 insertions(+), 110 deletions(-) diff --git a/frontend/coprs_frontend/coprs/logic/builds_logic.py b/frontend/coprs_frontend/coprs/logic/builds_logic.py index ace4bfa..3992ed0 100644 --- a/frontend/coprs_frontend/coprs/logic/builds_logic.py +++ b/frontend/coprs_frontend/coprs/logic/builds_logic.py @@ -9,6 +9,7 @@ import time import flask from sqlalchemy import or_ from sqlalchemy import and_ +from sqlalchemy.orm import joinedload from sqlalchemy.orm.exc import NoResultFound from sqlalchemy.sql import false from werkzeug.utils import secure_filename @@ -37,6 +38,7 @@ class BuildsLogic(object): def get(cls, build_id): return models.Build.query.filter(models.Build.id == build_id) + # todo: move methods operating with BuildChroot to BuildChrootLogic @classmethod def get_build_tasks(cls, status): return models.BuildChroot.query.filter(models.BuildChroot.status == status) \ @@ -521,6 +523,34 @@ class BuildChrootsLogic(object): .filter(BuildChroot.mock_chroot_id == mc.id) ) + @classmethod + def get_multiply(cls): + query = ( + models.BuildChroot.query + .join(models.BuildChroot.build) + .join(models.BuildChroot.mock_chroot) + .join(models.Build.copr) + .join(models.Copr.owner) + + ) + return query + + @classmethod + def filter_by_build_id(cls, query, build_id): + return query.filter(models.Build.id == build_id) + + @classmethod + def filter_by_project_id(cls, query, project_id): + return query.filter(models.Copr.id == project_id) + + @classmethod + def filter_by_project_owner_name(cls, query, username): + return query.filter(models.User.username == username) + + @classmethod + def filter_by_state(cls, query, state): + return query.filter(models.BuildChroot.status == StatusEnum(state)) + class BuildsMonitorLogic(object): @classmethod diff --git a/frontend/coprs_frontend/coprs/rest_api/__init__.py b/frontend/coprs_frontend/coprs/rest_api/__init__.py index 7276c2f..0f87c4f 100644 --- a/frontend/coprs_frontend/coprs/rest_api/__init__.py +++ b/frontend/coprs_frontend/coprs/rest_api/__init__.py @@ -7,7 +7,7 @@ from coprs.exceptions import InsufficientRightsException from coprs.rest_api.exceptions import ApiError from coprs.rest_api.resources.build import BuildListR, BuildR -from coprs.rest_api.resources.build_chroot import BuildChrootListR, BuildChrootR +from coprs.rest_api.resources.build_task import BuildTaskListR, BuildTaskR from coprs.rest_api.resources.mock_chroot import MockChrootListR, MockChrootR from coprs.rest_api.resources.project import ProjectListR, ProjectR from coprs.rest_api.resources.project_chroot import ProjectChrootListR, ProjectChrootR @@ -20,9 +20,19 @@ class RootR(Resource): return { "_links": { "self": {"href": url_for(".rootr")}, - "projects": {"href": url_for(".projectlistr")}, + "projects": { + "href": url_for(".projectlistr"), + "query_params": [ + { + "name": u"owner", + "type": u"string", + "description": u"Select only project owned by the user" + } + ] + }, "mock_chroots": {"href": url_for(".mockchrootlistr")}, "builds": {"href": url_for(".buildlistr")}, + "build_tasks": {"href": url_for(".buildtasklistr")}, } } @@ -30,14 +40,15 @@ class RootR(Resource): class MyApi(Api): # flask-restfull error handling quite buggy right now def error_router(self, original_handler, e): + # import ipdb; ipdb.set_trace() return original_handler(e) - # def handle_error(self, e): - # - # if isinstance(e, sqlalchemy.orm.exc.NoResultFound): - # return self.make_response(str(e), 404) - # - # - # super(MyApi, self).handle_error(e) +# # def handle_error(self, e): +# # +# # if isinstance(e, sqlalchemy.orm.exc.NoResultFound): +# # return self.make_response(str(e), 404) +# # +# # +# # super(MyApi, self).handle_error(e) # def register_api(app, db): @@ -59,11 +70,10 @@ api.add_resource(BuildR, "/builds/<int:build_id>") api.add_resource(ProjectChrootListR, "/projects/<int:project_id>/chroots") api.add_resource(ProjectChrootR, "/projects/<int:project_id>/chroots/<name>") -api.add_resource(BuildChrootListR, "/builds/<int:build_id>/chroots") -api.add_resource(BuildChrootR, "/builds/<int:build_id>/chroots/<name>") - -# app.register_blueprint(rest_api_bp, url_prefix=URL_PREFIX) +api.add_resource(BuildTaskListR, "/build_tasks") +# todo: add redirect from /build_tasks/<int:build_id> -> /build_tasks?build_id=<build_id> +api.add_resource(BuildTaskR, "/build_tasks/<int:build_id>/<name>") def register_api_error_handler(app): @@ -72,7 +82,7 @@ def register_api_error_handler(app): """ :param ApiError error: """ - + # import ipdb; ipdb.set_trace() content = { "message": error.msg, } diff --git a/frontend/coprs_frontend/coprs/rest_api/common.py b/frontend/coprs_frontend/coprs/rest_api/common.py index 72c5eae..7bc9c81 100644 --- a/frontend/coprs_frontend/coprs/rest_api/common.py +++ b/frontend/coprs_frontend/coprs/rest_api/common.py @@ -5,7 +5,7 @@ import functools from logging import getLogger from coprs.logic.builds_logic import BuildsLogic from coprs.logic.coprs_logic import CoprsLogic -from coprs.rest_api.schemas import BuildChrootSchema +from coprs.rest_api.schemas import BuildTaskSchema from coprs.rest_api.util import mm_serialize_one, get_one_safe log = getLogger(__name__) @@ -41,7 +41,7 @@ def render_build(build, self_params=None): "_links": { "self": {"href": url_for(".buildr", build_id=build.id, **self_params)}, "project": {"href": url_for(".projectr", project_id=build.copr_id)}, - "chroots": {"href": url_for(".buildchrootlistr", build_id=build.id)} + "tasks": {"href": url_for(".buildtasklistr", build_id=build.id)} } } @@ -60,15 +60,15 @@ def render_project(copr, self_params=None): } -def render_build_chroot(chroot): +def render_build_task(chroot): """ :type chroot: models.BuildChroot """ return { - "chroot": mm_serialize_one(BuildChrootSchema, chroot), + "task": mm_serialize_one(BuildTaskSchema, chroot), "_links": { "project": {"href": url_for(".projectr", project_id=chroot.build.copr_id)}, - "self": {"href": url_for(".buildchrootr", + "self": {"href": url_for(".buildtaskr", build_id=chroot.build.id, name=chroot.name)}, } diff --git a/frontend/coprs_frontend/coprs/rest_api/resources/build.py b/frontend/coprs_frontend/coprs/rest_api/resources/build.py index 6f4a792..7d31412 100644 --- a/frontend/coprs_frontend/coprs/rest_api/resources/build.py +++ b/frontend/coprs_frontend/coprs/rest_api/resources/build.py @@ -2,28 +2,24 @@ import flask from flask import url_for, make_response - -# from flask_restful_swagger import swagger - -from coprs import db -from coprs.exceptions import ActionInProgressException, InsufficientRightsException, RequestCannotBeExecuted -from coprs.logic.builds_logic import BuildsLogic -from coprs.rest_api.common import get_project_safe -from coprs.rest_api.exceptions import MalformedRequest, CannotProcessRequest, AccessForbidden -from ..common import render_build, rest_api_auth_required, render_build_chroot, get_build_safe, get_user_safe - -from coprs.rest_api.schemas import BuildSchema, BuildCreateSchema, BuildCreateFromUrlSchema - -from coprs.rest_api.util import mm_deserialize - from flask_restful import Resource, reqparse +from ... import db +from ...exceptions import ActionInProgressException, InsufficientRightsException, RequestCannotBeExecuted +from ...helpers import StatusEnum +from ...logic.builds_logic import BuildsLogic +from ..common import get_project_safe +from ..exceptions import MalformedRequest, CannotProcessRequest, AccessForbidden +from ..common import render_build, rest_api_auth_required, render_build_task, get_build_safe, get_user_safe +from ..schemas import BuildSchema, BuildCreateSchema, BuildCreateFromUrlSchema +from ..util import mm_deserialize, get_request_parser + class BuildListR(Resource): def get(self): - parser = reqparse.RequestParser() + parser = get_request_parser() parser.add_argument('owner', type=str,) parser.add_argument('project_id', type=int) @@ -31,6 +27,7 @@ class BuildListR(Resource): parser.add_argument('limit', type=int) parser.add_argument('offset', type=int) + parser.add_argument('state', type=str) # parser.add_argument('package', type=str) req_args = parser.parse_args() @@ -158,7 +155,7 @@ class BuildListR(Resource): class BuildR(Resource): def get(self, build_id): - parser = reqparse.RequestParser() + parser = get_request_parser() parser.add_argument('show_chroots', type=bool, default=False) req_args = parser.parse_args() @@ -171,7 +168,7 @@ class BuildR(Resource): result = render_build(build, self_params) if req_args["show_chroots"]: result["build_chroots"] = [ - render_build_chroot(chroot) + render_build_task(chroot) for chroot in build.build_chroots ] diff --git a/frontend/coprs_frontend/coprs/rest_api/resources/build_chroot.py b/frontend/coprs_frontend/coprs/rest_api/resources/build_chroot.py deleted file mode 100644 index 3f6da84..0000000 --- a/frontend/coprs_frontend/coprs/rest_api/resources/build_chroot.py +++ /dev/null @@ -1,47 +0,0 @@ -# coding: utf-8 - -from flask import url_for -from flask_restful import Resource - -from coprs.rest_api.common import render_build_chroot -from ...exceptions import MalformedArgumentException -from ...logic.builds_logic import BuildsLogic, BuildChrootsLogic -from ..exceptions import MalformedRequest -from ..util import get_one_safe - - -class BuildChrootListR(Resource): - def get(self, build_id): - build = get_one_safe(BuildsLogic.get(build_id), - "Build {} Not found".format(build_id)) - - return { - "chroots": [ - render_build_chroot(chroot) - for chroot in build.build_chroots - ], - "_links": { - "self": {"href": url_for(".buildchrootlistr", build_id=build_id)} - } - } - - -class BuildChrootR(Resource): - - @staticmethod - def _get_chroot_safe(build_id, name): - try: - chroot = get_one_safe( - BuildChrootsLogic.get_by_build_id_and_name(build_id, name), - "Build chroot {} for build {} not found" - ) - except MalformedArgumentException as err: - raise MalformedRequest("Bad mock chroot name: {}".format(err)) - return chroot - - def get(self, build_id, name): - chroot = self._get_chroot_safe(build_id, name) - return render_build_chroot(chroot) - - # todo: add put method: allows only to pass status: cancelled to cancel build - diff --git a/frontend/coprs_frontend/coprs/rest_api/resources/build_task.py b/frontend/coprs_frontend/coprs/rest_api/resources/build_task.py new file mode 100644 index 0000000..c17a1d1 --- /dev/null +++ b/frontend/coprs_frontend/coprs/rest_api/resources/build_task.py @@ -0,0 +1,111 @@ +# coding: utf-8 + +from flask import url_for +from flask_restful import Resource +from flask_restful import Resource, reqparse +from flask_restful.reqparse import Argument + +from marshmallow import pprint +from marshmallow import Schema, fields +from marshmallow import Schema, fields, validates_schema, ValidationError, validate + +from coprs.helpers import StatusEnum +from coprs.rest_api.common import render_build_task +from ...exceptions import MalformedArgumentException +from ...logic.builds_logic import BuildsLogic, BuildChrootsLogic +from ..exceptions import MalformedRequest +from ..util import get_one_safe, get_request_parser + + +# todo: add redirect from /build_tasks/<int:build_id> -> /build_tasks?build_id=<build_id> +# class BuildTaskListRedirectBuildIdR(Resource): +# def get(self, build_id): +# resp = make_response("", 302) +# resp.headers["Location"] == url_for(".buildtasklistr", build_id=build_id) + + +class BuildTaskListQuerySchema(Schema): + + build_id = fields.Int() + + +class BuildTaskListR(Resource): + state_choices = StatusEnum.vals.keys() + + def get(self): + + parser = get_request_parser() + + parser.add_argument('owner', type=str,) + parser.add_argument('project_id', type=int) + parser.add_argument('build_id', type=int) + + parser.add_argument('limit', type=int) + parser.add_argument('offset', type=int) + + parser.add_argument( + 'state', type=str, choices=self.state_choices, + help=u"allowed states: {}".format(" ".join(self.state_choices))) + + req_args = parser.parse_args() + + self_params = dict(req_args) + + query = BuildChrootsLogic.get_multiply() + if self_params.get("build_id") is not None: + query = BuildChrootsLogic.filter_by_build_id( + query, self_params["build_id"]) + elif self_params.get("project_id") is not None: + query = BuildChrootsLogic.filter_by_project_id( + query, self_params["project_id"]) + elif self_params.get("owner") is not None: + query = BuildChrootsLogic.filter_by_project_owner_name( + query, self_params["owner"]) + + state = self_params.get("state") + if state: + query = BuildChrootsLogic.filter_by_state(query, state) + + if req_args["limit"] is not None: + limit = req_args["limit"] + if limit <= 0 or limit > 100: + limit = 100 + else: + limit = 100 + self_params["limit"] = limit + query = query.limit(limit) + + if "offset" in self_params is not None: + query = query.offset(self_params["offset"]) + + build_chroots = query.all() + return { + "tasks": [ + render_build_task(chroot) + for chroot in build_chroots + ], + "_links": { + "self": {"href": url_for(".buildtasklistr", **self_params)} + } + } + + +class BuildTaskR(Resource): + + @staticmethod + def _get_chroot_safe(build_id, name): + try: + chroot = get_one_safe( + BuildChrootsLogic.get_by_build_id_and_name(build_id, name), + "Build task {} for build {} not found" + ) + except MalformedArgumentException as err: + raise MalformedRequest("Bad mock chroot name: {}".format(err)) + return chroot + + def get(self, build_id, name): + chroot = self._get_chroot_safe(build_id, name) + return render_build_task(chroot) + + # todo: add put method: allows only to pass status: cancelled to cancel build + diff --git a/frontend/coprs_frontend/coprs/rest_api/resources/mock_chroot.py b/frontend/coprs_frontend/coprs/rest_api/resources/mock_chroot.py index c7135de..a81569c 100644 --- a/frontend/coprs_frontend/coprs/rest_api/resources/mock_chroot.py +++ b/frontend/coprs_frontend/coprs/rest_api/resources/mock_chroot.py @@ -6,7 +6,7 @@ from flask_restful import Resource, reqparse from ...logic.coprs_logic import MockChrootsLogic from ..schemas import MockChrootSchema -from ..util import get_one_safe +from ..util import get_one_safe, get_request_parser def render_mock_chroot(chroot): @@ -21,7 +21,7 @@ def render_mock_chroot(chroot): class MockChrootListR(Resource): def get(self): - parser = reqparse.RequestParser() + parser = get_request_parser() parser.add_argument('active_only', type=bool) req_args = parser.parse_args() active_only = False diff --git a/frontend/coprs_frontend/coprs/rest_api/resources/project.py b/frontend/coprs_frontend/coprs/rest_api/resources/project.py index 148891b..e3157b8 100644 --- a/frontend/coprs_frontend/coprs/rest_api/resources/project.py +++ b/frontend/coprs_frontend/coprs/rest_api/resources/project.py @@ -1,7 +1,4 @@ from logging import getLogger -from coprs.rest_api.exceptions import MalformedRequest - -log = getLogger(__name__) import flask from flask import url_for, make_response @@ -19,7 +16,9 @@ from ...exceptions import DuplicateException from ..common import rest_api_auth_required, render_copr_chroot, render_build, render_project, get_project_safe from ..schemas import ProjectSchema, ProjectCreateSchema from ..exceptions import ObjectAlreadyExists, CannotProcessRequest, AccessForbidden -from ..util import mm_deserialize +from ..util import mm_deserialize, get_request_parser + +log = getLogger(__name__) class ProjectListR(Resource): @@ -52,7 +51,7 @@ class ProjectListR(Resource): return resp def get(self): - parser = reqparse.RequestParser() + parser = get_request_parser() parser.add_argument('owner', type=str) parser.add_argument('name', type=str) parser.add_argument('limit', type=int) @@ -111,7 +110,7 @@ class ProjectR(Resource): return None, 204 def get(self, project_id): - parser = reqparse.RequestParser() + parser = get_request_parser() parser.add_argument('show_builds', type=bool, default=False) parser.add_argument('show_chroots', type=bool, default=False) req_args = parser.parse_args() diff --git a/frontend/coprs_frontend/coprs/rest_api/schemas.py b/frontend/coprs_frontend/coprs/rest_api/schemas.py index cb6f867..b28d1c6 100644 --- a/frontend/coprs_frontend/coprs/rest_api/schemas.py +++ b/frontend/coprs_frontend/coprs/rest_api/schemas.py @@ -128,13 +128,14 @@ class CoprChrootCreateSchema(CoprChrootSchema): name = fields.Str(required=True) -class BuildChrootSchema(Schema): +class BuildTaskSchema(Schema): # used only for presentation state = fields.Str() started_on = fields.Int(dump_only=True) ended_on = fields.Int(dump_only=True) git_hash = fields.Str(dump_only=True) - name = fields.Str(dump_only=True) + chroot_name = fields.Str(dump_only=True, attribute="name") + build_id = fields.Int(dump_only=True) result_dir_url = fields.Str(dump_only=True) diff --git a/frontend/coprs_frontend/coprs/rest_api/util.py b/frontend/coprs_frontend/coprs/rest_api/util.py index d937784..402edd1 100644 --- a/frontend/coprs_frontend/coprs/rest_api/util.py +++ b/frontend/coprs_frontend/coprs/rest_api/util.py @@ -1,10 +1,13 @@ # coding: utf-8 import json +from flask import Response, url_for, Blueprint import sqlalchemy.orm.exc + +from flask_restful.reqparse import Argument, RequestParser + from .exceptions import ObjectNotFoundError, MalformedRequest -from schemas import AllowedMethodSchema -from flask import Response, url_for, Blueprint +from .schemas import AllowedMethodSchema class AllowedMethod(object): @@ -62,3 +65,21 @@ def mm_deserialize(schema, json_string): def mm_serialize_one(schema, obj): return schema().dump(obj)[0] + + +class MyArg(Argument): + def handle_validation_error(self, error, bundle_errors): + # dirty monkey patching, better to switch to webargs + # bundle errors are ignored + data = {u"error": unicode(error)} + if self.help: + data["help"] = self.help + raise MalformedRequest( + "Failed to validate query parameter: {}".format(self.name), + data=data + ) + + +def get_request_parser(): + return RequestParser(argument_class=MyArg) + #return RequestParser(bundle_errors=True) diff --git a/frontend/docs/api_2/source/Resources/build_chroot.rst b/frontend/docs/api_2/source/Resources/build_task.rst similarity index 73% rename from frontend/docs/api_2/source/Resources/build_chroot.rst rename to frontend/docs/api_2/source/Resources/build_task.rst index f767fbe..c5871a4 100644 --- a/frontend/docs/api_2/source/Resources/build_chroot.rst +++ b/frontend/docs/api_2/source/Resources/build_task.rst @@ -1,15 +1,16 @@ -Build Chroot -============ +Build Task +========== -Build chroot represents information about individual build tasks for each chroot. +Build task represents information about individual build tasks per each chroot. -Structure of the build chroot entity ------------------------------------- +Structure of the build task entity +---------------------------------- .. code-block:: javascript { "name": "fedora-rawhide-x86_64", + "build_id": 12345, "started_on": 1440753865, "ended_on": 1440753919, "state": "succeeded", @@ -22,7 +23,8 @@ Build chroot fields ================== ==================== =============== Field Type Description ================== ==================== =============== -name str chroot name +chroot_name str chroot name +build_id int unique build identifier state str current build state started_on int(unixtime UTC) time when the build chroot started ended_on int(unixtime UTC) time when the build chroot ended @@ -38,11 +40,24 @@ result_dir_url str(URL) location of the build results List build chroots ------------------ -.. http:get:: /api_2/builds/(int:build_id)/chroots +.. http:get:: /api_2/builds_tasks - Returns list of build chroots contained in the one build + Returns list of build tasks according to the given query parameters - :param int build_id: a unique identifier of the build + :query str owner: select build tasks from projects owned by this user + :query int project_id: + select build tasks from one project, + when used query parameter ``owner`` is ignored + :query int build_id: + select build tasks from one project, + when used query parameters ``owner`` and ``project_id`` are ignored + + :query int offset: offset number, default value is 0 + :query int limit: limit number, default value is 100 + :query str state: + select builds in particular state, allowed values: + ``failed``, ``succeeded``, ``canceled``, ``running``, + ``pending``, ``starting``, ``importing`` :statuscode 200: no error :statuscode 404: build not found @@ -51,7 +66,7 @@ List build chroots .. sourcecode:: http - GET /api_2/builds/106882/chroots HTTP/1.1 + GET /api_2/builds_tasks?build_id=106882 HTTP/1.1 Host: copr.fedoraproject.org **Response** @@ -91,18 +106,18 @@ List build chroots -Get build chroot details ------------------------- +Get build task details +---------------------- .. http:get:: /api_2/builds/(int:build_id)/chroots/(str:name) - Returns details about one build chroot + Returns details about one build task :param int build_id: a unique identifier of the build :param str name: chroot name :statuscode 200: no error - :statuscode 404: build or build chroot not found + :statuscode 404: build or build task not found **Example request** diff --git a/frontend/docs/api_2/source/index.rst b/frontend/docs/api_2/source/index.rst index d7dfcba..3ed6a68 100644 --- a/frontend/docs/api_2/source/index.rst +++ b/frontend/docs/api_2/source/index.rst @@ -22,7 +22,7 @@ Resources Resources/project Resources/project_chroot Resources/build - Resources/build_chroot + Resources/build_task Resources/mock_chroot

8 years, 8 months

1
0
0 / 0

[copr] master: [frontend][API] added section about errors (057cd1f)

by vgologuz＠fedoraproject.org

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit 057cd1f4f452b65201ca1e0960588417e444cf50 Author: Valentin Gologuzov <vgologuz(a)redhat.com> Date: Mon Sep 7 16:38:13 2015 +0200 [frontend][API] added section about errors >--------------------------------------------------------------- .../docs/api_2/source/Resources/project_chroot.rst | 1 - frontend/docs/api_2/source/index.rst | 28 +++++++++++++++++++- 2 files changed, 27 insertions(+), 2 deletions(-) diff --git a/frontend/docs/api_2/source/Resources/project_chroot.rst b/frontend/docs/api_2/source/Resources/project_chroot.rst index a4d6829..48288dc 100644 --- a/frontend/docs/api_2/source/Resources/project_chroot.rst +++ b/frontend/docs/api_2/source/Resources/project_chroot.rst @@ -48,7 +48,6 @@ List project chroots GET /api_2/projects/2482/chroots HTTP/1.1 Host: copr.fedoraproject.org - Accept: application/json **Response** diff --git a/frontend/docs/api_2/source/index.rst b/frontend/docs/api_2/source/index.rst index f860a63..d7dfcba 100644 --- a/frontend/docs/api_2/source/index.rst +++ b/frontend/docs/api_2/source/index.rst @@ -103,7 +103,33 @@ GET requests would return the following structures: Errors ______ -todo: + +To distinguish errors we use standard HTTP codes: https://en.wikipedia.org/wiki/List_of_HTTP_status_codes. +Additional information may be contained in the response body, it SHOULD have `application/json` Content-Type. +Inside json object, there are MUST be key ``message`` with error description, +additional information MAY be present at key ``data`` + + **Example** + + .. sourcecode:: http + + GET /api_2/builds?project_id=999999999 HTTP/1.1 + Host: copr.fedoraproject.org + + **Response** + + .. sourcecode:: http + + HTTP/1.1 404 NOT FOUND + Content-Type: application/json + + { + "message": "Project with id `999999999` not found", + "data": { + "project_id": 999999999 + } + } + .. _BasicAuth: https://en.wikipedia.org/wiki/Basic_access_authentication .. _CoprAPI: https://copr.fedoraproject.org/api

8 years, 9 months

1
0
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

copr-commits September 2015