Antoni Segura Puimedon has uploaded a new change for review.
Change subject: tests: Add time per test if nose-timer is available ......................................................................
tests: Add time per test if nose-timer is available
This patch adds an output like:
resourceManagerTests.ResourceManagerTests.testStressTest: 12.8313s resourceManagerTests.ResourceManagerTests.testResourceAcquireTimeout: 1.0069s resourceManagerTests.ResourceManagerTests.testResourceLockSwitch: 0.0108s resourceManagerTests.ResourceManagerTests.testResourceAutorelease: 0.0101s resourceManagerTests.ResourceManagerTests.testResourceInvalidation: 0.0093s resourceManagerTests.ResourceManagerTests.testCrashOnSwitch: 0.0067s resourceManagerTests.ResourceManagerTests.testResourceSwitchLockTypeFail: 0.0062s resourceManagerTests.ResourceManagerTests.testCancelExclusiveBetweenShared: 0.0049s resourceManagerTests.ResourceManagerTests.testAcquireResourceExclusive: 0.0042s resourceManagerTests.ResourceManagerTests.testCancelRequest: 0.0031s resourceManagerTests.ResourceManagerTests.testRequestRefCmp: 0.0030s resourceManagerTests.ResourceManagerTests.testRequestRecancel: 0.0030s resourceManagerTests.ResourceManagerTests.testResourceStatuses: 0.0029s resourceManagerTests.ResourceManagerTests.testAccessAttributeNotExposedByRequestRef: 0.0029s resourceManagerTests.ResourceManagerTests.testFailCreateAfterSwitch: 0.0025s resourceManagerTests.ResourceManagerTests.testRequestWithBadCallbackOnCancel: 0.0025s resourceManagerTests.ResourceManagerTests.testAcquireResourceShared: 0.0023s resourceManagerTests.ResourceManagerTests.testRereleaseResource: 0.0022s resourceManagerTests.ResourceManagerTests.testResourceWrapper: 0.0020s resourceManagerTests.ResourceManagerTests.testRequestWithBadCallbackOnGrant: 0.0020s resourceManagerTests.ResourceManagerTests.testRequestRefStr: 0.0020s resourceManagerTests.ResourceManagerTests.testAccessAttributeNotExposedByWrapper: 0.0019s resourceManagerTests.ResourceManagerTests.testErrorInFactory: 0.0017s resourceManagerTests.ResourceManagerTests.testAcquireNonExistingResource: 0.0016s resourceManagerTests.ResourceManagerTests.testRequestInvalidResource: 0.0015s resourceManagerTests.ResourceManagerTests.testRequestRegrant: 0.0015s resourceManagerTests.ResourceManagerTests.testResourceAcquireInvalidTimeout: 0.0014s resourceManagerTests.ResourceManagerTests.testReleaseInvalidResource: 0.0014s resourceManagerTests.ResourceManagerTests.testSingleton: 0.0014s resourceManagerTests.ResourceManagerTests.testForceRegisterNamespace: 0.0013s resourceManagerTests.ResourceManagerTests.testListNamespaces: 0.0013s resourceManagerTests.ResourceManagerTests.testReregisterNamespace: 0.0012s resourceManagerTests.ResourceManagerTests.testRegisterInvalidNamespace: 0.0012s
At the end of the usual tests if nose-timer is detected in the system. It shows the tests over 500ms in yellow and those over 5s in red.
If you want to get this output, do: pip install nose-timer
Change-Id: I4960b30532a84abd259fb268c7e40a1751847a96 Signed-off-by: Antoni S. Puimedon asegurap@redhat.com --- M tests/run_tests_local.sh.in M tests/testrunner.py 2 files changed, 18 insertions(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/71/30171/1
diff --git a/tests/run_tests_local.sh.in b/tests/run_tests_local.sh.in index d19f22c..f51473d 100644 --- a/tests/run_tests_local.sh.in +++ b/tests/run_tests_local.sh.in @@ -7,4 +7,9 @@ @top_srcdir@/tests/makecert.sh fi
-PYTHONDONTWRITEBYTECODE=1 LC_ALL=C PYTHONPATH="@top_srcdir@/lib:@top_srcdir@/vdsm:@top_srcdir@/client:@top_srcdir@/vdsm_api:$PYTHONPATH" "$PYTHON_EXE" @top_srcdir@/tests/testrunner.py --local-modules $@ +python -c 'import nosetimer' > /dev/null 2>&1 +if [ $? -eq 0 ]; then + EXTRA_ARGS='--with-timer' +fi + +PYTHONDONTWRITEBYTECODE=1 LC_ALL=C PYTHONPATH="@top_srcdir@/lib:@top_srcdir@/vdsm:@top_srcdir@/client:@top_srcdir@/vdsm_api:$PYTHONPATH" "$PYTHON_EXE" @top_srcdir@/tests/testrunner.py --local-modules $@ "$EXTRA_ARGS" diff --git a/tests/testrunner.py b/tests/testrunner.py index eed743f..ec24c2b 100644 --- a/tests/testrunner.py +++ b/tests/testrunner.py @@ -40,6 +40,16 @@ from nose import config from nose import core from nose import result +try: + from nosetimer.plugin import TimerPlugin +except ImportError: + timer_plugin = None +else: + timer_plugin = TimerPlugin() + timer_plugin.enabled = True + timer_plugin.timer_ok = 500 # Okay <= 500ms + timer_plugin.timer_warning = 5000 # Warn > 5s + timer_plugin.timer_no_color = False # Enable color output
from testValidation import SlowTestsPlugin, StressTestsPlugin
@@ -378,6 +388,8 @@ plugins=core.DefaultPluginManager()) conf.plugins.addPlugin(SlowTestsPlugin()) conf.plugins.addPlugin(StressTestsPlugin()) + if timer_plugin is not None: + conf.plugins.addPlugin(timer_plugin)
runner = VdsmTestRunner(stream=conf.stream, verbosity=conf.verbosity,
Francesco Romani has posted comments on this change.
Change subject: tests: Add time per test if nose-timer is available ......................................................................
Patch Set 1: Code-Review+1
nice! IIUC this is optional, so one can avoid to be overwhelmed by output if (s)he likes so. If so, then +1 from me!
oVirt Jenkins CI Server has posted comments on this change.
Change subject: tests: Add time per test if nose-timer is available ......................................................................
Patch Set 1:
Build Successful
http://jenkins.ovirt.org/job/vdsm_master_unit_tests_gerrit_el/10264/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/11049/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/11206/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_network_functional_tests_gerrit/163... : SUCCESS
Antoni Segura Puimedon has posted comments on this change.
Change subject: tests: Add time per test if nose-timer is available ......................................................................
Patch Set 1: Verified+1
Nir Soffer has posted comments on this change.
Change subject: tests: Add time per test if nose-timer is available ......................................................................
Patch Set 1:
(1 comment)
http://gerrit.ovirt.org/#/c/30171/1/tests/testrunner.py File tests/testrunner.py:
Line 48: timer_plugin = TimerPlugin() Line 49: timer_plugin.enabled = True Line 50: timer_plugin.timer_ok = 500 # Okay <= 500ms Line 51: timer_plugin.timer_warning = 5000 # Warn > 5s Line 52: timer_plugin.timer_no_color = False # Enable color output These defaults are not correct for all tests.
Some tests are slow because there is no other way to test without waiting, and for these the coloring means nothing.
This should be per test method or per class setting, to allow meaningful output. Line 53: Line 54: from testValidation import SlowTestsPlugin, StressTestsPlugin Line 55: Line 56: import zombiereaper
Nir Soffer has posted comments on this change.
Change subject: tests: Add time per test if nose-timer is available ......................................................................
Patch Set 1:
(1 comment)
http://gerrit.ovirt.org/#/c/30171/1/tests/run_tests_local.sh.in File tests/run_tests_local.sh.in:
Line 9: Line 10: python -c 'import nosetimer' > /dev/null 2>&1 Line 11: if [ $? -eq 0 ]; then Line 12: EXTRA_ARGS='--with-timer' Line 13: fi Can we do this in testrunner.py? can be much simpler. Line 14:
Dan Kenigsberg has posted comments on this change.
Change subject: tests: Add time per test if nose-timer is available ......................................................................
Patch Set 1:
(1 comment)
http://gerrit.ovirt.org/#/c/30171/1/tests/testrunner.py File tests/testrunner.py:
Line 48: timer_plugin = TimerPlugin() Line 49: timer_plugin.enabled = True Line 50: timer_plugin.timer_ok = 500 # Okay <= 500ms Line 51: timer_plugin.timer_warning = 5000 # Warn > 5s Line 52: timer_plugin.timer_no_color = False # Enable color output
These defaults are not correct for all tests.
these numbers are indeed a bit arbitrary, but the alternative of tweaking the plugin and running all over our tests defining acceptable times do not sound appealing, either.
Can we drop colours instead? Line 53: Line 54: from testValidation import SlowTestsPlugin, StressTestsPlugin Line 55: Line 56: import zombiereaper
Antoni Segura Puimedon has posted comments on this change.
Change subject: tests: Add time per test if nose-timer is available ......................................................................
Patch Set 2: Verified+1
oVirt Jenkins CI Server has posted comments on this change.
Change subject: tests: Add time per test if nose-timer is available ......................................................................
Patch Set 2:
Build Successful
http://jenkins.ovirt.org/job/vdsm_master_unit_tests_gerrit_el/10905/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/11847/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/11690/ : SUCCESS
Nir Soffer has posted comments on this change.
Change subject: tests: Add time per test if nose-timer is available ......................................................................
Patch Set 2:
(1 comment)
http://gerrit.ovirt.org/#/c/30171/2/tests/run_tests_local.sh.in File tests/run_tests_local.sh.in:
Line 9: Line 10: python -c 'import nosetimer' > /dev/null 2>&1 Line 11: if [ $? -eq 0 ]; then Line 12: EXTRA_ARGS='--with-timer' Line 13: fi Can we do this inside testlib.py?
I would like to eliminate this wrapper. the tests should run using:
nosetest Line 14:
Dan Kenigsberg has abandoned this change.
Change subject: tests: Add time per test if nose-timer is available ......................................................................
Abandoned
automation@ovirt.org has posted comments on this change.
Change subject: tests: Add time per test if nose-timer is available ......................................................................
Patch Set 2:
* Update tracker::IGNORE, no Bug-Url found
Jenkins CI RO has abandoned this change.
Change subject: tests: Add time per test if nose-timer is available ......................................................................
Abandoned
Abandoned due to no activity - please restore if still relevant
gerrit-hooks has posted comments on this change.
Change subject: tests: Add time per test if nose-timer is available ......................................................................
Patch Set 3:
* Update tracker: IGNORE, no Bug-Url found
vdsm-patches@lists.fedorahosted.org