[SciPy-user] numpy.testing and other test runners

[Neilen Marais]

Hi

I've been playing around with using other test runners together with my
numpy.testing based tests. I've tried nosetest and TestGears so far. Has anyone
else tried any others, and what have your experiences been?

Nosetest is kinda nice since it allows you to run pdb on tests that error out,
and allows you to specify tests by name from the commandline. I've been using
nosetest on another project that uses only normal unittest derrived tests for a
while. On NumpyTestCase derived tests it would not run at first, but I got it
working by hacking a little on numpy/testing/numpytest.py:

--- numpy/testing/numpytest.py.orig 2006-06-16 22:40:11.000000000 +0200
+++ numpy/testing/numpytest.py 2006-06-28 14:18:45.000000000 +0200
@@ -139,7 +139,7 @@
result.stream = _dummy_stream(save_stream)
unittest.TestCase.__call__(self, result)
if nof_errors != len(result.errors):
- test, errstr = result.errors[-1]
+ test, errstr = result.errors[-1][0:2]
if isinstance(errstr, tuple):
errstr = str(errstr[0])
elif isinstance(errstr, str):

I don't know the unittest API well enough to know if this is a bug in numpytest
or in nose, but it seems to work OK.

TestGears looks interesting since it can generate a unittest compatible
test-suite using its own collector. Ideally you could then feed this testsuite
to a GUI runner or something similar. I've only tried it quickly, but couldn't
get it to work.

Cheers
Neilen

[David M. Cooke]

On Wed, 28 Jun 2006 16:09:12 +0200

Post by Neilen Marais
Hi
I've been playing around with using other test runners together with my
numpy.testing based tests. I've tried nosetest and TestGears so far. Has
anyone else tried any others, and what have your experiences been?
Nosetest is kinda nice since it allows you to run pdb on tests that error
out, and allows you to specify tests by name from the commandline. I've
been using nosetest on another project that uses only normal unittest
derrived tests for a while. On NumpyTestCase derived tests it would not run
at first, but I got it working by hacking a little on
--- numpy/testing/numpytest.py.orig 2006-06-16 22:40:11.000000000 +0200
+++ numpy/testing/numpytest.py 2006-06-28 14:18:45.000000000 +0200
@@ -139,7 +139,7 @@
result.stream = _dummy_stream(save_stream)
unittest.TestCase.__call__(self, result)
- test, errstr = result.errors[-1]
+ test, errstr = result.errors[-1][0:2]
errstr = str(errstr[0])
I don't know the unittest API well enough to know if this is a bug in
numpytest or in nose, but it seems to work OK.

I applied the patch as it won't impact numpy (it just loosens the requirement
that len(result.errors) == 2 to >= 2).

[Bryce Hendrix]

We use a slightly customized version of Testoob for testing enthought
packages. I've ran it on numpy in the past, but have temporarily
disabled continuous building and testing of numpy.

Bryce

Post by David M. Cooke
On Wed, 28 Jun 2006 16:09:12 +0200

Post by Neilen Marais
Hi
I've been playing around with using other test runners together with my
numpy.testing based tests. I've tried nosetest and TestGears so far. Has
anyone else tried any others, and what have your experiences been?
Nosetest is kinda nice since it allows you to run pdb on tests that error
out, and allows you to specify tests by name from the commandline. I've
been using nosetest on another project that uses only normal unittest
derrived tests for a while. On NumpyTestCase derived tests it would not run
at first, but I got it working by hacking a little on
--- numpy/testing/numpytest.py.orig 2006-06-16 22:40:11.000000000 +0200
+++ numpy/testing/numpytest.py 2006-06-28 14:18:45.000000000 +0200
@@ -139,7 +139,7 @@
result.stream = _dummy_stream(save_stream)
unittest.TestCase.__call__(self, result)
- test, errstr = result.errors[-1]
+ test, errstr = result.errors[-1][0:2]
errstr = str(errstr[0])
I don't know the unittest API well enough to know if this is a bug in
numpytest or in nose, but it seems to work OK.

I applied the patch as it won't impact numpy (it just loosens the requirement
that len(result.errors) == 2 to >= 2).

[Bryce Hendrix]

We use a slightly customized version of Testoob for testing enthought
packages. I've ran it on numpy in the past, but have temporarily
disabled continuous building and testing of numpy.

Bryce

Post by David M. Cooke
On Wed, 28 Jun 2006 16:09:12 +0200

Post by Neilen Marais
Hi
I've been playing around with using other test runners together with my
numpy.testing based tests. I've tried nosetest and TestGears so far. Has
anyone else tried any others, and what have your experiences been?
Nosetest is kinda nice since it allows you to run pdb on tests that error
out, and allows you to specify tests by name from the commandline. I've
been using nosetest on another project that uses only normal unittest
derrived tests for a while. On NumpyTestCase derived tests it would not run
at first, but I got it working by hacking a little on
--- numpy/testing/numpytest.py.orig 2006-06-16 22:40:11.000000000 +0200
+++ numpy/testing/numpytest.py 2006-06-28 14:18:45.000000000 +0200
@@ -139,7 +139,7 @@
result.stream = _dummy_stream(save_stream)
unittest.TestCase.__call__(self, result)
- test, errstr = result.errors[-1]
+ test, errstr = result.errors[-1][0:2]
errstr = str(errstr[0])
I don't know the unittest API well enough to know if this is a bug in
numpytest or in nose, but it seems to work OK.

I applied the patch as it won't impact numpy (it just loosens the requirement
that len(result.errors) == 2 to >= 2).

[David M. Cooke]

On Wed, 28 Jun 2006 16:09:12 +0200

Post by Neilen Marais
Hi
I've been playing around with using other test runners together with my
numpy.testing based tests. I've tried nosetest and TestGears so far. Has
anyone else tried any others, and what have your experiences been?
Nosetest is kinda nice since it allows you to run pdb on tests that error
out, and allows you to specify tests by name from the commandline. I've
been using nosetest on another project that uses only normal unittest
derrived tests for a while. On NumpyTestCase derived tests it would not run
at first, but I got it working by hacking a little on
--- numpy/testing/numpytest.py.orig 2006-06-16 22:40:11.000000000 +0200
+++ numpy/testing/numpytest.py 2006-06-28 14:18:45.000000000 +0200
@@ -139,7 +139,7 @@
result.stream = _dummy_stream(save_stream)
unittest.TestCase.__call__(self, result)
- test, errstr = result.errors[-1]
+ test, errstr = result.errors[-1][0:2]
errstr = str(errstr[0])
I don't know the unittest API well enough to know if this is a bug in
numpytest or in nose, but it seems to work OK.

I applied the patch as it won't impact numpy (it just loosens the requirement
that len(result.errors) == 2 to >= 2).