>
> The existing run_tests.py script is a non-starter with python3, so if we
> want to even look at adding python3 support to Roundup the run_tests.py
> script would need to be updated to add python3 support. This could be done,
> but also begs the question of why are we maintaining our own test runner
> script? Why not just push that maintenance burden off to some other project
> (ie. pytest)?
>

Seems fair. As long as we maintain our own *short* user level usage
documentation. Stating that we use pytest will redirect people to
comprehensive pytest docs, which I most people doing testing don't really
have time to read. And I also would like to avoid a ton of options that we
don't need. Some testing utilities are frightening me away when I try to
read through their options..

> I personally find the documentation for the existing script ('run_tests.py
> -h') a bit confusing, I'm not sure if I have ever figured out how to run a
> single test case using the existing script, and some options don't behave
> as expected (ie. 'run_tests.py --dir test' works as one might expect, but
> 'run_tests.py --dir test/' doesn't).
>

Yes. I remember I was doing this part, but I am not sure if it was SCons or
Roundup, so my requirement for test utility are:
1. list tests with quick filter (preview test selection)
2. run tests filtered by list
3. run single test
4. run tests that match substring

In ideal world I'd wrap pytest or similar project and strip all the option
that are not used in a project unless user specifies -v -h (--verbose
--help).

There are also a number of benefits that adopting pytest could provide
> (less boilerplate, fixtures, using 'assert', etc.) if we were to fully
> embrace it, but the one nice benefit I have included in the current
> patchset is the support for test skipping decorators it brings to python2.6
> (not available in unittest before python2.7). When running tests you now
> know how may tests have been skipped (and why), instead of just being told
> some tests won't be run.
>

This is a much better argumentation. Thanks. =) Are the tests become hard
depending on pytest, or it is some general practice how to skip stuff?

> If the consensus is to use another tool, then I am OK with that.
>

I don't have a preference, but it would be nice to keep tests in some
format that is independent of the tool, and for this specific problem
choose tool that is not huge. =)

Also, it looks like we don't have continuous testing setup. There are
plenty of services which we can add to the home page http://shields.io/(And
link to PyPI)