Roundup Tracker - Issues

Issue 2550905

classification
Title: Improving the test coverage of the Roundup test suite
Type: Severity: normal
Components: Versions:
process
Status: new Resolution:
Dependencies: Superseder:
Assigned To: pefu Nosy List: ber, pefu, rouilj
Priority: normal Keywords: Effort-High

Created on 2016-03-08 12:28 by pefu, last changed 2016-06-29 10:03 by pefu.

Files
File name Uploaded Description Edit Remove
roundup_coverage_report.txt pefu, 2016-03-08 12:28
Messages
msg5488 Author: [hidden] (pefu) Date: 2016-03-08 12:28
After installing the latest py.test and the pytest-cov plugin using pip
I ran the following command in my hg Roundup source repository dir::

  py.test --cov=.

The output of this test run is attached as roundup_coverage_report.txt

My proposal is that we developers try to increase the test coverage
on the roundup code base over time.  
This is an engineering task which will not be easy.  
But I believe it will pay off in the end.  
I create this general issue here now so we have one to refer to in
commit messages.
msg5490 Author: [hidden] (ber) Date: 2016-03-09 15:48
On Tuesday 08 March 2016 at 13:28:37, Peter Funk wrote:
> My proposal is that we developers try to increase the test coverage
> on the roundup code base over time.  

It is always good to improve automatic tests.
But 100% test coverage usually is a mistake,
it has to be a reasonable coverage,
balancing out the drawbacks of too many tests.
(One drawback is that too many tests make
are additional code that has defects and raise maintenance costs 
and the bars for changing the code.)
msg5491 Author: [hidden] (pefu) Date: 2016-03-09 15:58
Bernhard Reiter, 09.03.2016, 15:48:
> On Tuesday 08 March 2016 at 13:28:37, Peter Funk wrote:
> > My proposal is that we developers try to increase the test coverage
> > on the roundup code base over time.  
> 
> It is always good to improve automatic tests.
> But 100% test coverage usually is a mistake,
> it has to be a reasonable coverage,
> balancing out the drawbacks of too many tests.
> (One drawback is that too many tests make
> are additional code that has defects and raise maintenance costs 
> and the bars for changing the code.)

Yes, I agree:  
Writing maintainable test code is hard and requires engineering.
And I've to admit that I've still a lot to learn in this area.
When I studied computer science TDD wasn't even invented yet! ☺

Regards, Peter Funk
-- 
Peter Funk, home: ✉Oldenburger Str.86, D-27777 Ganderkesee
mobile:+49-179-640-8878 phone:+49-421-20419-0 <http://www.artcom-gmbh.de/>
office: ArtCom GmbH, ✉Haferwende 2, D-28357 Bremen, Germany
msg5681 Author: [hidden] (rouilj) Date: 2016-06-29 02:49
I am in the process of adding some tests for some do_*
functions in admin.py. Do we have a percentage coverage
target, or any criteria for what we should check first?

My particular round of additional testing was due to
issue2550572 bug in roundup_admin. So I was testing the
code paths that failed.

-- rouilj
msg5682 Author: [hidden] (ber) Date: 2016-06-29 08:16
| Do we have a percentage coverage
| target, or any criteria for what we should check first?

AFAIK we don't.
msg5683 Author: [hidden] (pefu) Date: 2016-06-29 10:03
Hello all,

Bernhard Reiter added the comment29.06.2016 um 08:16:
> | Do we have a percentage coverage
> | target, or any criteria for what we should check first?
> 
> AFAIK we don't.

Trying to achieve a certain coverage percentage would not
automatically make Roundup a better software.  

However:  IMHO It would help the project a lot if we could find 
and agree on some way (for example a naming scheme ?) so that 
we can distinguish unit tests from component or integration tests.

Also it would help, if unit tests aleady exist, to document where
to find those unit tests belonging to certain parts of the 
roundup code base.

I'm feeling myself still not experienced enough with test driven 
development to propose the right steps in this direction.
 
Best Regards, Peter Funk
History
Date User Action Args
2016-06-29 10:03:21pefusetmessages: + msg5683
2016-06-29 08:16:06bersetmessages: + msg5682
2016-06-29 02:49:24rouiljsetnosy: + rouilj
messages: + msg5681
2016-03-09 15:58:36pefusetmessages: + msg5491
2016-03-09 15:48:28bersetmessages: + msg5490
2016-03-08 12:28:37pefucreate