Huh? That is irrelevant to what I actually wrote.

My experience of it was that is was pretty pointless for true TDD - after all, there is no code for it to generate tests for, until after you’ve already written the tests, right?

Have you tried its assertions?

This statement seems to imply something unusual about your understanding of TDD to me.

My experience of it was that is was pretty pointless for true TDD - after all, there is no code for it to generate tests for, until after you’ve already written the tests, right?

You could run it against your code later, as some sort of sanity check, but I didn’t find that a useful exercise.

For existing non-unit tested code (which is approximately all the code ever written) it comes into its own, although I frequently found a need to delete irrelevant tests in my trials.

That’s not true. Of course you can have non-RT methods where you know you have non-executing code paths.

But Pex works on methods, not classes. It won’t solve the problem of state. It is really useful for TDD, though, since it can find cases you might have missed, with nothing more than a prototype.

It still seems to me that you are worrying about the number of input values to methods.

No. I’m concerned about being able to hit the "interesting" cases. Exhaustive testing is just a really, really inefficent way of doing that.

It is possible that you will not be able to exercise all the code paths in the private methods by calling the public method. But that actually doesn’t matter, since any code calling that method can’t hit those code paths either.

Once more: Unless the methods under test are truly referentially transparent, you can’t know this.

> The argument is, how do you ensure that you
> have hit the useful test cases? Without
> thehidden input of state, this is really easy
> to do. When state is an input to the function,
> it is very difficult to do.

It still seems to me that you are worrying about the number of input values to methods. You are then getting concerned about combinatorial explosions of the number of test cases that might be required if internal (not visible to unit test) state is used in those methods.

But input values are not how you work out what tests are required. You only need to be concerned about code paths, both the obvious ones that coverage tools can measure (eg cyclomatic complexity), and less obvious ones like exceptions (and non-exceptions like the integer multiplication example, possibly).

Now, if a public method uses two private methods, you only need to supply the public method with sufficient input values to exercise all the code paths in those 3 methods. That’s got to be orders of magnitude less tests in almost all cases

It is possible that you will not be able to exercise all the code paths in the private methods by calling the public method. But that actually doesn’t matter, since any code calling that method can’t hit those code paths either. If you extend the same reasoning throughout the whole public interface of a class, you can see that it may not be necessary to get 100% test coverage.

This is one reason why private methods rarely need direct testing.

My experience suggests the heuristic of only testing public methods works pretty much alway the time. Any complex private stuff that needed testing I’ve always had to extract elsewhere, as far as I can remember.

Needing to unit test a private method directly is a code smell, IMO.

Generally speaking, I find a human brain is required to determine the required unit tests, but there is an interesting tool under development at MS:

http://research.microsoft.com/en-us/projects/Pex/

Since I prefer to do TDD, it’s not that useful to me currently (all new code, fortunately), but it might be quite handy for legacy code.

That’s a lot, and perhaps we would not want to test every possible case…

If it seems that I’ve based my argument on the possible number of inputs, then either I was not clear, or you miss the point. The argument is, how do you ensure that you have hit the useful test cases? Without the hidden input of state, this is really easy to do. When state is an input to the function, it is very difficult to do.

If the public function under test is guaranteed to be a referentially transparent function, then I agree that testing the public function is all that is required, no matter what it may do internally. But it is both difficult and rare to make that guarantee in Delphi or C# code.

Actually, no I don’t.

In the real world, I wouldn’t normally test it, as, yes, I think it is too simple to test (in most situations, anyway). It is certainly not worth making it public solely so you could test it.

The example tests I gave were just to demonstrate that 2^32 tests are not required to completely exercise the code. I think 2 or 3 does it, don’t you?

Basing your argument on the possible number of inputs was a red herring, IMO.

Neither do I - and that’s exactly what I said.

> Finally, the specification I referred to was the one in your comment, namely that Foo "can
> also throw an exception for sufficiently large a." This is demonstrably incorrect.

I see. Well, as you yourself implied, the code **can** throw an exception. You need the right compiler options, but it still can.

So if you were testing Foo you would write at least one test:

[Test]
public void Foo()
{
Assert.That(Foo(1), Is.EqualTo(2));
}

So you’re covered if someone changes Foo to multiply by 3 or something.

If you wanted the code to throw an exception (you might not, of course) you might also test for that - the boundary conditions, if you will :

[Test]
[ExpectedException(typeof(OverflowException))]
public void FooMaxInt()
{
Foo(int.MaxValue);
}

[Test]
[ExpectedException(typeof(OverflowException))]
public void FooMinInt()
{
Foo(int.MinValue);
}

At this point you could discover you do not have the compiler switch right, so the tests fail. You turn the switch on and the tests pass. TDD in action.

> I feel like we’re going in circles here, and getting away from the original point, namely
> that testing is by itself a valid use case.

I’m a big proponent of unit testing, but I didn’t agree with your 2^32 numbers and the subsequent argument. I still don’t.

That you should write testable code I completely agree with, hence my use of TDD whenever possible. That you might need to refactor to be able to test something more easily is of course true. Testability is one measure of design quality.

However, I do not (in general) equate unit tests with use cases. Unit testing is for classes; use cases should not be that detailed (you’d have to refactor your use cases all the time, it’d just be silly).

In principle, I wouldn’t make unit testing a use case either, I don’t think. In my mind it is just good, professional programming practice. You should just do it whenever it is appropriate. I see no need to explicitly say when it should be done.

Higher level tests (UI, integration, performance etc) are required to test use cases or user stories, IMO (as they are higher level constructs than classes). And I wouldn’t get so recursive as to make that sort of testing a use case

I would get someone else to do that type of testing, too (seriously).