Adventures in C#: Using NUnit

Some tips and techniques for using NUnit, the testing framework used in this book.

• Testing: a Core Practice
• NUnit
• Setting Up NUnit
• Finding a Simple Pattern
• Paired Tags
• Paired Unknown Tags
• Unmatched Tags
• What's Left?
• A Short Look at Debugging Under NUnit
• The Debug Startup Program
• Attach to the NUnit Process
• Choosing a Technique
• Summary

As you read this book, you'll see that we do a lot of testing. We write most of our tests before we write the code, one test a a time. A little test, a little code to make it work, and so on. We work that way for some good reasons, which you'll see as we go along:

• Most important, well-tested code works better. Customers like it better.
• Testing as we write means we spend less time debugging. We get our program done faster/.
• Testing as we write means that we don't have those long testing cycles at the end of our projects. We like working without that death match thing.
• Our tests are the first users of our code. We experience what it is like to use our code very quickly. The design turns out better.
• Tests support refactoring. Since we want to ship useful function early and often, we know that we'll be evolving the design with refactoring.
• Tests give us confidence. We're able to work with less stress, and we're not afraid to experiment as we go.
• Testing before coding is more interesting than testing after we code. Because it's interesting, we find it easier to maintain what we know is a good practice.

But I'm not here to sell testing, I'm here to tell you how we do it. Let's move along.

As we work with the XML Notepad, I'm expecting that we'll have to look at text strings a lot and figure out whether they have any XML tags in them. XML tags, as you probably know, come in matched pairs of strings embedded in squiggles: <SomeTag>like this</SomeTag>. Now there are some powerful features in .NET for working with XML, and later on in the book we'll look at some of them. But we're here to learn C#, and part of that will mean digging down a little deeper into how things work. Since C# has some powerful support for regular expressions, I decided to learn a little about them.

Often when I experiment with something new, I just write some throwaway code that prints out results to the console. Some good angel inspired me to do it this time by writing tests using NUnit, the .NET testing framework. The result, as you'll see, is some permanent information instead of just something that goes into my head, perhaps later to be forgotten. And something that others can learn from, making their job just a bit easier. Read on.

In this book, we're testing with NUnit, a .NET testing framework written by Jim Newkirk, Michael Two, Alexei Vorontsov, and Charlie Poole, based on the original NUnit by Philip Craig. You can find it at nunit.sourceforge.net. NUnit is much the same as all the Extreme Programming test frameworks (xUnits), with two important differences:

1. NUnit uses the "attribute" feature of .NET to identify tests. As you'll see, test fixtures are identified by the attribute [TestFixture] and individual tests by [Test]. This means that you can write tests that don't inherit from a fixed TestCase superclass, and NUnit can still find them. At first I thought this was just gratuitously different, but I'm finding that it works just fine and I rather like it.
2. NUnit allows you to write tests in any .NET language. So even though NUnit itself is written in C#, you can write your tests in Visual Basic or C++, or, I suppose, even in ML or Eiffel. Language interoperation is a key characteristic of .NET, and one that I consider to be quite important.

When you write tests using NUnit, you can run the NUnit application, and it will run and report on your tests. There are two versions of the app, one that runs at the command prompt, and a GUI version. I prefer the GUI version and will be using it throughout most of this book. When I say NUnit, I'm usually talking about that GUI, which looks like this:

TODO: Image here

When you press the Run button, NUnit looks at your program, finds all the tests, and runs them. Each test fixture in your program has its own setup and teardown methods, and each test case runs with a fresh setup. This means that your tests are all automatically independent, and that NUnit will run them all. If the test all run correctly, NUnit displays a gree bar, as show in the picture above. If any of the tests don't run, the bar is red.

Extreme Programming teams write lots of tests, and the rule is that we only release code back to the repository if all the tests run correctly. This may seem like a big burden, but it turns out not to be. And imagine how confident you can be in your code if you have lots of tests and every one of them is working correctly!

But enough warmup. Let's take a look at what it's all about.

To write tests using NUnit, your project needs to reference the nunit.framework.dll. In Visual Studio, you click on the project and Add Reference. Then navigate to the nunit.framework.dll, and select it. I keep nunit.framework.dll in the directory that holds all my .NET project directories, to make it easy to find.

The usual way to write tests is to set up a "test fixture", which is a class. The convention is to use a separate fixture for every test topic -- for example, we'll have a fixture for our Regex tests. Since each fixture has only one setup and one teardown, all the tests in the class want to be somewhat alike, and usually we wind up with one or a few fixtures for each important class in our system. The basic test fixture class looks like this:

				using System;
	using NUnit.Framework;
	using System.Text.RegularExpressions;

	namespace Notepad
	{
		[TestFixture] publicclass TestRegex : Assertion {
		}
	}

The [TestFixture] is one of those attributes I was telling you about. It tells NUnit that this class has tests in it.

NUnit tests allow us to write "assertions", and it checks those assertions to see if they are true. So if we wanted to test whether 2+2 is 4, we might write:

	[Test] publicvoid TwoPlusTwo() {
		AssertEquals(4, 2+2);
	}

NUnit has a number of different assertion methods for your convenience. We'll see more of them as we move on in the book. By far, the most commonly used one is AssertEquals().

We don't have to have the test fixture inherit from Assertion, as is shown above, but I prefer that because it means I can say AssertEquals rather than Assertion.AssertEquals. Note also in this example, that I'm already showing the reference to System.Text.RegularExpressions. I have to do that to write any references to the Regex class. So we're all set to go.

Our mission in the grand scheme of things is to find XML tags in text. So our first test will just look for a paragraph tag, <p>, in a string.

	[Test] publicvoid SimplePattern() {
		Regex r = new Regex("<p>");
		Match m = r.Match("contains <p> here");
		Assert(m.Success);
		m = r.Match("contains no para");
		Assert(!m.Success);
	}

So. There's the [Test] attribute, telling NUnit this is a test method. We create a Regex that searches for <p>, then test it against two strings. We expect the first two succeed, and the second to fail. (As far as I can tell, the Match class doesn't have a Failure property, only a Success property.

(If you're not familiar with C#, notice that the code says m.Success, not m.Success(). C# has methods, which take parenthesized lists of arguments (sometimes null), and "properties", which act like instance variables, with setters and getters. They can be implemented with member variables, or any other way you might like. They aren't member variables, they just act that way.

OK, we get a Green Bar, and we have a running Regex test now. They reality was about that easy, even though the only documentation I had was the C# Help, which is comprehensive but not easy to dig through.

Now, to some people, that test may seem too simple. You'll want to find your own balance, but I always make my first test really simple. Sometimes I even write a test that just says

	Assert(true);

to get things going. I do that for a few reasons. First of all, it takes a little discipline to do testing, even when I do them first. So I like to make it really easy to get started. Second, for me, the first tests usually include a lot of learning about something new, and rather than spend time debugging, I'm perfectly happy to start with something trivial. It only takes a moment to do, and it starts me off on the right foot. Third, sometimes event the simplest test I can think of doesn't work. The simpler it is, the sooner I'll figure out what it is I'm missing about whatever I'm testing. I'd suggest that you try tests ranging from very simple to more difficult, and see what works best for you. But please do try trivial tests -- they're surprisingly useful.

It occurs to me as I write this that in the course of this book, you will probably encounter examples of doing things in ways that seem almost obtusely simple. I assure you that I'm not doing it to make the book suitable for third-graders, nor because I myself am also simple. I work that way because in the half-dozen years I've been doing Extreme Programming, I've been working in simpler and simpler ways, and my work seems to be getting better and better. So, please, when you see something here that looks odd, give it a try. Often I think you'll find it interesting.

OK, enough advertising. Let's do another test.

In XML, tags must always be paired and properly nested, unlike HTML, which is more liberal. So in a legal paragaph in XML, we have a leading <p> and a trailing </p>. We'll write a test to see if we can recognize such a string.

	[Test] publicvoid PairedParagraphs() {
		Regex r = new Regex("<p>.*</p>");
		Match m = r.Match("<p>this is a para</p>");
		Assert(m.Success);
	}

So there we are. The Regex looks for a p tag, an un-p tag, as I call them, and anything in between. (In Regex, dot means "any character", and star means "any number of the preceding".) Again, we get the green bar. I'm having good luck so far, maybe because I've done some stuff with regular expressiosn in Ruby, and read most of the famous regular expressions books, Mastering Regular Expressions, by Jeffrey E. F. Friedl. The next problem, though, gets a bit trickier, and as we'll see, I have some problems with it.

OK, tags come in pairs, and there's stuff in between them. I want to see if I can recognize two tags at once, and pull out the stuff in the middle. Here's the test as it wound up: we'll talk about how it got that way.

	[Test] publicvoid PairedUnknown() {
		Regex r = new Regex("<(?<prefix>.*)>(?<body>.*)</(?<suffix>.*)>");
		Match m = r.Match("<p>this is a para</p>");
		Assert(m.Success);
		AssertEquals("p",m.Groups["prefix"].Value);
		AssertEquals("p",m.Groups["suffix"].Value);
		AssertEquals("this is a para",m.Groups["body"].Value);
		m = r.Match("<H2>this is a heading</H2>");
		Assert(m.Success);
		AssertEquals("H2",m.Groups["prefix"].Value);
		AssertEquals("H2",m.Groups["suffix"].Value);
		AssertEquals("this is a heading",m.Groups["body"].Value);
	}

Here I got into a little trouble. I knew that I wanted three "groups" in regular expressions, one for each section of the input, the leading tag, the content, the closing tag. In the kinds of expressios I'm used to, the expression would have been something like this:

				"(<.*>)(.*)(<.*>)"
		

That means threre are three groups, in the parents. The first is a less than, any sequence of characters, and a greater than. The second is any sequence of characters. The third is a less than, any sequence, a greater than. The second is any sequence of characters. The third is a less than, any sequence, a greater than. This isn't good enough to last for the ages. We'll probably want to have the sequences inside the tags not include a greater than, and so on. But it seemed enough to start with. So I tried that sequence exactly. I conldn't figure out how to make it work. I was trying things like

	AssertEquals("p", m.Groups(0).Value);

which turns out not to be right. I finally dug through the help and found an example. .NET Regex allows you to specify the groups to have names, as shown in the ?<prefix> and similar strings. It still didn't work. The only example I had was in Visual Basic. I guess that VB does subscripting with parentheses, not sequare brackets. When I finally figured out what the C# compiler was complaining about, and entered the syntax as shown in the example, the test worked.

So what we've got is that the Groups property is a collection of groups. It is indexable by name (and it turns out, by integer), and returns a Group when you index into it. That Group has a Value attribute, which is the string matched.

Here's a key thing that's happening in this test. We might have been doing this experimentation with print statements ... in fact I actually did some to find out what was coming out of these methods, when all else failed. But then I turned them back into tests. These tests are documenting for me (and for you) what I've learned so far about how to work Regex. It's not the whole story, but the story begins to take shape. I can look back to these tests and remind myself how things are supposed to work, in a way that actual code using the Regex might not. Here, I see what the exact inputs are and what the outputs are. Very clear, no need to induce what's up.

OK, now we have a regular expression that includes opening and closing tags, and can extract which tag it found and what was between the tags. Good stuff. Still not robust enough, but nearly enough for now. To make it more robust, what we might want is to ensure that the closing tag matches the opening one, H2 and /H2, and so on. I think there's a way to do that with. .NET Regex, but I've not gotten there yet. One last test for now. I still want to understand how those Groups work a bit better. I think that if I leave the names off, I should be able to index by integers. After a little fiddling around, and some more printing as I go, I got this test to work:

	[Test] publicvoid NumberedGroups() {
		Regex r = new Regex("<(.*)>(.*)</(.*)>");
		Match m = r.Match("<p>this is a para</p>");
		Assert(m.Success);
		// foreach(Group g in m.Groups){//    Console.WriteLine(g.Value);// }
		AssertEquals("<p>this is a para</p>",m.Groups[0].Value);
		AssertEquals("p",m.Groups[1].Value);
		AssertEquals("p",m.Groups[3].Value);
		AssertEquals("this is a para",m.Groups[2].Value);
	}

So you see what I learned. Group 0 is the whole matched string. I believe that if there was extra stuff outside the tags, it wouldn't included in this. Maybe I'll write another test to verify that, or modify this one. And the groups are then in order, 1, 2, 3. It all makes sense. I can't remember for sure, but I think my original problems, solved with the named groups in the preceding test, all came from not knowing the syntax for the Groups methods. So I got the more sophisticated way to work first, then backed into the simple solution I had been working toward to being with.

In the matching tag examples above, my input has matched p and H2 tags, and the Regex finds them just fine. However, there is nothing in the regular expression itself that rquires the opening and closing tags to match. I'm going to add a test that shows that unmatched tags will still pass this REgex, and then see if I can figure out how to require them to match: I seem to remember that there's a way to do what with the Regex class. Here's the new test:

	[Test] publicvoid InvalidXmlNotHandledYet() {
		Regex r = new Regex("<(?<prefix>.*)>(?<body>.*)</(?<suffix>.*)>");
		Match m = r.Match("<p>this is a para</H2>");
		Assert(m.Success);
		AssertEquals("p",m.Groups["prefix"].Value);
		AssertEquals("H2",m.Groups["suffix"].Value);
	}

Just as expected, the same Regex matches a p followed by an H2. Not what we really want, but we want to be sure we understand what our code does. This test now motivates the next extension, to a Regex that does forece the tags to match. I'm not sure we will need this -- We may already have gone beyond our current need for regular expressions, but my mission here is to learn as much as I can, in a reasonable time, about how Regex works. Now I'll have to search the Help a bit. Hold on...

The documentation seems to suggest that you can have named backreferences, using \k. I'll write a test. Hold on again... all right! Worked almost the first time: just a smple mistake away from prefect. Here's the new test:

	[Test] publicvoid Backreference() {
		Regex r = new Regex("<(?<prefix>.*)>(?<body>.*)</\\k<prefix>.*>");
		Match m = r.Match("<p>this is a para</p>");
		Assert(m.Success);
		m = r.Match("<p>this is a para</H2>");
		Assert(!m.Success);
	}

In this test, notice that we had to type \\k to get the \k into the expression. This is because C# strings, like most languages' strings, already use the backslash to prefix newlines and other special characters. So we have to type two of them to get one backslash into the string. The amazing thing is that I actually remembered to do that the first time! The mistake? I left the word "suffix" there instead of saying \k <prefix> as was my intent.

We're in pretty good shape for regular expressions now, though there are a number of things we might want to learn about. One complication is that you can apparently get all the matches that are possible when a pattern matches several times on one string. I have no immediate use for that, so probably won't go there. Another complication is that Regex has a Replace method that will produce a new string with a replacement done based on the pattern. We'll probably need that one, when we replace a p-tagged string with an H2-tagged string. I'm bored with this game right now, so will leave the Replace tests until I get that story.

In the example above, I have the \k, but it is followed by a .* that is left over from the original pattern. I bet that's redundant in this case. I'll change the test to remove that and see if it still runs. Sure enough, it still works. The backreference eats all the characters, given the patten we have. I bet there are some very tricky things you could do with that feature, embedding backreferences in other paterns... but I'm not going to mess with with that for now. For the record, here's the final version of the test. The only difference is the removal of the .* after the second occurence of "prefix".

	[Test] publicvoid Backreference() {
		Regex r = new Regex("<(?<prefix>.*)>(?<body>.*)</(\\k<prefix>)>");
		Match m = r.Match("<p>this is a para</p>");
		Assert(m.Success);
		m = r.Match("<p>this is a para</H2>");
		Assert(!m.Success);
	}

That's enough playing with Regex for now. What have we learned? What we have here is a small set of tests that were used to learn how to use Regex, and that now document htat learning so that I can refer to it later, and so that other programmers, like Chet when he sees this, can learn from it as well. It took a little more discipline to start testing Regex instead of just doing little experiments with printing, but to me the payoff is clear: this material can be used to pass on the learning to others, and to my future forgetful self. I think it's worth it, and I commend the idea to your attention.

My standard way of using NUnit is in its standalone GUI form. The NUnit gui runs as a separate program which is looking at your program. When you press the button, NUnit runs your tests and displays the results in its panes. My usual work cycle is to write or enhance a test, make it work, run the tests, and repeat. The cycle, for me, is quite short: usually less than ten minutes, often only two.

Working this way, I really don't get very many bugs, and I don't very often get confused. That's good, because I don't like hunting for bugs, and I don't like being confused. Both slow me down. Sometimes, though, it is hard to write a test for something. Often this happens when I'm using some .NET object that I'm not familiar with, and I don't even know what kind of result it is going to give me back.

My first reaction is often to do a Console.WriteLine to se what is going on. NUnit displays the console output right in one of its panes, and I build my software -- even forms software -- in console mode, so this works pretty well. But it has its limitations, as we'll see.

Now of couse Visual Studio has very strong debugging capabilities, and you can set breakpoints, run your program, stop at the breakpoints, and look around. You can inspect variables, step the program, and so on. It's all good. Sometimes in working with the XML Notepad, I've done that to figure out what was going on. But I always lean away from using the debugger. For me, it is a potential trap, a slippery slope. All too many times in the past, I have written vast amounts of code, and then spent hours in the debugger trying to make it work. The test-driven style I use now, which you're seeing in this book, allows me to work at a more constant rate of progress, without those long, frustrating, high-pressure debugging sessions. I'm afraid that if I get too good at using the debugger, I might fall back into bad habits. So I've held off on learning to use debuggers. I've been programming in Ruby for two years or more, and don't know how to use its debugger at all, and don't miss it a bit. Still, there are some times when Visual Studio's debugger can be helpful, and there are a few ways to use it in conjunction with NUnit. Let's look at a couple of them.

Visual Studio understands a number of configurations of your program. In particular, it understands a "production" configuration and a "debug" one. You can set up your debug configuration to start up a new copy of NUnit. So if you set a breakpoint, then start with debugging, NUnit will run. You can select the test of interest in the NUnit GUI, then run, and voila! the breakpoint you were interested in.

To set up your startup program, check the Help in Visual Studio for Modifying Project Properties and Configuration Settings. Don't expect it to be entirely accurate, however. It's not in my version, though it is close. Here's how to do it in my VS, "Microsoft Development Environment 2002, Version 7.0.9466.

Select the project in the Solution Explorer. Bring up the project Properties, using a right click on the project name, or from the Project menu. Click Configuration Properties, Debugging. You'll get a window like this:

TODO: Image here

Set the window as shown: Under Debug Mode, select "Program". Then, under Start Application, use the little"..." button to browse to the NUnit-GUI.exe file and select it. Apply or OK the window. From now on, when you do Debug/Start, or press F5, NUnit will start up. Works fine.

Visual Studio can attach to a running process and debug it. At this writing I have no idea how this works, but I can tell you how to make it do something you might want. Start the NUnit GUI, if it isn't already running. Select the menu item Debug/Processes. This will bring up a Processes window:

TODO: Image here

Scroll to find nunit-gui.exe, click on it, press Attach. This will bring up another window:

TODO: Image here

Select Common Language Runtime (at least), press OK, close the Debug Processes Window. Now, you can set a breakpoint in Visual Studio, go to the NUnit GUI, run a test, and badda-bing badda-boom, you're at your breakpoint.

Using this technique, if you rebuild your app, the connectio to the NUnit process is broken, so you'll have to reset it.

The Debug Startup technique takes more clicks to set up, but once it is done, you can always set a breakpoint and start a new NUnit using Debug/Start. Attaching to the process is a bit easier, but it only works until the next time you build the project. I prefere the first way, but often work for days without using either one. Try them both, choose your favorite.

Almost all the code in this book will be developed with associated tests in NUnit. I expect that you'll see that working this way serves well to keep the problems small, keep the code working, and easy to improve. Once in a while, we'll be working without tests, either because we can't figure out how to write a test, or because we get lazy. If that happens, I bet there will be bugs in the code.

Because this is a book about learning, it there are bugs we're going to let you see them. Assess whether our bugs would have been reduced by writing more, or better tests. My bet right now is that you'll come to see how effective the testing approach is. That's not all we're here to learn, but it's one thing that will come out.

Installing NUnit into your .NET development process is quite easy. It should only take you an hour or so to write your first test. And I hope you'll try it, and never stop.

posted on 2006-03-22 13:18 毒菇求Buy 阅读(384) 评论(1)  编辑 收藏 引用 所属分类: C#