Writing Unit Tests Makes Me Feel Stupid

Writing unit tests makes me feel stupid.

Allow me to explain.

I'm not talking about full-blown TDD, but simply about a beginner trying to write some tests in order to make sure code is working as expected in as many cases as possible. What I almost invariably run up against is my own questionable design choices that usually boil down to three issues: lots of dependencies, needing non-trivial data for a meaningful test, or needing a full ASP.NET web request for a meaningful test.

Sure, some classes are small and simple and will have clear tests supporting them. A few more classes will interact with those first ones clearly and cleanly, too. That'll be about ½ of 1% of my total system. While trying to write tests for the rest of the code I'll be mumbling

"How the heck am I going to test that?"

"Ugh. I need to inject that into that into that just to test one method."

"Why would I write it like this? Was I in that much of a rush?"

The more I think about testing my own code on current projects, the more I understand the usefulness of TDD and that the goal is not simply tests but an easily consumable, testable design. I've read Pragmatic Unit Testing for a baseline. Hopefully The Art of Unit Testing and Clean Code will help me avoid painting myself into these corners in the future. In the meantime, I've been watching James Shore's Let's Play TDD series, which is really cool because you're essentially looking over his shoulder as he works on a small app.

I've started to wonder if other people feel the same way and if they avoid testing because of it or if they realize that it's a learning opportunity.

Regex Matches in C#

static void Main(string[] args)
{
  string sample = "Swedish field marshal ... fact that Gustav II father the Thirty Years' War";

  Regex r = new Regex(@"JS_DisplayEntry\((\d*?)\)", RegexOptions.IgnoreCase);

  MatchCollection collection = r.Matches(sample);

  foreach (Match m in collection)
  {
    Console.WriteLine(m.Groups[1].Value);
  }

  Console.ReadLine();
}

XSL Embedded Resources, xsl:include, Saxon HE, and You

I wanted to:

1. Set my XSL files in my .NET console application to be embedded resources so they wouldn't be so easy to tinker with
2. Use xsl:include to modularize some XSL where appropriate
3. Pass a parameter into one of the XSL files
4. Use Saxon for its XSL 2.0 support

At first, before I needed xsl:include, the code worked absolutely fine. The problem I ran into is apparently related to where the .NET framework was looking for the included XSL file when both were embedded resources. Step 1 to fixing the problem is implementing your own XmlUrlResolver. Step 2 is figuring out how to then reference your XSL from the calling code AND in the xsl:include href attribute.

I won't pretend to understand the details of what's going on here. My understanding is general, but this does work.

Here is where I ended up with my implementation of an XmlUrlResolver, including the sources I used.

public class EmbeddedXslResolver : XmlUrlResolver
{
  private readonly Assembly _assembly;

  public EmbeddedXslResolver()
  {
    _assembly = Assembly.GetExecutingAssembly();
  }

  public override object GetEntity(Uri absoluteUri, string role, Type ofObjectToReturn)
  {
    // Based on 
    //http://www.tkachenko.com/blog/archives/000653.html
    //http://www.brandonmartinez.com/2009/07/06/xmlurlresolver-using-embedded-xslt-resources-in-c/

    // Seems to assume content, not embedded resource
    //Stream s = _assembly.GetManifestResourceStream(this.GetType(), Path.GetFileName(absoluteUri.AbsolutePath));

    // Assumes input that would be only the file name like PrepGpgDtd.xsl and hard-codes the assembly details
    //Stream s = _assembly.GetManifestResourceStream("Infrastructure.Xsl." + absoluteUri.Segments[absoluteUri.Segments.Length - 1]);

    // Assumes input that would be fully qualified resource name like Infrastructure.Xsl.PrepGpgDtd.xsl
    //Stream s = _assembly.GetManifestResourceStream(absoluteUri.Segments[absoluteUri.Segments.Length - 1]);

    return _assembly.GetManifestResourceStream(absoluteUri.Segments[absoluteUri.Segments.Length - 1]);
    }
  }

Here's how I used it in my calling code. Note the string for my XSL file, "Infrastructure.Xsl.SplitDtd.xsl". My XSL was in an assembly (a C# class library project) named Infrastructure, in the Xsl folder, and the file was named SplitDtd.xsl. Under its properties, that file was set to "Embedded Resource" and "Do Not Copy".

...
EmbeddedXslResolver resolver = new EmbeddedXslResolver();
XmlReaderSettings settings = new XmlReaderSettings();
settings.XmlResolver = resolver;

using (XmlReader reader = XmlReader.Create("Infrastructure.Xsl.SplitGpgDtd.xsl", settings))
  {
    // Here are the Saxon details.
    // Create a Processor instance.
    Processor p = new Processor();
                    
    // Load the source document.
    XdmNode node = p.NewDocumentBuilder().Build(new Uri(preparedXmlFile.FullName));

    // Create a transformer for the stylesheet. Saxon needs the resolver, too.
    XsltCompiler compiler = p.NewXsltCompiler();
    compiler.XmlResolver = resolver;
    XsltTransformer transformer = compiler.Compile(reader).Load();
                    
    // Set the root node of the source document to be the initial context node.
    transformer.InitialContextNode = node;
                    
    // BaseOutputUri is only necessary for xsl:result-document, which I'm using.
    transformer.BaseOutputUri = new Uri(destination.FullName);
                    
    transformer.SetParameter(new QName("", "", "a-head"), new XdmAtomicValue(splitOnAHead.ToString().ToLower()));
                    
    // Create a serializer.
    Serializer serializer = null;
    try
    {
      serializer = new Serializer();
                       
      // Transform the source XML to System.out.
      transformer.Run(serializer);
    }
    finally
    {
      if(serializer != null) serializer.Close();
    }
  }

Then in SplitDtd.xsl, the xsl:include file was set to the following. Note how it's path looks incorrect in terms of a physical URI

If you're seeing an odd closing xsl:include tag, that seems to be a bug in the syntax highlighting.

With this configuration, .NET can find the primary XSL file and the include correctly.

One Saxon-specific note: I originally tried to use xsl:variable to catch the incoming parameter, but that doesn't work. You'll need to use xsl:param.

The information here is scattered around the Internet. I'm presenting nothing new. But, I didn't find this all in one place and I had a difficult time putting all the pieces together. Hopefully this post can prevent that in the future.

Meatballs

Preheat the oven to 350.

• ~1 lb of mixed ground beef, sausage, and veal OR 1/3 lb of each.
• 1 egg
• 1 clove of garlic
• 1 cup of dried breadcrumbs
• 1/4 cup of olive oil
• 1/4 cup of water
  
• Salt and pepper to your taste
• Parsley or oregano to your taste

Mix it all together. Make balls about the size of a baseball. Place them on a baking sheet.

Bake for about 40-45 minutes.

Obstacles & Bootstrapping

Obstacles

The biggest obstacles in my way on this road are time and distractions. They're similar, but definitely not one in the same, and it's easy to let them interfere with my goals.

Time

This one's pretty simple. There's just not enough time in the day to learn everything I want to know about. More to the point, there's just enough time in the day to learn just enough to do my job, and I think that "just enough" is dangerous. I haven't been 100% coding for all of my working years, but that's no excuse for placing only a thin veneer over ignorance.

I need to make time to go deeper. I need to make time to explore and understand more thoroughly. For example, I don't want to be just a consumer of the .NET framework. I want to understand how the framework actually works, why it makes certain choices, and how in some cases (like the MVC layer) I can change it. I want to learn Ruby (or another dynamic language) because I've heard that it's very different from C# and I think juxtaposing them would be interesting. And on, and on. This is time well spent, but it must be fought for.

Distractions

As I work to make time to go deeper, the other significant obstacle I face is an unending stream of distractions. I don't mean the normal distractions of life, but technical distractions born out of the type of curiosity that got me into this line of work in the first place.

"I need to move and compress these log files. I wonder if I can do that in PowerShell?"

"Uncle Bob is talking about Monads. Should I know about Monads?"

"I bet I could do this easier in XSL 2.0 instead of 1.0. And there are those new features...."

All valid topics to consider. The problem is that they can become death by a thousand cuts. Since time is so very scarce, every distraction that leads me outside of my current scope can become nothing more than an interruption. Yes, I'll certainly learn something from it. Wonderful. But I've lost focus on the thing I was really trying to learn...and now I'm out of time today...again.

Bootstrapping

Here's how I'm getting started despite the obstacles.

First, probably about a year and a half ago, I started thinking about where my knowledge gaps are, came up with a book list, and began working through it. This was fine for basic material but became difficult the more advanced it became primarily because there's no one to ask for feedback or help. Every once in awhile you find a blog post that's helpful, which is always like finding a needle in a haystack. Sites like Stack Overflow are great for specific questions, but asking more general ones smacks of "homework help" and tends to draw trollish responses. The book list and reading solo was a necessary first step, but a limited one.

Next, I started attending every community developer event that I could manage the time for, which was and continues to be Microsoft-sponsored "firestarter" events, code camps, and local user group meetings. This is where concepts started to mesh a bit more for me. The 2009 ASP.NET MVC firestarter in particular brought together 3 foundational concepts that I had read about individually but hadn't applied: separation of concerns, unit testing, and, not necessarily domain-driven design, but the general idea of thinking more carefully about design. I'm only about a year separated from that event and it's almost amusing (definitely sad) how I wasn't thinking about those issues beforehand. Yet, there is still so much to learn.

Then, while at one of these community events I had the opportunity to meet via Twitter a highly-regarded developer who was closely involved with the community. I contacted him to ask if he knew of anyone in the area who would be willing to work with me as a mentor and he graciously agreed to that himself. To be completely honest, I felt like I had won the lottery. It is still early in that process, but we've setup a basic framework that is working well.

1. We set up a structured reading list that is broken down by general areas and progresses from easy to difficult.
2. We selected a project for us both to work on that's complex enough to stretch me and could actually see the light of day as something for the community.
   
3. We have a 1 hour call every 3-4 weeks where we talk about questions, trouble spots for me, and the project we selected.
   

Finally, I read O'Reilly's Apprenticeship Patterns. I didn't just read it. I plowed through. It was not only intensely comforting to realize that others are going through the same process as I am, but it was extremely helpful to read about issues I hadn't thought of on my own yet. Rather than list those, I would simply say if you're reading this because you feel like you're in the same position as I am, find it, read it, and use it to check on your progress.

I feel like I have a reasonable handle on the obstacles in front of me and that there is a clear bootstrapping process in place.