Writing Unit Tests Makes Me Feel Stupid

Writing unit tests makes me feel stupid.

Allow me to explain.

I'm not talking about full-blown TDD, but simply about a beginner trying to write some tests in order to make sure code is working as expected in as many cases as possible. What I almost invariably run up against is my own questionable design choices that usually boil down to three issues: lots of dependencies, needing non-trivial data for a meaningful test, or needing a full ASP.NET web request for a meaningful test.

Sure, some classes are small and simple and will have clear tests supporting them. A few more classes will interact with those first ones clearly and cleanly, too. That'll be about ½ of 1% of my total system. While trying to write tests for the rest of the code I'll be mumbling

"How the heck am I going to test that?"

"Ugh. I need to inject that into that into that just to test one method."

"Why would I write it like this? Was I in that much of a rush?"

The more I think about testing my own code on current projects, the more I understand the usefulness of TDD and that the goal is not simply tests but an easily consumable, testable design. I've read Pragmatic Unit Testing for a baseline. Hopefully The Art of Unit Testing and Clean Code will help me avoid painting myself into these corners in the future. In the meantime, I've been watching James Shore's Let's Play TDD series, which is really cool because you're essentially looking over his shoulder as he works on a small app.

I've started to wonder if other people feel the same way and if they avoid testing because of it or if they realize that it's a learning opportunity.

Regex Matches in C#

static void Main(string[] args)
{
  string sample = "Swedish field marshal ... fact that Gustav II father the Thirty Years' War";

  Regex r = new Regex(@"JS_DisplayEntry\((\d*?)\)", RegexOptions.IgnoreCase);

  MatchCollection collection = r.Matches(sample);

  foreach (Match m in collection)
  {
    Console.WriteLine(m.Groups[1].Value);
  }

  Console.ReadLine();
}

XSL Embedded Resources, xsl:include, Saxon HE, and You

I wanted to:

1. Set my XSL files in my .NET console application to be embedded resources so they wouldn't be so easy to tinker with
2. Use xsl:include to modularize some XSL where appropriate
3. Pass a parameter into one of the XSL files
4. Use Saxon for its XSL 2.0 support

At first, before I needed xsl:include, the code worked absolutely fine. The problem I ran into is apparently related to where the .NET framework was looking for the included XSL file when both were embedded resources. Step 1 to fixing the problem is implementing your own XmlUrlResolver. Step 2 is figuring out how to then reference your XSL from the calling code AND in the xsl:include href attribute.

I won't pretend to understand the details of what's going on here. My understanding is general, but this does work.

Here is where I ended up with my implementation of an XmlUrlResolver, including the sources I used.

public class EmbeddedXslResolver : XmlUrlResolver
{
  private readonly Assembly _assembly;

  public EmbeddedXslResolver()
  {
    _assembly = Assembly.GetExecutingAssembly();
  }

  public override object GetEntity(Uri absoluteUri, string role, Type ofObjectToReturn)
  {
    // Based on 
    //http://www.tkachenko.com/blog/archives/000653.html
    //http://www.brandonmartinez.com/2009/07/06/xmlurlresolver-using-embedded-xslt-resources-in-c/

    // Seems to assume content, not embedded resource
    //Stream s = _assembly.GetManifestResourceStream(this.GetType(), Path.GetFileName(absoluteUri.AbsolutePath));

    // Assumes input that would be only the file name like PrepGpgDtd.xsl and hard-codes the assembly details
    //Stream s = _assembly.GetManifestResourceStream("Infrastructure.Xsl." + absoluteUri.Segments[absoluteUri.Segments.Length - 1]);

    // Assumes input that would be fully qualified resource name like Infrastructure.Xsl.PrepGpgDtd.xsl
    //Stream s = _assembly.GetManifestResourceStream(absoluteUri.Segments[absoluteUri.Segments.Length - 1]);

    return _assembly.GetManifestResourceStream(absoluteUri.Segments[absoluteUri.Segments.Length - 1]);
    }
  }

Here's how I used it in my calling code. Note the string for my XSL file, "Infrastructure.Xsl.SplitDtd.xsl". My XSL was in an assembly (a C# class library project) named Infrastructure, in the Xsl folder, and the file was named SplitDtd.xsl. Under its properties, that file was set to "Embedded Resource" and "Do Not Copy".

...
EmbeddedXslResolver resolver = new EmbeddedXslResolver();
XmlReaderSettings settings = new XmlReaderSettings();
settings.XmlResolver = resolver;

using (XmlReader reader = XmlReader.Create("Infrastructure.Xsl.SplitGpgDtd.xsl", settings))
  {
    // Here are the Saxon details.
    // Create a Processor instance.
    Processor p = new Processor();
                    
    // Load the source document.
    XdmNode node = p.NewDocumentBuilder().Build(new Uri(preparedXmlFile.FullName));

    // Create a transformer for the stylesheet. Saxon needs the resolver, too.
    XsltCompiler compiler = p.NewXsltCompiler();
    compiler.XmlResolver = resolver;
    XsltTransformer transformer = compiler.Compile(reader).Load();
                    
    // Set the root node of the source document to be the initial context node.
    transformer.InitialContextNode = node;
                    
    // BaseOutputUri is only necessary for xsl:result-document, which I'm using.
    transformer.BaseOutputUri = new Uri(destination.FullName);
                    
    transformer.SetParameter(new QName("", "", "a-head"), new XdmAtomicValue(splitOnAHead.ToString().ToLower()));
                    
    // Create a serializer.
    Serializer serializer = null;
    try
    {
      serializer = new Serializer();
                       
      // Transform the source XML to System.out.
      transformer.Run(serializer);
    }
    finally
    {
      if(serializer != null) serializer.Close();
    }
  }

Then in SplitDtd.xsl, the xsl:include file was set to the following. Note how it's path looks incorrect in terms of a physical URI

If you're seeing an odd closing xsl:include tag, that seems to be a bug in the syntax highlighting.

With this configuration, .NET can find the primary XSL file and the include correctly.

One Saxon-specific note: I originally tried to use xsl:variable to catch the incoming parameter, but that doesn't work. You'll need to use xsl:param.

The information here is scattered around the Internet. I'm presenting nothing new. But, I didn't find this all in one place and I had a difficult time putting all the pieces together. Hopefully this post can prevent that in the future.

Meatballs

Preheat the oven to 350.

• ~1 lb of mixed ground beef, sausage, and veal OR 1/3 lb of each.
• 1 egg
• 1 clove of garlic
• 1 cup of dried breadcrumbs
• 1/4 cup of olive oil
• 1/4 cup of water
  
• Salt and pepper to your taste
• Parsley or oregano to your taste

Mix it all together. Make balls about the size of a baseball. Place them on a baking sheet.

Bake for about 40-45 minutes.

Obstacles & Bootstrapping

Obstacles

The biggest obstacles in my way on this road are time and distractions. They're similar, but definitely not one in the same, and it's easy to let them interfere with my goals.

Time

This one's pretty simple. There's just not enough time in the day to learn everything I want to know about. More to the point, there's just enough time in the day to learn just enough to do my job, and I think that "just enough" is dangerous. I haven't been 100% coding for all of my working years, but that's no excuse for placing only a thin veneer over ignorance.

I need to make time to go deeper. I need to make time to explore and understand more thoroughly. For example, I don't want to be just a consumer of the .NET framework. I want to understand how the framework actually works, why it makes certain choices, and how in some cases (like the MVC layer) I can change it. I want to learn Ruby (or another dynamic language) because I've heard that it's very different from C# and I think juxtaposing them would be interesting. And on, and on. This is time well spent, but it must be fought for.

Distractions

As I work to make time to go deeper, the other significant obstacle I face is an unending stream of distractions. I don't mean the normal distractions of life, but technical distractions born out of the type of curiosity that got me into this line of work in the first place.

"I need to move and compress these log files. I wonder if I can do that in PowerShell?"

"Uncle Bob is talking about Monads. Should I know about Monads?"

"I bet I could do this easier in XSL 2.0 instead of 1.0. And there are those new features...."

All valid topics to consider. The problem is that they can become death by a thousand cuts. Since time is so very scarce, every distraction that leads me outside of my current scope can become nothing more than an interruption. Yes, I'll certainly learn something from it. Wonderful. But I've lost focus on the thing I was really trying to learn...and now I'm out of time today...again.

Bootstrapping

Here's how I'm getting started despite the obstacles.

First, probably about a year and a half ago, I started thinking about where my knowledge gaps are, came up with a book list, and began working through it. This was fine for basic material but became difficult the more advanced it became primarily because there's no one to ask for feedback or help. Every once in awhile you find a blog post that's helpful, which is always like finding a needle in a haystack. Sites like Stack Overflow are great for specific questions, but asking more general ones smacks of "homework help" and tends to draw trollish responses. The book list and reading solo was a necessary first step, but a limited one.

Next, I started attending every community developer event that I could manage the time for, which was and continues to be Microsoft-sponsored "firestarter" events, code camps, and local user group meetings. This is where concepts started to mesh a bit more for me. The 2009 ASP.NET MVC firestarter in particular brought together 3 foundational concepts that I had read about individually but hadn't applied: separation of concerns, unit testing, and, not necessarily domain-driven design, but the general idea of thinking more carefully about design. I'm only about a year separated from that event and it's almost amusing (definitely sad) how I wasn't thinking about those issues beforehand. Yet, there is still so much to learn.

Then, while at one of these community events I had the opportunity to meet via Twitter a highly-regarded developer who was closely involved with the community. I contacted him to ask if he knew of anyone in the area who would be willing to work with me as a mentor and he graciously agreed to that himself. To be completely honest, I felt like I had won the lottery. It is still early in that process, but we've setup a basic framework that is working well.

1. We set up a structured reading list that is broken down by general areas and progresses from easy to difficult.
2. We selected a project for us both to work on that's complex enough to stretch me and could actually see the light of day as something for the community.
   
3. We have a 1 hour call every 3-4 weeks where we talk about questions, trouble spots for me, and the project we selected.
   

Finally, I read O'Reilly's Apprenticeship Patterns. I didn't just read it. I plowed through. It was not only intensely comforting to realize that others are going through the same process as I am, but it was extremely helpful to read about issues I hadn't thought of on my own yet. Rather than list those, I would simply say if you're reading this because you feel like you're in the same position as I am, find it, read it, and use it to check on your progress.

I feel like I have a reasonable handle on the obstacles in front of me and that there is a clear bootstrapping process in place.

Stop All Visual Studio Web Applications From Starting When Debugging

If you have a Visual Studio solution with more than one web application project, when you start the debugger you'll likely find that all the web applications will start their own ASP.NET Development Server instance.

I wanted to stop that because I didn't need them all to start and I wanted to keep the system resources. Select the web application project. Go to the properties tab. Find the "Always Start When Debugging" option and set it to False.

Storing a Directory Path with a Space and Using it in PowerShell

Step 1: Store the path in a variable like so
$7Zip = "C:\Program Files (x86)\7za.exe"

Step 2: Wrap the reference in the double quotes and precede with the call operator like so
ForEach-Object { &"$7Zip" a ($_.fullname -replace "txt", "zip") $_.fullname }

Step 3: Complain silently in your head about the time you wasted trying to figure that out.

Find a Column in Your Database by Name

SELECT t.name AS table_name,
SCHEMA_NAME(schema_id) AS schema_name,
c.name AS column_name
FROM sys.tables AS t
INNER JOIN sys.columns c ON t.OBJECT_ID = c.OBJECT_ID
WHERE c.name LIKE '%isTextbook%'
ORDER BY schema_name, table_name;

Props to SQL Authority.

Convert HTML Stored in SQL Server as varbinary to Text

I have a wonderfully exciting legacy database I'm handling and it contains a varbinary(MAX) field with HTML. For debugging purposes, every once in awhile I need to dig in and review what's stored there. Here's how I do that:

SELECT CONVERT(VARCHAR(MAX), mainText) AS htmlString FROM entry WHERE entryid = 1380915

A Little URL Rewriting with Managed Fusion Rewriter

I'm stuck on IIS 6 for now and I needed to do a bunch of URL rewriting. No one wants to spend money these days, so we didn't spring for the Helicon ISAPI_Rewrite product. I have a lot of legacy URLs to handle, but we won't keep them in place for more than a year and by that time we'll likely be on IIS 7 with its own rewriting tools.

I went with the Managed Fusion Rewriter tool both for its promised flexibility and the fact that it used the Apache mod_rewrite syntax.

I like it, but I ran into a few quirks I thought I'd pass along.

First, if you're using RewriteCond and trying to grab query string parameter values, %1 will hold the name/value pair. %2 will hold the value you're probably looking for. If you have multiple values you're catching you would just continue to increment -- %3, %4, etc. Here's an example that works as I expected it:

RewriteCond %{QUERY_STRING} ^lessonid=([0-9]+)$
RewriteRule ^/Explore/KeyQuestions.aspx$ /Analyze/Display/%2 [R=301,NC,L]

Second, in mod_rewrite you can apparently stack multiple RewriteCond one on top of another. I was not able to get it to work with this tool.

New Phrase: Smells Like Mountain Oysters To Me

I suggest the casual usage of a new phrase: smells like mountain oysters to me.

What does it mean? Roughly, it means someone's trying to make you look like an fool, possibly in front of others, but definitely for their own gain. It could also simply mean someone's trying to feed you a line of bullshit.

Would you eat a bull's testicles? Neither would I, but that's what mountain oysters are. The thought of that makes me want to vomit. No one should be eating anyone else's balls. Ever. Calling them "mountain oysters" is so ridiculously misleading to me that it makes me want to punch your mom.

By the way, if you're reading this and you would eat them, stop reading my fucking blog and never ever come back.

So....

The next time you suspect someone's trying to get you to do something and you think they're withholding key information, smells like mountain oysters to me.

The next time someone's asking you questions and you think they're going to use the information against you in some way, smells like mountain oysters to me.

The next time you think someone's intentionally trying to lead you into making a bad decision to advance their own goals or just to entertain himself, smells like mountain oysters to me.

Fiddler, Localhost, and Visual Studio

When you're running a site through Visual Studio's internal web server, Fiddler doesn't pick up the traffic. The help documentation gives a few tips for working around this problem, but the one that worked for me I found on Loren Halvorson's blog: add a dot [.] after localhost and before the port colon.

http://localhost.:2951/

Props to Loren.

XQuery Link-Love

Hi there.

I recently received some link-love from Pete Aven (Twitter, Blog) for some XQuery and MarkLogic Server posts I have here. The posts are kind of out-dated at this point, but I leave them up in case they help someone out. When I was just starting out, this kind of information was invaluable to me so I'm just trying to pay it forward.

I'm currently on a big SQL Server based project and my last MarkLogic project had some hellish deadlines that didn't leave much time for posting newer bits. With a little luck the next project I have lined up will bring me back into the XQuery/MarkLogic fold!

In the meantime, read Pete's blog and the other bloggers he has listed here.

TinyGet and URL Parameters

TinyGet is still a tool I use for some quick and dirty load testing, but I never needed to pass it URL parameters before. Simply adding an amerpsand was failing. The trick is escape it with the ^ character.

tinyget -srv:mydomain.com -uri:/Search/Results?q=food^&pIndex=5 -loop:10

Thanks to the folks on the IIS forums for their help!

Quick Console App to Ping a Server

This is based mostly off of something in the Microsoft documentation. It's just a quick hack to let me know when a remote server comes back up.

static void Main(string[] args)
{
  Ping pingSender = new Ping();
  PingOptions options = new PingOptions();
  // Create a buffer of 32 bytes of data to be transmitted.
  string data = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
  byte[] buffer = Encoding.ASCII.GetBytes(data);
  int timeout = 120;
  //PingReply reply = pingSender.Send(args[0], timeout, buffer, options);
  PingReply reply = pingSender.Send("server.domain.com", timeout, buffer, options);
  for (; ; )
  {
   if (reply.Status == IPStatus.Success)
   {
    Console.WriteLine("Address: {0}", reply.Address.ToString());
    for (int i = 0; i < 5; i++) Console.WriteLine("\a");
    // Or System.Media.SystemSounds.Beep.Play();
    // Or Console.Beep();
    break;                    
   }
   else
   {
    Console.WriteLine("Failed " + System.DateTime.Now);
    Thread.Sleep(30000);
   }
  }
 Console.ReadKey();
}

Center the Effing Div

.image {
  margin-left: auto;
  margin-right: auto;
  width: 415px; /* fix the width while we're at it */
  text-align: center; /* and we'll center the caption as well */
}