In the latest Hanselminutes podcast, Roy Osherove describes some of his best practices in unit testing techniques. In the discussion, Roy describes one of the key tenets from his book which says that a unit test must be “trustworthy”.
On the surface, this tenet seems fairly obvious. Of course, we need to trust our tests! However, as he describes some situations where this rule is often violated, I realize I would be guilty on many occasions.
For example, I always struggle with the idea to include data access layer tests in my unit tests. Yes, I know that testing the database is really an integration test and “purists” would argue these should not be included. However, I feel the only way to test the data access code in a meaningful way is to hit the database. In other words, the code in the data access layer and the related code in the database are a single logical unit.
In this podcast, Roy helps me to understand the difference, which I have summarized here:
Clarification # 1 – Data Access Layer Unit Tests are not trustworthy
With a trustworthy test, you must be able trust the results 100% of the time. If the test fails, I know the code is broken and must be fixed. This means I shouldn’t have to ask things like “Was the database down?”, “Was the connection string OK?”, “Was the stored procedure modified?”. By asking these questions, it shows that I don’t trust the results and I have a poorly designed “unit test”.
Clarification # 2 – Data Access Layer tests must be integration tests
In the podcast, Roy also states that the data access layer is generally a very thin layer on top of the database, invoking some execution logic within the database itself. For example, the data access layer will issue a command/query such as a SELECT, INSERT, UPDATE, DELETE or similar set of statements using a stored procedure. There generally is not much else that sits inside the data access code. So, Roy also believes that this type of code must be tested as a single unit.
However, Roy also points out that these must be also be integration tests. I had made the assumption that since they must be tested as a unit, then they must be unit tests! However, by doing so, I have created untrustworthy tests. Now, it makes sense to me that the data access layer will likely not include any unit tests at all! They will only be created as integration tests.
Clarification # 3 – Separation of Unit Tests from Integration Tests
It probably seems like an obvious observation to make, but I feel its important to distinguish: Unit Tests and Integration Tests are often both developed using a “Unit Test Framework”. We’re using the same tools to create both types of tests.
However, the problem occurs when we blend the two types of tests together. Instead, we should create different projects/assemblies, so they can be run easily and independently from one other. It should be obvious and easy for someone to get all the code out of source control, build the solution, and know which tests should pass from the beginning. If you mix integration tests into the same project, the other developer will have to determine which tests “should” pass and which “should” fail. This makes things more difficult than necessary for other developers and may likely mean that your tests will ignored and no longer maintained.