September 3, 2008

Carbon Five Community Has Moved

The Carbon Five Community Blog has moved to http://blog.carbonfive.com, please update your bookmarks and feed aggregator subscriptions.

The RSS feed is located at: http://blog.carbonfive.com/feed

There are new posts on the new blog... and you're missing em!

Posted by Christian Nelson at 10:22 AM | Comments (0)

August 28, 2008

Carbon Five Community Has Moved

The Carbon Five Community Blog has moved to http://blog.carbonfive.com, please update your bookmarks and feed aggregator subscriptions.

The RSS feed is located at: http://blog.carbonfive.com/feed

No new posts will show up at this location.

Posted by Christian Nelson at 12:15 PM | Comments (0)

July 31, 2008

Database Testing with Spring 2.5 and DBUnit

We've been using DB Unit on our Java projects for years and the mechanics of how it's used has evolved over time. I've recently spent some time making it work a little nicer for how we typically write database tests. What I've created makes using DBUnit on a project that is already using Spring and the testing support added in Spring 2.5 just a little easier through the application of convention and annotations.

In general, we've adopted the convention of loading data off the classpath from a flat dataset file named after the test located next to the test on the classpath. For example (in the maven standard directory structure):

  • src/test/java/com/acme/TripRepositoryTest.java - Java Test Code
  • src/test/resources/com/acme/TripRepositoryTest.xml - DB Unit Data Set for TripRepositoryTest

For most tests, the data set is loaded inside the test's transaction and rolled back when the test completes so that nothing needs to be cleaned up (see Spring's reference). For other tests -- service or integration tests -- the data is loaded outside of a transaction and must be cleared out manually. Most projects have a mix of both strategies and both should be easily supported.

When Spring 2.5 came out with its new testing framework, I threw together a custom TestExecutionListener that looks for test methods that are annotated with @DataSet, and when found, loads the data using DB Unit. Here's a transaction-per-test example:

@ContextConfiguration(locations = {"classpath:applicationContext.xml"})
public class TripRepositoryImplTest extends AbstractTransactionalDataSetTestCase {
 
     @Autowired TripRepository repository;
 
     @Test
     @DataSet
     public void forIdShouldFindTrip() throws Exception {
          Trip trip = repository.forId(2);
          assertThat(trip, not(nullValue()));
      }
}

The high-level execution path for this example looks like:

  1. Inject dependencies (DependencyInjectionTestExecutionListener)
  2. Start transaction (TransactionalTestExecutionListener)
  3. Load dbunit data set from TripRepositoryImplTest.xml (DataSetTestExecutionListener) using the setup operation (default is CLEAN_INSERT)
  4. Execute test
  5. Optionally cleanup dbunit data using the tear down operation (default is NONE)
  6. Rollback transaction (TransactionalTestExecutionListener)

Here's the trimmed down log output for this test: 

INFO: Began transaction (1): transaction manager; rollback [true] (TransactionalTestExecutionListener.java:259)
INFO: Loading dataset from location 'classpath:/eg/domain/TripRepositoryImplTest.xml' using operation 'CLEAN_INSERT'. (DataSetTestExecutionListener.java:152)
INFO: Tearing down dataset using operation 'NONE', leaving database connection open. (DataSetTestExecutionListener.java:67)
INFO: Rolled back transaction after test execution for test context (TransactionalTestExecutionListener.java:279)

For this to work in its current incarnation, a single datasource must be available for lookup in the application context. One of the interesting details is what to do with the connection used to load the data. The framework assumes that if it's a transactional connection it should be left open because whatever started the transaction should do the closing. When it's non-transactional it's closed after the dataset is loaded. This convention works well for how I typically write my database tests.

In addition to the @DataSet annotation, we must add the DataSetTestExecutionListener to the set of listeners that are applied to the test class. As in the above example, you can extend AbstractTransactionalDataSetTestCase which does this for you or you can specify the listener using the class-level annotation @TestExecutionListeners (see example). It's important that the listener is triggered after the TransactionalTestExecutionListener.

If all test methods use the dataset, then the test class (or super class) can be annotated and every test will load the dataset. Also, if a different dataset should be loaded, the name of the resource can be specified in the annotation (e.g. @DataSet("TripRepositoryImplTest-foo.xml") or @DataSet("classpath:/db/trips.xml")). Lastly, the setup and teardown database operations can be overriden (e.g. @DataSet(setupOperation = "INSERT", teardownOperation="DELETE")).

This functionality is part of the C5 Test Support package and is available in our maven repository. To use it, first add the C5 Public Maven repository to your pom.xml: 

<repository>
    <id>c5-public-repository</id>
    <url>http://mvn.carbonfive.com/public</url>
</repository>

Then add the necessary dependencies: 

<dependency>
    <groupId>org.dbunit</groupId>
    <artifactId>dbunit</artifactId>
    <version>2.2.3</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>com.carbonfive</groupId>
    <artifactId>test-support</artifactId>
    <version>0.6</version>
    <scope>test</scope>
</dependency>

Check out the sample application for details. It's mavenized and utilizes an in-memory database. Just check it out of subversion, look over the code, and give it a run using your IDE or from the command-line (mvn install). I'd be psyched to hear what you think and of course, welcome comments and suggestions.

Resources:

Posted by Christian Nelson at 3:31 PM | Comments (0)

May 5, 2008

Multithreaded Testing

Every now and then you'll work on something that needs to handle requests from multiple concurrent threads in a special way. I say "special way" because in a web application, everything needs to handle being executed concurrently and there are a slew of techniques used to handle this (prototypes, thread locals, stateless services, etc). Here's an example of what I mean by "special"...

On my current project, we have a queue of articles that need human-user attention. Each article must be doled out to only one user and there are multiple instances of the web application servicing requests in the cluster. We can't rely on Java synchronization because it only works within the JVM instance, not across instances.

The simplified version of the service interface we're working on looks like this:

public interface ArticleService {
 
     Article findNextArticleForModeration();
 
}

What makes this interesting is that we must ensure that the service doesn't hand out the same Article to more than one user. This is impossible to assert using a single thread. We've all been told that multiple threads and automated testing don't mix. It's generally true and should be avoided if at all possible, but in some cases it's the only way we can truly assert specific behavior. I've found a pretty simple way to do this type of testing in a reliable, consistent, and non-disruptive manner. Despite the fact that the technique leverages Java 1.5 built-in concurrency utilities, most of the engineers who have seen it are surprised and weren't aware that such testing was so easy to implement.

Given the above service interface, here's a test that will assert that no single article is given out to more than one invoker of the method findNextArticleForModeration(). The scenario we're simulating is 10 users feverishly moderating a queue of 250 articles as quickly as possible.

public void findNextArticleForModerationStressTest() throws Exception
{
     final int ARTICLE_COUNT = 250;
     final int THREAD_COUNT = 10;
 
     // Create test data and callable tasks
     //
     Set<Article> testArticles = new HashSet<Article>();
 
     Collection<Callable<Article>> tasks = new ArrayList<Callable<Article>>();
     for (int i = 0; i < ARTICLE_COUNT; i++)
     {
          // Test data
          testArticles.add(new Article());
  
          // Tasks - each task makes exactly one service invocation.
          tasks.add(new Callable<Article>()
          {
               public Article call() throws Exception
               {
                    return articleService.findNextArticleForModeration();
                }
           });
      }
     articleService.createArticles(testArticles);
 
     // Execute tasks
     //
     ExecutorService executorService = Executors.newFixedThreadPool(THREAD_COUNT);
     // invokeAll() blocks until all tasks have run...
     List<Future<Article>> futures = executorService.invokeAll(tasks);    
     assertThat(futures.size(), is(ARTICLE_COUNT));
 
     // Assertions
     //
     Set<Long> articleIds = new HashSet<Long>(ARTICLE_COUNT);
     for (Future<Article> future : futures)
     {
          // get() will throw an exception if an exception was thrown by the service.
          Article article = future.get();
          // Did we get an article?
          assertThat(article, not(nullValue()));
          // Did the service lock the article before returning?
          assertThat(article.isLocked(), is(true));
          // Is the article id unique (see Set.add() javadoc)?
          assertThat(articleIds.add(article.getId()), is(true));
      }
     // Did we get the right number of article ids?
     assertThat(articleIds.size(), is(ARTICLE_COUNT));
}

The test starts off by creating 250 test articles to be moderated. It also creates 250 'tasks', each designed to make a single service invocation of findNextArticleForModeration(). The real magic happens in Executors.newFixedThreadPool() and executorService.invokeAll(). The first creates a new ExecutorService backed by a thread pool of the specified size. This is a generic ExecutorService that is designed to churn through tasks using all of the threads in the pool. invokeAll blocks until every task has finished executing. In this test, 10 threads will rip through 250 tasks, each making a single call to our service and capturing the result of that call. Each task execution results in a Future, which is a handle to the results of the task (and more).

Iterating over each resulting future, we make several assertions. The most important one is the last, where we assert that every task is given a unique Article. Thanks to the natural semantics of Set, this is easy to do in an elegant way. Another useful, though unexpected, feature is that if an exception occurs during the task execution, an ExecutionException will be thrown when get() is called on the corresponding Future. If our service fails for some reason, the test will fail because no exceptions are expected.

This technique makes simulating a multi-threaded environment in a test easy and readable. It's important to only use this technique when it's really necessary. The resulting test is more of an integration test than a unit test, and its run time is an order of magnitude or more than a unit test, so overuse of the technique will artificially inflate the time it takes to runs the tests. After I've finished working on the component under test, I will reduce the test-data size and thread count to a level that the test still provides value, but is no longer a stress test (e.g. 10 articles and 2 threads). The next time the component is being worked on, the developer can crank up the values and run the tests to be confident that the behavior isn't broken.

The complete source for a working example of this technique is available here. You'll need Maven (or IntelliJ IDEA 7.x) to build and run the test. By default, the tests run against an in-memory H2Database instance, but if you look at application.properties you'll see configurations for PostgreSQL and MySQL as well.

Happy testing!

Posted by Christian Nelson at 3:06 PM | Comments (0)