On my current project, we have a queue of articles that need human-user attention. Each article must be doled out to only one user and there are multiple instances of the web application servicing requests in the cluster. We can't rely on Java synchronization because it only works within the JVM instance, not across instances.

The simplified version of the service interface we're working on looks like this:

public interface ArticleService {
 
     Article findNextArticleForModeration();
 
}

What makes this interesting is that we must ensure that the service doesn't hand out the same Article to more than one user. This is impossible to assert using a single thread. We've all been told that multiple threads and automated testing don't mix. It's generally true and should be avoided if at all possible, but in some cases it's the only way we can truly assert specific behavior. I've found a pretty simple way to do this type of testing in a reliable, consistent, and non-disruptive manner. Despite the fact that the technique leverages Java 1.5 built-in concurrency utilities, most of the engineers who have seen it are surprised and weren't aware that such testing was so easy to implement.

Given the above service interface, here's a test that will assert that no single article is given out to more than one invoker of the method findNextArticleForModeration(). The scenario we're simulating is 10 users feverishly moderating a queue of 250 articles as quickly as possible.

public void findNextArticleForModerationStressTest() throws Exception
{
     final int ARTICLE_COUNT = 250;
     final int THREAD_COUNT = 10;
 
     // Create test data and callable tasks
     //
     Set<Article> testArticles = new HashSet<Article>();
 
     Collection<Callable<Article>> tasks = new ArrayList<Callable<Article>>();
     for (int i = 0; i < ARTICLE_COUNT; i++)
     {
          // Test data
          testArticles.add(new Article());
  
          // Tasks - each task makes exactly one service invocation.
          tasks.add(new Callable<Article>()
          {
               public Article call() throws Exception
               {
                    return articleService.findNextArticleForModeration();
                }
           });
      }
     articleService.createArticles(testArticles);
 
     // Execute tasks
     //
     ExecutorService executorService = Executors.newFixedThreadPool(THREAD_COUNT);
     // invokeAll() blocks until all tasks have run...
     List<Future<Article>> futures = executorService.invokeAll(tasks);    
     assertThat(futures.size(), is(ARTICLE_COUNT));
 
     // Assertions
     //
     Set<Long> articleIds = new HashSet<Long>(ARTICLE_COUNT);
     for (Future<Article> future : futures)
     {
          // get() will throw an exception if an exception was thrown by the service.
          Article article = future.get();
          // Did we get an article?
          assertThat(article, not(nullValue()));
          // Did the service lock the article before returning?
          assertThat(article.isLocked(), is(true));
          // Is the article id unique (see Set.add() javadoc)?
          assertThat(articleIds.add(article.getId()), is(true));
      }
     // Did we get the right number of article ids?
     assertThat(articleIds.size(), is(ARTICLE_COUNT));
}

The test starts off by creating 250 test articles to be moderated. It also creates 250 'tasks', each designed to make a single service invocation of findNextArticleForModeration(). The real magic happens in Executors.newFixedThreadPool() and executorService.invokeAll(). The first creates a new ExecutorService backed by a thread pool of the specified size. This is a generic ExecutorService that is designed to churn through tasks using all of the threads in the pool. invokeAll blocks until every task has finished executing. In this test, 10 threads will rip through 250 tasks, each making a single call to our service and capturing the result of that call. Each task execution results in a Future, which is a handle to the results of the task (and more).

Iterating over each resulting future, we make several assertions. The most important one is the last, where we assert that every task is given a unique Article. Thanks to the natural semantics of Set, this is easy to do in an elegant way. Another useful, though unexpected, feature is that if an exception occurs during the task execution, an ExecutionException will be thrown when get() is called on the corresponding Future. If our service fails for some reason, the test will fail because no exceptions are expected.

This technique makes simulating a multi-threaded environment in a test easy and readable. It's important to only use this technique when it's really necessary. The resulting test is more of an integration test than a unit test, and its run time is an order of magnitude or more than a unit test, so overuse of the technique will artificially inflate the time it takes to runs the tests. After I've finished working on the component under test, I will reduce the test-data size and thread count to a level that the test still provides value, but is no longer a stress test (e.g. 10 articles and 2 threads). The next time the component is being worked on, the developer can crank up the values and run the tests to be confident that the behavior isn't broken.

The complete source for a working example of this technique is available here. You'll need Maven (or IntelliJ IDEA 7.x) to build and run the test. By default, the tests run against an in-memory H2Database instance, but if you look at application.properties you'll see configurations for PostgreSQL and MySQL as well.

Happy testing!

<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
    <property name="location" value="classpath:application.properties"/>
</bean>

<!-- A sample bean that needs some settings. -->
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
    <property name="driverClassName" value="${jdbc.driver}"/>
    <property name="url" value="${jdbc.url}"/>
    <property name="username" value="${jdbc.username}"/>
    <property name="password" value="${jdbc.password}"/>
</bean>

And application.properties might look like this:

jdbc.driver=org.h2.Driver
jdbc.url=jdbc:h2:mem:example
jdbc.username=sa
jdbc.password=

Note, you can achieve the same simple configuration using the new spring 2.x style schema configuration, but it doesn't allow for any further customization so we're going to use the old style.

<!-- Example of new Spring 2.x style -->
<context:property-placeholder location="classpath:application.properties"/>

This handles the simple case of replacing placeholders (e.g. ${jdbc.url}) with values found in a properties files (e.g. jdbc.url=jdbc:h2:mem:example). In a real-world application, we not only need to collect settings, but also override them in different environments. Many of our applications are deployed in 4 or more environments (developer machine, build server, staging server, and production), each requiring different databases at the very least.

There are a few ways to enable overriding of properties. Let's take a look at them in turn:

1. Setting the system properties mode to override (default is fallback)

<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
    <property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE"/>
    <property name="location" value="classpath:application.properties"/>
</bean>

When configured in this mode, any value specified as a system property to the JVM will override any values set in properties files. For example, adding -Djdbc.url=jdbc:h2:mem:cheesewhiz to the JVM arguments would override the value in the file (jdbc:h2:mem:example). On a Java 1.5 or newer platform, Spring will also look for an environment variable called jdbc.url is no system property was found.

2. Specifying an optional properties file

<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
    <property name="ignoreResourceNotFound" value="true"/>
    <property name="locations">
        <list>
            <value>classpath:application.properties</value>
            <value>classpath:local.properties</value>
        </list>
    </property>
</bean>

When ignoreResourceNotFound is set to true, Spring will ignore resources that don't exist. You can imagine application.properties, containing all of the default settings, versioned in your SCM system. Developers have the option of creating a properties file called local.properties to override any settings that differ in their environment. This file should be unversioned and ignored by your SCM system. This works because properties are loaded in order and replace previous values.

3. Web Application overrides

In a web application environment, Spring also supports specifying values in web.xml as context params or in your application server specific meta-data as servlet attributes. For example, if you're using Tomcat you can specify one or more parameter elements in your context.xml, and Spring will can inject those values into placeholders.

<bean class="org.springframework.web.context.support.ServletContextPropertyPlaceholderConfigurer">
    <property name="location" value="classpath:application.properties"/>
</bean>

The ServletContextPropertyPlaceholderConfigurer conveniently works in non servlet environments by falling back to the behavior of a PropertyPlaceholderConfigurer. This is great when running unit tests.

4. Combining techniques

There's no reason why these techniques can't be combined. Technique #1 is great for overriding a few values while #2 is better for overriding many. #3 just expands the field of view when Spring goes to resolve placeholders. When combined, system properties override those in files. When using technique #3, there are some settings available for adjusting the override behavior (see contextOverride). Test the resolution order when combining to ensure it's behaving as expected.

Optional External Properties

There's another use case that applies to some projects. Often in non-developer environments, system admins want to keep properties for the environment outside of the deployable archive or the application server, and they don't want to deal with keeping those files in a Tomcat context file; they prefer a simple properties file. They also don't want to have to place the file in a hard-coded location (e.g. /var/acmeapp/application.properties) or they may keep configuration for multiple servers in the same network directory, each file names after the server. With a little trickery, it's easy to support an optional external properties file that isn't in a hard-coded location. The location of the file is passed as a single system property to the JVM, for example: -Dconfig=file://var/acmeapp/server1.properties. Here's the configuration to make it happen:

<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
    <property name="ignoreUnresolvablePlaceholders" value="true"/>
</bean>

<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
    <property name="ignoreResourceNotFound" value="true"/>
    <property name="location" value="${config}"/>
</bean>

The first definition enables basic property resolution through system properties (in fallback mode). The second bean loads the resource from the location resolved from the system property -Dconfig. All spring resource urls are supported, making this very flexible.

Putting it all together

Here's a configuration that does more than most people would need, but allows for ultimate flexibility:

<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
    <property name="ignoreUnresolvablePlaceholders" value="true"/>
</bean>

<bean class="org.springframework.web.context.support.ServletContextPropertyPlaceholderConfigurer">
    <property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE"/>
    <property name="searchContextAttributes" value="true"/>
    <property name="contextOverride" value="true"/>
    <property name="ignoreResourceNotFound" value="true"/>
    <property name="locations">
        <list>
            <value>classpath:application.properties</value>
            <value>classpath:local.properties</value>
            <value>${config}</value>
        </list>
    </property>
</bean>

Every placeholder goes through the following resolution process. Once a value is found it's set and the next placeholder is resolved:

1. (optional) Property value specified as a system or environment property; useful for overriding specific placeholders.
   e.g. -Djdbc.host=devdb
   e.g. -Djdbc.username=carbon5
2. (optional) Context parameters located in web.xml or context attributes specified in application server meta-data (e.g. a Tomcat context.xml).
3. (optional) Properties file located by the system/environment variable called "config"; useful for externalizing configuration. All URL types are supported.
   e.g. -Dconfig=c://hmc.properties
4. (optional) Properties file identified by classpath:local.properties; useful for specific developer overrides.
5. (required) Properties file identified by classpath:application.properties, which contains default settings for our application.

Best Practices

• Deploy the same exact artifact (e.g. war, ear, etc) across all environments by externalizing configuration. This may seem daunting, but the emergent benefits are huge in terms of simplicity.
• Only make things that can safely change across environments configurable. Also, only things that need to be configurable should be configurable, it's easy to go overboard.
• Configure the minimal properties search path that meets your requirements.
• When looking for properties files in the project tree, use classpath resources whenever possible. This makes finding those files easy, consistent, and insensitive to the working-dir, which is great when running tests from your IDE and command line.
• Aim for a zero-configuration check-out, build, run-tests cycle for the environment where its happens most: development.

As you may know, we often use Spring for lifecycle management of our services and dependency injection (among other uses). In a Stripes + Spring application, you can imagine Spring-managed services being used by Stripes Actions (i.e. controllers). Every incoming HTTP request results in a new instance of a Stripes Action, thus the newly created actions must have their dependencies injected for every request.

Stripes ships with a SpringInterceptor that supports annotating fields and methods on your actions with @SpringBean. While this works fine, I was pretty interested in being able to use Spring 2.5's annotations for marking what should be injected on my actions, so I created a Spring25Interceptor (see code below).

The Spring25Interceptor is configured the same way as the out-of-the-box SpringInterceptor. In your web.xml:

...
<filter>
    <display-name>Stripes Filter</display-name>
    <filter-name>StripesFilter</filter-name>
    <filter-class>net.sourceforge.stripes.controller.StripesFilter</filter-class>
    <init-param>
        <param-name>Interceptor.Classes</param-name>
        <param-value>
            net.sourceforge.stripes.integration.spring.Spring25Interceptor
        </param-value>
    </init-param>
    ...
</filter>
...

Since it uses the same annotations that Spring 2.5's supports (@Autowired, @Resource, and @Qualifier), annotating your Stripes Actions is easy and should look pretty familiar (see Juergen's blog for a comprehensive overview). Here are a few hypothetical examples:

public class LandingActionBean extends AbstractActionBean
{
     // Autowire by type (looks for a bean in the application context of type ServiceA)
     @Autowired ServiceA serviceA;
 
     // Autowire by name (looks for a bean with the name 'serviceB')
     @Resource ServiceB serviceB;
     
     // Autowire by name (looks for a bean with the name 'serviceC')
     @Autowired @Qualifier("serviceC") ServiceC serviceCee;
     
     // Method injection examples (all of the above can be applied to methods as well)
 
     @Autowired
     public void setServiceA(ServiceA a) { this.serviceA = a; }
     
     @Autowired
     public void setServices(ServiceA a, ServiceB b, ServiceC c) {
          this.serviceA = a;
          this.serviceB = b;
          this.serviceC = c;
      }
 
     @DefaultHandler
     public Resolution execute()
     {
          return new ForwardResolution("/landing.jsp");
      }
}

While it may be considered a subtle improvement, I really like the fact that with this new interceptor, my application has a consistent syntax for dependency injection across the tiers. Additionally, the semantics of the Spring annotations are also consistent and shared throughout. An added bonus is that the new Interceptor is significantly smaller than the existing one.

Here's the Interceptor code that does the dependency injection into actions:

import net.sourceforge.stripes.action.*;
import net.sourceforge.stripes.controller.*;
import net.sourceforge.stripes.util.*;
import org.springframework.beans.factory.config.*;
import org.springframework.context.*;
import org.springframework.util.*;
import org.springframework.web.context.support.*;
import javax.servlet.*;

@Intercepts(LifecycleStage.ActionBeanResolution)
public class Spring25Interceptor implements Interceptor
{
     private static final Log log = Log.getInstance(Spring25Interceptor.class);
 
     public Resolution intercept(ExecutionContext context) throws Exception
     {
          Resolution resolution = context.proceed();
          log.debug("Running Spring dependency injection for instance of ", context.getActionBean().getClass().getSimpleName());
          ServletContext servletContext = StripesFilter.getConfiguration().getServletContext();
          ApplicationContext applicationContext = WebApplicationContextUtils.getWebApplicationContext(servletContext);
          AutowireCapableBeanFactory beanFactory = applicationContext.getAutowireCapableBeanFactory();
          beanFactory.autowireBeanProperties(context.getActionBean(), AutowireCapableBeanFactory.AUTOWIRE_NO, false);
          beanFactory.initializeBean(context.getActionBean(), StringUtils.uncapitalize(context.getActionBean().getClass().getSimpleName()));
          return resolution;
      }
}

I've created a enhancement request for this feature, though the comment-thread is a little bit all over the place. You can find the latest version of the code and Javadoc as an attachment on the issue.

Cooper talked for a while about the concept of craftsmanship as it relates to designers and developers. He emphasized that craftsmanship prioritizes quality over cost and time. While keeping costs down and accomplishing tasks in a timely matter are important, craftspeople do not sacrifice quality for either. They take the time and spend the money that it takes to do it right.

The Triad

He proposes that interaction designers and programmers are craftspeople who should unite around their common commitment to quality and demand that business take the time and spend the money to do things right. Doing things right in his proposal means a three-stage process of:

Interaction Design » Design Engineering » Production Engineering

Interaction Design starts with interaction designers talking to stakeholders, both customers and users, to understand the domain and behavioral requirements for software. They then design behaviors for the software based on this primary research.

Design Engineering starts with programmers working with interaction designers to evaluate the feasibility of the designed behavior, iterating over the designs in collaboration with the interaction designers. In this phase, programmers also hash out how the final system will work through a series of short iterations of development focused on identifying and solving technical problems. This product of this work is not production code, it is the definition of how the final system should be built. He feels that Agile methods are very appropriate for this phase.

Production Engineering is the process of cranking out the final product based on detailed written specifications produced in the prior two phases. Cooper claims that since all the details have been worked out, both in behavior and technology, this phase will be maximally efficient — a straight line of productive software development from start to delivery. He feels that a process like RUP is appropriate for this phase.

Cooper calls this process the Triad and elaborates on his web site.

His split between Design Engineering and Production Engineering is based partly on his claim that programmers can fundamentally be divided into two types — those who above all else want to do it right and those who want to get it done and out the door. The ones who want to do it right are the craftsmen and are best suited to be Design Engineering practitioners. The ones who want to get it done are best suited to be Production Engineering practitioners.

Hmmm

Cooper seems to have come a long way in his understanding and appreciation of Agile practices since his notorious 2002 conversation with Kent Beck. Iteration, collaboration and feedback are all important characteristics of his Design Engineering phase.

However, it's not clear why Production Engineering would not benefit from an Agile process focused on frequent delivery of working software and tracking toward an end goal that has good definition. It's also not clear why Design Engineering shouldn't produce early versions of working software that can be tested with users and be the foundation of the final delivered software.

Some projects may benefit from his phasing. I could see his process benefiting a year long initiative to build shrink wrapped software where there is a desire to offshore a portion of the development.

But web software development does not work like that. You can release web software early and often without compromising quality. Yes, heed Cooper's warning not to rush prematurely to market with ill-conceived products. Stay focused on what matters to users and let them use it as early as you can. Cooper himself cites Sergey Brin of Google as saying, paraphrased, "that early on they were in no rush to have people try Google today. Tomorrow it would be better. Tomorrow would be fine." Google was in fact out early with something simple but well-considered in users' hands. They've evolved their product daily and weekly to what it is today. They are proof that releasing early does not necessarily mean prematurely.

Cooper's attempt to place interaction designers between programmers and their customers and users is also troubling. He claims that programmers don't want to talk to users which we know is simply wrong. In our experience, the further we are from our customer and end user in a project the greater the likelihood the project will go awry. I'd rather see an approach that figures out how to get programmers and interaction designers understanding the needs of their customers and users together and using that understanding to direct work in their specialized areas of expertise.

As a programmer, Cooper can be a bit hard to swallow. He definitely encourages an "us and them" perspective with his tongue-in-cheek comments about the inscrutability of programmers and their work. Of course he does the same for other relationships with a slew of disparaging claims about management and business executives.

Yet it's encouraging to hear him emphasize the importance of true collaboration between designers and programmers through the life of a project. I hope his understanding of programmers and their craft continues to evolve to support this goal.

I like the semantics of paginating_find over will_paginate but am not married to that choice if anyone wants to try to convince me otherwise.

In any case, I wanted to paginate the finders created by has_finder. I wrote a simple ActiveRecord mixin that provides a class method 'acts_as_paginated and is used in your model like:

class Visit < ActiveRecord::Base
  belongs_to :person

  acts_as_paginated :size=> 10
  
  has_finder :for_person, lambda { |person| {
        :conditions => { :person_id => person},
        :order => 'datetime DESC'
    } }

  has_finder :in_date_range, lambda { |from,to| {
        :conditions => [ "visits.datetime >= ? and visits.datetime <= ?", from, to ]
    } }
end

Client code looks like:

 from, to = Date.new(2008,2,1), Date.new(2008,2,3)
 visits = Visit.for_person(@person).in_date_range(from, to).paginate(:page => 3)

The mixin looks like:

# Provides Model.paginate(options={}) to allow
# the paginating_find plugin to be used with the has_finder plugin.
#
# Usage:
#   acts_as_paginated
#   acts_as_paginated :page => 2, size => 10
# 
# Install:
#   Save this file as lib/acts_as_paginated.rb and require 'acts_as_paginated'
#   in environment.rb.
#
module HasFinder
  module PaginatingFind #:nodoc:

    def self.included(mod)
      mod.extend(ClassMethods)
    end

    module ClassMethods
      def acts_as_paginated(options={ :page => 1, :size => 20})
        cattr_accessor :paginate_defaults
        self.paginate_defaults = options

        self.class_eval do
          extend HasFinder::PaginatingFind::SingletonMethods
        end
      end
    end

    module SingletonMethods
      def paginate(args={})
        options = self.paginate_defaults.clone
        options[:page] = args[:page].to_i if args[:page]
        options[:size] = args[:size].to_i if args[:size]
        find(:all, :page => { :size => options[:size], :current => options[:page], :first => 1 })
      end
    end
  end
end

ActiveRecord::Base.send(:include, HasFinder::PaginatingFind)

It also works to paginate associations:

@person.visits.paginate(:size => 4)

One project wrote a rake task that converts ERB views to Markaby. Then Ingar created a plugin that re-implements the base Rails scaffolding to create Markaby views that output the same HTML as the base Rails scaffolding. I took his work one standard refactor further to introduce a MarkabyHelper with helper functions for the layout of form inputs and model views.

Usage

./script/plugin install http://svn.carbonfive.com/public/carbonfive/rails/plugins/markaby_scaffold/trunk/

Use it as you would regular Rails scaffolding:

./script/generate markaby_scaffold Post title:string body:text published:boolean

Prerequisites

markaby_scaffold requires that you install Markaby as a Rails plugin.

Gravy

Unfortunately, Markaby does not appear to be getting the love that it deserves. There is an issue using Markaby with Rails 2.0.2 for which Randy has submitted a patch. This plugin includes a monkey patch for Markaby and Rails 2.0.2 that provides this fix so you don't have to apply it yourself.

Additionally, the generated MarkabyHelper includes support for using Markaby in your helpers, provides a quick start for refactoring the scaffolding layouts and includes test cases to get you started testing your Markaby helpers.

During development, it’s a big deal because each engineer has two instances of the database, one for unit tests and another for running the application. We need an easy way to create a new, up-to-date database and update existing databases. Once a project has launched, it’s a big deal because we need a way to migrate a database teeming with important production data to the latest version without losing critical information.

High Level Requirements

• Initiate a migration from the command-line as a Maven plugin
• Programmatically migrate a database during application startup
• Convention over Configuration
• Initially support migrations written in SQL

At a high level, the migration process looks like this:

1. Query the database (table db_version) to find the current version.
2. Determine the latest database schema version available.
3. If the database is out of date, run each migration in order in its own transaction, updating the db_version for each migration.

We’d identified two usage patterns, the first is more akin to the Rails Migration model in that you explicitly migrate the database via the command line. The second is automatic migration when an application starts up, before Hibernate initializes or any other data access takes place. I’ll discuss each in turn.

Migrating using Maven

This functionality is easy to enable in a mavenized project. First you add the Carbon Five public plugin repository:

<pluginRepositories>
  <pluginRepository>
    <id>c5-public-repository</id>
    <url>http://mvn.carbonfive.com/public</url>
  </pluginRepository>
</pluginRepositories>

And then you configure the migration plugin:

<plugin>
  <groupId>com.carbonfive</groupId>
  <artifactId>maven-migration-plugin</artifactId>
  <version>0.9-SNAPSHOT</version>
  <configuration>
    <defaultEnvironment>test</defaultEnvironment>
    <environments>
      <environment>
        <name>default</name>
        <driver>com.mysql.jdbc.Driver</driver>
        <username>dev</username>
        <password>dev</password>
      </environment>
      <environment>
        <name>test</name>
        <url>jdbc:mysql://localhost/myapp_test</url>
      </environment>
      <environment>
        <name>development</name>
        <url>jdbc:mysql://localhost/myapp_development</url>
      </environment>
    </environments>
  </configuration>
  <dependencies>
    <dependency>
      <groupId>mysql</groupId>
      <artifactId>mysql-connector-java</artifactId>
      <version>5.1.5</version>
    </dependency>
  </dependencies>
</plugin>

You’ll notice that we’ve got 2 environments configured. You can have as many as you need and you can specify which you want to migrate on the command line. If none are specified the default environment will be migrated. In this example we’re specifying the dependency on our JDBC driver so that the plugin has access to the code it needs to connect the database.

Lastly, you drop in your migration scripts into the src/main/resources/db/migrations directory, naming them using the pattern NNN_description.sql, where NNN is three digits indicating the script sequence. Some examples might be:

• 001_create_users_table.sql
• 002_add_default_users.sql
• 003_add_lastvisit_column.sql

The description is optional and isn’t used for anything, it’s just there so that other developers can get an idea of what a script does without having to open it.

From the command line, you can run the migration plugin like this:

Note that he database must exist for the migrations to take place as we do not create missing databases (yet).

I’ve created a simple, complete sample that shows off this functionality, it’s on the C5 public subversion repository here. Check it out and then read the readme.txt at the top of the project.

Migrating from your Application

The other usage scenario is to auto-migrate during application startup. At the core of the framework, there’s an interface called MigrationManager which has two implementations: DataSourceMigrationManager and DriverManagerMigrationManager. Migration happens right after a datasource (of the javax.sql variety) is created.

Migrating from your application is as easy as instantiating one of these early in the startup cycle and invoking the migrate() method, something like this:

MigrationManager migrationManager = new
    DriverManagerMigrationManager(“com.mysql.jdbc.Driver”, “jdbc:mysql://localhost/myapp_test”,
“dev”, “dev”);
migrationManager.migrate();

Of course this needs to happen before anything else in the application uses the database; we want to database to be updated completely before it’s used.

Spring is part of our standard development stack on our Java projects, and it’s easy to enforce these dependencies in Spring configuration. First we define a data source for the application:

<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
  <property name="driverClassName" value="org.h2.Driver"/>
  <property name="url" value="jdbc:h2:file:~/.h2/migration_sample2_test"/>
  <property name="username" value="dev"/>
  <property name="password" value="dev"/>
</bean>

And now we declare our MigrationManager instance. Note the ‘init-method’ attribute:

<bean id="migrationManager"
    class="com.carbonfive.db.migration.DataSourceMigrationManager"
    init-method="migrate">
  <constructor-arg ref="dataSource"/>
</bean>

And then you define something that’s going to use the defined DataSource. Note the ‘depends-on’ attribute:

 <bean id="userService"
    class="com.carbonfive.migration.sample2.UserService"
    depends-on="migrationManager">
  <constructor-arg ref="dataSource"/>
</bean>

This is obviously a little contrived for the sake of example, but you get the point. In a typical application the thing that would depend on the datasource is a Hibernate SessionFactory.

You can visit the source code for this example on the C5 public subversion repository here.

Best Practices

Here are a few of the things we’ve learned along the way:

• Start using migrations early. Definitely start by time there’s more than one person on a project. I usually start off letting hibernate generate my schema while I’m experimenting with things, but as soon as I’m really working on features I’ll switch over to migrations.
• All database changes are captured as a new migration.
• Migration scripts cannot be changed once *anyone* has run them and make further changes in a new migrations.

Source Code Access

The Carbon Five db-support project which contains all of this migration goodness is available on the C5 public subversion repository at https://svn.carbonfive.com/public/carbonfive/db-support/trunk. It’s a maven project and should compile and pass its tests out of the box. I encourage you to look through the code and check out the tests.

Future

If you look through the code you’ll see some of what’s in store for this project. We’ve got initial support for writing migrations in Groovy and JRuby and we’re thinking about added Java support as well. We’re looking for feedback to drive the future direction of the project, so feel free to write us and let us know what you think.

We migrated to Maven 2 for builds partially because it’s convention driven, so when questions of how we should do something come up we can default to Maven best practices. Another reason is that it enabled us to stick to the Don’t Repeat Yourself (DRY) principle. We didn’t want to duplicate build meta-data by maintaining both command-line configuration (Maven pom.xmls) and IntelliJ IDEA configuration. Maven deals with this pretty well via the maven-idea-plugin.

The downside to running “mvn idea:module” every time there’s a pom change is that it eats at least a few minutes with all of the checks for sources and javadocs. To make matters worse, we always had to run it from the master directory so that intra-module dependencies were modeled correctly as IDEA module dependencies, not jar dependencies. Each developer would go through this routine when there was a change because we weren’t versioning our IDEA meta-data in subversion. When considering all of the engineer’s time, this process would consume over half an hour in total.

I wanted to change the process so that we still used the maven-idea-plugin to generate the meta-data (staying true to DRY), but then commit the generated files to subversion so that only one person would have to go through the pain of re-generating (presumably whoever made the changes to the poms) and the rest of the team could continue without much disruption. The problem is that each generated .iml file contains absolute paths which are specific to their machine.

After a bit of sleuthing, I discovered an IDEA option called Path Variables which is available in the general settings section. There isn’t much to tell you what it does, so it’s easily overlooked. You can specify one or more variables and IDEA will automatically replace file paths in project (.ipr) and module files (.iml) with the provided variables. Everyone on the development team added a Path Variable called “M2_REPO” which points to their local maven repository (~/.m2/repository).

After running “mvn idea:module” we just open the project in IDEA and it will do the replacing behind the scenes. Now the module meta-data can be versioned and shared.

It’s a small feature, but it’s helping us streamline everyone’s day-to-day.

IDEA 7 includes native Maven support which is getting better with each point release. The process described herein works great with the native plugin as well.

Here's a toy chat client and parrot bot I wrote for the Google Talk service using JRuby and the Java Smack API from Jive's open source portal ignite realtime.

JRubyGems allows you to package RubyGems on your Java application classpath. It removes the file system dependencies on locally installed gems that get awkward when using JRuby and dependent gems in a Java environment.

Use JRubyGems as a replacement for RubyGems. Instead of

require 'rubygems'
gem 'activerecord'

use

require 'jrubygems'
gem 'activerecord'

Behind the scenes JRubyGems requires RubyGems and injects behavior specific to finding gems on the Java classpath if they are not already installed.

You make JRubyGems available to your application by putting the jar on your application classpath.

Looking forward, I want to be sure that we take maximum advantage of our existing knowledge and new learning in both Java and Ruby. I have high hopes for JRuby as a valuable bridge that spans both languages and gives developers the flexibility to use the strengths of the two. Sun appears to feel the same way as they have hired the two core JRuby developers, Charles Nutter and Thomas Enebo, to work on JRuby full time.

In preparation for more specific posts, here are some light musings on JRuby.

The post is the first of a series on user experience (UX) design practices for agile software development teams. In an upcoming project with our partner Hot Studios we will be exploring new ways to integrate their user experience design process with our software development process. We'll share some learnings from that, experiences from the past and new ideas for the future.

First we have a growing reading list of existing resources on the topic that I am posting here for you. If you have other resources for us, please share.

User Research as a Commodity
Added 2008-02-14
Alex at Pivotal Labs noted this post from a user experience manager at Kaiser Permanente who describes a strategy for including regular user testing in an agile development process. The strategy is attractive because it fits with a key characteristic of agile where simple practices performed regularly produce high level complex results.

Kent Beck v. Alan Cooper
This is a 2002 article with the following abstract "Kent Beck is known as the father of 'extreme programming,' a process created to help developers design and build software that effectively meets user expectations. Alan Cooper is the prime proponent of interaction design, a process with similar goals but different methodology. We brought these two visionaries together to compare philosophies, looking for points of consensus—and points of irreconcilable difference." The difference in perspective between the two is very expressive of the conflicts we have seen between design and agile development processes. Many of the following resources reference this discussion.

Note: the original source for this great interview appears to have been taken over by another entity that is no longer making the article available. I am linking to it in the Internet Archive.

Waterfall Bad, Washing Machine Good
Leisa Reichelt's IA Summit 07 presentation on the role of IA in the design process identifies the failures of a waterfall design process and argues that an agile, iterative design process is superior. Slide 14 is right on.

User Experience Activities on Agile Development Projects
Recent article by Scott Ambler addresses the gap between Beck and Cooper by identifying a middle ground that requires UX and agile practitioners to learn a bit more about each others' practices and to adopt some of each.

Holistic Product Design & Development
A series of blog posts by Jeff Patton on agile software development and user centered design. Jeff moderates the agile-usability Yahoo group.

Agile Usability Yahoo! Group
The description for this group provides a good summary of the issues we are tackling here. You are invited to jump into the fray.

Experiences Integrating UED Practices
In this experience report Paul Hodgetts presents his "teams' struggles to come to grips with the often programming-centric orientation of agile processes, and their ongoing efforts to integrate their UED (user experience design) best practices into the incremental, collaborative world of agile processes." He provides an overview to the range of UX practices, identifies challenges and discusses practices that he found were particularly useful in integrating UX and and agile development.

Agile User-Centered Design
A post to ID-discuss that presents one designer's struggle to reconcile agile and user-centered design.

Process Agility and Software Usability: Toward Lightweight Usage-Centered Design
2001 paper by Larry Constantine presents concrete practices for UX that map to agile practices. His approach is simple and pragmatic.

The Challenge of Agile Design
These are slides from a talk given by our friends at Pivotal Labs and Satisfaction.

User Centered Design Round Table
Podcast of a discussion on user-centered design and agile development with Lynn Miller, Jeff Patton and Rebecca Wirfs-Brock.

Originally posted 2007-09-06
Last updated 2008-02-14

We chose to use the Dojo Toolkit (version 0.4.2) for both the AJAX (ie: remote calls to the server) and the Widegtry side of the Web 2.0 style. It's been working fairly well. Dojo has a lot of functionality, but also has a few areas that still need some refinement. The most interesting / infuriating to me so far has to do with Dojo drag-and-drop functionality within IE.

I finally got fed up with the situation and worked out a solution that lets developers specify path parameters in the dispatcher mappings, which will appear as request parameters in the controller and view. I've made the project available on our public SVN server. The solution requires only 4 classes, so you can download them instead. Read on for configuration information.