Unit tests, Mock objects, and App Engine

For my [still a secret] project which is running on Google App Engine infrastructure [1], I want to make it as solid as possible from the beginning by applying most of the best practices of the Agile methodology [2].

Update 2009/12/05:
With the release of the App Engine Java SDK 1.2.8 (read release notes, I had to update my code and this post on two points:

• Without the specification of the JDO inheritance type, the environment assumes it's superclass-table. This type is not supported by App Engine. Only subclass-table and complete-table are supported. In the Entity class described below, I had to add @Inheritance(strategy = InheritanceStrategy.SUBCLASS_TABLE). Read the documentation about Defining data classes for more information.
• With the automation of the task execution, the MockAppEngineEnvironment class listed below had to be updated to serve an expected value when the Queue runs in the live environment. Read the details on the thread announcing the 1.2.8. SDK prerelease on Google Groups.

Now, all tests pass again ;)

As written on my post from September 18, I had to develop many mock classes to keep reaching the mystical 100% of code coverage (by unit tests) [3]. A good introduction of mock objects is given by Vincent Massol in his book “JUnit in Action” [4]. To summarize, mock objects are especially useful to inject behavior and force the code using them to exercise complex control flows.

Developing applications for Google App Engine is not that complex because the system has a good documentation and an Eclipse plug-in ease the first steps.

Use case description

Let's consider a simple class organization implementing a common J2EE pattern:

• A DTO class for a Consumer;
• The DAO class getting the Consumer from the persistence layer, and sending it back with updates; and
• A Controller class routing REST requests. The Controller is an element of the implemented MVC pattern

Use case illustration

The code for the DTO class is instrumented with JDO annotations [5]:

Consumer DTO class definition

@PersistenceCapable(identityType = IdentityType.APPLICATION, detachable="true")
@Inheritance(strategy = InheritanceStrategy.SUBCLASS_TABLE)
public class Consumer extends Entity {
    @Persistent
    private String address;
 
    @Persistent
    private String displayName;
 
    @Persistent
    private String email;
 
    @Persistent
    private String facebookId;
 
    @Persistent
    private String jabberId;
 
    @Persistent
    private Long locationKey;
 
    @Persistent
    private String twitterId;
 
    /** Default constructor */
    public Consumer() {
        super();
    }
 
    /**
     * Creates a consumer
     * @param in HTTP request parameters
     */
    public Consumer(JsonObject parameters) {
        this();
        fromJson(parameters);
    }
 
    public String getAddress() {
        return address;
    }
    
    public void setAddress(String address) {
        this.address = address;
    }
    
    //...
}

My approach for the DAO class is modular:

• When the calling code is doing just one call, like the ConsumerOperations.delete(String) method deleting the identified Consumer instance, the call can be done without the persistence layer knowledge.
• When many calls to the persistence layer are required, the DAO API offers the caller to pass a PersistenceManager instance that can be re-used from call to call. With the combination of the detachable="true" parameter specified in the JDO annotation for the Consumer class, it saves many cycles.

Excerpt from the ConsumerOperations DAO class definition

/**
 * Persist the given (probably updated) resource
 * @param consumer Resource to update
 * @return Updated resource
 * @see ConsumerOperations#updateConsumer(PersistenceManager, Consumer)
 */
public Consumer updateConsumer(Consumer consumer) {
    PersistenceManager pm = getPersistenceManager();
    try {
        // Persist updated consumer
        return updateConsumer(pm, consumer);
    }
    finally {
        pm.close();
    }
}
 
/**
 * Persist the given (probably updated) resource while leaving the given persistence manager open for future updates
 * @param pm Persistence manager instance to use - let opened at the end to allow possible object updates later
 * @param consumer Resource to update
 * @return Updated resource
 */
public Consumer updateConsumer(PersistenceManager pm, Consumer consumer) {
    return pm.makePersistent(consumer);
}

The following piece of the abstract class BaseOperations shows the accessor made availabe to any controller code to get one handle of a valid PersistenceManager instance.

Excerpt from the abstract BaseOperations DAO class definition

/**
 * Accessor isolated to facilitate tests by IOP
 * @return Persistence manager instance
 */
public PersistenceManager getPersistenceManager() {
    PersistenceManager pm = getPersistenceManagerFactory().getPersistenceManager();
    pm.setDetachAllOnCommit(true);
    pm.setCopyOnAttach(false);
    return pm;
}

To finish the use case setup, here is a part of the controller code which deals with incoming HTTP requests and serves or operates accordingly. This specific piece of code replies to a GET request like:

• Invocation: http://<host:port>/API/Consumer/43544"
• Response:

  • {key:43544, displayName:"John", address:"75, Queen, Montréal, Qc, Canada", 
    locationKey:3245, location: {id:3245, postalCode:"H3C2N6", countryCode:"CA",
    latitude:43.3, longitude:-73.4}, ...}

Excerpt from the ConsumerRestlet Controller class definition

@Override
protected JsonObject getResource(JsonObject parameters, String resourceId, User loggedUser) throws DataSourceException {
    PersistenceManager pm = getBaseOperations().getPersistenceManager();
    try {
        // Get the consumer instance
        Consumer consumer = getConsumerOperations().getConsumer(pm, Long.valueOf(resourceId));
        JsonObject output = consumer.toJson();
        // Get the related information
        Long locationKey = consumer.getLocationKey();
        if (locationKey != null) {
            Location location = getLocationOperations().getLocation(pm, locationKey);
            output.put(Consumer.LOCATION, location.toJson());
        }
        // Return the complete set of information
        return output;
    }
    finally {
        pm.close();
    }
}

Simple mock

Now, it's time to test! To start slowly, let's deal with the Restlet getResource() method to verify:

• Just one and only one instance of PersistenceManager is loaded by the function;
• The PersistenceManager instance is cleanly closed at the end of the process;
• There's a call issued to get the identified Consumer instance;
• There's possibly a call issued to get the identified Location instance;
• The output value has the expected information.

In the corresponding unit test series, we don't want to interfere with the App Engine infrastructure (the following chapter will address that aspect). So we'll rely on a mock for the PersistenceManager class that will be injected into the ConsumerRestlet code. The full source of this class is available on my open source project two-tiers-utils: javax.jdo.MockPersistenceManager.

Custom part of the mock for the PersistenceManager class

public class MockPersistenceManager implements PersistenceManager {
    private boolean closed = false; // To keep track of the "closed" state
    public void close() {
        closed = true;
    }
    public boolean isClosed() {
        return closed;
    }

    // ...
}

Here are the unit tests verifying the different flow paths:

• When an exception is thrown, because the back-end does not serve the data for example;
• When the Consumer instance returns without location coordinates;
• When the Consumer instance is fully documented.

Three tests validating the behavior of the ConsumerRestlet.getResource() method

@Test(expected=IllegalArgumentException.class)
public void testUnexpectedError() {
    // Test prepration
    final PersistenceManager pm = new MockPersistenceManager();
    final BaseOperations baseOps = new BaseOperations() {
        boolean askedOnce = false;
        @Override
        PersistenceManager getPersistenceManager() {
            if (askedOnce) {
                fail("Expects only one call");
            }
            askedOnce = true;
            return pm;
        }
    };
    final Long consumerId = 12345L;
    final ConsumerOperations consumerOps = new ConsumerOperations() {
        @Override
        Consumer getConsumer(PersistenceManager pm, Long id) {
            assertEquals(consumerId, id);
            throw new IllegalArgumentException("Done in purpose!");
        }
    };
    ConsumerRestlet restlet = new ConsumerRestlet() {
        @Override BaseOperation getBaseOperations() { return baseOps; }
        @Override ConsumerOperation getConsumerOperations() { return consumerOps; }
    }
    
    // Test itself
    JsonObject response = restlet.getResource(null, consumerId.toString, null);
}

@Test
public void testGettingOneConsumer() {
    // Test prepration
    final PersistenceManager pm = new MockPersistenceManager();
    final BaseOperations baseOps = new BaseOperations() {
        boolean askedOnce = false;
        @Override
        PersistenceManager getPersistenceManager() {
            if (askedOnce) {
                fail("Expects only one call");
            }
            askedOnce = true;
            return pm;
        }
    };
    final Long consumerId = 12345L;
    final ConsumerOperations consumerOps = new ConsumerOperations() {
        @Override
        Consumer getConsumer(PersistenceManager pm, Long id) {
            assertEquals(consumerId, id);
            Consumer consumer = new Consumer();
            consumer.setId(consumerId);
            return consumer;
        }
    };
    final Long locationId = 67890L;
    final LocationOperations locationOps = new LocationOperations() {
        @Override
        Location getLocation(PersistenceManager pm, Long id) {
            fail("Call not expected here!");
            return null;
        }
    };
    ConsumerRestlet restlet = new ConsumerRestlet() {
        @Override BaseOperation getBaseOperations() { return baseOps; }
        @Override ConsumerOperation getConsumerOperations() { return consumerOps; }
        @Override LocationOperation getLocationOperations() { return locationOps; }
    }
    
    // Test itself
    JsonObject response = restlet.getResource(null, consumerId.toString, null);
    
    // Post-test verifications
    assertTrue(pm.isClosed());
    assertNotSame(0, response.size());
    assertTrue(response.containsKey(Consumer.ID);
    assertEquals(consumerId, response.getLong(Consumer.ID));
}

@Test
public void testGettingConsumerWithLocation() {
    // Test prepration
    final PersistenceManager pm = new MockPersistenceManager();
    final BaseOperations baseOps = new BaseOperations() {
        boolean askedOnce = false;
        @Override
        PersistenceManager getPersistenceManager() {
            if (askedOnce) {
                fail("Expects only one call");
            }
            askedOnce = true;
            return pm;
        }
    };
    final Long consumerId = 12345L;
    final Long locationId = 67890L;
    final ConsumerOperations consumerOps = new ConsumerOperations() {
        @Override
        Consumer getConsumer(PersistenceManager pm, Long id) {
            assertEquals(consumerId, id);
            Consumer consumer = new Consumer();
            consumer.setId(consumerId);
            consumer.setLocationId(locationId);
            return consumer;
        }
    };
    final LocationOperations locationOps = new LocationOperations() {
        @Override
        Location getLocation(PersistenceManager pm, Long id) {
            assertEquals(locationId, id);
            Location location = new Location();
            location.setId(locationId);
            return location;
        }
    };
    ConsumerRestlet restlet = new ConsumerRestlet() {
        @Override BaseOperation getBaseOperations() { return baseOps; }
        @Override ConsumerOperation getConsumerOperations() { return consumerOps; }
        @Override LocationOperation getLocationOperations() { return locationOps; }
    }
    
    // Test itself
    JsonObject response = restlet.getResource(null, consumerId.toString, null);
    
    // Post-test verifications
    assertTrue(pm.isClosed());
    assertNotSame(0, response.size());
    assertTrue(response.containsKey(Consumer.ID);
    assertEquals(consumerId, response.getLong(Consumer.ID));
    assertTrue(response.containsKey(Consumer.LOCATION_ID);
    assertEquals(locationId, response.getLong(Consumer.LOCATION_ID));
    assertTrue(response.containsKey(Consumer.LOCATION);
    assertEquals(consumerId, response.getJsonObject(Consumer.LOCATION).getLong(Location.ID));
}

Note that I would have been able to override just the PersistenceManager class to have the Object getObjectById(Object arg0) method returning the expected exception, Consumer, and Location instances. But I would have pass over the strict limit of a unit test by then testing also the behavior of the ConsumerOperations.getConsumer() and LocationOperations.getLocation() methods.

App Engine environment mock

Now, testing the ConsumerOperations class offers a better challenge.

As suggested above, I could override many pieces of the PersistenceManager class to be sure to control the flow. But to do a nice simulation, I almost need to have the complete specification of the Google App Engine infrastructure to be sure I mock it correctly. This is especially crucial when processing Query because Google data store has many limitations [6] that others traditional database, like MySQL, don't have...

Because this documentation is partially available and because Google continues to update its infrastructure, I looked for a way to use the standalone environment made available with the App Engine SDK [1]. This has not been easy because I wanted to have the test running independently from the development server itself. I found first some documentation on Google Code website: Unit Testing With Local Service Implementations, but it was very low level and did not fit with my JDO instrumentation of the DTO classes. Hopefully, I found this article JDO and unit tests from App Engine Fan, a great community contributor I mentioned many times in previous posts!

By cooking information gathered on Google Code website and on App Engine Post, I've produced a com.google.apphosting.api.MockAppEngineEnvironment I can use for my JUnit4 tests.

Three tests validating the behavior of the ConsumerRestlet.getResource() method

package com.google.apphosting.api;
 
// import ...
 
/**
 * Mock for the App Engine Java environment used by the JDO wrapper.
 *
 * These class has been build with information gathered on:
 * - App Engine documentation: http://code.google.com/appengine/docs/java/howto/unittesting.html
 * - App Engine Fan blog: http://blog.appenginefan.com/2009/05/jdo-and-unit-tests.html
 *
 * @author Dom Derrien
 */
public class MockAppEngineEnvironment {
 
    private class ApiProxyEnvironment implements ApiProxy.Environment {
        public String getAppId() {
          return "test";
        }
 
        public String getVersionId() {
          return "1.0";
        }
 
        public String getEmail() {
          throw new UnsupportedOperationException();
        }
 
        public boolean isLoggedIn() {
          throw new UnsupportedOperationException();
        }
 
        public boolean isAdmin() {
          throw new UnsupportedOperationException();
        }
 
        public String getAuthDomain() {
          throw new UnsupportedOperationException();
        }
 
        public String getRequestNamespace() {
          return "";
        }
 
        public Map getAttributes() {
            Map out = new HashMap();

            // Only necessary for tasks that are added when there is no "live" request
            // See: http://groups.google.com/group/google-appengine-java/msg/8f5872b05214...
            out.put("com.google.appengine.server_url_key", "http://localhost:8080");

            return out;
        }
    };
 
    private final ApiProxy.Environment env;
    private PersistenceManagerFactory pmf;
 
    public MockAppEngineEnvironment() {
        env = new ApiProxyEnvironment();
    }
 
    /**
     * Setup the mock environment
     */
    public void setUp() throws Exception {
        // Setup the App Engine services
        ApiProxy.setEnvironmentForCurrentThread(env);
        ApiProxyLocalImpl proxy = new ApiProxyLocalImpl(new File(".")) {};
 
        // Setup the App Engine data store
        proxy.setProperty(LocalDatastoreService.NO_STORAGE_PROPERTY, Boolean.TRUE.toString());
        ApiProxy.setDelegate(proxy);
    }
 
    /**
     * Clean up the mock environment
     */
    public void tearDown() throws Exception {
        // Verify that there's no pending transaction (ie pm.close() has been called)
        Transaction transaction = DatastoreServiceFactory.getDatastoreService().getCurrentTransaction(null);
        boolean transactionPending = transaction != null;
        if (transactionPending) {
            transaction.rollback();
        }
 
        // Clean up the App Engine data store
        ApiProxyLocalImpl proxy = (ApiProxyLocalImpl) ApiProxy.getDelegate();
        if (proxy != null) {
            LocalDatastoreService datastoreService = (LocalDatastoreService) proxy.getService("datastore_v3");
            datastoreService.clearProfiles();
        }
 
        // Clean up the App Engine services
        ApiProxy.setDelegate(null);
        ApiProxy.clearEnvironmentForCurrentThread();
 
        // Report the issue with the transaction still open
        if (transactionPending) {
            throw new IllegalStateException("Found a transaction nor commited neither rolled-back." +
                    "Probably related to a missing PersistenceManager.close() call.");
        }
    }
 
    /**
     * Creates a PersistenceManagerFactory on the fly, with the exact same information
     * stored in the /WEB-INF/META-INF/jdoconfig.xml file.
     */
    public PersistenceManagerFactory getPersistenceManagerFactory() {
        if (pmf == null) {
            Properties newProperties = new Properties();
            newProperties.put("javax.jdo.PersistenceManagerFactoryClass",
                    "org.datanucleus.store.appengine.jdo.DatastoreJDOPersistenceManagerFactory");
            newProperties.put("javax.jdo.option.ConnectionURL", "appengine");
            newProperties.put("javax.jdo.option.NontransactionalRead", "true");
            newProperties.put("javax.jdo.option.NontransactionalWrite", "true");
            newProperties.put("javax.jdo.option.RetainValues", "true");
            newProperties.put("datanucleus.appengine.autoCreateDatastoreTxns", "true");
            newProperties.put("datanucleus.appengine.autoCreateDatastoreTxns", "true");
            pmf = JDOHelper.getPersistenceManagerFactory(newProperties);
        }
        return pmf;
    }
 
    /**
     * Gets an instance of the PersistenceManager class
     */
    public PersistenceManager getPersistenceManager() {
        return getPersistenceManagerFactory().getPersistenceManager();
    }
}

With such a class, the unit test part is easy and I can build complex test cases without worrying about the pertinence of my mock classes! That's really great.

Excerpt of the TestConsumerOperations class

public class TestConsumerOperations {
 
    private MockAppEngineEnvironment mockAppEngineEnvironment;
 
    @Before
    public void setUp() throws Exception {
        mockAppEngineEnvironment = new MockAppEngineEnvironment();
        mockAppEngineEnvironment.setUp();
    }
 
    @After
    public void tearDown() throws Exception {
        mockAppEngineEnvironment.tearDown();
    }
 
    @Test
    public void testCreateVI() throws DataSourceException, UnsupportedEncodingException {
        final String email = "unit@test.net";
        final String name = "Mr Unit Test";
        Consumer newConsumer = new Consumer();
        newConsumer.setDisplayName(name);
        newConsumer.setEmail(email);
        assertNull(newConsumer.getId());
 
        // Verify there's no instance
        Query query = new Query(Consumer.class.getSimpleName());
        assertEquals(0, DatastoreServiceFactory.getDatastoreService().prepare(query).countEntities());
 
        // Create the user once
        ConsumerOperations ops = new ConsumerOperations();
        Consumer createdConsumer = ops.createConsumer(newConsumer);
 
        // Verify there's one instance
        query = new Query(Consumer.class.getSimpleName());
        assertEquals(1, DatastoreServiceFactory.getDatastoreService().prepare(query).countEntities());
 
        assertNotNull(createdConsumer.getId());
        assertEquals(email, createdConsumer.getEmail());
        assertEquals(name, createdConsumer.getName());
    }
    
    // ...
}

Conclusion

As a big fan of TDD, I'm now all set to cover the code of my [still a secret] project efficiently. It does not mean everything is correct, more that everything I thought about is correctly covered. At the time of this writing, just for the server-side logic, the code I produced covers more than 10,000 lines and the unit tests bring an additional set of 23,400 lines.

When it's time to refactor a bit or to add new features (plenty of them are aligned in my task list ;), I feel comfortable because I know I can detect most of regressions (if not all) after 3 minutes of running the test suite.

If you want to follow this example, feel free to get the various mock classes I have added to my two-tiers-utils open-source project. In addition to mock classes for the App Engine environment, you'll find:

• Basic mock classes for the servlet (see javax.servlet) and javamocks.io packages -- I had to adopt the root javamocks because the JVM class loader does not accept the creation on the fly of classes in the java root).
• A mock class for twitter4j.TwitterUser -- I needed a class with public constructor and a easy way to create a default account.
• A series of mock class for David Yu's Project which I use to allow users with OpenID credentials to log in. Read the discussion I had with David on ways to test his code, in fact the code he produced and I customized for my own needs and for other security reasons.

For other details on my library, read my post Internationalization and my two-tiers-utils library.

I hope this helps.

A+, Dom
--
References:

1. Google App Engine: the homepage and the SDK page.
2. See my post on Agile: SCRUM is Hype, but XP is More Important... where I mentionned the following techniques: Continuous Integration (CI), Unit testing and code coverage (CQC), and Continuous refactoring.
3. I know that keeping 100% as the target for code coverage numbers is a bit extreme. I read this article Don't be fooled by the coverage report soon after I started using Cobertura. In addition to reducing the exposition to bugs, the 100% coverage gives a very high chance to detect regressions before pushing the updates to the source control system!
4. Vincent Massol; JUnit in Action; Editions Manning; www.manning.com/massol and Petar Tahchiev, Felipe Leme, Vincent Massol, and Gary Gregory; JUnit in Action, Second Edition; Editions Massol; www.manning.com/tahchiev. I was used to asking any new developer joigning my team to read at least this chapter 7: Testing in isolation with mock objects.
5. JDO stands for Java Data Objects and is an attempt abstract the data storage manipulation. The code is instrumented with Java annotations like @Persistent, it is instrumented at compile time, and dynamically connect to the data source thanks few properties files. Look at the App Engine - Using the Datastore with JDO documentation for more information.
6. For general limitations, check this page Will it play in App Engine. For JDO related limitations, check the bottom of the page Usin JDO with App Engine.