About Me

My Photo
A Sun certified Java professional with time proven experience in architecture and designing of mid/large scale Java/JEE based applications. Creator of the EasyTest Framework.A lot of experience with technologies such as Spring framework, Hibernate , JPA, Java4-6, REST, SOA , YUI , JUnit , Cloud Foundry PaaS and other technologies.

Monday, August 13, 2012

JUnit Theories and ParametersSuppliedBy Annotation


Its been a long journey since we started discussing JUnit and its use for data driven testing. We discussed the long standing traditional testing , we discussed parameters based testing and we also discussed JUnit's Theories based data driven testing . We saw the pros and cons of each of the testing methodology and went through a complete example to prove our point.

Next in line for achieving our ultimate goal of Data Driven Testing with JUnit is the annotation @ParametersSuppliedBy . This is the annotation that gives us the power on defining how the List<PotentialAssignment> gets created for a given set of input parameters. Just a quick thought on PotentialAssignment. PotentialAssignment is an abstract class that JUnit Theories uses to wrap our test data and provide that test data to our test methods in a consistent manner. It has a static method forValue that you can use to get an instance of PotentialAssignment.

Right, so lets see how we can write data driven test cases using @ParametersSuppliedBy annotation.

We will continue to use the same example as we used in previous posts:

public interface ItemService {

    public List getItems(@NotNull Id libraryId , String searchText , String itemType);

    public Item findItem(@NotNull Id libraryId , @NotNull Id itemId);
}

Lets also revise what our Theory to test was:

Given that the test data to run my tests against already exists in my system, I would like to test the method getItems with a defined set of test data to run my test against. For eg. I should be able to verify positively that  a search for an ebook with searchText="potter" in Library 1 will always return me an Item. I know this assumption should hold true, because I know the data is present in the persistent storage for the given set of input parameters.
Thus one of my tests could be : I want to confirm that there exists ebooks in library 1 for all the search texts : "potter" , "poppins" and "superman".
Another one could be : I want to confirm that there exists books in library 2 for all the search texts :"potter" , "spiderman" and "batman".
Yet another one could be :  I want to confirm that there exists journals in library 3 for all the search texts :"java" , "junit" , "nasa". And so on.


Argument: We can argue that Theories is not meant for this type of testing since the original specifications of Theories called it as "specifications that catch bugs". And that we should be using Parameterized Runner for this. But I will keep this discussion for later. Also I will show how Theories can be used to do such type of testing.

So lets see how we can write a test case for the above theory using @ParametersSuppliedBy annotation:


@RunWith(Theories.class)
public class ItemServiceTheoriesTest {

private ItemService testSubject;

    @Before
    public void setUp(){
        testSubject = new RealItemService();
    }

    public static class LibraryIdSupplier extends ParameterSupplier {
        @Override
        public List getValueSources(ParameterSignature sig) {
            List list = new ArrayList();
            list.add(PotentialAssignment.forValue("", new  LibraryId(1L)));
            return list;
        }
   
    }

    public static class ItemTypeSupplier extends ParameterSupplier {
        @Override
        public List getValueSources(ParameterSignature sig) {
            List list = new ArrayList();
            list.add(PotentialAssignment.forValue("", "ebook"));
            return list;
        }
   
    }

    public static class SearchTextSupplier extends ParameterSupplier {
        @Override
        public List getValueSources(ParameterSignature sig) {
            List list = new ArrayList();
            list.add(PotentialAssignment.forValue("", new String[]{"potter" , "poppins" , "superman"}));
            return list;
        }
   
    }

    @Theory
    public void testGetItems(@ParametersSuppliedBy( LibraryIdSupplier.class) LibraryId libraryId ,
        @ParametersSuppliedBy(ItemTypeSupplier.class)String itemType ,
        @ParametersSuppliedBy(SearchTextSupplier.class)String[] searchText ) {
        for(String searchTextData : searchText){
            List items = testSubject.getItems(libraryId, searchTextData, itemType);
            Assert.assertNotNull(items);
        }        
There are two new things in the above example:
  • @ParametersSuppliedBy which is an annotation applied on the parameter to the test to let the Theories Runner know that the parameter needs to be supplied to this test method.
  • Classes  InstitutionIdSupplier ,  ItemTypeSupplier ,  SearchTextSupplier which all extends  ParameterSupplier. parameterSupplier is an abstract class and we can extend it to provide the test data in a consistent manner to our test method. Also we get a lot more control on the test Data than with using @DataPoint(s).
In the above example we are telling JUnit to fetch the values for libraryId from the class LibraryIdSupplier, the values for itemType from ItemTypeSupplier and the values for searchText from SearchTextSupplier . 
When the above test is run the method getItems will be called 3 times with the following values :


library Id : 1 and item type : ebook and search text array :potter
library Id : 1 and item type : ebook and search text array :poppins
library Id : 1 and item type : ebook and search text array :superman
This is really kewl(cool in normal english :)). I now control how my data is passed to my test case more precisely. I do not have to worry that wrong values will be passed to my test method.

Next, lets extend our test case to test that Library 1 has books(not ebooks) with searchText as="spiderman".
In the perfect world, I would expect to add the test data to my parameter supplier and hope it to work seamlessly. Thus I do the following :

public static class ItemTypeSupplier extends ParameterSupplier {

        @Override
        public List getValueSources(ParameterSignature sig) {
            List list = new ArrayList();
            list.add(PotentialAssignment.forValue("", "ebook"));
            list.add(PotentialAssignment.forValue("", "book"));
            return list;
        }
   
    }

    public static class SearchTextSupplier extends ParameterSupplier {
        @Override
        public List getValueSources(ParameterSignature sig) {
            List list = new ArrayList();
            list.add(PotentialAssignment.forValue("", new String[]{"potter" , "poppins" , "superman"}));
            list.add(PotentialAssignment.forValue("", new String[]{"spiderman"}));
            return list;
        }
   
    } 
And when i run my test case, it is run 8 times with the following input values :


library Id : 1 and item type : ebook and search text array :potter
library Id : 1 and item type : ebook and search text array :poppins
library Id : 1 and item type : ebook and search text array :superman
library Id : 1 and item type : ebook and search text array :spiderman
library Id : 1 and item type : book and search text array :potter
library Id : 1 and item type : book and search text array :poppins
library Id : 1 and item type : book and search text array :superman
library Id : 1 and item type : book and search text array :spiderman

So whats exactly happening. Theories Runner is recursively calling the test method for each and every permutation of the input test data. Whether this is good or bad is not for me to decide. All I can say is that Theories Runner cannot understand or assume what a user wants and how the user wants its data to be supplied to the test methods. Therefore it calls the test method for each and every permutation of the test data.

there are many ways around this problem. The simplest one(and also not the cleanest one in my opinion) is to pass a single parameter that acts as a holder of our input test data against which our test needs to be run.
So something like this :


@RunWith(Theories.class)
public class ItemServiceTheoriesTest {

private ItemService testSubject;

    @Before
    public void setUp(){
        testSubject = new RealItemService();
    }

public static class GetItemsDataSupplier extends ParameterSupplier {
        @Override
        public List getValueSources(ParameterSignature sig) {
            List list = new ArrayList();
            HashMap inputData = new HashMap();
            inputData.put("LibraryId", new LibraryId(1L));
            inputData.put("itemType", "ebook");
            inputData.put("searchText", new String[]{"potter" , "poppins" , "superman"});
            list.add(PotentialAssignment.forValue("", inputData));
            HashMap inputData1 = new HashMap();
            inputData1.put("LibraryId", new LibraryId(1L));
            inputData1.put("itemType", "book");
            inputData1.put("searchText", new String[]{"spiderman"});
            list.add(PotentialAssignment.forValue("", inputData1));
            return list;
        }
    }

    @Theory
    public void testGetItems(@ParametersSuppliedBy(GetItemsDataSupplier.class)HashMap inputData) {
        //Its is not a good practice to explicitly cast but is safe here since we know the return type
        LibraryId libraryId = (LibraryId)inputData.get("LibraryId");
        String itemType = (String)inputData.get("itemType");
        String[] searchText = (String[])inputData.get("searchText");
        for(String searchTextData : searchText){
            List items = testSubject.getItems(libraryId, searchTextData, itemType);
            Assert.assertNotNull(items);
        }
    }
    }
}

What we have done is instead of defining DataSupplier for each input parameter, we have supplied the test data as a single data structure for the method. The above test, when run, will run 4 times with the following input parameters :


library Id : 1 and item type : ebook and search text array :potter
library Id : 1 and item type : ebook and search text array :poppins
library Id : 1 and item type : ebook and search text array :superman
library Id : 1 and item type : book and search text array :spiderman

This is more like what we had desired for. And it is easy to extend this for any set of test data.
Side Note 1:  I have kept GetItemsDataSupplier class in the Test class itself. It is only for example purpose and in the real world should be moved outside of the test class.
Side Note 2: By making use of @ParametersSuppliedBy annotation and extending the  ParameterSupplier   class, we have now more control over our test method, its data and also its behavior. We have also managed to move the data out of the test class, although it still has to be defined as part of a Java class and everytime new data needs to be added, we have to change the java class. We will see in the next post how we can overcome this problem by using CSV to load the input test data.

One of the drawbacks that I consider of defining a generic parameter supplier that returns a generic data structure is that we have to write extra lines of code in our test method which in my opinion should be a configuration option. but that's just me. :)
Another potential drawback is that for each test method, you have to define a new Class extending  ParameterSupplier class. In my opinion, if we could somehow get the facility to generically load the input test data, then it would be wonderful. As we will see in the next post, it is achievable. So stay tuned or become a member to get regular updates.


SideNote: You can find the code base that improves JUnit's support for Data Driven Testing here : https://github.com/EaseTech/easytest-core

I am looking for feedback to further improve the project.

2 comments:

Anonymous said...

Nice work.

Anonymous said...

... we are legion.