One of the most frustrating things about mobile app development is finding a nasty problem in a just-released version and being forced to pull it off the market. It is this frustration that drove me to explore automated testing, and specifically unit testing for the Android platform.
The goal of automated testing in general, and unit testing in particular, is to let the machines do the heavy lifting by testing for problems in code automatically. Unit testing is focused on testing isolated “units” of codes, without requiring the entire application to be running. This is great for testing complex logic, especially in cases where a developer isn’t 100% confident about some part of their code. In this post, I would like to outline an introduction to unit testing on the Android platform, without requiring a physical device or emulator to test on.In Part I, I will discuss setting up the testing environment and in part 2, I will discuss the actual testing code.
Device Testing vs. Non-Device Testing
Before diving into the technical details, I would like to say a few words about on-device testing. My impression from reading the official Android testing guide seems to be that Google is focusing heavily on tests running on a physical device or an emulator. They do have a small section on local unit tests but those seem to be secondary in importance (at least the way I am reading it). There are also tools to facilitate on-device testing like Calabash and Google’s Cloud Test Lab. I prefer non-device tests instead.
My preference is for local unit tests for several reasons:
1. They are faster since they run in memory and do not need a physical device.
2. Less integration issues with having to connect to a physical device.
3. Can be run anywhere such as a server or a developer’s laptop.
4. Can be run in parallel.
5. Cheaper since they do not need a physical device.
Now with this being said, unit tests are really great for testing logic, but not so great for testing the actual UI. There are certainly cases where on-device testing is preferable , especially where compatibility issues between devices are being tested.
Laying the Foundation
Being that Android programming is done primarily in Java, the unit testing tools that used in Java are also useful for Android. Some common tools used are JUnit and many other libraries built to work with it. I settled on a combination of JUnit4 for core testing, AssertJ for making assertions easier and Mockito for mocking..
To start, you would configure your Gradle file as outlined below. Note that these are “testCompile”, not “compile” dependencies which indicates to Gradle that these will only be used during testing. Also note that I am using AssertJ v1 because that is the only version compatible with Android.
Android Testing Support Library Side Effects
Before discussing the actual mechanics of writing unit tests, it would be wise to mention an important point regarding the Android testing support library, which is included with the Android plugin for Gradle. This library will always be included when you run your tests and it stubs all of Android’s classes. The problem is that it does not actually supply any running code, instead every time you try to access an Android system class that is mocked, you get the following error:
Google provides the following explanation:
The android.jar file that is used to run unit tests does not contain any actual code – that is provided by the Android system image on real devices. Instead, all methods throw exceptions (by default). This is to make sure your unit tests only test your code and do not depend on any particular behaviour of the Android platform (that you have not explicitly mocked e.g. using Mockito).
Practically speaking, that means that you MUST mock any Android method you try to access in your code as I will outline later on. HOWEVER, there is another nasty side effect: the testing support library also includes some non-Android code such as org.json.* classes, resulting in errors whenever you try to do anything with JSON objects. A bug report has been filed with Google for this.
If you are using JSON classes, the workaround, as described on StackOverflow, is to include the JSON jar file manually in your gradle file as follows (Maven won’t work for plugin v1.1.0 and earlier):
If you are using the Gradle Android plugin v1.2.3 or later, then you can use Maven instead:
Referencing resources in the R file
Another common issue that comes up is referencing items in the R file which contains the resources used by the application (see also this overview of what the R file is). Because this file is automatically generated during the build process, it is not available to the testing code. The workaround, as described on Stack Overflow, is to include the directories where the R file is located manually. This approach requires running “assemble” first. NOTE, that including the R file does not actually give you access to the strings or other resources the R file references.
Putting It All Together
A complete example in Gradle is below:
To run the tests for the first time, do one of the following:
To run the tests again, do one of the following:
The results of the tests will show up here (relative to project path):
Example report below: