The Android development tools project has seen big changes over the last year. The original Eclipse ADT development environment was superseded late last year by Android Studio — a new IDE based on Intellij. Under the hood Android Studio also uses a new command line build system based on Gradle, replacing the previous Ant-based system. I’ve been keen to find out how these changes impact the integration of Android test reports with continuous integration servers like Pulse.
Summary
- Android JUnit Report is redundant.
- Run on-device Android tests with:
./gradlew connectedAndroidTest
- Collect reports from:
app/build/outputs/androidTest-results/connected/*.xml
Details
The original Ant-based build system for Android didn’t produce XML test reports for instrumentation tests (i.e. those that run on-device), prompting me to create the Android JUnit Report project. Android JUnit Report produced XML output similar to the Ant JUnit task, making it compatible with most continuous integration servers. The good news is: Android JUnit Report is now redundant. The new Gradle-based build system produces sane XML test reports out of the box. In fact, they’re even more complete than those produced by Android JUnit Report, so should work with even more continuous integration servers.
The only downside is the documentation, which is a little confusing (while there are still documents for the old system about) and not very detailed. With a bit of experimentation and poking around I found how to run on-device (or emulator) tests and where the XML reports were stored. With a default project layout as created by Android Studio:
ASDemo.iml
app/
app.iml
build.gradle
libs/
proguard-rules.pro
src/
androidTest/
main/
build.gradle
gradle
gradle.properties
gradlew
gradlew.bat
local.properties
settings.gradle
You get a built-in version of Gradle to use for building your project, launched via gradlew. To see available tasks, run:
$ ./gradlew tasks
(This will download a bunch of dependencies when first run.) Amongst plenty of output, take a look at the Verification Tasks section:
Verification tasks ------------------ check - Runs all checks. connectedAndroidTest - Installs and runs the tests for Debug build on connected devices. connectedCheck - Runs all device checks on currently connected devices. deviceCheck - Runs all device checks using Device Providers and Test Servers. lint - Runs lint on all variants. lintDebug - Runs lint on the Debug build. lintRelease - Runs lint on the Release build. test - Run all unit tests. testDebug - Run unit tests for the Debug build. testRelease - Run unit tests for the Release build.
The main testing target test does not run on-device tests, only unit tests that run locally. For on-device tests you use the connectedAndroidTest task. Try it:
$ ./gradlew connectedAndroidTest ... :app:compileDebugAndroidTestJava :app:preDexDebugAndroidTest :app:dexDebugAndroidTest :app:processDebugAndroidTestJavaRes UP-TO-DATE :app:packageDebugAndroidTest :app:assembleDebugAndroidTest :app:connectedAndroidTest :app:connectedCheck BUILD SUCCESSFUL Total time: 33.372 secs
It’s not obvious, but this produces compatible XML reports under:
app/build/outputs/androidTest-results/connected
with names based on the application module and device. In your continuous integration setup you can just collect all *.xml files in this directory for reporting.
Although the new build system has killed the need for my little Android JUnit Report project, this is a welcome development. Now all Android developers get better test reporting without an external dependency. Perhaps it will even encourage a few more people to use continuous integration servers like Pulse to keep close tabs on their tests!