Android, Robotium and Spoon: A Practical Guide for Testing on Android Devices

Developing for Android is a lot of fun. It has actually become a niche for me here at SEP. Android has changed quite a bit over the past 3 years since I first started hacking on it – action bars, more screen sizes, and probably the biggest addition…fragments.

But one thing certainly has not changed. Testing on real devices is not easy.

I’m not going to go into detail about the importance of unit testing and functional testing. I am assuming that since you are reading this, you already know that both types of testing are important.

Instead, this is a practical guide showing how my teammates and I are using Robotium and Spoon to perform functional testing on real Android devices.

Robotium

Robotium is a testing framework that works with Android’s “Instrumentation” tests.

The biggest benefit of using a tool like Robotium is that you can write tests as if you were a user (with very little knowledge about the implementation).
For example, your tests might look like the following:

At first pass, this looks terrible. I can still hear myself saying, out loud, “why do I have to use these awful indexes instead of IDs!?!” Honestly, though, that is the secret sauce.
The above example simply looks for the first text box, and types in the string “user-name”…same for the password. Then Robotium looks for the first button, and triggers the “performClick()” action on that button.

There are many other great benefits from using Robotium. It takes care of a lot of the gross/hard work…like how long do I have to wait before I assert that a new Activity started? Or, how can I assert that the text is actually visible to the user?

Setup is super simple. Include the “robotium-solo-*.jar” and create a new instance of “Solo” for use in your testing project.

Now that you are writing functional tests, you need a way to get feedback about how those functional tests behave on a multitude of devices.
Meet Spoon.

Spoon

The people at Square released a wonderful tool called Spoon.

Spoon itself is a separate testrunner which can be run from the command line. Spoon then runs your functional tests on all of the devices connected to the computer (including emulators running). Spoon will then record the results, save off the logcat logs, and take screenshots of the app during testing.

Tips

Both Robotium and Spoon have greatly improved our functional testing. We have a stable setup that the entire team has grown to trust in.

We did have some issues come up as we went.  Below are some additional tips that we have learned during our adventure.

• make sure you follow the OEM USB Drivers documentation in order to enable USB Debugging on all devices and platforms
• take the Spoon screenshot, right before you assert – this helps so that you can see the screen right before the assert…if the test fails, you still can see what potentially went wrong.

• use “waitForIdleSync()” to ensure the views are fully loaded

• disable the lock screen (Settings -> Security -> Screen lock -> None) – by disabling the lock screen, you don’t need to include the extra permissions for Spoon to unlock the device
• dim the brightness, to protect from draining the batteries while plugged into a USB port (USB is typically a slow charger)
• enable the “Stay Awake” setting to keep the screen turned on (Settings -> Developer options -> Stay awake) – Alternatively, use the “KeepScreenOn” app

There is no doubt that functional testing is important.  Robotium and Spoon have both helped us to write better functional tests.

Next up…getting Monkey and MonkeyRunner to play well with our application.