Cucumber based automation for android

Automating android application using robotium is simple but it is really missing cucumber feature test. LessPainful open sourced their functional testing tool for Android and iOS mobile apps based on cucumber tool, which is called Calabash. There are separate repository for android and iPhone.

Installation of calabash-android is very simple and explained here. Once you installed calabash-android, modify build.properties file and add details of your android application (Make sure that the application has INTERNET permission).

I have used the sample notepad application from android SDK and changed the build.properties file like below.

tested.package_name=com.example.android.notepad
tested.main_activity=com.example.android.notepad.NotesList
tested.project.apk=NotePad.apk

android.api.level=15 # using Android 4

key.store=${user.home}/.android/debug.keystore
key.store.password=android
key.alias=androiddebugkey
key.alias.password=android

The cucumber feature files need to be added into features directory. If you are following imperative style coding in cucumber (each step level) then most of the basic steps are already presented in the features/step_definitionsdirectory. You can add custom steps in the same directory in order to achieve declarative style steps. I have added both type of tests (refer the below screenshot).

Run android emulator and run your cucumber test using ant clean test. The test result will be available in the console. It is also possible to run cucumber command, instead of ant. If you are running cucumber command there is no need to update the build.properties file. See the below command for running cucumber directly.

PACKAGE_NAME=com.example.android.notepad TEST_PACKAGE_NAME=com.example.android.notepad.test APP_PATH=NotePad.apk TEST_APP_PATH=bin/Test.apk TEST_SERVER_PORT=34777 cucumber

This is the html result generated by cucumber.

You can also run your Calabash tests on multiple devices using LessPainful.com.

 

Update: The latest version of calabash-android setup is really simple.

1. Install the gem : gem install calabash-android
2. Run calabash-android gen : to generate feature directory and support filess, you can run this command from any directory( application source code  is not  required)
3. Run calabash-android run <<apkfile_path>>

Redcukes – running redmine stories using cucumber

I have seen projects where userstories are managed in redmine along with acceptance criteria. But while automating with cucumber all these stories and scenarios are again replicated in cucumber feature files.This leads to inconsistency due to the story modification in redmine which is not synced with cucumber feature files.

To avoid this problem I have developed a ruby gem called Redcukes for integrating redmine with cucumber and we can run cucumber against the user stories stored in redmine and update execution status back to redmine.

For using this gem, make sure that the REST API is enabled in redmine (Administration > Settings -> Authentication -> Enable REST web service). It is better to define another user role for cucumber tests with required issue modification rights.

Install gem using gem install redcukes. In cucumber projects directory, create env.rb under support directory.

Edit env.rb file and add following configuration

require 'redcukes'

Redcukes::Redmine.configure do |config|
 config.site = redmine url
 config.user = redmine username
 config.password = redmine password
 end

It is also possible to add issue filter using search_filter property

e.g : config.search_filter = {:project_id => ‘iframe’,:tracker_id => 2}

The default status updates are “Resolved” or “Rejected” according to feature passed or failed. This can be customized like below

config.result_status = {:passed => 7,:failed => 8}

I have created two more issue status called Passed, Failed and gave given these status ids in the above configuration.

This will be a perfect executable documentation for your project.

Android robotium tests with Jenkins

Robotium is a test framework for automating android native application. This post describes how to integrate the robotium tests with the most popular CI called Jenkins. This is on the assumption that you already have an android project and it’s test project uses Robotium.

The first step is to create a build file for both main project and test project. If you have created the project using android command line option then the build file already gets created, so you can skip this step.

Go to the directory of the main android project and run the following command ( Android SDK/tools directory should be in system path)

android update project -p .

The build file might be created in the project directory. You can run ‘ant debug’ to check whether the project build was successful or not. Now we need to create the build file for the test project by running the command which is given below.

android update test-project -p . -m MAIN_PROJECT_PATH

Here we have to give the main project’s path in the command line. You can change the main project path in ant.properties file which has been created by the above command.

To verify if the build script is working, start the emulator and run the following command from test project directory

ant debug install test

This command will build the project, install the application and instrumentation apk files in emulator and then run the tests.

Before integrating the project in Jenkins, we need to install “Android Emulator Plugin” , this will help us to run android emulator in the build server as a pre-build action. Specify the andoid sdk path (if it exist in the CI server ) in Jenkins → Manage Jenkins → Configure System , otherwise the plugin will automatically download the SDK from Internet.

Now create a new Job(free-style) in jenkins. While configuring SCM specify the root directory which contains both main and test project

Under build environment section, select the option to Run android emulator during  build option. It is possible to select an existing emulator or create a new one with custom properties .Make sure that “Use emulator snapshots” option has been selected which will help to start emulator faster. In build section select “Invoke Ant” option and specify the following targets “clean debug install test” ,also in the advanced section specify the build file from the test project. We are now  done with the basic configuration. Let’s build the project in jenkins and check the build is successful.

One more task which is missing in Jenkins is to report junit test results. Since the tests are running inside emulator we don’t have access to the junit test result.

To generate standard junit xml report in emulator we have a custom junit runner available called android-junit-report. Steps for adding this junit runner in our project is available in the README file’s section “USING WITH ANT BUILDS”

Once you are done with these steps, append  “fetch-test-report” target in the jenkins build configuration. Also enable “Publish JUnit test result report” under Post-build Actions. Now we are done with robotium jenkins integration

Redmine tracker control plugin released

For the last couple of months I have started using redmine project management tool for my official work . The tool has most of the features including issue tracker, built-in wiki, forum and can even integrate SCM. Redmine is developed in ruby on rails platform and supports plug-ins which helps to add new features to the tool.

One of the feature which my team was looking at is to control the tracker wise issue creation, for example developers should not be able to create new requirement(tracker) issue. Redmine has access permission only in the issue creation level but not in tracker wise.

To add the above functionality to redmine, I have created a new redmine plugin called Redmine Tracker Control

The plugin supports following features,

• Role based permission for creating issue with specific Tracker
• Project Module to enable/disable tracker permission
• Projects which are not enabled for tracker control module, uses default redmine behavior

Installation

• Copy redmine_track_control directory to #{RAILS_ROOT}/vendor/plugins
• Restart Redmine
• Goto Redmine -> Administration -> Roles and Permissions  -> Permissions report
• Configure the permissions in Tracker Permissions section

To enable tracker permissions to project, Go to the project -> Settings -> Modules and enable Tracker Permissions Module

You can download plugin from Github

PageObjects and PageFactory

PageObject pattern is familiar to everyone who is using webdriver. It makes test code less noisy and maintainable. Consider the first example in the PageObjects documentation. There are multiple actions(like loginAs, loginAsExpectingError) sharing the same code. Here the main thing to keep in mind is to avoid code duplication (findElement) for the same element in multiple methods which violates DRY Principle and also when there is a UI change, need to update the element property in multiple methods.

When a page has more actions and elements, class will become more verbose with all findElements and also its pretty hard to check all the elements in the page. This is where PageFactory is helping us. PageFactory solves this issue by declaring the element with property as annotation and the factory creates the object when required.

See the pagefactory example below.

public class LoginPage {

 private final WebDriver driver;

 @FindBy(id = "username") private WebElement _username;
 @FindBy(id = "password") private WebElement _password;
 @FindBy(id = "login") private WebElement _login;
 @FindBy(id = "error-msg") private WebElement _error;

 public LoginPage(WebDriver driver) { 
    this.driver = driver;
    if (!"Login".equals(driver.getTitle())) {
       throw new IllegalStateException("This is not the login page");
    }
 }

public HomePage loginAs(String username, String password) {
  login(username, password);
  return new HomePage(driver);
 }

public LoginPage loginAsExpectingError(String username, String password) {
  login(username, password);
  return new LoginPage(driver);
 }

public String getErrorMessage() {
 return _error.getText();
 }

private void login(String username, String password) {
 _username.sendKeys(username);
 _password.sendKeys(password);
 _login.submit();
 }

}

PageObject which initializes all elements in the constructor has one implicit advantage over PageFactory. Your test will fail if any one of the element is missing in the page. But in the case of PageFactory, web elements are evaluated lazily. So test will pass even if all the elements are not present in the page (will check only for the elements required for that particular test).

JUnit – Readable Test Names

Java camel-case function names are readable if it has only two or three words in it. Test names are little longer for specifying the context and what it’s testing. This is anyway better than adding a detailed comment for the test which will get outdated pretty soon.

The problem comes when reading the jUnit test report which contains big names. It is hard to read such test names. Here are some of the tests from the famous webdriver project:

These are pretty long names, Isn’t it!

One option to make it readable is to use snake-case

eg. test_should_return_null_when_getting_the_value_of_an_attribute_that_is_not_listed.

But this will be a violation towards java coding style, Also it is difficult to follow two standards in TDD(snake-case for tests and camel-case for normal functions).

Another option is to make a custom runner in jUnit which converts camel-case tests names to readable text (humanize) . Here is the code

public class ReadableTest extends BlockJUnit4ClassRunner {
@Override
 protected String testName(FrameworkMethod method) {
 return StringUtil.humanize(method.getMethod().getName());
 }

public ReadableTest(Class<?> klass) throws InitializationError {
 super(klass);
 }

}

Annotate the jUnit Test class with this runner eg: @RunWith(ReadableTest.class) and run your tests.

That’s it. See the following screenshot

The same humanized test names can be seen in Jenkins also.

Introduction

This blog will be my reference to my day to day learning, findings on programming, technologies…