Blog: Integrating TestNG and Appium for mobile app test automation

For mobile healthcare applications with many connected devices, complex database migration projects, and huge enterprise-level desktop multiplatform products, a solid DevOps environment is critical. In this blog, we’ll show how we used the TestNG and Appium frameworks for mobile app test automation using a CI/CD pipeline. We’ll also look at how we used AWS CloudFormation and PowerShell to deploy the test environment as well. The result was a 3.6 times reduction in testing person-hours and a reduction of the full-regression test cycle time by 25%.

Long release cycles crippled innovation

Initially, our client had a cross-platform application and web portals with poor test coverage and no test automation! It took the team of 4 engineers 4 weeks to go through a full regression test cycle, so they couldn’t afford to do it regularly. The client wanted to ship the updates faster and with fewer bugs by using test automation. They also wanted to replace manual work where possible to save both resources and budget.

Test automation with TestNG and the Appium framework

As this project proceeded, the DB Best team looked for ways to improve the following areas: web and mobile development, development for the Bluetooth-connected devices, and incorporation of a DevOps CI/CD pipeline with continuous testing.

The specific area covered in this post centers around concerns the test automation for mobile apps. Since our client offers cross-platform mobile applications with integrated hardware devices, a platform for cross-platform test automation was essential.

Numerous patients use these devices daily. And medical records need to be taken properly and transferred in a timely manner to mobile devices, and subsequently — to doctors.

This diagram represents the test automation logic and sequence of steps:

The flow of test automation

For automation testing of the mobile app, we used TestNG and a Selenium-based Appium framework, which is open-source and allows testing of different platforms like Android, iOS, and Windows.

  • We applied the Appium classes and created additional automation classes using Java to customize the tests for our client libraries and other aspects of the project.
  • We also implemented a continuous testing practice. Now, when we have a couple of new builds ready at the end of the day, the latest build goes through a set of tests that are run automatically during the night. The next morning, our engineer has a report from the testing on his plate.
  • For running automated tests that would involve testing of our hardware and the way it interacts with the software, we had to emulate the use cases for the devices. To enable this, our low-level engineer built similar devices so we could send requests to them. Now, our tests connect to the devices through the Wi-Fi API controller, and we have the emulation of the real-life process.

Automating the test environment deployment using CloudFormation and PowerShell

One of the ways our DevOps team optimized reduced the cycle time involved deploying the test environment to AWS. We used three steps using AWS CloudFormation and PowerShell to automate the deployment process.

  1. Create the CloudFormation Template
  2. Build Stack PowerShell script for creating AWS test resources
  3. Configure Webserver for automated testing

Let’s take a look at the sample code used for the deployment of the test environment. These scripts assume that you have installed the latest version of the AWS.Tools.CloudFormation module of AWS Tools for Powershell. This is as simple as issuing the following PowerShell command.

Install-Module -Name AWS.Tools.CloudFormation

Create the CloudFormation Template

Our team used the AWS CloudFormation Designer to author the initial template. This code example shows a fragment of the complete JSON file generated for our customer’s test automation stack.

{   "AWSTemplateFormatVersion": "2010-09-09",
    "Description": "Creates a load balanced and auto scaled web site running under IIS. It uses a SQL Server database. It deploys the web site code to the auto scaling group, so it can be installed on new EC2 web servers. By updating some of the parameters below, you can create a multi-az deployment - so if the database server fails, it automatically fails over to a hot standby server.",

    "NOTE:": "This is just a snippet of the full CloudFormation template",
    "Highlights": "Parameters used for the Build Stack and Configureation scripts",
    "See Also": "Resources section below for embedded PowerShell script",

    "Parameters": {
        "AppZipName": {
            "Description": "Code version to be deployed. This doubles as the key of the zip file with deployed files in the S3 bucket.",
            "Type": "String"
        "KeyName": {
            "Description": "The EC2 Key Pair to allow RDP access to the instances",
            "Type": "AWS::EC2::KeyPair::KeyName",
            "ConstraintDescription": "must be the name of an existing EC2 KeyPair."
        "AppName": {
            "Description": "Application Name for logs",
            "Type": "String"
        "DBUser": {
            "Description": "Database user name",
            "Type": "String",
            "ConstraintDescription": "must begin with a letter and contain only alphanumeric characters.",
            "Default": "bsAppUser"
        "RoleName": {
            "Description": "Role name with S3 access from EC2",
            "Type": "String",
            "Default": "web-server-role"
        "EnvironmentName": {
            "Description": "Type of environment. For example QA, Prod, DEV",
            "Type": "String",
            "Default": "test-web-server"
    "Resources": {
        "LaunchConfig": {
            "Type": "AWS::AutoScaling::LaunchConfiguration",
            "Properties": {
                "KeyName": {"Ref": "KeyName"},
                "ImageId": {"Ref": "ImageId"},
                "SecurityGroups": [{"Ref": "WebInstanceSecurityGroupId"}],
                "InstanceType": {"Ref": "InstanceType"},
                "IamInstanceProfile": {"Ref": "InstanceProfile"},
                "UserData": {"Fn::Base64": {"Fn::Join": ["",
                    "$ErrorActionPreference = "Continue"\n",
                    "Start-Transcript -path C:\\deploy_log.txt -append\n",
                    {"Fn::Sub": " Read-S3Object -BucketName ${DeploymentBucketName} -Key ${BucketFolderSource}${AppZipName}.zip -File c:\\inetpub\\deploy\\${AppZipName}.zip\n"},
                    " Add-Type -assembly 'System.IO.Compression.FileSystem'\n",
                    " [System.IO.Compression.ZipFile]::ExtractToDirectory('c:\\inetpub\\deploy\",{"Ref": "AppZipName"},".zip', 'c:\\inetpub\\deploy')\n",
                    "$executionPolicy = Get-ExecutionPolicy\n",
                    "Set-ExecutionPolicy Bypass -Scope Process -force\n",
                    "get-childitem c:\\inetpub\\deploy -recurse -force | ?{$ -eq '
deploy.ps1'} | \n",
                    {"Fn::Sub": "  ForEach-Object { Invoke-Expression ($_.FullName + '
''${DbServerAddress}'' ''${DBUser}'' ''${DBUserPassword}'' ''${AppName}'' ''${AppPortalLink}'' ''${ContentDB}'' ''${ConfigDB}'' ') }\n"},
                    "Set-ExecutionPolicy $executionPolicy -Scope Process -force\n",
                    "cfn-signal -e 0 --stack ",{"Ref": "AWS::StackName"}," --resource '
'WebServerGroup'' --region ",{"Ref": "AWS::Region"},"\n",

Build Stack PowerShell script for creating AWS test resources

We used the following script to create all of the AWS resources needed to spin up the EC2 instance used as the test system to validate a specific branch of code.

<# 1) Build Stack — This script receives as arguments a set of parameters,
including access to the DB server, and creates several objects for CloudFormation.
Then it creates a new stack based on the parameters and the template
stored in the S3 bucket. #>

param([string]$sourceZipName ,[string]$AppLink ,[string]$applicationName ,[string]$bucketFilePath
    ,[string]$dbServer ,[string]$dbUserName ,[securestring]$dbPassword ,[string]$instancePrefix
    ,[string]$defaultContentDB ,[string]$defaultConfigDB ,[string]$folderSourcePath
Import-Module -Name AWSPowerShell

# Examples for setting various parameters for QA Automation
# LaunchConfiguration Properties
$KeyName = new-object Amazon.CloudFormation.Model.Parameter -Property @{ParameterKey="KeyName"; ParameterValue="EC2::KeyPair::KeyName"}
$ImageId = new-object Amazon.CloudFormation.Model.Parameter -Property @{ParameterKey="ImageId"; ParameterValue="ami-xxxxxxxxxxxxxxx"}
$WebInstanceSecurityGroupId = new-object Amazon.CloudFormation.Model.Parameter -Property @{ParameterKey="WebInstanceSecurityGroupId"; ParameterValue="sg-xxxxxx"}
$InstanceType = new-object Amazon.CloudFormation.Model.Parameter -Property @{ParameterKey="InstanceType"; ParameterValue="c5.large"}

# LaunchConfig UserData Properties
$DeploymentBucketName = new-object Amazon.CloudFormation.Model.Parameter -Property @{ParameterKey="DeploymentBucketName"; ParameterValue="$bucketFilePath"}
$BucketFolderSource = new-object Amazon.CloudFormation.Model.Parameter -Property @{ParameterKey="BucketFolderSource"; ParameterValue="$folderSourcePath"}
$AppZipName = new-object Amazon.CloudFormation.Model.Parameter -Property @{ParameterKey="AppZipName"; ParameterValue="$sourceZipName"}
$DbServerAddress = new-object Amazon.CloudFormation.Model.Parameter -Property @{ParameterKey="DbServerAddress"; ParameterValue="$dbServer"}
$DBUser = new-object Amazon.CloudFormation.Model.Parameter -Property @{ParameterKey="DBUser"; ParameterValue="$dbUserName"}
$DBUserPassword = new-object Amazon.CloudFormation.Model.Parameter -Property @{ParameterKey="DBUserPassword"; ParameterValue="$dbPassword"}
$AppName = new-object Amazon.CloudFormation.Model.Parameter -Property @{ParameterKey="AppName"; ParameterValue="$applicationName"}
$AppPortalLink = new-object Amazon.CloudFormation.Model.Parameter -Property @{ParameterKey="AppPortalLink"; ParameterValue="$AppLink"}
$ContentDB = new-object Amazon.CloudFormation.Model.Parameter -Property @{ParameterKey="ContentDB"; ParameterValue="$defaultContentDB"}
$ConfigDB = new-object Amazon.CloudFormation.Model.Parameter -Property @{ParameterKey="ConfigDB"; ParameterValue="$defaultConfigDB"}
$EnvironmentName = new-object Amazon.CloudFormation.Model.Parameter -Property @{ParameterKey="EnvironmentName"; ParameterValue="$instancePrefix"}
$RoleName = new-object Amazon.CloudFormation.Model.Parameter -Property @{ParameterKey="RoleName"; ParameterValue="web-server-role"}

# Create a PowerShell object with the Key Value pairs of properties
$Stack_Parameters = @($AppZipName, $KeyName, $AppName, $AppPortalLink, $DbServerAddress, $DBUser, $DBUserPassword, $DeploymentBucketName, $InstanceType, $ImageId, $RoleName, $WebInstanceSecurityGroupId, $EnvironmentName, $ContentDB, $ConfigDB, $BucketFolderSource)

$CFNStackParams = @{
Stackname = "$applicationName-web-servers-stack"
TemplateURL = ""
Parameters = $Stack_Parameters
Region = $setRegion
Capability = "CAPABILITY_IAM"
New-CFNStack @CFNStackParams

Configure Webserver for automated testing

The third code example in the process, takes the parameters for the test automation run and configures the web server.

<# 2) Configuration — Relies on the Build Stack script to pass in parameters
from the CloudFormation template. Other user properties like the database
connection parameters are transferred to this script. These parameters, are  
used for the web.config file to configure web applications (the entity is on the
web server) and replaces the placeholders with the correct values.
Then it restarts all the services. #>

# Define the database connection and other web site parameters for script.
  [Parameter(Mandatory=$False, HelpMessage="Name of the database server")] [string]$dbServer,
  [Parameter(Mandatory=$False, HelpMessage="User name to be used when accessing the database")] [string]$dbUsername,
  [Parameter(Mandatory=$False, HelpMessage="Password to be used when accessing the database")] [securestring]$dbPassword,
  [Parameter(Mandatory=$False, HelpMessage="Application name")] [string]$appName,
  [Parameter(Mandatory=$False, HelpMessage="App portal link")] [string]$caregiverPortalLink,
  [Parameter(Mandatory=$False, HelpMessage="Default content db name")] [string]$contentDB,
  [Parameter(Mandatory=$False, HelpMessage="Default config db name")][string]$configDB
Function get-website-physicalpath([string]$siteName)
    $webSitePhysicalPath = (cmd /c %systemroot%\system32\inetsrv\APPCMD list vdirs "$siteName/" /text:physicalPath) | Out-String
    Return [System.Environment]::ExpandEnvironmentVariables($webSitePhysicalPath).Trim()
Add-Type -Path "C:\Program Files (x86)\AWS SDK for .NET\bin\Net45\AWSSDK.EC2.dll"

# Find the directory that this script is running in
$scriptpath = $MyInvocation.MyCommand.Path
$scriptDir = Split-Path $scriptpath

<# Replace placeholders in the web.config file. Both this script file and
web.config are assumed to sit in the root directory of the web site.
Note: IIS web server requires UTF8 encoding for the web.config #>

Function switch-config-prompts([string]$configFilePath)
    (Get-Content $configFilePath) | Foreach-Object {$_ `
        -replace '{{DbServer}}', $dbServer `
        -replace '{{ContentDB}}', $contentDB `
        -replace '{{ConfigDB}}', $configDB `
        -replace '{{DbUsername}}', $dbUsername `
        -replace '{{DbPassword}}', $dbPassword `
        -replace '{{AppName}}', $appName `
        -replace '{{AppPortalLink}}', $AppPortalLink `

    } | Out-File $configFilePath -encoding UTF8

$projects = Get-ChildItem $scriptDir -Directory
$projects | ForEach-Object { if (Test-Path $scriptDir\$_\web.config) { switch-config-prompts("$scriptDir\$_\web.config")}}

Import-Module WebAdministration

# Get pool name by the site name:
$adultApiPool = (Get-Item "IIS:\Sites\Applications"| Select-Object applicationPool).applicationPool

# Deploy the files to the root directory of the API
$physicalPath = get-website-physicalpath("Applications/api")
Copy-Item "$scriptDir\WebAPI\*" -Destination $physicalPath -Recurse -Force
$physicalPath = get-website-physicalpath("Applications/admin")
Copy-Item "$scriptDir\InstitutionPortal\*" -Destination $physicalPath -Recurse -Force
$physicalPath = get-website-physicalpath("Applications/provider")
Copy-Item "$scriptDir\WebProvidersBackEnd\*" -Destination $physicalPath -Recurse -Force
$physicalPath = get-website-physicalpath("Applications/app-portal")
Copy-Item "$scriptDir\App-Portal\*" -Destination $physicalPath -Recurse -Force
Copy-Item "$scriptDir\NotificationAssembly\*" -Destination "C:\Workspace\Assembly" -Recurse -Force

Remove-Item C:\inetpub\wwwroot\web.config -Force

# Start the web site again
Restart-WebAppPool $adultApiPool

What the client achieved with automation testing

The benefits of test automation
With the deployment of test automation, our client’s solution became much more stable and reliable, which is crucial for the healthcare industry. We managed to eliminate the human factor when possible, push updates with minimum bugs, and generally improve the testing process to meet the latest industry standards.

Even though we’re still working on automating more processes in this project, our customer already enjoys the following benefits from test automation:

  • Reduced the full regression test cycle from 4 weeks to 3 weeks
  • Decreased testing costs for person-hours from 640 to 180
  • Increased test coverage: 96 % of code is covered by tests
  • Stabilized releases that take place more often
  • Created a modern testing process that meets the latest industry requirements

Go modern with DB Best!

To ensure decent quality for your products, optimize processes, and reach the most efficient resource allocation, we came up with an optimal quality assurance strategy for complex mobile solutions. Our approach includes test automation and continuous testing. So, no matter how challenging the task is, don’t hesitate to turn to us for help. We will be glad to build software of superior quality and performance for you. Let’s chat to start optimizing your project today!

Share this...
Share on Facebook
Tweet about this on Twitter
Share on LinkedIn