Native Application Monitoring with Catchpoint and Appium

Smartphones contribute to the major share as a preferred device to access content over the internet. Nowadays, applications are developed using responsive design that renders seamlessly on any device irrespective of the screen width and resolution.

With the increase in mobile users, there is a boost in demand for native applications. Native applications are mobile device-specific, several such native apps are used for tasks such as sending messages, sharing media, ordering food, booking a cab, or preparing a to-do list. These apps have become integral to mobile end user experience and so has maintaining application performance. It is necessary to monitor these apps proactively for performance issues.

Catchpoint offers different monitors to suit different monitoring requirements. Performance monitoring of browser-based mobile web applications can be done using synthetic monitoring while native applications require a slightly different approach using custom monitors. In this blog, we explain how you can use Catchpoint to monitor native applications.

Measuring Native Application Performance

One of the major differences between web and native application is that they depend a lot on underlying frameworks provided by the operating system. It can be one of the many available operating systems like IOS, Android and so on, rather than the traditional client-server architecture. Even with these differences, native applications still depend on remote servers to download content and should be considered when analysing application performance.

Below are a few other factors that contribute to performance:

1. App start-up/launch time.
2. Battery or power status of a device.
3. Memory used by the app.
4. App crashes and bugs

With our new Custom Monitor, we enable monitoring of native applications for Android devices.

Understanding the Technologies Used

To run these test cases on an Android device and record performance metrics, it relies on a few open source frameworks and libraries as discussed below:

1. Appium Appium is an open-source test automation framework used for mobile native applications. It allows native, hybrid and web application testing and supports automation test on physical devices or emulator. Read more about Appium here.
2. Appium Client [Python] Appium offers multiple client libraries to choose from [Java, Ruby, Python, PHP, JavaScript, and C#]. This supports Appium’s extensions over WebDriver protocol. We chose Python as a client library to write our test cases. Find more details here.
3. Firebase performance SDK [Optional]This service helps you to gain insight into the performance characteristics of your iOS, Android, and web apps. This need to be embedded in the application APK file. It records the application start-up time, HTTP/S network requests and few other metrics. Read more here.
4. Mitmproxy [Optional] mitmproxy is a free and open-source interactive HTTPS proxy. This enables to record and report network traffic metrics in detail. Like connect time, SSL time, send time and more for individual requests. Read more here.

Custom Monitor Architecture

Flow for this Custom Monitor is divided into multiple sections based on its functionality. It begins with the Catchpoint infrastructure, then the Custom Monitor script used to run the test case, followed by an actual Android device used for running test cases and a proxy to capture network traffic.

1. Catchpoint Portal initiates the custom script on a node. It passes script related details to the node for execution. Example: script file name.
2. The Catchpoint Linux node is where the custom script is placed. All related dependencies are installed in advance.
3. Next, the custom monitor script initiates Appium client, a python script is used in this demonstration. An Appium server will always be running on this machine.
4. Appium server checks if the device is connected to the machine/node and executes the test cases.
5. The Android device executes the commands sent from Appium server on the native application. If Firebase is already configured into the APK, then it will record network requests, metrics like response time, and total bytes consumed.
6. Mitmproxy is used to capture individual network call details in depth. This helps to capture metrics like connect, SSL, Wait, etc.
7. All the network metrics are parsed from Mitmproxy logs, here a Python script is used for parsing.
8. Custom script sends back all the captured metrics by printing them on the console.
9. Catchpoint captures these metrics using insights feature from console logs and plots them into graphs.

Native Application Performance Troubleshooting

To troubleshoot performance issues, let us look at the data captured through Custom Monitor. The test case we are discussing here involves opening an application and clicking on a button to download 3 images. The below graph plots Response Time, which represents the time taken to run the entire test case. If you look at the graph, there is a slight spike in Response Time.

Graph plotting the response time for an hour.

This can be caused by various factors, like the mobile device hardware or latency in network requests. To investigate this further, we look at the device level metrics. For example, memory info, battery usage and CPU usage. Below graph plots memory and battery usage for the same time frame as above.

Graph plots battery status and memory usage by the app.

There does not seem to be any change in device-level metrics, as they remain consistent in the timeframe we are analysing. The next logical step here is to check the performance of each network request. The graph below plots the performance of images requests. This data is pulled from the Firebase SDK.

Graph plots total time taken for individual requests.

We can see that there was a spike in response time in Image 1, this has contributed to the increase in the overall Response Time.

The custom monitor we used here mad it is easy to locate performance degradation which in turn helped rectify the situation at the earliest.

Conclusion

This custom monitor is a great way to examine how an application user journey behaves over time. With the help of Appium and other open-source libraries, we can get in-depth metrics about the UI, device, and network requests. Catchpoint alerts can be set up to notify a team whenever the alert criteria are met. For example, when the tests run fails or when performance metric values increase/decrease abnormally. The test case we explained in this blog is specific to Android devices but Appium provides support for IOS and desktop applications as well.