Introduction

What device should be used to run the tests? What operating system version, manufacturer and model to use? How to get these devices?

These are some of the questions we ask ourselves when planning a mobile application testing strategy. If we consider automating these tests, other questions will surely come to mind.

Designing and executing automated tests for mobile devices, both for native applications and responsive web apps, pose technical and non-technical challenges, many of them related to the devices and platforms on which the tests will be run.

In automation processes, it is essential to have integrated platforms that cover the entire test cycle, like Katalon, which provide capabilities for automating tests, simplifying test execution setup, and providing solutions for multi-device operation.

The challenge of fragmentation and access to devices

Every day, new models of mobile devices are launched on the market, featuring new operating system versions, new manufacturers and technologies, wider resolutions and screen sizes, and more. End customers use them to download different apps, so they are ideal devices to carry out the tests.

So, how to ensure apps will run smoothly on these devices? What options are available to access such a range of devices? How much time will it take to make them available, configure and run tests on them? What will happen to those devices when the testing phase ends? What costs will be associated with their purchase and maintenance?

These are some questions that arise, and each one in itself represents a challenge to be addressed.  

A few alternatives

Real devices vs. Emulators

As an alternative to some of the previous questions, tests could be performed by emulating the devices required for the automated tests, configuring the necessary emulators for execution and, once the tests are completed, proceeding to discard them. At first glance, it looks like a good choice to avoid purchasing real devices and minimize automated test setup and execution efforts.

But emulators have an extremely important disadvantage that affects the quality of our work. An emulator is not a real device, but rather a piece of software that runs the app as if it were on a mobile device. Therefore, there is always a risk that the applications under test behave differently on an emulated device than on a real device. 

Read more: Mobile Emulator vs Simulator vs Real Device Testing Differences 

The power of the cloud within our reach

The answer to the questions and challenges mentioned before might be up in the cloud. With the advantages of PaaS (Platforms as a Service), the cloud provides the ability to access a large and diverse collection of devices from anywhere in the world.

These real mobile device platforms in the cloud are the solution to the above challenges:

• They are real devices, not emulators, so the environment is 100% real.
• A variety of devices from different manufacturers and models, operating system versions, screen resolutions, etc., are available.
• This minimizes the acquisition costs of the devices to be tested, and there is no need to dispose of them upon completion of the tests.

Katalon TestCloud

What is TestCloud?

Katalon TestCloud is one of the most recent products developed by Katalon for the execution of cloud-based automated tests. TestCloud allows you to run tests of a Katalon project on a wide range of browsers, operating systems and mobile devices.

What does it offer?

Katalon TestCloud allows you to:

• Run automated tests on more than 300 real devices, both Android and iOS, cell phones and tablets, and a wide variety of manufacturers and models.
• Run tests for both native and web-responsive apps (through a web browser on the mobile device.)
• Minimize setup efforts and simplify access to devices through Katalon platform by simply selecting the devices to be used for the implementation.
• Reduces coordination efforts typically required when using real mobile devices while avoiding the need for managing their purchase, delivery and maintenance.
• Parallelize test execution to run on a wide range of operating system-browser combinations. This significantly speeds up test cycles and improves coverage, ensuring applications perform optimally across all these scenarios.
• Access log files and manage test results on a per-device basis for proper tracking and incident management.

How does TestCloud work?

TestCloud is integrated into the Katalon platform. From this platform, test executions (Test Runs) are managed and the devices where the tests will be executed (Environments) are assigned.

To start, you must schedule a Test Run, select the project and the test suite to be executed, and then access the “Environment ”section.

From here it is possible to select the “TestCloud” option and then choose among the devices available.

As shown in the previous images, the Katalon platform provides details of all the devices available to run the tests. By selecting multiple devices, the same tests can be run on all of them, either sequentially or simultaneously.

When confirming the execution, TestCloud will run the test suite on each selected device. You can view the execution log for each device in real time and also access test information and results, and even export them to popular formats such as CSV, HTML and PDF.

Katalon TestCloud and other cloud solutions

Although we have only mentioned TestCloud's features so far, Katalon has always stood out for its high integration capabilities with other tools and platforms. It provides the ability to integrate and run automated tests on other mobile device providers in the cloud, such as BrowserStack, SauceLabs and Kobiton. This feature provides Katalon users with the possibility of having several alternatives when running mobile tests through the cloud, which makes Katalon a versatile and highly extensible tool.

Since TestCloud is a solution developed and maintained by Katalon, it has certain advantages when dealing with Katalon projects.

• Instant integration: It has 100% integration capability and no extra configuration is required. With other cloud appliance vendors, this requires modifying the project or test scripts.
• Centralized information: All test information and devices used are stored and available in one place, the Katalon platform. Because of this, post-test activities, such as the analysis of results, are made simpler since log files and device information no longer need to be gathered from another cloud.

Conclusion

Mobile test automation poses several challenges with different levels of technical complexity, such as the need to configure, maintain and manage the devices where tests are to be run. To get an idea of this, consider how many different manufacturers, models, screen resolutions, and operating systems there are today.

The use of emulators is a risky solution, as the behavior of the system under test may vary depending on whether it is a real device or an emulator.

The cloud -a better alternative- allows us to access a wide range of real devices, avoiding acquisition, maintenance and/or management costs.

Through the Katalon platform, Katalon TestCloud allows access to more than 300 different real devices, both Android and iOS. It provides a high-level interface for the search and selection of devices, supports simultaneous runs, and offers information and reports showing the results for each device.  

Try Mobile Testing With Katalon Now