Data-driven testing approach with Katalon Studio
- What is Data-driven testing?
- Data-driven testing approach with Katalon Studio
- Tutorial video
1. What is Data-driven testing?
Data-driven testing (DDT) is a common term in the computer software testing area. Its define a testing has been done using a table of conditions directly as test inputs and verifiable outputs. Also, in the data-driven testing process, the test environment settings and control are not hard-coded.
This testing process is quite popular and normally applied right after the record and playback mode. In other words, once a set of testing object under a test case is captured, testers will need to add more input values from the external data files to ensure the application can handle the most number of scenarios in the future.
In the control methodology, the external information is “read” from a database that can be any of the below data files.
- Excel files
- ADO objects
- CSV files
- ODBC sources
The most simple way to better understand the explanation of this data testing approach could be under a record of an automated test. In some cases, the values you select during record may not generate any bugs or errors but the external values which is missed during the records may lead them. Then, it is when the tester needs a set of database to diversify their testing inputs.
Also, if there is any change you want to make like putting more or cutting down the data, you are totally free to reshape and customize without causing harm to the hard-coded value. If the change is in hard-coded value itself, so yes, you have to change both in code and data.
Data-driven framework can be best understood by the following diagram:
The Advantages of Data-driven testing
- The scripts could generate parallelly whilst developers are building the application/software. The whole process of designing, planning and running the database is independent towards the application development.
- The tester can avoid the irrelevant and unneeded duplication of testing scripts generated since we already carry a standard file using for testing inputs and verifying the data.
- All the outcomes from inputs, outputs to feasible testing results are collected and stored under a well-managed text records.
- Test cases will be handled under numerous set of data since it is the external information collected separately to the test case. The information is stored under external material which stems from Excel files, XML or any other texting sources.
- The test data can’t be impacted if the test scripts are changing since data-driven testing is running separately with the development. The tester will have no influence towards developers even though they make multiple changes.
- It is a good news for tester if errors or bugs generated in future will be minimized to the lowest probability since there are multiple scenarios and inputs tested using data-driven framework approach.
2. Data-driven testing approach with Katalon Studio
Katalon Studio supports data-driven testing which allows users to define data sets and execute test scripts faster and easier compared to data-driven testing in other frameworks.
This tutorial will provide you a simple example of how to create an automation test case and execute it multiple times using different sets of data.
The Data-Driven Testing Sample Project
The simple example of Data-driven testing is described through a testing booking service. The booking form has 5 different fields, each field contains multiple options. The data used for a specific case may not cause any error to the application, but for the other sets of data, it probably does. In order to make sure the application will be working probably regardless of the inputs, you may leverage data-driven testing for this scenario to execute the test automatically multiple times using different data tests.
Given a sample test case whose scenario is as below:
- Open the login page of the Katalon demo AUT website (http://demoaut.katalon.com/profile.php#login)
- Login using three different accounts
- Validate if the login is success
- Close the browser
You can use the following steps to automate the above test scenario.
Create a new test case using Web Record function
1. Create a new test case and proceed to generate the steps to:
- Access http://demoaut.katalon.com/profile.php#login (the Katalon demo AUT website)
- Enter username/password
- Click Login
- Validate if the Appointment page is displayed
You can utilize the Web Record function to quickly generate test steps. Refer to this guide for more details on the Record & Playback feature of Katalon Studio.
Step 1: Click on the New Test Case button on the main toolbar. Provide a name for your test case and click OK. An empty test case will be created.
Step 2: Click on Record from the main toolbar.
Step 3: Click on the Record button from The Record dialog to start recording test case.
Step 4: Follow the scenario above.
2. The generated test case should be similar to the following:
You can see that the input values for username and password are hard-coded as what you typed during recording (in this case it’s admin/abc123). In order to run this script multiple times using different accounts, you need to parameterize these values. Please continue to the next step.
Modifying the Data-Driven Test and Assigning Input Values
3. Switch to the Variables tab of the test case and create two variables named ‘demo_usn’ and ‘demo_pwd’.
Step 1: Switch to Variables tab of your Test Case. Then click Add.
Step 2: A new row is added to the variable list. Create two variables named ‘demo_usn’ and ‘demo_pwd’. and save the test case when you’re done.
4. Switch back to the Manual view of the test case and set those two variables as inputs for the username/password fields.
Now that you have done the necessary steps to parameterize the required fields for login, proceed to the next steps to prepare data for execution.
5. Create a data file in Katalon to have a dataset containing three login accounts. You can generate data file from sources such as Excel, CSV, Database etc… Refer to Manage Test Data for more details on how to create test data from different sources. The following example shows the internal data file with three login accounts for http://demoaut.katalon.com (note that only ‘John Doe’ is valid):
Data-driven Tests Execution
From here you can apply Data-driven using two methods, either using Test Suites or Test Cases.
A. Execution from test suites
6. Next, create a test suite to execute the test case using the defined dataset.
7. Expand the Data Binding section, add the created data file to the Test Data section and proceed to bind the two variables ‘demo_usr’ and ‘demo_pwd’ to the respective columns of the data file. You may refer to Data for test execution for more details about variable binding.
8. Finally, you can run the test suite, and your login test case will be executed three times using the accounts defined in the test data file.
B. Execute from a test case
6. We can also implement Data-driven tests in a test case. Just create a new test case and switch to Script Mode. To iterate tests with multiple sets of data, we need to use FOR statement and call the test data objects. Copy and paste below code:
- Import InternalData class from Katalon built-in library and define data variable to locate test data table
- For statement to loop through all row of test data table which indicates how many times the test case will run
- To get a row value in test data table, use getRowNumbers() method syntax. For example:
Username field: data.internallyGetValue(“demo_usn”, index)
Password field: data.internallyGetValue(“demo_pwd”, index)\
7. When you have done in Scripts view, switch back to Manual view, the test case will be showed as the following screenshot:
8. Finally, you can run the test case, and your login test case will be executed three times using the accounts defined in the test data file.
Congratulations! You now understand how to approach Data-driven testing with Katalon Studio.
3. Tutorial video is contributed by Shiva Raj