When we develop products we need to check the quality before delivering it to the customer. Accordingly, testing is necessary for us to ensure that our output meets the highest standard of quality available. Sometimes, if products are constantly being added to or updated with features, the number of test cases also increases along with these features. So, teams will need to check more details and take more time to make these products stable and of the highest quality.
This is the precise reason why we use automated testing. It ensures that teams can test efficiently and effectively.
Testing on a simulator is easier and more flexible than on real devices. The main reason for this is we can test without real devices and can select the OS's version of the platform to test. That said, everything has pros and cons.
When our team was developing automated parallel testing, we ran into many problems. For example, simulators consume a lot of the host machine's resources, leading to unstable testing, mismatched UI between the simulator and real device, and sometimes the test itself can be unstable. These challenges inspired us to develop automated parallel testing on real devices.
After we developed, experimented, and ran tests, real devices turned out to be more stable and consumed much less of the host machine's resources when compared to the simulator. My laptop (MacBook Quad-Core Intel Core i7), for example, can run the parallel test with six real devices, both iOS & Android (maybe even more, we’ve yet to test the limits)
Real devices aren't flexible for changing OS versions like the simulator, but it's not a big obstacle since we can run a parallel test on both real devices and the simulator at the same time.
Appium is a server-based, cross-platform automation testing tool. It's compatible with various platforms, programming languages, and test frameworks because the automated test can be used on many devices and configurations.
Appium 2.0 adds more features, and this version of the driver can be installed separately from the Appium driver. We can install the driver that we need to use, update, or remove it without changing the version of Appium. Moreover, Appium 2.0 has many new plugins we can use to improve our work in the future.
Let’s dive into the details of the implementation Here’s a quick overview of what will be covered
npm install -g appium@next`.
appium driver install xcuitest` brew install carthage` Appium dependencies manager. `npm install appium-doctor -g` package for diagnosing Appium setup. Put this command into the terminal
`appium-doctor --ios`
The result will look like the image below, it shows the necessary dependencies and those that are optional in your machine.
If you want to migrate from simulator to real device, you'll need to change and add some of the existing capabilities.
ipa` instead of ` app` and change your absolute path to the application` xcrun xctrace list devices`
The UDIDis used to configure the desired capability to identify the devices that we would like to run the automated testing.
After set up, your device will prompt you to run the test script. When you first open an app that you've manually installed, you’ll see the notification “Untrusted developer”, you can allow it by accessing Settings > General > VPN & Device Management ( IOS >= 16.0 )
Now, we’ve completed setting up our device, but in this case the test has not worked and the Appium server log shows us message ` info XCUITest xcodebuild exited with code '65' and signal 'null'` which means the code signing is incorrect. You need to manually configure WebDriverAgent as follows.
WebDriverAgent is an iOS Webdriver server that can be used to remote control iOS devices, and it’s compatible with Appium. It allows you to tap, scroll, launch and kill applications. WebDriverAgent is developed and used at Facebook for end-to-end testing and has been successfully adopted by Appium. Let’s provision and authorize the WebDriverAgent as follows:
1. First, we’ll need to open the WebDriverAgent project in the Xcode. For Appium 2.X you can find the location of webDriverAgent.xcodeproj by using the command
` echo "$(dirname "$(find "$HOME/.appium" -name WebDriverAgent.xcodeproj)")" `
This will show the full path of WDA's source folder.
2. Go to the location path
`cd <PATH>`
Then run the following command to set the project. The next step is to put the command in the terminal.
`open ./`
The Finder will open the folder. We can then open WebDriverAgent.xcodeproj directly.
3. Select WebDriverAgentLib from the Target tab and check to select "Automatically manage signing" in the "Signing & Capability" tab and then select your Development Team. This should also auto-select the Signing Certificate. The outcome should look like this:
4. Select WebDriverAgentRunner from the Target tab and check to select "Automatically manage signing" in the "Signing & Capability" tab and then select your Development Team. This should also auto-select Signing Certificate. The outcome should look like this:
5. If the Bundle Identifier fails, you can change the name of the bundle, because the bundle ID is unique. If someone has already taken the ID, you’ll need to change it.
6. You can check if it’s a success by selecting Integration from the Target tab and building on your device.
7. Finally, verify that everything works by building the project with the command:
xcodebuild build-for-testing com.paloit.th.mobile.testfarm.automation.test-without-building -project WebDriverAgent.xcodeproj -scheme WebDriverAgentRunner -destination 'id=<udid>'
If this was successful, the output should end with something like:
To complete verification, you can try accessing the WebDriverAgent server status.
(Note: you must be on the same network as the device, and know its IP address, from Settings > Wi-Fi > Current Network) ` export DEVICE_URL='http://<device IP>:8100' export JSON_HEADER='-H "Content-Type: application/json;charset=UTF-8, accept: application/json"' curl -X GET $JSON_HEADER $DEVICE_URL/status `
You ought to get back an output that looks something like this:
`
{
"value": {
"state": "success",
"os": {
"name": "iOS",
"version": "16.0"
},
"ios" : {
"simulatorVersion" : "16.0",
"ip" : "192.168.1.114"
},
"build": {
"time": "Nov 23 2022 12:33:18"
}
},
"sessionId" : "8951A7BE-F0AD-416A-A8FB-D042F31F58E9",
"status" : 0
}
`
Tip : If you would like to run automated testing with the real device without a cable, you can set your device to connect with a machine by using WIFI.
You need to use the same network for both your machine and device, connect the device with the lightning cable and open the Xcode. Then, go to Window > Devices & Simulators ( Cmd ⌘ + Shift ⇧ + 2 ) Then check “connect via network”.
After that, pull the lightning cable off and open Devices & Simulators again. You'll see the icon after the device name. This means you can now run the test via WIFI.
One limitation: the developer mode on iOS Devices is only available on iOS version 16.0 and later.
Automated testing on real devices, lets us know if the product has any bugs or problems, while using the same device your user is likely to use, while reducing testing time and consuming less of a machine’s resources than a simulator. consume the machine's resources less than the simulator.
To understand more about automating technical processes for better business outcomes, get in touch with our team of experts to learn what's possible.
References
https://github.com/appium/appium-xcuitest-driver/blob/master/docs/real-device-config.md
https://github.com/facebookarchive/WebDriverAgent