Mobile Automation Stories — XCUITest — 4
Running our tests in parallel with MockServer…
This article will show you the cause of the main problem that will occur in your tests that you run in parallel with XCUITest on the mock server and how to solve it.
Introduction
In this article, I will talk about running our tests in parallel on the mock server and solving the server connection error that occurred during this parallel test run process. We will proceed by playing on a project that was written using the swifter library before. You can access this project from the link.
When we examine many of the tests already written with the mock server, you will see that they do not encounter such a problem because they do not include parallel work. Although some mock server libraries solve the problem themselves, we need to produce our own solution for Swifter.
As a first step, let’s look at how we use the Swifter library in our project. Next, let’s explain the use of mock data and the solution to this problem that came up in our parallel tests.
What is Swifter?
Swifter is actually a library developed with the swift language that allows us to create an http server engine. With this library, we can read the data of requests such as GET, POST, PUT, DELETE from a json file and return them through this mock server.
- In a certain period of time, the server gets up over the port and can be stopped again after completing its operations.
- It is completely free, but it is necessary to understand the framework and to know the swift programming language.
- It creates a flexible space. Accessing parameters on requests is easy to manipulate the response.
Implementing Swifter in iOS
Install the Swifter library using your preferred add the following line to your project’s Podfile:
use_frameworks!
pod 'Swifter', '~> 1.5.0'Let’s show how we worked on the BookStore project we developed on before. You can add it to the Podfile as follows.
After Podfile adds the relevant line, pod install is run from the terminal in the file directory where the Podfile is located.
pod installOnce the library is added, import the Swifter module in your test class file
import SwifterYou may have understood how to use json and network class from the file structure I explained in previous articles.
In the example picture below, the MockServer class is a class that we will use the server start, stop and json read-write functions of the swifter library. We also have json files in Files.
To decide which request and which data the tests will work with;
1. We must define the requests PATH.
2. We have to define the json filename of the requests.
private let django123SearchPath = "/1.0/search/django123/1"
private let djangoSearchPath = "/1.0/search/django/1"
private let bookInfoPath = "/1.0/books/9781783984404"
private let emptySearchJSONFilename = "django123.json"
private let normalSearchJSONFilename = "django.json"
private let bookInfoJSONFilename = "bookInfo.json"A server is started on port 4567 with the startServer function. The same server is stopped with the stopMockServer function. For Example:
class MockServer {
let server = HttpServer()
public func startMockServer() {
do {
try server.start(4567)
print("Server status: \(server.state)")
} catch {
print("Server start error: \(error)")
}
}
func stopMockServer() {
server.stop()
}
func configureSearchTest() {
do {
let emptySearchJSONPath = try TestUtil.path(for: emptySearchJSONFilename, in: type(of: self))
server[django123SearchPath] = shareFile(emptySearchJSONPath)
let searchJSONPath = try TestUtil.path(for: normalSearchJSONFilename, in: type(of: self))
server[djangoSearchPath] = shareFile(searchJSONPath)
let bookInfoJSONPath = try TestUtil.path(for: bookInfoJSONFilename, in: type(of: self))
server[bookInfoPath] = shareFile(bookInfoJSONPath)
startMockServer()
} catch {
XCTAssert(false, "Swifter Server failed to start.")
}
}
}Then, we can make a definition as follows so that these defined requests and json data are captured in incoming requests and the file is returned.
func configureSearchTest() {
do {
let emptySearchJSONPath = try TestUtil.path(for: emptySearchJSONFilename, in: type(of: self))
server[django123SearchPath] = shareFile(emptySearchJSONPath)
let searchJSONPath = try TestUtil.path(for: normalSearchJSONFilename, in: type(of: self))
server[djangoSearchPath] = shareFile(searchJSONPath)
let bookInfoJSONPath = try TestUtil.path(for: bookInfoJSONFilename, in: type(of: self))
server[bookInfoPath] = shareFile(bookInfoJSONPath)
startMockServer()
} catch {
XCTAssert(false, "Swifter Server failed to start.")
}
}As a different usage, the following code can be preferred.
func configureSearchTest() {
do {
let emptySearchJSONPath = try TestUtil.path(for: emptySearchJSONFilename, in: type(of: self))
server.GET[django123SearchPath] = shareFile(emptySearchJSONPath)
let searchJSONPath = try TestUtil.path(for: normalSearchJSONFilename, in: type(of: self))
server.POST[djangoSearchPath] = shareFile(searchJSONPath)
let bookInfoJSONPath = try TestUtil.path(for: bookInfoJSONFilename, in: type(of: self))
server.GET[bookInfoPath] = shareFile(bookInfoJSONPath)
startMockServer()
} catch {
XCTAssert(false, "Swifter Server failed to start.")
}
}The purpose of this usage is to return different response when there are both GET and POST requests with the same request path.
We can also capture json files with the path function in TestUtil.
import Foundation
import XCTest
enum TestUtilError: Error {
case fileNotFound
}
class TestUtil {
static func path(for fileName: String, in bundleClass: AnyClass) throws -> String {
if let path = Bundle(for: bundleClass).path(forResource: fileName, ofType: nil) {
return path
} else {
throw TestUtilError.fileNotFound
}
}
}We must set the launchArgument and launchEnvironment parameters in BaseTest. We also write functions that will launch the application and start and stop the mock server.
class BaseTest: XCTestCase {
let mockServer = MockServer()
override func setUp() {
continueAfterFailure = false
bookApp.launchArguments = ["-uitesting"]
bookApp.launchEnvironment = ["USE_MOCK_SERVER": "true"]
mockServer.startMockServer()
}
override func tearDown() {
mockServer.stopMockServer()
super.tearDown()
}
}Now let’s write our tests.
private let searchTableViewIdentifier = "SearchTableView"
class SearchTests: BaseTest {
override func setUp() {
super.setUp()
mockServer.configureSearchTest()
bookApp.launch()
}
override func tearDown() {
super.tearDown()
}
func testCancel() {
moveToSearchTab()
bookApp.buttons["Cancel"].tap()
XCTAssert(bookApp.keyboards.count == 0, "Keyboard should have dismissed.")
}
func testSearchButton() {
moveToSearchTab()
bookApp.searchFields["Search"].typeText(XCUIKeyboardKey.return.rawValue)
XCTAssert(bookApp.buttons["Cancel"].exists == false, "Cancel button should disappear.")
XCTAssert(bookApp.keyboards.count == 0, "Keyboard should dismiss.")
}
func testEmptySearch() {
moveToSearchTab()
bookApp.searchFields["Search"].typeText("django123")
XCTAssert(bookApp.staticTexts["Your search did not have any results."].waitForExistence(timeout: 2))
}
func testSearch() {
moveToSearchTab()
bookApp.searchFields["Search"].typeText("django")
XCTAssert(bookApp.tables[searchTableViewIdentifier].waitForExistence(timeout: 2))
XCTAssert(bookApp.tables[searchTableViewIdentifier].cells.count > 0)
XCTAssert(bookApp.staticTexts["Your search did not have any results."].exists == false)
bookApp.tables[searchTableViewIdentifier].cells.firstMatch.tap()
XCTAssert(bookApp.staticTexts["Learning Django Web Development"].waitForExistence(timeout: 2))
XCTAssert(bookApp.buttons["Close"].exists)
bookApp.buttons["Close"].tap()
XCTAssert(bookApp.searchFields["Search"].waitForExistence(timeout: 2))
bookApp.searchFields["Search"].tap()
bookApp.searchFields["Search"].clearText()
XCTAssert(bookApp.tables.count == 0)
XCTAssert(bookApp.staticTexts["Your search did not have any results."].exists == false)
}
private func moveToSearchTab() {
XCTContext.runActivity(named: "Move to Search Tab") { _ in
bookApp.tabBars.firstMatch.buttons["Search"].tap()
XCTAssert(bookApp.buttons["Cancel"].waitForExistence(timeout: 2), "Search textfield cancel button missing.")
XCTAssert(bookApp.keyboards.count > 0, "Keyboard should be shown.")
XCTAssert(bookApp.searchFields["Search"].waitForExistence(timeout: 2))
}
}
}
extension XCUIElement {
func clearText() {
guard let stringValue = value as? String else {
return
}
let deleteString = String(repeating: XCUIKeyboardKey.delete.rawValue, count: stringValue.count)
typeText(deleteString)
}
}Now we can see on AppDelegate how to make the application look at the mock http server.
class AppDelegate: UIResponder, UIApplicationDelegate {
var window: UIWindow?
func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
if ProcessInfo.processInfo.arguments.contains("-uitesting") {
UIView.setAnimationsEnabled(false)
if ProcessInfo.processInfo.environment["USE_MOCK_SERVER"] == "true" {
BookStoreConfiguration.shared.setBaseURL(URL(string: "http://localhost:4567")!)
}
else {
BookStoreConfiguration.shared.setBaseURL(URL(string: "http://localhost:8000")!)
}
}
return true
}
}Actions taken here:
- If arguments is -uitesting it turns off animations.
- If environment “USE_MOCK_SERVER” is true, baseUrl is set to “http://localhost:4567".
Now we can run our tests.
We see that all the tests ended successfully. 🎉
Here, we were able to view the requests coming from localhost as follows🌹
Solution for using the same port in parallel tests
As can be seen in the record below, you can see that 4 simulators using the same port on the local server cause an error or wait in server requests.
In order to solve this situation, we created random ports at certain intervals and sent them to AppDelegate and enabled each simulator to communicate with a different port.
Example for startMockServer in MockServer:
func startMockServer(port: in_port_t = in_port_t.random(in: 8000..<10000),
maximumOfAttempts: Int = 10)
{
guard maximumOfAttempts > 0 else { return }
do {
try server.start(port, forceIPv4: true)
bookApp.launchEnvironment["SERVER_PORT"] = "\(port)"
} catch SocketError.bindFailed(let message) where message == "Address already in use" {
startMockServer(maximumOfAttempts: maximumOfAttempts - 1)
} catch {
print("Server start error: \(error)")
}
}Example for AppDelegate:
class AppDelegate: UIResponder, UIApplicationDelegate {
var window: UIWindow?
func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
if ProcessInfo.processInfo.arguments.contains("-uitesting") {
UIView.setAnimationsEnabled(false)
if ProcessInfo.processInfo.environment["USE_MOCK_SERVER"] == "true" {
BookStoreConfiguration.shared.setBaseURL(URL(string: "http://localhost:" + (ProcessInfo.processInfo.environment["SERVER_PORT"] ?? "8000/"))!)
}
else {
BookStoreConfiguration.shared.setBaseURL(URL(string: "http://localhost:8000")!)
}
}
return true
}
}And Successfullll 🎉
No waiting, different ports, successful request results 🌹
Conclusion
As a result, we replicated our tests and ran parallel runs. And 60 tests took about 3 minutes. 90% success was achieved for 60 tests. We have failed tests for some requests that are not mocked. Here is a screenshot of the results with test allure.
Feedback 📬
If you have patiently followed me up to this point and you think we could do it better this way, here is the reason for you to contact me! I am ready to listen to your advice, criticism and feedback that can improve me or you. Contact Us. Thanks…
