Skip to content

Appelium XPCService - Strongly typed bidirectional communication channel between UIApplication and XCUIApplication

Published: 20th July 2022; Updated: 3rd September 2022

When running UI automated tests at scale, sooner or later you will hit that wall where the native APIs are just not enough and you wish for it to be able to do more. One of those "more things" is being able to communicate with an app under test and perhaps vice versa.

Disclaimer

The whole topic of whether there should or shouldn't be any direct link between the UI automated test and the app under test is a delicate one, with strong arguments on both sides. Going forward will only focus on the practical and technological aspects of the introduced XPCService that enables bidirectional communication.

Standard app interaction in XCTests

Let's first explore what standard ways there are for interacting with the app under test in XCTest. By its nature, UI automated tests on iOS are almost purely Black-box tests.

What you can do

The APIs essentially allows you to:

  • Read the currently presented content of the screen (texts, labels, buttons, etc.)
  • Interact with the those elements via tap(), typeText(...), swipeUp(), etc.

What you can't do

From the things that XCTest doesn't allow you to do, we can mention:

  • Access the memory of the app under test (such as UIApplication.shared) because it is a separate executable
  • Invoke any function/method inside the app under test (doesn't matter if it is marked with public or not)
  • Access the storage of the app under test (NSUserDefaults, keychain, file system etc.)
  • Check the status of permissions of the app under test (PHAuthorizationStatus, CLAuthorizationStatus, etc.)

The "grey area"

In addition to these two groups, there is also a small "grey area" of useful APIs which don't really fall under the black-box testing (that is why we said it is almost black-box testing). Here are the things like:

  • Passing flags and configs at app launch time via launchEnvironment and launchArguments
  • Resetting authorization status for resources (photos, location, contacts etc.). Even though this would have been possible by resetting the privacy settings by navigating to Settings.app, doing it with APIs is far more practical.

XCTests limitations

Practically speaking, by being able to read the content of the screen, tap on the visible elements and eventually pass a flag to the app at launch time, the XCTest covers the vast majority of cases that you will want to automate.

However, if you are running UI automated tests at scale, you have probably already hit the limits of available APIs. Let's explore a few of those examples and see later how the XPCServer addresses them.

Getting a UUID of a user (or any data from the app)

Imagine testing a scenario where you log in a user into the app but soon after login, you want to block that user on your backend and want to make sure that the user gets logged out from the app. To achieve this, you might need to extract some kind of user UUID from the app and afterward, make a request to your backend which will ban or block that user:

1
2
3
let app = XCUIApplication()
let userUuid = app.getCurrentUserUuid()
self.backedHttpClient.banUser(userUuid)

now, when you resume the app, the user should be logged out with an error message displayed:

1
2
app.activate()
XCTAssertTrue(app.staticTexts["Account blocked"].exists)

Simulating a camera (or Bluetooth, NFC, etc)

Imagine that you are implementing an app that allows you to scan invoices. If the scanned invoice is too blurry, you want the app to show an alert to the user asking him/her to re-take the photo. The next time, the captured photo should appear sharp, and the app should show a success screen.

Strictly speaking, you could pass a launchArgument like -simulateBlurryPhoto, but since this argument will be preserved for the entire lifetime of the app, you will not be able to test the "re-take" flow where the second camera scan should appear sharp.

What you would need instead, is to be able to change those launch arguments during app runtime:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
let app = XCUIApplication()

app.cameraSimulationMode = .blurry
app.buttons["Take photo"].tap()
XCTAssertTrue(app.staticTexts["Image too blurry"].exists)

app.cameraSimulationMode = .sharp
app.buttons["Re-take photo"].tap()

XCTAssertTrue(app.staticTexts["Success"].exists)

Checking the authorization status

If your test cases are complex and you are writing a lot of reusable testing code, you might have wished for being able to check if the app has permission to access contacts/photos/camera and verify the screen based on this knowledge. The way you would use this is:

1
2
3
4
5
if XCUIApplication().locationAuthorizationStatus != .notDetermined {
    XCTAssertTrue(app.staticTexts["Allow location access to get local restaurants recommendations"].exists)
} else {
    XCTAssertTrue(app.staticTexts["Local restaurants recommended to you:"].exists)
}

The screen content is different depending on whether the user has granted the location permission in the past.

Simulating a network request failure (or timeout, being offline etc.)

In some more advanced scenarios, assuming you are running E2E UI automated tests (against a real backend service, not a local mock server), you might wish to be able to test those edge cases with networking:

  • What happens if the GET /posts times out?
  • What happens if the POST /messages request returns 401?
  • What happens if the DELETE /articles request throws a "no internet connection" error?

As with simulating camera, some of these cases can be achieved via launchArguments but, as we mentioned above, it simply doesn't scale and becomes impractical very soon:

  1. The arguments remain in memory for the entire lifetime of the app
  2. Changing the arguments requires you to restart the whole app, which will make the tests very slow

XPCService to the rescue

As you might have expected, we developed the XPCService to address all of the limitations mentioned above and much more. It is built on the concept of messaging and message handlers, and it exposes APIs for bidirectional communication between UIApplication and XCUIApplication.

There are 2 "layers" of APIs that it offers:

  1. General-purpose APIs for dispatching messages and encoding/decoding payloads
  2. Higher-level APIs built on top of Layer 1, that are useful in specific test flows (like checking app permissions)

General-purpose APIs

Each XCUIApplication() now has an xpcService property, which can be used for sending messages to the application under test:

XCUIApplication().xpcService.send("setCameraMode", content: CameraMode.blurry)

In addition to sending messages (or invoking methods by name), you can attach a custom payload to each message. The content has to conform to Encodable which gives you the flexibility of easily passing all core types (String, Int, Bool enums etc.), but also the robustness of passing more complex objects - you just need to make them conform to Encodable protocol.

When a message has a response, you can easily decode it by passing the expected type (analogous to the JSONDecoder):

let userUuid: UUID = try XCUIApplication().xpcService.send("getUserUuid")

You can use this mechanism for extracting any kind of data from the app.

Sync vs async

Notice that the above method throws an error. Each send(...) method is synchronous/blocking and as such, it can throw an error (timeout, encoding error etc.). For each synchronous method, there is a corresponding sendAsync(...) method which takes a completion block with generic Swift.Result type:

XCUIApplication().xpcService.sendAsync("getUserUuid", decoding: UUID.self) { result in
    // result is Result<UUID, XPCError>
}

So far we have explored the APIs from the client's point of view. Now let's take a look at how we will handle those calls on the "server" side (in our case it is the app under test).

Registering the handlers can be done conveniently in application:didFinishLaunchingWithOptions: method:

AppeliumFeedback.xpcService?.on("setCameraMode", decoding: CameraMode.self) { cameraMode in
    /// store cameraMode for later use when simulating camera capture
}

Note that in the above example, the XPCService is optional. This is because the service is only initialized if the app is being tested, otherwise it is nil.

Finally, let's explore the case where we need to return the UUID of the current user:

AppeliumFeedback.xpcService?.on("getUserUuid", encoding: UUID.self) {
    return UUID() // fetch the UUID from keychain for example
}

High level APIs

Building on top of the low-level messaging APIs, we went a step further and implemented a couple of handy APIs for retrieving those authorization statuses. Here are a few examples:

let app = XCUIApplication()

_ = app.contactsAuthorizationStatus
_ = app.locationAuthorizationStatus
_ = app.mediaAuthorizationStatus(for: .video)
_ = app.eventKitAuthorizationStatus(for: .event)
_ = app.userNotificationSettingsAuthorizationStatus
_ = app.photoLibraryAuthorizationStatus(for: .readWrite)
_ = app.healthKitAuthorizationStatus(for: .workoutType())

Following a similar setup, you can create XCUIApplication extensions in your tests for common operations:

extension XCUIApplication {
    func getUserUuid() throws -> UUID {
        return try XCUIApplication().xpcService.send("getUserUuid", decoding: UUID.self)
    }
}

As the popular saying goes: The limit is only your imagination! :)

Performance 🚀

The performance comes into the spotlight whenever dealing with anything at scale. For this reason, the XCPService is designed to be extremely fast. In Appelium SDKs, we are using it from dispatching simple events to synchronizing individual frames during screen recording.

Practically speaking, two use-cases are worth mentioning when it comes to performance:

Test Average duration
Dispatching an event (fire and forget) 0.3ms
Request-response round trip 3.3ms
  • Dispatching an event consists of sending a message to the other party (App -> Test or Test -> App) without waiting for the response or a completion block to be invoked.
  • Request-response round trip consists of sending a message to the other party and waiting for a response to arrive (regardless of whether the waiting is done in a blocking or non-blocking way).

As visible from the benchmarks, the speed even allows us to do sustained 120 fps round-trips (if ever needed)1.

An alternative approach with an HTTP server

It wouldn't be fair to talk about XCTService without bringing its alternative into comparison. For years, iOS devs were bypassing the XCTest limitations by hosting an HTTP server inside the app under test and then interacting with it using REST-like calls done from the UI tests.

Dragging an HTTP server into a mobile app for purpose of sending a flag during UI tests, already speaks a lot about how needed bidirectional communication is and how far are engineers willing to go in order to work around this limitation.

Performance comparison

Let's first extend the above table and add the HTTP server to the comparison:

Test XPCService HTTP server
Dispatching an event (fire and forget) 0.3ms 3ms (3x slower)
Request-response round trip 3.3ms 12ms (3.5x slower)

Based on the performance alone, XPCService is a clear winner1. But, putting raw benchmarks aside, we believe there are far greater benefits than the performance itself.

HTTP server and port(s)

The way a typical setup with HTTP server works is that you open a specific port for listening within your app. This particular step has some far-reaching and quite undesired consequences:

  1. If you are running the tests in Xcode, macOS might keep annoying you about accepting incoming connections.

  2. Depending on your company security policy and setup, you might not even be able to open connections and might need some additional rights in order to put the firewall exception in place.

  3. If you are running the tests on parallel Simulators, the setup starts to be even more complicated, because you need to assign a different port to each simulator instance given that they all share the same network of the host computer (Mac). Since there is no centralized coordination mechanism in XCTest (each XCTestRunner runs in a stand-alone process without references to other runners), you will have yet another "interesting" problem that requires solving.

  4. If you want to be able to initiate a call from the app) (perhaps to send a signal to the test when a particular event occurs), you will have to embed yet another HTTP server inside your tests.

Codable support

Finally, the HTTP traffic is based on concepts such as IP addresses, ports, URLs, headers, status codes, HTTP methods and raw bytes in the payload. Like it or not, implementing a communication mechanism based on HTTP traffic will expose you to most if not all of these concepts, even if you just want to send an enum value to the app under test.

Understandably the Codable layer can be added on top of the HTTP-based APIs, but typically it doesn't come out of the box (Swifter, Embassy etc.).

Contrary to the HTTP server setup, the XPCService supports Codable in every exposed API. Sending things like enums, strings or whole objects and later decoding them on the other side works by default.

XPCService summary

  • A powerful bidirectional communication channel between UIApplication and XCUIApplication
  • Robust and elegant APIs for transferring anything from Bool to complex objects conforming to Codable
  • High level APIs for accessing permission authorization statuses of the app under test (camera, location etc.)
  • Extremely fast: 3ms for a round-trip
  • No incoming connections prompt from Xcode or firewall exception to deal with

AppeliumTests SDK

The XCTService APIs are part of AppeliumTests SDK for iOS. The availability of XCTService is not tied to any Appelium license, but we believe that you can get the biggest benefits when combining these APIs with the powerful UI tests recording and analysis functionality of the Appelium platform. With just a moment that takes to integrate the SDK, you will get:

  • Complete test execution reporting in an online dashboard (with test trends and stats)
  • Intelligent test failures grouping and annotations (like you are used to treating crash reports)
  • Network and console logs of the app under test (unavailable with plain XCTest)
  • Screen recordings of test failures
  • Queries, assertions, actions, network logs, console logs and stack traces of the UITest-Runner
  • Test performance improvement suggestions for each test execution

  1. Benchmarks were done using iPhone 12 Pro Max, iOS 15.5, Xcode 13.4.1 and Swifter HTTP server ↩↩