Avoiding Release Anxiety 1: Block-based UI Testing in Swift

When deadlines are tight and the product faces considerable changes, it's common for developers to make concessions in the project's quality to make sure it gets shipped in time. This leads to release anxiety - that stressful feeling where you're unsure if you're shipping something that actually works. As a result, teams resort to heavy amounts of manual testing, staying overtime to make sure nothing fell apart and an unhappy environment in general. Avoiding Release Anxiety is a series of posts where I show the things I do to develop durable Swift apps that allow me and my team to sleep worry-free at night.

Resilient UI Tests

I feel that UI Tests are very underrated by the community when compared to other forms of testing, but they're my favorite. When the subject is making sure that a feature works, my opinion is that users don't care if a button isn't the right color or if a font is slightly off -- they only care if it does what it should do. UI Testing is the closest you can get to the actual user experience, so if the UI Tests are working, it's likely that the user's experience in these flows will work as well. As a complement to the usual unit tests, I make sure to always write UI tests that navigate through all variations of the important flows of the app to reduce stress in release days.

With that said, how can you effectively write and maintain these tests if your app has several screens and variations of them?

I deal with this by implementing something that I call block-based testing. Instead of putting all the test logic directly in the test method, I instead divide each step of the navigation into its own "exploration" method. Here's an example:

private func exploreBalanceDetailsScreen() {
    let balanceView = app.tables.buttons["WALLET_HEADER"]
    expect(balanceView.isHittable) == true
    balanceView.tap()
    let detailsView = app.otherElements["BALANCE_DETAILS_VC_ROOT_VIEW"]
    detailsView.waitForExistence("Expected to be in the Balance Details Screen!")
    //Conditions that test if "Balance Details" is behaving correctly
    app.tapNavigationBackButton()
}

Exploration methods start by moving to the relevant screen of the method (assuming that the app is already in a position to do so). After it's confirmed that the relevant screen is working, the app is moved back to where it was before the exploration started.

To confirm that a screen change happened, I like to use this waitForExistence() helper in a view controller's root view property:

extension XCUIElement {
    func waitForExistence(_ description: String? = nil, timeout: TimeInterval = 0.2) {
        let predicate = NSPredicate(format: "exists == true")
        let hasAppeared = XCTNSPredicateExpectation(predicate: predicate,
                                                    object: self)
        _ = XCTWaiter.wait(for: [hasAppeared], timeout: timeout)
    }
}

When all relevant flows are created in this structure, building test cases is just a matter of connecting your LEGO bricks:

class BaseUITestCase: XCTestCase {

    private(set) var app: XCUIApplication! = nil

    override func setUp() {
        super.setUp()
        continueAfterFailure = false
        if app == nil {
            app = XCUIApplication()
            app.add(MockFlags.Environment.isUITest)
        }
    }
}

class WalletViewUITests: BaseUITestCase {
    func testLoggedOnHomeToQRScannerFlow() {
        app.launch()
        expectToBeInHome()
        exploreAvailableCardsWidget()
        exploreBalanceDetailsScreen()
        explorePaymentDetailsScreen()
        exploreTransactionsListScreen()
        goToQRScanner()
    }
}

By reorganizing these blocks, you can test several variations of a flow with little effort. Here's an example of how I test the logged out version of the previous test:

func testLoggedOffHomeToQRScannerFlow() {
    app.add(MockFlags.User.isNotLogged)
    app.launch()
    expectToBeInHome(canInteract: false) //There should be a "logged off" empty state covering the screen.
    exploreLoggedOffScreen(shouldLogin: true)
    testLoggedOnHomeToQRScannerFlow() //Being logged out initially should not affect the rest of the app.
}

One of the things that bothers me the most in programming is when the tests are so complicated that they end up becoming a burden. Developing tests with a defined structure like this makes them a lot easier to maintain and expand from.

Extra: Typesafe Mocks

Although the block structure helps with the development of the tests themselves, we still need to make sure the app can properly navigate the screens and understand when an alternate flow is required.

As spoiled by the previous example, the way we chose to handle the latter is by using MockFlags, which is a simple enum created to represent launch arguments:

public protocol MockFlag {
    var value: String { get }
}

extension MockFlag {
    public var value: String {
        return "\(String(describing: type(of: self)))-\(String(describing: self))"
    }
}

public enum MockFlags {
    public enum Environment: MockFlag {
        case isUITest
        case isUnitTest
    }

    public enum User: MockFlag {
        case isNotLogged
    }
}

extension XCUIApplication {
    func add(_ mockFlag: MockFlag) {
        launchArguments.append(mockFlag.value)
    }
}

public func has(_ mockFlag: MockFlag) -> Bool {
    return CommandLine.arguments.contains(mockFlag.value)
}

By running app.add(theFlagIWantToTest) before launching the UI test, the desired flag becomes available for inspection during runtime as a launch argument. The tough part is how you efficiently react to these flags, but developing a system that can do this well is very beneficial to the evolution of your project.

The way I like to do this is by using protocols in all important components of the app. Every one of these components, such as clients and persistence modules, are initialized once in the app's launch and propagated through the app -- but when the app is in the Unit/UI test environment, different, mocked versions of them are created instead. This is the usual stuff, but the catch here is that since everything is done with protocols, the app itself is completely isolated from the environment peculiarities. This prevents the tests from making the project's maintenance harder.

Here's a basic example of how a client is mocked in this structure:

protocol HTTPRequest {
    associatedtype Response //Omitting paths, parameters, headers and etc for simplicity
}

protocol HTTPClient: AnyObject {
    func send<R>(_ request: R) -> Promise<R.Response> where R: HTTPRequest
}

final class MockClient: HTTPClient {
    func send<R>(_ request: R) -> Promise<R.Response> where R: HTTPRequest {
        guard let value = (request as? Mockable)?.mockedValue else {
            Logger.log("Request \(request) has no mocked version!")
            return Promise(error: HTTPError.generic)
        }
        return Promise(value: value)
    }
}

To make the mocks typesafe, we created a Mockable protocol that allows anything to expose a mocked version of itself based on the available MockFlags. This way, we don't have to deal with HTTP stub libraries and raw json strings that need to be replaced every now and then, and changes in the app's models will make so the mocks have to be updated in compile time as well. The mocked client ends up being simply a class that retrieves these Mockable HTTPRequest's properties.

protocol Mockable {
    associatedtype MockValue
    var mockedValue: MockValue { get }
}

extension HTTPRequest where Self: Mockable {
    typealias MockValue = Response
}

extension UserBalanceRequest: Mockable {
    var mockedValue: UserBalanceResponse {
        return UserBalanceResponse(balance: has(MockFlag.User.noBalance) ? 0 : 1_000_000)
    }
}

To start the logic that propagates the mocked components forward, I like to use the factory pattern to hide the decision that results in their creation. In the case of the client, we allow the AppDelegate to create and retain these components before injecting them to the app's first screen.

func isTestEnvironment() -> Bool {
    return has(MockFlags.Environment.isUITest) || has(MockFlags.Environment.isUnitTest)
}

enum HTTPClientFactory {
   static func create() -> HTTPClient {
        if isTestEnvironment() {
            return MockClient()
        }
        return URLSessionHTTPClient() //The regular HTTPClient
    }
}

class AppDelegate: UIResponder, UIApplicationDelegate {
    let client = HTTPClientFactory.create()
    ...

Although creating the components is easy enough, how you efficiently push them forward to the rest of the app depends on your architecture. We use MVVM-C, so in our case we chose to never use singletons to instead have each Coordinator/ViewModel directly receive and retain the components that are important to them.

final class WalletCoordinator: Coordinator {

    let client: HTTPClient
    let persistence: Persistence

    init(client: HTTPClient, persistence: Persistence) {
        self.client = client
        self.persistence = persistence
        let viewModel = WalletViewModel(client: client, persistence: persistence)
        let viewController = WalletViewController(viewModel: viewModel)
        super.init(rootViewController: viewController, delegate: delegate)
    }
}

This is beneficial to the development of Unit/UI tests as testing certain interactions can be done by simply passing custom versions of the components to each class.

func testViewStates() {
    let mockedClient = MockClient()
    mockedClient.alwaysFail = true
    let viewModel = WalletViewModel(client: mockedClient, persistence: MockPersistence())
    expect(viewModel.state) == .none
    viewModel.load()
    expect(viewModel.state) == .failed
}

func testLockId() {
    class FailableClient: MockClient {
        var shouldFail = false
        func send<R>(_ resource: R) -> Promise<R.Value> where R : HTTPRequest {
            guard shouldFail else {
                return super.send(resource)
            }
            return Promise(error: HTTPError.generic)
        }
    }
    let client = FailableClient()
    let viewModel = CheckoutViewModel(client: client)
    // some tests
    client.shouldFail = true
    // more tests
}

The downside of this approach is that the initializers tend to get really big when a screen has tons of responsibilities, which can be a bit jarring. There are several alternatives to dependency injection out there that don't hurt the testability of the classes, but that's a topic for another blog post.

Conclusion

With everything setup, we're able to create not just UI but also Unit Tests while still being able to scale our project stress-free. By separating the test necessities from the project itself, we can develop conditions for tests without hurting the project's quality. After creating exploration methods and developing UI Tests for important flows that snap these methods together, we can rely on a resilient test suite that allows us to safely release new versions of our apps.

Follow me on my Twitter - @rockbruno_, and let me know of any suggestions and corrections you want to share.