A simple URLSession mock

There are so many people advocating for the use of URLProtocol for mocking HTTP requests in Swift that I couldn’t believe how quickly it fell apart for me. In fact, I found more writing about using URLProtocol as a mock than I did about using URLProtocol for its intended purpose. This post is about the shortcomings that I encountered, and how I solved them by mocking URLSession instead.

The trouble with URLProtocol

Most of the posts advocating for mocking with URLProtocol recommend storing a single request handler as a static variable on the URLProtocol subclass. The closure gives you a chance to assert against the request and inject a specific response or raise an exception. I like that this approach lets me keep all of my #expect and #require calls in the test function, but I ran into trouble using it to test my API client. One method implements the pretty common pattern of getting a pre-signed S3 URL from my API, and then using that URL to upload some data to a bucket. These two requests have to happen every time I upload something, so it made sense to treat them as a single method in the client rather than exposing the two-step process.

The requests have a fixed sequence, so I modified the mock to store an array of request handlers instead, and simply popped one handler off of the array for each request. That worked well when I ran the test, but then I found another problem that really highlighted the weakness of the URLProtocol approach. Up until this point I had been iterating by running one test case at a time. When I tried running the entire test plan, things got messy.

URLProtocols are instantiated by the system, so I never get to handle an instance of one directly. That is why most mock implementations store the request handlers in a static property, in other words, as a global variable. When tests run in parallel, there is no isolation for that global state. Put simply, all of the tests were filling up the requestHandlers array with their own handlers, and then racing to get the next handler in the list regardless of whether it belonged to them or not.

So I set out to find a way to make sure each test case could keep track of its own handlers. My first idea was to store the handlers in a dictionary keyed on the anticipated request. I’m using HTTPRequest from the HTTPTypes package, which conforms to Hashable so that was an easy adjustment. Unfortunately, I quickly realized that many of my tests produced identical requests. Making every request in my test cases unique in some way would solve the problem, but that felt like sweeping the problem under the rug.

Another way to solve this would be to serialize my requests. There are a lots of ways to accomplish this, but making my tests run serially seems like the easiest. That approach would work as long as the order of requests within a single test case is static, but I still had one big objection: slow tests don’t get run often enough. By forcing all of my tests to run serially, I’ve slowed down my test suite considerably. I felt like I could do better.

Acting locally

To reiterate, the core problem with URLProtocols is that I don’t have any say about how they are instantiated, and I never get access to the instance that serves a particular request. That means that my requestHandlers would always be global state in a URLProtocol based approach. Fortunately if I abandon URLProtocol there is a better solution that is also conceptually simpler: I can mock URLSession instead.

I can create a really simple mock of URLSession that lets me make all of the necessary requests without a lot of effort. This solution is nearly as transparent to our application code as the URLProtocol approach, but it gives me the control I need over the lifecycle of our mock.

I started by creating a protocol that my mock and URLSession will share:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

import Foundation
import HTTPTypes

public protocol HTTPRequestable {
    func data(
        for request: HTTPTypes.HTTPRequest
    ) async throws -> (Data, HTTPTypes.HTTPResponse)

    func upload(
        for request: HTTPTypes.HTTPRequest,
        from data: Data
    ) async throws -> (Data, HTTPTypes.HTTPResponse)
}

Now that I have a protocol, I can make URLSession conform to it. An extension to URLSession from HTTPTypesFoundation already makes it conform to my protocol, so I can leave the body of the extension empty:

1
2
3

import HTTPTypesFoundation

extension URLSession: HTTPRequestable {}

Finally, I can build my new MockURLSession that conforms to HTTPRequestable:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

typealias RequestHandler = (HTTPRequest, Data?) throws -> (HTTPResponse, Data)

class MockURLSession: HTTPRequestable {
    public var requestHandlers: [RequestHandler] = .init()
    
    func data(
        for request: HTTPTypes.HTTPRequest
    ) async throws -> (Data, HTTPTypes.HTTPResponse) {
        let (response, responseData) = requestHandlers.removeFirst()(request, nil)
        
        return (responseData, response)
    }
    
    func upload(
        for request: HTTPTypes.HTTPRequest,
        from data: Data
    ) async throws -> (Data, HTTPTypes.HTTPResponse) {
        let (response, responseData) = requestHandlers.removeFirst()(request, data)
        
        return (responseData, response)
    }
}

And that is all there is to it. The protocol and mock can be expanded to include other methods of URLSession as needed, but most API clients won’t need more than this.

Let’s take a look at how to use this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

import Testing
import HTTPTypes
@testable import MyAPIClient

@Suite("Test a client method call that makes multiple requests")
struct TestComplexCalls {
    var session = MockURLSession()
    
    @Test
    func callSucceeds() {
        let firstRequestInput = Data("First Request".utf8)
        let firstRequest = HTTPRequest(
            method: .post,
            url: "https://example.com/request1"
        )
        let firstResponse = HTTPResponse(status: .ok)
        let firstResponseOutput = Data("First Response".utf8)
        
        session.eventHandlers.append({ request, data in
            #expect(request == firstRequest)
            #expect(data == firstRequestInput)
            
            return (firstResponse, firstResponseOutput)
        })
        
        let secondRequest = HTTPRequest(
            method: .put,
            url: "https://example.com/request2"
        )
        let secondResponse = HTTPResponse(status: .created)
        let secondResponseOutput = Data("Second Response".utf8)
        
        session.eventHandlers.append({ request, data in
            #expect(request == secondRequest)
            #expect(data == firstRequestOutput)
            
            return (secondResponse, secondResponseOutput)
        })
        
        let client = MyAPIClient(session: session)
        let result = client.upload(firstRequestInput)
    }
}

One thing to note about the example above is that in Swift Testing a suite is reinitialized for every test case, so no two test cases will share an instance of MockURLSession.

This mock works great as long as the code in my API client performs the requests sequentially. If you need to be able to handle parallel requests, you can adapt the requestHandlers property of MockURLSession to be a dictionary keyed on the request instead. I leave it as an exercise for the reader to handle identical parallel requests that need different handlers in the same test case. I suspect this use case is uncommon enough that most users won’t need to worry about it.

This ought to cover most use cases for testing API clients, and it solves the shortcomings of the URLProtocol-based approach nicely. I welcome feedback and suggestions on Mastodon.