Callbacks in C++ API design

Callbacks are a perfect way for clients to interface with an API abstracting an events source. How to design an API for such abstraction to make it unambiguous and easy to use? The simplest choice is functor injection, similarly as it is done in many places in STL (or even C stdlib by means of function pointers). Can we do better though? What are the alternatives available? In this post, I’ll try to provide a step by step overview of a design process for an imaginary HttpServer class.

Event producers

Let’s focus on an example, enter HttpServer class:

1
2
3
4

class HttpServer {
public:
    void listen(int port);
};

At the moment it doesn’t do much. Ideally, we want to get notified whenever there’s a new connection or a new pending request. As a result, application code can process new connections and serve incoming requests. This can be achieved in (at least) two ways.

Decoupling through inheritance

We could design the HttpServer to be an extendible interface, requiring the API clients to extend it and provide the needed functionality. Consider the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

class HttpServer {
public:
    virtual ~HttpServer() = default;

    struct ClientInfo {
        ...
    };

    struct Request {
        ...
    };

    void listen(int port) final;

    virtual void onConnect() {}

    virtual void onRequest() {}

    virtual void onDisconnect() {}
};

...

class ConcreteHttpServer : public HttpServer {
public:
    void onConnect() override {}

    void onRequest() override {}

    void onDisconnect() override {}
};

Inheriting from HttpServer semantically means that we are re-implementing the server every time. Imagine you wish to implement a bare bones HttpHandler. This object would just handle requests and doesn’t care about server intrinsics. It’s not a server, it’s just a handler. The “is-a” relation, created through inheritance, implies something to the contrary though. This doesn’t seem to be optimal from the design standpoint. It’s clear that this approach is a form of a compromise and further exploration is required.

Decoupling through callbacks

Another option is to inject the needed functionality via callbacks. The implementation would look the following way:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

class HttpServer {
public:

    struct ClientInfo {
        ...
    };

    struct Request {
        ...
    };

    void listen(int port);

    void onConnect(std::function<void(ClientInfo)> callback);

    void onRequest(std::function<void(Request)> callback);

    void onDisconnect(std::function<void(ClientInfo)> callback);
};

Callbacks and inheritance are a form of double dispatch, performance-wise both solutions are equivalent. From the design perspective though, the callbacks approach doesn’t seem optimal either. The API is a bit too verbose and not as elegant as the former approach. The callbacks approach is much cleaner in regards to separation of concerns. Event handlers are entirely decoupled through callbacks and none of the server details are leaking. It may not look as clean but it’s definitely an improvement.

Callbacks from clients perspective

Consider the following:

1
2
3
4
5
6

class HttpHandler {
public:
    void handleConn(HttpServer::ClientInfo client);

    void handleRequest(HttpServer::Request request);
};

To use this code with our server we’d have to transform the member functions to delegates, either with std::bind or via lambdas:

1
2
3
4
5
6
7
8
9

HttpServer server{};
HttpHandler handler{};

...
server.onRequest([&handler](auto request){ handler.handleRequest(request); });
server.onConnect([&handler](auto clientInfo){ handler.handleConn(clientInfo); });

...
server.listen();

It’s clear that further improvement is required to make this code easier to use. This is becoming more apparent with the number of different types of events a given event producer may provide. Registering all of them can be a bit of a burden. Additionally, the API seems disorganised and arbitrary which may be confusing for its clients.

Decoupling through callbacks object.

Third approach combines the prior two. Consider the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

class HttpServerCallbacks {
public:
    virtual ~HttpServerCallbacks() = default;

    virtual void onConnect(ClientInfo) = 0;

    virtual void onRequest(Request) = 0;

    virtual void onDisconnect(ClientInfo) = 0;
}

HttpServerCallbacks is a dedicated object defining callbacks for all events that HttpServer may produce. HttpServer accepts it as a mandatory dependency:

1
2
3
4
5
6

class HttpServer {
public:
    explicit HttpServer(std::unique_ptr<HttpServerCallbacks> callbacks);

    void listen(int port);
};

The callbacks themselves became an implementation of HttpServerCallbacks interface. This implementation is totally independent from the server. Effectively HttpServerCallbacks interface is an integration edge. It allows for custom callbacks implementation and separates the server details. As a result, we’ve achieved all the design goals:

• it’s clear, from the client’s perspective, what has to be implemented and how the interface looks like,
• it’s easy to use the API, as callbacks registration is trivial and is done automatically when instantiating HttpServer,
• it’s easy to test the API (which I’m gonna discuss later),
• object hierarchy is not disturbed. There’s a clear distinction on what’s a server, what’s a callback and the relationship between them.

Usage

In order to use this code, an implementation of the callbacks is required. HttpHandler would be implementing the HttpServerCallbacks interface:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

class HttpHandler : public HttpServerCallbacks {
public:
    void onConnect(ClientInfo) override {}

    void onRequest(Request) override {}

    void onDisconnect(ClientInfo) override {}
};

...
auto handler = std::make_unique<HttpHandler>();
HttpServer server{std::move(handler)};
...
server.listen();

Testing

Let’s consider the testability of all presented approaches.

Inheritance

It’s difficult to test both HttpServer and the implementation of its callbacks in isolation. We either implement a testable version of HttpServer with mocked implementation of the callbacks or the entire thing. It’s impossible to focus solely on callbacks themselves. This is a major drawback of this design. Strictly hypothetically, consider an implementation of HttpServer’s dependencies:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

class HttpServer {
public:
    HttpServer(Certificate c,
            Keys k,
            Socket s,
            ...)
};

class ConcreteHttpServer : public HttpServer {
    ConcreteHttpServer(Certificate c,
            Keys k,
            Socket s,
            ...) :
        HttpServer(c, k, s, ...)

}

All of that would have to be mocked and injected even for the purpose of only testing the callbacks, which proves that this approach is not viable in the long run for a more complex, real life, scenario.

Callbacks

The situation looks much better with callbacks, although still not ideal. HttpServer and its callbacks implementation exist entirely in separation. It’s possible to inject mocked callbacks to HttpServer implementation, making the object very easy to test.

1
2
3
4
5
6
7

HttpServer server{};
HttpHandlerMock mock;

...
server.onRequest([&mock](auto request){ mock.handleRequest(request); });
server.onConnect([&mock](auto clientInfo){ mock.handleConn(clientInfo); });
...

At the same time, it’s possible to focus on the callbacks without having to instantiate the server.

1
2
3
4
5

TEST(test_httpHandler) {
    HttpHandler handler;
    handler.handleRequest(clientInfo);
    ...
}

This solution has clear advantage over the former one. The only inconvenience comes from the fact that mocked objects have to be wrapped in lambdas as well for the purpose of testing HttpServer.

Callbacks object

It’s possible to test HttpServer in isolation by injecting mocked implementation of callbacks:

1
2
3
4
5

class MockHttpServerCallbacks : public HttpServerCallbacks {
    ...
};

HttpServer server{std::make_unique<MockHttpServerCallbacks>()};

Same applies to the implementation of HttpServerCallbacks. It can be tested without the server itself, in separation.

1
2
3
4
5

TEST(test_httpServerCallbacks) {
    HttpHandler handler;
    handler.handleRequest(clientInfo);
    ...
}

Conclusion

In the following overview I’ve proven that using callback objects to decouple the client implementation from an event producer API has great advantage over inheritance. It became quite apparent as well that as much as lambdas are a perfect mechanism to inject a single callback, for the purpose of extending a generic algorithm, or as means to handle an event, they may not be a perfect solution in case an API interface requiring the client to register many different types of callbacks - in such situation, callback objects are definitely a superior solution.