Handling signals in Linux with C++

Why signals are important

Signals are the simplest form of IPC in Unix-like systems. They are notifiers to the process that an external event has happened. As this is a system specific mechanism, there’s no portable C++ STL library providing support for signal handling. But it’s fairly easy to write one. First though a short introduction is required.

Signal disposition

Each thread in a process has a signal mask which can be modified either via setprocmask (for single threaded processes) or pthread_sigmask (for multithreaded processes). Every raised signal is checked against the mask. If it’s blocked, it’s residing in a PENDING state and it will be immediately handled once unblocked. If it’s not blocked, its corresponding signal handler will be called or the default action executed in case there is none. Of course there are signals which can’t be blocked or its disposition can’t be altered (like SIGKILL or SIGSTOP). In other words, pending signal is handled as soon as thread’s signal mask unblocks it. It’s also worth to remember that signals are not queued (aside from realtime signals - these are queued). Multiple subsequently raised pending signals will be handled as if a single one was raised.

What’s in the toolbox?

Most Unix like OSes provide the following APIs to allow for signal handling:

• signal(2)
• sigaction(2)
• pause(2)
• sigwaitinfo(2) and sigtimedwait(2)
• signalfd

Which one to use? The use of signal is discouraged due to portability issues (the interface is not common for all Unix-like OSes). sigaction seems like a reasonable candidate but there’s a caveat. The thread on which the signal handler, registered with sigaction, is gonna be called is unknown. This introduces a level of unpredictability and this is something that is not desired.

What about the remaining pause, sigwaitinfo, sigtimedwait and signalfd? pause sounds good however it implies that we’ve already registered a signal handler with something like sigaction in order to stop blocking so, it won’t solve the problem of predictable thread selection for execution of the signal handler. The last two options are the ones I’m gonna explore. But first, let’s create some utilities.

Masking signals

sigwaitinfo and friends blocks until there’s a pending signal, which it then returns along with signal details enclosed in siginfo_t structure. In order for signal to become pending, its default disposition has to be altered. All signals on all threads have to be blocked so, the default disposition is not interfering with the signal handler I’m trying to build. How to assure that this happens on all threads? There’s no way to do that. To achieve that, signal mask has to be configured prior to creation of any other threads. As a result, all children threads will inherit it. With that in mind let’s define a class: ScopedSigSet. By default, it will take the provided signal mask and block all signals in it. The original mask will be saved. Once the object is destroyed, it will be restored.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

class ScopedSigSet {
public:
    ScopedSigSet(const ::sigset_t& sigSet) :
        oldSet{setSigMask(sigSet)}
    {
    }

    ~ScopedSigSet() {
        setSigMask(oldSet);
    }

    static ::sigset_t fullSet() {
        ::sigset_t set;
        ::sigfillset(&set);
        return set;
    }

private:
    const ::sigset_t oldSet;

    static ::sigset_t setSigMask(const ::sigset_t& newSet) {
        ::sigset_t oldSet;
        if (0 != ::pthread_sigmask(SIG_SETMASK, &newSet, &oldSet)) {
            const std::string errStr = ::strerror(errno);
            throw std::runtime_error(errStr);
        }
        return oldSet;
    }
};

ScopedSigSet treats the signal mask as a resource and is a RAII wrapper around it. Once it goes out of scope, the original mask is restored. Unfortunately, since the signal mask can be altered externally anyway by 3rd party libraries, ScopedSigSet can’t provide any strict guarantees. It’s purpose, is purely for convenience.

A couple more utilities will come in handy, specifically in regards to the signal set construction. For that purpose I defined the following functions:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

template <typename ... SigT>
void extendSet(::sigset_t& sigSet, SigT... sigs) {
    (sigaddset(&sigSet, sigs), ... );
}

template <typename ... SigT>
::sigset_t createSet(SigT... sigs) {
    ::sigset_t set;
    sigemptyset(&set);
    extendSet(set, std::forward<SigT>(sigs)...);
    return set;
}

extendSet is using c++17 fold expression to perform a series of sigaddset invocations, every each extending the provided signal set with a new signal. Thanks to these two functions it’s possible to create signal sets in a very convenient way, by just specifying the desired signals. The sigset_t will be automatically constructed and populated, i.e.:

1

const auto sigSet = createSet(SIGUSR1, SIGUSR2, SIGTERM, SIGINT);

Signal handler callbacks

In spirit of my previous blog post, signals will be handled with a callbacks object. The interface of this object will look the following way:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

class SignalHandlerCallbacks {
public:
    virtual ~SignalHandlerCallbacks() = default;

    /**
     * @brief Callback for signal handling
     *
     * @param sigNo signal number
     * @param senderPid sender's PID
     *
     * @return return true to request termination of the signal handler, false otherwise
     */
    virtual bool onSignal(int sigNo, ::pid_t senderPid) = 0;
};

Very simple interface with only one callback. The onSignal callback returns a bool value. If the returned value is true it’s gonna be interpreted by the handler as a request to terminate.

sigwaitinfo based SignalHandler implementation

With a basic set of prerequisites out of the way it’s possible to proceed with a SignalHandler implementation. For that, I’m gonna define a class SigWaitHandler, the interface is gonna look the following way:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

class SigWaitHandler {
public:
    SigWaitHandler(const ::sigset_t& waitSet,
            std::unique_ptr<SignalHandlerCallbacks> cb);

    ~SigWaitHandler() {
        stop();
    }

    void stop();

    void run();
};

The interface is as simple as it can get. stop() allows to explicitly request termination of the handler. As a result the run() API will terminate immediately. run() may be executed on any thread. Callbacks are gonna be dispatched on the same thread. run() will block until signal handler is explicitly stopped.

Full implementation is available in a form of library on gitlab. In this post, I’m gonna focus only on the most important bits:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

    void stop() {
        isRunning.store(false);
        ::raise(SIGUSR1);
    }

    void run() {
        isRunning.store(true);

        while (isRunning.load()) {
            siginfo_t siginfo;
            const int sigNo = ::sigwaitinfo(&waitSet, &siginfo);
            const auto err = errno;

            if (sigNo == -1 && err != EAGAIN) {
                const std::string errStr = ::strerror(errno);
                throw std::runtime_error(errStr);
            } else if (sigNo > 0) {
                if (!isRunning.load() && sigNo == SIGUSR1 && siginfo.si_pid == pid) {
                    // stop has been explicitly requested
                    break;
                } else {
                    // call user's signal handler
                    if (callbacks->onSignal(sigNo, siginfo.si_pid)) {
                        isRunning.store(false);
                        break;
                    }
                }
            }
        }
    }

SIGUSR1 serves a double purpose here. It may be used by the client API but internally, it’s used to wake up the sigwaitinfo for the purpose of termination. If the sender’s pid matches the process’ and at the same time, the isRunning flag has been set to false it’s being interpreted as a request to terminate. SIGUSR1 is always unmasked in the constructor.

signalfd based SignalHandler implementation

signalfd is another mean allowing to implement a modern signal handler. signalfd allows for creation of a file descriptor. Read operations on the file descriptor will be unblocked once there’s a pending signal. This allows for creation of a similar “busy” loop as in case of sigwaitinfo. The only difference is that the loop will be blocked on read operation rather than explicitly wait for a signal. This is a perfect pattern for writing efficient server applications. The file descriptor created by signalfd can be multiplexed by APIs like poll, epoll or select along with client connections. In this case, multiplexing is still gonna be required. Another file descriptor is necessary, acting as an event source. Only one type of events is needed; event indicating termination request, which will, in result, break the dispatching loop. For that purpose, I’m gonna use eventfd, which is another convenient API designed specifically for that purpose.

Even more utilities

Prior to that, I’m gonna need some more utilities. First, a RAII style file descriptor wrapper:

ScopedFd

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

class ScopedFd {
    constexpr static int InvalidFd = -1;
public:
    explicit ScopedFd(int fd) :
        fd{fd}
    {
    }

    ScopedFd(ScopedFd&& o) :
        fd{o.fd}
    {
        o.fd = InvalidFd;
    }

    ScopedFd& operator=(ScopedFd&& o) {
        ScopedFd tmp = std::move(*this);
        std::swap(fd, o.fd);
        return *this;
    }

    ~ScopedFd() {
        if (fd != InvalidFd) {
            close(fd);
        }
    }

    // copying is forbidden
    ScopedFd(const ScopedFd&) = delete;
    ScopedFd& operator=(const ScopedFd&) = delete;

    int get() const noexcept {
        return fd;
    }

    bool isValid() const noexcept {
        return fd != InvalidFd;
    }

private:
    int fd;
};

It’s a movable only object. Copying is disabled. Interesting bit is the move assignment operator. The pattern implemented there is very similar to “copy & swap” idiom but slightly different.

On rvalue assignment, the object moves itself to a temporary. As a result, the temporary will close the current file descriptor (once it goes out of scope) and at the same time it’s possible to safely overwrite the fd with the one from the right hand side object’s (that’s what the swap operation is doing).

FdWrapper

Another wrapper is required around ScopedFd as it will become later apparent. The purpose of FdWrapper is solely to provide a uniform interface and, as well, to provide some rudimentary error handling. Otherwise this code would be duplicated in further objects. The implementation looks like so:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

class FdWrapper {
public:
    virtual ~FdWrapper() = default;

    FdWrapper(int fd, const std::string& name) :
        fd{fd},
        name{name}
    {
        if (!this->fd.isValid()) {
            const std::string errStr = ::strerror(errno);
            throw std::runtime_error(name + ": " + errStr);
        }
    }

    int get() const noexcept {
        return fd.get();
    }

protected:
    ScopedFd fd;
    const std::string name;
};

If the provided file descriptor is, for any reason, invalid, this wrapper will provide us with some common error handling. Having FdWrapper implemented, I can now continue with some convenience wrappers around Linux’s APIs.

SignalFd

With FdWrapper at hand, creation of wrappers for Linux APIs becomes trivial:

1
2
3
4
5
6
7

class SignalFd : public FdWrapper {
public:
    SignalFd(const ::sigset_t& sigSet) :
        FdWrapper(signalfd(-1, &sigSet, SFD_CLOEXEC), "signalfd")
    {
    }
};

Looks very simple. Thanks to all of the prior efforts, the file descriptor obtained from the system that way is automatically guaranteed to be closed and I can benefit from error handling in case signalfd fails. Some more wrappers are needed.

EventFd

This one is equally simple:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

class EventFd : public FdWrapper {
public:
    EventFd() :
        FdWrapper(eventfd(0, EFD_CLOEXEC), "eventfd")
    {
    }

    void notify() noexcept {
        uint64_t u = 0;
        ::write(get(), &u, sizeof(u));
    }

    void wait() {
        uint64_t u = 0;
        if (ssize_t n = ::read(get(), &u, sizeof(u)); n != sizeof(uint64_t)) {
            throw std::runtime_error("expected to read an event of size uint64");
        }
    }
};

It extends the FdWrapper interface with two more APIs allowing for sending and receiving events. For the sake of simplicity, error handling is in a quite rudimentary shape. One last wrapper is required.

EpollFd

Since multiplexing is required, I’ve decided to use epoll for that purpose. The wrapper looks the following way:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

class EpollFd : public FdWrapper {
public:
    EpollFd() :
        FdWrapper(epoll_create1(EPOLL_CLOEXEC), "epollfd")
    {
    }

    void addEventSource(const FdWrapper& fdw) {
        struct epoll_event ev;
        ev.events = EPOLLIN;
        ev.data.fd = fdw.get();
        if (-1 == epoll_ctl(fd.get(), EPOLL_CTL_ADD, ev.data.fd, &ev)) {
            throw std::runtime_error("unable to add event source");
        }
    }
};

In here, I benefit from a generic API that FdWrapper provides to allow to add file descriptors epoll_wait is gonna operate on.

The required tool set is complete. It’s possible to proceed with the signalfd based signal handler implementation now. It’s gonna comply with the requirements defined by the generic signal handler interface defined earlier (in fact this interface has been extracted for code de-duplication purposes in the library implementation in the gitlab repository).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

class SignalFdHandler {
public:
    SignalFdHandler(const ::sigset_t& sigSet,
            std::unique_ptr<SignalHandlerCallbacks> cb) :
        signalFd{sigSet},
        cb{std::move(cb)}
    {
        epollFd.addEventSource(eventFd);
        epollFd.addEventSource(signalFd);
    }

    void stop() {
        if (isRunning.load()) {
            isRunning.store(false);
            eventFd.notify();
        }
    }

    void run() {
        isRunning.store(true);

        while (isRunning.load()) {
            const std::size_t maxEvents = 16;
            struct epoll_event events[maxEvents];
            const int nfds = epoll_wait(epollFd.get(), events, maxEvents, -1);
            if (-1 == nfds) {
                throw std::runtime_error("error waiting for an event");
            }

            for (int i = 0; i < nfds; i++) {
                if (events[i].data.fd == eventFd.get()) {
                    eventFd.wait();
                    break;
                } else {
                    signalfd_siginfo sigInfo;
                    ssize_t s = ::read(events[i].data.fd, &sigInfo, sizeof(sigInfo));

                    if (cb->onSignal(sigInfo.ssi_signo, sigInfo.ssi_pid)) {
                        stop();
                    }
                }
            }
        }
    }

private:
    EventFd eventFd;
    SignalFd signalFd;
    EpollFd epollFd;
    std::atomic<bool> isRunning{false};
    std::unique_ptr<SignalHandlerCallbacks> cb;
};

Error handling is simplified in this example. The implementation’s intrinsics are quite obvious I think. The file descriptor set is comprised of two sources; signalfd and eventfd. The former one, provides means to retrieve and handle pending signals, the latter one serves as a notification to terminate the handler.

How to use this implementation?

Both SigWaitHandler and SignalFdHandler are implementing the same interface, the usage is very simple. An example program skeleton may look the following way:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

class MyCallbacks :
    public signals::SignalHandlerCallbacks
{
public:
    bool onSignal(int sigNo, ::pid_t pid) {
        std::cout << "received signal: " << sigNo << ", from: " << pid << std::endl;
        return (sigNo == SIGTERM);
    }
};

int main(int argc, const char *argv[])
{
    // masks all signals, make sure this call is done prior to any children thread creation
    ScopedSigSet scopedSigSet{ScopedSigSet::fullSet()};

    const auto sigSet = createSet(SIGUSR1, SIGUSR2, SIGTERM, SIGINT);

    SignalFdHandler sfh{sigSet, std::make_unique<MyCallbacks>()};

    std::thread t{[&](){
        sfh.run();
    }};

    std::cout << "in main thread" << std::endl;
    t.join();
    return 0;
}

libsignals library

I’ve summarised the discussed implementation as an independent library available on my gitlab account. There are some minor differences from the discussed implementation, mainly an interface clean up. I’m interested in feedback regarding this implementation so, feel free to reach out!

What about boost::asio::signal_set?

Yes, I’m aware that a similar solution is provided by boost. Additional benefit is that it’s portable and not only Linux specific. It’s something definitely recommended. I’ve decided to explore my own implementation though for educational purposes and in situations when integration with boost asio is not desired.