# Handling signals in Linux with C++


## Why signals are important

Signals are the simplest form of IPC in Unix-like systems.  They are notifiers
to the process that an external event has happened.  As this is a system
specific mechanism, there's no portable C++ STL library providing support for
signal handling.  But it's fairly easy to write one.  First though a short
introduction is required.

## Signal disposition

Each thread in a process has a signal mask which can be modified either via
`setprocmask` (for single threaded processes) or `pthread_sigmask` (for
multithreaded processes).  Every raised signal is checked against the mask.  If
it's blocked, it's residing in a **PENDING** state and it will be immediately
handled once unblocked.  If it's not blocked, its corresponding signal handler
will be called or the default action executed in case there is none.  Of course
there are signals which can't be blocked or its disposition can't be altered
(like _SIGKILL_ or _SIGSTOP_).  In other words, _pending_ signal is handled as
soon as thread's signal mask unblocks it.  It's also worth to remember that
signals are not queued (aside from realtime signals - these are queued).
Multiple subsequently raised *pending* signals will be handled as if a single
one was raised.

## What's in the toolbox?

Most Unix like OSes provide the following APIs to allow for signal handling:

- `signal(2)`
- `sigaction(2)`
- `pause(2)`
- `sigwaitinfo(2)` and `sigtimedwait(2)`
- `signalfd`

Which one to use?  The use of `signal` is discouraged due to portability issues
(the interface is not common for all Unix-like OSes).  `sigaction` seems like a
reasonable candidate but there's a caveat.  The thread on which the signal
handler, registered with `sigaction`, is gonna be called is unknown.  This
introduces a level of unpredictability and this is something that is not
desired.

What about the remaining `pause`, `sigwaitinfo`, `sigtimedwait` and `signalfd`?
`pause` sounds good however it implies that we've already registered a signal
handler with something like `sigaction` in order to stop blocking so, it won't
solve the problem of predictable thread selection for execution of the
signal handler.  The last two options are the ones I'm gonna explore.
But first, let's create some utilities.

## Masking signals

`sigwaitinfo` and friends blocks until there's a pending signal, which it then
returns along with signal details enclosed in `siginfo_t` structure.  In order
for signal to become pending, its default disposition has to be altered.  All
signals on **all threads** have to be blocked so, the default disposition is
not interfering with the signal handler I'm trying to build.  How to assure
that this happens on *all* threads?  There's no way to do that.  To achieve
that, signal mask has to be configured **prior** to creation of any other
threads.  As a result, all children threads will inherit it.  With that in mind
let's define a class: `ScopedSigSet`.  By default, it will take the provided
signal mask and block all signals in it.  The original mask will be saved.
Once the object is destroyed, it will be restored.

```C++
class ScopedSigSet {
public:
    ScopedSigSet(const ::sigset_t& sigSet) :
        oldSet{setSigMask(sigSet)}
    {
    }

    ~ScopedSigSet() {
        setSigMask(oldSet);
    }

    static ::sigset_t fullSet() {
        ::sigset_t set;
        ::sigfillset(&set);
        return set;
    }

private:
    const ::sigset_t oldSet;

    static ::sigset_t setSigMask(const ::sigset_t& newSet) {
        ::sigset_t oldSet;
        if (0 != ::pthread_sigmask(SIG_SETMASK, &newSet, &oldSet)) {
            const std::string errStr = ::strerror(errno);
            throw std::runtime_error(errStr);
        }
        return oldSet;
    }
};
```

`ScopedSigSet` treats the signal mask as a resource and is a *RAII* wrapper
around it.  Once it goes out of scope, the original mask is restored.
Unfortunately, since the signal mask can be altered externally anyway by 3rd
party libraries, `ScopedSigSet` can't provide any strict guarantees.  It's
purpose, is purely for convenience.

A couple more utilities will come in handy, specifically in regards to the
signal set construction.  For that purpose I defined the following functions:

```C++
template <typename ... SigT>
void extendSet(::sigset_t& sigSet, SigT... sigs) {
    (sigaddset(&sigSet, sigs), ... );
}

template <typename ... SigT>
::sigset_t createSet(SigT... sigs) {
    ::sigset_t set;
    sigemptyset(&set);
    extendSet(set, std::forward<SigT>(sigs)...);
    return set;
}
```

`extendSet` is using c++17 fold expression to perform a series of `sigaddset`
invocations, every each extending the provided signal set with a new signal.
Thanks to these two functions it's possible to create signal sets in a very
convenient way, by just specifying the desired signals.  The `sigset_t` will be
automatically constructed and populated, i.e.:

```C++
const auto sigSet = createSet(SIGUSR1, SIGUSR2, SIGTERM, SIGINT);
```

### Signal handler callbacks

In spirit of my previous blog post, signals will be handled with a callbacks
object.  The interface of this object will look the following way:

```C++
class SignalHandlerCallbacks {
public:
    virtual ~SignalHandlerCallbacks() = default;

    /**
     * @brief Callback for signal handling
     *
     * @param sigNo signal number
     * @param senderPid sender's PID
     *
     * @return return true to request termination of the signal handler, false otherwise
     */
    virtual bool onSignal(int sigNo, ::pid_t senderPid) = 0;
};
```

Very simple interface with only one callback.  The `onSignal` callback returns
a bool value.  If the returned value is `true` it's gonna be interpreted by the
handler as a request to terminate.

## `sigwaitinfo` based SignalHandler implementation

With a basic set of prerequisites out of the way it's possible to proceed with
a `SignalHandler` implementation.  For that, I'm gonna define a class
`SigWaitHandler`, the interface is gonna look the following way:

```C++
class SigWaitHandler {
public:
    SigWaitHandler(const ::sigset_t& waitSet,
            std::unique_ptr<SignalHandlerCallbacks> cb);

    ~SigWaitHandler() {
        stop();
    }

    void stop();

    void run();
};
```

The interface is as simple as it can get.  `stop()` allows to explicitly
request termination of the handler.  As a result the `run()` API will terminate
immediately. `run()` may be executed on any thread.  Callbacks are gonna be
dispatched on the same thread.  `run()` will block until signal handler is
explicitly stopped.

Full implementation is available in a form of library on
[gitlab](https://gitlab.com/twdev_projects/signals).  In this post, I'm gonna focus
only on the most important bits:

```C++
    void stop() {
        isRunning.store(false);
        ::raise(SIGUSR1);
    }

    void run() {
        isRunning.store(true);

        while (isRunning.load()) {
            siginfo_t siginfo;
            const int sigNo = ::sigwaitinfo(&waitSet, &siginfo);
            const auto err = errno;

            if (sigNo == -1 && err != EAGAIN) {
                const std::string errStr = ::strerror(errno);
                throw std::runtime_error(errStr);
            } else if (sigNo > 0) {
                if (!isRunning.load() && sigNo == SIGUSR1 && siginfo.si_pid == pid) {
                    // stop has been explicitly requested
                    break;
                } else {
                    // call user's signal handler
                    if (callbacks->onSignal(sigNo, siginfo.si_pid)) {
                        isRunning.store(false);
                        break;
                    }
                }
            }
        }
    }
```

`SIGUSR1` serves a double purpose here.  It may be used by the client API but
internally, it's used to wake up the `sigwaitinfo` for the purpose of
termination.  If the sender's `pid` matches the process' and at the same time,
the `isRunning` flag has been set to `false` it's being interpreted as a
request to terminate.  `SIGUSR1` is always unmasked in the constructor.

## `signalfd` based SignalHandler implementation

`signalfd` is another mean allowing to implement a modern signal handler.
`signalfd` allows for creation of a file descriptor.  Read operations on the
file descriptor will be unblocked once there's a pending signal.  This allows
for creation of a similar "busy" loop as in case of `sigwaitinfo`.  The only
difference is that the loop will be blocked on `read` operation rather than
explicitly wait for a signal.  This is a perfect pattern for writing efficient
server applications.  The file descriptor created by `signalfd` can be
multiplexed by APIs like `poll`, `epoll` or `select` along with client
connections.  In this case, multiplexing is still gonna be required.  Another
file descriptor is necessary, acting as an event source.  Only one type of
events is needed; event indicating termination request, which will, in result,
break the dispatching loop.  For that purpose, I'm gonna use `eventfd`, which is
another convenient API designed specifically for that purpose.


### Even more utilities

Prior to that, I'm gonna need some more utilities.  First, a RAII style file
descriptor wrapper:

#### `ScopedFd`

```C++
class ScopedFd {
    constexpr static int InvalidFd = -1;
public:
    explicit ScopedFd(int fd) :
        fd{fd}
    {
    }

    ScopedFd(ScopedFd&& o) :
        fd{o.fd}
    {
        o.fd = InvalidFd;
    }

    ScopedFd& operator=(ScopedFd&& o) {
        ScopedFd tmp = std::move(*this);
        std::swap(fd, o.fd);
        return *this;
    }

    ~ScopedFd() {
        if (fd != InvalidFd) {
            close(fd);
        }
    }

    // copying is forbidden
    ScopedFd(const ScopedFd&) = delete;
    ScopedFd& operator=(const ScopedFd&) = delete;

    int get() const noexcept {
        return fd;
    }

    bool isValid() const noexcept {
        return fd != InvalidFd;
    }

private:
    int fd;
};
```

It's a movable only object.  Copying is disabled.  Interesting bit is the move
assignment operator.  The pattern implemented there is very similar to ["copy &
swap"](https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Copy-and-swap) idiom but slightly different.

On rvalue assignment, the object moves itself to a temporary.  As a result, the
temporary will close the current file descriptor (once it goes out of scope)
and at the same time it's possible to safely overwrite the fd with the one from
the right hand side object's (that's what the `swap` operation is doing).

#### `FdWrapper`

Another wrapper is required around
`ScopedFd` as it will become later apparent.  The purpose of `FdWrapper` is
solely to provide a uniform interface and, as well, to provide some rudimentary
error handling.  Otherwise this code would be duplicated in further objects.
The implementation looks like so:

```C++
class FdWrapper {
public:
    virtual ~FdWrapper() = default;

    FdWrapper(int fd, const std::string& name) :
        fd{fd},
        name{name}
    {
        if (!this->fd.isValid()) {
            const std::string errStr = ::strerror(errno);
            throw std::runtime_error(name + ": " + errStr);
        }
    }

    int get() const noexcept {
        return fd.get();
    }

protected:
    ScopedFd fd;
    const std::string name;
};
```

If the provided file descriptor is, for any reason, invalid, this wrapper will
provide us with some common error handling.  Having `FdWrapper` implemented, I
can now continue with some convenience wrappers around Linux's APIs.

#### `SignalFd`

With `FdWrapper` at hand, creation of wrappers for Linux APIs becomes trivial:

```C++
class SignalFd : public FdWrapper {
public:
    SignalFd(const ::sigset_t& sigSet) :
        FdWrapper(signalfd(-1, &sigSet, SFD_CLOEXEC), "signalfd")
    {
    }
};
```

Looks very simple.  Thanks to all of the prior efforts, the file descriptor
obtained from the system that way is automatically guaranteed to be closed and
I can benefit from error handling in case `signalfd` fails.  Some more wrappers
are needed.

#### `EventFd`

This one is equally simple:

```C++
class EventFd : public FdWrapper {
public:
    EventFd() :
        FdWrapper(eventfd(0, EFD_CLOEXEC), "eventfd")
    {
    }

    void notify() noexcept {
        uint64_t u = 1;
        ::write(get(), &u, sizeof(u));
    }

    void wait() {
        uint64_t u = 0;
        if (ssize_t n = ::read(get(), &u, sizeof(u)); n != sizeof(uint64_t)) {
            throw std::runtime_error("expected to read an event of size uint64");
        }
    }
};
```

It extends the `FdWrapper` interface with two more APIs allowing for sending
and receiving events.  For the sake of simplicity, error handling is in a quite
rudimentary shape.  One last wrapper is required.

#### `EpollFd`

Since multiplexing is required, I've decided to use `epoll` for that purpose.
The wrapper looks the following way:

```C++
class EpollFd : public FdWrapper {
public:
    EpollFd() :
        FdWrapper(epoll_create1(EPOLL_CLOEXEC), "epollfd")
    {
    }

    void addEventSource(const FdWrapper& fdw) {
        struct epoll_event ev;
        ev.events = EPOLLIN;
        ev.data.fd = fdw.get();
        if (-1 == epoll_ctl(fd.get(), EPOLL_CTL_ADD, ev.data.fd, &ev)) {
            throw std::runtime_error("unable to add event source");
        }
    }
};
```

In here, I benefit from a generic API that `FdWrapper` provides to allow to add
file descriptors `epoll_wait` is gonna operate on.

The required tool set is complete.  It's possible to proceed with the
`signalfd` based signal handler implementation now.  It's gonna comply with the
requirements defined by the generic signal handler interface defined earlier
(in fact this interface has been extracted for code de-duplication purposes in
the library implementation in the [gitlab](https://gitlab.com/twdev_projects/signals) repository).

```C++
class SignalFdHandler {
public:
    SignalFdHandler(const ::sigset_t& sigSet,
            std::unique_ptr<SignalHandlerCallbacks> cb) :
        signalFd{sigSet},
        cb{std::move(cb)}
    {
        epollFd.addEventSource(eventFd);
        epollFd.addEventSource(signalFd);
    }

    void stop() {
        if (isRunning.load()) {
            isRunning.store(false);
            eventFd.notify();
        }
    }

    void run() {
        isRunning.store(true);

        while (isRunning.load()) {
            const std::size_t maxEvents = 16;
            struct epoll_event events[maxEvents];
            const int nfds = epoll_wait(epollFd.get(), events, maxEvents, -1);
            if (-1 == nfds) {
                throw std::runtime_error("error waiting for an event");
            }

            for (int i = 0; i < nfds; i++) {
                if (events[i].data.fd == eventFd.get()) {
                    eventFd.wait();
                    break;
                } else {
                    signalfd_siginfo sigInfo;
                    ssize_t s = ::read(events[i].data.fd, &sigInfo, sizeof(sigInfo));

                    if (cb->onSignal(sigInfo.ssi_signo, sigInfo.ssi_pid)) {
                        stop();
                    }
                }
            }
        }
    }

private:
    EventFd eventFd;
    SignalFd signalFd;
    EpollFd epollFd;
    std::atomic<bool> isRunning{false};
    std::unique_ptr<SignalHandlerCallbacks> cb;
};
```

Error handling is simplified in this example.  The implementation's intrinsics
are quite obvious I think. The file descriptor set is comprised of two sources;
`signalfd` and `eventfd`.  The former one, provides means to retrieve and
handle pending signals, the latter one serves as a notification to terminate
the handler.

## How to use this implementation?

Both `SigWaitHandler` and `SignalFdHandler` are implementing the same
interface, the usage is very simple.  An example program skeleton may look the
following way:

```C++
class MyCallbacks :
    public signals::SignalHandlerCallbacks
{
public:
    bool onSignal(int sigNo, ::pid_t pid) {
        std::cout << "received signal: " << sigNo << ", from: " << pid << std::endl;
        return (sigNo == SIGTERM);
    }
};

int main(int argc, const char *argv[])
{
    // masks all signals, make sure this call is done prior to any children thread creation
    ScopedSigSet scopedSigSet{ScopedSigSet::fullSet()};

    const auto sigSet = createSet(SIGUSR1, SIGUSR2, SIGTERM, SIGINT);

    SignalFdHandler sfh{sigSet, std::make_unique<MyCallbacks>()};

    std::thread t{[&](){
        sfh.run();
    }};

    std::cout << "in main thread" << std::endl;
    t.join();
    return 0;
}
```

## `libsignals` library

I've summarised the discussed implementation as an independent library
available on my [gitlab account](https://gitlab.com/twdev_projects/signals).  There are some minor differences from the
discussed implementation, mainly an interface clean up.  I'm interested in
feedback regarding this implementation so, feel free to reach out!

## What about `boost::asio::signal_set`?

Yes, I'm aware that a similar solution is provided by `boost`.  Additional
benefit is that it's portable and not only Linux specific.  It's something
definitely recommended.  I've decided to explore my own implementation though
for educational purposes and in situations when integration with boost asio is
not desired.

