c - Linux, timerfd accuracy

Question

I have a system that needs at least 10 mseconds of accuracy for timers.
I went for timerfd as it suits me perfectly, but found that even for times up to 15 milliseconds it is not accurate at all, either that or I don't understand how it works.

The times I have measured were up to 21 mseconds on a 10 mseconds timer.
I have put together a quick test that shows my problem.
Here a test:

#include <sys/timerfd.h>
#include <time.h>
#include <string.h>
#include <stdint.h>
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <inttypes.h>

int main(int argc, char *argv[]){

    int timerfd = timerfd_create(CLOCK_MONOTONIC,0);
    int milliseconds = atoi(argv[1]);
    struct itimerspec timspec;
    bzero(&timspec, sizeof(timspec));
    timspec.it_interval.tv_sec = 0;
    timspec.it_interval.tv_nsec = milliseconds * 1000000;
    timspec.it_value.tv_sec = 0;
    timspec.it_value.tv_nsec = 1;

    int res = timerfd_settime(timerfd, 0, &timspec, 0);
    if(res < 0){
       perror("timerfd_settime:");
       return 1;
    }
    uint64_t expirations = 0;
    int iterations = 0;
    while( res = read(timerfd, &expirations, sizeof(expirations))){
        if(res < 0){ perror("read:"); continue; }
        if(expirations > 1){
            printf("%" PRIu64 " expirations, %d iterations
", expirations, iterations);
            break;
        }
        iterations++;
    }
    return 0;
}

And executed like this:

Zack ~$ for i in 2 4 8 10 15; do echo "intervals of $i milliseconds"; ./test $i;done
intervals of 2 milliseconds
2 expirations, 1 iterations
intervals of 4 milliseconds
2 expirations, 6381 iterations
intervals of 8 milliseconds
2 expirations, 21764 iterations
intervals of 10 milliseconds
2 expirations, 1089 iterations
intervals of 15 milliseconds
2 expirations, 3085 iterations

Even assuming some possible delays, 15 milliseconds delays sounds too much for me.

Answer 1

Try altering it as follows, this should pretty much garuntee that it'll never miss a wakeup, but be careful with it since running realtime priority can lock your machine hard if it doesn't sleep, also you may need to set things up so that your user has the ability to run stuff at realtime priority (see /etc/security/limits.conf)

#include <sys/timerfd.h>
#include <time.h>
#include <string.h>
#include <stdint.h>
#include <stdio.h>
#include <sched.h>

int main(int argc, char *argv[]) 
{
    int timerfd = timerfd_create(CLOCK_MONOTONIC,0);
    int milliseconds = atoi(argv[1]);
    struct itimerspec timspec;
    struct sched_param schedparm;

    memset(&schedparm, 0, sizeof(schedparm));
    schedparm.sched_priority = 1; // lowest rt priority
    sched_setscheduler(0, SCHED_FIFO, &schedparm);

    bzero(&timspec, sizeof(timspec));
    timspec.it_interval.tv_sec = 0;
    timspec.it_interval.tv_nsec = milliseconds * 1000000;
    timspec.it_value.tv_sec = 0;
    timspec.it_value.tv_nsec = 1;

    int res = timerfd_settime(timerfd, 0, &timspec, 0);
    if(res < 0){
       perror("timerfd_settime:");
    }
    uint64_t expirations = 0;
    int iterations = 0;
    while( res = read(timerfd, &expirations, sizeof(expirations))){
        if(res < 0){ perror("read:"); continue; }
        if(expirations > 1){
            printf("%ld expirations, %d iterations
", expirations, iterations);
            break;
        }
        iterations++;
    }
}

If you are using threads you should use pthread_setschedparam instead of sched_setscheduler.

Realtime also isn't about low latency, it's about guarantees, RT means that if you want to wake up exactly once every second on the second, you WILL, the normal scheduling does not give you this, it might decide to wake you up 100ms later, because it had some other work to do at that time anyway. If you want to wake up every 10ms and you REALLY do need to, then you should set yourself to run as a realtime task then the kernel will wake you up every 10ms without fail. Unless a higher priority realtime task is busy doing stuff.

If you need to guarantee that your wakeup interval is exactly some time it doesn't matter if it's 1ms or 1 second, you won't get it unless you run as a realtime task. There are good reasons the kernel will do this to you (saving power is one of them, higher throughput is another, there are others), but it's well within it's rights to do so since you never told it you need better guarantees. Most stuff doesn't actually need to be this accurate, or need to never miss so you should think hard about whether or not you really do need it.

quoting from http://www.ganssle.com/articles/realtime.htm

A hard real time task or system is one where an activity simply must be completed - always - by a specified deadline. The deadline may be a particular time or time interval, or may be the arrival of some event. Hard real time tasks fail, by definition, if they miss such a deadline.

Notice this definition makes no assumptions about the frequency or period of the tasks. A microsecond or a week - if missing the deadline induces failure, then the task has hard real time requirements.

Soft realtime is pretty much the same, except that missing a deadline, while undesirable, is not the end of the world (for example video and audio playback are soft realtime tasks, you don't want to miss displaying a frame, or run out of buffer, but if you do it's just a momentary hiccough, and you simply continue). If what you are trying to do is 'soft' realtime I wouldn't bother with running at realtime priority since you should generally get your wakeups in time (or at least close to it).

EDIT:

If you aren't running realtime the kernel will by default give any timers you make some 'slack' so that it can merge your request to wake up with other events that happen at times close to the one you asked for (that is if the other event is within your 'slack' time it will not wake you at the time you asked, but a little earlier or later, at the same time it was already going to do something else, this saves power).

For a little more info see High- (but not too high-) resolution timeouts and Timer slack (note I'm not sure if either of those things is exactly what's really in the kernel since both those articles are about lkml mailing list discussions, but something like the first one really is in the kernel.