Over the past couple of weeks I have been working on several pieces of
software that critically rely on precise time synchronization across
different components.
I found it quite surprising to come across multiple third party
components that caused problems because they implemented their own
NTP clients and got the processing of NTP timestamps wrong.
This is what prompted me to write this very short introduction
to NTP timestamps and how to convert them to other common
timestamp formats.
The most common timestamp format one encounters in systems programming
is arguable struct timeval which is defined on Windows
in winsock.h and on Unix-like systems in
sys/time.h. This is what we get by e.g. calling
gettimeofday().
struct timeval is typically defined like this:
struct timeval {
time_t tv_sec; /* seconds */
suseconds_t tv_usec; /* microseconds */
};
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| tv_sec |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| tv_usec |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
As we can see here the timestamp consists of two values. One
representing seconds, the other representing microseconds.
The sizes of the variables representing those two fields will vary
across different systems and architectures. E.g. on 64 bit systems
both will usually be 64 bits wide instead of the 32 bits depicted above.
Additionally the meaning of the seconds component will vary depending
on context. When reading the current time on a Unix system tv_sec
will represent the seconds since January 1st 1970, whereas the
reference point can be a different one on other systems.
The same structure is also widely used across APIs to pass relative
time information to arbitrary functions, for example timeouts for
requests. In that case tv_sec will just represent an absolute number
of seconds.
Since we are concerned with synchronizing times across applications and focusing on Unix systems, from here on tv_sec is assumed to represent seconds since January 1st 1970.
tv_usec on the other hand will always contain a number between
zero and 999,999 representing the microseconds part of the given
timestamp and is therefore not directly dependent on the context
in which the timestamp is used.
There are also several other similar formats in common use which
all behave mostly the same as described above but only differ
in the resolution of the sub-second component.
For example struct timespec is typically defined
as:
struct timespec {
time_t ts_sec; /* seconds */
long ts_nsec; /* nanoseconds */
};
In contrast to the structures described so far we now come to NTP timestamps which are defined in RFC 1305 as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Integer part of seconds |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Fractional part of seconds |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
struct ntp_ts_t {
uint32_t seconds;
uint32_t fraction;
};
The first difference to the other types of timestamps we have covered
so far is that the fields of an NTP timestamp are always 32 bits wide,
regardless of the underlying architecture.
(There is also another NTP timestamp format that is 128 bits wide,
usually called NTP datestamp, but we will not cover that here.)
The second difference, to Unix timestamps in particular, is that the seconds part of the NTP timestamp refers to the seconds since January 1st 1900, not 1970.
From what we have learned so far the conversion between Unix and
NTP timestamps should be pretty straight forward:
Just add or subtract 70 years to or from the seconds part and watch
out for potential overflows on systems that don't use 32 bit integers.
Unfortunately there still is the fractional part of the NTP timestamp and this is where it gets a little tricky.
It seems easy to conclude that "Fractional part of seconds" here
effectively means the same as microseconds in the above timestamps.
After all 100 microseconds are .1 seconds or 1/10 of a second which
kind of is a fractional part of a second.
And as far as I can tell this seems to have been the reasoning of
the authors of the software that gave me trouble over the past
couple of days.
What "Fractional part of seconds" actually means here though is this:
A value of 1 represents
1/(2^32) of a second which is about .2 nanoseconds.
So how do we convert between Unix and NTP timestamps ?
As already mentioned converting the seconds part is trivial. Depending on the direction of the conversion, we just add or remove 70 years, which is 2208988800 seconds.
To convert the fractional/sub-second part here's what we do:
To convert from NTP to Unix time we multiply by an appropriate
factor to get to the required subdivision of seconds, e.g.
10^6 in the case of microseconds, and divide the result by 2^32.
The conversion from Unix to NTP time is just the opposite. We
multiply by 2^32 and divide the result by 10^6.
Since the intermediate values can get larger than 32 bits
we'll have to watch out for overflows and use sufficiently large
types for the conversion.
Now that we know what to do, here's some example C code in that performs the actual conversion:
void ntp_to_timeval(struct ntp_ts_t *ntp, struct timeval *tv) {
tv->tv_sec = ntp->seconds - 2208988800;
tv->tv_usec = (uint32_t)((double)ntp->fraction * 1.0e6 / (double)(1LL << 32));
}
void timeval_to_ntp(struct timeval *tv, struct ntp_ts_t *ntp) {
ntp->seconds = tv->tv_sec + 2208988800;
ntp->fraction =
(uint32_t)((double)(tv->tv_usec + 1) * (double)(1LL << 32) * 1.0e-6);
}