25

The chrony documentation warns

BE WARNED: Certain software will be seriously affected by such jumps in the system time. (That is the reason why chronyd uses slewing normally.) Documentation

But the documentation gives no examples. What are examples of software that will be seriously affected? Is the OS or any background processes at risk?

Cecilia
  • 425

10 Answers10

32

This is a bit of open question but let me give some examples:

  • databases - most of them rely a lot of precise time for storing records, indexes, etc
  • security - precise time is very important for security to map action to time and gaps or time duplication is not accepted
  • digital signing - usually part of signed document is the timestamp so wrong time may invalidate the signature
  • scheduling software - may skip or repeat twice jobs depend of time jump direction.
  • clustering software - probably any cluster will need to be in sync and any jump of one or more nodes may have unpredictable result.
Romeo Ninov
  • 6,677
13

I recently got bit by a bug that dates back to 1999 and affects both the JVM and Android Runtime: https://bugs.java.com/bugdatabase/view_bug.do?bug_id=4290274

... two extra executions are fired (unexpectedly) when the system clock is set ahead one minute after the task is scheduled using scheduleAtFixedRate().

I work on a device that starts with the 1970 epoch as the current time, then receives the correct network time a little later. Occasionally a 3rd party library would initialize before the time was set, causing it to experience a 50 year time jump.

The result was scheduleAtFixedRate attempting to catch up on ~50 years worth of invocations... which was about 27 million back-to-back invocations with no delay between them.

That would cause the GC to go haywire and generally bog down the system until it was restarted

9

All software that interacts with real-live hardware. If you have a toaster that toasts bread for 20 seconds, and its software is stupid enough to check against the wall clock, you'll either get white or burned bread if you correct the clock while waiting for your toast.

Practically all applications that control any kind of industrial device need precise timings, like, for example, "open a valve for 5.3 seconds to get the correct amount of fluid". Being off by more than a few milliseconds ruins your product.

Applications that position anything using motors will either use step motors (which are slow) or end switches to determine when to stop. But often, you don't have a switch at every important position, so you'll do some "x m/s for A milliseconds, then y m/s for B milliseconds" logic. Now imagine your NTP daemon adjusts the time by even a single millisecond while this logic is running ...

4

We had an issue with an on-vehicle embedded system where the clock would significant lose time (due to an electrical problem). But the wireless connections were intermittent, so the time only occasionally corrected. The upshot was that when the vehicles finally received wireless, and then an NTP update, the clock would jump forward significantly.

Various systems were checking the "last valid" time of certain things like GPS readings, etc. Suddenly all of these were "old", despite being updated only 0.5 seconds before.

Obviously a reconfiguration could fix the issue, but it was an issue.

TRiG
  • 1,193
  • 3
  • 14
  • 30
Kingsley
  • 141
2

Dovecot IMAP server is affected and (in older versions) it (deliberately) suicides if it detects the system time having jumped backwards. In v2.0, it at least tries to remedy the situation.

See https://wiki.dovecot.org/TimeMovedBackwards

2

Plenty of examples...

filo
  • 431
1

It's already in a comment, but I thought I'd post it as an answer too:

Applications that should have used the steady monotonic clock but don't are also affected. For example, if software checks client keep-alives using the current time, a jump in time may kick out all clients.

I've seen regularly that software uses the wrong clock.

Halfgaar
  • 8,534
1

Most game engines use an update loop that take a delta of the time between the previous and current time. Sometimes a time change or program suspension/resume will cause this delta to be huge. Typically you just filter out large deltas as an outlier.

0

Everyday normal web browsing

Really.

Anything to do with encryption deals in certificates. The certificates must be validated before they are accepted. Part of the validation process is checking the certificate is not expired, which obviously implicates your computer clock. If your let your computer clock get too far out of sync with reality, certificate validation on the computer will fail.

This matters, because pretty much every web page you access these days is transmitted via HTTPS, which uses TLS encryption (and certificate validation) to ensure the integrity of the page contents.

In other words, if you let your clock get off, you might not be able to even browse the web normally.

Now playing with an NTP daemon — where the whole point is keeping your system clock more or less accurate — is unlikely to create a shift large enough to matter. But point it at the wrong time source, and you could easily create this effect.

Additionally, a number of things that deal in authentication rely on the clocks between the user's computer and the server being relatively in sync, with tolerances limited to sometimes no more than a few minutes difference.

Joel Coel
  • 13,117
0

Timing Things

It appears to be the obvious, but according to Falsehoods Programmers Believe About Time, due to lack of knowledge or support for a monotonically time source, developers often use system time to measure how long a process takes, which can account for incorrect measures, if between the two measurements the system clock has changed, like a value which is:

  • slightly bigger than what was the correct;
  • negative (which can likely crash system rellying in a positive value);
  • a hugely bigger than what is correct due to the integer signal bit flip for negative numbers and a incorrect type conversion (a signed -1 has the same memory representation as the maximum unsigned integer value 0xFFFFFFFF)
// this C code prints 4294967295
printf("%u", (unsigned int) -1);

// this comparison is true if (0xFFFFFFFF == (unsigned int) -1) { }

Cache Invalidation

As a wise man once said:

There are only two hard things in Computer Science: cache invalidation and naming things. Phil Karlton

Often distributed systems, use a very short time for cache invalidation, which can be a conservative value, like 60s or higher (which most of dynamic DNS uses), down to a few milliseconds.

Using NTP you can ensure that all computers in a local network are synced down bellow a millisecond or a few milliseconds over internet regarding the correct UTC time.

With that in mind, even an submillisecond call to a cache server in the local network (like Redis) can also be cached in local memory for nanosecond response time.

However, there is a thing called Leap Second which makes this kind of aggressive millisecond caching very hard, as the reference clock can either jump one second ahead or behind the current clock.

The difference between 1s or -1s could mean that the current value that we thing is correct is not the more recent value or a value that is still correct are treated as if it is already too old, causing the system to query the source of true too often, slowing down the system or even crashing it.