Couldn't the guests somehow inherit the system time of the host ?
It seems kind of pointless to run the same daemon to get the same results on the same machine multiple times but I didn't find anything related to time when reading KVM or Xen articles. My understanding is that the guest gets the host time on boot but that it might then drift apart. Is that correct ?
In a perfect world, your VM guests would keep perfect time, or at least as perfect as the host provides. Unfortunately we don't live in a perfect world.
Based on my experience with virtually every hypervisor known to man, I always run an NTP client in virtual machines, without exception. My usual setup is ntpd with the -g option, or ntpdate starting right before it for old systems, to step the clock (which may be far out of sync at system boot).
KVM has nearly the perfect setup, with its paravirtualized realtime clock; guests with the appropriate driver (all recent Linux, at least) will keep time as well as the host. But still things go wrong here: For instance, the host may not be running NTP, the host may have an incorrect time zone set, the host's clock may just be plain wrong, etc.
VMware and Hyper-V fall in the middle. Each has a tool meant to be run on the guest which syncs the clock with the host periodically, but again, this is vulnerable to any existing problems with the host clock.
Guests on my test Hyper-V server also exhibited some strange behavior: even with integration services, the guest clock would drift faster than 500 ppm, preventing ntpd from working (it considers the clock insane if it drifts faster than this). I had to switch these guests to chrony, which allows this value to be adjusted.
Xen is the worst in this respect; it has absolutely no synchronization and running NTP in the guests is pretty much required. (I am told that very recent versions of Xen have some sort of synchronization but haven't personally worked with it yet.)
Things just get worse if the host hypervisor isn't under your control, such as a public cloud. You are at the mercy of the provider with respect to the host clock, and if they aren't diligent in keeping it synchronized, you lose.
With all that, running NTP clients in your virtual machines is pretty much required if you need even a semi-accurate clock. NB: If you run Windows virtual machines, get a third party NTP client that adjusts the clock continuously; the poor excuse for a client that comes with Windows only adjusts the clock once a week, which is utterly ridiculous.
This is correct. It should be noted that time not only "can" drift away, but will drift away due to the fact that intervals between timer interrupts (which timekeeping on OS is often based on) are stretched and compressed as the hypervisor would see fit.
A commonly taken workaround across most virtualization platforms (Hyper-V integration services, VMWare tools) is running a daemon on the guest which periodically synchronizes clocks with the VM host. As noted by Hauke in a comment to your question, KVM additionally provides a paravirtualized clock which would need the respective driver loaded on the guest OS to work.
Further reading:Timekeeping in VMWare Virtual Machines (vmware.com)
KVM guests clock synchronisation (s19n.net)
I would recommend to use NTP because it is well known and has been around a long time. Adjusting the clock is not trivial. NTP has solved this issue.
The official line of VMware is to use one mechanism, with NTP preferred because it is more fine-grained and take smaller steps to adjust the time. The internal VMware solution takes bigger steps. When you run both they could fight each other. With the internal VMware solution taking a big step and then NTP adjusting it and taking it back a bit.
In practice however we run both at the same time and I haven't seen a problem yet.
$ ntpq
ntpq> peers
remote refid st t when poll reach delay offset jitter
==============================================================================
something.org 172.2.1.5 2 u 57 64 377 1.597 -2.409 5.952
$ vmware-toolbox-cmd timesync status
Enabled
$ vmware-toolbox-cmd help timesync
timesync: functions for controlling time synchronization on the guest OS
Usage: vmware-toolbox-cmd timesync <subcommand>
Subcommands:
enable: enable time synchronization
disable: disable time synchronization
status: print the time synchronization status
I wonder why this aspect wasn't mentioned in the answers so far:
VMs give you virtual machines, not being related to any other machine (that's the theory), so you could even have the freedom to set any VM to some arbitrary time.
That could be the reason why some environments do not sync the VMs time periodically to the host. (Some systems that do, actually do a poor job with time jumping back and forth noticeably)
Another point is boot time of a VM: You do not really expect the date to be the start of the epoch, like Jauary 1st 1970. Instead you expect the boot time to "continue" where you left the VM (possibly requiring a virtual RTC chip). So if you set your clock ahead of time by (let's say) two hours, and you reboot, you do not really expect boot time to jump back two hours, do you? Again not all environments do a good job there, however.
So in summary:
If you want all VMs to have the correct time, you can either synchronize each using NTP (and disable any other mechanism). The advantage is that NTP even allows to monitor the time being sync'ed correctly.
As an alternative optionally sync the VM host's clock, then sync the VMs from the host's clock using some mechanism the VM environment provides.
Final remark:
With massively over-committing CPUs, any VM will have a hard time to keep the correct time. Imagine your host has four CPUs, and you configured four VMs with two CPUs each. Then every VM runs two processes that read the CPU's cycle counter (TSC) at full speed. Which values should they see, and how could the VM host help doing so?
