For a relatively large network (thousands of hosts) - what are the arguments for and against running a locally managed (pool of) NTP server(s) (perhaps periodically set via some public NTP server) and having all other hosts on the network use that (pool of) NTP server(s) versus having all hosts simply use public NTP servers directly, say via ntp.pool.org?
Aside from the pros and cons, What is typical best practice today?
The best practice is to run your own pool of NTP servers set to sync from public NTP servers. In the event that your organization was to lose internet access, you would not want your clocks to become skewed. Further, it is rude to set thousands of hosts to public servers when you could (and should) operate a mirror.
Finally, if you have a secure computing requirement, then you should operate your own independent NTP hosts. You would require special hardware for these systems to operate.
EDIT: Since there was discussion of it, here is some hardware:
Any hardware supporting PPS seems to work on a modern ntpd. This includes some GPS units, although this seems to be rare, at least as rare as serial GPS units are these days. There are hardware devices sold explicitly for this function, however, including one product called TSync-PCIe. According to the manufacturer's site:
The TSync-PCIe offers several configurations of a synchronized timecode reader/generator package offering flexibility and easy integration of precise timing into an embedded computing application. Choose from synchronization to IRIG (and other similar timecodes), GPS (internal or external receivers), or Precise Time Protocol (PTP/IEEE-1588v2). - Site Link: http://i564f.6o.to
Even on a small network I use a local NTP service, which itself updates from an external one. One reason is purely historical, dating back to when the only connection to the Internet was via dial-up modems. The other is that if the NTP service is wrong for any reason I would prefer all the machines to still be consistent, which is more likely to be the case if they all update from a single source.
Best practice, setup 2 (or more) NTP hosts at your location, peer them. Have them sync against at least 4 (preferably, up to 8) external servers from 0.pool.ntp.org to 3.pool.ntp.org. If you use more than 4 you should adjust the frequency that they poll the pool members.
Here's an edited version of my ntp.conf:
server 0.us.pool.ntp.org minpoll 8 maxpoll 14
server 1.us.pool.ntp.org minpoll 8 maxpoll 14
server 2.us.pool.ntp.org minpoll 8 maxpoll 14
server 3.us.pool.ntp.org minpoll 8 maxpoll 14
peer ntp2.example.com
driftfile /var/db/drift.ntp
logfile /var/log/ntp.log
logconfig +sysall +syncall
You can omit the minpoll and maxpoll arguments, I add them so I'm a bit lighter on those servers. The values are 2^n seconds, where n is the argument; those values are higher than the defaults (6 & 10) because I already poll 12 different servers between my three NTP hosts.
If you're very concerned with accuracy you might add the following as well:
server tick.usno.navy.mil prefer minpoll 10 maxpoll 16
This will poll the navy's atomic clock. Note the high poll times as they're fairly heavily loaded and have requested people take it easy on their server (actually a 3 node cluster).
As others have mentioned, for thousands of internal hosts, providing your own time servers is the way to go. For reasons such as (as other already mentioned):
As far as best practices:
From http://www.ntp.org/ntpfaq/NTP-s-config-adv.htm, here is a recommended structure for NTP only sources.
EDITED - per Paul Gear and comment after diagram on ntp.org website
1a 1b 1c 1d 1e 1f outside
..\ /...\../..\/..\.../.\./...............
2a ---p--- 2b ---p--- 2c inside (stratum 2 has many stratum 1 sources)
\ /|\ /|\ /|\ /
ntp clients
ORIGINAL diagram
1a 1b 1c 1d 1e 1f outside
. \ / ...... \ / ...... \ / ..............
2a ---p--- 2b ---p--- 2c inside
/|\ /|\ /|\
/ | \ / | \ / | \
3a 3b 3c 3e 3f 3g 3h 3i 3j
Key: 1 = stratum-1, 2 = stratum-2, 3 = stratum-3, p = peer
Additional information for setting up an NTP server is from http://www.pool.ntp.org/join/configuration.html . Examples being:
I think most large networks use a small pool of dedicated internal ntp servers. ntp traffic is pretty light so you probably don't need many servers to serve a large organization.
As with all network services, the advantage of running your own ntp servers is you get more control and get to make more decisions. For example, if you lose network connectivity to the outside world, your machines can continue to talk to your internal ntp server and you don't have to worry about them all having to reconnect to external servers.
If you have thousands of servers you should also consider running your own dedicated time server, for example off a gps device or via a dedicated atomic clock. I'm not sure how much that costs these days but it can't be expensive relative to the thousands of systems you are already supporting.. Then you have an accurate time service completely independent of your connection to the outside world.
Another point to consider is that running your own ntp servers is more polite. That way you have just a few machines making external requests as opposed to thousands. I'm sure the admins of the publicly accessible ntp servers out there would appreciate that. Plus it will reduce your external network traffic slightly (very slightly) which is probably a good thing.
Also if you run your own ntp servers you can tighten up your firewall a little bit since just a few machines are connecting to the outside on port 123 instead of lots of machines. That might be useful.
ntp is easy to set up and once you have it running it requires very little maintenance. Every company I've ever been involved with has set up it's own ntp servers and that has worked just fine.
Best practice in that case would be running your own NTP server - or a pool as necessary - and pulling from the NTP pool nearest you geographically. This reduces the load that the public facing NTP servers have to bear but will still give you a high amount of accuracy. If you require even more accuracy, you could pull from the Stratum 1 servers, but doing so increases the load that the pool has to bear, so you should only do this if you are willing to contribute a server to the pool.
A good reason for running your own NTP server(s) in a large network is making sure all your machines agree on the correct time. Having lots of systems with their own settings for external time servers (or all using different pool.ntp.org members) can lead to small differences in time on systems which may lead to problems.
The other good reason is that having your own NTP server(s) means synchronized time will stay available from a few (monitored!) servers when the outside link goes down or is saturated with traffic.
All my opinion as a timegeek.
