Well, Same as the title, NTP client failed to  synchronize time from NTP server

it's also possible that it happens on real real host, more seems more likely happen on VM servers.

It just happened to me, 3 VM storage nodes went reboot(still working with Vendor for root cause), all of them had system clock problem after abnormal reboot.

I closely checked /var/log/messages, it seems to me that network interface was up when NTP tried to get server time, but failed.

Here is the log:

Jan 20 18:12:24 d04 ntpd[1797]: ntpd This email address is being protected from spambots. You need JavaScript enabled to view it. Sat Dec 20 19:00:16 UTC 2014 (1)
Jan 20 18:12:24 d04 ntpd[1798]: proto: precision = 0.118 usec
Jan 20 18:12:24 d04 ntpd[1798]: 0.0.0.0 c01d 0d kern kernel time sync enabled
Jan 20 18:12:24 d04 ntpd[1798]: Listen and drop on 0 v4wildcard 0.0.0.0 UDP 123
Jan 20 18:12:24 d04 ntpd[1798]: Listen normally on 1 lo 127.0.0.1 UDP 123
Jan 20 18:12:24 d04 ntpd[1798]: Listen normally on 2 eth2 1.1.1.2 UDP 123
Jan 20 18:12:24 d04 ntpd[1798]: peers refreshed
Jan 20 18:12:24 d04 ntpd[1798]: Listening on routing socket on fd #19 for interface updates
Jan 20 18:12:24 d04 ntpd[1798]: 0.0.0.0 c016 06 restart
Jan 20 18:12:24 d04 ntpd[1798]: 0.0.0.0 c012 02 freq_set kernel -5.655 PPM
Jan 20 18:12:27 dpool04 ntpd[1798]: 0.0.0.0 c515 05 clock_sync
Jan 20 18:15:40 dpool04 ntpd[1798]: 0.0.0.0 0617 07 panic_stop +28800 s; set clock manually within 1000 s.

The red lines show that the system time it got from VM host is too far behind the server clock time(>1000s), actually, to my case, it was 8 hours.

Solution:

What i did was to add a script to /etc/rc.local

 

cat fix_clock.sh 

#!/bin/sh

/etc/init.d/ntpd stop >/dev/null 2>&1
/usr/sbin/ntpdate -U ntp -s -b time1 time2 time3 >/dev/null 2>&1
/usr/sbin/ntpdate -U ntp -s -b time1 time2 time3 >/dev/null 2>&1
/sbin/hwclock --utc --systohc >/dev/null 2>&1
/etc/init.d/ntpd start >/dev/null 2>&1

 

 

It looks not nice for long run, but it works for now.

Googling tells me that there is another option to play with(I havn't got chance to test it yet, no access to VM host):

Solution2:

I copied from the link below:

http://stackoverflow.com/questions/26005056/synchronizing-clock-time-in-host-and-guest-under-kvm

  1. Set the hwclock on the host and to use UTC time. This is done with the --utc option of the hwclock command. I run the following command on my host OS:

    hwclock --utc --set --date="time-string"

  2. Tell CentOS that the hwclock is using UTC via the file /etc/adjtime. For example, you could initialize this file using

    echo -e "0.0 0 0.0\n0\n\nUTC" >/etc/adjtime

Create this file on both the host and your guest VMs. I create the file on my guests before I boot them for the first time by directly accessing the guest file system from the host.

  1. Set the time zone you want for your system time. Again, do this on both your host and your guests:

    ln -sf /usr/share/zoneinfo/time-zone /etc/localtime
    echo "ZONE=time-zone" >/etc/sysconfig/clock
    export TZ=time-zone

where time-zone is a standard CentOS time zone string, for example "US/Pacific".

  1. Set the system time on your host based on the hwclock. The --utc option is needed to tell CentOS that the hwclock is in UTC time. It will take the UTC time and set your system time based on the TZ environment variable:

    hwclock --utc --hctosys

  2. The steps above are all done once, when you are configuring your host and guests. To keep time synced on all your servers after they are up and running you'll want to configure NTP on your host and guests.

 

 

Comments powered by CComment