We have a blueprints defined cluster that has an accepted topology: ambari-server.log" 6839L, 998638C 16 Oct 2015 01:31:40,374 INFO [pool-2-thread-1] AmbariContext:375 - Config type hdfs-site is resolved in the cluster config. 16 Oct 2015 01:31:40,374 INFO [pool-2-thread-1] AmbariContext:375 - Config type hadoop-env is resolved in the cluster config. 16 Oct 2015 01:31:40,375 INFO [pool-2-thread-1] AmbariContext:375 - Config type hive-site is resolved in the cluster config. 16 Oct 2015 01:31:40,375 INFO [pool-2-thread-1] AmbariContext:375 - Config type yarn-site is resolved in the cluster config. 16 Oct 2015 01:31:40,375 INFO [pool-2-thread-1] AmbariContext:375 - Config type oozie-site is resolved in the cluster config. 16 Oct 2015 01:31:40,376 INFO [pool-2-thread-1] AmbariContext:375 - Config type webhcat-site is resolved in the cluster config. 16 Oct 2015 01:31:40,376 INFO [pool-2-thread-1] AmbariContext:375 - Config type cluster-env is resolved in the cluster config. 16 Oct 2015 01:31:40,376 INFO [pool-2-thread-1] AmbariContext:375 - Config type mapred-site is resolved in the cluster config. 16 Oct 2015 01:31:40,377 INFO [pool-2-thread-1] AmbariContext:375 - Config type core-site is resolved in the cluster config. 16 Oct 2015 01:31:40,377 INFO [pool-2-thread-1] AmbariContext:375 - Config type oozie-env is resolved in the cluster config. 16 Oct 2015 01:31:40,377 INFO [pool-2-thread-1] AmbariContext:381 - All required configuration types are in the* TOPOLOGY_RESOLVED state. Blueprint deployment can now continue.* 16 Oct 2015 01:31:40,377 INFO [pool-2-thread-1] TopologyManager:558 - TopologyManager.ConfigureClusterTask: Exiting
We then have a few messages that seem healthy: 16 Oct 2015 01:32:00,945 INFO [Thread-25] AbstractPoolBackedDataSource:462 - Initializing c3p0 pool... 16 Oct 2015 01:32:00,985 INFO [Thread-25] JobStoreTX:861 - Freed 0 triggers from 'acquired' / 'blocked' state. 16 Oct 2015 01:32:00,992 INFO [Thread-25] JobStoreTX:871 - Recovering 0 jobs that were in-progress at the time of the last shut-down. 16 Oct 2015 01:32:00,992 INFO [Thread-25] JobStoreTX:884 - Recovery complete. 16 Oct 2015 01:32:00,998 INFO [Thread-25] QuartzScheduler:575 - Scheduler ExecutionScheduler_$_NON_CLUSTERED started. 16 Oct 2015 01:33:34,658 INFO [qtp-client-72] JvmInformation:446 - Detected JVM data model settings of: 64-Bit HotSpot JVM with Compressed OOPs and Concurrent Mark-and-Sweep GC 16 Oct 2015 01:33:34,928 INFO [qtp-client-72] TimelineMetricCacheProvider:73 - .. However we then have a series of LONG (multiple minutes) delays, in between which there are some "Looking for keyname <foo>" : 016.novalocal/10.103.177.55-9ac6f6e0-9d4a-447e-b3ee-53926d6e16bf 16 Oct 2015 01:33:35,088 INFO [qtp-client-76] PersistKeyValueService:82 - Looking for keyName CLUSTER_CURRENT_STATUS 16 Oct 2015 01:33:36,272 INFO [qtp-client-72] PersistKeyValueService:82 - Looking for keyName user-pref-admin-dashboard 16 Oct 2015 01:33:43,550 INFO [qtp-client-72] PersistKeyValueService:82 - Looking for keyName time-range-service-HDFS 16 Oct 2015 01:33:44,995 WARN [pool-7-thread-1] JMXPropertyProvider:202 - Unable to get JMX metrics. No host name for NAMENODE 16 Oct 2015 01:34:00,930 INFO [qtp-client-72] PersistKeyValueService:82 - Looking for keyName admin-settings-show-bg-admin 16 Oct 2015 01:34:01,010 INFO [qtp-client-76] PersistKeyValueService:82 - Looking for keyName hostPopup-pagination-displayLength-admin 16 Oct 2015 01:36:34,123 INFO [qtp-client-82] PersistKeyValueService:82 - Looking for keyName time-range-service-AMBARI_METRICS 16 Oct 2015 01:36:42,903 INFO [qtp-client-125] PersistKeyValueService:82 - Looking for keyName time-range-service-HDFS 16 Oct 2015 01:36:44,138 WARN [pool-7-thread-35] JMXPropertyProvider:202 - Unable to get JMX metrics. No host name for NAMENODE 16 Oct 2015 01:36:57,590 INFO [qtp-client-75] PersistKeyValueService:82 - Looking for keyName admin-settings-show-bg-admin 16 Oct 2015 01:48:10,901 INFO [qtp-client-192] PersistKeyValueService:82 - Looking for keyName admin-settings-show-bg-admin 16 Oct 2015 01:53:06,487 INFO [qtp-client-214] PersistKeyValueService:82 - Looking for keyName user-pref-admin-dashboard After that last message there was another 15 minute delay and then nothing (another sixty minutes has passed).. How to determine what is happening here and why?
