Vivek Sharma created AMBARI-21364: ------------------------------------- Summary: DB consistency checker throws errors for missing 'parquet-logging' and 'product-info' configs after Ambari upgrade Key: AMBARI-21364 URL: https://issues.apache.org/jira/browse/AMBARI-21364 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.5.2 Reporter: Vivek Sharma Priority: Blocker Fix For: 2.5.2
*STR* # Deployed cluster with Ambari version: 2.5.1.0-159 and HDP version: 2.6.1.0-129 # Upgrade Ambari to 2.5.2.0-74 (hash: fd30644590991deb41241454d6e9091ed7a38e92) # Run "ambari-server start" {code} root@ctr-e133-1493418528701-156570-01-000005:/hwqe/hadoopqe# ambari-server restart Using python /usr/bin/python Restarting ambari-server Waiting for server stop... Ambari Server stopped Ambari Server running with administrator privileges. Organizing resource files at /var/lib/ambari-server/resources... Ambari database consistency check started... Server PID at: /var/run/ambari-server/ambari-server.pid Server out at: /var/log/ambari-server/ambari-server.out Server log at: /var/log/ambari-server/ambari-server.log Waiting for server start.................. DB configs consistency check failed. Run "ambari-server start --skip-database-check" to skip. You may try --auto-fix-database flag to attempt to fix issues automatically. If you use this "--skip-database-check" option, do not make any changes to your cluster topology or perform a cluster upgrade until you correct the database consistency issues. See /var/log/ambari-server/ambari-server-check-database.log for more details on the consistency issues. ERROR: Exiting with exit code -1. REASON: Ambari Server java process has stopped. Please check the logs for more information. {code} DB log: ambari-server-check-database.log {code} 2017-06-27 13:51:38,743 INFO - Executing query 'GET_SERVICES_WITH_CONFIGS' 2017-06-27 13:51:38,748 INFO - Comparing service configs from stack with configs that we got from db 2017-06-27 13:51:38,748 INFO - Getting services from metainfo 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / KAFKA 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / PIG 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / ZEPPELIN 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / LOGSEARCH 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / MAPREDUCE2 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / SLIDER 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / HIVE 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / TEZ 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / HBASE 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / OOZIE 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / FLUME 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / MAHOUT 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / HDFS 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / DRUID 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / AMBARI_METRICS 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / SPARK 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / SMARTSENSE 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / AMBARI_INFRA 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / YARN 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / FALCON 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / SPARK2 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / ZOOKEEPER 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / ATLAS 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / SQOOP 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / STORM 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / KNOX 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / KERBEROS 2017-06-27 13:51:38,749 INFO - Comparing required service configs from stack with mapped service configs from db 2017-06-27 13:51:38,751 ERROR - Required config(s): product-info is(are) not available for service SMARTSENSE with service config version 2 in cluster cl1 2017-06-27 13:51:38,751 ERROR - Required config(s): parquet-logging is(are) not available for service HIVE with service config version 7 in cluster cl1 2017-06-27 13:51:38,751 INFO - Getting services which has mapped configs which are not selected in clusterconfigmapping 2017-06-27 13:51:38,755 INFO - Checking Topology tables 2017-06-27 13:51:38,757 INFO - Checking for tables with large physical size 2017-06-27 13:51:38,759 INFO - The database table host_role_command is currently 16.703 MB and is within normal limits (3000.000) 2017-06-27 13:51:38,760 INFO - The database table execution_command is currently 12.516 MB and is within normal limits (3000.000) 2017-06-27 13:51:38,761 INFO - The database table stage is currently 0.109 MB and is within normal limits (3000.000) 2017-06-27 13:51:38,761 INFO - The database table request is currently 0.078 MB and is within normal limits (3000.000) 2017-06-27 13:51:38,762 INFO - The database table alert_history is currently 0.563 MB and is within normal limits (3000.000) 2017-06-27 13:51:38,762 INFO - ******************************* Check database completed ******************************* {code} Looks like the issue is because the DB query does not return 'parquet-logging' and 'product-info' in type_name, while they exist in the respective metainfo files. See attached SQL query output and below: {code} root@ctr-e133-1493418528701-156570-01-000005:/var/lib/ambari-server/resources/common-services/HIVE/0.12.0.2.0# cat /var/lib/ambari-server/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml | grep -C2 parquet <configFile> <type>env</type> <fileName>parquet-logging.properties</fileName> <dictionaryName>parquet-logging</dictionaryName> </configFile> </configFiles> -- <config-type>webhcat-site</config-type> <config-type>webhcat-env</config-type> <config-type>parquet-logging</config-type> <config-type>ranger-hive-plugin-properties</config-type> <config-type>ranger-hive-audit</config-type> {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)