[ https://issues.apache.org/jira/browse/AMBARI-21580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16107659#comment-16107659 ]
Hudson commented on AMBARI-21580: --------------------------------- FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #7827 (See [https://builds.apache.org/job/Ambari-trunk-Commit/7827/]) AMBARI-21580 - Replace Hard Coded stack-select Structures (jhurley: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=2bab2159ba9f17a0fcdefd20c8ab401c0042fdb9]) * (edit) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py * (edit) ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py * (edit) ambari-server/src/main/resources/common-services/SPARK/2.2.0/package/scripts/spark_client.py * (edit) ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py * (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/rest_api.py * (edit) ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/ranger_admin.py * (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py * (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/zkfc_slave.py * (edit) ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/hbase_client.py * (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py * (edit) ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/ranger_usersync.py * (edit) ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py * (edit) ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/pig_client.py * (edit) ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop_client.py * (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat_client.py * (edit) ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py * (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/ui_server.py * (edit) ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_master.py * (edit) ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_client.py * (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/drpc_server.py * (edit) ambari-server/src/main/resources/common-services/PIG/0.16.1.3.0/package/scripts/pig_client.py * (edit) ambari-server/src/main/resources/common-services/SQOOP/1.4.4.3.0/package/scripts/sqoop_client.py * (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/mapreduce2_client.py * (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_client.py * (edit) ambari-server/src/test/python/stacks/2.1/STORM/test_storm_nimbus_prod.py * (edit) ambari-server/src/main/resources/common-services/SLIDER/0.60.0.2.2/package/scripts/slider_client.py * (edit) ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py * (edit) ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py * (edit) ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/service_check.py * (edit) ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py * (edit) ambari-server/src/test/python/stacks/2.3/SPARK/test_spark_thrift_server.py * (edit) ambari-server/src/test/python/stacks/2.0.6/FLUME/test_flume.py * (edit) ambari-server/src/test/python/stacks/2.1/HIVE/test_hive_metastore.py * (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/journalnode.py * (edit) contrib/management-packs/odpi-ambari-mpack/src/main/resources/stacks/ODPi/2.0/hooks/after-INSTALL/scripts/shared_initialization.py * (edit) ambari-common/src/main/python/resource_management/libraries/functions/stack_tools.py * (edit) ambari-server/src/main/resources/common-services/SQOOP/1.4.4.3.0/package/scripts/service_check.py * (edit) ambari-server/src/test/python/stacks/2.0.6/PIG/test_pig_client.py * (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py * (edit) ambari-server/src/test/python/stacks/2.1/STORM/test_storm_supervisor.py * (edit) ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py * (add) ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_select_packages.json * (edit) ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py * (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py * (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/namenode.py * (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/application_timeline_server.py * (edit) ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/slider_client.py * (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py * (edit) ambari-server/src/test/python/stacks/2.1/STORM/test_storm_ui_server.py * (edit) ambari-server/src/main/resources/common-services/RANGER/1.0.0.3.0/package/scripts/ranger_admin.py * (edit) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py * (edit) ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py * (edit) ambari-server/src/test/python/stacks/2.2/SPARK/test_job_history_server.py * (edit) ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/hive_server_interactive.py * (edit) ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/hive_metastore.py * (edit) ambari-server/src/test/python/stacks/2.0.6/HIVE/test_webhcat_server.py * (edit) ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/accumulo_client.py * (edit) ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.3.0/package/scripts/master.py * (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py * (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/supervisor.py * (edit) ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/hive_server.py * (edit) ambari-server/src/test/python/stacks/utils/RMFTestCase.py * (edit) ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume_handler.py * (edit) ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py * (edit) ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms_server.py * (edit) ambari-server/src/main/resources/common-services/RANGER/1.0.0.3.0/package/scripts/ranger_usersync.py * (edit) ambari-server/src/main/resources/common-services/KAFKA/0.10.0.3.0/package/scripts/kafka_broker.py * (edit) ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/spark_thrift_server.py * (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/supervisor_prod.py * (edit) ambari-server/src/test/python/stacks/2.2/SLIDER/test_slider_client.py * (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/datanode.py * (edit) ambari-server/src/main/resources/common-services/SPARK/2.2.0/package/scripts/job_history_server.py * (edit) ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hcat_client.py * (edit) ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/accumulo_script.py * (edit) ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/upgrade.py * (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/nimbus_prod.py * (edit) ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py * (edit) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/params.py * (edit) ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/phoenix_queryserver.py * (edit) ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_regionserver.py * (edit) ambari-server/src/test/python/stacks/2.0.6/SQOOP/test_sqoop.py * (edit) contrib/management-packs/odpi-ambari-mpack/src/main/resources/stacks/ODPi/2.0/hooks/after-INSTALL/scripts/hook.py * (edit) ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/druid_node.py * (edit) ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/job_history_server.py * (edit) contrib/management-packs/hdf-ambari-mpack/src/main/resources/stacks/HDF/2.0/hooks/after-INSTALL/scripts/shared_initialization.py * (edit) ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/mahout_client.py * (edit) ambari-server/src/main/resources/stacks/HDP/3.0/configuration/cluster-env.xml * (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_client.py * (edit) ambari-server/src/test/python/stacks/2.0.6/HBASE/test_phoenix_queryserver.py * (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/nfsgateway.py * (edit) ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_client.py * (edit) ambari-common/src/main/python/resource_management/libraries/script/script.py * (edit) ambari-server/src/test/python/stacks/2.1/STORM/test_storm_drpc_server.py * (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py * (edit) ambari-server/src/test/python/stacks/2.0.6/HDFS/test_hdfs_client.py * (edit) ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py * (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/supervisor_prod.py * (edit) ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py * (edit) ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py * (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/pacemaker.py * (edit) ambari-server/src/test/python/stacks/2.0.6/HDFS/test_nfsgateway.py * (edit) ambari-server/src/test/python/stacks/2.1/YARN/test_apptimelineserver.py * (edit) ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/upgrade.py * (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/yarn_client.py * (edit) ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml * (edit) ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/job_history_server.py * (edit) ambari-server/src/main/resources/common-services/SPARK/2.2.0/package/scripts/livy_server.py * (edit) ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_client.py * (edit) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/after-INSTALL/scripts/hook.py * (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/drpc_server.py * (edit) ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py * (edit) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/after-INSTALL/scripts/shared_initialization.py * (edit) ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_server.py * (edit) ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/hbase_master.py * (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/hdfs_client.py * (edit) ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_client.py * (edit) ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py * (edit) ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py * (edit) ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java * (edit) ambari-server/src/main/resources/common-services/KNOX/0.5.0.3.0/package/scripts/knox_gateway.py * (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/snamenode.py * (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/nimbus.py * (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/snamenode.py * (edit) ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java * (edit) ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_client.py * (edit) ambari-server/src/test/python/stacks/2.1/STORM/test_storm_nimbus.py * (edit) ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka_broker.py * (add) ambari-server/src/main/resources/stacks/HDP/3.0/properties/stack_select_packages.json * (edit) ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/superset.py * (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py * (edit) ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/hive_client.py * (edit) ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/webhcat_server.py * (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/pacemaker.py * (edit) ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/hbase_regionserver.py * (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py * (edit) ambari-server/src/test/python/stacks/2.3/MAHOUT/test_mahout_client.py * (edit) ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py * (edit) ambari-server/src/test/python/stacks/2.1/STORM/test_storm_supervisor_prod.py * (edit) contrib/management-packs/hdf-ambari-mpack/src/main/resources/stacks/HDF/2.0/hooks/after-INSTALL/scripts/hook.py * (edit) ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/atlas_client.py * (edit) ambari-server/src/test/python/stacks/2.2/SPARK/test_spark_client.py * (edit) ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_client.py * (edit) ambari-server/src/main/resources/common-services/ATLAS/0.7.0.3.0/package/scripts/metadata_server.py * (edit) ambari-server/src/main/resources/common-services/RANGER_KMS/1.0.0.3.0/package/scripts/kms_server.py * (edit) ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py * (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py * (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/historyserver.py * (edit) ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py * (edit) ambari-server/src/main/resources/common-services/RANGER_KMS/1.0.0.3.0/package/scripts/upgrade.py * (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py * (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/ui_server.py * (edit) ambari-server/src/main/resources/common-services/OOZIE/4.2.0.3.0/package/scripts/oozie_client.py * (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/supervisor.py * (edit) ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_client.py * (edit) ambari-server/src/main/resources/common-services/TEZ/0.9.0.3.0/package/scripts/tez_client.py * (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_yarn_client.py * (edit) ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/package/scripts/zookeeper_client.py * (edit) ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/livy2_server.py * (edit) ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py * (edit) ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py * (edit) ambari-server/src/main/resources/common-services/SPARK/2.2.0/package/scripts/spark_thrift_server.py * (edit) ambari-server/src/main/java/org/apache/ambari/server/upgrade/FinalUpgradeCatalog.java * (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/rest_api.py * (edit) ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/spark_client.py * (edit) ambari-server/src/main/resources/common-services/RANGER/1.0.0.3.0/package/scripts/upgrade.py * (edit) ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/hcat_client.py * (edit) ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/ranger_tagsync.py * (edit) ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py * (edit) ambari-server/src/test/python/stacks/2.2/ACCUMULO/test_accumulo_client.py * (edit) ambari-server/src/main/resources/common-services/RANGER/1.0.0.3.0/package/scripts/ranger_tagsync.py * (edit) ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_thrift_server.py * (edit) ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py * (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/nimbus_prod.py * (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py * (edit) ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/upgrade.py * (edit) ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py * (edit) ambari-server/src/main/resources/common-services/ATLAS/0.7.0.3.0/package/scripts/atlas_client.py * (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_server.py * (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/nimbus.py * (edit) ambari-server/src/main/resources/common-services/OOZIE/4.2.0.3.0/package/scripts/oozie_server.py * (edit) ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/phoenix_queryserver.py * (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/resourcemanager.py * (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/nfsgateway.py * (edit) ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/package/scripts/zookeeper_server.py * (edit) ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_regionserver.py * (edit) ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_client.py > Replace Hard Coded stack-select Structures > ------------------------------------------ > > Key: AMBARI-21580 > URL: https://issues.apache.org/jira/browse/AMBARI-21580 > Project: Ambari > Issue Type: Task > Affects Versions: 2.5.3 > Reporter: Jonathan Hurley > Assignee: Jonathan Hurley > Priority: Critical > Fix For: 2.5.3 > > > Currently, both the stack-select and conf-select python files have hard-coded > structures inside of them which contain mappings for things like > configuration directories and stack-select component names. > This is mainly used for pairing the Ambari role name (defined from > metainfo.xml) with the {{stack-select}} package name. There are places, like > {{params.py}} where we have not yet entered the target python file (like > {{AccumuloScript}}) where we could get this information. Some components need > this to be able to build things like the Hadoop conf dir: > {code:title=stack_select.py} > SERVER_ROLE_DIRECTORY_MAP = { > 'ACCUMULO_MASTER' : 'accumulo-master', > 'ACCUMULO_MONITOR' : 'accumulo-monitor', > 'ACCUMULO_GC' : 'accumulo-gc', > 'ACCUMULO_TRACER' : 'accumulo-tracer', > 'ACCUMULO_TSERVER' : 'accumulo-tablet', > 'ATLAS_SERVER' : 'atlas-server', > 'FLUME_HANDLER' : 'flume-server', > 'FALCON_SERVER' : 'falcon-server', > ... > {code} > With the coming of management packs replacing stacks, we can no longer hard > code this in Python. My suggestion is to begin moving this data into a > property of some sort. Today, {{cluster-env}} exists and would be the only > place for it (where the stack-feature tools are now). However, I believe that > {{cluster-env}} is also going away and being replaced with something similar > to "Cluster Settings". > In any event, this Jira is to track the work needed to replace this logic. -- This message was sent by Atlassian JIRA (v6.4.14#64029)