----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/42051/#review114925 -----------------------------------------------------------
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py (line 50) <https://reviews.apache.org/r/42051/#comment175757> When did spark get added. Will get_hdp_version work for all versions of HDP? ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_service.py (line 37) <https://reviews.apache.org/r/42051/#comment175758> This may have RU implications. Can you discuss with Alejandro/Nate? ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml (line 25) <https://reviews.apache.org/r/42051/#comment175759> Existing installations will still have these properties - is that OK? ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml (line 70) <https://reviews.apache.org/r/42051/#comment175760> Ambari upgrade will automatically add these properties for already installed cluster. Is that OK? can you test one upgrade scenario where spark is already installed using an older version of Ambari? - Sumit Mohanty On Jan. 16, 2016, 1:13 a.m., Jeff Zhang wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/42051/ > ----------------------------------------------------------- > > (Updated Jan. 16, 2016, 1:13 a.m.) > > > Review request for Ambari, Alejandro Fernandez, Saisai Shao, and Sumit > Mohanty. > > > Bugs: AMBARI-14561 > https://issues.apache.org/jira/browse/AMBARI-14561 > > > Repository: ambari > > > Description > ------- > > This is for HDP stack, in HDP stack, we'd like to do following changes for > spark thrift server > - Remove advanced properties except queue name > - Use yarn-client mode > - Enable Dynamic Resource Allocation > - Enable Fair scheduling policy > > > Diffs > ----- > > > ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py > c3ffc7b > > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py > 68240bd > > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_spark.py > 4b38572 > > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_service.py > d4c6732 > > ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-defaults.xml > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-fairscheduler.xml > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml > PRE-CREATION > ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml > 6c0e393 > > Diff: https://reviews.apache.org/r/42051/diff/ > > > Testing > ------- > > This patch doesn't solve the issue completly. The weird thing is that the > configuration in HDP 2.3's spark-thrift-sparkconf.xml will be appended to > HDP-2.4's spark-thrift-sparkconf.xml. And what I want is remove some > configuration in 2.3. And actually I make Spark 1.6 of HDP 2.4 extend Spark > 1.2 in common-services, not sure why it still use spark-thrift-sparkconf.xml > in HDP 2.3 > > > Thanks, > > Jeff Zhang > >
