[ https://issues.apache.org/jira/browse/AMBARI-20225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891049#comment-15891049 ]
Hudson commented on AMBARI-20225: --------------------------------- SUCCESS: Integrated in Jenkins build Ambari-trunk-Commit #6906 (See [https://builds.apache.org/job/Ambari-trunk-Commit/6906/]) AMBARI-20225. Take care of hive-interactive-site's (sshridhar: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=baefbcca0b60caac801f3addf0d015df23a5e96d]) * (edit) ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml * (edit) ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/tez-interactive-site.xml * (edit) ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py * (edit) ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/config-upgrade.xml * (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/service_advisor.py * (edit) ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py * (edit) ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.6.xml * (edit) ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml > Take care of hive-interactive-site's 'hive.tez.container.size' config during > HDP upgrade from 2.5 to 2.6 and update default value for > tez-interactive-site's config 'tez.am.resource.memory.mb'. > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > > Key: AMBARI-20225 > URL: https://issues.apache.org/jira/browse/AMBARI-20225 > Project: Ambari > Issue Type: Bug > Components: ambari-server > Affects Versions: trunk, 2.5.0 > Reporter: Swapan Shridhar > Assignee: Swapan Shridhar > Fix For: trunk, 2.5.0 > > Attachments: AMBARI-20225.patch > > > *Background :* > We are dealing with 2 specific configs here. > *(1).* 'tez.am.resource.memory.mb' for tez-interactive-site and > *(2).* 'hive.tez.container.size' for hive-interactive-site. > - 'tez.am.resource.memory.mb' already exists in tez-interactive-site from > Ambari-2.4 onwards and recently its default value had been changed to > 'SET_ON_FIRST_INVOCATION' in its hive-interacive-site. > - 'hive.tez.container.size' was inherited and used from Hive1/hive-site in > Ambari 2.4. Starting from Ambari 2.5, it was added as a config explicitly in > hive-interactive-site with a default value of 'SET_ON_FIRST_INVOCATION'. > The non-standard 'SET_ON_FIRST_INVOCATION' was introduced, so as to give a > hint in Stack Advisor (SA) call for the 1st time on Hive Server Interactive > ON (enable_hive_interactive config), so that config values for both can be > calculated and set for the 1st time. Subsequesnt calls in the lifecycle, will > just read the value set by SA or if later changed by user manually. > *Problem :* > 'SET_ON_FIRST_INVOCATION', being non-standard was breaking the code in > upgrade scenario. > *Fix:* > Removing the use of 'SET_ON_FIRST_INVOCATION' and adding code for correct > behavior of configs during HDP upgrade. > > *Possible scenarios:* > *1. Ambari upgrade from 2.4 - > 2.5 having HDP 2.5. Implying > 2.5/stack_advisor.py code would have been updated with new LLAP GA logic.* > - 'tez.am.resource.memory.mb' will already be there whether Hive Server > Interactive (HSI) is ON or OFF after Ambari upgrade. > - If HSI if OFF before the upgrade was done, nothing to do over there. > - If HSI on ON before AMbari upgrade, or is made ON after Ambari > upgrade, SA logic has been updated to take of reading > 'hive.tez.container.size' from hive-site as it wont exist in > hive-interactive-site at this point. It will show up in hive-interative-site > aftre the HDP upgrade to 2.6 > *2. Ambari 2.4 having HDP 2.5 -> Upgrade to Ambari 2.5 -> Upgraded to HDP 2.6* > - 'tez.am.resource.memory.mb' will already be there > - As part of HDP 2.5 upgrade to HDP 2.6, 'hive.tez.container.size' would > have got added, having current value picked from > hive-site/hive.tez.container.size. SA code carries a calculation logic for > config 'hive.tez.container.size', which is bound to yield different value > compared to hive-site/'hive.tez.container.size'. The config's value will get > updated/caluclated (1). when HSI is made ON (meaning > 'enabled_hive_interactive' config detected as **True** in > changed-configurations). > > *3. Ambari 2.5 with HDP 2.5 -> Upgraded to HDP 2.6* > - Both 'hive.tez.container.size' and 'tez.am.resource.memory.mb' (alreday > present at HDP 2.5 in hive-interactive-site) will exist after the HDP upgrade. > *Note* that a fresh value for 'hive.tez.container.size' and > 'tez.am.resource.memory.mb' is caluclated at below conditions: > (1). when HSI is made ON, meaning 'enabled_hive_interactive' config detected > as **True** in changed-configurations, or > (2). it's a cluster create operation, where HSI also is supposed to be > started. (BP scenario) -- This message was sent by Atlassian JIRA (v6.3.15#6346)