[ https://issues.apache.org/jira/browse/HADOOP-12988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun Singh reassigned HADOOP-12988: ----------------------------------- Assignee: Ali Bajwa Please let me know if any additional info is needed. We will get from customer/eng. Thx. > upgrading the HDP stack through Ambari (from 2.3 to 2.4) > -------------------------------------------------------- > > Key: HADOOP-12988 > URL: https://issues.apache.org/jira/browse/HADOOP-12988 > Project: Hadoop Common > Issue Type: Bug > Environment: SLES11 SP4 > Reporter: Arun Singh > Assignee: Ali Bajwa > > - Issue 3: When upgrading the HDP stack through Ambari (from 2.3 to 2.4), at > some point a YARN smokescreen test is performed. This smoke screen test > fails, as it is trying to call an API command using curl with the --negotiate > option. The problem is that on SUSE 11, the version of curl does not ship > with one that understands "--negotiate", grinding the whole upgrade process > to a halt. > > There are quite a few files in Ambari where this seems to be the case, > although I personally only encountered it during the YARN component: > /var/lib/ambari-server/resources/common-services/RANGER/0.4.0/package/scripts/service_check.py: > > Execute(format("curl -s -o /dev/null -w'%{{http_code}}' --negotiate -u: > -k {ranger_external_url}/login.jsp | grep 200"), > /var/lib/ambari-server/resources/common-services/SPARK/1.2.0.2.2/package/scripts/service_check.py: > > Execute(format("curl -s -o /dev/null -w'%{{http_code}}' --negotiate -u: > -khttp://{spark_history_server_host}:{spark_history_ui_port} | grep 200"), > /var/lib/ambari-server/resources/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py: > > get_app_info_cmd = "curl --negotiate -u : -ksL --connect-timeout " + > CURL_CONNECTION_TIMEOUT + " " + info_app_url > /var/lib/ambari-server/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py: > > smoke_cmd = format('curl --negotiate -u : -b ~/cookiejar.txt -c > ~/cookiejar.txt -s -o /dev/null -w > "%{{http_code}}"http://{metadata_host}:{metadata_port}/') > /var/lib/ambari-server/resources/common-services/HIVE/0.12.0.2.0/package/files/templetonSmoke.sh: > cmd="${kinitcmd}curl --negotiate -u : -s -w 'http_code <%{http_code}>' > $ttonurl/status 2>&1" > /var/lib/ambari-server/resources/common-services/HIVE/0.12.0.2.0/package/files/templetonSmoke.sh: > > cmd="${kinitcmd}curl --negotiate -u : -s -w 'http_code <%{http_code}>' > $ttonurl/status?user.name=$smoke_test_user 2>&1" > /var/lib/ambari-server/resources/common-services/HIVE/0.12.0.2.0/package/files/templetonSmoke.sh: > cmd="${kinitcmd}curl --negotiate -u : -s -w 'http_code <%{http_code}>' -d > \@${destdir}/show_db.post.txt $ttonurl/ddl 2>&1" > For example, in this file: > /var/lib/ambari-server/resources/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py > You will see the code as: > for rm_webapp_address in params.rm_webapp_addresses_list: > info_app_url = params.scheme + "://" + rm_webapp_address + > "/ws/v1/cluster/apps/" + application_name > get_app_info_cmd = "curl --negotiate -u : -ksL --connect-timeout " + > CURL_CONNECTION_TIMEOUT + " " + info_app_url > return_code, stdout, _ = get_user_call_output(get_app_info_cmd, > user=params.smokeuser, > > path='/usr/sbin:/sbin:/usr/local/bin:/bin:/usr/bin', > ) > > > There is no code checking whether RHEL vs SUSE, to run the correct usage of > curl. Or alternatively, there is no code to check for version of curl, and > run a "deprecated" version of the command as a fallback should it detect that > the installed curl does not support --negotiate. This is just blindly > assuming to work on SUSE 11 (or any version of curl). > > Information about the curl installed on the system: > hdplab02:~ # curl -V > curl 7.45.0 (x86_64-pc-linux-gnu) libcurl/7.45.0 OpenSSL/1.0.2d zlib/1.2.8 > Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp > smb smbs smtp smtps telnet tftp > Features: IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP UnixSockets > -- This message was sent by Atlassian JIRA (v6.3.4#6332)