Robin Zuo created AMBARI-25987:
----------------------------------
Summary: Enable Spark to use hadoop native libraries for better
performance
Key: AMBARI-25987
URL: https://issues.apache.org/jira/browse/AMBARI-25987
Project: Ambari
Issue Type: Improvement
Components: ambari-server
Reporter: Robin Zuo
Currently, when run spark tasks, spark will report cannot load hadoop native
libraries.
Also, the hadoop short-circuit read feature cannot be applied. For better
performance, add hadoop native libraries support.
Some WARN log as following:
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
23/08/15 13:43:22 WARN NativeCodeLoader: Unable to load native-hadoop library
for your platform... using builtin-java classes where applicable
...
23/08/15 13:43:25 INFO Client: Requesting a new application from cluster with 3
NodeManagers
23/08/15 13:43:26 WARN DomainSocketFactory: The short-circuit local reads
feature cannot be used because libhadoop cannot be loaded.
23/08/15 13:43:26 INFO Configuration: resource-types.xml not found
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
