It took some digging, but I think I found it.
It's Hadoop code that's trying to get group information, which might not be
available if you use Kerberos:
cacheTimeout =
conf.getLong(CommonConfigurationKeys.HADOOP_SECURITY_GROUPS_CACHE_SECS,
5*60) * 1000;
public static final String HADOOP_SECURITY_GROUPS_CACHE_SECS =
"hadoop.security.groups.cache.secs";
The config is in *core-default.xml*, default value is 300 seconds (based on
code above):
http://hadoop.apache.org/docs/r2.0.3-alpha/hadoop-project-dist/hadoop-common/core-default.xml
Let me know if it helps...
-----
Madhu
https://www.linkedin.com/in/msiddalingaiah
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/why-is-Spark-0-9-1-context-creation-so-slow-on-my-OSX-laptop-tp5535p5706.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
