You can add resolvers on SBT using resolvers += "Sonatype OSS Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots"
On Thu, Feb 26, 2015 at 4:09 PM, MEETHU MATHEW <meethu2...@yahoo.co.in> wrote: > Hi, > > I am not able to read from HDFS(Intel distribution hadoop,Hadoop version > is 1.0.3) from spark-shell(spark version is 1.2.1). I built spark using the > command > mvn -Dhadoop.version=1.0.3 clean package and started spark-shell and read > a HDFS file using sc.textFile() and the exception is > > WARN hdfs.DFSClient: Failed to connect to /10.88.6.133:50010, add to > deadNodes and continuejava.net.SocketTimeoutException: 120000 millis > timeout while waiting for channel to be ready for read. ch : > java.nio.channels.SocketChannel[connected local=/10.88.6.131:44264 > remote=/10.88.6.133:50010] > > The same problem is asked in the this mail. > RE: Spark is unable to read from HDFS > <http://mail-archives.us.apache.org/mod_mbox/spark-user/201309.mbox/%3cf97adee4fba8f6478453e148fe9e2e8d3cca3...@hasmsx106.ger.corp.intel.com%3E> > > > > > > > RE: Spark is unable to read from HDFS > <http://mail-archives.us.apache.org/mod_mbox/spark-user/201309.mbox/%3cf97adee4fba8f6478453e148fe9e2e8d3cca3...@hasmsx106.ger.corp.intel.com%3E> > Hi, Thanks for the reply. I've tried the below. > View on mail-archives.us.apache.org > <http://mail-archives.us.apache.org/mod_mbox/spark-user/201309.mbox/%3cf97adee4fba8f6478453e148fe9e2e8d3cca3...@hasmsx106.ger.corp.intel.com%3E> > Preview by Yahoo > > > As suggested in the above mail, > *"In addition to specifying HADOOP_VERSION=1.0.3 in the > ./project/SparkBuild.scala file, you will need to specify the > libraryDependencies and name "spark-core" resolvers. Otherwise, sbt will > fetch version 1.0.3 of hadoop-core from apache instead of Intel. You can > set up your own local or remote repository that you specify" * > > Now HADOOP_VERSION is deprecated and -Dhadoop.version should be used. Can > anybody please elaborate on how to specify tat SBT should fetch hadoop-core > from Intel which is in our internal repository? > > Thanks & Regards, > Meethu M > -- [image: Sigmoid Analytics] <http://htmlsig.com/www.sigmoidanalytics.com> *Arush Kharbanda* || Technical Teamlead ar...@sigmoidanalytics.com || www.sigmoidanalytics.com