Hi Lewis, Many thanks for your answer. It is not easy for a beginner to download so many staffs. I am struggling with java and hadoop Now I have downloaded jdk1.8.0_71 from Oracle, it seems ok. But hadoop refuses to run properly! When I run hadoop the terminal shows: /usr/local/hadoop/hadoop-common-project/hadoop-common/src/main/bin/hadoop: line 27: /usr/local/hadoop/hadoop-common-project/hadoop-common/src/main/bin/../libexec/hadoop-config.sh: No such file or directory
I have downloaded hadoop 2.7.2 and followed the instruction on https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html#Fully-Distributed_Operation It seems that I cannot find etc/hadoop as the instruction described. In the .bashrc, the paths are difined as export JAVA_HOME=/usr/local/jdk1.8.0_71/ export PATH=$PATH:$JAVA_HOME/bin export HADOOP_INSTALL=/usr/local/hadoop #export PATH=$PATH:$HADOOP_INSTALL/bin #no bin file are found #export PATH=$PATH:$HADDOP_INSTALL/sbin #not sbin file are found export HADOOP_MAPRED_HOME=$HADOOP_INSTALL/hadoop-mapreduce-project/bin export HADOOP_COMMON_HOME=$HADOOP_INSTALL/hadoop-common-project/hadoop-common/src/main/bin export HADOOP_HDFS_HOME=$HADOOP_INSTALL/hadoop-hdfs-project/hadoop-hdfs/src/main/bin export YARN_HOME=$HADOOP_INSTALL/hadoop-yarn-project/hadoop-yarn/bin export HADOOP_COMMOM_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native export HADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib" The problem is that there are not bin and sbin folders under hadoop, the bin folders are distributed under different folders. When I go to the $HADOOP_COMMON_HOME and run hadoop, the terminal shows /usr/local/hadoop/hadoop-common-project/hadoop-common/src/main/bin/hadoop: line 27: /usr/local/hadoop/hadoop-common-project/hadoop-common/src/main/bin/../libexec/hadoop-config.sh: No such file or directory Usage: hadoop [--config confdir] [COMMAND | CLASSNAME] CLASSNAME run the class named CLASSNAME or where COMMAND is one of: fs run a generic filesystem user client version print the version jar <jar> run a jar file note: please use "yarn jar" to launch YARN applications, not this command. checknative [-a|-h] check native hadoop and compression libraries availability distcp <srcurl> <desturl> copy file or directories recursively archive -archiveName NAME -p <parent path> <src>* <dest> create a hadoop archive classpath prints the class path needed to get the credential interact with credential providers Hadoop jar and the required libraries daemonlog get/set the log level for each daemon trace view and modify Hadoop tracing settings Sorry to turn this into a hadoop problem! Just to give you a feedback and in case anyone has an answer... Thanks! /Dan ________________________________________ Från: Lewis John Mcgibbney [[email protected]] Skickat: den 28 januari 2016 00:49 Till: [email protected] Ämne: Re: configuration nutch with hbase and elasticserach Hi Dan, Which version of Nutch 2.X are you using? The document you've highlighted below stated Nutch 2.3 with gora-hbase 0.5. Both of these are old and I would strongly advise you to use Nutch 2.3.1 (just released last week) along with one of the following backends The recommended Gora backends for this Nutch release are - Apache Avro 1.7.6 - Apache Hadoop 1.2.1 and 2.5.2 - Apache HBase 0.98.8-hadoop2 (although also tested with 1.X) - Apache Cassandra 2.0.2 - Apache Solr 4.10.3 - MongoDB 2.6.X - Apache Accumlo 1.5.1 - Apache Spark 1.4.1 Please also note that you shoudl upgrade your JDK to 1.7. Thanks On Wed, Jan 27, 2016 at 3:21 PM, <[email protected]> wrote: > > Hi, > I am a beginner with nutch and everything. Can anyone help me with the > configuration? > > I follower the instruction in > https://gist.github.com/xrstf/b48a970098a8e76943b9 > > It seems that my building process complain plugin: indexer-elastic? My > thanks in advance!!! > > compile: > [echo] Compiling plugin: indexer-elastic > [javac] Compiling 3 source files to > /home/dan/apache-nutch-2.3/build/indexer-elastic/classes > [javac] warning: [options] bootstrap class path not set in conjunction > with -source 1.6 > [javac] > /home/dan/apache-nutch-2.3/src/plugin/indexer-elastic/src/java/org/apache/nutch/indexwriter/elastic/ElasticIndexWriter.java:108: > error: no suitable constructor found for > InetSocketTransportAddress(String,int) > [javac] .addTransportAddress(new > InetSocketTransportAddress(host, port)); > [javac] ^ > [javac] constructor > InetSocketTransportAddress.InetSocketTransportAddress(InetSocketAddress) is > not applicable > [javac] (actual and formal argument lists differ in length) > [javac] constructor > InetSocketTransportAddress.InetSocketTransportAddress(InetAddress,int) is > not applicable > [javac] (actual argument String cannot be converted to > InetAddress by method invocation conversion) > [javac] constructor > InetSocketTransportAddress.InetSocketTransportAddress() is not applicable > [javac] (actual and formal argument lists differ in length) > [javac] constructor > InetSocketTransportAddress.InetSocketTransportAddress(StreamInput) is not > applicable > [javac] (actual and formal argument lists differ in length) > [javac] > /home/dan/apache-nutch-2.3/src/plugin/indexer-elastic/src/java/org/apache/nutch/indexwriter/elastic/ElasticIndexWriter.java:107: > error: constructor TransportClient in class TransportClient cannot be > applied to given types; > [javac] client = new TransportClient(settings) > [javac] ^ > [javac] required: Injector > [javac] found: Settings > [javac] reason: actual argument Settings cannot be converted to > Injector by method invocation conversion > [javac] 2 errors > [javac] 1 warning > > BUILD FAILED > /home/dan/apache-nutch-2.3/build.xml:113: The following error occurred > while executing this line: > /home/dan/apache-nutch-2.3/src/plugin/build.xml:35: The following error > occurred while executing this line: > /home/dan/apache-nutch-2.3/src/plugin/build-plugin.xml:117: Compile > failed; see the compiler error output for details. > > >
