Mengwei, Issue is solved. MapReduce configuration file had invalid configuration and was creating problem.
*File Name:* /etc/hadoop/conf/mapred-site.xml *Parameter Name:* mapred.job.tracker *Original Value: *neededForHive:999999 *Modified Value: *<machinename>:9001 After this change, this are working fine now. Note that I have followed suggestions provided by Mengwei in this thread. Regards, Madhan On Thu, Jul 4, 2013 at 11:08 PM, Mengwei Ding <[email protected]>wrote: > Ok, Madhan, why not. > > Could you kindly provide you availabilities of time and communication > tools. I will be more than happy to help you out with this. > > Best, > Mengwei > > > On Thu, Jul 4, 2013 at 1:15 AM, Madhanmohan Savadamuthu < > [email protected]> wrote: > >> After doing changes in catalina.properties also, same issue is coming. >> >> is there any possibility for interactive discussion on this issue? >> >> Regards, >> Madhan >> >> On Wed, Jul 3, 2013 at 11:05 PM, Mengwei Ding <[email protected] >> > wrote: >> >>> Thank you for your prompt response, sir. Please don't worry, I can help >>> you out with this until your problem is done. >>> >>> Well, let's try out our new method of adding dependency jar files, and >>> forget about the addtowar.sh script. >>> >>> Please following these instructions: >>> >>> " >>> Installing Dependencies >>> >>> Hadoop libraries must be available on node where you are planning to run >>> Sqoop server with proper configuration for major services - NameNode and >>> either JobTracker or ResourceManager depending whether you are running >>> Hadoop 1 or 2. There is no need to run any Hadoop service on the same node >>> as Sqoop server, just the libraries and configuration must be available. >>> >>> Path to Hadoop libraries is stored in file catalina.properties inside >>> directory server/conf. You need to change property called common.loader to >>> contain all directories with your Hadoop libraries. The default expected >>> locations are /usr/lib/hadoop and /usr/lib/hadoop/lib/. Please check out >>> the comments in the file for further description how to configure different >>> locations. >>> >>> Lastly you might need to install JDBC drivers that are not bundled with >>> Sqoop because of incompatible licenses. You can add any arbitrary Java jar >>> file to Sqoop server by copying it into lib/ directory. You can create this >>> directory if it do not exists already. >>> " >>> >>> I can give you my configuration as an example. So in my >>> catalina.properties file, I have the following line: >>> >>> * >>> common.loader=${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar,${catalina.home}/../lib/*.jar,/usr/lib/hadoop/client-0.20/*.jar,/home/mengweid/Downloads/mysql-connector-java-5.1.25-bin.jar >>> * >>> >>> The */usr/lib/hadoop/client-0.20/*.jar *is used to include all >>> hadoop-related jars, and *mysql-connector-java-5.1.25-bin.jar *is used >>> for JDBC driver. >>> >>> Please try this, and let me know whether it works. Thank you. >>> >>> Best, >>> Mengwei >>> >>> >>> On Wed, Jul 3, 2013 at 9:18 AM, Madhanmohan Savadamuthu < >>> [email protected]> wrote: >>> >>>> I did deployment as sugggested in below thread. I am not able to >>>> successfully use sqoop2. I am attaching the services log for your >>>> references. >>>> >>>> I made sure that exact same set of JAR files in appropriate location >>>> and also deleted sqoop folder before starting the sqoop server. >>>> >>>> *Error Message:* >>>> Exception has occurred during processing command >>>> Exception: com.sun.jersey.api.client.UniformInterfaceException Message: >>>> GET http://<ipaddress>:12013/sqoop/version returned a response status >>>> of 404 Not Found >>>> >>>> Regards, >>>> Madhan >>>> >>>> On Wed, Jul 3, 2013 at 7:30 PM, Mengwei Ding < >>>> [email protected]> wrote: >>>> >>>>> Hi Madhanmohan, >>>>> >>>>> Thank you for providing all these detailed information. Help a lot to >>>>> diagnose the problem. >>>>> >>>>> First, the addtowar.sh is not good enough for every situation, >>>>> we apologize for that. We have already figured out a new way to add >>>>> dependency library, which will coming out along with next version of >>>>> Sqoop2. >>>>> >>>>> Currently, it seems like the hadoop-core.jar has not been added. I >>>>> could show you all the libraries existing in the >>>>> webapps/sqoop/WEB-INF/lib folder, please check below: >>>>> avro-1.7.4.jar >>>>> commons-cli-1.2.jar >>>>> commons-configuration-1.6.jar >>>>> commons-dbcp-1.4.jar >>>>> commons-lang-2.5.jar >>>>> commons-logging-1.1.1.jar >>>>> commons-pool-1.5.4.jar >>>>> derby-10.8.2.2.jar >>>>> guava-11.0.2.jar >>>>> hadoop-auth-2.0.0-cdh4.3.0.jar >>>>> hadoop-common-2.0.0-cdh4.3.0.jar >>>>> *hadoop-core-2.0.0-mr1-cdh4.3.0.jar* >>>>> hadoop-hdfs-2.0.0-cdh4.3.0.jar >>>>> hadoop-mapreduce-client-app-2.0.0-cdh4.3.0.jar >>>>> hadoop-mapreduce-client-common-2.0.0-cdh4.3.0.jar >>>>> hadoop-mapreduce-client-core-2.0.0-cdh4.3.0.jar >>>>> hadoop-mapreduce-client-jobclient-2.0.0-cdh4.3.0.jar >>>>> hadoop-yarn-api-2.0.0-cdh4.3.0.jar >>>>> hadoop-yarn-common-2.0.0-cdh4.3.0.jar >>>>> jackson-core-asl-1.8.8.jar >>>>> jackson-mapper-asl-1.8.8.jar >>>>> json-simple-1.1.jar >>>>> log4j-1.2.16.jar >>>>> mysql-connector-java-5.1.25-bin.jar >>>>> protobuf-java-2.4.0a.jar >>>>> slf4j-api-1.6.1.jar >>>>> slf4j-log4j12-1.6.1.jar >>>>> sqoop-common-1.99.2.jar >>>>> sqoop-connector-generic-jdbc-1.99.2.jar >>>>> sqoop-core-1.99.2.jar >>>>> sqoop-execution-mapreduce-1.99.2-hadoop200.jar >>>>> sqoop-repository-derby-1.99.2.jar >>>>> sqoop-spi-1.99.2.jar >>>>> sqoop-submission-mapreduce-1.99.2-hadoop200.jar >>>>> >>>>> I have the same Hadoop and Sqoop2 installation directories with you. >>>>> And I am running a pseudo cluster in a single Ubuntu virtual machine. >>>>> >>>>> So, now, you could try to add the hadoop-core.jar manually, and then >>>>> go ahead to see whether the sqoop2 server could run. Please follow the >>>>> following steps: >>>>> >>>>> *./bin/addtowar.sh >>>>> -jars /usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.3.0.jar >>>>> * >>>>> >>>>> Please find the hadoop-core.jar in your own machine. It should be in a >>>>> similar place. But still, if you have problem, please let me know. >>>>> >>>>> >>>>> The reason why it's better to remove the "sqoop" folder is to clear >>>>> the cached old servlet. Because Tomcat cannot alway extract the sqoop.war >>>>> file immediately after you add dependency library to sqoop.war file. By >>>>> removing the sqoop folder, the Tomcat is forced to extract the sqoop.war >>>>> to >>>>> keep the sqoop folder up-to-date. So in this way, you could know whether >>>>> you have correctly setup the dependency library. Does this explanation >>>>> help? >>>>> >>>>> Best, >>>>> Mengwei >>>>> >>>>> >>>>> On Tue, Jul 2, 2013 at 9:19 PM, Madhanmohan Savadamuthu < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Mengwei, >>>>>> >>>>>> Following are details >>>>>> >>>>>> Hadoop Version: Hadoop 2.0.0-cdh4.2.1 >>>>>> Linux Version: Linux version 2.6.32-358.2.1.el6.x86_64 ( >>>>>> [email protected]) (gcc version 4.4.7 >>>>>> 20120313 (Red Hat 4.4.7-3) (GCC) ) #1 SMP Wed Feb 20 12:17:37 EST 2013 >>>>>> Hadoop Installation Location: /usr/lib/hadoop >>>>>> Sqoop2 Installation Location: /usr/lib/sqoop2 >>>>>> Sqoop2 Dependency Configuration Command Used: ./bin/addtowar.sh >>>>>> -hadoop-auto >>>>>> Files in : >>>>>> >>>>>> avro-1.7.3.jar >>>>>> commons-cli-1.2.jar >>>>>> commons-configuration-1.6.jar >>>>>> commons-dbcp-1.4.jar >>>>>> commons-lang-2.5.jar >>>>>> commons-logging-1.1.1.jar >>>>>> commons-pool-1.5.4.jar >>>>>> derby-10.8.2.2.jar >>>>>> guava-11.0.2.jar >>>>>> hadoop-auth-2.0.0-cdh4.2.1.jar >>>>>> hadoop-common-2.0.0-cdh4.2.1-tests.jar >>>>>> hadoop-hdfs-2.0.0-cdh4.2.1.jar >>>>>> hadoop-mapreduce-client-app-2.0.0-cdh4.2.1.jar >>>>>> hadoop-mapreduce-client-common-2.0.0-cdh4.2.1.jar >>>>>> hadoop-mapreduce-client-core-2.0.0-cdh4.2.1.jar >>>>>> hadoop-mapreduce-client-jobclient-2.0.0-cdh4.2.1.jar >>>>>> hadoop-yarn-api-2.0.0-cdh4.2.1.jar >>>>>> hadoop-yarn-common-2.0.0-cdh4.2.1.jar >>>>>> jackson-core-asl-1.8.8.jar >>>>>> jackson-mapper-asl-1.8.8.jar >>>>>> json-simple-1.1.jar >>>>>> log4j-1.2.16.jar >>>>>> mysql-connector-java-5.1.25-bin.jar >>>>>> protobuf-java-2.4.0a.jar >>>>>> slf4j-api-1.6.1.jar >>>>>> slf4j-log4j12-1.6.1.jar >>>>>> sqoop-common-1.99.2.jar >>>>>> sqoop-connector-generic-jdbc-1.99.2.jar >>>>>> sqoop-core-1.99.2.jar >>>>>> sqoop-execution-mapreduce-1.99.2-hadoop200.jar >>>>>> sqoop-repository-derby-1.99.2.jar >>>>>> sqoop-spi-1.99.2.jar >>>>>> sqoop-submission-mapreduce-1.99.2-hadoop200.jar >>>>>> >>>>>> Can you elaborate more about deletion of 'sqoop' folder? >>>>>> >>>>>> Regards, >>>>>> Madhanmohan S >>>>>> >>>>>> On Tue, Jul 2, 2013 at 10:50 PM, Mengwei Ding < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Madhanmohan, >>>>>>> >>>>>>> Thank you for your interest in Sqoop2. It's really great to hear >>>>>>> this. And thank you for providing details for your question. Let me help >>>>>>> you out with this. >>>>>>> >>>>>>> This main reason for your situation is that the Sqoop servlet has >>>>>>> not been started successfully, so the client get connection refused. I >>>>>>> have >>>>>>> gone through you attachments. The reason of servlet failure is that your >>>>>>> Hadoop dependency library has not be configured correctly. Could you >>>>>>> kindly >>>>>>> answer my following questions, so that I could help with you further. >>>>>>> >>>>>>> 1. Your Hadoop version and installation location? You operating >>>>>>> system? >>>>>>> 2. The details of how you configure the dependency library for sqoop? >>>>>>> 3. Could you kindly go to >>>>>>> [sqoop_install_dir]/server/server/webapps/sqoop/WEB-INF/lib and list all >>>>>>> the jar files? >>>>>>> >>>>>>> PS: remember to delete the sqoop folder under server/server/webapps >>>>>>> every time after you configure the dependency library. >>>>>>> >>>>>>> Best, >>>>>>> Mengwei >>>>>>> >>>>>>> >>>>>>> On Tue, Jul 2, 2013 at 10:05 AM, Madhanmohan Savadamuthu < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> I have setup Sqoop 1.99.2 as mentioned in >>>>>>>> sqoop.apache.orginstruction. When I try to show version --all command, >>>>>>>> following error is >>>>>>>> coming. >>>>>>>> >>>>>>>> Sqoop 1.99.2 revision 3e31b7d3eefb3696d4970704364dea05a9ea2a59 >>>>>>>> Compiled by homeuser on Mon Apr 15 20:50:13 PDT 2013 >>>>>>>> Exception has occurred during processing command >>>>>>>> Exception: com.sun.jersey.api.client.ClientHandlerException >>>>>>>> Message: java.net.ConnectException: Connection refused >>>>>>>> >>>>>>>> all log files are attached for reference. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Thanks and Regards, >>>>>>>> Madhanmohan S >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Thanks and Regards, >>>>>> Madhanmohan S >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Thanks and Regards, >>>> Madhanmohan S >>>> >>> >>> >> >> >> -- >> Thanks and Regards, >> Madhanmohan S >> > > -- Thanks and Regards, Madhanmohan S
