Hi Jarek, thnx for the response. actually I only have one node running where the namenode, secondarynode, datanode,jobtracker in one PC only.
the remote PC contains the remote database which I failed to access using sqoop version 1.4.4 my configuration IP: 192.168.0.150 - running namenode, secondarynode, datanode, jobstracker IP: 192.168.0.151 - running the remote mysql database FYI, I did try accessing local database at IP 192.168.0.150 and it works. any other things I need to look into? tq On Sunday, November 3, 2013 1:59 AM, Jarek Jarcec Cecho <[email protected]> wrote: Hi sir, Sqoop will need access to your database from *all* machines in your cluster. Having db access only from some limited number of machines (such as gateways, NameNode or other "special" nodes) is not sufficient. I would recommend trying that mysql command from all your nodes or checking your user configuration on MySQL server side. Jarcec On Sat, Nov 02, 2013 at 01:24:22AM +0800, wan razali wrote: > Hi all, > > I tried to import remote database using sqoop > > I did try the solutions mentioned here: > http://solaimurugan.blogspot.com/2013/07/resolved-issue-error-in-sqoop-14.html > https://cwiki.apache.org/confluence/download/attachments/27361435/sqoop_meetup_kate_ting_110711.pdf?version=1&modificationDate=1323721159000 > > & did test the connection using mysql command below & successfully > connected to the remote mysql server (@ 192.168.0.151):- > > > > >mysql -h 192.168.0.151 -u sqoopuser -pscooppassword > > mysql> show databases; > +--------------------+ > | Database | > +--------------------+ > | information_schema | > | mysql | > | performance_schema | > | propertytrend | > | test | > | webscrapper | > +--------------------+ > 6 rows in set (0.39 sec) > > ------------------------------------- > mysql> status; > -------------- > mysql Ver 14.14 Distrib 5.5.32, for debian-linux-gnu (x86_64) using > readline 6.2 > > Connection id: 73 > Current database: > Current user: [email protected] <mailto:[email protected]> > SSL: Not in use > Current pager: stdout > Using outfile: '' > Using delimiter: ; > Server version: 5.5.32-0ubuntu7 (Ubuntu) > Protocol version: 10 > Connection: 192.168.0.151 via TCP/IP > Server characterset: latin1 > Db characterset: latin1 > Client characterset: utf8 > Conn. characterset: utf8 > TCP port: 3306 > Uptime: 32 min 47 sec > > > > > > I want to import the remote db into HFDS with setting options as follows: > ----------------------- > import > --connect > jdbc:mysql://198.168.0.151/propertytrend > --verbose > --username > sqoopuser > --password > scooppasword > --as-textfile > --split-by > id > --table > hd_columns_varchar > --columns > id,rowId,value > --where > columnId = 5 AND reportId = 200 > --target-dir > /user/hduser/sqoop/import/output > > ------------------------ > > > But when I ran the sqoop import I got these errors: > > --------------------------------------- > bin/sqoop --options-file options/setting-import3.txt > Warning: /usr/lib/hbase does not exist! HBase imports will fail. > Please set $HBASE_HOME to the root of your HBase installation. > Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail. > Please set $HCAT_HOME to the root of your HCatalog installation. > Warning: $HADOOP_HOME is deprecated. > > 13/11/01 23:42:15 DEBUG tool.BaseSqoopTool: Enabled debug logging. > 13/11/01 23:42:15 WARN tool.BaseSqoopTool: Setting your password on > the command-line is insecure. Consider using -P instead. > 13/11/01 23:42:16 DEBUG sqoop.ConnFactory: Loaded manager factory: > com.cloudera.sqoop.manager.DefaultManagerFactory > 13/11/01 23:42:16 DEBUG sqoop.ConnFactory: Trying ManagerFactory: > com.cloudera.sqoop.manager.DefaultManagerFactory > 13/11/01 23:42:16 DEBUG manager.DefaultManagerFactory: Trying with > scheme: jdbc:mysql: > 13/11/01 23:42:16 INFO manager.MySQLManager: Preparing to use a > MySQL streaming resultset. > 13/11/01 23:42:16 DEBUG sqoop.ConnFactory: Instantiated ConnManager > org.apache.sqoop.manager.MySQLManager@7618d289 > 13/11/01 23:42:16 INFO tool.CodeGenTool: Beginning code generation > 13/11/01 23:42:16 DEBUG manager.SqlManager: Execute > getColumnTypesRawQuery : SELECT t.* FROM `hd_columns_varchar` AS t > LIMIT 1 > 13/11/01 23:42:16 DEBUG manager.SqlManager: No connection > paramenters specified. Using regular API for making connection. > 13/11/01 23:43:20 ERROR manager.SqlManager: Error executing > statement: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: > Communications link failure > > The last packet sent successfully to the server was 0 milliseconds > ago. The driver has not received any packets from the server. > com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: > Communications link failure > > The last packet sent successfully to the server was 0 milliseconds > ago. The driver has not received any packets from the server. > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at >sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > at >sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) > at >com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1121) > at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:357) > at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2482) > at >com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2519) > at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2304) > at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:834) > at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at >sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > at >sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) > at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:416) > at >com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:346) > at java.sql.DriverManager.getConnection(DriverManager.java:571) > at java.sql.DriverManager.getConnection(DriverManager.java:215) > at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:801) > at >org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) > at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:660) > at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:683) > at >org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240) > at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:223) > at >org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:347) > at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1277) > at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1089) > at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96) > at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396) > at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502) > at org.apache.sqoop.Sqoop.run(Sqoop.java:145) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181) > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220) > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229) > at org.apache.sqoop.Sqoop.main(Sqoop.java:238) > Caused by: java.net.ConnectException: Connection timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) > at >java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) > at >java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) > at >java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > at java.net.Socket.connect(Socket.java:579) > at java.net.Socket.connect(Socket.java:528) > at java.net.Socket.<init>(Socket.java:425) > at java.net.Socket.<init>(Socket.java:241) > at >com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:259) > at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:307) > ... 32 more > 13/11/01 23:43:20 ERROR tool.ImportTool: Encountered IOException > running import job: java.io.IOException: No columns to generate for > ClassWriter > at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1095) > at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96) > at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396) > at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502) > at org.apache.sqoop.Sqoop.run(Sqoop.java:145) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181) > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220) > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229) > at org.apache.sqoop.Sqoop.main(Sqoop.java:238) > > --------------------------------- > > any assistance is much appreciated. > > thank you
