Thanks for the response, Stanley:
On Wed, Mar 18, 2015 at 6:29 AM, Xu, Qian A <[email protected]> wrote: > Hi Traiano, > > This is usually a firewall issue of the server xx.yy.xx.zz. Please make sure > the xx.yy.xx.zz :3306 is accessible from where your Sqoop query initiates. I can confirm this is not the case: - Can connect to port 3306 on xx.yy.xx.zz: --- [root@lol-dev-hdpdn8 ~]# nc xx.yy.xx.zz 3306 [ 5.5.41-0ubuntu0.12.04.1�\FUV10;xn��>*VT!F{[7A|omysql_native_password ^C --- - Can connect using the mysql client and list tables in the db: --- [root@lol-dev-hdpdn8 ~]# mysql -h xx.yy.xx.zz -u<user> -p <db> Enter password: Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 6076407 --- - Finally, when I run a packet sniff listening for outbound packets to xx.yy.xx.zz from the sqoop server, I see no packets leaving the node to the db server: --- [root@lol-dev-hdpdn8 jdbc]# tcpdump host xx.yy.xx.zz and port 3306 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes ^C 0 packets captured 0 packets received by filter 0 packets dropped by kernel --- Is there any other way I could drill down deeper into this ? > > Stanley > > > -----Original Message----- > From: Traiano Welcome [mailto:[email protected]] > Sent: Tuesday, March 17, 2015 11:02 PM > To: [email protected] > Subject: How to debug SQOOP / JDBC Driver Failures for MySQL Import Jobs > > Hi > > I'm following the guide for testing Sqoop data imports from mysql: > > http://blog.cloudera.com/blog/2013/11/sqooping-data-with-hue/ > > But the mysql sqoop job fails with "" in the HUE interface. I'd appreciate > some help interpreting the sqoop java error trace I'm seeing in the sqoop > logs: > > My connection settings are as follows: > > --- > JDBC driver class: com.mysql.jdbc.Driver JDBC Connection string: > jdbc:mysql://xx.yy.xx.zz/dbname > --- > > I've downloaded and added the jdbc driver from Oracle in /var/lib/sqoop on > the sqoop server node, following the cloudera guide > here: > > > http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_ig_jdbc_driver_install.html > > However, the job keeps failing with the following java trace from the sqoop > logs (excuse the noise!): > > > --- > 2015-03-17 17:44:06,368 INFO > org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction > commit > 2015-03-17 17:44:06,385 INFO > org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction > commit > 2015-03-17 17:44:06,393 INFO > org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction > commit > 2015-03-17 17:45:09,396 ERROR > org.apache.sqoop.server.SqoopProtocolServlet: Exception in POST > http://lol-dev-hdpdn8.hadoop.local:12000/sqoop/v1/submission/action/2 > org.apache.sqoop.common.SqoopException: > GENERIC_JDBC_CONNECTOR_0001:Unable to get a connection > at > org.apache.sqoop.connector.jdbc.GenericJdbcExecutor.<init>(GenericJdbcExecutor.java:51) > at > org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer.configureJdbcProperties(GenericJdbcImportInitializer.java:125) > at > org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer.initialize(GenericJdbcImportInitializer.java:49) > at > org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer.initialize(GenericJdbcImportInitializer.java:40) > at org.apache.sqoop.framework.JobManager.submit(JobManager.java:378) > at > org.apache.sqoop.handler.SubmissionRequestHandler.submissionSubmit(SubmissionRequestHandler.java:152) > at > org.apache.sqoop.handler.SubmissionRequestHandler.handleActionEvent(SubmissionRequestHandler.java:122) > at > org.apache.sqoop.handler.SubmissionRequestHandler.handleEvent(SubmissionRequestHandler.java:75) > at > org.apache.sqoop.server.v1.SubmissionServlet.handlePostRequest(SubmissionServlet.java:44) > at > org.apache.sqoop.server.SqoopProtocolServlet.doPost(SqoopProtocolServlet.java:63) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:643) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:723) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) > at > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861) > at > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606) > at > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) > at java.lang.Thread.run(Thread.java:744) > Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: > Communications link failure > > The last packet sent successfully to the server was 0 milliseconds ago. The > driver has not received any packets from the server. > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > at com.mysql.jdbc.Util.handleNewInstance(Util.java:377) > at > com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1036) > at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:338) > at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2232) > at > com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2265) > at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2064) > at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:790) > at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:44) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > . > . > . > --- > > > I've checked for attempts to connect to the remove db server (tcpdump) > > I've checked for attempts to connect to the remove db server (tcpdump) but it > looks like connection attempts are not even leaving the sqoop node, so it > seems sqoop is not successful at creating a mysql connection to begin with. > I've checked the remote mysql server connection from the sqoop server using > the mysql client as well - no issues there. > > I'm using sqoop on cloudera hadoop with cloudera manager 5.2.0 and java java > version "1.7.0_45" > > Specific package details: > > hue-sqoop-3.6.0+cdh5.2.0+509-1.cdh5.2.0.p0.37.el6.x86_64 > sqoop2-client-1.99.3+cdh5.2.0+30-1.cdh5.2.0.p0.26.el6.noarch > sqoop2-1.99.3+cdh5.2.0+30-1.cdh5.2.0.p0.26.el6.noarch > sqoop-1.4.5+cdh5.2.0+47-1.cdh5.2.0.p0.26.el6.noarch > > I've checked the permissions on the .jar file in /usr/lib/sqoop2, and they > seem fine: > > --- > [root@lol-dev-hdpdn8 sqoop2]# ls -l /var/lib/sqoop2/ total 8680 > drwxr-xr-x 4 sqoop2 sqoop 4096 Sep 10 2014 mysql-connector-java-5.1.33 > -rw-r--r-- 1 sqoop2 sqoop 3797240 Sep 10 2014 > mysql-connector-java-5.1.33.tar.gz > -rwxr-xr-x 1 sqoop2 sqoop 960374 Mar 17 17:36 > mysql-connector-java-5.1.34-bin.jar > -rwxr-xr-x 1 sqoop2 sqoop 539705 Mar 17 17:37 postgresql-9.0-801.jdbc4.jar > drwxr-xr-x 3 sqoop2 sqoop 4096 Oct 31 21:02 repository > drwxr-xr-x 3 sqoop2 sqoop 4096 Nov 1 17:07 sqljdbc_4.0 > -rwxr-xr-x 1 sqoop2 sqoop 2391092 Nov 1 17:06 > sqljdbc_4.0.2206.100_enu.tar.gz -rwxr-xr-x 1 sqoop2 sqoop 584207 Nov 1 > 17:12 sqljdbc4.jar -rwxr-xr-x 1 sqoop2 sqoop 563117 Nov 1 17:12 sqljdbc.jar > drwxr-xr-x 5 sqoop2 sqoop 4096 Mar 17 17:37 tomcat-deployment > [root@lol-dev-hdpdn8 sqoop2]# > > --- > > I've also tested bot mysql-connector-java-5.1.34-bin.jar and > mysql-connector-java-5.1.33-bin.jar, restarting the sqoop service each time I > changed the module, to no avail. > > Is there anyway to debug further into why the job is failing? > > Many thanks in advance for any help! > > Traiano
