Hi Traiano, This is usually a firewall issue of the server xx.yy.xx.zz. Please make sure the xx.yy.xx.zz :3306 is accessible from where your Sqoop query initiates.
Stanley -----Original Message----- From: Traiano Welcome [mailto:[email protected]] Sent: Tuesday, March 17, 2015 11:02 PM To: [email protected] Subject: How to debug SQOOP / JDBC Driver Failures for MySQL Import Jobs Hi I'm following the guide for testing Sqoop data imports from mysql: http://blog.cloudera.com/blog/2013/11/sqooping-data-with-hue/ But the mysql sqoop job fails with "" in the HUE interface. I'd appreciate some help interpreting the sqoop java error trace I'm seeing in the sqoop logs: My connection settings are as follows: --- JDBC driver class: com.mysql.jdbc.Driver JDBC Connection string: jdbc:mysql://xx.yy.xx.zz/dbname --- I've downloaded and added the jdbc driver from Oracle in /var/lib/sqoop on the sqoop server node, following the cloudera guide here: http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_ig_jdbc_driver_install.html However, the job keeps failing with the following java trace from the sqoop logs (excuse the noise!): --- 2015-03-17 17:44:06,368 INFO org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction commit 2015-03-17 17:44:06,385 INFO org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction commit 2015-03-17 17:44:06,393 INFO org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction commit 2015-03-17 17:45:09,396 ERROR org.apache.sqoop.server.SqoopProtocolServlet: Exception in POST http://lol-dev-hdpdn8.hadoop.local:12000/sqoop/v1/submission/action/2 org.apache.sqoop.common.SqoopException: GENERIC_JDBC_CONNECTOR_0001:Unable to get a connection at org.apache.sqoop.connector.jdbc.GenericJdbcExecutor.<init>(GenericJdbcExecutor.java:51) at org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer.configureJdbcProperties(GenericJdbcImportInitializer.java:125) at org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer.initialize(GenericJdbcImportInitializer.java:49) at org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer.initialize(GenericJdbcImportInitializer.java:40) at org.apache.sqoop.framework.JobManager.submit(JobManager.java:378) at org.apache.sqoop.handler.SubmissionRequestHandler.submissionSubmit(SubmissionRequestHandler.java:152) at org.apache.sqoop.handler.SubmissionRequestHandler.handleActionEvent(SubmissionRequestHandler.java:122) at org.apache.sqoop.handler.SubmissionRequestHandler.handleEvent(SubmissionRequestHandler.java:75) at org.apache.sqoop.server.v1.SubmissionServlet.handlePostRequest(SubmissionServlet.java:44) at org.apache.sqoop.server.SqoopProtocolServlet.doPost(SqoopProtocolServlet.java:63) at javax.servlet.http.HttpServlet.service(HttpServlet.java:643) at javax.servlet.http.HttpServlet.service(HttpServlet.java:723) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:744) Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at com.mysql.jdbc.Util.handleNewInstance(Util.java:377) at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1036) at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:338) at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2232) at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2265) at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2064) at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:790) at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:44) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) . . . --- I've checked for attempts to connect to the remove db server (tcpdump) I've checked for attempts to connect to the remove db server (tcpdump) but it looks like connection attempts are not even leaving the sqoop node, so it seems sqoop is not successful at creating a mysql connection to begin with. I've checked the remote mysql server connection from the sqoop server using the mysql client as well - no issues there. I'm using sqoop on cloudera hadoop with cloudera manager 5.2.0 and java java version "1.7.0_45" Specific package details: hue-sqoop-3.6.0+cdh5.2.0+509-1.cdh5.2.0.p0.37.el6.x86_64 sqoop2-client-1.99.3+cdh5.2.0+30-1.cdh5.2.0.p0.26.el6.noarch sqoop2-1.99.3+cdh5.2.0+30-1.cdh5.2.0.p0.26.el6.noarch sqoop-1.4.5+cdh5.2.0+47-1.cdh5.2.0.p0.26.el6.noarch I've checked the permissions on the .jar file in /usr/lib/sqoop2, and they seem fine: --- [root@lol-dev-hdpdn8 sqoop2]# ls -l /var/lib/sqoop2/ total 8680 drwxr-xr-x 4 sqoop2 sqoop 4096 Sep 10 2014 mysql-connector-java-5.1.33 -rw-r--r-- 1 sqoop2 sqoop 3797240 Sep 10 2014 mysql-connector-java-5.1.33.tar.gz -rwxr-xr-x 1 sqoop2 sqoop 960374 Mar 17 17:36 mysql-connector-java-5.1.34-bin.jar -rwxr-xr-x 1 sqoop2 sqoop 539705 Mar 17 17:37 postgresql-9.0-801.jdbc4.jar drwxr-xr-x 3 sqoop2 sqoop 4096 Oct 31 21:02 repository drwxr-xr-x 3 sqoop2 sqoop 4096 Nov 1 17:07 sqljdbc_4.0 -rwxr-xr-x 1 sqoop2 sqoop 2391092 Nov 1 17:06 sqljdbc_4.0.2206.100_enu.tar.gz -rwxr-xr-x 1 sqoop2 sqoop 584207 Nov 1 17:12 sqljdbc4.jar -rwxr-xr-x 1 sqoop2 sqoop 563117 Nov 1 17:12 sqljdbc.jar drwxr-xr-x 5 sqoop2 sqoop 4096 Mar 17 17:37 tomcat-deployment [root@lol-dev-hdpdn8 sqoop2]# --- I've also tested bot mysql-connector-java-5.1.34-bin.jar and mysql-connector-java-5.1.33-bin.jar, restarting the sqoop service each time I changed the module, to no avail. Is there anyway to debug further into why the job is failing? Many thanks in advance for any help! Traiano
