[jira] [Commented] (SQOOP-2331) Snappy Compression Support in Sqoop-HCatalog
[ https://issues.apache.org/jira/browse/SQOOP-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16572680#comment-16572680 ] Shashank commented on SQOOP-2331: - [~jarcec] I have incorporated the changes.Please review the same. > Snappy Compression Support in Sqoop-HCatalog > > > Key: SQOOP-2331 > URL: https://issues.apache.org/jira/browse/SQOOP-2331 > Project: Sqoop > Issue Type: New Feature >Affects Versions: 1.4.7 >Reporter: Atul Gupta >Assignee: Shashank >Priority: Major > Fix For: 1.5.0 > > Attachments: SQOOP-2331_0.patch, SQOOP-2331_1.patch, > SQOOP-2331_2.patch, SQOOP-2331_2.patch > > > Current Apache Sqoop 1.4.7 does not compress in gzip format with > --compress option while using with --hcatalog-table option. It also does not > support option --compression-codec snappy with --hcatalog-table option. it > would be nice if we add both the options in the Sqoop future releases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SQOOP-3358) sqoop2 1.99.7 Schema name of Job not work for MariaDB
Wang created SQOOP-3358: --- Summary: sqoop2 1.99.7 Schema name of Job not work for MariaDB Key: SQOOP-3358 URL: https://issues.apache.org/jira/browse/SQOOP-3358 Project: Sqoop Issue Type: Bug Components: connectors/generic Affects Versions: 1.99.7 Environment: Mac 10.13.5 Sqoop 1.99.7 Reporter: Wang Hi guys, I find a problem when use sqoop2 1.99.7, that is: I want to export MariaDB(mysql) to hdfs using the connector org.apache.sqoop.connector.jdbc.GenericJdbcConnector。First, I create a Link using "dbc:mysql://server_host:server_port/db_name"。Then, I create a Job from mysql_link to hdfs_link。And The "Schema Name" is the db_name。Finally, I start the Job, but it responses that other_db_name.Table_name not exist. The other_db_name is also in the mysql server, but not the one I set in "Schema Name". Why? Is't that sqoop2 not support MariaDB. Other problem is that when I set "SQL statement" as "select * from table ${CONDITIONS}" and "Table name" empty. It will warns that "revise your input". Why ? Is't that it not support "SQL statement" -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (SQOOP-2949) SQL Syntax error when split-by column is of character type and min or max value has single quote inside it
[ https://issues.apache.org/jira/browse/SQOOP-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fero Szabo reassigned SQOOP-2949: - Assignee: Gireesh Puthumana > SQL Syntax error when split-by column is of character type and min or max > value has single quote inside it > -- > > Key: SQOOP-2949 > URL: https://issues.apache.org/jira/browse/SQOOP-2949 > Project: Sqoop > Issue Type: Bug >Affects Versions: 1.4.6 > Environment: Sqoop 1.4.6 > Run on Hadoop 2.6.0 > On Ubuntu >Reporter: Gireesh Puthumana >Assignee: Gireesh Puthumana >Priority: Major > > Did a sqoop import from mysql table "emp", with split-by column "ename", > which is a varchar(100) type. > +Used below command:+ > sqoop import --connect jdbc:mysql://localhost/testdb --username root > --password * --table emp --m 2 --target-dir /sqoopTest/5 --split-by ename; > +Ename has following records:+ > | ename | > | gireesh | > | aavesh | > | shiva' | > | jamir | > | balu| > | santosh | > | sameer | > Min value is "aavesh" and max value is "shiva'" (please note the single quote > inside max value). > When run, it tried to execute below query in mapper 2 and failed: > SELECT `ename`, `eid`, `deptid` FROM `emp` AS `emp` WHERE ( `ename` >= > 'jd聯聭聪G耀' ) AND ( `ename` <= 'shiva'' ) > +Stack trace:+ > {quote} > 2016-06-05 16:54:06,749 ERROR [main] > org.apache.sqoop.mapreduce.db.DBRecordReader: Top level exception: > com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error > in your SQL syntax; check the manual that corresponds to your MySQL server > version for the right syntax to use near ''shiva'' )' at line 1 > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:422) > at com.mysql.jdbc.Util.handleNewInstance(Util.java:404) > at com.mysql.jdbc.Util.getInstance(Util.java:387) > at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:942) > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3966) > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3902) > at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2526) > at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2673) > at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2549) > at > com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861) > at > com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1962) > at > org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111) > at > org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235) > at > org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553) > at > org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80) > at > org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) > at > org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > {quote} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SQOOP-2949) SQL Syntax error when split-by column is of character type and min or max value has single quote inside it
[ https://issues.apache.org/jira/browse/SQOOP-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16571394#comment-16571394 ] Fero Szabo commented on SQOOP-2949: --- Hi [~gireeshp], Great to hear! We are trying to improve the test coverage of Sqoop so that we can ensure that a new change won't break existing use cases. So the first step is to create a few basic test cases for the change. For this, you will need to install the docker images for the 3rd party tests if you haven't done so, yet. Please see COMPILING.txt in the root directory of the project for the details. (The relevant section starts at line 221: "Setting up and executing third-party tests...") *1. Creating tests* As a start, I'd suggest modifying SQLServerSplitByTest.java. A test for SQL server is a good start, though, since this is a fundamental change, a test for the following RDBMS implementations would also be nice: * Oracle * MySQL * PostgresSQL The committers / reviewers might ask for these anyway, as Oracle is the most popular database implementation used with Sqoop. *2. Creating a review* Please go to Review Board at [https://reviews.apache.org/account/login/] and register if you haven't done so far. Then, create a patch by invoking _git diff > SQOOP-2949-1.patch **_on the command line. Finally, create a review using the _sqoop-trunk_ repository and your patch. Fill in the necessary fields, as for example, in this reivew: [https://reviews.apache.org/r/65607/] (no need for a description this long, nobody likes to read this much :) ). Once this is done, we'll continue the technical discussion on ReviewBoard! Please let me know if you've any questions here, or via email (you should be able to see my email address under my name). Best Regards, Fero > SQL Syntax error when split-by column is of character type and min or max > value has single quote inside it > -- > > Key: SQOOP-2949 > URL: https://issues.apache.org/jira/browse/SQOOP-2949 > Project: Sqoop > Issue Type: Bug >Affects Versions: 1.4.6 > Environment: Sqoop 1.4.6 > Run on Hadoop 2.6.0 > On Ubuntu >Reporter: Gireesh Puthumana >Priority: Major > > Did a sqoop import from mysql table "emp", with split-by column "ename", > which is a varchar(100) type. > +Used below command:+ > sqoop import --connect jdbc:mysql://localhost/testdb --username root > --password * --table emp --m 2 --target-dir /sqoopTest/5 --split-by ename; > +Ename has following records:+ > | ename | > | gireesh | > | aavesh | > | shiva' | > | jamir | > | balu| > | santosh | > | sameer | > Min value is "aavesh" and max value is "shiva'" (please note the single quote > inside max value). > When run, it tried to execute below query in mapper 2 and failed: > SELECT `ename`, `eid`, `deptid` FROM `emp` AS `emp` WHERE ( `ename` >= > 'jd聯聭聪G耀' ) AND ( `ename` <= 'shiva'' ) > +Stack trace:+ > {quote} > 2016-06-05 16:54:06,749 ERROR [main] > org.apache.sqoop.mapreduce.db.DBRecordReader: Top level exception: > com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error > in your SQL syntax; check the manual that corresponds to your MySQL server > version for the right syntax to use near ''shiva'' )' at line 1 > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:422) > at com.mysql.jdbc.Util.handleNewInstance(Util.java:404) > at com.mysql.jdbc.Util.getInstance(Util.java:387) > at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:942) > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3966) > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3902) > at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2526) > at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2673) > at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2549) > at > com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861) > at > com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1962) > at > org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111) > at > org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235) > at > org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553) > at > org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80) >
Review Request 68254: Change MainframeImportTool to refer to MainframeManager class directly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68254/ --- Review request for Sqoop. Bugs: SQOOP-3357 https://issues.apache.org/jira/browse/SQOOP-3357 Repository: sqoop-trunk Description --- MainframeImportTool used to refer to the MainframeManager with a string. I changed the string to a MainframeManager class reference. Diffs - src/java/org/apache/sqoop/tool/MainframeImportTool.java 8883301d src/test/org/apache/sqoop/manager/TestMainframeManager.java 97e48e88 Diff: https://reviews.apache.org/r/68254/diff/1/ Testing --- I ran ant test and did not add any more test case because my change was already covered. Thanks, Nguyen Truong
[jira] [Assigned] (SQOOP-2949) SQL Syntax error when split-by column is of character type and min or max value has single quote inside it
[ https://issues.apache.org/jira/browse/SQOOP-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fero Szabo reassigned SQOOP-2949: - Assignee: (was: Fero Szabo) > SQL Syntax error when split-by column is of character type and min or max > value has single quote inside it > -- > > Key: SQOOP-2949 > URL: https://issues.apache.org/jira/browse/SQOOP-2949 > Project: Sqoop > Issue Type: Bug >Affects Versions: 1.4.6 > Environment: Sqoop 1.4.6 > Run on Hadoop 2.6.0 > On Ubuntu >Reporter: Gireesh Puthumana >Priority: Major > > Did a sqoop import from mysql table "emp", with split-by column "ename", > which is a varchar(100) type. > +Used below command:+ > sqoop import --connect jdbc:mysql://localhost/testdb --username root > --password * --table emp --m 2 --target-dir /sqoopTest/5 --split-by ename; > +Ename has following records:+ > | ename | > | gireesh | > | aavesh | > | shiva' | > | jamir | > | balu| > | santosh | > | sameer | > Min value is "aavesh" and max value is "shiva'" (please note the single quote > inside max value). > When run, it tried to execute below query in mapper 2 and failed: > SELECT `ename`, `eid`, `deptid` FROM `emp` AS `emp` WHERE ( `ename` >= > 'jd聯聭聪G耀' ) AND ( `ename` <= 'shiva'' ) > +Stack trace:+ > {quote} > 2016-06-05 16:54:06,749 ERROR [main] > org.apache.sqoop.mapreduce.db.DBRecordReader: Top level exception: > com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error > in your SQL syntax; check the manual that corresponds to your MySQL server > version for the right syntax to use near ''shiva'' )' at line 1 > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:422) > at com.mysql.jdbc.Util.handleNewInstance(Util.java:404) > at com.mysql.jdbc.Util.getInstance(Util.java:387) > at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:942) > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3966) > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3902) > at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2526) > at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2673) > at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2549) > at > com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861) > at > com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1962) > at > org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111) > at > org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235) > at > org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553) > at > org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80) > at > org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) > at > org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > {quote} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (SQOOP-2949) SQL Syntax error when split-by column is of character type and min or max value has single quote inside it
[ https://issues.apache.org/jira/browse/SQOOP-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fero Szabo reassigned SQOOP-2949: - Assignee: (was: Fero Szabo) > SQL Syntax error when split-by column is of character type and min or max > value has single quote inside it > -- > > Key: SQOOP-2949 > URL: https://issues.apache.org/jira/browse/SQOOP-2949 > Project: Sqoop > Issue Type: Bug >Affects Versions: 1.4.6 > Environment: Sqoop 1.4.6 > Run on Hadoop 2.6.0 > On Ubuntu >Reporter: Gireesh Puthumana >Priority: Major > > Did a sqoop import from mysql table "emp", with split-by column "ename", > which is a varchar(100) type. > +Used below command:+ > sqoop import --connect jdbc:mysql://localhost/testdb --username root > --password * --table emp --m 2 --target-dir /sqoopTest/5 --split-by ename; > +Ename has following records:+ > | ename | > | gireesh | > | aavesh | > | shiva' | > | jamir | > | balu| > | santosh | > | sameer | > Min value is "aavesh" and max value is "shiva'" (please note the single quote > inside max value). > When run, it tried to execute below query in mapper 2 and failed: > SELECT `ename`, `eid`, `deptid` FROM `emp` AS `emp` WHERE ( `ename` >= > 'jd聯聭聪G耀' ) AND ( `ename` <= 'shiva'' ) > +Stack trace:+ > {quote} > 2016-06-05 16:54:06,749 ERROR [main] > org.apache.sqoop.mapreduce.db.DBRecordReader: Top level exception: > com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error > in your SQL syntax; check the manual that corresponds to your MySQL server > version for the right syntax to use near ''shiva'' )' at line 1 > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:422) > at com.mysql.jdbc.Util.handleNewInstance(Util.java:404) > at com.mysql.jdbc.Util.getInstance(Util.java:387) > at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:942) > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3966) > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3902) > at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2526) > at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2673) > at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2549) > at > com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861) > at > com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1962) > at > org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111) > at > org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235) > at > org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553) > at > org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80) > at > org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) > at > org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > {quote} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (SQOOP-2949) SQL Syntax error when split-by column is of character type and min or max value has single quote inside it
[ https://issues.apache.org/jira/browse/SQOOP-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fero Szabo reassigned SQOOP-2949: - Assignee: Fero Szabo > SQL Syntax error when split-by column is of character type and min or max > value has single quote inside it > -- > > Key: SQOOP-2949 > URL: https://issues.apache.org/jira/browse/SQOOP-2949 > Project: Sqoop > Issue Type: Bug >Affects Versions: 1.4.6 > Environment: Sqoop 1.4.6 > Run on Hadoop 2.6.0 > On Ubuntu >Reporter: Gireesh Puthumana >Assignee: Fero Szabo >Priority: Major > > Did a sqoop import from mysql table "emp", with split-by column "ename", > which is a varchar(100) type. > +Used below command:+ > sqoop import --connect jdbc:mysql://localhost/testdb --username root > --password * --table emp --m 2 --target-dir /sqoopTest/5 --split-by ename; > +Ename has following records:+ > | ename | > | gireesh | > | aavesh | > | shiva' | > | jamir | > | balu| > | santosh | > | sameer | > Min value is "aavesh" and max value is "shiva'" (please note the single quote > inside max value). > When run, it tried to execute below query in mapper 2 and failed: > SELECT `ename`, `eid`, `deptid` FROM `emp` AS `emp` WHERE ( `ename` >= > 'jd聯聭聪G耀' ) AND ( `ename` <= 'shiva'' ) > +Stack trace:+ > {quote} > 2016-06-05 16:54:06,749 ERROR [main] > org.apache.sqoop.mapreduce.db.DBRecordReader: Top level exception: > com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error > in your SQL syntax; check the manual that corresponds to your MySQL server > version for the right syntax to use near ''shiva'' )' at line 1 > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:422) > at com.mysql.jdbc.Util.handleNewInstance(Util.java:404) > at com.mysql.jdbc.Util.getInstance(Util.java:387) > at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:942) > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3966) > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3902) > at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2526) > at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2673) > at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2549) > at > com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861) > at > com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1962) > at > org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111) > at > org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235) > at > org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553) > at > org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80) > at > org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) > at > org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > {quote} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (SQOOP-2949) SQL Syntax error when split-by column is of character type and min or max value has single quote inside it
[ https://issues.apache.org/jira/browse/SQOOP-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fero Szabo reassigned SQOOP-2949: - Assignee: Fero Szabo > SQL Syntax error when split-by column is of character type and min or max > value has single quote inside it > -- > > Key: SQOOP-2949 > URL: https://issues.apache.org/jira/browse/SQOOP-2949 > Project: Sqoop > Issue Type: Bug >Affects Versions: 1.4.6 > Environment: Sqoop 1.4.6 > Run on Hadoop 2.6.0 > On Ubuntu >Reporter: Gireesh Puthumana >Assignee: Fero Szabo >Priority: Major > > Did a sqoop import from mysql table "emp", with split-by column "ename", > which is a varchar(100) type. > +Used below command:+ > sqoop import --connect jdbc:mysql://localhost/testdb --username root > --password * --table emp --m 2 --target-dir /sqoopTest/5 --split-by ename; > +Ename has following records:+ > | ename | > | gireesh | > | aavesh | > | shiva' | > | jamir | > | balu| > | santosh | > | sameer | > Min value is "aavesh" and max value is "shiva'" (please note the single quote > inside max value). > When run, it tried to execute below query in mapper 2 and failed: > SELECT `ename`, `eid`, `deptid` FROM `emp` AS `emp` WHERE ( `ename` >= > 'jd聯聭聪G耀' ) AND ( `ename` <= 'shiva'' ) > +Stack trace:+ > {quote} > 2016-06-05 16:54:06,749 ERROR [main] > org.apache.sqoop.mapreduce.db.DBRecordReader: Top level exception: > com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error > in your SQL syntax; check the manual that corresponds to your MySQL server > version for the right syntax to use near ''shiva'' )' at line 1 > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:422) > at com.mysql.jdbc.Util.handleNewInstance(Util.java:404) > at com.mysql.jdbc.Util.getInstance(Util.java:387) > at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:942) > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3966) > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3902) > at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2526) > at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2673) > at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2549) > at > com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861) > at > com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1962) > at > org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111) > at > org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235) > at > org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553) > at > org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80) > at > org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) > at > org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > {quote} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SQOOP-3357) Change MainframeImportTool to refer to MainframeManager class directly
[ https://issues.apache.org/jira/browse/SQOOP-3357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nguyen Truong updated SQOOP-3357: - Attachment: SQOOP-3357.patch > Change MainframeImportTool to refer to MainframeManager class directly > -- > > Key: SQOOP-3357 > URL: https://issues.apache.org/jira/browse/SQOOP-3357 > Project: Sqoop > Issue Type: Improvement >Affects Versions: 1.4.7 >Reporter: Nguyen Truong >Priority: Major > Fix For: 3.0.0 > > Attachments: SQOOP-3357.patch > > > Currently MainframeImportTool refers to the MainframeManager with a string > which can create a problem in refactoring. It would be beneficial to change > the string to a MainframeManager class reference. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SQOOP-3357) Change MainframeImportTool to refer to MainframeManager class directly
Nguyen Truong created SQOOP-3357: Summary: Change MainframeImportTool to refer to MainframeManager class directly Key: SQOOP-3357 URL: https://issues.apache.org/jira/browse/SQOOP-3357 Project: Sqoop Issue Type: Improvement Affects Versions: 1.4.7 Reporter: Nguyen Truong Fix For: 3.0.0 Currently MainframeImportTool refers to the MainframeManager with a string which can create a problem in refactoring. It would be beneficial to change the string to a MainframeManager class reference. -- This message was sent by Atlassian JIRA (v7.6.3#76005)