[jira] [Commented] (SQOOP-2903) Add Kudu connector for Sqoop

2017-07-06 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/SQOOP-2903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16077283#comment-16077283
 ] 

ASF GitHub Bot commented on SQOOP-2903:
---

GitHub user cammachusa opened a pull request:

https://github.com/apache/sqoop/pull/37

[SQOOP-2903] - Add Kudu connector for Sqoop



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/InspurUSA/sqoop SQOOP-2903

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/sqoop/pull/37.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #37


commit c9c07b2e8eabfb2066187c42a4d533e29ceded6c
Author: cam 
Date:   2017-07-04T00:04:41Z

[SQOOP-2903] - Add Kudu connector for Sqoop




> Add Kudu connector for Sqoop
> 
>
> Key: SQOOP-2903
> URL: https://issues.apache.org/jira/browse/SQOOP-2903
> Project: Sqoop
>  Issue Type: Improvement
>  Components: connectors
>Reporter: Sameer Abhyankar
>Assignee: Sameer Abhyankar
> Attachments: SQOOP-2903.1.patch, SQOOP-2903.2.patch, SQOOP-2903.patch
>
>
> Sqoop currently does not have a connector for Kudu. We should add the 
> functionality to allow Sqoop to ingest data directly into Kudu.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] sqoop pull request #37: [SQOOP-2903] - Add Kudu connector for Sqoop

2017-07-06 Thread cammachusa 
GitHub user cammachusa opened a pull request:

https://github.com/apache/sqoop/pull/37

[SQOOP-2903] - Add Kudu connector for Sqoop



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/InspurUSA/sqoop SQOOP-2903

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/sqoop/pull/37.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #37


commit c9c07b2e8eabfb2066187c42a4d533e29ceded6c
Author: cam 
Date:   2017-07-04T00:04:41Z

[SQOOP-2903] - Add Kudu connector for Sqoop




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (SQOOP-3197) Database credentials cannot be configured in PostgreSQL third party tests

2017-07-06 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/SQOOP-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076668#comment-16076668
 ] 

Hudson commented on SQOOP-3197:
---

SUCCESS: Integrated in Jenkins build Sqoop-hadoop200 #1118 (See 
[https://builds.apache.org/job/Sqoop-hadoop200/1118/])
SQOOP-3197: Database credentials cannot be configured in PostgreSQL 
(annaszonyi: 
[https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=7a6d34946582cb2598dcc50edc9f389e9a42c969])
* (edit) 
src/test/com/cloudera/sqoop/manager/DirectPostgreSQLExportManualTest.java
* (edit) src/test/com/cloudera/sqoop/manager/PostgresqlExportTest.java
* (edit) src/test/com/cloudera/sqoop/manager/PostgresqlImportTest.java
* (edit) build.xml
* (edit) 
src/test/com/cloudera/sqoop/manager/PostgresqlExternalTableImportTest.java


> Database credentials cannot be configured in PostgreSQL third party tests
> -
>
> Key: SQOOP-3197
> URL: https://issues.apache.org/jira/browse/SQOOP-3197
> Project: Sqoop
>  Issue Type: Test
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
> Attachments: SQOOP-3197.patch
>
>
> PostgreSQL third party tests look like they support configurable database 
> parameters (e.g. they load the database user, name and password from a system 
> property) but the property names in the test cases do not match the property 
> names in the build.xml and because of that these parameters cannot be 
> specified from the command line.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (SQOOP-3197) Database credentials cannot be configured in PostgreSQL third party tests

2017-07-06 Thread Anna Szonyi (JIRA) 

[ 
https://issues.apache.org/jira/browse/SQOOP-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076627#comment-16076627
 ] 

Anna Szonyi commented on SQOOP-3197:


Hey Szabolcs,

Thanks for your continued improvements/standardization to the tests we are 
running!

Thanks,
Anna

> Database credentials cannot be configured in PostgreSQL third party tests
> -
>
> Key: SQOOP-3197
> URL: https://issues.apache.org/jira/browse/SQOOP-3197
> Project: Sqoop
>  Issue Type: Test
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
> Attachments: SQOOP-3197.patch
>
>
> PostgreSQL third party tests look like they support configurable database 
> parameters (e.g. they load the database user, name and password from a system 
> property) but the property names in the test cases do not match the property 
> names in the build.xml and because of that these parameters cannot be 
> specified from the command line.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (SQOOP-3197) Database credentials cannot be configured in PostgreSQL third party tests

2017-07-06 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/SQOOP-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076622#comment-16076622
 ] 

ASF subversion and git services commented on SQOOP-3197:


Commit 7a6d34946582cb2598dcc50edc9f389e9a42c969 in sqoop's branch 
refs/heads/trunk from [~anna.szonyi]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=7a6d349 ]

SQOOP-3197: Database credentials cannot be configured in PostgreSQL third party 
tests

(Szabolcs Vasas via Anna Szonyi)


> Database credentials cannot be configured in PostgreSQL third party tests
> -
>
> Key: SQOOP-3197
> URL: https://issues.apache.org/jira/browse/SQOOP-3197
> Project: Sqoop
>  Issue Type: Test
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
> Attachments: SQOOP-3197.patch
>
>
> PostgreSQL third party tests look like they support configurable database 
> parameters (e.g. they load the database user, name and password from a system 
> property) but the property names in the test cases do not match the property 
> names in the build.xml and because of that these parameters cannot be 
> specified from the command line.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 59710: SQOOP-3190: Remove dependency on PSQL for postgres direct import

2017-07-06 Thread Szabolcs Vasas 


> On July 3, 2017, 9:55 a.m., Szabolcs Vasas wrote:
> > Hi Alex,
> > 
> > I can still see some reformats in the diffs, can you please fix those as 
> > well? Apart from that when I tried to apply the patch locally I got the 
> > following warnings, please address these as well:
> > 
> > /Users/szabolcsvasas/Downloads/DirectPostgresqlManager_v2.patch:312: 
> > trailing whitespace.
> >   private CopyManager getPostgresCopyAPI(ImportJobContext context) 
> > /Users/szabolcsvasas/Downloads/DirectPostgresqlManager_v2.patch:314: 
> > trailing whitespace.
> >   
> > /Users/szabolcsvasas/Downloads/DirectPostgresqlManager_v2.patch:322: 
> > trailing whitespace.
> >   
> > /Users/szabolcsvasas/Downloads/DirectPostgresqlManager_v2.patch:324: 
> > trailing whitespace.
> > conn = dbConf.getConnection(); 
> > /Users/szabolcsvasas/Downloads/DirectPostgresqlManager_v2.patch:335: 
> > trailing whitespace.
> >   
> > warning: squelched 8 whitespace errors
> > warning: 13 lines add whitespace errors.
> > 
> > I have also replied to some of your comments below.
> 
> Alex Ang wrote:
> I have replied to the above comments. 
> On the format issues, is there any preferred code templates and/or 
> formatter configuration that I can use with Eclipse?

Unfortunately we do not have a formatter all the contributors use our general 
guideline is however not to reformat the parts of the code which are otherwise 
not changed by the patch. So restoring the original state is fine.


- Szabolcs


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59710/#review179489
---


On June 30, 2017, 5:17 p.m., Alex Ang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59710/
> ---
> 
> (Updated June 30, 2017, 5:17 p.m.)
> 
> 
> Review request for Sqoop and Szabolcs Vasas.
> 
> 
> Bugs: SQOOP-3190
> https://issues.apache.org/jira/browse/SQOOP-3190
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> ---
> 
> Current Postgres direct import requires the installation of the psql utility 
> on all hadoop nodes. However, due to system constraints, this may not always 
> be possible. 
> It should be noted that Postgres provides a Java API which can execute the 
> same copy command as the psql utility. As such, it would be more ideal to 
> remove the psql utility dependency and switch to the Postgres Java API.
> 
> 
> Diffs
> -
> 
>   src/java/org/apache/sqoop/manager/DirectPostgresqlManager.java 63b0704 
> 
> 
> Diff: https://reviews.apache.org/r/59710/diff/2/
> 
> 
> Testing
> ---
> 
> Tested with PostgresqlImportTest and passed all tests.
> 
> 
> Thanks,
> 
> Alex Ang
> 
>

Re: Review Request 59710: SQOOP-3190: Remove dependency on PSQL for postgres direct import

2017-07-06 Thread Szabolcs Vasas 


> On June 30, 2017, 1 p.m., Szabolcs Vasas wrote:
> > src/java/org/apache/sqoop/manager/DirectPostgresqlManager.java
> > Lines 281 (patched)
> > 
> >
> > The password is not set in the Configuration and because of this some 
> > test cases in PostgresqlImportTest fail.
> 
> Alex Ang wrote:
> Added the password into the config.
> 
> Szabolcs Vasas wrote:
> I think adding the password is a bit trickier than just adding to the 
> conf, PostgresqlImportTest still fails with the same error on my side.
> 
> Alex Ang wrote:
> I have some difficulty setting the postgres db where the original code 
> and PostgresqlExportTest also fail due to  org.postgresql.util.PSQLException: 
> FATAL: password authentication failed for user "sqooptest". 
> This would take some time as I have to sort out the environment. 
> Need to fix this issue before I can post the new patch.
> 
> Pls note that in the PostgresqlImportTest and the cwiki 
> (https://cwiki.apache.org/confluence/display/SQOOP/Setting+up+Sqoop+1#SettingupSqoop1-SettingupPostgreSQL),
>  the environment setup uses trust where password are not required. 
> Would appreciate if you are able to guide me to the correct setup.
> FYI, I had success using --password-file in another setup.

Sorry, I haven't taken into consideration that 
https://issues.apache.org/jira/browse/SQOOP-3197 is not yet committed. A few 
weeks ago I have realized that PostgreSQL third party tests cannot handle 
database passwords, most probably this was the reason the Wiki suggested 
creating a user without password. But this need to be fixed since users have 
passwords in production. I will try to have SQOOP-3197 committed then you can 
verify your code change. Without the fix you could test it manually. If you 
want to specify a password for a new user you can use this command:

CREATE USER sqooptest with PASSWORD 'sqooptest';


> On June 30, 2017, 1 p.m., Szabolcs Vasas wrote:
> > src/java/org/apache/sqoop/manager/DirectPostgresqlManager.java
> > Lines 283 (patched)
> > 
> >
> > I am not sure why the JobConf object is needed here. The 
> > DBConfiguration constructor seems to expect a Configuration object, so conf 
> > could be passed as a parameter, couldn't it?
> 
> Alex Ang wrote:
> DBConfiguration expects a org.apache.hadoop.mapred.JobConf and not 
> org.apache.hadoop.conf.Configuration. Without the wrapper, ClassCastException 
> will be thrown.
> 
> Szabolcs Vasas wrote:
> Are you sure you use the trunk branch? On trunk I can see this 
> constructor: 
> http://github.mtv.cloudera.com/CDH/sqoop/blob/cdh5-1.4.6/src/java/org/apache/sqoop/mapreduce/db/DBConfiguration.java#L266
> 
> Alex Ang wrote:
> I am using the trunk from https://github.com/apache/sqoop 
> Is there somewhere else I should be getting the source from?

Sorry, I have posted a wrong URL, but I use the same repo as you. Meanwhile I 
have realized what you mean, the constructor parameter is defined as 
Configuration but later it is casted to JobConf.


> On June 30, 2017, 1 p.m., Szabolcs Vasas wrote:
> > src/java/org/apache/sqoop/manager/DirectPostgresqlManager.java
> > Line 505 (original), 312 (patched)
> > 
> >
> > I think the string encoding should be specified here to avoid using 
> > defaults.
> 
> Alex Ang wrote:
> Could you advise where the encoding can be retrieved or I should default 
> it to UTF-8? Updated to UTF-8 in the new diff.
> 
> Szabolcs Vasas wrote:
> UTF-8 seems to be used in 
> org.apache.sqoop.mapreduce.postgresql.PostgreSQLCopyExportMapper too but I 
> think you should check what is the encoding used by the psql command because 
> some users might rely on it.
> 
> Alex Ang wrote:
> I have defaulted the encoding to follow the database as per the postgres 
> CopyManager code (line 94 
> https://github.com/pgjdbc/pgjdbc/blob/master/pgjdbc/src/main/java/org/postgresql/copy/CopyManager.java).
>  
> Pls let me know if this is acceptable.

I am not sure what you mean, but I think it is a good idea to use the Encoding 
object to encode the byte to strings. And I guess you should initialize it like 
this: this.encoding = connection.getEncoding();


- Szabolcs


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59710/#review177994
---


On June 30, 2017, 5:17 p.m., Alex Ang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59710/
> ---
> 
> (Updated June 30, 2017, 5:17 p.m.)
> 
> 
> Review reque

[jira] [Commented] (SQOOP-3206) Make sqoop fail if user uses --direct connector and tries to encode a null value when using a MySQL database

2017-07-06 Thread Anna Szonyi (JIRA) 

[ 
https://issues.apache.org/jira/browse/SQOOP-3206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076455#comment-16076455
 ] 

Anna Szonyi commented on SQOOP-3206:


Hey Zach,

Thank you for your great contribution! :)

Thanks,
Anna

> Make sqoop fail if user uses --direct connector and tries to encode a null 
> value when using a MySQL database
> 
>
> Key: SQOOP-3206
> URL: https://issues.apache.org/jira/browse/SQOOP-3206
> Project: Sqoop
>  Issue Type: Improvement
>Reporter: Zach Berkowitz
>Assignee: Zach Berkowitz
>Priority: Minor
> Attachments: sqoop-3206.patch
>
>
> The MySQL tools, mysqldump and mysqlimport, do not support custom null value 
> options, so sqoop should fail fast when --direct mode and 
> --input-null-string, --input-null-non-string, --null-string, or 
> --null-non-string are used.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (SQOOP-3186) Add Sqoop1 (import + --incremental + --check-column) support for functions/expressions

2017-07-06 Thread Eric Lin (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SQOOP-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Lin reassigned SQOOP-3186:
---

Assignee: Eric Lin

> Add Sqoop1 (import + --incremental + --check-column) support for 
> functions/expressions
> --
>
> Key: SQOOP-3186
> URL: https://issues.apache.org/jira/browse/SQOOP-3186
> Project: Sqoop
>  Issue Type: Improvement
>Reporter: Markus Kemper
>Assignee: Eric Lin
>
> Add Sqoop1 (import + --incremental + --check-column) support for 
> functions/expressions, for example:
> *Example*
> {noformat}
> sqoop import \
> --connect $MYCONN --username $MYUSER --password $MYPSWD \
> --table T1 --target-dir /path/directory --merge-key C1 \
> --incremental lastmodified  --last-value '2017-01-01 00:00:00.0' \
> --check-column nvl(C4,to_date('2017-01-01 00:00:00')
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (SQOOP-3188) Sqoop1 (import + --target-dir) with empty directory (/usr/lib/hive) fails with error (java.lang.NoClassDefFoundError: org/json/JSONObject)

2017-07-06 Thread Eric Lin (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SQOOP-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Lin reassigned SQOOP-3188:
---

Assignee: Eric Lin

> Sqoop1 (import + --target-dir) with empty directory (/usr/lib/hive) fails 
> with error (java.lang.NoClassDefFoundError: org/json/JSONObject)
> --
>
> Key: SQOOP-3188
> URL: https://issues.apache.org/jira/browse/SQOOP-3188
> Project: Sqoop
>  Issue Type: Bug
>Reporter: Markus Kemper
>Assignee: Eric Lin
>
> Sqoop1 (import + --target-dir) with empty directory (/usr/lib/hive) fails 
> with error (java.lang.NoClassDefFoundError: org/json/JSONObject), see test 
> case below.
> *Test Case*
> {noformat}
> #
> # STEP 01 - Create Table and Data
> #
> export MYCONN=jdbc:mysql://mysql.sqoop.com:3306/sqoop
> export MYUSER=sqoop
> export MYPSWD=sqoop
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query 
> "drop table t1"
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query 
> "create table t1 (c1 int, c2 date, c3 varchar(10))"
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query 
> "insert into t1 values (1, current_date, 'some data')"
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query 
> "select * from t1"
> Output:
> -
> | c1  | c2 | c3 | 
> -
> | 1   | 2017-05-10 | some data  | 
> -
> #
> # STEP 02 - Import Data into HDFS 
> #
> sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --table 
> t1 --target-dir /user/root/t1 --delete-target-dir --num-mappers 1
> hdfs dfs -cat /user/root/t1/part*
> Output:
> 17/05/10 13:46:24 INFO mapreduce.ImportJobBase: Transferred 23 bytes in 22.65 
> seconds (1.0155 bytes/sec)
> 17/05/10 13:46:24 INFO mapreduce.ImportJobBase: Retrieved 1 records.
> ~
> 1,2017-05-10,some data
> #
> # STEP 03 - Create Bogus Hive Directory and Attempt to Import into HDFS
> #
> mkdir /usr/lib/hive
> chmod 777 /usr/lib/hive
> sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --table 
> t1 --target-dir /user/root/t1 --delete-target-dir --num-mappers 1
> Output:
> 17/05/10 13:47:44 INFO mapreduce.ImportJobBase: Beginning import of t1
> Exception in thread "main" java.lang.NoClassDefFoundError: org/json/JSONObject
>   at 
> org.apache.sqoop.util.SqoopJsonUtil.getJsonStringforMap(SqoopJsonUtil.java:43)
>   at org.apache.sqoop.SqoopOptions.writeProperties(SqoopOptions.java:776)
>   at 
> org.apache.sqoop.mapreduce.JobBase.putSqoopOptionsToConfiguration(JobBase.java:388)
>   at org.apache.sqoop.mapreduce.JobBase.createJob(JobBase.java:374)
>   at 
> org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:256)
>   at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692)
>   at 
> org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:127)
>   at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:513)
>   at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621)
>   at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
>   at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
>   at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
>   at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
> Caused by: java.lang.ClassNotFoundException: org.json.JSONObject
>   at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>   at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>   ... 15 more
> #
> # STEP 04 - Remove Bogus Hive Directory and Attempt to Import into HDFS 
> #
> rm -rf /usr/lib/hive
> sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --table 
> t1 --target-dir /user/root/t1 --delete-target-dir --num-mappers 1
> hdfs dfs -cat /user/root/t1/part*
> Output:
> 17/05/10 13:52:30 INFO mapreduce.ImportJobBase: Transferred 23 bytes in 
> 22.6361 seconds (1.0161 bytes/sec)
> 17/05/10 13:52:30 INFO mapreduce.ImportJobBase: Retrieved 1 records.
> ~
> 1,2017-05-10,some data
> {noformat}



--
This mes

[jira] [Commented] (SQOOP-3189) Issue while reading double(data type) data from Sql server DB and loading in to Hive String (data type) with Sqoop

2017-07-06 Thread Eric Lin (JIRA) 

[ 
https://issues.apache.org/jira/browse/SQOOP-3189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076266#comment-16076266
 ] 

Eric Lin commented on SQOOP-3189:
-

Hi [~sravan777],

Can you please clarify whether this is Sqoop1 or Sqoop2 issue? Can you please 
also provide the full command you used to run Sqoop?

This will help us to understand the issue better.

Thanks

> Issue while reading double(data type) data from Sql server DB and loading in 
> to Hive String (data type) with Sqoop
> --
>
> Key: SQOOP-3189
> URL: https://issues.apache.org/jira/browse/SQOOP-3189
> Project: Sqoop
>  Issue Type: Bug
>  Components: sqoop2-client
>Reporter: sravan kumar
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (SQOOP-3193) Sqoop comamnd to overwrite the table

2017-07-06 Thread Eric Lin (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SQOOP-3193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Lin updated SQOOP-3193:

Component/s: (was: sqoop2-shell)

> Sqoop comamnd to overwrite the table
> 
>
> Key: SQOOP-3193
> URL: https://issues.apache.org/jira/browse/SQOOP-3193
> Project: Sqoop
>  Issue Type: Bug
>Affects Versions: 1.4.7
> Environment: linux
>Reporter: Jayanthi R
>  Labels: features
> Fix For: no-release
>
>
> I am using sqoop import command and I want to overwrite the data when I use 
> it for second or third time (except the first time)
> sqoop import --connect jdbc:postgresql://reuxeuls677.bp.com/auditdb 
> --username audit_user --password theaudituserloginpassword --table 
> test_automation --hive-overwrite --hive-table sqoophive -m 1  --hive-import 
> --hive-database 'test_automation_db' --create-hive-table --hive-table 
> test_automation;
> But, I am getting the following error:
> 17/06/06 09:34:13 ERROR tool.ImportTool: Encountered IOException running 
> import job: org.apache.hadoop.mapred.FileAlreadyExistsException: Output 
> directory hdfs://HDPEMDCPROD/user/dluser/test_automation already exists



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (SQOOP-3193) Sqoop comamnd to overwrite the table

2017-07-06 Thread Eric Lin (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SQOOP-3193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Lin updated SQOOP-3193:

Affects Version/s: (was: 1.99.7)
   1.4.7

> Sqoop comamnd to overwrite the table
> 
>
> Key: SQOOP-3193
> URL: https://issues.apache.org/jira/browse/SQOOP-3193
> Project: Sqoop
>  Issue Type: Bug
>Affects Versions: 1.4.7
> Environment: linux
>Reporter: Jayanthi R
>  Labels: features
> Fix For: no-release
>
>
> I am using sqoop import command and I want to overwrite the data when I use 
> it for second or third time (except the first time)
> sqoop import --connect jdbc:postgresql://reuxeuls677.bp.com/auditdb 
> --username audit_user --password theaudituserloginpassword --table 
> test_automation --hive-overwrite --hive-table sqoophive -m 1  --hive-import 
> --hive-database 'test_automation_db' --create-hive-table --hive-table 
> test_automation;
> But, I am getting the following error:
> 17/06/06 09:34:13 ERROR tool.ImportTool: Encountered IOException running 
> import job: org.apache.hadoop.mapred.FileAlreadyExistsException: Output 
> directory hdfs://HDPEMDCPROD/user/dluser/test_automation already exists



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (SQOOP-3206) Make sqoop fail if user uses --direct connector and tries to encode a null value when using a MySQL database

2017-07-06 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/SQOOP-3206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076253#comment-16076253
 ] 

Hudson commented on SQOOP-3206:
---

SUCCESS: Integrated in Jenkins build Sqoop-hadoop200 #1117 (See 
[https://builds.apache.org/job/Sqoop-hadoop200/1117/])
SQOOP-3206: Make sqoop fail if user uses --direct connector and tries to 
(annaszonyi: 
[https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=2c199a7f4fc744f653f036d5afb6f49b0d4c445c])
* (edit) src/test/com/cloudera/sqoop/manager/DirectMySQLTest.java
* (edit) src/test/org/apache/sqoop/tool/TestValidateImportOptions.java
* (edit) src/test/com/cloudera/sqoop/manager/DirectMySQLExportTest.java
* (edit) src/java/org/apache/sqoop/tool/ExportTool.java
* (edit) src/java/org/apache/sqoop/tool/ImportTool.java
* (edit) src/test/org/apache/sqoop/tool/TestExportToolValidateOptions.java


> Make sqoop fail if user uses --direct connector and tries to encode a null 
> value when using a MySQL database
> 
>
> Key: SQOOP-3206
> URL: https://issues.apache.org/jira/browse/SQOOP-3206
> Project: Sqoop
>  Issue Type: Improvement
>Reporter: Zach Berkowitz
>Assignee: Zach Berkowitz
>Priority: Minor
> Attachments: sqoop-3206.patch
>
>
> The MySQL tools, mysqldump and mysqlimport, do not support custom null value 
> options, so sqoop should fail fast when --direct mode and 
> --input-null-string, --input-null-non-string, --null-string, or 
> --null-non-string are used.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (SQOOP-3206) Make sqoop fail if user uses --direct connector and tries to encode a null value when using a MySQL database

2017-07-06 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/SQOOP-3206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076225#comment-16076225
 ] 

ASF subversion and git services commented on SQOOP-3206:


Commit 2c199a7f4fc744f653f036d5afb6f49b0d4c445c in sqoop's branch 
refs/heads/trunk from [~anna.szonyi]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=2c199a7 ]

SQOOP-3206: Make sqoop fail if user uses --direct connector and tries to encode 
a null value when using a MySQL database

(Zach Berkowitz via Anna Szonyi)


> Make sqoop fail if user uses --direct connector and tries to encode a null 
> value when using a MySQL database
> 
>
> Key: SQOOP-3206
> URL: https://issues.apache.org/jira/browse/SQOOP-3206
> Project: Sqoop
>  Issue Type: Improvement
>Reporter: Zach Berkowitz
>Assignee: Zach Berkowitz
>Priority: Minor
> Attachments: sqoop-3206.patch
>
>
> The MySQL tools, mysqldump and mysqlimport, do not support custom null value 
> options, so sqoop should fail fast when --direct mode and 
> --input-null-string, --input-null-non-string, --null-string, or 
> --null-non-string are used.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)