[jira] [Commented] (SQOOP-3435) Avoid NullPointerException due to different JSONObject library in classpath
[
https://issues.apache.org/jira/browse/SQOOP-3435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16808871#comment-16808871
]
Tak Lon (Stephen) Wu commented on SQOOP-3435:
-
Thanks again [~vasas]
> Avoid NullPointerException due to different JSONObject library in classpath
> ---
>
> Key: SQOOP-3435
> URL: https://issues.apache.org/jira/browse/SQOOP-3435
> Project: Sqoop
> Issue Type: Bug
>Affects Versions: 1.4.7, 1.5.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Major
> Fix For: 1.4.7, 1.5.0
>
> Attachments: SQOOP-3435.trunk.001.patch, SQOOP-3435.trunk.002.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In line of
> [SqoopOptions.java#L785|https://github.com/apache/sqoop/blob/branch-1.4.7/src/java/org/apache/sqoop/SqoopOptions.java#L785]
> with
> {\{SqoopJsonUtil.getJsonStringforMap((Map) f.get(this))); }}
> Above line should check NULL pointer like in line of
> [SqoopOptions.java#L778|https://github.com/apache/sqoop/blob/branch-1.4.7/src/java/org/apache/sqoop/SqoopOptions.java#L778]
> which has
> {{f.get(this) == null ? "null" : f.get(this).toString()}}
> Please see the stacktrace below when running command:
> {{sqoop job --create myjob -- import --connect jdbc:mysql://localhost/db
> --username root --table employee --m 1}}
> {code:java}
> 19/02/02 01:09:21 ERROR sqoop.Sqoop: Got exception running Sqoop:
> java.lang.NullPointerException
> java.lang.NullPointerException
> at org.json.JSONObject.(JSONObject.java:144)
> at
> org.apache.sqoop.util.SqoopJsonUtil.getJsonStringforMap(SqoopJsonUtil.java:43)
> at org.apache.sqoop.SqoopOptions.writeProperties(SqoopOptions.java:785)
> at
> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.createInternal(HsqldbJobStorage.java:399)
> at
> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.create(HsqldbJobStorage.java:379)
> at org.apache.sqoop.tool.JobTool.createJob(JobTool.java:181)
> at org.apache.sqoop.tool.JobTool.run(JobTool.java:294)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
> {code}
> The above {{NullPointerException}} is due to use the of
> [{{com.tdunning}}|https://github.com/tdunning/open-json/blob/rc1.8/src/main/java/org/json/JSONObject.java#L141-L155]
> as part of the HIVE libs (if one is reusing the {{HADOOP_CLASSPATH}}) in the
> classpath. but I think we can better have a checker of {{null}} in
> {{SqoopJsonUtil.getJsonStringforMap(Map map)}} before calling
> {{JSONObject pathPartMap = new JSONObject(map);}}
> Reporting this bug and the right behavior need to be decided by the assignee.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SQOOP-3134) Add option to configure Avro schema output file name with (import + --as-avrodatafile)
[
https://issues.apache.org/jira/browse/SQOOP-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16808645#comment-16808645
]
Daniel Voros commented on SQOOP-3134:
-
Tests have passed for this patch:
https://travis-ci.org/dvoros/sqoop/builds/515049441
> Add option to configure Avro schema output file name with (import +
> --as-avrodatafile)
> ---
>
> Key: SQOOP-3134
> URL: https://issues.apache.org/jira/browse/SQOOP-3134
> Project: Sqoop
> Issue Type: Improvement
>Reporter: Markus Kemper
>Assignee: Eric Lin
>Priority: Major
> Attachments: SQOOP-3134.1.patch
>
>
> Please consider adding an option to configure the Avro schema output file
> name that is created with Sqoop (import + --as-avrodatafile), example cases
> below.
> {noformat}
> #
> # STEP 01 - Create Data
> #
> export MYCONN=jdbc:mysql://mysql.cloudera.com:3306/db_coe
> export MYUSER=sqoop
> export MYPSWD=cloudera
> sqoop list-tables --connect $MYCONN --username $MYUSER --password $MYPSWD
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query
> "drop table t1"
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query
> "create table t1 (c1 int, c2 date, c3 varchar(10))"
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query
> "insert into t1 values (1, current_date, 'some data')"
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query
> "select * from t1"
> -
> | c1 | c2 | c3 |
> -
> | 1 | 2017-02-13 | some data |
> -
> #
> # STEP 02 - Import + --table + --as-avrodatafile
> #
> sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --table
> t1 --target-dir /user/root/t1 --delete-target-dir --num-mappers 1
> --as-avrodatafile
> ls -l ./*
> Output:
> 17/02/13 12:14:52 INFO mapreduce.ImportJobBase: Transferred 413 bytes in
> 20.6988 seconds (19.9529 bytes/sec)
> 17/02/13 12:14:52 INFO mapreduce.ImportJobBase: Retrieved 1 records.
>
> -rw-r--r-- 1 root root 492 Feb 13 12:14 ./t1.avsc < want option to
> configure this file name
> -rw-r--r-- 1 root root 12462 Feb 13 12:14 ./t1.java
> #
> # STEP 03 - Import + --query + --as-avrodatafile
> #
> sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --query
> "select * from t1 where \$CONDITIONS" --split-by c1 --target-dir
> /user/root/t1 --delete-target-dir --num-mappers 1 --as-avrodatafile
> ls -l ./*
> Output:
> 17/02/13 12:16:58 INFO mapreduce.ImportJobBase: Transferred 448 bytes in
> 25.2757 seconds (17.7245 bytes/sec)
> 17/02/13 12:16:58 INFO mapreduce.ImportJobBase: Retrieved 1 records.
> ~
> -rw-r--r-- 1 root root 527 Feb 13 12:16 ./AutoGeneratedSchema.avsc <
> want option to configure this file name
> -rw-r--r-- 1 root root 12590 Feb 13 12:16 ./QueryResult.java
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SQOOP-3435) Avoid NullPointerException due to different JSONObject library in classpath
[
https://issues.apache.org/jira/browse/SQOOP-3435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16808512#comment-16808512
]
Hudson commented on SQOOP-3435:
---
SUCCESS: Integrated in Jenkins build Sqoop-hadoop200 #1256 (See
[https://builds.apache.org/job/Sqoop-hadoop200/1256/])
SQOOP-3435: Avoid NullPointerException due to different JSONObject (vasas:
[https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=e90e244396ecffeb332633c89e641357adcf8eda])
* (edit) src/java/org/apache/sqoop/util/SqoopJsonUtil.java
> Avoid NullPointerException due to different JSONObject library in classpath
> ---
>
> Key: SQOOP-3435
> URL: https://issues.apache.org/jira/browse/SQOOP-3435
> Project: Sqoop
> Issue Type: Bug
>Affects Versions: 1.4.7, 1.5.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Major
> Fix For: 1.4.7, 1.5.0
>
> Attachments: SQOOP-3435.trunk.001.patch, SQOOP-3435.trunk.002.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In line of
> [SqoopOptions.java#L785|https://github.com/apache/sqoop/blob/branch-1.4.7/src/java/org/apache/sqoop/SqoopOptions.java#L785]
> with
> {\{SqoopJsonUtil.getJsonStringforMap((Map) f.get(this))); }}
> Above line should check NULL pointer like in line of
> [SqoopOptions.java#L778|https://github.com/apache/sqoop/blob/branch-1.4.7/src/java/org/apache/sqoop/SqoopOptions.java#L778]
> which has
> {{f.get(this) == null ? "null" : f.get(this).toString()}}
> Please see the stacktrace below when running command:
> {{sqoop job --create myjob -- import --connect jdbc:mysql://localhost/db
> --username root --table employee --m 1}}
> {code:java}
> 19/02/02 01:09:21 ERROR sqoop.Sqoop: Got exception running Sqoop:
> java.lang.NullPointerException
> java.lang.NullPointerException
> at org.json.JSONObject.(JSONObject.java:144)
> at
> org.apache.sqoop.util.SqoopJsonUtil.getJsonStringforMap(SqoopJsonUtil.java:43)
> at org.apache.sqoop.SqoopOptions.writeProperties(SqoopOptions.java:785)
> at
> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.createInternal(HsqldbJobStorage.java:399)
> at
> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.create(HsqldbJobStorage.java:379)
> at org.apache.sqoop.tool.JobTool.createJob(JobTool.java:181)
> at org.apache.sqoop.tool.JobTool.run(JobTool.java:294)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
> {code}
> The above {{NullPointerException}} is due to use the of
> [{{com.tdunning}}|https://github.com/tdunning/open-json/blob/rc1.8/src/main/java/org/json/JSONObject.java#L141-L155]
> as part of the HIVE libs (if one is reusing the {{HADOOP_CLASSPATH}}) in the
> classpath. but I think we can better have a checker of {{null}} in
> {{SqoopJsonUtil.getJsonStringforMap(Map map)}} before calling
> {{JSONObject pathPartMap = new JSONObject(map);}}
> Reporting this bug and the right behavior need to be decided by the assignee.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (SQOOP-3435) Avoid NullPointerException due to different JSONObject library in classpath
[
https://issues.apache.org/jira/browse/SQOOP-3435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szabolcs Vasas reassigned SQOOP-3435:
-
Assignee: Tak Lon (Stephen) Wu
> Avoid NullPointerException due to different JSONObject library in classpath
> ---
>
> Key: SQOOP-3435
> URL: https://issues.apache.org/jira/browse/SQOOP-3435
> Project: Sqoop
> Issue Type: Bug
>Affects Versions: 1.4.7, 1.5.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Major
> Attachments: SQOOP-3435.trunk.001.patch, SQOOP-3435.trunk.002.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In line of
> [SqoopOptions.java#L785|https://github.com/apache/sqoop/blob/branch-1.4.7/src/java/org/apache/sqoop/SqoopOptions.java#L785]
> with
> {\{SqoopJsonUtil.getJsonStringforMap((Map) f.get(this))); }}
> Above line should check NULL pointer like in line of
> [SqoopOptions.java#L778|https://github.com/apache/sqoop/blob/branch-1.4.7/src/java/org/apache/sqoop/SqoopOptions.java#L778]
> which has
> {{f.get(this) == null ? "null" : f.get(this).toString()}}
> Please see the stacktrace below when running command:
> {{sqoop job --create myjob -- import --connect jdbc:mysql://localhost/db
> --username root --table employee --m 1}}
> {code:java}
> 19/02/02 01:09:21 ERROR sqoop.Sqoop: Got exception running Sqoop:
> java.lang.NullPointerException
> java.lang.NullPointerException
> at org.json.JSONObject.(JSONObject.java:144)
> at
> org.apache.sqoop.util.SqoopJsonUtil.getJsonStringforMap(SqoopJsonUtil.java:43)
> at org.apache.sqoop.SqoopOptions.writeProperties(SqoopOptions.java:785)
> at
> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.createInternal(HsqldbJobStorage.java:399)
> at
> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.create(HsqldbJobStorage.java:379)
> at org.apache.sqoop.tool.JobTool.createJob(JobTool.java:181)
> at org.apache.sqoop.tool.JobTool.run(JobTool.java:294)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
> {code}
> The above {{NullPointerException}} is due to use the of
> [{{com.tdunning}}|https://github.com/tdunning/open-json/blob/rc1.8/src/main/java/org/json/JSONObject.java#L141-L155]
> as part of the HIVE libs (if one is reusing the {{HADOOP_CLASSPATH}}) in the
> classpath. but I think we can better have a checker of {{null}} in
> {{SqoopJsonUtil.getJsonStringforMap(Map map)}} before calling
> {{JSONObject pathPartMap = new JSONObject(map);}}
> Reporting this bug and the right behavior need to be decided by the assignee.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SQOOP-3435) Avoid NullPointerException due to different JSONObject library in classpath
[
https://issues.apache.org/jira/browse/SQOOP-3435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szabolcs Vasas updated SQOOP-3435:
--
Summary: Avoid NullPointerException due to different JSONObject library in
classpath (was: SqoopOptions missed a null check)
> Avoid NullPointerException due to different JSONObject library in classpath
> ---
>
> Key: SQOOP-3435
> URL: https://issues.apache.org/jira/browse/SQOOP-3435
> Project: Sqoop
> Issue Type: Bug
>Affects Versions: 1.4.7, 1.5.0
>Reporter: Tak Lon (Stephen) Wu
>Priority: Major
> Attachments: SQOOP-3435.trunk.001.patch, SQOOP-3435.trunk.002.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In line of
> [SqoopOptions.java#L785|https://github.com/apache/sqoop/blob/branch-1.4.7/src/java/org/apache/sqoop/SqoopOptions.java#L785]
> with
> {\{SqoopJsonUtil.getJsonStringforMap((Map) f.get(this))); }}
> Above line should check NULL pointer like in line of
> [SqoopOptions.java#L778|https://github.com/apache/sqoop/blob/branch-1.4.7/src/java/org/apache/sqoop/SqoopOptions.java#L778]
> which has
> {{f.get(this) == null ? "null" : f.get(this).toString()}}
> Please see the stacktrace below when running command:
> {{sqoop job --create myjob -- import --connect jdbc:mysql://localhost/db
> --username root --table employee --m 1}}
> {code:java}
> 19/02/02 01:09:21 ERROR sqoop.Sqoop: Got exception running Sqoop:
> java.lang.NullPointerException
> java.lang.NullPointerException
> at org.json.JSONObject.(JSONObject.java:144)
> at
> org.apache.sqoop.util.SqoopJsonUtil.getJsonStringforMap(SqoopJsonUtil.java:43)
> at org.apache.sqoop.SqoopOptions.writeProperties(SqoopOptions.java:785)
> at
> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.createInternal(HsqldbJobStorage.java:399)
> at
> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.create(HsqldbJobStorage.java:379)
> at org.apache.sqoop.tool.JobTool.createJob(JobTool.java:181)
> at org.apache.sqoop.tool.JobTool.run(JobTool.java:294)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
> {code}
> The above {{NullPointerException}} is due to use the of
> [{{com.tdunning}}|https://github.com/tdunning/open-json/blob/rc1.8/src/main/java/org/json/JSONObject.java#L141-L155]
> as part of the HIVE libs (if one is reusing the {{HADOOP_CLASSPATH}}) in the
> classpath. but I think we can better have a checker of {{null}} in
> {{SqoopJsonUtil.getJsonStringforMap(Map map)}} before calling
> {{JSONObject pathPartMap = new JSONObject(map);}}
> Reporting this bug and the right behavior need to be decided by the assignee.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SQOOP-3435) SqoopOptions missed a null check
[
https://issues.apache.org/jira/browse/SQOOP-3435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16808479#comment-16808479
]
ASF subversion and git services commented on SQOOP-3435:
Commit e90e244396ecffeb332633c89e641357adcf8eda in sqoop's branch
refs/heads/trunk from Tak Lon (Stephen) Wu
[ https://gitbox.apache.org/repos/asf?p=sqoop.git;h=e90e244 ]
SQOOP-3435: Avoid NullPointerException due to different JSONObject library in
classpath
> SqoopOptions missed a null check
>
>
> Key: SQOOP-3435
> URL: https://issues.apache.org/jira/browse/SQOOP-3435
> Project: Sqoop
> Issue Type: Bug
>Affects Versions: 1.4.7, 1.5.0
>Reporter: Tak Lon (Stephen) Wu
>Priority: Major
> Attachments: SQOOP-3435.trunk.001.patch, SQOOP-3435.trunk.002.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In line of
> [SqoopOptions.java#L785|https://github.com/apache/sqoop/blob/branch-1.4.7/src/java/org/apache/sqoop/SqoopOptions.java#L785]
> with
> {\{SqoopJsonUtil.getJsonStringforMap((Map) f.get(this))); }}
> Above line should check NULL pointer like in line of
> [SqoopOptions.java#L778|https://github.com/apache/sqoop/blob/branch-1.4.7/src/java/org/apache/sqoop/SqoopOptions.java#L778]
> which has
> {{f.get(this) == null ? "null" : f.get(this).toString()}}
> Please see the stacktrace below when running command:
> {{sqoop job --create myjob -- import --connect jdbc:mysql://localhost/db
> --username root --table employee --m 1}}
> {code:java}
> 19/02/02 01:09:21 ERROR sqoop.Sqoop: Got exception running Sqoop:
> java.lang.NullPointerException
> java.lang.NullPointerException
> at org.json.JSONObject.(JSONObject.java:144)
> at
> org.apache.sqoop.util.SqoopJsonUtil.getJsonStringforMap(SqoopJsonUtil.java:43)
> at org.apache.sqoop.SqoopOptions.writeProperties(SqoopOptions.java:785)
> at
> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.createInternal(HsqldbJobStorage.java:399)
> at
> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.create(HsqldbJobStorage.java:379)
> at org.apache.sqoop.tool.JobTool.createJob(JobTool.java:181)
> at org.apache.sqoop.tool.JobTool.run(JobTool.java:294)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
> {code}
> The above {{NullPointerException}} is due to use the of
> [{{com.tdunning}}|https://github.com/tdunning/open-json/blob/rc1.8/src/main/java/org/json/JSONObject.java#L141-L155]
> as part of the HIVE libs (if one is reusing the {{HADOOP_CLASSPATH}}) in the
> classpath. but I think we can better have a checker of {{null}} in
> {{SqoopJsonUtil.getJsonStringforMap(Map map)}} before calling
> {{JSONObject pathPartMap = new JSONObject(map);}}
> Reporting this bug and the right behavior need to be decided by the assignee.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [sqoop] szvasas merged pull request #77: SQOOP-3435 Avoid NullPointerException due to different JSONObject library in classpath
szvasas merged pull request #77: SQOOP-3435 Avoid NullPointerException due to different JSONObject library in classpath URL: https://github.com/apache/sqoop/pull/77 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
