Review Request 65480: HIVE-18545 Add UDF to parse complex types from json
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/65480/ --- Review request for hive and Ashutosh Chauhan. Bugs: HIVE-18545 https://issues.apache.org/jira/browse/HIVE-18545 Repository: hive-git Description --- add json_read udf Diffs - itests/hive-jmh/src/main/java/org/apache/hive/benchmark/udf/json_read/JsonReadBench.java PRE-CREATION itests/hive-jmh/src/main/resources/org/apache/hive/benchmark/udf/json_read/val1.json PRE-CREATION itests/hive-jmh/src/main/resources/org/apache/hive/benchmark/udf/json_read/val1.type PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 76e85636d1ecddc720d6b6e3680194354a6e157c ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFJsonRead.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFJsonRead.java PRE-CREATION ql/src/test/queries/clientpositive/udf_json_read.q PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out 43e4a5de393d4b23c4c0257f08c32dd650eaaadc ql/src/test/results/clientpositive/udf_json_read.q.out PRE-CREATION Diff: https://reviews.apache.org/r/65480/diff/1/ Testing --- Thanks, Zoltan Haindrich
Re: Review Request 65480: HIVE-18545 Add UDF to parse complex types from json
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/65480/ --- (Updated June 15, 2018, 2:12 p.m.) Review request for hive and Ashutosh Chauhan. Changes --- use the same in serde/udf; some refactor Bugs: HIVE-18545 https://issues.apache.org/jira/browse/HIVE-18545 Repository: hive-git Description (updated) --- * add a generalized reader for json2hive struct parsing * add json_read udf ; tests * rename the new "copied" the new jsonserde "serde2.JsonSerde" to "serde2.json.JsonSerde2" - I think it would be bad to have 2 classes named the same ; in the same project, it would even confuse users - because only the package path was different * make the hcat jsonserde use the "serde2" version as a back end - so that only 1 real jsonserde exists in hive right now... Diffs (updated) - hcatalog/core/src/main/java/org/apache/hive/hcatalog/data/JsonSerDe.java 114c205c83 hcatalog/core/src/test/java/org/apache/hive/hcatalog/data/TestJsonSerDe.java 8aeb4f4f10 itests/hive-jmh/src/main/java/org/apache/hive/benchmark/udf/json_read/JsonReadBench.java PRE-CREATION itests/hive-jmh/src/main/resources/org/apache/hive/benchmark/udf/json_read/val1.json PRE-CREATION itests/hive-jmh/src/main/resources/org/apache/hive/benchmark/udf/json_read/val1.type PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e77fe18129 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFJsonRead.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFJsonRead.java PRE-CREATION ql/src/test/queries/clientpositive/json_serde2.q PRE-CREATION ql/src/test/queries/clientpositive/udf_json_read.q PRE-CREATION ql/src/test/results/clientpositive/json_serde2.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out 91d3660aa0 ql/src/test/results/clientpositive/udf_json_read.q.out PRE-CREATION serde/src/java/org/apache/hadoop/hive/serde2/JsonSerDe.java 1119fa2af5 serde/src/java/org/apache/hadoop/hive/serde2/json/HiveJsonStructReader.java PRE-CREATION serde/src/java/org/apache/hadoop/hive/serde2/json/JsonSerDe2.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/StrictJsonWriter.java cabb64cdcc Diff: https://reviews.apache.org/r/65480/diff/2/ Changes: https://reviews.apache.org/r/65480/diff/1-2/ Testing (updated) --- * added some new tests for the udf Thanks, Zoltan Haindrich
Re: Review Request 65480: HIVE-18545 Add UDF to parse complex types from json
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/65480/ --- (Updated June 19, 2018, 7:02 a.m.) Review request for hive and Ashutosh Chauhan. Changes --- update patch6; does not moves serde2.jsonserde Bugs: HIVE-18545 https://issues.apache.org/jira/browse/HIVE-18545 Repository: hive-git Description (updated) --- * add a generalized reader for json2hive struct parsing * add json_read udf ; tests * make the hcat jsonserde use the "serde2" version as a back end - so that only 1 real jsonserde exists in hive right now... Diffs (updated) - hcatalog/core/src/main/java/org/apache/hive/hcatalog/data/JsonSerDe.java 114c205c83 hcatalog/core/src/test/java/org/apache/hive/hcatalog/data/TestJsonSerDe.java 8aeb4f4f10 itests/hive-jmh/src/main/java/org/apache/hive/benchmark/udf/json_read/JsonReadBench.java PRE-CREATION itests/hive-jmh/src/main/resources/org/apache/hive/benchmark/udf/json_read/val1.json PRE-CREATION itests/hive-jmh/src/main/resources/org/apache/hive/benchmark/udf/json_read/val1.type PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e77fe18129 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFJsonRead.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFJsonRead.java PRE-CREATION ql/src/test/queries/clientpositive/json_serde2.q PRE-CREATION ql/src/test/queries/clientpositive/udf_json_read.q PRE-CREATION ql/src/test/results/clientpositive/json_serde2.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out 91d3660aa0 ql/src/test/results/clientpositive/udf_json_read.q.out PRE-CREATION serde/src/java/org/apache/hadoop/hive/serde2/JsonSerDe.java 1119fa2af5 serde/src/java/org/apache/hadoop/hive/serde2/json/HiveJsonStructReader.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/StrictJsonWriter.java cabb64cdcc Diff: https://reviews.apache.org/r/65480/diff/3/ Changes: https://reviews.apache.org/r/65480/diff/2-3/ Testing --- * added some new tests for the udf Thanks, Zoltan Haindrich