[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 1: (7 comments) http://gerrit.cloudera.org:8080/#/c/19304/1/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java File fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java: http://gerrit.cloudera.org:8080/#/c/19304/1/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@32 PS1, Line 32: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/1/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java: http://gerrit.cloudera.org:8080/#/c/19304/1/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@25 PS1, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/1/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@26 PS1, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveWritableObjectInspector; line too long (104 > 90) http://gerrit.cloudera.org:8080/#/c/19304/1/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java File fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/1/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@25 PS1, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/1/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@26 PS1, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory; line too long (95 > 90) http://gerrit.cloudera.org:8080/#/c/19304/1/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/1/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@25 PS1, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/1/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@26 PS1, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory; line too long (95 > 90) -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 01 Dec 2022 18:06:21 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 2: (7 comments) http://gerrit.cloudera.org:8080/#/c/19304/2/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java File fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java: http://gerrit.cloudera.org:8080/#/c/19304/2/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@32 PS2, Line 32: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/2/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java: http://gerrit.cloudera.org:8080/#/c/19304/2/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@25 PS2, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/2/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@26 PS2, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveWritableObjectInspector; line too long (104 > 90) http://gerrit.cloudera.org:8080/#/c/19304/2/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java File fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/2/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@25 PS2, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/2/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@26 PS2, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory; line too long (95 > 90) http://gerrit.cloudera.org:8080/#/c/19304/2/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/2/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@25 PS2, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/2/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@26 PS2, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory; line too long (95 > 90) -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 01 Dec 2022 18:08:05 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Csaba Ringhofer has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19304 Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. IMPALA-11549: Support Hive GenericUdfs that return primitive java types Before this patch only the Writable* types were accepted in GenericUdfs as return types, while some GenericUdfs in the wild return primitive java types (e.g. Integer instead of IntWritable). For legacy Hive UDFs these return types were already handled, so the only change needed was to map the ObjectInspector subclasses (e.g. JavaIntObjectInspector) to the correct JavaUdfDataType in Impala. Testing: - Added a subclass for TestGenericUdf (TestGenericUdfWithJavaReturnTypes) that returns primitive java types (probably inheriting in the opposite direction would be more logical, but the diff is smaller this way). - Changed EE tests to also use TestGenericUdfWithJavaReturnTypes. - Changed FE tests (UdfExecutorTest) to check both TestGenericUdfWithJavaReturnTypes and TestGenericUdf. - Also added a test with BINARY type to UdfExecutorTest as this was forgotten during the original BINARY patch. Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 --- M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java A fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java A java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test 10 files changed, 670 insertions(+), 183 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/19304/1 -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19304 to look at the new patch set (#2). Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. IMPALA-11549: Support Hive GenericUdfs that return primitive java types Before this patch only the Writable* types were accepted in GenericUdfs as return types, while some GenericUdfs in the wild return primitive java types (e.g. Integer instead of IntWritable). For legacy Hive UDFs these return types were already handled, so the only change needed was to map the ObjectInspector subclasses (e.g. JavaIntObjectInspector) to the correct JavaUdfDataType in Impala. Testing: - Added a subclass for TestGenericUdf (TestGenericUdfWithJavaReturnTypes) that returns primitive java types (probably inheriting in the opposite direction would be more logical, but the diff is smaller this way). - Changed EE tests to also use TestGenericUdfWithJavaReturnTypes. - Changed FE tests (UdfExecutorTest) to check both TestGenericUdfWithJavaReturnTypes and TestGenericUdf. - Also added a test with BINARY type to UdfExecutorTest as this was forgotten during the original BINARY patch. Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 --- M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java A fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java A java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test 10 files changed, 670 insertions(+), 184 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/19304/2 -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19304 to look at the new patch set (#3). Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. IMPALA-11549: Support Hive GenericUdfs that return primitive java types Before this patch only the Writable* types were accepted in GenericUdfs as return types, while some GenericUdfs in the wild return primitive java types (e.g. Integer instead of IntWritable). For legacy Hive UDFs these return types were already handled, so the only change needed was to map the ObjectInspector subclasses (e.g. JavaIntObjectInspector) to the correct JavaUdfDataType in Impala. Testing: - Added a subclass for TestGenericUdf (TestGenericUdfWithJavaReturnTypes) that returns primitive java types (probably inheriting in the opposite direction would be more logical, but the diff is smaller this way). - Changed EE tests to also use TestGenericUdfWithJavaReturnTypes. - Changed FE tests (UdfExecutorTest) to check both TestGenericUdfWithJavaReturnTypes and TestGenericUdf. - Also added a test with BINARY type to UdfExecutorTest as this was forgotten during the original BINARY patch. Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 --- M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java A fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java A java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test 10 files changed, 670 insertions(+), 184 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/19304/3 -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 3 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 3: (7 comments) http://gerrit.cloudera.org:8080/#/c/19304/3/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java File fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java: http://gerrit.cloudera.org:8080/#/c/19304/3/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@32 PS3, Line 32: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/3/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java: http://gerrit.cloudera.org:8080/#/c/19304/3/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@25 PS3, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/3/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@26 PS3, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveWritableObjectInspector; line too long (104 > 90) http://gerrit.cloudera.org:8080/#/c/19304/3/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java File fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/3/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@25 PS3, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/3/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@26 PS3, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory; line too long (95 > 90) http://gerrit.cloudera.org:8080/#/c/19304/3/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/3/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@25 PS3, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/3/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@26 PS3, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory; line too long (95 > 90) -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 3 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 01 Dec 2022 18:21:40 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19304 to look at the new patch set (#4). Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. IMPALA-11549: Support Hive GenericUdfs that return primitive java types Before this patch only the Writable* types were accepted in GenericUdfs as return types, while some GenericUdfs in the wild return primitive java types (e.g. Integer instead of IntWritable). For legacy Hive UDFs these return types were already handled, so the only change needed was to map the ObjectInspector subclasses (e.g. JavaIntObjectInspector) to the correct JavaUdfDataType in Impala. Testing: - Added a subclass for TestGenericUdf (TestGenericUdfWithJavaReturnTypes) that returns primitive java types (probably inheriting in the opposite direction would be more logical, but the diff is smaller this way). - Changed EE tests to also use TestGenericUdfWithJavaReturnTypes. - Changed FE tests (UdfExecutorTest) to check both TestGenericUdfWithJavaReturnTypes and TestGenericUdf. - Also added a test with BINARY type to UdfExecutorTest as this was forgotten during the original BINARY patch. Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 --- M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java A fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java A java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test 10 files changed, 670 insertions(+), 184 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/19304/4 -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 4 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 4: (7 comments) http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java File fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java: http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@32 PS4, Line 32: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java: http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@25 PS4, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@26 PS4, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveWritableObjectInspector; line too long (104 > 90) http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java File fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@25 PS4, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@26 PS4, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory; line too long (95 > 90) http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@25 PS4, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@26 PS4, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory; line too long (95 > 90) -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 4 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 01 Dec 2022 18:22:52 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/11935/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 01 Dec 2022 18:25:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/11936/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 01 Dec 2022 18:28:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 3: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/11937/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 3 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 01 Dec 2022 18:34:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/11938/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 4 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 01 Dec 2022 18:42:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 4: (11 comments) Thanks, looks good. http://gerrit.cloudera.org:8080/#/c/19304/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19304/4//COMMIT_MSG@18 PS4, Line 18: probably inheriting in the opposite : direction would be more logical We could also have an abstract base class and a concrete class for both kinds of return types. http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java: http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@116 PS4, Line 116: // Only primitive objects are supported currently. We could insert a precondition check for that. http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java File fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java: http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java@441 PS4, Line 441: TestHiveUdf(UDFBin.class, createText("1100100"), createBigInt(100)); We only have one Bytes test now, is it intentional? http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java@514 PS4, Line 514: TestGenericUdf(createBoolean(true), Can we also test a function with more than one BINARY arguments? http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java: http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@56 PS4, Line 56: first Now we return the last (see L322). I wonder if concatenation would be possible for BINARY, too? http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@57 PS4, Line 57: * that argument. Not new in this patch, but we could also mention that if any argument is null, the result is also null. http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@114 PS4, Line 114: last Shouldn't this be "first"? http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@123 PS4, Line 123: ObjectInspector Can't we take PrimitiveObjectInspector here? No need for the cast then. http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@389 PS4, Line 389: if (result == null) return null; Before this change we returned a BytesWritable without setting it, not 'null'. Also, we null-check 'result' on L391, too, which is unnecessary. http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@42 PS4, Line 42: * Similarly to TestGenericUdf this class also has copy in the FE. If we wrote here both places where this file is, this line wouldn't have to be different in the the versions (like with TestGenericUdf.java). That's less error prone (though a bit less informative) because a simple copy is enough to sync the files. http://gerrit.cloudera.org:8080/#/c/19304/4/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test File testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test: http://gerrit.cloudera.org:8080/#/c/19304/4/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test@24 PS4, Line 24: #Test GenericUDF functions We could also check some of the multi-argument functions with Java Object return types. -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 4 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 02 Dec 2022 13:45:22 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 5: (7 comments) http://gerrit.cloudera.org:8080/#/c/19304/5/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java File fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java: http://gerrit.cloudera.org:8080/#/c/19304/5/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@32 PS5, Line 32: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/5/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java: http://gerrit.cloudera.org:8080/#/c/19304/5/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@25 PS5, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/5/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@26 PS5, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveWritableObjectInspector; line too long (104 > 90) http://gerrit.cloudera.org:8080/#/c/19304/5/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java File fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/5/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@25 PS5, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/5/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@26 PS5, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory; line too long (95 > 90) http://gerrit.cloudera.org:8080/#/c/19304/5/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/5/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@25 PS5, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/5/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@26 PS5, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory; line too long (95 > 90) -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 5 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 06 Dec 2022 14:27:14 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 6: (7 comments) http://gerrit.cloudera.org:8080/#/c/19304/6/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java File fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java: http://gerrit.cloudera.org:8080/#/c/19304/6/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@32 PS6, Line 32: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/6/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java: http://gerrit.cloudera.org:8080/#/c/19304/6/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@25 PS6, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/6/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@26 PS6, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveWritableObjectInspector; line too long (104 > 90) http://gerrit.cloudera.org:8080/#/c/19304/6/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java File fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/6/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@25 PS6, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/6/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@26 PS6, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory; line too long (95 > 90) http://gerrit.cloudera.org:8080/#/c/19304/6/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/6/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@25 PS6, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/6/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@26 PS6, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory; line too long (95 > 90) -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 6 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 06 Dec 2022 14:30:48 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Hello Daniel Becker, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19304 to look at the new patch set (#5). Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. IMPALA-11549: Support Hive GenericUdfs that return primitive java types Before this patch only the Writable* types were accepted in GenericUdfs as return types, while some GenericUdfs in the wild return primitive java types (e.g. Integer instead of IntWritable). For legacy Hive UDFs these return types were already handled, so the only change needed was to map the ObjectInspector subclasses (e.g. JavaIntObjectInspector) to the correct JavaUdfDataType in Impala. Testing: - Added a subclass for TestGenericUdf (TestGenericUdfWithJavaReturnTypes) that returns primitive java types (probably inheriting in the opposite direction would be more logical, but the diff is smaller this way). - Changed EE tests to also use TestGenericUdfWithJavaReturnTypes. - Changed FE tests (UdfExecutorTest) to check both TestGenericUdfWithJavaReturnTypes and TestGenericUdf. - Also added a test with BINARY type to UdfExecutorTest as this was forgotten during the original BINARY patch. Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 --- M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java A fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java A java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test 10 files changed, 709 insertions(+), 198 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/19304/5 -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 5 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Hello Daniel Becker, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19304 to look at the new patch set (#6). Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. IMPALA-11549: Support Hive GenericUdfs that return primitive java types Before this patch only the Writable* types were accepted in GenericUdfs as return types, while some GenericUdfs in the wild return primitive java types (e.g. Integer instead of IntWritable). For legacy Hive UDFs these return types were already handled, so the only change needed was to map the ObjectInspector subclasses (e.g. JavaIntObjectInspector) to the correct JavaUdfDataType in Impala. Testing: - Added a subclass for TestGenericUdf (TestGenericUdfWithJavaReturnTypes) that returns primitive java types (probably inheriting in the opposite direction would be more logical, but the diff is smaller this way). - Changed EE tests to also use TestGenericUdfWithJavaReturnTypes. - Changed FE tests (UdfExecutorTest) to check both TestGenericUdfWithJavaReturnTypes and TestGenericUdf. - Also added a test with BINARY type to UdfExecutorTest as this was forgotten during the original BINARY patch. Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 --- M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java A fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java A java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test 10 files changed, 711 insertions(+), 200 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/19304/6 -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 6 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 6: (8 comments) http://gerrit.cloudera.org:8080/#/c/19304/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19304/4//COMMIT_MSG@18 PS4, Line 18: probably inheriting in the opposite : direction would be more logical > We could also have an abstract base class and a concrete class for both kin I would prefer to keep is this way in this patch so that the changes in the actual logic behind the udf is more visible. https://gerrit.cloudera.org/#/c/19177/ also modifies these classes, I can do this refactor after merging this and rebasing that patch. http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java: http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@116 PS4, Line 116: > We could insert a precondition check for that. Done http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java File fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java: http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java@441 PS4, Line 441: TestHiveUdf(UDFSign.class, createDouble(1), createDouble(3)); > We only have one Bytes test now, is it intentional? There are not many builtin functions in Hive that use BINARY. Added a tests for UDFBase64 and UDFUnbase64 http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java@514 PS4, Line 514: TestGenericUdf(createFloat(1.1f + 1.2f + 1.3f), > Can we also test a function with more than one BINARY arguments? Done http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java: http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@56 PS4, Line 56: it re > Now we return the last (see L322). Rewrote the function to concatenate the arrays. http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@57 PS4, Line 57: * > Not new in this patch, but we could also mention that if any argument is nu Oops, forgot this, will do in next patch. http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@389 PS4, Line 389: BytesWritable resultBinary = new BytesWritable(); > Before this change we returned a BytesWritable without setting it, not 'nul In the original commit evaluateBinary returned null IMO, see if (input == null) { return null; } The result == null handling was needed for the 0 argument case. http://gerrit.cloudera.org:8080/#/c/19304/4/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test File testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test: http://gerrit.cloudera.org:8080/#/c/19304/4/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test@24 PS4, Line 24: #Test GenericUDF functions > We could also check some of the multi-argument functions with Java Object r I am not sure if this useful in this case, as the change is about the return types and does not change argument type handling. -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 6 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 06 Dec 2022 14:40:05 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/11978/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 5 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 06 Dec 2022 14:45:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Hello Daniel Becker, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19304 to look at the new patch set (#7). Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. IMPALA-11549: Support Hive GenericUdfs that return primitive java types Before this patch only the Writable* types were accepted in GenericUdfs as return types, while some GenericUdfs in the wild return primitive java types (e.g. Integer instead of IntWritable). For legacy Hive UDFs these return types were already handled, so the only change needed was to map the ObjectInspector subclasses (e.g. JavaIntObjectInspector) to the correct JavaUdfDataType in Impala. Testing: - Added a subclass for TestGenericUdf (TestGenericUdfWithJavaReturnTypes) that returns primitive java types (probably inheriting in the opposite direction would be more logical, but the diff is smaller this way). - Changed EE tests to also use TestGenericUdfWithJavaReturnTypes. - Changed FE tests (UdfExecutorTest) to check both TestGenericUdfWithJavaReturnTypes and TestGenericUdf. - Also added a test with BINARY type to UdfExecutorTest as this was forgotten during the original BINARY patch. Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 --- M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java A fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java A java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test 10 files changed, 719 insertions(+), 204 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/19304/7 -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 7 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 7: (7 comments) http://gerrit.cloudera.org:8080/#/c/19304/7/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java File fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java: http://gerrit.cloudera.org:8080/#/c/19304/7/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@32 PS7, Line 32: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/7/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java: http://gerrit.cloudera.org:8080/#/c/19304/7/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@25 PS7, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/7/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@26 PS7, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveWritableObjectInspector; line too long (104 > 90) http://gerrit.cloudera.org:8080/#/c/19304/7/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java File fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/7/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@25 PS7, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/7/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@26 PS7, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory; line too long (95 > 90) http://gerrit.cloudera.org:8080/#/c/19304/7/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/7/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@25 PS7, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/7/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@26 PS7, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory; line too long (95 > 90) -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 7 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 06 Dec 2022 14:48:32 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/11979/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 6 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 06 Dec 2022 14:50:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 7: (4 comments) http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java: http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@57 PS4, Line 57: * > Oops, forgot this, will do in next patch. Done http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@114 PS4, Line 114: ctor > Shouldn't this be "first"? Done http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@123 PS4, Line 123: > Can't we take PrimitiveObjectInspector here? No need for the cast then. Done, I think I just wanted to avoid breaking the line by using the shorter name :) http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@42 PS4, Line 42: * Similarly to TestGenericUdf this class also has copy in the FE. > If we wrote here both places where this file is, this line wouldn't have to The file cannot be simply copied because of the different package, see line 18 -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 7 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 06 Dec 2022 14:51:46 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 7: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/11980/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 7 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 06 Dec 2022 15:05:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 7: (6 comments) http://gerrit.cloudera.org:8080/#/c/19304/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19304/4//COMMIT_MSG@18 PS4, Line 18: probably inheriting in the opposite : direction would be more logical > I would prefer to keep is this way in this patch so that the changes in the Sounds good. http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java: http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@389 PS4, Line 389: if (result == null) return null; > In the original commit evaluateBinary returned null IMO, see Right, thanks. http://gerrit.cloudera.org:8080/#/c/19304/7/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java: http://gerrit.cloudera.org:8080/#/c/19304/7/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@54 PS7, Line 54: string Now it is string and binary types, right? http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@42 PS4, Line 42: * Similarly to TestGenericUdf this class also has copy in the FE. > The file cannot be simply copied because of the different package, see line Ok, thanks. http://gerrit.cloudera.org:8080/#/c/19304/4/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test File testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test: http://gerrit.cloudera.org:8080/#/c/19304/4/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test@24 PS4, Line 24: #Test GenericUDF functions > I am not sure if this useful in this case, as the change is about the retur We could still add one or two cases so that we can see that the Java Object return type also works with multi-arg functions. http://gerrit.cloudera.org:8080/#/c/19304/7/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test File testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test: http://gerrit.cloudera.org:8080/#/c/19304/7/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test@107 PS7, Line 107: strings binaries -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 7 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 06 Dec 2022 15:13:57 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Hello Daniel Becker, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19304 to look at the new patch set (#8). Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. IMPALA-11549: Support Hive GenericUdfs that return primitive java types Before this patch only the Writable* types were accepted in GenericUdfs as return types, while some GenericUdfs in the wild return primitive java types (e.g. Integer instead of IntWritable). For legacy Hive UDFs these return types were already handled, so the only change needed was to map the ObjectInspector subclasses (e.g. JavaIntObjectInspector) to the correct JavaUdfDataType in Impala. Testing: - Added a subclass for TestGenericUdf (TestGenericUdfWithJavaReturnTypes) that returns primitive java types (probably inheriting in the opposite direction would be more logical, but the diff is smaller this way). - Changed EE tests to also use TestGenericUdfWithJavaReturnTypes. - Changed FE tests (UdfExecutorTest) to check both TestGenericUdfWithJavaReturnTypes and TestGenericUdf. - Also added a test with BINARY type to UdfExecutorTest as this was forgotten during the original BINARY patch. Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 --- M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java A fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java A java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test 10 files changed, 736 insertions(+), 204 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/19304/8 -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 8 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 8: (7 comments) http://gerrit.cloudera.org:8080/#/c/19304/8/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java File fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java: http://gerrit.cloudera.org:8080/#/c/19304/8/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@32 PS8, Line 32: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/8/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java: http://gerrit.cloudera.org:8080/#/c/19304/8/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@25 PS8, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/8/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@26 PS8, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveWritableObjectInspector; line too long (104 > 90) http://gerrit.cloudera.org:8080/#/c/19304/8/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java File fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/8/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@25 PS8, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/8/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@26 PS8, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory; line too long (95 > 90) http://gerrit.cloudera.org:8080/#/c/19304/8/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/8/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@25 PS8, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/8/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@26 PS8, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory; line too long (95 > 90) -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 8 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 07 Dec 2022 12:32:33 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Hello Daniel Becker, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19304 to look at the new patch set (#9). Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. IMPALA-11549: Support Hive GenericUdfs that return primitive java types Before this patch only the Writable* types were accepted in GenericUdfs as return types, while some GenericUdfs in the wild return primitive java types (e.g. Integer instead of IntWritable). For legacy Hive UDFs these return types were already handled, so the only change needed was to map the ObjectInspector subclasses (e.g. JavaIntObjectInspector) to the correct JavaUdfDataType in Impala. Testing: - Added a subclass for TestGenericUdf (TestGenericUdfWithJavaReturnTypes) that returns primitive java types (probably inheriting in the opposite direction would be more logical, but the diff is smaller this way). - Changed EE tests to also use TestGenericUdfWithJavaReturnTypes. - Changed FE tests (UdfExecutorTest) to check both TestGenericUdfWithJavaReturnTypes and TestGenericUdf. - Also added a test with BINARY type to UdfExecutorTest as this was forgotten during the original BINARY patch. Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 --- M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java A fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java A java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test 10 files changed, 740 insertions(+), 208 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/19304/9 -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 9 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 9: (7 comments) http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java File fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java: http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@32 PS9, Line 32: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java: http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@25 PS9, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@26 PS9, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveWritableObjectInspector; line too long (104 > 90) http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java File fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@25 PS9, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@26 PS9, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory; line too long (95 > 90) http://gerrit.cloudera.org:8080/#/c/19304/9/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java: http://gerrit.cloudera.org:8080/#/c/19304/9/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@25 PS9, Line 25: import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/19304/9/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@26 PS9, Line 26: import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory; line too long (95 > 90) -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 9 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 07 Dec 2022 12:34:55 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 9: (3 comments) http://gerrit.cloudera.org:8080/#/c/19304/7/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java: http://gerrit.cloudera.org:8080/#/c/19304/7/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@54 PS7, Line 54: string > Now it is string and binary types, right? Done http://gerrit.cloudera.org:8080/#/c/19304/4/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test File testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test: http://gerrit.cloudera.org:8080/#/c/19304/4/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test@24 PS4, Line 24: #Test GenericUDF functions > We could still add one or two cases so that we can see that the Java Object Done. It would be good to have some generic functions where note all argument types are the same, but I would do this in a different patch (probably https://gerrit.cloudera.org/#/c/19177/ ) http://gerrit.cloudera.org:8080/#/c/19304/7/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test File testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test: http://gerrit.cloudera.org:8080/#/c/19304/7/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test@107 PS7, Line 107: binarie > binaries Done -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 9 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 07 Dec 2022 12:36:05 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 8: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/11987/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 8 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 07 Dec 2022 12:51:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 9: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/11988/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 9 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 07 Dec 2022 12:53:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 9: Code-Review+1 (1 comment) LGTM http://gerrit.cloudera.org:8080/#/c/19304/4/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test File testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test: http://gerrit.cloudera.org:8080/#/c/19304/4/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test@24 PS4, Line 24: #Test GenericUDF functions > Done. It would be good to have some generic functions where note all argume Ok, that's fine. Thanks. -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 9 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 07 Dec 2022 12:53:50 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Steve Carlin has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 9: Code-Review+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java File fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java: http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@152 PS9, Line 152: if (returnOi_ != getInspector(retType_, true) optional nit: Maybe it makes more sense to check if the returnOi_ is valid within initializeWrapper before setting the variable? It's nice to know that the variable is already valid by the time it is set. -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 9 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Steve Carlin Gerrit-Comment-Date: Wed, 07 Dec 2022 17:17:55 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 10: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 10 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Steve Carlin Gerrit-Comment-Date: Thu, 08 Dec 2022 12:41:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 10: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8885/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 10 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Steve Carlin Gerrit-Comment-Date: Thu, 08 Dec 2022 12:41:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 9: (1 comment) http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java File fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java: http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@152 PS9, Line 152: if (returnOi_ != getInspector(retType_, true) > optional nit: Maybe it makes more sense to check if the returnOi_ is valid In the state it would be more logical to check it here, but t my other GenericUdf related review changes this are and uses initializeWrapper in different ways: https://gerrit.cloudera.org/#/c/19177/6/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java When using GenericUdf in "generic" way the return type is unknown beforehand and we know it only after initialize(). I would prefer to keep it this way in this patch possibly beatifying it after this patch is merged and the other is rebased. -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 9 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Steve Carlin Gerrit-Comment-Date: Thu, 08 Dec 2022 12:38:36 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 9: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 9 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Steve Carlin Gerrit-Comment-Date: Thu, 08 Dec 2022 12:40:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. Patch Set 10: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 10 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Steve Carlin Gerrit-Comment-Date: Thu, 08 Dec 2022 17:50:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19304 ) Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive java types .. IMPALA-11549: Support Hive GenericUdfs that return primitive java types Before this patch only the Writable* types were accepted in GenericUdfs as return types, while some GenericUdfs in the wild return primitive java types (e.g. Integer instead of IntWritable). For legacy Hive UDFs these return types were already handled, so the only change needed was to map the ObjectInspector subclasses (e.g. JavaIntObjectInspector) to the correct JavaUdfDataType in Impala. Testing: - Added a subclass for TestGenericUdf (TestGenericUdfWithJavaReturnTypes) that returns primitive java types (probably inheriting in the opposite direction would be more logical, but the diff is smaller this way). - Changed EE tests to also use TestGenericUdfWithJavaReturnTypes. - Changed FE tests (UdfExecutorTest) to check both TestGenericUdfWithJavaReturnTypes and TestGenericUdf. - Also added a test with BINARY type to UdfExecutorTest as this was forgotten during the original BINARY patch. Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Reviewed-on: http://gerrit.cloudera.org:8080/19304 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java A fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java A java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test 10 files changed, 740 insertions(+), 208 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/19304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63 Gerrit-Change-Number: 19304 Gerrit-PatchSet: 11 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Steve Carlin