[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-01 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 1:

(7 comments)

http://gerrit.cloudera.org:8080/#/c/19304/1/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
File 
fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java:

http://gerrit.cloudera.org:8080/#/c/19304/1/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@32
PS1, Line 32: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/1/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java:

http://gerrit.cloudera.org:8080/#/c/19304/1/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@25
PS1, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/1/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@26
PS1, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveWritableObjectInspector;
line too long (104 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/1/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java
File 
fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/1/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@25
PS1, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/1/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@26
PS1, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
line too long (95 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/1/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
File 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/1/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@25
PS1, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/1/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@26
PS1, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
line too long (95 > 90)



--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 01 Dec 2022 18:06:21 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-01 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 2:

(7 comments)

http://gerrit.cloudera.org:8080/#/c/19304/2/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
File 
fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java:

http://gerrit.cloudera.org:8080/#/c/19304/2/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@32
PS2, Line 32: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/2/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java:

http://gerrit.cloudera.org:8080/#/c/19304/2/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@25
PS2, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/2/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@26
PS2, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveWritableObjectInspector;
line too long (104 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/2/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java
File 
fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/2/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@25
PS2, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/2/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@26
PS2, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
line too long (95 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/2/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
File 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/2/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@25
PS2, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/2/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@26
PS2, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
line too long (95 > 90)



--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 2
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 01 Dec 2022 18:08:05 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-01 Thread Csaba Ringhofer (Code Review)
Csaba Ringhofer has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/19304


Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..

IMPALA-11549: Support Hive GenericUdfs that return primitive java types

Before this patch only the Writable* types were accepted in GenericUdfs
as return types, while some GenericUdfs in the wild return primitive java
types (e.g. Integer instead of IntWritable). For legacy Hive UDFs these
return types were already handled, so the only change needed was to
map the ObjectInspector subclasses (e.g. JavaIntObjectInspector) to the
correct JavaUdfDataType in Impala.

Testing:
- Added a subclass for TestGenericUdf (TestGenericUdfWithJavaReturnTypes)
  that returns primitive java types (probably inheriting in the opposite
  direction would be more logical, but the diff is smaller this way).
- Changed EE tests to also use TestGenericUdfWithJavaReturnTypes.
- Changed FE tests (UdfExecutorTest) to check both
  TestGenericUdfWithJavaReturnTypes and TestGenericUdf.
- Also added a test with BINARY type to UdfExecutorTest as this was
  forgotten during the original BINARY patch.

Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
---
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java
A 
fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
A 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M 
testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
10 files changed, 670 insertions(+), 183 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/19304/1
--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer 


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-01 Thread Csaba Ringhofer (Code Review)
Hello Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19304

to look at the new patch set (#2).

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..

IMPALA-11549: Support Hive GenericUdfs that return primitive java types

Before this patch only the Writable* types were accepted in GenericUdfs
as return types, while some GenericUdfs in the wild return primitive java
types (e.g. Integer instead of IntWritable). For legacy Hive UDFs these
return types were already handled, so the only change needed was to
map the ObjectInspector subclasses (e.g. JavaIntObjectInspector) to the
correct JavaUdfDataType in Impala.

Testing:
- Added a subclass for TestGenericUdf (TestGenericUdfWithJavaReturnTypes)
  that returns primitive java types (probably inheriting in the opposite
  direction would be more logical, but the diff is smaller this way).
- Changed EE tests to also use TestGenericUdfWithJavaReturnTypes.
- Changed FE tests (UdfExecutorTest) to check both
  TestGenericUdfWithJavaReturnTypes and TestGenericUdf.
- Also added a test with BINARY type to UdfExecutorTest as this was
  forgotten during the original BINARY patch.

Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
---
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java
A 
fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
A 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M 
testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
10 files changed, 670 insertions(+), 184 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/19304/2
--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 2
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-01 Thread Csaba Ringhofer (Code Review)
Hello Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19304

to look at the new patch set (#3).

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..

IMPALA-11549: Support Hive GenericUdfs that return primitive java types

Before this patch only the Writable* types were accepted in GenericUdfs
as return types, while some GenericUdfs in the wild return primitive java
types (e.g. Integer instead of IntWritable). For legacy Hive UDFs these
return types were already handled, so the only change needed was to
map the ObjectInspector subclasses (e.g. JavaIntObjectInspector) to the
correct JavaUdfDataType in Impala.

Testing:
- Added a subclass for TestGenericUdf (TestGenericUdfWithJavaReturnTypes)
  that returns primitive java types (probably inheriting in the opposite
  direction would be more logical, but the diff is smaller this way).
- Changed EE tests to also use TestGenericUdfWithJavaReturnTypes.
- Changed FE tests (UdfExecutorTest) to check both
  TestGenericUdfWithJavaReturnTypes and TestGenericUdf.
- Also added a test with BINARY type to UdfExecutorTest as this was
  forgotten during the original BINARY patch.

Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
---
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java
A 
fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
A 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M 
testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
10 files changed, 670 insertions(+), 184 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/19304/3
--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-01 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 3:

(7 comments)

http://gerrit.cloudera.org:8080/#/c/19304/3/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
File 
fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java:

http://gerrit.cloudera.org:8080/#/c/19304/3/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@32
PS3, Line 32: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/3/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java:

http://gerrit.cloudera.org:8080/#/c/19304/3/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@25
PS3, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/3/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@26
PS3, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveWritableObjectInspector;
line too long (104 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/3/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java
File 
fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/3/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@25
PS3, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/3/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@26
PS3, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
line too long (95 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/3/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
File 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/3/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@25
PS3, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/3/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@26
PS3, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
line too long (95 > 90)



--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 01 Dec 2022 18:21:40 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-01 Thread Csaba Ringhofer (Code Review)
Hello Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19304

to look at the new patch set (#4).

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..

IMPALA-11549: Support Hive GenericUdfs that return primitive java types

Before this patch only the Writable* types were accepted in GenericUdfs
as return types, while some GenericUdfs in the wild return primitive java
types (e.g. Integer instead of IntWritable). For legacy Hive UDFs these
return types were already handled, so the only change needed was to
map the ObjectInspector subclasses (e.g. JavaIntObjectInspector) to the
correct JavaUdfDataType in Impala.

Testing:
- Added a subclass for TestGenericUdf (TestGenericUdfWithJavaReturnTypes)
  that returns primitive java types (probably inheriting in the opposite
  direction would be more logical, but the diff is smaller this way).
- Changed EE tests to also use TestGenericUdfWithJavaReturnTypes.
- Changed FE tests (UdfExecutorTest) to check both
  TestGenericUdfWithJavaReturnTypes and TestGenericUdf.
- Also added a test with BINARY type to UdfExecutorTest as this was
  forgotten during the original BINARY patch.

Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
---
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java
A 
fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
A 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M 
testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
10 files changed, 670 insertions(+), 184 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/19304/4
--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 4
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-01 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 4:

(7 comments)

http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
File 
fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java:

http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@32
PS4, Line 32: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java:

http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@25
PS4, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@26
PS4, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveWritableObjectInspector;
line too long (104 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java
File 
fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@25
PS4, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@26
PS4, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
line too long (95 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
File 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@25
PS4, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@26
PS4, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
line too long (95 > 90)



--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 4
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 01 Dec 2022 18:22:52 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-01 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 1:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11935/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 01 Dec 2022 18:25:09 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-01 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 2:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11936/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 2
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 01 Dec 2022 18:28:13 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-01 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 3:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/11937/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 01 Dec 2022 18:34:51 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-01 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 4:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11938/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 4
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 01 Dec 2022 18:42:54 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-02 Thread Daniel Becker (Code Review)
Daniel Becker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 4:

(11 comments)

Thanks, looks good.

http://gerrit.cloudera.org:8080/#/c/19304/4//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19304/4//COMMIT_MSG@18
PS4, Line 18: probably inheriting in the opposite
:   direction would be more logical
We could also have an abstract base class and a concrete class for both kinds 
of return types.


http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java:

http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@116
PS4, Line 116:   // Only primitive objects are supported currently.
We could insert a precondition check for that.


http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
File fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java:

http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java@441
PS4, Line 441: TestHiveUdf(UDFBin.class, createText("1100100"), 
createBigInt(100));
We only have one Bytes test now, is it intentional?


http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java@514
PS4, Line 514: TestGenericUdf(createBoolean(true),
Can we also test a function with more than one BINARY arguments?


http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java:

http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@56
PS4, Line 56: first
Now we return the last (see L322).
I wonder if concatenation would be possible for BINARY, too?


http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@57
PS4, Line 57:  * that argument.
Not new in this patch, but we could also mention that if any argument is null, 
the result is also null.


http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@114
PS4, Line 114: last
Shouldn't this be "first"?


http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@123
PS4, Line 123: ObjectInspector
Can't we take PrimitiveObjectInspector here? No need for the cast then.


http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@389
PS4, Line 389: if (result == null) return null;
Before this change we returned a BytesWritable without setting it, not 'null'.
Also, we null-check 'result' on L391, too, which is unnecessary.


http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
File 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@42
PS4, Line 42:  * Similarly to TestGenericUdf this class also has copy in the FE.
If we wrote here both places where this file is, this line wouldn't have to be 
different in the the versions (like with TestGenericUdf.java). That's less 
error prone (though a bit less informative) because a simple copy is enough to 
sync the files.


http://gerrit.cloudera.org:8080/#/c/19304/4/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
File 
testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test:

http://gerrit.cloudera.org:8080/#/c/19304/4/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test@24
PS4, Line 24: #Test GenericUDF functions
We could also check some of the multi-argument functions with Java Object 
return types.



--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 4
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 02 Dec 2022 13:45:22 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-06 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 5:

(7 comments)

http://gerrit.cloudera.org:8080/#/c/19304/5/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
File 
fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java:

http://gerrit.cloudera.org:8080/#/c/19304/5/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@32
PS5, Line 32: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/5/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java:

http://gerrit.cloudera.org:8080/#/c/19304/5/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@25
PS5, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/5/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@26
PS5, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveWritableObjectInspector;
line too long (104 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/5/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java
File 
fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/5/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@25
PS5, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/5/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@26
PS5, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
line too long (95 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/5/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
File 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/5/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@25
PS5, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/5/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@26
PS5, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
line too long (95 > 90)



--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 5
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Tue, 06 Dec 2022 14:27:14 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-06 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 6:

(7 comments)

http://gerrit.cloudera.org:8080/#/c/19304/6/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
File 
fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java:

http://gerrit.cloudera.org:8080/#/c/19304/6/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@32
PS6, Line 32: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/6/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java:

http://gerrit.cloudera.org:8080/#/c/19304/6/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@25
PS6, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/6/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@26
PS6, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveWritableObjectInspector;
line too long (104 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/6/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java
File 
fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/6/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@25
PS6, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/6/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@26
PS6, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
line too long (95 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/6/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
File 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/6/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@25
PS6, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/6/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@26
PS6, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
line too long (95 > 90)



--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 6
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Tue, 06 Dec 2022 14:30:48 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-06 Thread Csaba Ringhofer (Code Review)
Hello Daniel Becker, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19304

to look at the new patch set (#5).

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..

IMPALA-11549: Support Hive GenericUdfs that return primitive java types

Before this patch only the Writable* types were accepted in GenericUdfs
as return types, while some GenericUdfs in the wild return primitive java
types (e.g. Integer instead of IntWritable). For legacy Hive UDFs these
return types were already handled, so the only change needed was to
map the ObjectInspector subclasses (e.g. JavaIntObjectInspector) to the
correct JavaUdfDataType in Impala.

Testing:
- Added a subclass for TestGenericUdf (TestGenericUdfWithJavaReturnTypes)
  that returns primitive java types (probably inheriting in the opposite
  direction would be more logical, but the diff is smaller this way).
- Changed EE tests to also use TestGenericUdfWithJavaReturnTypes.
- Changed FE tests (UdfExecutorTest) to check both
  TestGenericUdfWithJavaReturnTypes and TestGenericUdf.
- Also added a test with BINARY type to UdfExecutorTest as this was
  forgotten during the original BINARY patch.

Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
---
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java
A 
fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
A 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M 
testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
10 files changed, 709 insertions(+), 198 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/19304/5
--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 5
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-06 Thread Csaba Ringhofer (Code Review)
Hello Daniel Becker, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19304

to look at the new patch set (#6).

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..

IMPALA-11549: Support Hive GenericUdfs that return primitive java types

Before this patch only the Writable* types were accepted in GenericUdfs
as return types, while some GenericUdfs in the wild return primitive java
types (e.g. Integer instead of IntWritable). For legacy Hive UDFs these
return types were already handled, so the only change needed was to
map the ObjectInspector subclasses (e.g. JavaIntObjectInspector) to the
correct JavaUdfDataType in Impala.

Testing:
- Added a subclass for TestGenericUdf (TestGenericUdfWithJavaReturnTypes)
  that returns primitive java types (probably inheriting in the opposite
  direction would be more logical, but the diff is smaller this way).
- Changed EE tests to also use TestGenericUdfWithJavaReturnTypes.
- Changed FE tests (UdfExecutorTest) to check both
  TestGenericUdfWithJavaReturnTypes and TestGenericUdf.
- Also added a test with BINARY type to UdfExecutorTest as this was
  forgotten during the original BINARY patch.

Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
---
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java
A 
fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
A 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M 
testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
10 files changed, 711 insertions(+), 200 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/19304/6
--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 6
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-06 Thread Csaba Ringhofer (Code Review)
Csaba Ringhofer has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 6:

(8 comments)

http://gerrit.cloudera.org:8080/#/c/19304/4//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19304/4//COMMIT_MSG@18
PS4, Line 18: probably inheriting in the opposite
:   direction would be more logical
> We could also have an abstract base class and a concrete class for both kin
I would prefer to keep is this way in this patch so that the changes in the 
actual logic behind the udf is more visible.

https://gerrit.cloudera.org/#/c/19177/ also modifies these classes, I can do 
this refactor after merging this and rebasing that patch.


http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java:

http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@116
PS4, Line 116:
> We could insert a precondition check for that.
Done


http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
File fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java:

http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java@441
PS4, Line 441: TestHiveUdf(UDFSign.class, createDouble(1), createDouble(3));
> We only have one Bytes test now, is it intentional?
There are not many builtin functions in Hive that use BINARY. Added a tests for 
UDFBase64 and UDFUnbase64


http://gerrit.cloudera.org:8080/#/c/19304/4/fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java@514
PS4, Line 514: TestGenericUdf(createFloat(1.1f + 1.2f + 1.3f),
> Can we also test a function with more than one BINARY arguments?
Done


http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java:

http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@56
PS4, Line 56: it re
> Now we return the last (see L322).
Rewrote the function to concatenate the arrays.


http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@57
PS4, Line 57:  *
> Not new in this patch, but we could also mention that if any argument is nu
Oops, forgot this, will do in next patch.


http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@389
PS4, Line 389: BytesWritable resultBinary = new BytesWritable();
> Before this change we returned a BytesWritable without setting it, not 'nul
In the original commit evaluateBinary returned null IMO, see
  if (input == null) {
return null;
  }
The result == null handling was needed for the 0 argument case.


http://gerrit.cloudera.org:8080/#/c/19304/4/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
File 
testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test:

http://gerrit.cloudera.org:8080/#/c/19304/4/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test@24
PS4, Line 24: #Test GenericUDF functions
> We could also check some of the multi-argument functions with Java Object r
I am not sure if this useful in this case, as the change is about the return 
types and does not change argument type handling.



--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 6
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Tue, 06 Dec 2022 14:40:05 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-06 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 5:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11978/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 5
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Tue, 06 Dec 2022 14:45:46 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-06 Thread Csaba Ringhofer (Code Review)
Hello Daniel Becker, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19304

to look at the new patch set (#7).

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..

IMPALA-11549: Support Hive GenericUdfs that return primitive java types

Before this patch only the Writable* types were accepted in GenericUdfs
as return types, while some GenericUdfs in the wild return primitive java
types (e.g. Integer instead of IntWritable). For legacy Hive UDFs these
return types were already handled, so the only change needed was to
map the ObjectInspector subclasses (e.g. JavaIntObjectInspector) to the
correct JavaUdfDataType in Impala.

Testing:
- Added a subclass for TestGenericUdf (TestGenericUdfWithJavaReturnTypes)
  that returns primitive java types (probably inheriting in the opposite
  direction would be more logical, but the diff is smaller this way).
- Changed EE tests to also use TestGenericUdfWithJavaReturnTypes.
- Changed FE tests (UdfExecutorTest) to check both
  TestGenericUdfWithJavaReturnTypes and TestGenericUdf.
- Also added a test with BINARY type to UdfExecutorTest as this was
  forgotten during the original BINARY patch.

Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
---
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java
A 
fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
A 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M 
testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
10 files changed, 719 insertions(+), 204 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/19304/7
--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 7
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-06 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 7:

(7 comments)

http://gerrit.cloudera.org:8080/#/c/19304/7/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
File 
fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java:

http://gerrit.cloudera.org:8080/#/c/19304/7/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@32
PS7, Line 32: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/7/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java:

http://gerrit.cloudera.org:8080/#/c/19304/7/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@25
PS7, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/7/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@26
PS7, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveWritableObjectInspector;
line too long (104 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/7/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java
File 
fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/7/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@25
PS7, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/7/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@26
PS7, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
line too long (95 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/7/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
File 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/7/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@25
PS7, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/7/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@26
PS7, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
line too long (95 > 90)



--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 7
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Tue, 06 Dec 2022 14:48:32 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-06 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 6:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11979/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 6
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Tue, 06 Dec 2022 14:50:00 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-06 Thread Csaba Ringhofer (Code Review)
Csaba Ringhofer has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 7:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java:

http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@57
PS4, Line 57:  *
> Oops, forgot this, will do in next patch.
Done


http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@114
PS4, Line 114: ctor
> Shouldn't this be "first"?
Done


http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@123
PS4, Line 123:
> Can't we take PrimitiveObjectInspector here? No need for the cast then.
Done, I think I just wanted to avoid breaking the line by using the shorter 
name :)


http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
File 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@42
PS4, Line 42:  * Similarly to TestGenericUdf this class also has copy in the FE.
> If we wrote here both places where this file is, this line wouldn't have to
The file cannot be simply copied because of the different package, see line 18



--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 7
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Tue, 06 Dec 2022 14:51:46 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-06 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 7:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11980/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 7
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Tue, 06 Dec 2022 15:05:06 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-06 Thread Daniel Becker (Code Review)
Daniel Becker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 7:

(6 comments)

http://gerrit.cloudera.org:8080/#/c/19304/4//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19304/4//COMMIT_MSG@18
PS4, Line 18: probably inheriting in the opposite
:   direction would be more logical
> I would prefer to keep is this way in this patch so that the changes in the
Sounds good.


http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java:

http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@389
PS4, Line 389: if (result == null) return null;
> In the original commit evaluateBinary returned null IMO, see
Right, thanks.


http://gerrit.cloudera.org:8080/#/c/19304/7/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java:

http://gerrit.cloudera.org:8080/#/c/19304/7/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@54
PS7, Line 54: string
Now it is string and binary types, right?


http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
File 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/4/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@42
PS4, Line 42:  * Similarly to TestGenericUdf this class also has copy in the FE.
> The file cannot be simply copied because of the different package, see line
Ok, thanks.


http://gerrit.cloudera.org:8080/#/c/19304/4/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
File 
testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test:

http://gerrit.cloudera.org:8080/#/c/19304/4/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test@24
PS4, Line 24: #Test GenericUDF functions
> I am not sure if this useful in this case, as the change is about the retur
We could still add one or two cases so that we can see that the Java Object 
return type also works with multi-arg functions.


http://gerrit.cloudera.org:8080/#/c/19304/7/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
File 
testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test:

http://gerrit.cloudera.org:8080/#/c/19304/7/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test@107
PS7, Line 107: strings
binaries



--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 7
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Tue, 06 Dec 2022 15:13:57 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-07 Thread Csaba Ringhofer (Code Review)
Hello Daniel Becker, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19304

to look at the new patch set (#8).

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..

IMPALA-11549: Support Hive GenericUdfs that return primitive java types

Before this patch only the Writable* types were accepted in GenericUdfs
as return types, while some GenericUdfs in the wild return primitive java
types (e.g. Integer instead of IntWritable). For legacy Hive UDFs these
return types were already handled, so the only change needed was to
map the ObjectInspector subclasses (e.g. JavaIntObjectInspector) to the
correct JavaUdfDataType in Impala.

Testing:
- Added a subclass for TestGenericUdf (TestGenericUdfWithJavaReturnTypes)
  that returns primitive java types (probably inheriting in the opposite
  direction would be more logical, but the diff is smaller this way).
- Changed EE tests to also use TestGenericUdfWithJavaReturnTypes.
- Changed FE tests (UdfExecutorTest) to check both
  TestGenericUdfWithJavaReturnTypes and TestGenericUdf.
- Also added a test with BINARY type to UdfExecutorTest as this was
  forgotten during the original BINARY patch.

Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
---
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java
A 
fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
A 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M 
testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
10 files changed, 736 insertions(+), 204 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/19304/8
--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 8
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-07 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 8:

(7 comments)

http://gerrit.cloudera.org:8080/#/c/19304/8/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
File 
fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java:

http://gerrit.cloudera.org:8080/#/c/19304/8/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@32
PS8, Line 32: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/8/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java:

http://gerrit.cloudera.org:8080/#/c/19304/8/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@25
PS8, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/8/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@26
PS8, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveWritableObjectInspector;
line too long (104 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/8/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java
File 
fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/8/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@25
PS8, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/8/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@26
PS8, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
line too long (95 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/8/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
File 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/8/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@25
PS8, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/8/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@26
PS8, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
line too long (95 > 90)



--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 8
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 07 Dec 2022 12:32:33 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-07 Thread Csaba Ringhofer (Code Review)
Hello Daniel Becker, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19304

to look at the new patch set (#9).

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..

IMPALA-11549: Support Hive GenericUdfs that return primitive java types

Before this patch only the Writable* types were accepted in GenericUdfs
as return types, while some GenericUdfs in the wild return primitive java
types (e.g. Integer instead of IntWritable). For legacy Hive UDFs these
return types were already handled, so the only change needed was to
map the ObjectInspector subclasses (e.g. JavaIntObjectInspector) to the
correct JavaUdfDataType in Impala.

Testing:
- Added a subclass for TestGenericUdf (TestGenericUdfWithJavaReturnTypes)
  that returns primitive java types (probably inheriting in the opposite
  direction would be more logical, but the diff is smaller this way).
- Changed EE tests to also use TestGenericUdfWithJavaReturnTypes.
- Changed FE tests (UdfExecutorTest) to check both
  TestGenericUdfWithJavaReturnTypes and TestGenericUdf.
- Also added a test with BINARY type to UdfExecutorTest as this was
  forgotten during the original BINARY patch.

Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
---
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java
A 
fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
A 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M 
testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
10 files changed, 740 insertions(+), 208 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/19304/9
--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 9
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-07 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 9:

(7 comments)

http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
File 
fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java:

http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@32
PS9, Line 32: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
File fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java:

http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@25
PS9, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java@26
PS9, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveWritableObjectInspector;
line too long (104 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java
File 
fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@25
PS9, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java@26
PS9, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
line too long (95 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/9/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
File 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java:

http://gerrit.cloudera.org:8080/#/c/19304/9/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@25
PS9, Line 25: import 
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;
line too long (96 > 90)


http://gerrit.cloudera.org:8080/#/c/19304/9/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java@26
PS9, Line 26: import 
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
line too long (95 > 90)



--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 9
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 07 Dec 2022 12:34:55 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-07 Thread Csaba Ringhofer (Code Review)
Csaba Ringhofer has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 9:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/19304/7/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
File java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java:

http://gerrit.cloudera.org:8080/#/c/19304/7/java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java@54
PS7, Line 54: string
> Now it is string and binary types, right?
Done


http://gerrit.cloudera.org:8080/#/c/19304/4/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
File 
testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test:

http://gerrit.cloudera.org:8080/#/c/19304/4/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test@24
PS4, Line 24: #Test GenericUDF functions
> We could still add one or two cases so that we can see that the Java Object
Done. It would be good to have some generic functions where note all argument 
types are the same, but I would do this in a different patch (probably 
https://gerrit.cloudera.org/#/c/19177/ )


http://gerrit.cloudera.org:8080/#/c/19304/7/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
File 
testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test:

http://gerrit.cloudera.org:8080/#/c/19304/7/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test@107
PS7, Line 107: binarie
> binaries
Done



--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 9
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 07 Dec 2022 12:36:05 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-07 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 8:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11987/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 8
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 07 Dec 2022 12:51:26 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-07 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 9:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11988/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 9
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 07 Dec 2022 12:53:56 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-07 Thread Daniel Becker (Code Review)
Daniel Becker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 9: Code-Review+1

(1 comment)

LGTM

http://gerrit.cloudera.org:8080/#/c/19304/4/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
File 
testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test:

http://gerrit.cloudera.org:8080/#/c/19304/4/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test@24
PS4, Line 24: #Test GenericUDF functions
> Done. It would be good to have some generic functions where note all argume
Ok, that's fine. Thanks.



--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 9
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 07 Dec 2022 12:53:50 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-07 Thread Steve Carlin (Code Review)
Steve Carlin has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 9: Code-Review+1

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
File 
fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java:

http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@152
PS9, Line 152: if (returnOi_ != getInspector(retType_, true)
optional nit: Maybe it makes more sense to check if the returnOi_ is valid 
within initializeWrapper before setting the variable?

It's nice to know that the variable is already valid by the time it is set.



--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 9
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Steve Carlin 
Gerrit-Comment-Date: Wed, 07 Dec 2022 17:17:55 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-08 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 10: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 10
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Steve Carlin 
Gerrit-Comment-Date: Thu, 08 Dec 2022 12:41:11 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-08 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 10:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8885/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 10
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Steve Carlin 
Gerrit-Comment-Date: Thu, 08 Dec 2022 12:41:12 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-08 Thread Csaba Ringhofer (Code Review)
Csaba Ringhofer has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 9:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
File 
fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java:

http://gerrit.cloudera.org:8080/#/c/19304/9/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@152
PS9, Line 152: if (returnOi_ != getInspector(retType_, true)
> optional nit: Maybe it makes more sense to check if the returnOi_ is valid
In the state it would be more logical to check it here, but t my other 
GenericUdf related review changes this are and uses initializeWrapper in 
different ways:
https://gerrit.cloudera.org/#/c/19177/6/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java

When using GenericUdf in "generic" way the return type is unknown beforehand 
and we know it only after initialize(). I would prefer to keep it this way in 
this patch possibly beatifying it after this patch is merged and the other is 
rebased.



--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 9
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Steve Carlin 
Gerrit-Comment-Date: Thu, 08 Dec 2022 12:38:36 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-08 Thread Daniel Becker (Code Review)
Daniel Becker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 9: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 9
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Steve Carlin 
Gerrit-Comment-Date: Thu, 08 Dec 2022 12:40:37 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-08 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..


Patch Set 10: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 10
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Steve Carlin 
Gerrit-Comment-Date: Thu, 08 Dec 2022 17:50:58 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11549: Support Hive GenericUdfs that return primitive java types

2022-12-08 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/19304 )

Change subject: IMPALA-11549: Support Hive GenericUdfs that return primitive 
java types
..

IMPALA-11549: Support Hive GenericUdfs that return primitive java types

Before this patch only the Writable* types were accepted in GenericUdfs
as return types, while some GenericUdfs in the wild return primitive java
types (e.g. Integer instead of IntWritable). For legacy Hive UDFs these
return types were already handled, so the only change needed was to
map the ObjectInspector subclasses (e.g. JavaIntObjectInspector) to the
correct JavaUdfDataType in Impala.

Testing:
- Added a subclass for TestGenericUdf (TestGenericUdfWithJavaReturnTypes)
  that returns primitive java types (probably inheriting in the opposite
  direction would be more logical, but the diff is smaller this way).
- Changed EE tests to also use TestGenericUdfWithJavaReturnTypes.
- Changed FE tests (UdfExecutorTest) to check both
  TestGenericUdfWithJavaReturnTypes and TestGenericUdf.
- Also added a test with BINARY type to UdfExecutorTest as this was
  forgotten during the original BINARY patch.

Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Reviewed-on: http://gerrit.cloudera.org:8080/19304
Reviewed-by: Impala Public Jenkins 
Tested-by: Impala Public Jenkins 
---
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java
A 
fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
A 
java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M 
testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
10 files changed, 740 insertions(+), 208 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

--
To view, visit http://gerrit.cloudera.org:8080/19304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I30679045d6693ebd35718b6f1a22aaa4963c1e63
Gerrit-Change-Number: 19304
Gerrit-PatchSet: 11
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Steve Carlin