chenliang created SPARK-33704:
---------------------------------
Summary: Support latest version of initialize() in HiveGenericUDTF
Key: SPARK-33704
URL: https://issues.apache.org/jira/browse/SPARK-33704
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.4.3, 2.3.4, 2.2.0
Reporter: chenliang
For HiveGenericUDTF , there are two initialization methods:
{code:java}
public StructObjectInspector initialize(StructObjectInspector argOIs)
throws UDFArgumentException {
List<? extends StructField> inputFields = argOIs.getAllStructFieldRefs();
ObjectInspector[] udtfInputOIs = new ObjectInspector[inputFields.size()];
for (int i = 0; i < inputFields.size(); i++) {
udtfInputOIs[i] = inputFields.get(i).getFieldObjectInspector();
}
return initialize(udtfInputOIs);
}
@Deprecated
public StructObjectInspector initialize(ObjectInspector[] argOIs)
throws UDFArgumentException {
throw new IllegalStateException("Should not be called directly");
}
{code}
As https://issues.apache.org/jira/browse/HIVE-5737 mentioned, hive provided
StructObjectInspector for UDTFs rather than ObjectInspect[], but Spark SQL
still only support deprecated function.
An exception will be reported before fix:
Error in query: No handler for UDF/UDAF/UDTF 'FeatureParseUDTF1':
java.lang.IllegalStateException: Should not be called directly
Please make sure your function overrides public StructObjectInspector
initialize(ObjectInspector[] args).; line 1 pos 7
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
