Cheng Hao created SPARK-3904: -------------------------------- Summary: HQL doesn't support the ConstantObjectInspector to pass into UDFs Key: SPARK-3904 URL: https://issues.apache.org/jira/browse/SPARK-3904 Project: Spark Issue Type: Bug Components: SQL Reporter: Cheng Hao
In HQL, we convert all of the data type into normal ObjectInspectors for UDFs, most of cases it work, however, some of the UDF actually requires the input ObjectInspector to be the ConstantObjectInspector, which will cause exception. e.g. {panel} select named_struct("x", "str") from src limit 1 {panel} It will throws exception like {panel} 14/10/10 16:25:17 INFO parse.ParseDriver: Parsing command: select named_struct("x", "str") from src 14/10/10 16:25:17 INFO parse.ParseDriver: Parse Completed 14/10/10 16:25:17 INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=src 14/10/10 16:25:17 INFO HiveMetaStore.audit: ugi=hcheng ip=unknown-ip-addr cmd=get_table : db=default tbl=tmp2 14/10/10 16:25:17 ERROR thriftserver.SparkSQLDriver: Failed in [select named_struct("x", "str") from src] org.apache.hadoop.hive.ql.exec.UDFArgumentTypeException: Even arguments to NAMED_STRUCT must be a constant STRING.org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaStringObjectInspector@2f2dbcfc at org.apache.hadoop.hive.ql.udf.generic.GenericUDFNamedStruct.initialize(GenericUDFNamedStruct.java:55) at org.apache.spark.sql.hive.HiveGenericUdf.returnInspector$lzycompute(hiveUdfs.scala:129) at org.apache.spark.sql.hive.HiveGenericUdf.returnInspector(hiveUdfs.scala:129) at org.apache.spark.sql.hive.HiveGenericUdf.eval(hiveUdfs.scala:158) at org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$6$$anonfun$applyOrElse$2.applyOrElse(Optimizer.scala:267) at org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$6$$anonfun$applyOrElse$2.applyOrElse(Optimizer.scala:260) {panel} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org