[ https://issues.apache.org/jira/browse/SPARK-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Armbrust resolved SPARK-3904. ------------------------------------- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2762 [https://github.com/apache/spark/pull/2762] > HQL doesn't support the ConstantObjectInspector to pass into UDFs > ----------------------------------------------------------------- > > Key: SPARK-3904 > URL: https://issues.apache.org/jira/browse/SPARK-3904 > Project: Spark > Issue Type: Bug > Components: SQL > Reporter: Cheng Hao > Assignee: Cheng Hao > Fix For: 1.2.0 > > > In HQL, we convert all of the data type into normal ObjectInspectors for > UDFs, most of cases it work, however, some of the UDF actually requires the > input ObjectInspector to be the ConstantObjectInspector, which will cause > exception. > e.g. > {panel} > select named_struct("x", "str") from src limit 1 > {panel} > It will throws exception like > {panel} > 14/10/10 16:25:17 INFO parse.ParseDriver: Parsing command: select > named_struct("x", "str") from src > 14/10/10 16:25:17 INFO parse.ParseDriver: Parse Completed > 14/10/10 16:25:17 INFO metastore.HiveMetaStore: 0: get_table : db=default > tbl=src > 14/10/10 16:25:17 INFO HiveMetaStore.audit: ugi=hcheng > ip=unknown-ip-addr cmd=get_table : db=default tbl=tmp2 > 14/10/10 16:25:17 ERROR thriftserver.SparkSQLDriver: Failed in [select > named_struct("x", "str") from src] > org.apache.hadoop.hive.ql.exec.UDFArgumentTypeException: Even arguments to > NAMED_STRUCT must be a constant > STRING.org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaStringObjectInspector@2f2dbcfc > at > org.apache.hadoop.hive.ql.udf.generic.GenericUDFNamedStruct.initialize(GenericUDFNamedStruct.java:55) > at > org.apache.spark.sql.hive.HiveGenericUdf.returnInspector$lzycompute(hiveUdfs.scala:129) > at > org.apache.spark.sql.hive.HiveGenericUdf.returnInspector(hiveUdfs.scala:129) > at org.apache.spark.sql.hive.HiveGenericUdf.eval(hiveUdfs.scala:158) > at > org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$6$$anonfun$applyOrElse$2.applyOrElse(Optimizer.scala:267) > at > org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$6$$anonfun$applyOrElse$2.applyOrElse(Optimizer.scala:260) > {panel} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org