GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/6179
[SPARK-6747] [SQL] Support List<> as a return type in Hive UDF This patch supports List<> as a return type in Hive UDF. We assume an UDF below; public class UDFToListString extends UDF { public List evaluate(Object o) { return Arrays.asList("xxx", "yyy", "zzz"); } } An exception of scala.MatchError is thrown as follows when the UDF used in the current implementation. scala.MatchError: interface java.util.List (of class java.lang.Class) at org.apache.spark.sql.hive.HiveInspectors$class.javaClassToDataType(HiveInspectors.scala:174) at org.apache.spark.sql.hive.HiveSimpleUdf.javaClassToDataType(hiveUdfs.scala:76) at org.apache.spark.sql.hive.HiveSimpleUdf.dataType$lzycompute(hiveUdfs.scala:106) at org.apache.spark.sql.hive.HiveSimpleUdf.dataType(hiveUdfs.scala:106) at org.apache.spark.sql.catalyst.expressions.Alias.toAttribute(namedExpressions.scala:131) at org.apache.spark.sql.catalyst.planning.PhysicalOperation$$anonfun$collectAliases$1.applyOrElse(patterns.scala:95) at org.apache.spark.sql.catalyst.planning.PhysicalOperation$$anonfun$collectAliases$1.applyOrElse(patterns.scala:94) at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:33) at scala.collection.TraversableLike$$anonfun$collect$1.apply(TraversableLike.scala:278) ... You can merge this pull request into a Git repository by running: $ git pull https://github.com/maropu/spark FixBugInHiveInspectors Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/6179.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #6179 ---- commit e553f10c67f129e59c526d6149e60c77348f546a Author: Takeshi YAMAMURO <linguin....@gmail.com> Date: 2015-04-07T16:41:17Z Support List as a return type in Hive UDF commit e21ce7e33139b96581dbd37785f99854e60379fd Author: Takeshi YAMAMURO <linguin....@gmail.com> Date: 2015-04-08T02:18:22Z Add a blank line at the end of UDFToListString commit 94064169380d129a41574e7ee3ccc994532f71f0 Author: Takeshi YAMAMURO <linguin....@gmail.com> Date: 2015-04-14T16:46:44Z Apply review comments commit f965c34bcea8efa6b58443ef31c4d857b5feb09e Author: Takeshi YAMAMURO <linguin....@gmail.com> Date: 2015-04-14T18:04:31Z Fix code-style errors commit 1c7b9d1aae14de43ca9bf6e3414a45f1104e5043 Author: Takeshi YAMAMURO <linguin....@gmail.com> Date: 2015-04-17T00:51:14Z Remove a new type commit a4887127834fc73063851fd57f095ff212f9b483 Author: Takeshi YAMAMURO <linguin....@gmail.com> Date: 2015-05-15T04:36:50Z Add StringToUtf8 to comvert String into UTF8String commit 2b3f8a1dc9153aab0bd13767d12fc258ce5a1782 Author: Takeshi YAMAMURO <linguin....@gmail.com> Date: 2015-05-15T07:18:40Z Add TOD comments in UDFToListString of HiveUdfSuite ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org