Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22812#discussion_r228391626
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -2384,14 +2384,23 @@ class Analyzer(
case UnresolvedMapObjects(func, inputData, cls) if
inputData.resolved =>
inputData.dataType match {
case ArrayType(et, cn) =>
- val expr = MapObjects(func, inputData, et, cn, cls)
transformUp {
+ MapObjects(func, inputData, et, cn, cls) transformUp {
case UnresolvedExtractValue(child, fieldName) if
child.resolved =>
ExtractValue(child, fieldName, resolver)
}
- expr
case other =>
throw new AnalysisException("need an array field but got
" + other.catalogString)
}
+ case u: UnresolvedCatalystToExternalMap if u.child.resolved =>
+ u.child.dataType match {
+ case _: MapType =>
+ CatalystToExternalMap(u) transformUp {
+ case UnresolvedExtractValue(child, fieldName) if
child.resolved =>
--- End diff --
TBH I don't quite remember why I did this for `MapObjects`, so I just
follow it here. Maybe we can remove it in a followup PR.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
