[ https://issues.apache.org/jira/browse/SPARK-33330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Miguel Duarte updated SPARK-33330: ---------------------------------- Description: Given that: # JavaTypeInference maps [Enums to StringType|https://github.com/apache/spark/blob/55105a0784459331d5506eee9f37c2e655a2a6a0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/JavaTypeInference.scala#L135] # The converter for [StringType is StringConverter|https://github.com/apache/spark/blob/55105a0784459331d5506eee9f37c2e655a2a6a0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/CatalystTypeConverters.scala#L64] # StringConverter is unable to convert an [Enums to UTF8String|https://github.com/apache/spark/blob/55105a0784459331d5506eee9f37c2e655a2a6a0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/CatalystTypeConverters.scala#L294], failing with an InvalidArgumentException It can be argued that CatalystTypeConverters should align with the exprectations set by JavaTypeInference and convert enums to their string representation. Edit: Added PRs for 3.0.X and 2.4.X branches was: Given that: # JavaTypeInference maps [Enums to StringType|https://github.com/apache/spark/blob/55105a0784459331d5506eee9f37c2e655a2a6a0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/JavaTypeInference.scala#L135] # The converter for [StringType is StringConverter|https://github.com/apache/spark/blob/55105a0784459331d5506eee9f37c2e655a2a6a0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/CatalystTypeConverters.scala#L64] # StringConverter is unable to convert an [Enums to UTF8String|https://github.com/apache/spark/blob/55105a0784459331d5506eee9f37c2e655a2a6a0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/CatalystTypeConverters.scala#L294], failing with an InvalidArgumentException It can be argued that CatalystTypeConverters should align with the exprectations set by JavaTypeInference and convert enums to their string representation. > Catalyst StringType converter unable to convert enum type > --------------------------------------------------------- > > Key: SPARK-33330 > URL: https://issues.apache.org/jira/browse/SPARK-33330 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.4.7, 3.0.1 > Reporter: Miguel Duarte > Priority: Major > > Given that: > # JavaTypeInference maps [Enums to > StringType|https://github.com/apache/spark/blob/55105a0784459331d5506eee9f37c2e655a2a6a0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/JavaTypeInference.scala#L135] > > # The converter for [StringType is > StringConverter|https://github.com/apache/spark/blob/55105a0784459331d5506eee9f37c2e655a2a6a0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/CatalystTypeConverters.scala#L64] > # StringConverter is unable to convert an [Enums to > UTF8String|https://github.com/apache/spark/blob/55105a0784459331d5506eee9f37c2e655a2a6a0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/CatalystTypeConverters.scala#L294], > failing with an InvalidArgumentException > > It can be argued that CatalystTypeConverters should align with the > exprectations set by JavaTypeInference and convert enums to their string > representation. > > Edit: > Added PRs for 3.0.X and 2.4.X branches -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org