[
https://issues.apache.org/jira/browse/SPARK-43357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-43357.
---------------------------------
Fix Version/s: 3.5.0
Resolution: Fixed
Issue resolved by pull request 41035
[https://github.com/apache/spark/pull/41035]
> Spark AWS Glue date partition push down broken
> ----------------------------------------------
>
> Key: SPARK-43357
> URL: https://issues.apache.org/jira/browse/SPARK-43357
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.1.0, 3.1.1, 3.1.2, 3.2.0, 3.1.3, 3.2.1, 3.3.0, 3.2.2,
> 3.3.1, 3.2.3, 3.2.4, 3.3.2
> Reporter: Stijn De Haes
> Assignee: Stijn De Haes
> Priority: Major
> Fix For: 3.5.0
>
>
> When using the following project:
> [https://github.com/awslabs/aws-glue-data-catalog-client-for-apache-hive-metastore]
> To have glue supported as as a hive metastore for spark there is an issue
> when reading a date-partitioned data set. Writing is fine.
> You get the following error:
> {quote}org.apache.hadoop.hive.metastore.api.InvalidObjectException:
> Unsupported expression '2023 - 05 - 03' (Service: AWSGlue; Status Code: 400;
> Error Code: InvalidInputException; Request ID:
> beed68c6-b228-442e-8783-52c25b9d2243; Proxy: null)
> {quote}
>
> A fix for this is making sure the date passed to glue is quoted
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
