Rahij Ramsharan created SPARK-32544: ---------------------------------------
Summary: Bucketing and Partitioning information are not passed on to non FileFormat datasource writes Key: SPARK-32544 URL: https://issues.apache.org/jira/browse/SPARK-32544 Project: Spark Issue Type: Improvement Components: Input/Output Affects Versions: 3.0.0 Reporter: Rahij Ramsharan When writing to a FileFormat datasource, bucket spec and partition columns are passed into InsertIntoHadoopFsRelationCommand: [https://github.com/apache/spark/blob/v3.0.0/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala#L474-L475]. However, from what I can tell, the RelationProvider API does not have a way to pass in these: [https://github.com/apache/spark/blob/v3.0.0/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala#L511-L513]. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org