[
https://issues.apache.org/jira/browse/SPARK-32935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-32935:
----------------------------------
Description:
Datasource V2 does not currently support bucketed reads or writes similar to
Datasource V1 does. See DatasourceScanExec and config
spark.sql.sources.bucketing.enabled. We need to add support to V2 as well.
Support writing file data source with bucketing looks like:
{code:java}
fileDf.write.bucketBy(...).sortBy(..)...
{code}
was:
Datasource V2 does not currently support bucketed reads similar to Datasource
V1 does. See DatasourceScanExec and config
spark.sql.sources.bucketing.enabled. We need to add support to V2 as well.
Support writing file data source with bucketing
{code:java}
fileDf.write.bucketBy(...).sortBy(..)...
{code}
> File source V2: support bucketing
> ---------------------------------
>
> Key: SPARK-32935
> URL: https://issues.apache.org/jira/browse/SPARK-32935
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.1.0
> Reporter: Gengliang Wang
> Priority: Major
>
> Datasource V2 does not currently support bucketed reads or writes similar to
> Datasource V1 does. See DatasourceScanExec and config
> spark.sql.sources.bucketing.enabled. We need to add support to V2 as well.
>
> Support writing file data source with bucketing looks like:
> {code:java}
>
> fileDf.write.bucketBy(...).sortBy(..)...
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
