[ https://issues.apache.org/jira/browse/ARROW-17046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566145#comment-17566145 ]
Alenka Frim commented on ARROW-17046: ------------------------------------- Great, now you only need to click on "Create pull request" so that the change to the code will be visible on Apache Arrow repo as a PR. When setting the title of the pull request please follow the {_}second item in the guidelines{_}: [https://arrow.apache.org/docs/developers/overview.html#pull-request-and-review] so that the PR gets connected to the Jira issue. > [Python] pyarrow.parquet.write_to_dataset fails to pass kwargs to write_table > function > -------------------------------------------------------------------------------------- > > Key: ARROW-17046 > URL: https://issues.apache.org/jira/browse/ARROW-17046 > Project: Apache Arrow > Issue Type: Improvement > Components: Python > Affects Versions: 8.0.0 > Reporter: Amir Khosroshahi > Priority: Minor > > According to PyArrow 8.0.0 > [documentation|https://arrow.apache.org/docs/python/generated/pyarrow.parquet.write_to_dataset.html] > {{kwargs}} is "Additional kwargs for {{write_table}} function." However when > I try to pass additional arguments, for example {{{}flavor{}}}, to the > underlying write_table I get the following error > {code:java} > TypeError: unexpected parquet write option: flavor{code} > This used to work in PyArrow versions as late as 7.0.0 but started to break > in 8.0.0. > Minimal example to reproduce the error > {code:java} > import pyarrow as pa > import pandas as pd > import pyarrow.parquet as pq > df = pd.DataFrame({'a': [1, 2], 'b': [3, 4]}) > tb = pa.Table.from_pandas(df) > pq.write_to_dataset(tb, "test.parquet", flavor="spark") {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)