[
https://issues.apache.org/jira/browse/ARROW-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223386#comment-17223386
]
Micah Kornfield commented on ARROW-10426:
-----------------------------------------
I think this is just an oversight. There is a limit of 4 bytes for encoding
the length of any one string
> [C++] Arrow type large_string cannot be written to Parquet type column
> descriptor
> ---------------------------------------------------------------------------------
>
> Key: ARROW-10426
> URL: https://issues.apache.org/jira/browse/ARROW-10426
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, R
> Affects Versions: 2.0.0
> Environment: R 4.0.3 on OSX 10.15.7
> Reporter: Gabriel Bassett
> Priority: Minor
> Labels: parquet
>
> When trying to write a dataset in parquet format, arrow errors with the
> message: "Arrow type large_string cannot be written to Parquet type column
> descriptor"
> {code:java}
> arrow::write_dataset(
> dataframe,
> "/directory/",
> "parquet",
> "partitioning" = c("col1", "col2")
> )
> {code}
> The dataframe in question is very large with one column containing the text
> of message board posts encoded in HTML.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
