[ https://issues.apache.org/jira/browse/ARROW-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223386#comment-17223386 ]
Micah Kornfield commented on ARROW-10426: ----------------------------------------- I think this is just an oversight. There is a limit of 4 bytes for encoding the length of any one string > [C++] Arrow type large_string cannot be written to Parquet type column > descriptor > --------------------------------------------------------------------------------- > > Key: ARROW-10426 > URL: https://issues.apache.org/jira/browse/ARROW-10426 > Project: Apache Arrow > Issue Type: Bug > Components: C++, R > Affects Versions: 2.0.0 > Environment: R 4.0.3 on OSX 10.15.7 > Reporter: Gabriel Bassett > Priority: Minor > Labels: parquet > > When trying to write a dataset in parquet format, arrow errors with the > message: "Arrow type large_string cannot be written to Parquet type column > descriptor" > {code:java} > arrow::write_dataset( > dataframe, > "/directory/", > "parquet", > "partitioning" = c("col1", "col2") > ) > {code} > The dataframe in question is very large with one column containing the text > of message board posts encoded in HTML. -- This message was sent by Atlassian Jira (v8.3.4#803005)