[ https://issues.apache.org/jira/browse/ARROW-6960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958058#comment-16958058 ]
Grant Nguyen commented on ARROW-6960: ------------------------------------- Thanks [~npr] for the very detailed explanation, that helps a lot! I will look into this in the next few days – the lz4 seems like a good starting point – not sure that I have quite the level of expertise to add zstd and brotli to rtools but will investigate further. > [R] Add information about zstd/lz4 codec installation and linkages for R users > ------------------------------------------------------------------------------ > > Key: ARROW-6960 > URL: https://issues.apache.org/jira/browse/ARROW-6960 > Project: Apache Arrow > Issue Type: Improvement > Components: R > Affects Versions: 0.15.0 > Environment: Windows 10 > Reporter: Grant Nguyen > Priority: Minor > > When I attempt to write a parquet file using lz4, zstd, or brotli compression > using R arrow 0.15.0, I am unable to do so due to the codec support not being > built (example below). > > {code:java} > > arrow::write_parquet(payout_strategy, sink = > > "records_test_lz4.parquet",compression = "lz4") > Error in parquet___arrow___FileWriter__WriteTable(self, table, chunk_size) : > Arrow error: IOError: Arrow error: NotImplemented: LZ4 codec support not > built{code} > > I believe that the error is generated through > [https://github.com/apache/arrow/blob/master/cpp/src/arrow/util/compression.cc#L124-L145], > but I am not sure how to call > {code:java} > install.packages("arrow"){code} > in R to enable the ARROW_WITH_ZSTD/LZ4/BROTLI flags, or whether I should be > doing installing zstd separately from arrow and then doing something pre- or > post-install to link zstd with arrow. From > [https://github.com/apache/arrow/issues/1209], it appears that zstd support > has been added to arrow and parquet in general, and the R package readme > ([https://github.com/apache/arrow/tree/master/r)|https://github.com/apache/arrow/tree/master/r] > notes "On macOS and Windows, installing a binary package from CRAN will > handle Arrow's C++ dependencies for you", but I get the sense that does not > apply to zstd. > > Is there guidance as to how to enable zstd and other compression codecs prior > to or after downloading the R arrow package? Could this be added to the R > documentation somewhere for future reference? -- This message was sent by Atlassian Jira (v8.3.4#803005)