[ https://issues.apache.org/jira/browse/ARROW-14998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Keane updated ARROW-14998: ----------------------------------- Summary: [R] Support for HTTPS Filesystem access (was: Support for HTTPS Filesystem access (from R client)) > [R] Support for HTTPS Filesystem access > --------------------------------------- > > Key: ARROW-14998 > URL: https://issues.apache.org/jira/browse/ARROW-14998 > Project: Apache Arrow > Issue Type: Wish > Components: R > Reporter: Carl Boettiger > Priority: Major > > Thanks for such an amazing project. I've been entirely blown away by the S3 > Filesystem access in the latest release; and excited to see other backends > like Azure being discussed in the issues. As you know, many https clients > also permit range requests, meaning (I think) that it should be possible to > access public data (parquet, csv files) over generic HTTPS connections too. > As you probably know, duckdb already has support for https based remote file > access, e.g. > [https://github.com/duckdb/duckdb/blob/master/test/sql/copy/parquet/test_parquet_remote.test|https://github.com/duckdb/duckdb/blob/master/test/sql/copy/parquet/test_parquet_remote.test.] > (though it is not available out-of-the-box in the R client there either). > > It would be wonderful to have a similar remote filesystem access that could > work over HTTPS like that in arrow. (I gather on the python side, fsspec > already gives access to a wide number of such abstractions, but we're more > limited in R so far, except for the geospatial data, where bindings to GDAL > mean we can access GDAL's rather amazing virtual file systems over https, S3, > FTP, etc, [https://gdal.org/user/virtual_file_systems.html] – a nice > array-data complement to the more database-oriented workflow of arrow...). > > Thanks for considering! -- This message was sent by Atlassian Jira (v8.20.1#820001)