ianmcook commented on issue #40597: URL: https://github.com/apache/arrow/issues/40597#issuecomment-2002770182
Ah, ok, I see what you mean now. You have a "dumb" HTTP server that is just serving files from a directory structure and doesn't know anything about Arrow. In that case, if the files are in the Arrow IPC file format and the server supports range requests, then the client can take advantage of the footers to download specific record batches, schemas, or other data blocks from the files. This is perhaps a bit of an obscure case; we don't recommend using the IPC file format for archival storage so there are not many large collections of Arrow IPC files as far as I know. Parquet files are vastly more common. But it might be worth mentioning describing this in a section of the Arrow-over-HTTP conventions document. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
