jorisvandenbossche commented on pull request #9685: URL: https://github.com/apache/arrow/pull/9685#issuecomment-799452300
> It does not autodetect the type of compression (but perhaps this could be added, by inspecting FileSource). Small note here: the python API for reading plain CSV files (using `pyarrow.csv`) automatically detects compressed files and doesn't have an explicit option for that. So *ideally*, the dataset CSV reading would work similarly, I think. But AFAIK, the decompressing for `pyarrow.csv` currently happens on the python side (and not in C++)? (i.e. `get_input_stream` in the cython code detects compression) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
