[
https://issues.apache.org/jira/browse/ARROW-8674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523434#comment-17523434
]
Dominik Moritz commented on ARROW-8674:
---------------------------------------
Looking at lz4js, it's so small
(https://cdn.jsdelivr.net/npm/lz4js@0.2.0/lz4.min.js) that it's probably okay
to pull in a dependency by default. I agree that having some system to register
a different decompress function could be nice. lz4js is a bit old so we would
want to carefully look at the available libraries. It would be nice to have
some out of the box support. To avoid increasing bundle sizes, we can decide
which functions actually use the decompression library.
Could you look at the available js libraries and see what their sizes are?
Also, is lz4 or zstd much more common than the other?
We also should look into how much benefit we actually get from compression
since most servers already support transparent gzip compression and so
compressing an already compressed file will just incur overhead.
If the libraries are too heavy, we can think about a plugin system. We could
make our registry be synchronous.
I definitely don't want to pull in wasm into the library as it will break
people's workflows.
> [JS] Implement IPC RecordBatch body buffer compression from ARROW-300
> ---------------------------------------------------------------------
>
> Key: ARROW-8674
> URL: https://issues.apache.org/jira/browse/ARROW-8674
> Project: Apache Arrow
> Issue Type: Sub-task
> Components: JavaScript
> Reporter: Wes McKinney
> Priority: Major
>
> This may not be a hard requirement for JS because this would require pulling
> in implementations of LZ4 and ZSTD which not all users may want
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
