[jira] [Commented] (ARROW-11593) [Rust] Parquet does not support wasm32-unknown-unknown target
[ https://issues.apache.org/jira/browse/ARROW-11593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332211#comment-17332211 ] Andrew Lamb commented on ARROW-11593: - Migrated to github: https://github.com/apache/arrow-rs/issues/180 > [Rust] Parquet does not support wasm32-unknown-unknown target > - > > Key: ARROW-11593 > URL: https://issues.apache.org/jira/browse/ARROW-11593 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust >Reporter: Dominik Moritz >Priority: Major > > The Arrow crate successfully compiles to WebAssembly (e.g. > https://github.com/domoritz/arrow-wasm) but the Parquet crate currently does > not support the`wasm32-unknown-unknown` target. > Try out the repository at > https://github.com/domoritz/parquet-wasm/commit/e877f9ad9c45c09f73d98fab2a8ad384a802b2e0. > The problem seems to be in liblz4, even if I do not include lz4 in the > feature flags. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11593) [Rust] Parquet does not support wasm32-unknown-unknown target
[ https://issues.apache.org/jira/browse/ARROW-11593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17319022#comment-17319022 ] Dominik Moritz commented on ARROW-11593: That's awesome. Do you want to add a note to https://issues.apache.org/jira/projects/ARROW/issues/ARROW-11615, which tracks DataFusion support for wasm? > [Rust] Parquet does not support wasm32-unknown-unknown target > - > > Key: ARROW-11593 > URL: https://issues.apache.org/jira/browse/ARROW-11593 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust >Reporter: Dominik Moritz >Priority: Major > > The Arrow crate successfully compiles to WebAssembly (e.g. > https://github.com/domoritz/arrow-wasm) but the Parquet crate currently does > not support the`wasm32-unknown-unknown` target. > Try out the repository at > https://github.com/domoritz/parquet-wasm/commit/e877f9ad9c45c09f73d98fab2a8ad384a802b2e0. > The problem seems to be in liblz4, even if I do not include lz4 in the > feature flags. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11593) [Rust] Parquet does not support wasm32-unknown-unknown target
[ https://issues.apache.org/jira/browse/ARROW-11593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17318970#comment-17318970 ] David Roher commented on ARROW-11593: - I just got a version of DataFusion working on wasm32-unknown-unknown – it required disabling both the LZ4 and ZSTD features on Parquet and tweaking the hash function: [https://github.com/apache/arrow/compare/master...droher:master] To add to [~AndyRedhead1974]'s point above, it would also be useful in a serverless context – for instance, Cloudflare Workers Unbound is in beta now and will allow WASM functions to run at unlimited CPU usage. in this context, DataFusion could be a serverless data lake engine like AWS Athena. Maybe it could even be useful as a Ballista worker. > [Rust] Parquet does not support wasm32-unknown-unknown target > - > > Key: ARROW-11593 > URL: https://issues.apache.org/jira/browse/ARROW-11593 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust >Reporter: Dominik Moritz >Priority: Major > > The Arrow crate successfully compiles to WebAssembly (e.g. > https://github.com/domoritz/arrow-wasm) but the Parquet crate currently does > not support the`wasm32-unknown-unknown` target. > Try out the repository at > https://github.com/domoritz/parquet-wasm/commit/e877f9ad9c45c09f73d98fab2a8ad384a802b2e0. > The problem seems to be in liblz4, even if I do not include lz4 in the > feature flags. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11593) [Rust] Parquet does not support wasm32-unknown-unknown target
[ https://issues.apache.org/jira/browse/ARROW-11593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17295597#comment-17295597 ] Andy Redhead commented on ARROW-11593: -- A WebAssembly compatible Rust library that can read data from Parquet files would be very useful to anyone who would like to do "browser based" data processing/visualisation (better still if that library is in a family that includes efficient in-memory "data structures"). > [Rust] Parquet does not support wasm32-unknown-unknown target > - > > Key: ARROW-11593 > URL: https://issues.apache.org/jira/browse/ARROW-11593 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust >Reporter: Dominik Moritz >Priority: Major > > The Arrow crate successfully compiles to WebAssembly (e.g. > https://github.com/domoritz/arrow-wasm) but the Parquet crate currently does > not support the`wasm32-unknown-unknown` target. > Try out the repository at > https://github.com/domoritz/parquet-wasm/commit/e877f9ad9c45c09f73d98fab2a8ad384a802b2e0. > The problem seems to be in liblz4, even if I do not include lz4 in the > feature flags. -- This message was sent by Atlassian Jira (v8.3.4#803005)
