[GitHub] [arrow] emkornfield commented on pull request #6731: [WIP] ARROW-8601: [Go][Flight] Added implementation of FlightDataWriter

2020-09-05 Thread GitBox
emkornfield commented on pull request #6731: URL: https://github.com/apache/arrow/pull/6731#issuecomment-687693582 @sbinet did you ever take a look at this? Seems this has stalled? This is an automated message from the Apach

[GitHub] [arrow] lidavidm commented on a change in pull request #8010: ARROW-9587: [FlightRPC][Java] clean up FlightStream/DoPut

2020-09-05 Thread GitBox
lidavidm commented on a change in pull request #8010: URL: https://github.com/apache/arrow/pull/8010#discussion_r483976105 ## File path: java/memory/memory-core/src/main/java/org/apache/arrow/memory/BaseAllocator.java ## @@ -41,7 +41,7 @@ abstract class BaseAllocator extends

[GitHub] [arrow] lidavidm commented on a change in pull request #8010: ARROW-9587: [FlightRPC][Java] clean up FlightStream/DoPut

2020-09-05 Thread GitBox
lidavidm commented on a change in pull request #8010: URL: https://github.com/apache/arrow/pull/8010#discussion_r483976063 ## File path: java/flight/flight-core/src/main/java/org/apache/arrow/flight/FlightStream.java ## @@ -158,29 +167,52 @@ public FlightDescriptor getDescript

[GitHub] [arrow] github-actions[bot] commented on pull request #8117: ARROW-9921: [Rust] Replace TryFrom by From in `StringArray` from `Vec>` (+50%)

2020-09-05 Thread GitBox
github-actions[bot] commented on pull request #8117: URL: https://github.com/apache/arrow/pull/8117#issuecomment-687641141 https://issues.apache.org/jira/browse/ARROW-9921 This is an automated message from the Apache Git Serv

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #8116: ARROW-9919: [Rust][DataFusion] Speedup math operations by 15%+

2020-09-05 Thread GitBox
jorgecarleitao commented on a change in pull request #8116: URL: https://github.com/apache/arrow/pull/8116#discussion_r483971257 ## File path: rust/datafusion/benches/math_query_sql.rs ## @@ -0,0 +1,100 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or mo

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #8116: ARROW-9919: [Rust][DataFusion] Speedup math operations by 15%+

2020-09-05 Thread GitBox
jorgecarleitao commented on a change in pull request #8116: URL: https://github.com/apache/arrow/pull/8116#discussion_r483971257 ## File path: rust/datafusion/benches/math_query_sql.rs ## @@ -0,0 +1,100 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or mo

[GitHub] [arrow] jorgecarleitao opened a new pull request #8117: ARROW-9921: [Rust] Replace TryFrom by From in `StringArray` from `Vec>` (+50%)

2020-09-05 Thread GitBox
jorgecarleitao opened a new pull request #8117: URL: https://github.com/apache/arrow/pull/8117 This PR: * adds a benchmark to StringArray's `try_from(Vec>)` * Replaces `TryFrom` by `From` * improves performance by 50% by not using `StringBuilder` * adds the same functionality

[GitHub] [arrow] andygrove closed pull request #8113: ARROW-9916: [RUST] Avoid cloning array data

2020-09-05 Thread GitBox
andygrove closed pull request #8113: URL: https://github.com/apache/arrow/pull/8113 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] andygrove commented on pull request #8116: ARROW-9919: [Rust][DataFusion] Speedup math operations by 15%+

2020-09-05 Thread GitBox
andygrove commented on pull request #8116: URL: https://github.com/apache/arrow/pull/8116#issuecomment-687635427 Wow, very cool. This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [arrow] jhorstmann commented on pull request #8092: ARROW-9895: [Rust] Improve sorting kernels

2020-09-05 Thread GitBox
jhorstmann commented on pull request #8092: URL: https://github.com/apache/arrow/pull/8092#issuecomment-687633205 Thanks a lot for the review @jorgecarleitao , I think I addressed nearly all of your comments. The logical and physical plan docs currently don't contain much documentation reg

[GitHub] [arrow] jhorstmann commented on a change in pull request #8092: ARROW-9895: [Rust] Improve sorting kernels

2020-09-05 Thread GitBox
jhorstmann commented on a change in pull request #8092: URL: https://github.com/apache/arrow/pull/8092#discussion_r483965365 ## File path: rust/datafusion/src/physical_plan/sort.rs ## @@ -223,4 +223,51 @@ mod tests { Ok(()) } + +#[test] +fn test_lex_sort

[GitHub] [arrow] jhorstmann commented on a change in pull request #8092: ARROW-9895: [Rust] Improve sorting kernels

2020-09-05 Thread GitBox
jhorstmann commented on a change in pull request #8092: URL: https://github.com/apache/arrow/pull/8092#discussion_r483965314 ## File path: rust/arrow/src/compute/kernels/sort.rs ## @@ -148,58 +206,165 @@ impl Default for SortOptions { } } -/// Sort primitive values, exc

[GitHub] [arrow] jhorstmann commented on a change in pull request #8092: ARROW-9895: [Rust] Improve sorting kernels

2020-09-05 Thread GitBox
jhorstmann commented on a change in pull request #8092: URL: https://github.com/apache/arrow/pull/8092#discussion_r483965152 ## File path: rust/arrow/src/array/ord.rs ## @@ -68,65 +80,233 @@ impl OrdArray for NullArray { } } +macro_rules! float_ord_cmp { +($NAME: id

[GitHub] [arrow] nevi-me commented on a change in pull request #8116: ARROW-9919: [Rust][DataFusion] Speedup math operations by 15%+

2020-09-05 Thread GitBox
nevi-me commented on a change in pull request #8116: URL: https://github.com/apache/arrow/pull/8116#discussion_r483954060 ## File path: rust/datafusion/src/physical_plan/math_expressions.rs ## @@ -19,23 +19,25 @@ use crate::error::{ExecutionError, Result}; -use arrow::arra

[GitHub] [arrow] alamb commented on a change in pull request #8116: ARROW-9919: [Rust][DataFusion] Speedup math operations by 15%+

2020-09-05 Thread GitBox
alamb commented on a change in pull request #8116: URL: https://github.com/apache/arrow/pull/8116#discussion_r483940437 ## File path: rust/datafusion/benches/math_query_sql.rs ## @@ -0,0 +1,100 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contri

[GitHub] [arrow] rymurr commented on a change in pull request #8010: ARROW-9587: [FlightRPC][Java] clean up FlightStream/DoPut

2020-09-05 Thread GitBox
rymurr commented on a change in pull request #8010: URL: https://github.com/apache/arrow/pull/8010#discussion_r483935015 ## File path: java/memory/memory-core/src/main/java/org/apache/arrow/memory/BaseAllocator.java ## @@ -41,7 +41,7 @@ abstract class BaseAllocator extends Ac

[GitHub] [arrow] jhorstmann commented on pull request #8113: ARROW-9916: [RUST] Avoid cloning array data

2020-09-05 Thread GitBox
jhorstmann commented on pull request #8113: URL: https://github.com/apache/arrow/pull/8113#issuecomment-687585539 I saw the biggest effect when doing lexicographic sorting and added some benchmarks for sorting. On my laptop the run to run variation was much to high, but on an ec2 m5a.xlarg

[GitHub] [arrow] github-actions[bot] commented on pull request #8116: ARROW-9919: [Rust][DataFusion] Speedup math operations by 15%+

2020-09-05 Thread GitBox
github-actions[bot] commented on pull request #8116: URL: https://github.com/apache/arrow/pull/8116#issuecomment-687567384 https://issues.apache.org/jira/browse/ARROW-9919 This is an automated message from the Apache Git Serv

[GitHub] [arrow] jorgecarleitao opened a new pull request #8116: ARROW-9919: [Rust][DataFusion] Speedup math operations by 15%+

2020-09-05 Thread GitBox
jorgecarleitao opened a new pull request #8116: URL: https://github.com/apache/arrow/pull/8116 This PR improves speed of all math operations by 15% (on default batch size), and changes the scaling (to faster) of how these operations scale with batch size. See main issue here: ARROW-