[GitHub] [arrow] arw2019 commented on issue #8610: Does Arrow work with Python 3.9+?

2020-11-07 Thread GitBox
arw2019 commented on issue #8610: URL: https://github.com/apache/arrow/issues/8610#issuecomment-723526024 xref #8386 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [arrow] shyamalschandra opened a new issue #8610: Does Arrow work with Python 3.9+?

2020-11-07 Thread GitBox
shyamalschandra opened a new issue #8610: URL: https://github.com/apache/arrow/issues/8610 Does Arrow work with Python 3.9+? This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [arrow] nealrichardson closed pull request #8547: ARROW-10228: [Julia] Contribute Julia implementation

2020-11-07 Thread GitBox
nealrichardson closed pull request #8547: URL: https://github.com/apache/arrow/pull/8547 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [arrow] nealrichardson commented on pull request #8547: ARROW-10228: [Julia] Contribute Julia implementation

2020-11-07 Thread GitBox
nealrichardson commented on pull request #8547: URL: https://github.com/apache/arrow/pull/8547#issuecomment-72350 72 hours have passed so I've closed the lazy consensus vote at incubator: https://lists.apache.org/thread.html/raa0c7dacd8022c4110774e1523a082afa374cb852c57a8d1ff5077b3%40%3

[GitHub] [arrow] github-actions[bot] commented on pull request #8609: ARROW-10513: [Rust]: ARMv7 compilation

2020-11-07 Thread GitBox
github-actions[bot] commented on pull request #8609: URL: https://github.com/apache/arrow/pull/8609#issuecomment-723499160 https://issues.apache.org/jira/browse/ARROW-10513 This is an automated message from the Apache Git Ser

[GitHub] [arrow] vertexclique opened a new pull request #8609: ARROW-10513: [Rust]: ARMv7 compilation

2020-11-07 Thread GitBox
vertexclique opened a new pull request #8609: URL: https://github.com/apache/arrow/pull/8609 todos: - [x] gnu abi hf - [ ] gnu abi - [x] musl abi hf - [ ] musl abi - [ ] start ci for armv7 This is an au

[GitHub] [arrow] jorgecarleitao commented on pull request #8543: ARROW-9361 [Rust] Move array types into their own modules

2020-11-07 Thread GitBox
jorgecarleitao commented on pull request #8543: URL: https://github.com/apache/arrow/pull/8543#issuecomment-723480908 @alamb and @nevi-me , thanks a lot for taking the time to review this. I have rebased this against master. I am fine with either order, rebasing this one is tedious b

[GitHub] [arrow] nevi-me closed pull request #8541: ARROW-10402: [Rust] Refactor array equality

2020-11-07 Thread GitBox
nevi-me closed pull request #8541: URL: https://github.com/apache/arrow/pull/8541 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [arrow] nevi-me commented on pull request #8541: ARROW-10402: [Rust] Refactor array equality

2020-11-07 Thread GitBox
nevi-me commented on pull request #8541: URL: https://github.com/apache/arrow/pull/8541#issuecomment-723471732 > @alamb , @nevi-me and @jhorstmann , thanks a lot for taking a close look into this, also for the amazing work yesterday and today on reviewing and merging stuff. Really impressi

[GitHub] [arrow] vertexclique edited a comment on pull request #8598: ARROW-10500: [Rust] Refactor bit slice, bit view iterator for array buffers

2020-11-07 Thread GitBox
vertexclique edited a comment on pull request #8598: URL: https://github.com/apache/arrow/pull/8598#issuecomment-723470048 I don't think that's needed at all. Optimizations were extremely premature. We should have thought about the future beforehand. I think we are prolonging the discussio

[GitHub] [arrow] jorgecarleitao commented on pull request #8541: ARROW-10402: [Rust] Refactor array equality

2020-11-07 Thread GitBox
jorgecarleitao commented on pull request #8541: URL: https://github.com/apache/arrow/pull/8541#issuecomment-723470287 @alamb , @nevi-me and @jhorstmann , thanks a lot for taking a close look into this, also for the amazing work yesterday and today on reviewing and merging stuff. Really imp

[GitHub] [arrow] vertexclique edited a comment on pull request #8598: ARROW-10500: [Rust] Refactor bit slice, bit view iterator for array buffers

2020-11-07 Thread GitBox
vertexclique edited a comment on pull request #8598: URL: https://github.com/apache/arrow/pull/8598#issuecomment-723470048 I don't think that's needed at all. Optimizations were extremely premature. We should have thought about the future beforehand. I think we are prolonging the discussio

[GitHub] [arrow] vertexclique commented on pull request #8598: ARROW-10500: [Rust] Refactor bit slice, bit view iterator for array buffers

2020-11-07 Thread GitBox
vertexclique commented on pull request #8598: URL: https://github.com/apache/arrow/pull/8598#issuecomment-723470048 I don't think that's needed at all. Optimizations were extremely premature. We should have thought about the future beforehand. I think we are prolonging the discussion witho

[GitHub] [arrow] nevi-me closed pull request #8606: ARROW-10510: [Rust] [DataFusion] Benchmark COUNT(DISTINCT) queries.

2020-11-07 Thread GitBox
nevi-me closed pull request #8606: URL: https://github.com/apache/arrow/pull/8606 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [arrow] jhorstmann commented on a change in pull request #8541: ARROW-10402: [Rust] Refactor array equality

2020-11-07 Thread GitBox
jhorstmann commented on a change in pull request #8541: URL: https://github.com/apache/arrow/pull/8541#discussion_r519187056 ## File path: rust/arrow/src/array/data.rs ## @@ -121,7 +120,7 @@ impl ArrayData { /// Returns whether the element at index `i` is null pub fn

[GitHub] [arrow] kiszk commented on pull request #8057: ARROW-9862: [Java] Enable UnsafeDirectLittleEndian on a big-endian platform

2020-11-07 Thread GitBox
kiszk commented on pull request #8057: URL: https://github.com/apache/arrow/pull/8057#issuecomment-723455119 I added tests to verify it works well on any architecture. This is an automated message from the Apache Git Service.

[GitHub] [arrow] nevi-me closed pull request #8562: ARROW-10445: [Rust] Added doubleEnded iterator to PrimitiveArrayIter

2020-11-07 Thread GitBox
nevi-me closed pull request #8562: URL: https://github.com/apache/arrow/pull/8562 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [arrow] nevi-me closed pull request #8560: ARROW-10448: [Rust] Remove PrimitiveArray::new that can cause UB

2020-11-07 Thread GitBox
nevi-me closed pull request #8560: URL: https://github.com/apache/arrow/pull/8560 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [arrow] nevi-me commented on pull request #8545: ARROW-9674: [Rust] Make the parquet read and writers Send

2020-11-07 Thread GitBox
nevi-me commented on pull request #8545: URL: https://github.com/apache/arrow/pull/8545#issuecomment-723442063 @Marwes may you please rebase this, I think the conflicts are from your other PR, which might share some commits with this one ---

[GitHub] [arrow] nevi-me closed pull request #8561: ARROW-10449 [Rust] Make Dictionary::keys be an array

2020-11-07 Thread GitBox
nevi-me closed pull request #8561: URL: https://github.com/apache/arrow/pull/8561 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [arrow] nevi-me closed pull request #8608: ARROW-10261: [Rust] [Breaking] Change List datatype to Box

2020-11-07 Thread GitBox
nevi-me closed pull request #8608: URL: https://github.com/apache/arrow/pull/8608 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [arrow] alamb commented on pull request #8571: ARROW-10461: [Rust] Fix offset bug in remainder bits

2020-11-07 Thread GitBox
alamb commented on pull request #8571: URL: https://github.com/apache/arrow/pull/8571#issuecomment-723439185 FYI @jorgecarleitao / @nevi-me I think we should merge this PR. This is an automated message from the Apache Git S

[GitHub] [arrow] alamb commented on a change in pull request #8541: ARROW-10402: [Rust] Refactor array equality

2020-11-07 Thread GitBox
alamb commented on a change in pull request #8541: URL: https://github.com/apache/arrow/pull/8541#discussion_r519169437 ## File path: rust/arrow/src/array/builder.rs ## @@ -3535,14 +3531,15 @@ mod tests { None, Some(6), Some(7), +

[GitHub] [arrow] nevi-me commented on pull request #8608: ARROW-10261: [Rust] [Breaking] Change List datatype to Box

2020-11-07 Thread GitBox
nevi-me commented on pull request #8608: URL: https://github.com/apache/arrow/pull/8608#issuecomment-723437224 > I have a question related to why do we use a `Box<>` instead of just putting it into the stack, but other parts of the code have that, and thus makes sense to keep it as is.

[GitHub] [arrow] jorgecarleitao commented on pull request #8608: ARROW-10261: [Rust] [Breaking] Change List datatype to Box

2020-11-07 Thread GitBox
jorgecarleitao commented on pull request #8608: URL: https://github.com/apache/arrow/pull/8608#issuecomment-723436980 Ready to merge if the tests pass. I went through all the changes and they seem reasonable. Most issues I had were the same that @alamb suggested, and the rest seems straigh

[GitHub] [arrow] nevi-me closed pull request #8575: ARROW-10476: [Rust] Allow string arrays to be built from Option<&str> or Option

2020-11-07 Thread GitBox
nevi-me closed pull request #8575: URL: https://github.com/apache/arrow/pull/8575 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [arrow] nevi-me closed pull request #8576: ARROW-10477: [Rust] Add iterator support for Binary arrays.

2020-11-07 Thread GitBox
nevi-me closed pull request #8576: URL: https://github.com/apache/arrow/pull/8576 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [arrow] nevi-me commented on a change in pull request #8608: ARROW-10261: [Rust] [Breaking] Change List datatype to Box

2020-11-07 Thread GitBox
nevi-me commented on a change in pull request #8608: URL: https://github.com/apache/arrow/pull/8608#discussion_r519169297 ## File path: rust/arrow/src/ipc/convert.rs ## @@ -537,59 +529,62 @@ pub(crate) fn get_fb_field_type<'a: 'b, 'b>( } } List(re

[GitHub] [arrow] nevi-me commented on a change in pull request #8608: ARROW-10261: [Rust] [Breaking] Change List datatype to Box

2020-11-07 Thread GitBox
nevi-me commented on a change in pull request #8608: URL: https://github.com/apache/arrow/pull/8608#discussion_r519169243 ## File path: rust/arrow/src/datatypes.rs ## @@ -1182,6 +1183,18 @@ impl Field { self.nullable } +/// IReturns the dictionary ID Review

[GitHub] [arrow] nevi-me commented on pull request #8608: ARROW-10261: [Rust] [Breaking] Change List datatype to Box

2020-11-07 Thread GitBox
nevi-me commented on pull request #8608: URL: https://github.com/apache/arrow/pull/8608#issuecomment-723436040 > We can always add such a convenience function later, however. It is definitely not required in this PR Yes @alamb, we can address it as a follow-up. There's more work that

[GitHub] [arrow] alamb commented on pull request #8543: ARROW-9361 [Rust] Move array types into their own modules

2020-11-07 Thread GitBox
alamb commented on pull request #8543: URL: https://github.com/apache/arrow/pull/8543#issuecomment-723435331 I also agree this PR is likely to cause a bunch of conflicts, so the less time it is left outstanding the better. T

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #8608: ARROW-10261: [Rust] [Breaking] Change List datatype to Box

2020-11-07 Thread GitBox
jorgecarleitao commented on a change in pull request #8608: URL: https://github.com/apache/arrow/pull/8608#discussion_r519166944 ## File path: rust/arrow/src/datatypes.rs ## @@ -1182,6 +1183,18 @@ impl Field { self.nullable } +/// IReturns the dictionary ID

[GitHub] [arrow] nevi-me closed pull request #8556: ARROW-10378: [Rust] Update take() kernel with support for LargeList.

2020-11-07 Thread GitBox
nevi-me closed pull request #8556: URL: https://github.com/apache/arrow/pull/8556 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [arrow] nevi-me commented on a change in pull request #8556: ARROW-10378: [Rust] Update take() kernel with support for LargeList.

2020-11-07 Thread GitBox
nevi-me commented on a change in pull request #8556: URL: https://github.com/apache/arrow/pull/8556#discussion_r519166695 ## File path: rust/arrow/src/compute/util.rs ## @@ -100,41 +99,55 @@ pub(super) fn compare_option_bitmap( /// Where a list array has indices `[0,2,5,10]`,

[GitHub] [arrow] alamb commented on a change in pull request #8608: ARROW-10261: [Rust] [Breaking] Change List datatype to Box

2020-11-07 Thread GitBox
alamb commented on a change in pull request #8608: URL: https://github.com/apache/arrow/pull/8608#discussion_r519166183 ## File path: rust/arrow/src/ipc/convert.rs ## @@ -537,59 +529,62 @@ pub(crate) fn get_fb_field_type<'a: 'b, 'b>( } } List(ref

[GitHub] [arrow] github-actions[bot] commented on pull request #8608: ARROW-10261: [Rust] [Breaking] Change List datatype to Box

2020-11-07 Thread GitBox
github-actions[bot] commented on pull request #8608: URL: https://github.com/apache/arrow/pull/8608#issuecomment-723431612 https://issues.apache.org/jira/browse/ARROW-10261 This is an automated message from the Apache Git Ser

[GitHub] [arrow] nevi-me opened a new pull request #8608: ARROW-10261: [Rust] [Breaking] Change List datatype to Box

2020-11-07 Thread GitBox
nevi-me opened a new pull request #8608: URL: https://github.com/apache/arrow/pull/8608 This changes a list datatype to use `Box` instead of `Box`. This change is needed in order to make both Parquet and IPC roundtrips pass. The C++ implementation uses `Field`, as it allows for pre

[GitHub] [arrow] nevi-me commented on pull request #8590: ARROW-10042: [Rust] BufferData's equality should depend on its capacity; ArrayData's equality should not depend on its BufferDatas' capacities

2020-11-07 Thread GitBox
nevi-me commented on pull request #8590: URL: https://github.com/apache/arrow/pull/8590#issuecomment-723429467 > One idea: > > 1. Make the array comparison on the parquet tests be done using the `PartialEq`, like this PR is doing > 2. Make `PartialEq` use the function `equal` bein

[GitHub] [arrow] kiszk commented on a change in pull request #8057: ARROW-9862: [Java] Enable UnsafeDirectLittleEndian on a big-endian platform

2020-11-07 Thread GitBox
kiszk commented on a change in pull request #8057: URL: https://github.com/apache/arrow/pull/8057#discussion_r519156391 ## File path: java/memory/memory-netty/src/main/java/io/netty/buffer/UnsafeDirectLittleEndian.java ## @@ -60,9 +59,6 @@ private UnsafeDirectLittleEndian

[GitHub] [arrow] kiszk commented on a change in pull request #8057: ARROW-9862: [Java] Enable UnsafeDirectLittleEndian on a big-endian platform

2020-11-07 Thread GitBox
kiszk commented on a change in pull request #8057: URL: https://github.com/apache/arrow/pull/8057#discussion_r519152228 ## File path: java/memory/memory-netty/src/main/java/io/netty/buffer/UnsafeDirectLittleEndian.java ## @@ -60,9 +59,6 @@ private UnsafeDirectLittleEndian

[GitHub] [arrow] kiszk commented on a change in pull request #8057: ARROW-9862: [Java] Enable UnsafeDirectLittleEndian on a big-endian platform

2020-11-07 Thread GitBox
kiszk commented on a change in pull request #8057: URL: https://github.com/apache/arrow/pull/8057#discussion_r519152228 ## File path: java/memory/memory-netty/src/main/java/io/netty/buffer/UnsafeDirectLittleEndian.java ## @@ -60,9 +59,6 @@ private UnsafeDirectLittleEndian

[GitHub] [arrow] kiszk commented on a change in pull request #8057: ARROW-9862: [Java] Enable UnsafeDirectLittleEndian on a big-endian platform

2020-11-07 Thread GitBox
kiszk commented on a change in pull request #8057: URL: https://github.com/apache/arrow/pull/8057#discussion_r519152228 ## File path: java/memory/memory-netty/src/main/java/io/netty/buffer/UnsafeDirectLittleEndian.java ## @@ -60,9 +59,6 @@ private UnsafeDirectLittleEndian

[GitHub] [arrow] stefan-lange-dataeng opened a new issue #8607: Deletion of existing file when write_table fails

2020-11-07 Thread GitBox
stefan-lange-dataeng opened a new issue #8607: URL: https://github.com/apache/arrow/issues/8607 https://github.com/apache/arrow/blob/47f2e0cb03ed8ad265e0688ada8162bf46066483/python/pyarrow/parquet.py#L1737 When write_table encounters a problem, the exception handler removes the atte