[GitHub] [arrow] nevi-me commented on pull request #8715: ARROW-10656: [Rust] Remove Field from (most) DataTypes, add nullability

2020-12-16 Thread GitBox
nevi-me commented on pull request #8715: URL: https://github.com/apache/arrow/pull/8715#issuecomment-745870189 Hey @ch-sc, I'm going to experiment with a slightly different approach, and I'll let you know how that goes. I'm aiming to have that done over the weekend. I ended up not getting

[GitHub] [arrow] jorisvandenbossche opened a new pull request #8934: ARROW-10863: [Python] Fix pandas skip in ExtensionArray.to_pandas test

2020-12-16 Thread GitBox
jorisvandenbossche opened a new pull request #8934: URL: https://github.com/apache/arrow/pull/8934 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [arrow] jorisvandenbossche commented on pull request #8934: ARROW-10863: [Python] Fix pandas skip in ExtensionArray.to_pandas test

2020-12-16 Thread GitBox
jorisvandenbossche commented on pull request #8934: URL: https://github.com/apache/arrow/pull/8934#issuecomment-745886207 @github-actions crossbow submit -g integration wheel This is an automated message from the Apache G

[GitHub] [arrow] jorisvandenbossche commented on a change in pull request #8887: ARROW-10863: [Python] Fix ExtensionArray.to_pandas to use underlying storage array

2020-12-16 Thread GitBox
jorisvandenbossche commented on a change in pull request #8887: URL: https://github.com/apache/arrow/pull/8887#discussion_r544093795 ## File path: python/pyarrow/tests/test_pandas.py ## @@ -4001,6 +4011,22 @@ def test_to_pandas_extension_dtypes_mapping(): assert isinstance

[GitHub] [arrow] github-actions[bot] commented on pull request #8934: ARROW-10863: [Python] Fix pandas skip in ExtensionArray.to_pandas test

2020-12-16 Thread GitBox
github-actions[bot] commented on pull request #8934: URL: https://github.com/apache/arrow/pull/8934#issuecomment-745889047 https://issues.apache.org/jira/browse/ARROW-10863 This is an automated message from the Apache Git Ser

[GitHub] [arrow] github-actions[bot] commented on pull request #8934: ARROW-10863: [Python] Fix pandas skip in ExtensionArray.to_pandas test

2020-12-16 Thread GitBox
github-actions[bot] commented on pull request #8934: URL: https://github.com/apache/arrow/pull/8934#issuecomment-745891443 Revision: 67851177ddb840e16b60477be00ccf01184d0080 Submitted crossbow builds: [ursa-labs/crossbow @ actions-798](https://github.com/ursa-labs/crossbow/branches/a

[GitHub] [arrow] jorisvandenbossche commented on pull request #8934: ARROW-10863: [Python] Fix pandas skip in ExtensionArray.to_pandas test

2020-12-16 Thread GitBox
jorisvandenbossche commented on pull request #8934: URL: https://github.com/apache/arrow/pull/8934#issuecomment-745915972 @github-actions crossbow submit -g wheel This is an automated message from the Apache Git Service. To r

[GitHub] [arrow] github-actions[bot] commented on pull request #8934: ARROW-10863: [Python] Fix pandas skip in ExtensionArray.to_pandas test

2020-12-16 Thread GitBox
github-actions[bot] commented on pull request #8934: URL: https://github.com/apache/arrow/pull/8934#issuecomment-745918495 Revision: 67851177ddb840e16b60477be00ccf01184d0080 Submitted crossbow builds: [ursa-labs/crossbow @ actions-799](https://github.com/ursa-labs/crossbow/branches/a

[GitHub] [arrow] jorisvandenbossche opened a new pull request #8935: ARROW-10934: [Python] Skip filesystem tests for in-memory fs for fsspec 0.8.5

2020-12-16 Thread GitBox
jorisvandenbossche opened a new pull request #8935: URL: https://github.com/apache/arrow/pull/8935 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [arrow] github-actions[bot] commented on pull request #8935: ARROW-10934: [Python] Skip filesystem tests for in-memory fs for fsspec 0.8.5

2020-12-16 Thread GitBox
github-actions[bot] commented on pull request #8935: URL: https://github.com/apache/arrow/pull/8935#issuecomment-745993339 https://issues.apache.org/jira/browse/ARROW-10934 This is an automated message from the Apache Git Ser

[GitHub] [arrow] pitrou commented on a change in pull request #7179: ARROW-8732: [C++] Add basic cancellation API

2020-12-16 Thread GitBox
pitrou commented on a change in pull request #7179: URL: https://github.com/apache/arrow/pull/7179#discussion_r544166587 ## File path: cpp/src/arrow/util/thread_pool.h ## @@ -218,7 +220,8 @@ class ARROW_EXPORT ThreadPool : public Executor { ThreadPool(); - Status SpawnR

[GitHub] [arrow] pitrou commented on pull request #7179: ARROW-8732: [C++] Add basic cancellation API

2020-12-16 Thread GitBox
pitrou commented on pull request #7179: URL: https://github.com/apache/arrow/pull/7179#issuecomment-746019377 I'll probably try to submit another PR with the "unified" model (i.e. consumer-issued stop token), though that may be after my end-of-year vacation. --

[GitHub] [arrow] pitrou closed pull request #8935: ARROW-10934: [Python] Skip filesystem tests for in-memory fs for fsspec 0.8.5

2020-12-16 Thread GitBox
pitrou closed pull request #8935: URL: https://github.com/apache/arrow/pull/8935 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] pitrou commented on pull request #8933: ARROW-10932: [C++] BinaryMemoTable::CopyOffsets access out-of-bound address when data is empty

2020-12-16 Thread GitBox
pitrou commented on pull request #8933: URL: https://github.com/apache/arrow/pull/8933#issuecomment-746033058 The CI failures are unrelated, will merge. This is an automated message from the Apache Git Service. To respond to

[GitHub] [arrow] pitrou closed pull request #8933: ARROW-10932: [C++] BinaryMemoTable::CopyOffsets access out-of-bound address when data is empty

2020-12-16 Thread GitBox
pitrou closed pull request #8933: URL: https://github.com/apache/arrow/pull/8933 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] pitrou commented on a change in pull request #8914: ARROW-10887: [Doc][C++] Document C++ IPC API

2020-12-16 Thread GitBox
pitrou commented on a change in pull request #8914: URL: https://github.com/apache/arrow/pull/8914#discussion_r544175713 ## File path: docs/source/cpp/api/ipc.rst ## @@ -0,0 +1,91 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license

[GitHub] [arrow] jhorstmann commented on pull request #8929: ARROW-10914: [Rust] Refactor simd arithmetic kernels to use chunked iteration

2020-12-16 Thread GitBox
jhorstmann commented on pull request #8929: URL: https://github.com/apache/arrow/pull/8929#issuecomment-746051869 I'm seeing bigger variations on the benchmark numbers between runs, the results below between current master and this pr are from one of the faster runs: ``` ~/arrow/

[GitHub] [arrow] pitrou commented on a change in pull request #8928: ARROW-10918: [Doc][C++] Document supported Parquet features

2020-12-16 Thread GitBox
pitrou commented on a change in pull request #8928: URL: https://github.com/apache/arrow/pull/8928#discussion_r544184629 ## File path: docs/source/cpp/parquet.rst ## @@ -27,15 +27,207 @@ Reading and writing Parquet files .. seealso:: :ref:`Parquet reader and writer API ref

[GitHub] [arrow] jorisvandenbossche commented on pull request #8934: ARROW-10863: [Python] Fix pandas skip in ExtensionArray.to_pandas test

2020-12-16 Thread GitBox
jorisvandenbossche commented on pull request #8934: URL: https://github.com/apache/arrow/pull/8934#issuecomment-746178196 @github-actions crossbow submit -g integration This is an automated message from the Apache Git Service

[GitHub] [arrow] github-actions[bot] commented on pull request #8934: ARROW-10863: [Python] Fix pandas skip in ExtensionArray.to_pandas test

2020-12-16 Thread GitBox
github-actions[bot] commented on pull request #8934: URL: https://github.com/apache/arrow/pull/8934#issuecomment-746187610 Revision: c83bdd9a179f9e6096fe4419ca87d3e220da8acd Submitted crossbow builds: [ursa-labs/crossbow @ actions-800](https://github.com/ursa-labs/crossbow/branches/a

[GitHub] [arrow] alamb commented on pull request #8930: ARROW-10929: [Rust] Change CI to use Stable Rust

2020-12-16 Thread GitBox
alamb commented on pull request #8930: URL: https://github.com/apache/arrow/pull/8930#issuecomment-746218093 Hmm, I ran everything under valgrind locally and it did not detect any errors and the test has passed on windows a subsequent run. However, now I am getting a clippy error. I will

[GitHub] [arrow] pitrou closed pull request #8914: ARROW-10887: [Doc][C++] Document C++ IPC API

2020-12-16 Thread GitBox
pitrou closed pull request #8914: URL: https://github.com/apache/arrow/pull/8914 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] mqy opened a new pull request #8936: ARROW-10938: [Rust] upgrade dependency "flatbuffers" to 0.8

2020-12-16 Thread GitBox
mqy opened a new pull request #8936: URL: https://github.com/apache/arrow/pull/8936 [flatbuffers](https://crates.io/crates/flatbuffers) 0.8.0 was released on Dec 10, 2020, with some notable changes: - new verifier - common rust traits to FlatBufferBuilder - new VectorIter -

[GitHub] [arrow] github-actions[bot] commented on pull request #8936: ARROW-10938: [Rust] upgrade dependency "flatbuffers" to 0.8

2020-12-16 Thread GitBox
github-actions[bot] commented on pull request #8936: URL: https://github.com/apache/arrow/pull/8936#issuecomment-746273042 https://issues.apache.org/jira/browse/ARROW-10938 This is an automated message from the Apache Git Ser

[GitHub] [arrow] sweb commented on pull request #8880: ARROW-10927: [Rust][Parquet] Add Decimal to ArrayBuilderReader

2020-12-16 Thread GitBox
sweb commented on pull request #8880: URL: https://github.com/apache/arrow/pull/8880#issuecomment-746306300 Interesting - now I need to find out why we are writing files in legacy mode ;) This is an automated message from th

[GitHub] [arrow] jorgecarleitao commented on pull request #8864: ARROW-10838: [Rust] [CI] Add arrow build targeting wasm32

2020-12-16 Thread GitBox
jorgecarleitao commented on pull request #8864: URL: https://github.com/apache/arrow/pull/8864#issuecomment-746329723 FWIW, I do not think we should merge this if there are no plans to merge #7767. This is an automated messa

[GitHub] [arrow] nevi-me commented on pull request #8864: ARROW-10838: [Rust] [CI] Add arrow build targeting wasm32

2020-12-16 Thread GitBox
nevi-me commented on pull request #8864: URL: https://github.com/apache/arrow/pull/8864#issuecomment-746332585 I think we should merge this given that we don't know when the PR with tests will be ready. This is an automated

[GitHub] [arrow] waynexia commented on pull request #8856: ARROW-10940: [Rust] Extend sort kernel to ListArray

2020-12-16 Thread GitBox
waynexia commented on pull request #8856: URL: https://github.com/apache/arrow/pull/8856#issuecomment-746365537 Okay, thanks for your help. I filed ARROW-10940 for this PR. There are some works to do like support fixed-size list and i64 indices type. I'll finish it in a few days :)

[GitHub] [arrow] pitrou commented on pull request #8934: ARROW-10863: [Python] Fix pandas skip in ExtensionArray.to_pandas test

2020-12-16 Thread GitBox
pitrou commented on pull request #8934: URL: https://github.com/apache/arrow/pull/8934#issuecomment-746379253 @jorisvandenbossche Are you happy with the build results? If so, will merge. This is an automated message from the

[GitHub] [arrow] pitrou commented on a change in pull request #8805: ARROW-10725: [Python][Compute] Expose sort options in Python bindings

2020-12-16 Thread GitBox
pitrou commented on a change in pull request #8805: URL: https://github.com/apache/arrow/pull/8805#discussion_r544344212 ## File path: python/pyarrow/tests/test_compute.py ## @@ -1017,3 +1017,51 @@ def test_partition_nth(): for i in range(pivot)) assert all

[GitHub] [arrow] pitrou closed pull request #8928: ARROW-10918: [Doc][C++] Document supported Parquet features

2020-12-16 Thread GitBox
pitrou closed pull request #8928: URL: https://github.com/apache/arrow/pull/8928 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] rdettai commented on a change in pull request #8917: ARROW-9828: [Rust] [DataFusion] Support filter pushdown optimisation for TableProvider implementations

2020-12-16 Thread GitBox
rdettai commented on a change in pull request #8917: URL: https://github.com/apache/arrow/pull/8917#discussion_r544337093 ## File path: rust/datafusion/src/datasource/datasource.rs ## @@ -48,9 +66,19 @@ pub trait TableProvider { &self, projection: &Option>,

[GitHub] [arrow] rdettai commented on a change in pull request #8917: ARROW-9828: [Rust] [DataFusion] Support filter pushdown optimisation for TableProvider implementations

2020-12-16 Thread GitBox
rdettai commented on a change in pull request #8917: URL: https://github.com/apache/arrow/pull/8917#discussion_r544353356 ## File path: rust/datafusion/src/datasource/datasource.rs ## @@ -48,9 +66,19 @@ pub trait TableProvider { &self, projection: &Option>,

[GitHub] [arrow] rdettai commented on a change in pull request #8917: ARROW-9828: [Rust] [DataFusion] Support filter pushdown optimisation for TableProvider implementations

2020-12-16 Thread GitBox
rdettai commented on a change in pull request #8917: URL: https://github.com/apache/arrow/pull/8917#discussion_r544353356 ## File path: rust/datafusion/src/datasource/datasource.rs ## @@ -48,9 +66,19 @@ pub trait TableProvider { &self, projection: &Option>,

[GitHub] [arrow] rdettai commented on a change in pull request #8917: ARROW-9828: [Rust] [DataFusion] Support filter pushdown optimisation for TableProvider implementations

2020-12-16 Thread GitBox
rdettai commented on a change in pull request #8917: URL: https://github.com/apache/arrow/pull/8917#discussion_r544355966 ## File path: rust/datafusion/src/datasource/parquet.rs ## @@ -65,6 +66,7 @@ impl TableProvider for ParquetTable { &self, projection: &Opt

[GitHub] [arrow] github-actions[bot] commented on pull request #8856: ARROW-10940: [Rust] Extend sort kernel to ListArray

2020-12-16 Thread GitBox
github-actions[bot] commented on pull request #8856: URL: https://github.com/apache/arrow/pull/8856#issuecomment-746412925 https://issues.apache.org/jira/browse/ARROW-10940 This is an automated message from the Apache Git Ser

[GitHub] [arrow] andygrove commented on a change in pull request #8936: ARROW-10938: [Rust] upgrade dependency "flatbuffers" to 0.8

2020-12-16 Thread GitBox
andygrove commented on a change in pull request #8936: URL: https://github.com/apache/arrow/pull/8936#discussion_r544366618 ## File path: rust/arrow/src/ipc/convert.rs ## @@ -124,8 +124,11 @@ pub fn fb_to_schema(fb: ipc::Schema) -> Schema { /// Deserialize an IPC message int

[GitHub] [arrow] rdettai commented on pull request #8917: ARROW-9828: [Rust] [DataFusion] Support filter pushdown optimisation for TableProvider implementations

2020-12-16 Thread GitBox
rdettai commented on pull request #8917: URL: https://github.com/apache/arrow/pull/8917#issuecomment-746426491 I am very happy with the general heading of this PR. Thanks for your great work @returnString 😄 . I can't shake the filling that we are adding too much complexity to the `

[GitHub] [arrow] andygrove commented on a change in pull request #8936: ARROW-10938: [Rust] upgrade dependency "flatbuffers" to 0.8

2020-12-16 Thread GitBox
andygrove commented on a change in pull request #8936: URL: https://github.com/apache/arrow/pull/8936#discussion_r544367525 ## File path: rust/parquet/src/arrow/schema.rs ## @@ -184,10 +184,13 @@ fn get_arrow_schema_from_metadata(encoded_meta: &str) -> Option { }

[GitHub] [arrow] andygrove commented on a change in pull request #8936: ARROW-10938: [Rust] upgrade dependency "flatbuffers" to 0.8

2020-12-16 Thread GitBox
andygrove commented on a change in pull request #8936: URL: https://github.com/apache/arrow/pull/8936#discussion_r544368023 ## File path: rust/arrow-flight/src/utils.rs ## @@ -113,7 +113,13 @@ pub fn flight_data_to_arrow_batch( schema: SchemaRef, ) -> Option> { // ch

[GitHub] [arrow] andygrove commented on pull request #8936: ARROW-10938: [Rust] upgrade dependency "flatbuffers" to 0.8

2020-12-16 Thread GitBox
andygrove commented on pull request #8936: URL: https://github.com/apache/arrow/pull/8936#issuecomment-746430232 Thanks @mqy this is looking good and I appreciate you breaking this down into the individual commits. I have some questions on error handling but other than that it LGTM.

[GitHub] [arrow] andygrove commented on a change in pull request #8931: ARROW-10933: [Rust] Update readme files in regard to nightly rust

2020-12-16 Thread GitBox
andygrove commented on a change in pull request #8931: URL: https://github.com/apache/arrow/pull/8931#discussion_r544369209 ## File path: rust/README.md ## @@ -111,10 +111,8 @@ from here on, this is a pure Rust project and `cargo` can be used to run tests, ## Code Formattin

[GitHub] [arrow] andygrove commented on a change in pull request #8931: ARROW-10933: [Rust] Update readme files in regard to nightly rust

2020-12-16 Thread GitBox
andygrove commented on a change in pull request #8931: URL: https://github.com/apache/arrow/pull/8931#discussion_r544369896 ## File path: rust/README.md ## @@ -111,10 +111,8 @@ from here on, this is a pure Rust project and `cargo` can be used to run tests, ## Code Formattin

[GitHub] [arrow] Yuhta commented on a change in pull request #8933: ARROW-10932: [C++] BinaryMemoTable::CopyOffsets access out-of-bound address when data is empty

2020-12-16 Thread GitBox
Yuhta commented on a change in pull request #8933: URL: https://github.com/apache/arrow/pull/8933#discussion_r544379156 ## File path: cpp/src/arrow/util/hashing.h ## @@ -690,7 +690,8 @@ class BinaryMemoTable : public MemoTable { DCHECK_LE(start, size()); const build

[GitHub] [arrow] pitrou commented on a change in pull request #8933: ARROW-10932: [C++] BinaryMemoTable::CopyOffsets access out-of-bound address when data is empty

2020-12-16 Thread GitBox
pitrou commented on a change in pull request #8933: URL: https://github.com/apache/arrow/pull/8933#discussion_r544382794 ## File path: cpp/src/arrow/util/hashing.h ## @@ -690,7 +690,8 @@ class BinaryMemoTable : public MemoTable { DCHECK_LE(start, size()); const buil

[GitHub] [arrow] Yuhta opened a new pull request #8937: Arrow 10932

2020-12-16 Thread GitBox
Yuhta opened a new pull request #8937: URL: https://github.com/apache/arrow/pull/8937 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [arrow] andygrove commented on pull request #8760: ARROW-10712: [Rust] Add tests to TPC-H benchmarks

2020-12-16 Thread GitBox
andygrove commented on pull request #8760: URL: https://github.com/apache/arrow/pull/8760#issuecomment-746470112 Thanks @seddonm1 I like this approach. We have a separate repo (arrow-testing) that is a git submodule of the main arrow repo where we check in files like this. There is sometim

[GitHub] [arrow] andygrove closed pull request #8931: ARROW-10933: [Rust] Update readme files in regard to nightly rust

2020-12-16 Thread GitBox
andygrove closed pull request #8931: URL: https://github.com/apache/arrow/pull/8931 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] andygrove closed pull request #8924: ARROW-10915: [Rust] README.md: set the Env vars as absolute dirs; several minor fixes.

2020-12-16 Thread GitBox
andygrove closed pull request #8924: URL: https://github.com/apache/arrow/pull/8924 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] andygrove closed pull request #8880: ARROW-10927: [Rust][Parquet] Add Decimal to ArrayBuilderReader

2020-12-16 Thread GitBox
andygrove closed pull request #8880: URL: https://github.com/apache/arrow/pull/8880 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] mqy commented on pull request #8936: ARROW-10938: [Rust] upgrade dependency "flatbuffers" to 0.8

2020-12-16 Thread GitBox
mqy commented on pull request #8936: URL: https://github.com/apache/arrow/pull/8936#issuecomment-746495690 > Thanks @mqy this is looking good and I appreciate you breaking this down into the individual commits. I have some questions on error handling but other than that it LGTM. @an

[GitHub] [arrow] andygrove commented on pull request #8880: ARROW-10927: [Rust][Parquet] Add Decimal to ArrayBuilderReader

2020-12-16 Thread GitBox
andygrove commented on pull request #8880: URL: https://github.com/apache/arrow/pull/8880#issuecomment-746502221 Thanks @sweb. I would like to assign the JIRA to you but need to add you to the contributor role there first but I'm there are multiple JIRA accounts with your name. Could you l

[GitHub] [arrow] github-actions[bot] commented on pull request #8937: ARROW-10932: [C++] BinaryMemoTable::CopyOffsets access out-of-bound address when data is empty

2020-12-16 Thread GitBox
github-actions[bot] commented on pull request #8937: URL: https://github.com/apache/arrow/pull/8937#issuecomment-746531687 https://issues.apache.org/jira/browse/ARROW-10932 This is an automated message from the Apache Git Ser

[GitHub] [arrow] jorisvandenbossche commented on a change in pull request #8805: ARROW-10725: [Python][Compute] Expose sort options in Python bindings

2020-12-16 Thread GitBox
jorisvandenbossche commented on a change in pull request #8805: URL: https://github.com/apache/arrow/pull/8805#discussion_r544417190 ## File path: python/pyarrow/tests/test_compute.py ## @@ -1017,3 +1017,51 @@ def test_partition_nth(): for i in range(pivot))

[GitHub] [arrow] jorisvandenbossche commented on a change in pull request #8805: ARROW-10725: [Python][Compute] Expose sort options in Python bindings

2020-12-16 Thread GitBox
jorisvandenbossche commented on a change in pull request #8805: URL: https://github.com/apache/arrow/pull/8805#discussion_r544417190 ## File path: python/pyarrow/tests/test_compute.py ## @@ -1017,3 +1017,51 @@ def test_partition_nth(): for i in range(pivot))

[GitHub] [arrow] sweb commented on pull request #8880: ARROW-10927: [Rust][Parquet] Add Decimal to ArrayBuilderReader

2020-12-16 Thread GitBox
sweb commented on pull request #8880: URL: https://github.com/apache/arrow/pull/8880#issuecomment-746549299 Thanks @andygrove. `florian.mueller` is the correct one! This is an automated message from the Apache Git Service. To

[GitHub] [arrow] jonkeane commented on pull request #8932: ARROW-10856: [R] CC and CXX environment variables passing to cmake

2020-12-16 Thread GitBox
jonkeane commented on pull request #8932: URL: https://github.com/apache/arrow/pull/8932#issuecomment-746551513 @github-actions crossbow submit -g r This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] nevi-me opened a new pull request #8938: ARROW-10770: [Rust] Json nested list reader

2020-12-16 Thread GitBox
nevi-me opened a new pull request #8938: URL: https://github.com/apache/arrow/pull/8938 Big one! This implements a JSON nested list reader, which means that we can now read `>>` and other variants. While working on this, I noticed some bugs in the reader, which I fixed. They were

[GitHub] [arrow] nevi-me commented on pull request #8938: ARROW-10770: [Rust] Json nested list reader

2020-12-16 Thread GitBox
nevi-me commented on pull request #8938: URL: https://github.com/apache/arrow/pull/8938#issuecomment-746579024 CC @houqp This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [arrow] nevi-me edited a comment on pull request #8938: ARROW-10770: [Rust] Json nested list reader

2020-12-16 Thread GitBox
nevi-me edited a comment on pull request #8938: URL: https://github.com/apache/arrow/pull/8938#issuecomment-746579024 CC @houqp @vertexclique (I didn't expand on the dictionary reader, but the first bug mentioned might be relevant to your work)

[GitHub] [arrow] pitrou opened a new pull request #8939: ARROW-10928: [C++] Better Parquet error when trying to write empty struct

2020-12-16 Thread GitBox
pitrou opened a new pull request #8939: URL: https://github.com/apache/arrow/pull/8939 An empty struct type (with no child fields) is not easy to write in Parquet, since Parquet only represents the data of leaf nodes. We would need a way to distinguish between null and non-null (empty)

[GitHub] [arrow] pitrou commented on a change in pull request #8937: ARROW-10932: [C++] BinaryMemoTable::CopyOffsets access out-of-bound address when data is empty

2020-12-16 Thread GitBox
pitrou commented on a change in pull request #8937: URL: https://github.com/apache/arrow/pull/8937#discussion_r56692 ## File path: cpp/src/arrow/util/hashing.h ## @@ -691,7 +691,7 @@ class BinaryMemoTable : public MemoTable { const builder_offset_type* offsets = bina

[GitHub] [arrow] github-actions[bot] commented on pull request #8939: ARROW-10928: [C++] Better Parquet error when trying to write empty struct

2020-12-16 Thread GitBox
github-actions[bot] commented on pull request #8939: URL: https://github.com/apache/arrow/pull/8939#issuecomment-746600174 https://issues.apache.org/jira/browse/ARROW-10928 This is an automated message from the Apache Git Ser

[GitHub] [arrow] github-actions[bot] commented on pull request #8938: ARROW-10770: [Rust] JSON nested list reader

2020-12-16 Thread GitBox
github-actions[bot] commented on pull request #8938: URL: https://github.com/apache/arrow/pull/8938#issuecomment-746600179 https://issues.apache.org/jira/browse/ARROW-10770 This is an automated message from the Apache Git Ser

[GitHub] [arrow] nevi-me commented on pull request #8880: ARROW-10927: [Rust][Parquet] Add Decimal to ArrayBuilderReader

2020-12-16 Thread GitBox
nevi-me commented on pull request #8880: URL: https://github.com/apache/arrow/pull/8880#issuecomment-746604355 @andygrove @sweb we've got CI failures, I suspect we merged too many changes without rebasing. This is an automat

[GitHub] [arrow] andygrove commented on pull request #8880: ARROW-10927: [Rust][Parquet] Add Decimal to ArrayBuilderReader

2020-12-16 Thread GitBox
andygrove commented on pull request #8880: URL: https://github.com/apache/arrow/pull/8880#issuecomment-746609688 Thanks @nevi-me I am looking now This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [arrow] andygrove commented on pull request #8880: ARROW-10927: [Rust][Parquet] Add Decimal to ArrayBuilderReader

2020-12-16 Thread GitBox
andygrove commented on pull request #8880: URL: https://github.com/apache/arrow/pull/8880#issuecomment-746610857 I will revert this PR for now This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [arrow] andygrove opened a new pull request #8940: ARROW-10927: [Rust][Parquet] Add Decimal to ArrayBuilderReader [REVERT]

2020-12-16 Thread GitBox
andygrove opened a new pull request #8940: URL: https://github.com/apache/arrow/pull/8940 This reverts commit 06ac7509adb14f3109e1089221fd5cb9a28c8519 which caused build failures and will need rebasing. This is an automated

[GitHub] [arrow] github-actions[bot] commented on pull request #8932: ARROW-10856: [R] CC and CXX environment variables passing to cmake

2020-12-16 Thread GitBox
github-actions[bot] commented on pull request #8932: URL: https://github.com/apache/arrow/pull/8932#issuecomment-746615281 Revision: 5690dc804bf3f3e90ce2a1efa32268807d1dc1ad Submitted crossbow builds: [ursa-labs/crossbow @ actions-801](https://github.com/ursa-labs/crossbow/branches/a

[GitHub] [arrow] returnString commented on a change in pull request #8917: ARROW-9828: [Rust] [DataFusion] Support filter pushdown optimisation for TableProvider implementations

2020-12-16 Thread GitBox
returnString commented on a change in pull request #8917: URL: https://github.com/apache/arrow/pull/8917#discussion_r544457191 ## File path: rust/datafusion/src/datasource/datasource.rs ## @@ -48,9 +66,19 @@ pub trait TableProvider { &self, projection: &Option

[GitHub] [arrow] nealrichardson commented on a change in pull request #8932: ARROW-10856: [R] CC and CXX environment variables passing to cmake

2020-12-16 Thread GitBox
nealrichardson commented on a change in pull request #8932: URL: https://github.com/apache/arrow/pull/8932#discussion_r544456115 ## File path: .env ## @@ -47,3 +47,4 @@ ARROW_R_DEV=TRUE R_ORG=rhub R_IMAGE=ubuntu-gcc-release R_TAG=latest +DEVTOOLSET_VERSION=-1 Review comment

[GitHub] [arrow] returnString commented on a change in pull request #8917: ARROW-9828: [Rust] [DataFusion] Support filter pushdown optimisation for TableProvider implementations

2020-12-16 Thread GitBox
returnString commented on a change in pull request #8917: URL: https://github.com/apache/arrow/pull/8917#discussion_r544457191 ## File path: rust/datafusion/src/datasource/datasource.rs ## @@ -48,9 +66,19 @@ pub trait TableProvider { &self, projection: &Option

[GitHub] [arrow] nevi-me commented on pull request #8880: ARROW-10927: [Rust][Parquet] Add Decimal to ArrayBuilderReader

2020-12-16 Thread GitBox
nevi-me commented on pull request #8880: URL: https://github.com/apache/arrow/pull/8880#issuecomment-746618724 The reason of the failure is not obvious to me. @sweb we'll need to rebase changes after we reopen this PR, so we can see what the issue is. -

[GitHub] [arrow] returnString commented on a change in pull request #8917: ARROW-9828: [Rust] [DataFusion] Support filter pushdown optimisation for TableProvider implementations

2020-12-16 Thread GitBox
returnString commented on a change in pull request #8917: URL: https://github.com/apache/arrow/pull/8917#discussion_r544460440 ## File path: rust/datafusion/src/datasource/datasource.rs ## @@ -34,6 +35,23 @@ pub struct Statistics { pub total_byte_size: Option, } +/// In

[GitHub] [arrow] nealrichardson commented on a change in pull request #8932: ARROW-10856: [R] CC and CXX environment variables passing to cmake

2020-12-16 Thread GitBox
nealrichardson commented on a change in pull request #8932: URL: https://github.com/apache/arrow/pull/8932#discussion_r544455545 ## File path: ci/scripts/r_docker_configure.sh ## @@ -39,6 +39,16 @@ if [ "$RHUB_PLATFORM" = "linux-x86_64-fedora-clang" ]; then rm -rf $(${R_BIN}

[GitHub] [arrow] returnString commented on pull request #8917: ARROW-9828: [Rust] [DataFusion] Support filter pushdown optimisation for TableProvider implementations

2020-12-16 Thread GitBox
returnString commented on pull request #8917: URL: https://github.com/apache/arrow/pull/8917#issuecomment-746628943 Rebase is done, plus I've augmented the test coverage to also use the SQL code paths now that providers are resolved eagerly (thanks @rdettai 🥇)

[GitHub] [arrow] jonkeane commented on pull request #8932: ARROW-10856: [R] CC and CXX environment variables passing to cmake

2020-12-16 Thread GitBox
jonkeane commented on pull request #8932: URL: https://github.com/apache/arrow/pull/8932#issuecomment-746631619 @github-actions crossbow submit -g r This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] emkornfield commented on a change in pull request #8939: ARROW-10928: [C++] Better Parquet error when trying to write empty struct

2020-12-16 Thread GitBox
emkornfield commented on a change in pull request #8939: URL: https://github.com/apache/arrow/pull/8939#discussion_r544474457 ## File path: cpp/src/parquet/arrow/schema.cc ## @@ -113,12 +114,21 @@ Status StructToNode(const std::shared_ptr<::arrow::StructType>& type,

[GitHub] [arrow] emkornfield commented on a change in pull request #8939: ARROW-10928: [C++] Better Parquet error when trying to write empty struct

2020-12-16 Thread GitBox
emkornfield commented on a change in pull request #8939: URL: https://github.com/apache/arrow/pull/8939#discussion_r544474953 ## File path: cpp/src/parquet/schema.cc ## @@ -550,11 +550,11 @@ std::unique_ptr Unflatten(const format::SchemaElement* elements, int lengt int fi

[GitHub] [arrow] andygrove commented on pull request #8930: ARROW-10929: [Rust] Change CI to use Stable Rust

2020-12-16 Thread GitBox
andygrove commented on pull request #8930: URL: https://github.com/apache/arrow/pull/8930#issuecomment-746661642 > thinking the windows CI build is failing when using stable: > > ``` > > failures: > > encodings::encoding::tests::test_bool stdout > thread 'en

[GitHub] [arrow] pitrou commented on a change in pull request #8939: ARROW-10928: [C++] Better Parquet error when trying to write empty struct

2020-12-16 Thread GitBox
pitrou commented on a change in pull request #8939: URL: https://github.com/apache/arrow/pull/8939#discussion_r544488560 ## File path: cpp/src/parquet/arrow/schema.cc ## @@ -113,12 +114,21 @@ Status StructToNode(const std::shared_ptr<::arrow::StructType>& type,

[GitHub] [arrow] pitrou commented on a change in pull request #8939: ARROW-10928: [C++] Better Parquet error when trying to write empty struct

2020-12-16 Thread GitBox
pitrou commented on a change in pull request #8939: URL: https://github.com/apache/arrow/pull/8939#discussion_r544488947 ## File path: cpp/src/parquet/schema.cc ## @@ -550,11 +550,11 @@ std::unique_ptr Unflatten(const format::SchemaElement* elements, int lengt int field_i

[GitHub] [arrow] pitrou commented on a change in pull request #8939: ARROW-10928: [C++] Better Parquet error when trying to write empty struct

2020-12-16 Thread GitBox
pitrou commented on a change in pull request #8939: URL: https://github.com/apache/arrow/pull/8939#discussion_r544489391 ## File path: cpp/src/parquet/schema.cc ## @@ -550,11 +550,11 @@ std::unique_ptr Unflatten(const format::SchemaElement* elements, int lengt int field_i

[GitHub] [arrow] pitrou opened a new pull request #8941: ARROW-10942: [C++] Fix S3FileSystem::Impl::IsEmptyDirectory on Amazon

2020-12-16 Thread GitBox
pitrou opened a new pull request #8941: URL: https://github.com/apache/arrow/pull/8941 Amazon and Minio S3 implementations unfortunately have slightly different conventions for recognizing and handling empty "directories". T

[GitHub] [arrow] sweb commented on pull request #8880: ARROW-10927: [Rust][Parquet] Add Decimal to ArrayBuilderReader

2020-12-16 Thread GitBox
sweb commented on pull request #8880: URL: https://github.com/apache/arrow/pull/8880#issuecomment-746697508 @nevi-me @andygrove I will take a look at it and try to resolve it once master is stable and this PR is reopened. Th

[GitHub] [arrow] Yuhta commented on a change in pull request #8937: ARROW-10932: [C++] BinaryMemoTable::CopyOffsets access out-of-bound address when data is empty

2020-12-16 Thread GitBox
Yuhta commented on a change in pull request #8937: URL: https://github.com/apache/arrow/pull/8937#discussion_r544500867 ## File path: cpp/src/arrow/util/hashing.h ## @@ -691,7 +691,7 @@ class BinaryMemoTable : public MemoTable { const builder_offset_type* offsets = binar

[GitHub] [arrow] github-actions[bot] commented on pull request #8940: ARROW-10927: [Rust][Parquet] Add Decimal to ArrayBuilderReader [REVERT]

2020-12-16 Thread GitBox
github-actions[bot] commented on pull request #8940: URL: https://github.com/apache/arrow/pull/8940#issuecomment-746721299 https://issues.apache.org/jira/browse/ARROW-10927 This is an automated message from the Apache Git Ser

[GitHub] [arrow] github-actions[bot] commented on pull request #8941: ARROW-10942: [C++] Fix S3FileSystem::Impl::IsEmptyDirectory on Amazon

2020-12-16 Thread GitBox
github-actions[bot] commented on pull request #8941: URL: https://github.com/apache/arrow/pull/8941#issuecomment-746721298 https://issues.apache.org/jira/browse/ARROW-10942 This is an automated message from the Apache Git Ser

[GitHub] [arrow] pitrou commented on a change in pull request #8937: ARROW-10932: [C++] BinaryMemoTable::CopyOffsets access out-of-bound address when data is empty

2020-12-16 Thread GitBox
pitrou commented on a change in pull request #8937: URL: https://github.com/apache/arrow/pull/8937#discussion_r544519106 ## File path: cpp/src/arrow/util/hashing.h ## @@ -691,7 +691,7 @@ class BinaryMemoTable : public MemoTable { const builder_offset_type* offsets = bina

[GitHub] [arrow] github-actions[bot] commented on pull request #8932: ARROW-10856: [R] CC and CXX environment variables passing to cmake

2020-12-16 Thread GitBox
github-actions[bot] commented on pull request #8932: URL: https://github.com/apache/arrow/pull/8932#issuecomment-746764124 Revision: 3711544fb1a2606741747affca51013e2cf4d22e Submitted crossbow builds: [ursa-labs/crossbow @ actions-802](https://github.com/ursa-labs/crossbow/branches/a

[GitHub] [arrow] pitrou commented on pull request #8937: ARROW-10932: [C++] BinaryMemoTable::CopyOffsets access out-of-bound address when data is empty

2020-12-16 Thread GitBox
pitrou commented on pull request #8937: URL: https://github.com/apache/arrow/pull/8937#issuecomment-746770741 Previous PR: #8933 This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [arrow] Yuhta commented on a change in pull request #8937: ARROW-10932: [C++] BinaryMemoTable::CopyOffsets access out-of-bound address when data is empty

2020-12-16 Thread GitBox
Yuhta commented on a change in pull request #8937: URL: https://github.com/apache/arrow/pull/8937#discussion_r544522474 ## File path: cpp/src/arrow/util/hashing.h ## @@ -691,7 +691,7 @@ class BinaryMemoTable : public MemoTable { const builder_offset_type* offsets = binar

[GitHub] [arrow] pitrou commented on a change in pull request #8937: ARROW-10932: [C++] BinaryMemoTable::CopyOffsets access out-of-bound address when data is empty

2020-12-16 Thread GitBox
pitrou commented on a change in pull request #8937: URL: https://github.com/apache/arrow/pull/8937#discussion_r544523627 ## File path: cpp/src/arrow/util/hashing.h ## @@ -691,7 +691,7 @@ class BinaryMemoTable : public MemoTable { const builder_offset_type* offsets = bina

[GitHub] [arrow] juanjgalvez commented on pull request #8941: ARROW-10942: [C++] Fix S3FileSystem::Impl::IsEmptyDirectory on Amazon

2020-12-16 Thread GitBox
juanjgalvez commented on pull request #8941: URL: https://github.com/apache/arrow/pull/8941#issuecomment-746786024 Btw, tested it and it works. This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [arrow] Yuhta commented on a change in pull request #8937: ARROW-10932: [C++] BinaryMemoTable::CopyOffsets access out-of-bound address when data is empty

2020-12-16 Thread GitBox
Yuhta commented on a change in pull request #8937: URL: https://github.com/apache/arrow/pull/8937#discussion_r544534942 ## File path: cpp/src/arrow/util/hashing.h ## @@ -691,7 +691,7 @@ class BinaryMemoTable : public MemoTable { const builder_offset_type* offsets = binar

[GitHub] [arrow] pitrou commented on a change in pull request #8937: ARROW-10932: [C++] BinaryMemoTable::CopyOffsets access out-of-bound address when data is empty

2020-12-16 Thread GitBox
pitrou commented on a change in pull request #8937: URL: https://github.com/apache/arrow/pull/8937#discussion_r544537227 ## File path: cpp/src/arrow/util/hashing.h ## @@ -691,7 +691,7 @@ class BinaryMemoTable : public MemoTable { const builder_offset_type* offsets = bina

[GitHub] [arrow] rdettai edited a comment on pull request #8917: ARROW-9828: [Rust] [DataFusion] Support filter pushdown optimisation for TableProvider implementations

2020-12-16 Thread GitBox
rdettai edited a comment on pull request #8917: URL: https://github.com/apache/arrow/pull/8917#issuecomment-746426491 I am very happy with the general heading of this PR. Thanks for your great work @returnString 😄 . I can't shake the feeling that we are adding too much complexity to

[GitHub] [arrow] pitrou closed pull request #8937: ARROW-10932: [C++] BinaryMemoTable::CopyOffsets access out-of-bound address when data is empty

2020-12-16 Thread GitBox
pitrou closed pull request #8937: URL: https://github.com/apache/arrow/pull/8937 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] jorgecarleitao opened a new pull request #8942: ARROW-10946: [Rust] Simplified bit chunk iterator

2020-12-16 Thread GitBox
jorgecarleitao opened a new pull request #8942: URL: https://github.com/apache/arrow/pull/8942 This PR makes the bit chunk iterator not depend on a `Buffer`, since we were only using it to keep track of the lifetime, and we can just use a byte slice for it. --

[GitHub] [arrow] github-actions[bot] commented on pull request #8942: ARROW-10946: [Rust] Simplified bit chunk iterator

2020-12-16 Thread GitBox
github-actions[bot] commented on pull request #8942: URL: https://github.com/apache/arrow/pull/8942#issuecomment-746876449 https://issues.apache.org/jira/browse/ARROW-10946 This is an automated message from the Apache Git Ser

  1   2   >