[GitHub] [arrow] ursabot edited a comment on pull request #9272: [WIP] Benchmark placebo

2021-02-25 Thread GitBox
ursabot edited a comment on pull request #9272: URL: https://github.com/apache/arrow/pull/9272#issuecomment-785654888 Benchmark runs are scheduled for baseline = 356c300c5ee1e2b23a83652514af11e3a731d596 and contender = 0f7cd4b8cb71cd5a7135404b2abc6e77de3aea7f. Results will be available as

[GitHub] [arrow] ritchie46 opened a new pull request #9571: ARROW-11778: [Rust] Cast from LargeUtf8 to Numerical and temporal types

2021-02-25 Thread GitBox
ritchie46 opened a new pull request #9571: URL: https://github.com/apache/arrow/pull/9571 Sorry that the PR's are not more clustered, but they occur to me in the wild. This PR allows casting from LargeUtf8 to numerical and temporal types. It also modifies the already existing strin

[GitHub] [arrow] ritchie46 opened a new pull request #9572: ARROW-11779: [Rust] make alloc module public

2021-02-25 Thread GitBox
ritchie46 opened a new pull request #9572: URL: https://github.com/apache/arrow/pull/9572 Polars uses the `arrow::memory` module. With the backwards incompatible change of ARROW-11627, the API is refactored to `arrow::alloc`. By making `alloc` public users can shift to the new change

[GitHub] [arrow] ursabot edited a comment on pull request #9272: [WIP] Benchmark placebo

2021-02-25 Thread GitBox
ursabot edited a comment on pull request #9272: URL: https://github.com/apache/arrow/pull/9272#issuecomment-785654888 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [arrow] github-actions[bot] commented on pull request #9571: ARROW-11778: [Rust] Cast from LargeUtf8 to Numerical and temporal types

2021-02-25 Thread GitBox
github-actions[bot] commented on pull request #9571: URL: https://github.com/apache/arrow/pull/9571#issuecomment-785746307 https://issues.apache.org/jira/browse/ARROW-11778 This is an automated message from the Apache Git Ser

[GitHub] [arrow] github-actions[bot] commented on pull request #9572: ARROW-11779: [Rust] make alloc module public

2021-02-25 Thread GitBox
github-actions[bot] commented on pull request #9572: URL: https://github.com/apache/arrow/pull/9572#issuecomment-785762556 https://issues.apache.org/jira/browse/ARROW-11779 This is an automated message from the Apache Git Ser

[GitHub] [arrow] xhochy commented on pull request #9569: ARROW-11695: [C++][FlightRPC] fix option to disable TLS verification

2021-02-25 Thread GitBox
xhochy commented on pull request #9569: URL: https://github.com/apache/arrow/pull/9569#issuecomment-785766872 It looks like https://github.com/apache/arrow/pull/9569/checks?check_run_id=1974857560#step:5:524 is a valid failure but the integration test seems like a flaky test. ---

[GitHub] [arrow] ursabot edited a comment on pull request #9272: [WIP] Benchmark placebo

2021-02-25 Thread GitBox
ursabot edited a comment on pull request #9272: URL: https://github.com/apache/arrow/pull/9272#issuecomment-785654888 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [arrow] nevi-me commented on pull request #9571: ARROW-11778: [Rust] Cast from LargeUtf8 to Numerical and temporal types

2021-02-25 Thread GitBox
nevi-me commented on pull request #9571: URL: https://github.com/apache/arrow/pull/9571#issuecomment-785778931 @ritchie46 you have unused imports somewhere This is an automated message from the Apache Git Service. To respond

[GitHub] [arrow] sagnikc-dremio commented on pull request #9450: ARROW-11565: [C++][Gandiva] Modify upper()/lower() logic to make them work for utf8 strings

2021-02-25 Thread GitBox
sagnikc-dremio commented on pull request #9450: URL: https://github.com/apache/arrow/pull/9450#issuecomment-785798201 @kou This patch seems to resolve the dependency issue. Thanks again. This is an automated message from the

[GitHub] [arrow] alamb commented on pull request #9568: ARROW-11709: [Rust][DataFusion] Move `expressions` and `inputs` into LogicalPlan ratherthan helpers in util

2021-02-25 Thread GitBox
alamb commented on pull request #9568: URL: https://github.com/apache/arrow/pull/9568#issuecomment-785830337 FYI @Dandandan and @houqp This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [arrow] alamb commented on pull request #9567: ARROW-11775: [Rust][DataFusion] Feature Flags for Dependencies

2021-02-25 Thread GitBox
alamb commented on pull request #9567: URL: https://github.com/apache/arrow/pull/9567#issuecomment-785843165 FYI @ovr -- given you contributed the original crypto functions, do you have any thoughts about making them optional (on by default)? -

[GitHub] [arrow] ovr commented on pull request #9567: ARROW-11775: [Rust][DataFusion] Feature Flags for Dependencies

2021-02-25 Thread GitBox
ovr commented on pull request #9567: URL: https://github.com/apache/arrow/pull/9567#issuecomment-785844998 > FYI @ovr -- given you contributed the original crypto functions, do you have any thoughts about making them optional (on by default)? Looks ok to me :) -

[GitHub] [arrow] ovr commented on a change in pull request #9434: ARROW-10816: [Rust][DF] Operations with Intervals

2021-02-25 Thread GitBox
ovr commented on a change in pull request #9434: URL: https://github.com/apache/arrow/pull/9434#discussion_r582777567 ## File path: rust/arrow/src/datatypes/interval.rs ## @@ -0,0 +1,33 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor li

[GitHub] [arrow] ovr commented on pull request #9434: ARROW-10816: [Rust][DF] Operations with Intervals

2021-02-25 Thread GitBox
ovr commented on pull request #9434: URL: https://github.com/apache/arrow/pull/9434#issuecomment-785846056 @alamb @andygrove @nevi-me @jorgecarleitao > Btw: There is a problem, DF executes `BinaryExpr` by casting left and right sides which is not needed for `timestamp - inter

[GitHub] [arrow] ovr edited a comment on pull request #9434: ARROW-10816: [Rust][DF] Operations with Intervals

2021-02-25 Thread GitBox
ovr edited a comment on pull request #9434: URL: https://github.com/apache/arrow/pull/9434#issuecomment-785846056 @alamb @andygrove @nevi-me @jorgecarleitao > Btw: >There is a problem, DF executes `BinaryExpr` by casting left and right sides which is not needed for `timestamp

[GitHub] [arrow] bkietz commented on pull request #9532: ARROW-11174: [C++][Dataset] Make expressions available to projection

2021-02-25 Thread GitBox
bkietz commented on pull request #9532: URL: https://github.com/apache/arrow/pull/9532#issuecomment-785854425 @nealrichardson thanks for testing! I'll add an analogous c++ test and get it passing This is an automated messag

[GitHub] [arrow] trucnguyenlam commented on pull request #9489: ARROW-11497: [Python] Provide parquet enable compliant nested type flag for python binding

2021-02-25 Thread GitBox
trucnguyenlam commented on pull request #9489: URL: https://github.com/apache/arrow/pull/9489#issuecomment-785862028 @jorisvandenbossche please have a look if it is ok to merge? I resolved all issues as far as I know. This i

[GitHub] [arrow] pitrou commented on pull request #9569: ARROW-11695: [C++][FlightRPC] fix option to disable TLS verification

2021-02-25 Thread GitBox
pitrou commented on pull request #9569: URL: https://github.com/apache/arrow/pull/9569#issuecomment-785871574 Opened https://github.com/grpc/grpc/issues/25556 upstream. This is an automated message from the Apache Git Service

[GitHub] [arrow] xhochy commented on pull request #9569: ARROW-11695: [C++][FlightRPC] fix option to disable TLS verification

2021-02-25 Thread GitBox
xhochy commented on pull request #9569: URL: https://github.com/apache/arrow/pull/9569#issuecomment-785875225 Added the patch to the conda recipe and seems to be working fine: https://github.com/conda-forge/arrow-cpp-feedstock/pull/362

[GitHub] [arrow] bkietz commented on a change in pull request #9532: ARROW-11174: [C++][Dataset] Make expressions available to projection

2021-02-25 Thread GitBox
bkietz commented on a change in pull request #9532: URL: https://github.com/apache/arrow/pull/9532#discussion_r582813969 ## File path: cpp/src/arrow/dataset/scanner.cc ## @@ -33,21 +34,10 @@ namespace arrow { namespace dataset { -ScanOptions::ScanOptions(std::shared_ptr sch

[GitHub] [arrow] lidavidm commented on pull request #9569: ARROW-11695: [C++][FlightRPC] fix option to disable TLS verification

2021-02-25 Thread GitBox
lidavidm commented on pull request #9569: URL: https://github.com/apache/arrow/pull/9569#issuecomment-785889363 Pushed a fix for the warning on MacOS. Thanks Antoine! To be fair to them, we're explicitly using things under a grpc::experimental namespace, but the macros would be nice.

[GitHub] [arrow] pitrou commented on pull request #8955: ARROW-9948: [C++] in Decimal128::FromString raise when scale is out of bounds

2021-02-25 Thread GitBox
pitrou commented on pull request #8955: URL: https://github.com/apache/arrow/pull/8955#issuecomment-785889550 @arw2019 Are you still interested in finishing this? This is an automated message from the Apache Git Service. To r

[GitHub] [arrow] pitrou commented on pull request #9569: ARROW-11695: [C++][FlightRPC] fix option to disable TLS verification

2021-02-25 Thread GitBox
pitrou commented on pull request #9569: URL: https://github.com/apache/arrow/pull/9569#issuecomment-785890135 Ah, thanks for pointing it out. I had missed the experimental part. This is an automated message from the Apache Gi

[GitHub] [arrow] arw2019 commented on pull request #8955: ARROW-9948: [C++] in Decimal128::FromString raise when scale is out of bounds

2021-02-25 Thread GitBox
arw2019 commented on pull request #8955: URL: https://github.com/apache/arrow/pull/8955#issuecomment-785898572 > @arw2019 Are you still interested in finishing this? @pitrou sorry for the delay. yes - will get to it today or over the weekend -

[GitHub] [arrow] pitrou commented on a change in pull request #9474: ARROW-10420: [C++] Refactor io and filesystem APIs to take an IOContext

2021-02-25 Thread GitBox
pitrou commented on a change in pull request #9474: URL: https://github.com/apache/arrow/pull/9474#discussion_r582862583 ## File path: cpp/src/arrow/csv/reader.cc ## @@ -935,18 +931,22 @@ class AsyncThreadedTableReader AsyncGenerator> buffer_generator_; }; +} // namespac

[GitHub] [arrow] pitrou commented on pull request #9474: ARROW-10420: [C++] Refactor io and filesystem APIs to take an IOContext

2021-02-25 Thread GitBox
pitrou commented on pull request #9474: URL: https://github.com/apache/arrow/pull/9474#issuecomment-785926294 @ursabot please benchmark This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [arrow] bkietz commented on a change in pull request #9532: ARROW-11174: [C++][Dataset] Make expressions available to projection

2021-02-25 Thread GitBox
bkietz commented on a change in pull request #9532: URL: https://github.com/apache/arrow/pull/9532#discussion_r582880766 ## File path: c_glib/arrow-dataset-glib/scanner.cpp ## @@ -265,7 +265,7 @@ gad_scan_options_class_init(GADScanOptionsClass *klass) gobject_class->set_prop

[GitHub] [arrow] bkietz commented on a change in pull request #9532: ARROW-11174: [C++][Dataset] Make expressions available to projection

2021-02-25 Thread GitBox
bkietz commented on a change in pull request #9532: URL: https://github.com/apache/arrow/pull/9532#discussion_r582881412 ## File path: java/dataset/src/test/java/org/apache/arrow/dataset/jni/TestReservationListener.java ## @@ -58,9 +53,8 @@ public void testDirectReservationLis

[GitHub] [arrow] lidavidm commented on pull request #9569: ARROW-11695: [C++][FlightRPC] fix option to disable TLS verification

2021-02-25 Thread GitBox
lidavidm commented on pull request #9569: URL: https://github.com/apache/arrow/pull/9569#issuecomment-785951838 Hmm, it seems MacOS is different about this and I'll need to actually include the dummy root cert still. Let me update the PR. (https://github.com/grpc/grpc/issues/21655 se

[GitHub] [arrow] mrkn commented on a change in pull request #9532: ARROW-11174: [C++][Dataset] Make expressions available to projection

2021-02-25 Thread GitBox
mrkn commented on a change in pull request #9532: URL: https://github.com/apache/arrow/pull/9532#discussion_r582900019 ## File path: c_glib/arrow-dataset-glib/scanner.cpp ## @@ -265,7 +265,7 @@ gad_scan_options_class_init(GADScanOptionsClass *klass) gobject_class->set_proper

[GitHub] [arrow] ursabot commented on pull request #9474: ARROW-10420: [C++] Refactor io and filesystem APIs to take an IOContext

2021-02-25 Thread GitBox
ursabot commented on pull request #9474: URL: https://github.com/apache/arrow/pull/9474#issuecomment-785966421 Benchmark runs are scheduled for baseline = b5ac048c75cc55f4039d279f554920be3112d7cd and contender = f16a80c7458753c224ab2d43a6985b0ab2ccc9db. Results will be available as each b

[GitHub] [arrow] bkietz commented on a change in pull request #9474: ARROW-10420: [C++] Refactor io and filesystem APIs to take an IOContext

2021-02-25 Thread GitBox
bkietz commented on a change in pull request #9474: URL: https://github.com/apache/arrow/pull/9474#discussion_r582925427 ## File path: cpp/src/arrow/io/interfaces.h ## @@ -49,16 +49,36 @@ struct ReadRange { }; // EXPERIMENTAL -struct ARROW_EXPORT AsyncContext { - ::arrow::

[GitHub] [arrow] elferherrera opened a new pull request #9573: Proposal for RFCs in Rust Arrow

2021-02-25 Thread GitBox
elferherrera opened a new pull request #9573: URL: https://github.com/apache/arrow/pull/9573 The PR proposes a "light" version of the RFC process followed by the Rust community to manage the introduction of new features. The RFC objective is to be a roadmap of the desired features that peo

[GitHub] [arrow] nealrichardson closed pull request #9566: ARROW-11756: [R] passing a partition as a schema leads to segfaults

2021-02-25 Thread GitBox
nealrichardson closed pull request #9566: URL: https://github.com/apache/arrow/pull/9566 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [arrow] edrevo opened a new pull request #9574: ARROW-11784: [Rust][DataFusion] CoalesceBatchesStream doesn't honor Stream interface

2021-02-25 Thread GitBox
edrevo opened a new pull request #9574: URL: https://github.com/apache/arrow/pull/9574 Unit tests now cover the bug to avoid regressions. This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [arrow] edrevo commented on pull request #9574: ARROW-11784: [Rust][DataFusion] CoalesceBatchesStream doesn't honor Stream interface

2021-02-25 Thread GitBox
edrevo commented on pull request #9574: URL: https://github.com/apache/arrow/pull/9574#issuecomment-786038308 cc @andygrove This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [arrow] westonpace commented on a change in pull request #9532: ARROW-11174: [C++][Dataset] Make expressions available to projection

2021-02-25 Thread GitBox
westonpace commented on a change in pull request #9532: URL: https://github.com/apache/arrow/pull/9532#discussion_r582990025 ## File path: cpp/src/arrow/dataset/scanner.cc ## @@ -33,21 +34,10 @@ namespace arrow { namespace dataset { -ScanOptions::ScanOptions(std::shared_ptr

[GitHub] [arrow] emkornfield commented on pull request #9504: ARROW-2229: [C++][Python] Add WriteCsv functionality.

2021-02-25 Thread GitBox
emkornfield commented on pull request #9504: URL: https://github.com/apache/arrow/pull/9504#issuecomment-786042209 I'll do a follow-up pr to expose a class/object for writing This is an automated message from the Apache Git S

[GitHub] [arrow] jonkeane commented on a change in pull request #9561: ARROW-11649: [R] Add support for null_fallback to R

2021-02-25 Thread GitBox
jonkeane commented on a change in pull request #9561: URL: https://github.com/apache/arrow/pull/9561#discussion_r582993386 ## File path: r/R/dataset-partition.R ## @@ -72,19 +77,22 @@ HivePartitioning$create <- dataset___HivePartitioning #' Because fields are named in the path

[GitHub] [arrow] jonkeane commented on a change in pull request #9561: ARROW-11649: [R] Add support for null_fallback to R

2021-02-25 Thread GitBox
jonkeane commented on a change in pull request #9561: URL: https://github.com/apache/arrow/pull/9561#discussion_r582994030 ## File path: r/R/dataset-partition.R ## @@ -72,19 +77,22 @@ HivePartitioning$create <- dataset___HivePartitioning #' Because fields are named in the path

[GitHub] [arrow] jonkeane commented on a change in pull request #9561: ARROW-11649: [R] Add support for null_fallback to R

2021-02-25 Thread GitBox
jonkeane commented on a change in pull request #9561: URL: https://github.com/apache/arrow/pull/9561#discussion_r582998490 ## File path: r/R/dataset-partition.R ## @@ -25,12 +25,17 @@ #' `DirectoryPartitioning` describes how to interpret raw path segments, in #' order. For ex

[GitHub] [arrow] houqp commented on a change in pull request #9534: ARROW-11707: [Rust] support CSV schema inference without file IO

2021-02-25 Thread GitBox
houqp commented on a change in pull request #9534: URL: https://github.com/apache/arrow/pull/9534#discussion_r583009292 ## File path: rust/arrow/src/csv/reader.rs ## @@ -99,7 +99,27 @@ fn infer_field_schema(string: &str) -> DataType { /// If `max_read_records` is not set, the

[GitHub] [arrow] alamb opened a new pull request #9575: ARROW-11773: [Rust] Support writing well formed JSON arrays

2021-02-25 Thread GitBox
alamb opened a new pull request #9575: URL: https://github.com/apache/arrow/pull/9575 ## Rationale Currently the Arrow json writer makes JSON that looks like this (one record per line): ```json {"foo":1} {"bar":1} ``` Which is not technically valid JSON, which would lo

[GitHub] [arrow] alamb commented on a change in pull request #9575: ARROW-11773: [Rust] Support writing well formed JSON arrays

2021-02-25 Thread GitBox
alamb commented on a change in pull request #9575: URL: https://github.com/apache/arrow/pull/9575#discussion_r583015629 ## File path: rust/arrow/src/json/writer.rs ## @@ -42,7 +45,39 @@ //! ); //! ``` //! -//! Serialize record batches into line-delimited JSON bytes: +//! ##

[GitHub] [arrow] alamb commented on pull request #9575: ARROW-11773: [Rust] Support writing well formed JSON arrays

2021-02-25 Thread GitBox
alamb commented on pull request #9575: URL: https://github.com/apache/arrow/pull/9575#issuecomment-786062677 cc @houqp This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [arrow] houqp commented on pull request #9575: ARROW-11773: [Rust] Support writing well formed JSON arrays

2021-02-25 Thread GitBox
houqp commented on pull request #9575: URL: https://github.com/apache/arrow/pull/9575#issuecomment-786070165 I personally prefer the more composable formatter trait approach. This is an automated message from the Apache Git S

[GitHub] [arrow] alamb commented on a change in pull request #9565: ARROW-11655: [Rust][DataFusion] Postgres String Functions: left, lpad, right, rpad

2021-02-25 Thread GitBox
alamb commented on a change in pull request #9565: URL: https://github.com/apache/arrow/pull/9565#discussion_r583024015 ## File path: rust/datafusion/src/physical_plan/type_coercion.rs ## @@ -168,20 +168,35 @@ fn maybe_data_types( pub fn can_coerce_from(type_into: &DataType, t

[GitHub] [arrow] jonkeane commented on a change in pull request #9561: ARROW-11649: [R] Add support for null_fallback to R

2021-02-25 Thread GitBox
jonkeane commented on a change in pull request #9561: URL: https://github.com/apache/arrow/pull/9561#discussion_r583026368 ## File path: r/R/dataset-write.R ## @@ -41,6 +41,9 @@ #' will yield `"part-0.feather", ...`. #' @param hive_style logical: write partition segments as H

[GitHub] [arrow] github-actions[bot] commented on pull request #9573: ARROW-11783: [Rust] Proposal for RFCs in Rust Arrow

2021-02-25 Thread GitBox
github-actions[bot] commented on pull request #9573: URL: https://github.com/apache/arrow/pull/9573#issuecomment-786075152 Thanks for opening a pull request! Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW Then could

[GitHub] [arrow] nealrichardson commented on a change in pull request #9521: ARROW-11683: [R] Support dplyr::mutate()

2021-02-25 Thread GitBox
nealrichardson commented on a change in pull request #9521: URL: https://github.com/apache/arrow/pull/9521#discussion_r583043183 ## File path: r/R/dplyr.R ## @@ -309,8 +355,27 @@ collect.arrow_dplyr_query <- function(x, as_data_frame = TRUE, ...) { # See dataset.R for Dat

[GitHub] [arrow] lidavidm commented on pull request #9569: ARROW-11695: [C++][FlightRPC] fix option to disable TLS verification

2021-02-25 Thread GitBox
lidavidm commented on pull request #9569: URL: https://github.com/apache/arrow/pull/9569#issuecomment-786091725 And now it passes (minus the integration test which I'll look at next). This is an automated message from the Apa

[GitHub] [arrow] github-actions[bot] commented on pull request #9573: ARROW-11783: [Rust] Proposal for RFCs in Rust Arrow

2021-02-25 Thread GitBox
github-actions[bot] commented on pull request #9573: URL: https://github.com/apache/arrow/pull/9573#issuecomment-786096429 https://issues.apache.org/jira/browse/ARROW-11783 This is an automated message from the Apache Git Ser

[GitHub] [arrow] github-actions[bot] commented on pull request #9574: ARROW-11784: [Rust][DataFusion] CoalesceBatchesStream doesn't honor Stream interface

2021-02-25 Thread GitBox
github-actions[bot] commented on pull request #9574: URL: https://github.com/apache/arrow/pull/9574#issuecomment-786096893 https://issues.apache.org/jira/browse/ARROW-11784 This is an automated message from the Apache Git Ser

[GitHub] [arrow] alamb closed pull request #9572: ARROW-11779: [Rust] make alloc module public

2021-02-25 Thread GitBox
alamb closed pull request #9572: URL: https://github.com/apache/arrow/pull/9572 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] alamb closed pull request #9570: ARROW-11777: [Rust] impl AsRef for StringBuilder/BinaryBuilder

2021-02-25 Thread GitBox
alamb closed pull request #9570: URL: https://github.com/apache/arrow/pull/9570 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] nbruno commented on a change in pull request #9151: ARROW-11173: [Java] Add map type in complex reader / writer

2021-02-25 Thread GitBox
nbruno commented on a change in pull request #9151: URL: https://github.com/apache/arrow/pull/9151#discussion_r583064206 ## File path: java/vector/src/main/codegen/templates/UnionFixedSizeListWriter.java ## @@ -169,6 +169,29 @@ public StructWriter struct(String name) { ret

[GitHub] [arrow] pierwill opened a new pull request #9576: Edit Rust README

2021-02-25 Thread GitBox
pierwill opened a new pull request #9576: URL: https://github.com/apache/arrow/pull/9576 Edits and fixes for some missing words, punctuation, and wording. This is an automated message from the Apache Git Service. To respond t

[GitHub] [arrow] github-actions[bot] commented on pull request #9575: ARROW-11773: [Rust] Support writing well formed JSON arrays

2021-02-25 Thread GitBox
github-actions[bot] commented on pull request #9575: URL: https://github.com/apache/arrow/pull/9575#issuecomment-786109433 https://issues.apache.org/jira/browse/ARROW-11773 This is an automated message from the Apache Git Ser

[GitHub] [arrow] nealrichardson closed pull request #9521: ARROW-11683: [R] Support dplyr::mutate()

2021-02-25 Thread GitBox
nealrichardson closed pull request #9521: URL: https://github.com/apache/arrow/pull/9521 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [arrow] pitrou opened a new pull request #9577: ARROW-11662: [C++] Support sorting decimal and fixed size binary data

2021-02-25 Thread GitBox
pitrou opened a new pull request #9577: URL: https://github.com/apache/arrow/pull/9577 Also enable nth_to_indices on decimal and fixed size binary data. This is an automated message from the Apache Git Service. To respond to

[GitHub] [arrow] andygrove commented on pull request #9573: ARROW-11783: [Rust] Proposal for RFCs in Rust Arrow

2021-02-25 Thread GitBox
andygrove commented on pull request #9573: URL: https://github.com/apache/arrow/pull/9573#issuecomment-786131322 I think this is a great way to share information about new features in one place rather than relying on mailing list discussions, JIRAs, and Google docs. Thank you for working o

[GitHub] [arrow] xhochy commented on pull request #9569: ARROW-11695: [C++][FlightRPC] fix option to disable TLS verification

2021-02-25 Thread GitBox
xhochy commented on pull request #9569: URL: https://github.com/apache/arrow/pull/9569#issuecomment-786131859 > And now it passes (minus the integration test which I'll look at next). As said before: It seems unrelated to me as I saw that failures in an other PR, too. -

[GitHub] [arrow] lidavidm commented on pull request #9569: ARROW-11695: [C++][FlightRPC] fix option to disable TLS verification

2021-02-25 Thread GitBox
lidavidm commented on pull request #9569: URL: https://github.com/apache/arrow/pull/9569#issuecomment-786132477 > > And now it passes (minus the integration test which I'll look at next). > > As said before: It seems unrelated to me as I saw that failures in an other PR, too.

[GitHub] [arrow] xhochy closed pull request #9569: ARROW-11695: [C++][FlightRPC] fix option to disable TLS verification

2021-02-25 Thread GitBox
xhochy closed pull request #9569: URL: https://github.com/apache/arrow/pull/9569 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] pitrou commented on pull request #9474: ARROW-10420: [C++] Refactor io and filesystem APIs to take an IOContext

2021-02-25 Thread GitBox
pitrou commented on pull request #9474: URL: https://github.com/apache/arrow/pull/9474#issuecomment-786138511 @bkietz Do you want to give this another look? This is an automated message from the Apache Git Service. To respond

[GitHub] [arrow] pitrou closed pull request #9504: ARROW-2229: [C++][Python] Add WriteCsv functionality.

2021-02-25 Thread GitBox
pitrou closed pull request #9504: URL: https://github.com/apache/arrow/pull/9504 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] ursabot commented on pull request #9474: ARROW-10420: [C++] Refactor io and filesystem APIs to take an IOContext

2021-02-25 Thread GitBox
ursabot commented on pull request #9474: URL: https://github.com/apache/arrow/pull/9474#issuecomment-786139971 Benchmark runs are scheduled for baseline = d7da16e57258a5fa74cd5e7534a93010058b9a3c and contender = a463936962b865e6d2fa65dc53d0a182b65526a3. Results will be available as each b

[GitHub] [arrow] ursabot commented on pull request #9272: [WIP] Benchmark placebo

2021-02-25 Thread GitBox
ursabot commented on pull request #9272: URL: https://github.com/apache/arrow/pull/9272#issuecomment-786140091 Benchmark runs are scheduled for baseline = 356c300c5ee1e2b23a83652514af11e3a731d596 and contender = 0f7cd4b8cb71cd5a7135404b2abc6e77de3aea7f. Results will be available as each b

[GitHub] [arrow] ritchie46 commented on pull request #9572: ARROW-11779: [Rust] make alloc module public

2021-02-25 Thread GitBox
ritchie46 commented on pull request #9572: URL: https://github.com/apache/arrow/pull/9572#issuecomment-786141749 I use an wrapper type that is basically a Rust `Vec` combined with an arrow alloctor. [AlignedVec](https://github.com/ritchie46/polars/blob/06864d5135d4dd133fe496a9cc7e99681dbb9

[GitHub] [arrow] lidavidm opened a new pull request #9578: ARROW-11786: [C++] Remove noisy CMake message

2021-02-25 Thread GitBox
lidavidm opened a new pull request #9578: URL: https://github.com/apache/arrow/pull/9578 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [arrow] github-actions[bot] commented on pull request #9576: Edit Rust README

2021-02-25 Thread GitBox
github-actions[bot] commented on pull request #9576: URL: https://github.com/apache/arrow/pull/9576#issuecomment-786146618 Thanks for opening a pull request! Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW Then could

[GitHub] [arrow] westonpace commented on a change in pull request #9561: ARROW-11649: [R] Add support for null_fallback to R

2021-02-25 Thread GitBox
westonpace commented on a change in pull request #9561: URL: https://github.com/apache/arrow/pull/9561#discussion_r583113776 ## File path: r/R/dataset-partition.R ## @@ -72,19 +77,22 @@ HivePartitioning$create <- dataset___HivePartitioning #' Because fields are named in the pa

[GitHub] [arrow] ursabot commented on pull request #9474: ARROW-10420: [C++] Refactor io and filesystem APIs to take an IOContext

2021-02-25 Thread GitBox
ursabot commented on pull request #9474: URL: https://github.com/apache/arrow/pull/9474#issuecomment-786153504 Benchmark runs are scheduled for baseline = 9a9baf6824db91be2c0913367d4b151d9390a4e6 and contender = da3ece91279509cd21130c7fee24a73e4b81eb4f. Results will be available as each b

[GitHub] [arrow] ursabot commented on pull request #9272: [WIP] Benchmark placebo

2021-02-25 Thread GitBox
ursabot commented on pull request #9272: URL: https://github.com/apache/arrow/pull/9272#issuecomment-786153566 Benchmark runs are scheduled for baseline = 356c300c5ee1e2b23a83652514af11e3a731d596 and contender = 0f7cd4b8cb71cd5a7135404b2abc6e77de3aea7f. Results will be available as each b

[GitHub] [arrow] ursabot removed a comment on pull request #9474: ARROW-10420: [C++] Refactor io and filesystem APIs to take an IOContext

2021-02-25 Thread GitBox
ursabot removed a comment on pull request #9474: URL: https://github.com/apache/arrow/pull/9474#issuecomment-785966421 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [arrow] paddyhoran commented on pull request #9573: ARROW-11783: [Rust] Proposal for RFCs in Rust Arrow

2021-02-25 Thread GitBox
paddyhoran commented on pull request #9573: URL: https://github.com/apache/arrow/pull/9573#issuecomment-786157815 @elferherrera this is a great idea. As someone finding it hard to spend any time on Arrow but trying to keep "in the loop" this is really valuable. Thank you. -

[GitHub] [arrow] github-actions[bot] commented on pull request #9577: ARROW-11662: [C++] Support sorting decimal and fixed size binary data

2021-02-25 Thread GitBox
github-actions[bot] commented on pull request #9577: URL: https://github.com/apache/arrow/pull/9577#issuecomment-786161449 https://issues.apache.org/jira/browse/ARROW-11662 This is an automated message from the Apache Git Ser

[GitHub] [arrow] pierwill closed pull request #9576: Edit Rust README

2021-02-25 Thread GitBox
pierwill closed pull request #9576: URL: https://github.com/apache/arrow/pull/9576 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [arrow] bkietz commented on a change in pull request #9474: ARROW-10420: [C++] Refactor io and filesystem APIs to take an IOContext

2021-02-25 Thread GitBox
bkietz commented on a change in pull request #9474: URL: https://github.com/apache/arrow/pull/9474#discussion_r583131021 ## File path: r/src/csv.cpp ## @@ -141,9 +141,9 @@ std::shared_ptr csv___TableReader__Make( const std::shared_ptr& read_options, const std::shared

[GitHub] [arrow] nevi-me commented on pull request #9575: ARROW-11773: [Rust] Support writing well formed JSON arrays

2021-02-25 Thread GitBox
nevi-me commented on pull request #9575: URL: https://github.com/apache/arrow/pull/9575#issuecomment-786168328 > Which is not technically valid JSON, which would look something like this: It's streaming JSON or NDJSON (https://en.wikipedia.org/wiki/JSON_streaming#Line-delimited_JSON)

[GitHub] [arrow] nevi-me commented on a change in pull request #9575: ARROW-11773: [Rust] Support writing well formed JSON arrays

2021-02-25 Thread GitBox
nevi-me commented on a change in pull request #9575: URL: https://github.com/apache/arrow/pull/9575#discussion_r583134343 ## File path: rust/arrow/src/json/writer.rs ## @@ -309,40 +353,167 @@ pub fn record_batches_to_json_rows( rows } -/// A JSON writer -#[derive(Debug)

[GitHub] [arrow] seddonm1 commented on pull request #9567: ARROW-11775: [Rust][DataFusion] Feature Flags for Dependencies

2021-02-25 Thread GitBox
seddonm1 commented on pull request #9567: URL: https://github.com/apache/arrow/pull/9567#issuecomment-786172573 Thanks @alamb I did try to wrap code blocks `{}` with the macro and that did not work. I will see if the compiler is clever enough to handle macros as I agree this gets p

[GitHub] [arrow] ursabot removed a comment on pull request #9474: ARROW-10420: [C++] Refactor io and filesystem APIs to take an IOContext

2021-02-25 Thread GitBox
ursabot removed a comment on pull request #9474: URL: https://github.com/apache/arrow/pull/9474#issuecomment-786153504 Benchmark runs are scheduled for baseline = 9a9baf6824db91be2c0913367d4b151d9390a4e6 and contender = da3ece91279509cd21130c7fee24a73e4b81eb4f. Results will be available as

[GitHub] [arrow] ursabot commented on pull request #9474: ARROW-10420: [C++] Refactor io and filesystem APIs to take an IOContext

2021-02-25 Thread GitBox
ursabot commented on pull request #9474: URL: https://github.com/apache/arrow/pull/9474#issuecomment-786177313 Benchmark runs are scheduled for baseline = 9a9baf6824db91be2c0913367d4b151d9390a4e6 and contender = da3ece91279509cd21130c7fee24a73e4b81eb4f. Results will be available as each b

[GitHub] [arrow] ursabot commented on pull request #9272: [WIP] Benchmark placebo

2021-02-25 Thread GitBox
ursabot commented on pull request #9272: URL: https://github.com/apache/arrow/pull/9272#issuecomment-786177417 Benchmark runs are scheduled for baseline = 356c300c5ee1e2b23a83652514af11e3a731d596 and contender = 0f7cd4b8cb71cd5a7135404b2abc6e77de3aea7f. Results will be available as each b

[GitHub] [arrow] seddonm1 commented on a change in pull request #9565: ARROW-11655: [Rust][DataFusion] Postgres String Functions: left, lpad, right, rpad

2021-02-25 Thread GitBox
seddonm1 commented on a change in pull request #9565: URL: https://github.com/apache/arrow/pull/9565#discussion_r583147592 ## File path: rust/datafusion/src/physical_plan/type_coercion.rs ## @@ -168,20 +168,35 @@ fn maybe_data_types( pub fn can_coerce_from(type_into: &DataType

[GitHub] [arrow] seddonm1 commented on a change in pull request #9565: ARROW-11655: [Rust][DataFusion] Postgres String Functions: left, lpad, right, rpad

2021-02-25 Thread GitBox
seddonm1 commented on a change in pull request #9565: URL: https://github.com/apache/arrow/pull/9565#discussion_r583147743 ## File path: rust/datafusion/tests/sql.rs ## @@ -530,17 +530,6 @@ async fn sqrt_f32_vs_f64() -> Result<()> { Ok(()) } -#[tokio::test] -async fn cs

[GitHub] [arrow] alamb commented on a change in pull request #9575: ARROW-11773: [Rust] Support writing well formed JSON arrays

2021-02-25 Thread GitBox
alamb commented on a change in pull request #9575: URL: https://github.com/apache/arrow/pull/9575#discussion_r583147570 ## File path: rust/arrow/src/json/writer.rs ## @@ -309,40 +353,167 @@ pub fn record_batches_to_json_rows( rows } -/// A JSON writer -#[derive(Debug)]

[GitHub] [arrow] edrevo commented on pull request #9523: ARROW-11687: [Rust][DataFusion] RepartitionExec Hanging

2021-02-25 Thread GitBox
edrevo commented on pull request #9523: URL: https://github.com/apache/arrow/pull/9523#issuecomment-786180030 @andygrove / @seddonm1, I think we should revert this: RepartitionExec will now load everything into memory, since we are now .awaiting untill the input stream has been completely

[GitHub] [arrow] seddonm1 commented on pull request #9523: ARROW-11687: [Rust][DataFusion] RepartitionExec Hanging

2021-02-25 Thread GitBox
seddonm1 commented on pull request #9523: URL: https://github.com/apache/arrow/pull/9523#issuecomment-786185550 @edrevo We should revert if there is an issue. This change puts the await per input partition so I thought it would wait until that partition is fully processed with the ma

[GitHub] [arrow] alamb commented on pull request #9523: ARROW-11687: [Rust][DataFusion] RepartitionExec Hanging

2021-02-25 Thread GitBox
alamb commented on pull request #9523: URL: https://github.com/apache/arrow/pull/9523#issuecomment-786194765 I reviewed the code -- and I think @edrevo is right (basically that the call to `execute` won't return now until after all the streams have been finished). I wonder if the right fi

[GitHub] [arrow] edrevo commented on pull request #9523: ARROW-11687: [Rust][DataFusion] RepartitionExec Hanging

2021-02-25 Thread GitBox
edrevo commented on pull request #9523: URL: https://github.com/apache/arrow/pull/9523#issuecomment-786198472 Unfortunately, it looks like #9574 isn't the fix for this: I've reverted this change in my branch with that fix and it does in fact still hang in the last partition.

[GitHub] [arrow] seddonm1 commented on pull request #9523: ARROW-11687: [Rust][DataFusion] RepartitionExec Hanging

2021-02-25 Thread GitBox
seddonm1 commented on pull request #9523: URL: https://github.com/apache/arrow/pull/9523#issuecomment-786198879 @edrevo @alamb Yes. I have read through that code again and agree. It will be blocking. Let me see if I can do some more debugging as my code is definitely reproducing the

[GitHub] [arrow] alamb commented on pull request #9434: ARROW-10816: [Rust][DF] Operations with Intervals

2021-02-25 Thread GitBox
alamb commented on pull request #9434: URL: https://github.com/apache/arrow/pull/9434#issuecomment-786199045 > There is a problem, DF executes BinaryExpr by casting left and right sides which is not needed for timestamp - interval, because this calculation is exception in this case and sho

[GitHub] [arrow] edrevo commented on pull request #9523: ARROW-11687: [Rust][DataFusion] RepartitionExec Hanging

2021-02-25 Thread GitBox
edrevo commented on pull request #9523: URL: https://github.com/apache/arrow/pull/9523#issuecomment-786199330 By the way, I am trying to repartition 100GB of memory and with this change I get OOMKilled (unfortunately for me I don't have 100GB of RAM 😂 ) and without this change the memory u

[GitHub] [arrow] alamb commented on pull request #9523: ARROW-11687: [Rust][DataFusion] RepartitionExec Hanging

2021-02-25 Thread GitBox
alamb commented on pull request #9523: URL: https://github.com/apache/arrow/pull/9523#issuecomment-786199845 Thanks @edrevo and @seddonm1 ! I am sorry to hear DataFusion is abusing your system :) Let me know if I can help ---

[GitHub] [arrow] github-actions[bot] commented on pull request #9578: ARROW-11786: [C++] Remove noisy CMake message

2021-02-25 Thread GitBox
github-actions[bot] commented on pull request #9578: URL: https://github.com/apache/arrow/pull/9578#issuecomment-786200289 https://issues.apache.org/jira/browse/ARROW-11786 This is an automated message from the Apache Git Ser

[GitHub] [arrow] seddonm1 commented on pull request #9523: ARROW-11687: [Rust][DataFusion] RepartitionExec Hanging

2021-02-25 Thread GitBox
seddonm1 commented on pull request #9523: URL: https://github.com/apache/arrow/pull/9523#issuecomment-786206569 @edrevo you have inspired me to work this out This is an automated message from the Apache Git Service. To respon

[GitHub] [arrow] jonkeane opened a new pull request #9579: ARROW-11774: [R] macos one line install

2021-02-25 Thread GitBox
jonkeane opened a new pull request #9579: URL: https://github.com/apache/arrow/pull/9579 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

  1   2   >