[GitHub] [arrow] sanjibansg commented on pull request #12426: ARROW-15672: [C++] Enable CSV writer to control the field delimiter

2022-02-21 Thread GitBox
sanjibansg commented on pull request #12426: URL: https://github.com/apache/arrow/pull/12426#issuecomment-1047511108 > Appveyor test failure looks related to this PR, needs be fixed. https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/42659260/job/qk6x3kld8ffabyuu#L2758

[GitHub] [arrow-datafusion] hntd187 commented on issue #1544: Streaming support for DataFusion

2022-02-21 Thread GitBox
hntd187 commented on issue #1544: URL: https://github.com/apache/arrow-datafusion/issues/1544#issuecomment-1047492411 Just an update I am still working on this, I've just been inundated with work so my progress had been slower than I'd like. I don't have any of the larger parts in a worki

[GitHub] [arrow-datafusion] matthewmturner commented on issue #1872: Add option to datafusion-cli to execute SQL from file and keep session open

2022-02-21 Thread GitBox
matthewmturner commented on issue #1872: URL: https://github.com/apache/arrow-datafusion/issues/1872#issuecomment-1047485499 @Jimexist FYI - as you've shown interest in datafusion-cli topics in the past. -- This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [arrow-datafusion] PierreZ commented on issue #1864: Add a DataFusion example capable of retrieving data from a custom datasource

2022-02-21 Thread GitBox
PierreZ commented on issue #1864: URL: https://github.com/apache/arrow-datafusion/issues/1864#issuecomment-1047478463 > Yes, I think it would be helpful to have a simplistic example in our user doc :) At the very minimal, we could point to our memtable implementation if coming up with a s

[GitHub] [arrow-datafusion] matthewmturner opened a new issue #1872: Add option to datafusion-cli to execute SQL from file and keep session open

2022-02-21 Thread GitBox
matthewmturner opened a new issue #1872: URL: https://github.com/apache/arrow-datafusion/issues/1872 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** A clear and concise description of what the problem is. Ex. I'm always frustrate

[GitHub] [arrow-datafusion] matthewmturner opened a new issue #1871: CREATE EXTERNAL TABLE sql should check if table exists before doing IO

2022-02-21 Thread GitBox
matthewmturner opened a new issue #1871: URL: https://github.com/apache/arrow-datafusion/issues/1871 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** A clear and concise description of what the problem is. Ex. I'm always frustrate

[GitHub] [arrow] sanjibansg commented on a change in pull request #12426: ARROW-15672: [C++] Enable CSV writer to control the field delimiter

2022-02-21 Thread GitBox
sanjibansg commented on a change in pull request #12426: URL: https://github.com/apache/arrow/pull/12426#discussion_r811598053 ## File path: cpp/src/arrow/csv/writer_test.cc ## @@ -171,6 +173,13 @@ std::vector GenerateTestCases() { "style is \"None\". See RFC4180. Inva

[GitHub] [arrow] sanjibansg commented on a change in pull request #12426: ARROW-15672: [C++] Enable CSV writer to control the field delimiter

2022-02-21 Thread GitBox
sanjibansg commented on a change in pull request #12426: URL: https://github.com/apache/arrow/pull/12426#discussion_r811597957 ## File path: cpp/src/arrow/csv/writer_test.cc ## @@ -189,6 +198,12 @@ std::vector GenerateTestCases() { /*expected_output*/ "", expected

[GitHub] [arrow] sanjibansg commented on a change in pull request #12426: ARROW-15672: [C++] Enable CSV writer to control the field delimiter

2022-02-21 Thread GitBox
sanjibansg commented on a change in pull request #12426: URL: https://github.com/apache/arrow/pull/12426#discussion_r811597854 ## File path: cpp/src/arrow/csv/writer_test.cc ## @@ -60,13 +60,15 @@ void PrintTo(const WriterTestParams& p, std::ostream* os) { WriteOptions Default

[GitHub] [arrow-rs] HaoYang670 opened a new issue #1351: Refactor `StructArray::from`

2022-02-21 Thread GitBox
HaoYang670 opened a new issue #1351: URL: https://github.com/apache/arrow-rs/issues/1351 **Describe the bug** The function `StructArray::from` use `vec.push` in a for loop, which may cause many times of memory reallocation. What about using `Iterator::collect` Instead? Need your opinion

[GitHub] [arrow-rs] HaoYang670 commented on issue #1350: Refactor `RecordBatch::validate_new_batch`

2022-02-21 Thread GitBox
HaoYang670 commented on issue #1350: URL: https://github.com/apache/arrow-rs/issues/1350#issuecomment-1047429549 Using `Iterator.find` or `Iterator.any` may be a good choice. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [arrow-rs] HaoYang670 opened a new issue #1350: Refactor `RecordBatch::validate_new_batch`

2022-02-21 Thread GitBox
HaoYang670 opened a new issue #1350: URL: https://github.com/apache/arrow-rs/issues/1350 **Describe the bug** The function `validate_new_batch` constains some repetitive code. We could find a way to refactor it. https://github.com/apache/arrow-rs/blob/master/arrow/src/record_batch.rs

[GitHub] [arrow-datafusion] matthewmturner opened a new issue #1870: Add a script for running db-benchmark

2022-02-21 Thread GitBox
matthewmturner opened a new issue #1870: URL: https://github.com/apache/arrow-datafusion/issues/1870 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** A clear and concise description of what the problem is. Ex. I'm always frustrate

[GitHub] [arrow-datafusion] matthewmturner commented on issue #147: Add DataFusion to h2oai/db-benchmark

2022-02-21 Thread GitBox
matthewmturner commented on issue #147: URL: https://github.com/apache/arrow-datafusion/issues/147#issuecomment-1047420945 Cross post from slack: I’m working on updating datafusions db-benchmark results based on datafusion v7. i just got a first cut of the results compared to what

[GitHub] [arrow] cyb70289 commented on pull request #12426: ARROW-15672: [C++] Enable CSV writer to control the field delimiter

2022-02-21 Thread GitBox
cyb70289 commented on pull request #12426: URL: https://github.com/apache/arrow/pull/12426#issuecomment-1047373300 Appveyor test failure looks related to this PR, needs be fixed. https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/42659260/job/qk6x3kld8ffabyuu#L2758 -

[GitHub] [arrow] cyb70289 commented on a change in pull request #12426: ARROW-15672: [C++] Enable CSV writer to control the field delimiter

2022-02-21 Thread GitBox
cyb70289 commented on a change in pull request #12426: URL: https://github.com/apache/arrow/pull/12426#discussion_r811530024 ## File path: cpp/src/arrow/csv/writer_test.cc ## @@ -171,6 +173,13 @@ std::vector GenerateTestCases() { "style is \"None\". See RFC4180. Invali

[GitHub] [arrow] cyb70289 commented on a change in pull request #12426: ARROW-15672: [C++] Enable CSV writer to control the field delimiter

2022-02-21 Thread GitBox
cyb70289 commented on a change in pull request #12426: URL: https://github.com/apache/arrow/pull/12426#discussion_r811529389 ## File path: cpp/src/arrow/csv/writer_test.cc ## @@ -189,6 +198,12 @@ std::vector GenerateTestCases() { /*expected_output*/ "", expected_s

[GitHub] [arrow-datafusion] houqp merged pull request #1866: Add doc for building `datafusion-cli` when connect the ballista

2022-02-21 Thread GitBox
houqp merged pull request #1866: URL: https://github.com/apache/arrow-datafusion/pull/1866 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-

[GitHub] [arrow-datafusion] houqp edited a comment on issue #1818: A corner bug in union

2022-02-21 Thread GitBox
houqp edited a comment on issue #1818: URL: https://github.com/apache/arrow-datafusion/issues/1818#issuecomment-1047364448 I agree, if this is only happening to union, realigning the column names for anonymous columns during union planning would be a better/simpler fix. -- This is an au

[GitHub] [arrow-datafusion] houqp commented on issue #1818: A corner bug in union

2022-02-21 Thread GitBox
houqp commented on issue #1818: URL: https://github.com/apache/arrow-datafusion/issues/1818#issuecomment-1047364448 I agree, if this is only happening to union, realigning the column names for anonymous columns during union planning would be a better fix. -- This is an automated message

[GitHub] [arrow] github-actions[bot] commented on pull request #12481: ARROW-15746: [Java] Add arrow-flight pom artifact to tasks.yml

2022-02-21 Thread GitBox
github-actions[bot] commented on pull request #12481: URL: https://github.com/apache/arrow/pull/12481#issuecomment-1047361840 Revision: 9e52535f75e0f2535759c71c300015fc08d3b943 Submitted crossbow builds: [ursacomputing/crossbow @ actions-1666](https://github.com/ursacomputing/crossbo

[GitHub] [arrow] BryanCutler commented on pull request #12481: ARROW-15746: [Java] Add arrow-flight pom artifact to tasks.yml

2022-02-21 Thread GitBox
BryanCutler commented on pull request #12481: URL: https://github.com/apache/arrow/pull/12481#issuecomment-1047361401 @github-actions crossbow submit java-jars -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [arrow] lidavidm commented on issue #12473: [Flight] If the flight server and client are just in different processes one same server, can we avoid the data copies in arrow flight?

2022-02-21 Thread GitBox
lidavidm commented on issue #12473: URL: https://github.com/apache/arrow/issues/12473#issuecomment-1047354995 Interesting! Would love to hear more once it's ready. As for examples: yes, we are working on more for the Cookbook. See https://arrow.apache.org/cookbook/cpp/flight.html

[GitHub] [arrow-rs] codecov-commenter commented on pull request #1349: Add `async` to default features

2022-02-21 Thread GitBox
codecov-commenter commented on pull request #1349: URL: https://github.com/apache/arrow-rs/pull/1349#issuecomment-1047353602 # [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1349?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [arrow-rs] HaoYang670 opened a new pull request #1349: Add `async` to default features

2022-02-21 Thread GitBox
HaoYang670 opened a new pull request #1349: URL: https://github.com/apache/arrow-rs/pull/1349 Signed-off-by: remzi <1371656737...@gmail.com> # Which issue does this PR close? Closes #1307. # Rationale for this change # What changes are included i

[GitHub] [arrow] BryanCutler commented on a change in pull request #12481: ARROW-15746: [Java] Add arrow-flight pom artifact to tasks.yml

2022-02-21 Thread GitBox
BryanCutler commented on a change in pull request #12481: URL: https://github.com/apache/arrow/pull/12481#discussion_r811507118 ## File path: dev/tasks/tasks.yml ## @@ -856,6 +857,16 @@ tasks: - flight-grpc-{no_rc_version}-tests.jar - flight-grpc-{no_rc_version}.j

[GitHub] [arrow-datafusion] Jimexist commented on a change in pull request #1866: Add doc for building `datafusion-cli` when connect the ballista

2022-02-21 Thread GitBox
Jimexist commented on a change in pull request #1866: URL: https://github.com/apache/arrow-datafusion/pull/1866#discussion_r811504952 ## File path: datafusion-cli/README.md ## @@ -66,6 +66,17 @@ DataFusion CLI v4.0.0-SNAPSHOT ``` ## Ballista +If you want to execute the SQL

[GitHub] [arrow] github-actions[bot] commented on pull request #12481: ARROW-15746: [Java] Add arrow-flight pom artifact to tasks.yml

2022-02-21 Thread GitBox
github-actions[bot] commented on pull request #12481: URL: https://github.com/apache/arrow/pull/12481#issuecomment-1047323281 Revision: 72b02f8a8aaa8cee9ed3e7c78a1353e110f97139 Submitted crossbow builds: [ursacomputing/crossbow @ actions-1665](https://github.com/ursacomputing/crossbo

[GitHub] [arrow] BryanCutler commented on pull request #12481: ARROW-15746: [Java] Add arrow-flight pom artifact to tasks.yml

2022-02-21 Thread GitBox
BryanCutler commented on pull request #12481: URL: https://github.com/apache/arrow/pull/12481#issuecomment-1047322789 @github-actions crossbow submit java-jars -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [arrow] BryanCutler edited a comment on pull request #12481: ARROW-15746: [Java] Add arrow-flight pom artifact to tasks.yml

2022-02-21 Thread GitBox
BryanCutler edited a comment on pull request #12481: URL: https://github.com/apache/arrow/pull/12481#issuecomment-1047321044 ~~Hmm, something doesn't seem right. Still not picking up flight-sql and integration-tests.~~ Nvm, I see you have to submit the job first and then check the status

[GitHub] [arrow] kou commented on pull request #12481: ARROW-15746: [Java] Add arrow-flight pom artifact to tasks.yml

2022-02-21 Thread GitBox
kou commented on pull request #12481: URL: https://github.com/apache/arrow/pull/12481#issuecomment-1047322157 We need to re-run `@github-actions crossbow submit java-jars` after we change `tasks.yml`. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [arrow] BryanCutler commented on pull request #12481: ARROW-15746: [Java] Add arrow-flight pom artifact to tasks.yml

2022-02-21 Thread GitBox
BryanCutler commented on pull request #12481: URL: https://github.com/apache/arrow/pull/12481#issuecomment-1047321044 Hmm, something doesn't seem right. Still not picking up flight-sql and integration-tests -- This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [arrow] BryanCutler removed a comment on pull request #12481: ARROW-15746: [Java] Add arrow-flight pom artifact to tasks.yml

2022-02-21 Thread GitBox
BryanCutler removed a comment on pull request #12481: URL: https://github.com/apache/arrow/pull/12481#issuecomment-1047320331 Ok, I think it has all the missing artifacts now. This is the crossbow output ``` [ state] Task / Branch Artifac

[GitHub] [arrow] BryanCutler commented on pull request #12481: ARROW-15746: [Java] Add arrow-flight pom artifact to tasks.yml

2022-02-21 Thread GitBox
BryanCutler commented on pull request #12481: URL: https://github.com/apache/arrow/pull/12481#issuecomment-1047320331 Ok, I think it has all the missing artifacts now. This is the crossbow output ``` [ state] Task / Branch Artifacts -

[GitHub] [arrow] kou commented on a change in pull request #12457: ARROW-15709: [C++] Compilation of ARROW_ENGINE fails if doing an "inline" build

2022-02-21 Thread GitBox
kou commented on a change in pull request #12457: URL: https://github.com/apache/arrow/pull/12457#discussion_r811483696 ## File path: cpp/src/arrow/engine/CMakeLists.txt ## @@ -34,94 +28,32 @@ set(ARROW_ENGINE_SRCS substrait/relation_internal.cc substrait/type_interna

[GitHub] [arrow] BryanCutler commented on pull request #12481: ARROW-15746: [Java] Add arrow-flight pom artifact to tasks.yml

2022-02-21 Thread GitBox
BryanCutler commented on pull request #12481: URL: https://github.com/apache/arrow/pull/12481#issuecomment-1047305253 Ah, I thought I was supposed to create an empty crossbow repo. No wonder why I had issues :disappointed: I'll try it out with a fork from ursa, that should work. -- This

[GitHub] [arrow] kou commented on a change in pull request #12457: ARROW-15709: [C++] Compilation of ARROW_ENGINE fails if doing an "inline" build

2022-02-21 Thread GitBox
kou commented on a change in pull request #12457: URL: https://github.com/apache/arrow/pull/12457#discussion_r811483696 ## File path: cpp/src/arrow/engine/CMakeLists.txt ## @@ -34,94 +28,32 @@ set(ARROW_ENGINE_SRCS substrait/relation_internal.cc substrait/type_interna

[GitHub] [arrow] kou commented on pull request #12481: ARROW-15746: [Java] Add arrow-flight pom artifact to tasks.yml

2022-02-21 Thread GitBox
kou commented on pull request #12481: URL: https://github.com/apache/arrow/pull/12481#issuecomment-1047304646 Thanks. It seems that there are more missing entries: https://github.com/ursacomputing/crossbow/releases/nightly-2022-02-21-0-github-java-jars * `flight-integ

[GitHub] [arrow] kou commented on pull request #12481: ARROW-15746: [Java] Add arrow-flight pom artifact to tasks.yml

2022-02-21 Thread GitBox
kou commented on pull request #12481: URL: https://github.com/apache/arrow/pull/12481#issuecomment-1047303776 We can verify this by the following command lines: ```console $ git clone https://github.com/ursacomputing/crossbow.git $ archery crossbow --github-token "${GITHUB_TOKEN

[GitHub] [arrow] github-actions[bot] commented on pull request #12481: ARROW-15746: [Java] Add arrow-flight pom artifact to tasks.yml

2022-02-21 Thread GitBox
github-actions[bot] commented on pull request #12481: URL: https://github.com/apache/arrow/pull/12481#issuecomment-1047302715 Revision: af7d3e2ba9d1ad71c5766db1828dbb24adf3d673 Submitted crossbow builds: [ursacomputing/crossbow @ actions-1664](https://github.com/ursacomputing/crossbo

[GitHub] [arrow] kou commented on pull request #12481: ARROW-15746: [Java] Add arrow-flight pom artifact to tasks.yml

2022-02-21 Thread GitBox
kou commented on pull request #12481: URL: https://github.com/apache/arrow/pull/12481#issuecomment-1047302316 @github-actions crossbow submit java-jars -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] github-actions[bot] commented on pull request #12482: ARROW-15701 [R] month() should allow integer inputs

2022-02-21 Thread GitBox
github-actions[bot] commented on pull request #12482: URL: https://github.com/apache/arrow/pull/12482#issuecomment-1047265737 https://issues.apache.org/jira/browse/ARROW-15701 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [arrow-datafusion] andrei-ionescu opened a new pull request #1392: Fix index out of bounds for stats on nested fields

2022-02-21 Thread GitBox
andrei-ionescu opened a new pull request #1392: URL: https://github.com/apache/arrow-datafusion/pull/1392 # Which issue does this PR close? Closes #1383 # Rationale for this change This is a step in supporting nested fields in data fusion being read from parquet in

[GitHub] [arrow-datafusion] houqp commented on issue #1864: Add a DataFusion example capable of retrieving data from a custom datasource

2022-02-21 Thread GitBox
houqp commented on issue #1864: URL: https://github.com/apache/arrow-datafusion/issues/1864#issuecomment-1047235365 join is an operator that operates on two tables, but the table provider trait is an abstraction for a single table source, that's why it doesn't make sense to push down join

[GitHub] [arrow] sanjibansg commented on a change in pull request #12426: ARROW-15672: [C++] Enable CSV writer to control the field delimiter

2022-02-21 Thread GitBox
sanjibansg commented on a change in pull request #12426: URL: https://github.com/apache/arrow/pull/12426#discussion_r811418379 ## File path: cpp/src/arrow/csv/writer_test.cc ## @@ -233,7 +241,16 @@ std::vector GenerateTestCases() { reject_structural_params({"0123456789",

[GitHub] [arrow] sanjibansg commented on a change in pull request #12426: ARROW-15672: [C++] Enable CSV writer to control the field delimiter

2022-02-21 Thread GitBox
sanjibansg commented on a change in pull request #12426: URL: https://github.com/apache/arrow/pull/12426#discussion_r811397787 ## File path: python/pyarrow/tests/test_csv.py ## @@ -326,7 +326,7 @@ def test_write_options(): opts = cls() check_options_class( -

[GitHub] [arrow] jonkeane commented on pull request #12466: [C++] If a projected_schema is not supplied but a bound projection expression is then we should use that to infer the projected_schema

2022-02-21 Thread GitBox
jonkeane commented on pull request #12466: URL: https://github.com/apache/arrow/pull/12466#issuecomment-1047177853 It's not super easy to find, but the logs from the benchmark: https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/159#6bba5997-49a2-4405-b8ae-6172b8a

[GitHub] [arrow] jcralmeida commented on pull request #11982: ARROW-15313: [C++][Java][FlightRPC] Implement type info method to flight-sql

2022-02-21 Thread GitBox
jcralmeida commented on pull request #11982: URL: https://github.com/apache/arrow/pull/11982#issuecomment-1047170058 Thanks for the review @pitrou. I've already address/replied your comments. > Also, was this discussed on the ML to get some input? Yes, it was sent. David Li mad

[GitHub] [arrow] jcralmeida commented on a change in pull request #11982: ARROW-15313: [C++][Java][FlightRPC] Implement type info method to flight-sql

2022-02-21 Thread GitBox
jcralmeida commented on a change in pull request #11982: URL: https://github.com/apache/arrow/pull/11982#discussion_r811382921 ## File path: cpp/src/arrow/flight/sql/example/sqlite_type_info.cc ## @@ -0,0 +1,116 @@ +// Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [arrow] jcralmeida commented on a change in pull request #11982: ARROW-15313: [C++][Java][FlightRPC] Implement type info method to flight-sql

2022-02-21 Thread GitBox
jcralmeida commented on a change in pull request #11982: URL: https://github.com/apache/arrow/pull/11982#discussion_r811382360 ## File path: cpp/src/arrow/flight/sql/example/sqlite_type_info.cc ## @@ -0,0 +1,116 @@ +// Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [arrow] jcralmeida commented on a change in pull request #11982: ARROW-15313: [C++][Java][FlightRPC] Implement type info method to flight-sql

2022-02-21 Thread GitBox
jcralmeida commented on a change in pull request #11982: URL: https://github.com/apache/arrow/pull/11982#discussion_r811382226 ## File path: cpp/src/arrow/flight/sql/example/sqlite_type_info.cc ## @@ -0,0 +1,116 @@ +// Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [arrow] jcralmeida commented on a change in pull request #11982: ARROW-15313: [C++][Java][FlightRPC] Implement type info method to flight-sql

2022-02-21 Thread GitBox
jcralmeida commented on a change in pull request #11982: URL: https://github.com/apache/arrow/pull/11982#discussion_r811382226 ## File path: cpp/src/arrow/flight/sql/example/sqlite_type_info.cc ## @@ -0,0 +1,116 @@ +// Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [arrow] jcralmeida commented on a change in pull request #11982: ARROW-15313: [C++][Java][FlightRPC] Implement type info method to flight-sql

2022-02-21 Thread GitBox
jcralmeida commented on a change in pull request #11982: URL: https://github.com/apache/arrow/pull/11982#discussion_r811381934 ## File path: cpp/src/arrow/flight/sql/example/sqlite_type_info.cc ## @@ -0,0 +1,116 @@ +// Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [arrow] jcralmeida commented on a change in pull request #11982: ARROW-15313: [C++][Java][FlightRPC] Implement type info method to flight-sql

2022-02-21 Thread GitBox
jcralmeida commented on a change in pull request #11982: URL: https://github.com/apache/arrow/pull/11982#discussion_r811378519 ## File path: cpp/src/arrow/flight/sql/client.h ## @@ -135,6 +135,19 @@ class ARROW_EXPORT FlightSqlClient { arrow::Result> GetTableTypes( co

[GitHub] [arrow] pitrou commented on a change in pull request #12457: ARROW-15709: [C++] Compilation of ARROW_ENGINE fails if doing an "inline" build

2022-02-21 Thread GitBox
pitrou commented on a change in pull request #12457: URL: https://github.com/apache/arrow/pull/12457#discussion_r811345577 ## File path: cpp/src/arrow/engine/CMakeLists.txt ## @@ -34,94 +28,32 @@ set(ARROW_ENGINE_SRCS substrait/relation_internal.cc substrait/type_inte

[GitHub] [arrow] pitrou commented on a change in pull request #12457: ARROW-15709: [C++] Compilation of ARROW_ENGINE fails if doing an "inline" build

2022-02-21 Thread GitBox
pitrou commented on a change in pull request #12457: URL: https://github.com/apache/arrow/pull/12457#discussion_r811344965 ## File path: cpp/cmake_modules/ThirdpartyToolchain.cmake ## @@ -610,6 +620,14 @@ else() endif() endif() +if(DEFINED ENV{ARROW_SUBSTRAIT_URL}) + set

[GitHub] [arrow] pitrou commented on pull request #11982: ARROW-15313: [C++][Java][FlightRPC] Implement type info method to flight-sql

2022-02-21 Thread GitBox
pitrou commented on pull request #11982: URL: https://github.com/apache/arrow/pull/11982#issuecomment-1047116142 cc @emkornfield for the Java bits. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [arrow] pitrou edited a comment on pull request #11982: ARROW-15313: [C++][Java][FlightRPC] Implement type info method to flight-sql

2022-02-21 Thread GitBox
pitrou edited a comment on pull request #11982: URL: https://github.com/apache/arrow/pull/11982#issuecomment-1047114887 Also, was this discussed on the ML to get some input? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [arrow] pitrou commented on pull request #11982: ARROW-15313: [C++][Java][FlightRPC] Implement type info method to flight-sql

2022-02-21 Thread GitBox
pitrou commented on pull request #11982: URL: https://github.com/apache/arrow/pull/11982#issuecomment-1047114887 Also, what this discussed on the ML to get some input? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [arrow] BryanCutler commented on pull request #12481: ARROW-15746: [Java] Add arrow-flight pom artifact to tasks.yml

2022-02-21 Thread GitBox
BryanCutler commented on pull request #12481: URL: https://github.com/apache/arrow/pull/12481#issuecomment-1047113662 @kou or @kszucs has there been any thought on making this list of java artifacts dynamically so we don't forget to add something in the future? -- This is an automated me

[GitHub] [arrow] BryanCutler commented on pull request #12481: ARROW-15746: [Java] Add arrow-flight pom artifact to tasks.yml

2022-02-21 Thread GitBox
BryanCutler commented on pull request #12481: URL: https://github.com/apache/arrow/pull/12481#issuecomment-1047112870 I would like to run this to verify, but I couldn't get crossbow setup correctly and keep getting errors. If there is an easier way to test this out, let me know. -- This

[GitHub] [arrow] pitrou commented on a change in pull request #11982: ARROW-15313: [C++][Java][FlightRPC] Implement type info method to flight-sql

2022-02-21 Thread GitBox
pitrou commented on a change in pull request #11982: URL: https://github.com/apache/arrow/pull/11982#discussion_r811336662 ## File path: cpp/src/arrow/flight/sql/example/sqlite_type_info.cc ## @@ -0,0 +1,116 @@ +// Licensed to the Apache Software Foundation (ASF) under one +//

[GitHub] [arrow] github-actions[bot] commented on pull request #12481: ARROW-15746: [Java] Add arrow-flight pom artifact to tasks.yml

2022-02-21 Thread GitBox
github-actions[bot] commented on pull request #12481: URL: https://github.com/apache/arrow/pull/12481#issuecomment-1047109296 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [arrow] BryanCutler opened a new pull request #12481: ARROW-15746: [Java] Add arrow-flight pom artifact to tasks.yml

2022-02-21 Thread GitBox
BryanCutler opened a new pull request #12481: URL: https://github.com/apache/arrow/pull/12481 This adds the new arrow-flight parent pom to the list of artifacts to deploy in tasks.yml -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [arrow] pitrou commented on a change in pull request #11982: ARROW-15313: [C++][Java][FlightRPC] Implement type info method to flight-sql

2022-02-21 Thread GitBox
pitrou commented on a change in pull request #11982: URL: https://github.com/apache/arrow/pull/11982#discussion_r811334683 ## File path: format/FlightSql.proto ## @@ -867,6 +867,167 @@ enum SqlSupportsConvert { SQL_CONVERT_VARCHAR = 19; } +enum SqlDataType { + SQL_TYPE_U

[GitHub] [arrow] dragosmg commented on a change in pull request #12431: ARROW-14826 [R] Implement bindings for `lubridate::dst()`

2022-02-21 Thread GitBox
dragosmg commented on a change in pull request #12431: URL: https://github.com/apache/arrow/pull/12431#discussion_r811291710 ## File path: r/tests/testthat/test-dplyr-funcs-datetime.R ## @@ -711,3 +711,40 @@ test_that("am/pm mirror lubridate", { ) }) +test_that("dst extra

[GitHub] [arrow] pitrou commented on a change in pull request #11982: ARROW-15313: [C++][Java][FlightRPC] Implement type info method to flight-sql

2022-02-21 Thread GitBox
pitrou commented on a change in pull request #11982: URL: https://github.com/apache/arrow/pull/11982#discussion_r811330096 ## File path: cpp/src/arrow/flight/sql/client.h ## @@ -135,6 +135,19 @@ class ARROW_EXPORT FlightSqlClient { arrow::Result> GetTableTypes( const

[GitHub] [arrow] pitrou commented on a change in pull request #11982: ARROW-15313: [C++][Java][FlightRPC] Implement type info method to flight-sql

2022-02-21 Thread GitBox
pitrou commented on a change in pull request #11982: URL: https://github.com/apache/arrow/pull/11982#discussion_r811329732 ## File path: cpp/src/arrow/flight/sql/client.h ## @@ -135,6 +135,19 @@ class ARROW_EXPORT FlightSqlClient { arrow::Result> GetTableTypes( const

[GitHub] [arrow] pitrou commented on a change in pull request #11982: ARROW-15313: [C++][Java][FlightRPC] Implement type info method to flight-sql

2022-02-21 Thread GitBox
pitrou commented on a change in pull request #11982: URL: https://github.com/apache/arrow/pull/11982#discussion_r811328594 ## File path: cpp/src/arrow/flight/sql/example/sqlite_type_info.cc ## @@ -0,0 +1,116 @@ +// Licensed to the Apache Software Foundation (ASF) under one +//

[GitHub] [arrow] pitrou commented on a change in pull request #11982: ARROW-15313: [C++][Java][FlightRPC] Implement type info method to flight-sql

2022-02-21 Thread GitBox
pitrou commented on a change in pull request #11982: URL: https://github.com/apache/arrow/pull/11982#discussion_r811328404 ## File path: cpp/src/arrow/flight/sql/example/sqlite_type_info.cc ## @@ -0,0 +1,116 @@ +// Licensed to the Apache Software Foundation (ASF) under one +//

[GitHub] [arrow] pitrou commented on a change in pull request #11982: ARROW-15313: [C++][Java][FlightRPC] Implement type info method to flight-sql

2022-02-21 Thread GitBox
pitrou commented on a change in pull request #11982: URL: https://github.com/apache/arrow/pull/11982#discussion_r811326482 ## File path: cpp/src/arrow/flight/sql/example/sqlite_type_info.cc ## @@ -0,0 +1,116 @@ +// Licensed to the Apache Software Foundation (ASF) under one +//

[GitHub] [arrow] pitrou commented on a change in pull request #11982: ARROW-15313: [C++][Java][FlightRPC] Implement type info method to flight-sql

2022-02-21 Thread GitBox
pitrou commented on a change in pull request #11982: URL: https://github.com/apache/arrow/pull/11982#discussion_r811326012 ## File path: cpp/src/arrow/flight/sql/example/sqlite_type_info.cc ## @@ -0,0 +1,116 @@ +// Licensed to the Apache Software Foundation (ASF) under one +//

[GitHub] [arrow] pitrou commented on a change in pull request #11982: ARROW-15313: [C++][Java][FlightRPC] Implement type info method to flight-sql

2022-02-21 Thread GitBox
pitrou commented on a change in pull request #11982: URL: https://github.com/apache/arrow/pull/11982#discussion_r811320485 ## File path: format/FlightSql.proto ## @@ -867,6 +867,185 @@ enum SqlSupportsConvert { SQL_CONVERT_VARCHAR = 19; } +/** + * It is an attribute that

[GitHub] [arrow-datafusion] Igosuki commented on issue #1785: Panic reading avro file at datafusion-6.0.0/src/avro_to_arrow/arrow_array_reader.rs:771:37

2022-02-21 Thread GitBox
Igosuki commented on issue #1785: URL: https://github.com/apache/arrow-datafusion/issues/1785#issuecomment-1047086840 @joshuarobinson if you want, you can use the arrow2 branch, it's the one I use to read avro -- This is an automated message from the Apache Git Service. To respond to th

[GitHub] [arrow] thisisnic commented on a change in pull request #12431: ARROW-14826 [R] Implement bindings for `lubridate::dst()`

2022-02-21 Thread GitBox
thisisnic commented on a change in pull request #12431: URL: https://github.com/apache/arrow/pull/12431#discussion_r811301516 ## File path: r/tests/testthat/test-dplyr-funcs-datetime.R ## @@ -711,3 +711,40 @@ test_that("am/pm mirror lubridate", { ) }) +test_that("dst extr

[GitHub] [arrow-cookbook] davisusanibar commented on a change in pull request #138: [Java]: Java cookbook for create arrow jni dataset

2022-02-21 Thread GitBox
davisusanibar commented on a change in pull request #138: URL: https://github.com/apache/arrow-cookbook/pull/138#discussion_r811292565 ## File path: java/source/dataset.rst ## @@ -0,0 +1,297 @@ +.. _arrow-dataset: + +=== +Dataset +=== + +* `Arrow Java Dataset`_: Java im

[GitHub] [arrow] dragosmg commented on a change in pull request #12431: ARROW-14826 [R] Implement bindings for `lubridate::dst()`

2022-02-21 Thread GitBox
dragosmg commented on a change in pull request #12431: URL: https://github.com/apache/arrow/pull/12431#discussion_r811291710 ## File path: r/tests/testthat/test-dplyr-funcs-datetime.R ## @@ -711,3 +711,40 @@ test_that("am/pm mirror lubridate", { ) }) +test_that("dst extra

[GitHub] [arrow] thisisnic commented on a change in pull request #12431: ARROW-14826 [R] Implement bindings for `lubridate::dst()`

2022-02-21 Thread GitBox
thisisnic commented on a change in pull request #12431: URL: https://github.com/apache/arrow/pull/12431#discussion_r811289144 ## File path: r/tests/testthat/test-dplyr-funcs-datetime.R ## @@ -711,3 +711,40 @@ test_that("am/pm mirror lubridate", { ) }) +test_that("dst extr

[GitHub] [arrow] hujianxin commented on issue #12473: [Flight] If the flight server and client are just in different processes one same server, can we avoid the data copies in arrow flight?

2022-02-21 Thread GitBox
hujianxin commented on issue #12473: URL: https://github.com/apache/arrow/issues/12473#issuecomment-1047046364 Do we have more examples besides `flight/integration_tests/test_integration_client.cc`,It's difficult to setup an demo flight serivce with custom service method for me 😄 -- Th

[GitHub] [arrow] jcralmeida commented on pull request #11982: ARROW-15313: [C++][Java][FlightRPC] Implement type info method to flight-sql

2022-02-21 Thread GitBox
jcralmeida commented on pull request #11982: URL: https://github.com/apache/arrow/pull/11982#issuecomment-1047045688 Any other concerns about the PRs @pitrou? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [arrow] rtpsw commented on pull request #12338: ARROW-15571: [C++] Add min/max binary scalar kernels to execution engine

2022-02-21 Thread GitBox
rtpsw commented on pull request #12338: URL: https://github.com/apache/arrow/pull/12338#issuecomment-1047044901 Thanks @cyb70289 ! @lidavidm: What do you think remains to be done? Should I add unit tests like in https://github.com/apache/arrow/pull/12412/ ? Or perhaps the binary sca

[GitHub] [arrow-datafusion] GavinRay97 edited a comment on issue #1864: Add a DataFusion example capable of retrieving data from a custom datasource

2022-02-21 Thread GitBox
GavinRay97 edited a comment on issue #1864: URL: https://github.com/apache/arrow-datafusion/issues/1864#issuecomment-1047043617 I had a read through the source of the `TableProvider` trait here: https://github.com/apache/arrow-datafusion/blob/0b1bef123acc5f46f225950d13f296104a543727/dat

[GitHub] [arrow-datafusion] GavinRay97 edited a comment on issue #1864: Add a DataFusion example capable of retrieving data from a custom datasource

2022-02-21 Thread GitBox
GavinRay97 edited a comment on issue #1864: URL: https://github.com/apache/arrow-datafusion/issues/1864#issuecomment-1047043617 I had a read through the source of the `TableProvider` trait here: https://github.com/apache/arrow-datafusion/blob/0b1bef123acc5f46f225950d13f296104a543727/dat

[GitHub] [arrow-datafusion] GavinRay97 commented on issue #1864: Add a DataFusion example capable of retrieving data from a custom datasource

2022-02-21 Thread GitBox
GavinRay97 commented on issue #1864: URL: https://github.com/apache/arrow-datafusion/issues/1864#issuecomment-1047043617 I had a read through the source of the `TableProvider` trait here: https://github.com/apache/arrow-datafusion/blob/0b1bef123acc5f46f225950d13f296104a543727/datafusion

[GitHub] [arrow] hujianxin commented on issue #12473: [Flight] If the flight server and client are just in different processes one same server, can we avoid the data copies in arrow flight?

2022-02-21 Thread GitBox
hujianxin commented on issue #12473: URL: https://github.com/apache/arrow/issues/12473#issuecomment-1047037048 Our project is a kudu/hbase-like storage system, we need to provide a native client for calculate engine to accelerate the data reading for it. And the calculate engine is deploye

[GitHub] [arrow] hujianxin commented on issue #12473: [Flight] If the flight server and client are just in different processes one same server, can we avoid the data copies in arrow flight?

2022-02-21 Thread GitBox
hujianxin commented on issue #12473: URL: https://github.com/apache/arrow/issues/12473#issuecomment-1047034705 Our work maybe open source this year or next year, Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [arrow] hujianxin edited a comment on issue #12473: [Flight] If the flight server and client are just in different processes one same server, can we avoid the data copies in arrow flight?

2022-02-21 Thread GitBox
hujianxin edited a comment on issue #12473: URL: https://github.com/apache/arrow/issues/12473#issuecomment-1047032327 And thanks for the advice on UCX, I'll consider your suggestion carefully. -- This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [arrow] hujianxin edited a comment on issue #12473: [Flight] If the flight server and client are just in different processes one same server, can we avoid the data copies in arrow flight?

2022-02-21 Thread GitBox
hujianxin edited a comment on issue #12473: URL: https://github.com/apache/arrow/issues/12473#issuecomment-1047031427 @lidavidm I'm investigating both ArrowFlight and Plasma-like plans. For plasma-like plan, I may do this like: 1. Setup a grpc service in our storage server. 2. C

[GitHub] [arrow] hujianxin commented on issue #12473: [Flight] If the flight server and client are just in different processes one same server, can we avoid the data copies in arrow flight?

2022-02-21 Thread GitBox
hujianxin commented on issue #12473: URL: https://github.com/apache/arrow/issues/12473#issuecomment-1047032327 And thanks for the advice on UCS, I'll consider your suggestion carefully. -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [arrow] hujianxin commented on issue #12473: [Flight] If the flight server and client are just in different processes one same server, can we avoid the data copies in arrow flight?

2022-02-21 Thread GitBox
hujianxin commented on issue #12473: URL: https://github.com/apache/arrow/issues/12473#issuecomment-1047031427 @lidavidm I'm investigating both ArrowFlight and Plasma-like plans. For plasma-like plan, I may do this like: 1. Setup a grpc service in our storage server. 2. Client c

[GitHub] [arrow-datafusion] matthewmturner commented on issue #1777: Improve DataFusions ability to write files

2022-02-21 Thread GitBox
matthewmturner commented on issue #1777: URL: https://github.com/apache/arrow-datafusion/issues/1777#issuecomment-104603 From what I see it looks like only the execution context can write files right now - let me know if im mistaken. I think it makes sense to add write functionality

[GitHub] [arrow] dragosmg commented on a change in pull request #12429: ARROW-14815 [R] bindings for `lubridate::semester()`

2022-02-21 Thread GitBox
dragosmg commented on a change in pull request #12429: URL: https://github.com/apache/arrow/pull/12429#discussion_r811219485 ## File path: r/R/dplyr-funcs-datetime.R ## @@ -115,6 +115,23 @@ register_bindings_datetime <- function() { return(Expression$create("strftime", x

[GitHub] [arrow-cookbook] lidavidm commented on a change in pull request #138: [Java]: Java cookbook for create arrow jni dataset

2022-02-21 Thread GitBox
lidavidm commented on a change in pull request #138: URL: https://github.com/apache/arrow-cookbook/pull/138#discussion_r81126 ## File path: java/source/dataset.rst ## @@ -0,0 +1,297 @@ +.. _arrow-dataset: + +=== +Dataset +=== + +* `Arrow Java Dataset`_: Java impleme

[GitHub] [arrow-cookbook] lidavidm merged pull request #136: [Java]: Java cookbook for create arrow read/write IPC format

2022-02-21 Thread GitBox
lidavidm merged pull request #136: URL: https://github.com/apache/arrow-cookbook/pull/136 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-u

[GitHub] [arrow] lidavidm commented on issue #12473: [Flight] If the flight server and client are just in different processes one same server, can we avoid the data copies in arrow flight?

2022-02-21 Thread GitBox
lidavidm commented on issue #12473: URL: https://github.com/apache/arrow/issues/12473#issuecomment-1046955888 BTW: is this work open source in any way? It would be nice to see more examples of Flight being used. And/or depending on the details it would help plan out how to support this. (I

[GitHub] [arrow] lidavidm commented on pull request #12338: ARROW-15571: [C++] Add min/max binary scalar kernels to execution engine

2022-02-21 Thread GitBox
lidavidm commented on pull request #12338: URL: https://github.com/apache/arrow/pull/12338#issuecomment-1046948297 @cyb70289 thanks for checking this! It should be possible to specialize the variadic kernel for this case (I haven't gotten a chance to look at the details yet though) -- T

[GitHub] [arrow] github-actions[bot] commented on pull request #12480: ARROW-15744: [Gandiva][C++] Add NEGATIVE function for interval types

2022-02-21 Thread GitBox
github-actions[bot] commented on pull request #12480: URL: https://github.com/apache/arrow/pull/12480#issuecomment-1046936127 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [arrow] github-actions[bot] commented on pull request #12480: [Gandiva][C++] Add NEGATIVE function for interval types

2022-02-21 Thread GitBox
github-actions[bot] commented on pull request #12480: URL: https://github.com/apache/arrow/pull/12480#issuecomment-1046932465 Thanks for opening a pull request! If this is not a [minor PR](https://github.com/apache/arrow/blob/master/CONTRIBUTING.md#Minor-Fixes). Could you op

[GitHub] [arrow] lidavidm commented on issue #12473: [Flight] If the flight server and client are just in different processes one same server, can we avoid the data copies in arrow flight?

2022-02-21 Thread GitBox
lidavidm commented on issue #12473: URL: https://github.com/apache/arrow/issues/12473#issuecomment-1046923910 Interesting. We are working on a UCX (https://github.com/openucx/ucx) backend, which I believe has optimizations for intraprocess data transfer. I'll investigate this more thorough

[GitHub] [arrow] dragosmg commented on a change in pull request #12429: ARROW-14815 [R] bindings for `lubridate::semester()`

2022-02-21 Thread GitBox
dragosmg commented on a change in pull request #12429: URL: https://github.com/apache/arrow/pull/12429#discussion_r811156508 ## File path: r/tests/testthat/test-dplyr-funcs-datetime.R ## @@ -711,3 +711,40 @@ test_that("am/pm mirror lubridate", { ) }) + +test_that("semeste

  1   2   >