[GitHub] [arrow-datafusion] Jimexist commented on pull request #622: fix 621, where unnamed window functions shall be differentiated by partition and order by clause

2021-06-25 Thread GitBox
Jimexist commented on pull request #622: URL: https://github.com/apache/arrow-datafusion/pull/622#issuecomment-868956823 if we are okay with dup names then i guess just keeping the function name would be enough -- This is an automated message from the Apache Git Service. To respond to th

[GitHub] [arrow-datafusion] Jimexist commented on pull request #622: fix 621, where unnamed window functions shall be differentiated by partition and order by clause

2021-06-25 Thread GitBox
Jimexist commented on pull request #622: URL: https://github.com/apache/arrow-datafusion/pull/622#issuecomment-868956556 Postgres is totally okay with dup names in the projection phase: ``` [postgres] # select max(c1) over (), max(c1) over (), max(c1) over (partition by c1) from t

[GitHub] [arrow-datafusion] Jimexist edited a comment on pull request #622: fix 621, where unnamed window functions shall be differentiated by partition and order by clause

2021-06-25 Thread GitBox
Jimexist edited a comment on pull request #622: URL: https://github.com/apache/arrow-datafusion/pull/622#issuecomment-868956435 > should we update physical column name as well for better readability in the query output? https://github.com/apache/arrow-datafusion/blob/master/datafusion/src/

[GitHub] [arrow-datafusion] Jimexist commented on pull request #622: fix 621, where unnamed window functions shall be differentiated by partition and order by clause

2021-06-25 Thread GitBox
Jimexist commented on pull request #622: URL: https://github.com/apache/arrow-datafusion/pull/622#issuecomment-868956435 > should we update physical column name as well for better readability in the query output? https://github.com/apache/arrow-datafusion/blob/master/datafusion/src/physica

[GitHub] [arrow-datafusion] Dandandan commented on a change in pull request #599: try to create a scheduled security audit every week

2021-06-25 Thread GitBox
Dandandan commented on a change in pull request #599: URL: https://github.com/apache/arrow-datafusion/pull/599#discussion_r659125692 ## File path: .github/workflows/audit.yml ## @@ -0,0 +1,33 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributo

[GitHub] [arrow-datafusion] houqp commented on pull request #622: fix 621, where unnamed window functions shall be differentiated by partition and order by clause

2021-06-25 Thread GitBox
houqp commented on pull request #622: URL: https://github.com/apache/arrow-datafusion/pull/622#issuecomment-868953367 should we update physical column name as well for better readability in the query output? https://github.com/apache/arrow-datafusion/blob/master/datafusion/src/physical_pla

[GitHub] [arrow] kou commented on pull request #10404: ARROW-12876: [R] Fix build flags on Raspberry Pi

2021-06-25 Thread GitBox
kou commented on pull request #10404: URL: https://github.com/apache/arrow/pull/10404#issuecomment-868951528 Can we see the build error log without this change? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [arrow] kou closed pull request #10305: ARROW-12730: [MATLAB] Update featherreadmex and featherwritemex to build against latest Arrow C++ APIs

2021-06-25 Thread GitBox
kou closed pull request #10305: URL: https://github.com/apache/arrow/pull/10305 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...

[GitHub] [arrow] kou commented on a change in pull request #10305: ARROW-12730: [MATLAB] Update featherreadmex and featherwritemex to build against latest Arrow C++ APIs

2021-06-25 Thread GitBox
kou commented on a change in pull request #10305: URL: https://github.com/apache/arrow/pull/10305#discussion_r659122423 ## File path: matlab/src/feather_reader.cc ## @@ -177,32 +182,32 @@ Status FeatherReader::Open(const std::string& filename, *feather_reader = std::shared_p

[GitHub] [arrow-datafusion] Jimexist opened a new pull request #622: fix 621, where unnamed window functions shall be differentiated by partition and order by clause

2021-06-25 Thread GitBox
Jimexist opened a new pull request #622: URL: https://github.com/apache/arrow-datafusion/pull/622 # Which issue does this PR close? Closes #621 # Rationale for this change # What changes are included in this PR? # Are there any user-facing changes?

[GitHub] [arrow-datafusion] Jimexist opened a new issue #621: unqualified window functions shall differentiate given partition and order by clause

2021-06-25 Thread GitBox
Jimexist opened a new issue #621: URL: https://github.com/apache/arrow-datafusion/issues/621 **Describe the bug** given window functions on the same col with same function but different partition by and order by, the schema is failing due to conflict. **To Reproduce** Steps

[GitHub] [arrow-datafusion] Jimexist commented on pull request #610: fix 593, reduce cloning by taking ownership in logical planner's `from` fn

2021-06-25 Thread GitBox
Jimexist commented on pull request #610: URL: https://github.com/apache/arrow-datafusion/pull/610#issuecomment-868940370 > If we really want to make this backwards compatible, we could use `impl Into` as the argument type, see #615. thanks for the suggestion. unless it's a goal, i'd

[GitHub] [arrow] edponce commented on a change in pull request #10349: ARROW-12744: [C++][Compute] Add rounding kernel

2021-06-25 Thread GitBox
edponce commented on a change in pull request #10349: URL: https://github.com/apache/arrow/pull/10349#discussion_r659082281 ## File path: cpp/src/arrow/compute/kernels/scalar_arithmetic.cc ## @@ -454,6 +456,159 @@ struct PowerChecked { } }; +using RoundState = internal::O

[GitHub] [arrow] emkornfield edited a comment on pull request #10603: ARROW-13191: [Go] allow external schema in ipc readers

2021-06-25 Thread GitBox
emkornfield edited a comment on pull request #10603: URL: https://github.com/apache/arrow/pull/10603#issuecomment-868881296 @zeroshade do you have thoughts here? Being able to pass in an external schema is useful in some contexts. Would it be workable to have the following logic:

[GitHub] [arrow] emkornfield commented on pull request #10603: ARROW-13191: [Go] allow external schema in ipc readers

2021-06-25 Thread GitBox
emkornfield commented on pull request #10603: URL: https://github.com/apache/arrow/pull/10603#issuecomment-868881296 @zeroshade do you have thoughts here? Being able to pass in an external schema is useful in some contexts. Would it be workable to have the following logic: ```

[GitHub] [arrow] shollyman edited a comment on pull request #10603: ARROW-13191: [Go] allow external schema in ipc readers

2021-06-25 Thread GitBox
shollyman edited a comment on pull request #10603: URL: https://github.com/apache/arrow/pull/10603#issuecomment-868867445 The issue seems to be the change of expectations. The internal/arrdata writes schema messages, but validation also passes in an external schema (which is being used fo

[GitHub] [arrow] github-actions[bot] commented on pull request #10604: ARROW-13190: [C++] [Gandiva] Change behavior of INITCAP function

2021-06-25 Thread GitBox
github-actions[bot] commented on pull request #10604: URL: https://github.com/apache/arrow/pull/10604#issuecomment-868878483 https://issues.apache.org/jira/browse/ARROW-13190 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [arrow] anthonylouisbsb commented on pull request #10604: ARROW-13190: [C++] [Gandiva] Change behavior of INITCAP function

2021-06-25 Thread GitBox
anthonylouisbsb commented on pull request #10604: URL: https://github.com/apache/arrow/pull/10604#issuecomment-868878502 @augustoasilva, please, could you review the PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [arrow] anthonylouisbsb opened a new pull request #10604: ARROW-13190: [C++] [Gandiva] Change behavior of INITCAP function

2021-06-25 Thread GitBox
anthonylouisbsb opened a new pull request #10604: URL: https://github.com/apache/arrow/pull/10604 The current behavior of the INITCAP function is to turn the first character of each word uppercase and remains the other as is. The desired behavior is to turn the first letter uppercase

[GitHub] [arrow-datafusion] Dandandan opened a new pull request #620: [WIP] Optimize count(*) with table statistics

2021-06-25 Thread GitBox
Dandandan opened a new pull request #620: URL: https://github.com/apache/arrow-datafusion/pull/620 # Which issue does this PR close? Closes #. # Rationale for this change # What changes are included in this PR? # Are there any user-facing changes?

[GitHub] [arrow] shollyman commented on pull request #10603: ARROW-13191: [Go] allow external schema in ipc readers

2021-06-25 Thread GitBox
shollyman commented on pull request #10603: URL: https://github.com/apache/arrow/pull/10603#issuecomment-868867445 And now I try to work out arrow testing. :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [arrow] github-actions[bot] commented on pull request #10603: ARROW-13191: [Go] allow external schema in ipc readers

2021-06-25 Thread GitBox
github-actions[bot] commented on pull request #10603: URL: https://github.com/apache/arrow/pull/10603#issuecomment-868866471 https://issues.apache.org/jira/browse/ARROW-13191 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [arrow] shollyman opened a new pull request #10603: ARROW-13191: [Go] allow external schema in ipc readers

2021-06-25 Thread GitBox
shollyman opened a new pull request #10603: URL: https://github.com/apache/arrow/pull/10603 This slightly changes the newReader to accept passed in schema as I believe is the intent. It also removes the existing behavior where the newReader uses a passed in schema merely for com

[GitHub] [arrow] ianmcook closed pull request #10563: ARROW-13117: [R] Retain schema in new Expressions

2021-06-25 Thread GitBox
ianmcook closed pull request #10563: URL: https://github.com/apache/arrow/pull/10563 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubs

[GitHub] [arrow] westonpace commented on pull request #10602: ARROW-13173: [C++] TestAsyncUtil.ReadaheadFailed asserts occasionally

2021-06-25 Thread GitBox
westonpace commented on pull request #10602: URL: https://github.com/apache/arrow/pull/10602#issuecomment-868823151 Ok, I think I got it this time. It turned out to be a little trickier. AsyncGenerator was using AddCallback instead of Then and there are no sequencing guarantees with AddC

[GitHub] [arrow] github-actions[bot] commented on pull request #10593: ARROW-13113: [R] use RTasks to manage parallel in converting arrow to R

2021-06-25 Thread GitBox
github-actions[bot] commented on pull request #10593: URL: https://github.com/apache/arrow/pull/10593#issuecomment-868820973 Revision: a0f18640e86ccd4f0ef4a55ece186dc614bc7559 Submitted crossbow builds: [ursacomputing/crossbow @ actions-511](https://github.com/ursacomputing/crossbow/

[GitHub] [arrow] nealrichardson commented on pull request #10593: ARROW-13113: [R] use RTasks to manage parallel in converting arrow to R

2021-06-25 Thread GitBox
nealrichardson commented on pull request #10593: URL: https://github.com/apache/arrow/pull/10593#issuecomment-868820557 @github-actions crossbow submit -g r -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [arrow] nealrichardson commented on pull request #10404: ARROW-12876: [R] Fix build flags on Raspberry Pi

2021-06-25 Thread GitBox
nealrichardson commented on pull request #10404: URL: https://github.com/apache/arrow/pull/10404#issuecomment-868820032 @kou @westonpace any objection to merging the R-only fix for this and ticketing a C++ followup to do it right? Or can one of you push a fix to cmake that does this the wa

[GitHub] [arrow] lidavidm commented on a change in pull request #9368: [WIP] [POC] Flight SQL

2021-06-25 Thread GitBox
lidavidm commented on a change in pull request #9368: URL: https://github.com/apache/arrow/pull/9368#discussion_r658934729 ## File path: format/FlightSQL.proto ## @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor licens

[GitHub] [arrow] github-actions[bot] commented on pull request #10602: ARROW-13173: [C++] TestAsyncUtil.ReadaheadFailed asserts occasionally

2021-06-25 Thread GitBox
github-actions[bot] commented on pull request #10602: URL: https://github.com/apache/arrow/pull/10602#issuecomment-868728119 https://issues.apache.org/jira/browse/ARROW-13173 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [arrow] westonpace opened a new pull request #10602: ARROW-13173: [C++] TestAsyncUtil.ReadaheadFailed asserts occasionally

2021-06-25 Thread GitBox
westonpace opened a new pull request #10602: URL: https://github.com/apache/arrow/pull/10602 As @cyb70289 pointed out the test was dependent on timing and when running on a slow CI machine it could lead to failure. I changed the test to use condition variables instead of sleeps so that it

[GitHub] [arrow] lidavidm edited a comment on pull request #10557: ARROW-13064: [C++] Implement select ('case when') function for fixed-width types

2021-06-25 Thread GitBox
lidavidm edited a comment on pull request #10557: URL: https://github.com/apache/arrow/pull/10557#issuecomment-868726123 Note there's some code here for fixed-width types that duplicates what's in ARROW-9430/#10412. They should probably get unified. -- This is an automated message from t

[GitHub] [arrow] lidavidm commented on pull request #10412: ARROW-9430: [C++] Implement replace_with_mask kernel

2021-06-25 Thread GitBox
lidavidm commented on pull request #10412: URL: https://github.com/apache/arrow/pull/10412#issuecomment-868726398 Note there's some code here for handling fixed-width types that now duplicates what's in ARROW-13064/#10557. We should probably unify those at some point (after one or the othe

[GitHub] [arrow] lidavidm commented on pull request #10557: ARROW-13064: [C++] Implement select ('case when') function for fixed-width types

2021-06-25 Thread GitBox
lidavidm commented on pull request #10557: URL: https://github.com/apache/arrow/pull/10557#issuecomment-868726123 Note there's some code here for fixed-width types that duplicates what's in ARROW-9430. They should probably get unified. -- This is an automated message from the Apache Git

[GitHub] [arrow] kylepbit commented on a change in pull request #9368: [WIP] [POC] Flight SQL

2021-06-25 Thread GitBox
kylepbit commented on a change in pull request #9368: URL: https://github.com/apache/arrow/pull/9368#discussion_r658923471 ## File path: format/FlightSQL.proto ## @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor licens

[GitHub] [arrow-datafusion] houqp commented on a change in pull request #619: RFC: Do not prune out unnecessary columns with unqualified references

2021-06-25 Thread GitBox
houqp commented on a change in pull request #619: URL: https://github.com/apache/arrow-datafusion/pull/619#discussion_r658910893 ## File path: datafusion/src/optimizer/projection_push_down.rs ## @@ -75,9 +78,12 @@ fn get_projected_schema( // // we discard non-existing

[GitHub] [arrow-datafusion] houqp commented on a change in pull request #619: RFC: Do not prune out unnecessary columns with unqualified references

2021-06-25 Thread GitBox
houqp commented on a change in pull request #619: URL: https://github.com/apache/arrow-datafusion/pull/619#discussion_r658910893 ## File path: datafusion/src/optimizer/projection_push_down.rs ## @@ -75,9 +78,12 @@ fn get_projected_schema( // // we discard non-existing

[GitHub] [arrow-datafusion] houqp commented on a change in pull request #619: RFC: Do not prune out unnecessary columns with unqualified references

2021-06-25 Thread GitBox
houqp commented on a change in pull request #619: URL: https://github.com/apache/arrow-datafusion/pull/619#discussion_r658910893 ## File path: datafusion/src/optimizer/projection_push_down.rs ## @@ -75,9 +78,12 @@ fn get_projected_schema( // // we discard non-existing

[GitHub] [arrow-datafusion] houqp commented on a change in pull request #619: RFC: Do not prune out unnecessary columns with unqualified references

2021-06-25 Thread GitBox
houqp commented on a change in pull request #619: URL: https://github.com/apache/arrow-datafusion/pull/619#discussion_r658910526 ## File path: datafusion/src/optimizer/projection_push_down.rs ## @@ -75,9 +78,12 @@ fn get_projected_schema( // // we discard non-existing

[GitHub] [arrow] kylepbit commented on pull request #9368: [WIP] [POC] Flight SQL

2021-06-25 Thread GitBox
kylepbit commented on pull request #9368: URL: https://github.com/apache/arrow/pull/9368#issuecomment-868694053 @lidavidm - that would be the goal. Am working to get this finalized, restrictions are just on time that can be made available. -- This is an automated message from the Apache

[GitHub] [arrow-rs] nevi-me closed issue #460: Implement RecordBatch::slice() to slice RecordBatches

2021-06-25 Thread GitBox
nevi-me closed issue #460: URL: https://github.com/apache/arrow-rs/issues/460 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@a

[GitHub] [arrow-rs] nevi-me merged pull request #490: Implement function slice for RecordBatch

2021-06-25 Thread GitBox
nevi-me merged pull request #490: URL: https://github.com/apache/arrow-rs/pull/490 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr

[GitHub] [arrow] anthonylouisbsb edited a comment on pull request #10195: ARROW-12595: [C++][Gandiva] Implement TO_HEX([binary] field), HEX, UNHEX and FROM_HEX([string]field] functions

2021-06-25 Thread GitBox
anthonylouisbsb edited a comment on pull request #10195: URL: https://github.com/apache/arrow/pull/10195#issuecomment-866789693 @jpedroantunes please, after @jvictorhuguenin applies the corrections, could you review it, please? -- This is an automated message from the Apache Git Service

[GitHub] [arrow] thisisnic commented on pull request #10588: ARROW-12868: [R] find_substring

2021-06-25 Thread GitBox
thisisnic commented on pull request #10588: URL: https://github.com/apache/arrow/pull/10588#issuecomment-868679514 > Meanwhile it looks like [ARROW-13157](https://issues.apache.org/jira/browse/ARROW-13157) has a PR, so if we wait a few days for that to get merged, then we can remove the sk

[GitHub] [arrow] ianmcook commented on pull request #10588: ARROW-12868: [R] find_substring

2021-06-25 Thread GitBox
ianmcook commented on pull request #10588: URL: https://github.com/apache/arrow/pull/10588#issuecomment-868670505 Meanwhile it looks like ARROW-13157 has a PR, so if we wait a few days for that to get merged, then we can remove the skipped test in this PR before we merge it. -- This is

[GitHub] [arrow] ianmcook commented on a change in pull request #10588: ARROW-12868: [R] find_substring

2021-06-25 Thread GitBox
ianmcook commented on a change in pull request #10588: URL: https://github.com/apache/arrow/pull/10588#discussion_r658876327 ## File path: r/tests/testthat/test-dplyr-string-functions.R ## @@ -725,6 +725,29 @@ test_that("errors in strptime", { ) }) +test_that("arrow_find_

[GitHub] [arrow] thisisnic commented on pull request #10588: ARROW-12868: [R] find_substring

2021-06-25 Thread GitBox
thisisnic commented on pull request #10588: URL: https://github.com/apache/arrow/pull/10588#issuecomment-868666882 Sounds like a plan! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [arrow] ianmcook commented on pull request #10588: ARROW-12868: [R] find_substring

2021-06-25 Thread GitBox
ianmcook commented on pull request #10588: URL: https://github.com/apache/arrow/pull/10588#issuecomment-868666454 >So, is the plan to merge this as-is, and then look at creating bindings for `str_locate` once the other tickets are implemented? Yes, if that sounds 👍 to you -- This

[GitHub] [arrow] lidavidm commented on pull request #9368: [WIP] [POC] Flight SQL

2021-06-25 Thread GitBox
lidavidm commented on pull request #9368: URL: https://github.com/apache/arrow/pull/9368#issuecomment-868657093 Thanks for picking this up again. Note the 5.0.0 timeline is soon-ish. It looks like we are nearing consensus on most issues raised so it would be helpful to see an updated docum

[GitHub] [arrow] lidavidm commented on a change in pull request #9368: [WIP] [POC] Flight SQL

2021-06-25 Thread GitBox
lidavidm commented on a change in pull request #9368: URL: https://github.com/apache/arrow/pull/9368#discussion_r658861293 ## File path: format/FlightSQL.proto ## @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor licens

[GitHub] [arrow] thisisnic commented on pull request #10588: ARROW-12868: [R] find_substring

2021-06-25 Thread GitBox
thisisnic commented on pull request #10588: URL: https://github.com/apache/arrow/pull/10588#issuecomment-868649192 @jonkeane @ianmcook I agree with what you both say here, that all makes sense. So, is the plan to merge this as-is, and then look at creating bindings for `str_locate` once t

[GitHub] [arrow-datafusion] alamb commented on issue #617: Projection pushdown removes unqualified column names even when they are used

2021-06-25 Thread GitBox
alamb commented on issue #617: URL: https://github.com/apache/arrow-datafusion/issues/617#issuecomment-868645886 https://github.com/apache/arrow-datafusion/pull/619 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [arrow-datafusion] alamb commented on pull request #619: RFC: Do not prune out unnecessary columns with unqualified references

2021-06-25 Thread GitBox
alamb commented on pull request #619: URL: https://github.com/apache/arrow-datafusion/pull/619#issuecomment-868645653 FYI @houqp -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [arrow-datafusion] alamb commented on a change in pull request #619: RFC: Do not prune out unnecessary columns with unqualified references

2021-06-25 Thread GitBox
alamb commented on a change in pull request #619: URL: https://github.com/apache/arrow-datafusion/pull/619#discussion_r658863004 ## File path: datafusion/src/optimizer/projection_push_down.rs ## @@ -75,9 +78,12 @@ fn get_projected_schema( // // we discard non-existing

[GitHub] [arrow-datafusion] alamb commented on a change in pull request #619: RFC: Do not prune out unnecessary columns with unqualified references

2021-06-25 Thread GitBox
alamb commented on a change in pull request #619: URL: https://github.com/apache/arrow-datafusion/pull/619#discussion_r658862205 ## File path: datafusion/src/optimizer/projection_push_down.rs ## @@ -75,9 +78,12 @@ fn get_projected_schema( // // we discard non-existing

[GitHub] [arrow-datafusion] alamb opened a new pull request #619: RFC: Do not prune out unnecessary columns with unqualified references

2021-06-25 Thread GitBox
alamb opened a new pull request #619: URL: https://github.com/apache/arrow-datafusion/pull/619 # Which issue does this PR close? Closes https://github.com/apache/arrow-datafusion/issues/617 but I am still not sure if this is a bug or not (explained below) # Rationale for this

[GitHub] [arrow] jonkeane commented on a change in pull request #10601: ARROW-13149 [R]: Convert named lists to structs instead of (unnamed) lists

2021-06-25 Thread GitBox
jonkeane commented on a change in pull request #10601: URL: https://github.com/apache/arrow/pull/10601#discussion_r658859592 ## File path: r/src/r_to_arrow.cpp ## @@ -1089,6 +1096,7 @@ class RStructConverter : public StructConverter { auto fields = this->struct_type_->f

[GitHub] [arrow] jonkeane commented on a change in pull request #10601: ARROW-13149 [R]: Convert named lists to structs instead of (unnamed) lists

2021-06-25 Thread GitBox
jonkeane commented on a change in pull request #10601: URL: https://github.com/apache/arrow/pull/10601#discussion_r658857664 ## File path: r/tests/testthat/test-metadata.R ## @@ -205,3 +205,47 @@ test_that("metadata of list elements (ARROW-10386)", { expect_identical(attr(as

[GitHub] [arrow] jonkeane commented on a change in pull request #10601: ARROW-13149 [R]: Convert named lists to structs instead of (unnamed) lists

2021-06-25 Thread GitBox
jonkeane commented on a change in pull request #10601: URL: https://github.com/apache/arrow/pull/10601#discussion_r658854104 ## File path: r/tests/testthat/test-json.R ## @@ -208,7 +208,7 @@ test_that("Can read json file with nested columns (ARROW-5503)", { expect_equal(str

[GitHub] [arrow-datafusion] alamb commented on issue #617: Projection pushdown removes unqualified column names even when they are used

2021-06-25 Thread GitBox
alamb commented on issue #617: URL: https://github.com/apache/arrow-datafusion/issues/617#issuecomment-868611882 Here is a reproducer (in projection_push_down.rs) -- it can also be found on https://github.com/alamb/arrow-datafusion/tree/alamb/repro_projection_pruning ```rust

[GitHub] [arrow] jonkeane commented on a change in pull request #10601: ARROW-13149 [R]: Convert named lists to structs instead of (unnamed) lists

2021-06-25 Thread GitBox
jonkeane commented on a change in pull request #10601: URL: https://github.com/apache/arrow/pull/10601#discussion_r658853111 ## File path: r/tests/testthat/test-Array.R ## @@ -839,3 +840,23 @@ test_that("Array to C-interface", { delete_arrow_schema(schema_ptr) delete_arro

[GitHub] [arrow] jonkeane commented on a change in pull request #10601: ARROW-13149 [R]: Convert named lists to structs instead of (unnamed) lists

2021-06-25 Thread GitBox
jonkeane commented on a change in pull request #10601: URL: https://github.com/apache/arrow/pull/10601#discussion_r658853111 ## File path: r/tests/testthat/test-Array.R ## @@ -839,3 +840,23 @@ test_that("Array to C-interface", { delete_arrow_schema(schema_ptr) delete_arro

[GitHub] [arrow] jonkeane commented on a change in pull request #10601: ARROW-13149 [R]: Convert named lists to structs instead of (unnamed) lists

2021-06-25 Thread GitBox
jonkeane commented on a change in pull request #10601: URL: https://github.com/apache/arrow/pull/10601#discussion_r658852266 ## File path: r/tests/testthat/latin1.R ## @@ -35,12 +35,12 @@ expect_identical(Encoding(levels(df[[2]])), "latin1") # Array expect_identical(as.vector

[GitHub] [arrow] jonkeane commented on a change in pull request #10601: ARROW-13149 [R]: Convert named lists to structs instead of (unnamed) lists

2021-06-25 Thread GitBox
jonkeane commented on a change in pull request #10601: URL: https://github.com/apache/arrow/pull/10601#discussion_r658850721 ## File path: r/src/type_infer.cpp ## @@ -133,8 +133,14 @@ static inline std::shared_ptr InferArrowTypeFromDataFrame( template <> std::shared_ptr In

[GitHub] [arrow] jonkeane commented on a change in pull request #10601: ARROW-13149 [R]: Convert named lists to structs instead of (unnamed) lists

2021-06-25 Thread GitBox
jonkeane commented on a change in pull request #10601: URL: https://github.com/apache/arrow/pull/10601#discussion_r658849841 ## File path: r/R/metadata.R ## @@ -56,7 +56,21 @@ apply_arrow_r_metadata <- function(x, r_metadata) { if (is.data.frame(x)) { if (length(nam

[GitHub] [arrow] jonkeane commented on a change in pull request #10601: ARROW-13149 [R]: Convert named lists to structs instead of (unnamed) lists

2021-06-25 Thread GitBox
jonkeane commented on a change in pull request #10601: URL: https://github.com/apache/arrow/pull/10601#discussion_r658849273 ## File path: r/src/array_to_vector.cpp ## @@ -723,9 +723,13 @@ class Converter_Struct : public Converter { auto colnames = arrow::r::to_r_strings(

[GitHub] [arrow] jonkeane commented on a change in pull request #10601: ARROW-13149 [R]: Convert named lists to structs instead of (unnamed) lists

2021-06-25 Thread GitBox
jonkeane commented on a change in pull request #10601: URL: https://github.com/apache/arrow/pull/10601#discussion_r658846244 ## File path: r/R/metadata.R ## @@ -56,7 +56,21 @@ apply_arrow_r_metadata <- function(x, r_metadata) { if (is.data.frame(x)) { if (length(nam

[GitHub] [arrow] ianmcook commented on a change in pull request #10563: ARROW-13117: [R] Retain schema in new Expressions

2021-06-25 Thread GitBox
ianmcook commented on a change in pull request #10563: URL: https://github.com/apache/arrow/pull/10563#discussion_r658842787 ## File path: r/tests/testthat/test-dplyr.R ## @@ -800,6 +808,21 @@ test_that("type checks with is_*()", { ) }) +test_that("type checks on expressi

[GitHub] [arrow] jonkeane commented on pull request #10601: ARROW-13149 [R]: Convert named lists to structs instead of (unnamed) lists

2021-06-25 Thread GitBox
jonkeane commented on pull request #10601: URL: https://github.com/apache/arrow/pull/10601#issuecomment-868570390 @github-actions autotune -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] sgilmore10 commented on a change in pull request #10305: ARROW-12730: [MATLAB] Update featherreadmex and featherwritemex to build against latest Arrow C++ APIs

2021-06-25 Thread GitBox
sgilmore10 commented on a change in pull request #10305: URL: https://github.com/apache/arrow/pull/10305#discussion_r658827138 ## File path: matlab/src/feather_reader.cc ## @@ -177,32 +182,32 @@ Status FeatherReader::Open(const std::string& filename, *feather_reader = std::s

[GitHub] [arrow] sgilmore10 commented on a change in pull request #10305: ARROW-12730: [MATLAB] Update featherreadmex and featherwritemex to build against latest Arrow C++ APIs

2021-06-25 Thread GitBox
sgilmore10 commented on a change in pull request #10305: URL: https://github.com/apache/arrow/pull/10305#discussion_r658819952 ## File path: matlab/src/feather_reader.cc ## @@ -52,11 +57,12 @@ mxArray* ReadNumericVariableData(const std::shared_ptr& column) { mxArray* variab

[GitHub] [arrow] lidavidm commented on a change in pull request #10579: ARROW-11932: [C++] Implement ArrayBuilder::AppendScalar

2021-06-25 Thread GitBox
lidavidm commented on a change in pull request #10579: URL: https://github.com/apache/arrow/pull/10579#discussion_r658801294 ## File path: cpp/src/arrow/array/builder_base.h ## @@ -116,6 +116,10 @@ class ARROW_EXPORT ArrayBuilder { /// This method is useful when appending nu

[GitHub] [arrow] rok edited a comment on pull request #10598: ARROW-13054: [C++] Add TemporalOptions

2021-06-25 Thread GitBox
rok edited a comment on pull request #10598: URL: https://github.com/apache/arrow/pull/10598#issuecomment-868522762 > However, we don't need to know all of these in advance, right? Indeed. Even the ones I listed above should be implemented just-in-time with their respective kernels I

[GitHub] [arrow] rok commented on pull request #10598: ARROW-13054: [C++] Add TemporalOptions

2021-06-25 Thread GitBox
rok commented on pull request #10598: URL: https://github.com/apache/arrow/pull/10598#issuecomment-868522762 > However, we don't need to know all of these in advance, right? Indeed. Even the ones I listed above should be implemented just-in-time with their respective kernels IMO. So

[GitHub] [arrow] sgilmore10 commented on a change in pull request #10305: ARROW-12730: [MATLAB] Update featherreadmex and featherwritemex to build against latest Arrow C++ APIs

2021-06-25 Thread GitBox
sgilmore10 commented on a change in pull request #10305: URL: https://github.com/apache/arrow/pull/10305#discussion_r658785123 ## File path: matlab/src/feather_reader.cc ## @@ -177,32 +182,32 @@ Status FeatherReader::Open(const std::string& filename, *feather_reader = std::s

[GitHub] [arrow] sgilmore10 commented on a change in pull request #10305: ARROW-12730: [MATLAB] Update featherreadmex and featherwritemex to build against latest Arrow C++ APIs

2021-06-25 Thread GitBox
sgilmore10 commented on a change in pull request #10305: URL: https://github.com/apache/arrow/pull/10305#discussion_r658778342 ## File path: matlab/CMakeLists.txt ## @@ -29,22 +30,51 @@ if(EXISTS "${CPP_CMAKE_MODULES}") set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${CPP_CMAKE_

[GitHub] [arrow] nastra commented on pull request #10600: ARROW-13172: Make TYPE_WIDTH publicly accessible

2021-06-25 Thread GitBox
nastra commented on pull request #10600: URL: https://github.com/apache/arrow/pull/10600#issuecomment-868511078 @emkornfield is this something that you could potentially review/commit? thanks in advance. -- This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [arrow] ianmcook closed pull request #10559: ARROW-13125: [R] Throw error when 2+ args passed to desc() in arrange()

2021-06-25 Thread GitBox
ianmcook closed pull request #10559: URL: https://github.com/apache/arrow/pull/10559 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubs

[GitHub] [arrow] lidavidm commented on a change in pull request #10544: ARROW-13095: [C++] Implement trig compute functions

2021-06-25 Thread GitBox
lidavidm commented on a change in pull request #10544: URL: https://github.com/apache/arrow/pull/10544#discussion_r658740181 ## File path: docs/source/cpp/compute.rst ## @@ -312,6 +312,40 @@ precision of `divide` is at least the sum of precisions of both operands with enough

[GitHub] [arrow] lidavidm commented on a change in pull request #10544: ARROW-13095: [C++] Implement trig compute functions

2021-06-25 Thread GitBox
lidavidm commented on a change in pull request #10544: URL: https://github.com/apache/arrow/pull/10544#discussion_r658740109 ## File path: cpp/src/arrow/compute/kernels/scalar_arithmetic.cc ## @@ -454,6 +462,200 @@ struct PowerChecked { } }; +struct Sin { + template +

[GitHub] [arrow] thisisnic edited a comment on pull request #10598: ARROW-13054: [C++] Add TemporalOptions

2021-06-25 Thread GitBox
thisisnic edited a comment on pull request #10598: URL: https://github.com/apache/arrow/pull/10598#issuecomment-868468794 > > About the actual options in this PR: that seems like a good start, but I don't think all those options would be needed in a single Options struct? For example, the

[GitHub] [arrow] lidavidm commented on a change in pull request #10544: ARROW-13095: [C++] Implement trig compute functions

2021-06-25 Thread GitBox
lidavidm commented on a change in pull request #10544: URL: https://github.com/apache/arrow/pull/10544#discussion_r658739906 ## File path: cpp/src/arrow/compute/kernels/scalar_arithmetic.cc ## @@ -454,6 +462,200 @@ struct PowerChecked { } }; +struct Sin { + template +

[GitHub] [arrow] lidavidm commented on a change in pull request #10349: ARROW-12744: [C++][Compute] Add rounding kernel

2021-06-25 Thread GitBox
lidavidm commented on a change in pull request #10349: URL: https://github.com/apache/arrow/pull/10349#discussion_r658733636 ## File path: cpp/src/arrow/compute/kernels/scalar_arithmetic.cc ## @@ -454,6 +456,159 @@ struct PowerChecked { } }; +using RoundState = internal::

[GitHub] [arrow] lidavidm commented on a change in pull request #10349: ARROW-12744: [C++][Compute] Add rounding kernel

2021-06-25 Thread GitBox
lidavidm commented on a change in pull request #10349: URL: https://github.com/apache/arrow/pull/10349#discussion_r658733299 ## File path: cpp/src/arrow/compute/kernels/scalar_arithmetic.cc ## @@ -454,6 +456,159 @@ struct PowerChecked { } }; +using RoundState = internal::

[GitHub] [arrow] thisisnic commented on pull request #10598: ARROW-13054: [C++] Add TemporalOptions

2021-06-25 Thread GitBox
thisisnic commented on pull request #10598: URL: https://github.com/apache/arrow/pull/10598#issuecomment-868468794 > > About the actual options in this PR: that seems like a good start, but I don't think all those options would be needed in a single Options struct? For example, the ambiguo

[GitHub] [arrow] github-actions[bot] commented on pull request #10601: ARROW-13149 [R]: Convert named lists to structs instead of (unnamed) lists

2021-06-25 Thread GitBox
github-actions[bot] commented on pull request #10601: URL: https://github.com/apache/arrow/pull/10601#issuecomment-868467415 https://issues.apache.org/jira/browse/ARROW-13149 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [arrow] jonkeane opened a new pull request #10601: ARROW-13149 [R]: Convert named lists to structs instead of (unnamed) lists

2021-06-25 Thread GitBox
jonkeane opened a new pull request #10601: URL: https://github.com/apache/arrow/pull/10601 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [arrow] rok commented on pull request #10598: ARROW-13054: [C++] Add TemporalOptions

2021-06-25 Thread GitBox
rok commented on pull request #10598: URL: https://github.com/apache/arrow/pull/10598#issuecomment-868449124 > Pandas actually has a different method (the one you link is an attribute that has been there for a long time, so but pandas added an additional method so it could have arguments)

[GitHub] [arrow] jorisvandenbossche commented on pull request #10598: ARROW-13054: [C++] Add TemporalOptions

2021-06-25 Thread GitBox
jorisvandenbossche commented on pull request #10598: URL: https://github.com/apache/arrow/pull/10598#issuecomment-868441918 > I was wondering about locale as well. Would it be a good idea to be able to set an arbitrary locale in some cases? [Pandas appears not to offer this option](https:/

[GitHub] [arrow] pitrou commented on pull request #10583: ARROW-11675: [CI][C++] Resolve ctest failures on VS 2019 builds

2021-06-25 Thread GitBox
pitrou commented on pull request #10583: URL: https://github.com/apache/arrow/pull/10583#issuecomment-868429468 TODO: perhaps add Spaced tests and specialization -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [arrow] edponce commented on a change in pull request #10349: ARROW-12744: [C++][Compute] Add rounding kernel

2021-06-25 Thread GitBox
edponce commented on a change in pull request #10349: URL: https://github.com/apache/arrow/pull/10349#discussion_r658690788 ## File path: cpp/src/arrow/compute/kernels/scalar_arithmetic.cc ## @@ -454,6 +456,159 @@ struct PowerChecked { } }; +using RoundState = internal::O

[GitHub] [arrow] edponce commented on pull request #10349: ARROW-12744: [C++][Compute] Add rounding kernel

2021-06-25 Thread GitBox
edponce commented on pull request #10349: URL: https://github.com/apache/arrow/pull/10349#issuecomment-868420891 @bkietz @jorisvandenbossche Need feedback on this PR. Specifically, the rounding options provided and kernel implementations. -- This is an automated message from the Apache G

[GitHub] [arrow] rok commented on pull request #10598: ARROW-13054: [C++] Add TemporalOptions

2021-06-25 Thread GitBox
rok commented on pull request #10598: URL: https://github.com/apache/arrow/pull/10598#issuecomment-868419790 > Again, I'd like input from others on this (I am not experienced in this and may be missing something!), but my initial thoughts are that if we do choose to extract the day of week

[GitHub] [arrow] thisisnic commented on pull request #10598: ARROW-13054: [C++] Add TemporalOptions

2021-06-25 Thread GitBox
thisisnic commented on pull request #10598: URL: https://github.com/apache/arrow/pull/10598#issuecomment-868407701 > > * `fiscal_start` - this is an argument to `quarter` which indicates the starting month of a fiscal year - I think this would be useful to implement > > Is this just

[GitHub] [arrow-datafusion] boaz-codota commented on pull request #342: Left join could use bitmap for left join instead of Vec

2021-06-25 Thread GitBox
boaz-codota commented on pull request #342: URL: https://github.com/apache/arrow-datafusion/pull/342#issuecomment-868400691 @alamb tried to refactor the code now, noticed that there is no `get_bit` equivalent of `set_bit` in `BooleanBufferBuilder` and calling `finish` to make receive a `Bu

[GitHub] [arrow] rok edited a comment on pull request #10598: ARROW-13054: [C++] Add TemporalOptions

2021-06-25 Thread GitBox
rok edited a comment on pull request #10598: URL: https://github.com/apache/arrow/pull/10598#issuecomment-868398341 Hey @thisisnic! Thanks for the input :). > * `fiscal_start` - this is an argument to `quarter` which indicates the starting month of a fiscal year - I think this would

[GitHub] [arrow] rok commented on pull request #10598: ARROW-13054: [C++] Add TemporalOptions

2021-06-25 Thread GitBox
rok commented on pull request #10598: URL: https://github.com/apache/arrow/pull/10598#issuecomment-868398341 Hey @thisisnic! Thanks for the input :). > * `fiscal_start` - this is an argument to `quarter` which indicates the starting month of a fiscal year - I think this would be usef

[GitHub] [arrow-datafusion] yjshen edited a comment on issue #616: Add support for reading distributed datasets (files on HDFS for example)

2021-06-25 Thread GitBox
yjshen edited a comment on issue #616: URL: https://github.com/apache/arrow-datafusion/issues/616#issuecomment-868364661 To make a digression, after searching on the Internet for a period of time, I found no suitable Rust HDFS client library that can be used directly for my use case. ☹️

[GitHub] [arrow-datafusion] yjshen commented on issue #616: Add support for reading distributed datasets (files on HDFS for example)

2021-06-25 Thread GitBox
yjshen commented on issue #616: URL: https://github.com/apache/arrow-datafusion/issues/616#issuecomment-868364661 To make a digression, after searching on the Internet for a period of time, I found no suitable Rust HDFS client library that can be used directly for my use case. ☹️ - [h

[GitHub] [arrow] thisisnic edited a comment on pull request #10598: ARROW-13054: [C++] Add TemporalOptions

2021-06-25 Thread GitBox
thisisnic edited a comment on pull request #10598: URL: https://github.com/apache/arrow/pull/10598#issuecomment-868358466 > Is this missing an option we will need in the near future? Should we remove something. > @jorisvandenbossche @thisisnic Thanks for this! I've taken a look at

  1   2   >