[GitHub] [arrow] sunchao commented on a change in pull request #9592: ARROW-11803: [Rust] [Parquet] Support v2 LogicalType

2021-02-27 Thread GitBox
sunchao commented on a change in pull request #9592: URL: https://github.com/apache/arrow/pull/9592#discussion_r584251475 ## File path: rust/parquet/src/basic.rs ## @@ -337,8 +396,11 @@ impl ColumnOrder { // If the max is -0, the row group may contain +0 values as

[GitHub] [arrow] houqp commented on pull request #9534: ARROW-11707: [Rust] support CSV schema inference without file IO

2021-02-27 Thread GitBox
houqp commented on pull request #9534: URL: https://github.com/apache/arrow/pull/9534#issuecomment-787410693 alright, pushed reader offset rewind fix. This is an automated message from the Apache Git Service. To respond to

[GitHub] [arrow] codecov-io edited a comment on pull request #9534: ARROW-11707: [Rust] support CSV schema inference without file IO

2021-02-27 Thread GitBox
codecov-io edited a comment on pull request #9534: URL: https://github.com/apache/arrow/pull/9534#issuecomment-782598126 # [Codecov](https://codecov.io/gh/apache/arrow/pull/9534?src=pr=h1) Report > Merging [#9534](https://codecov.io/gh/apache/arrow/pull/9534?src=pr=desc) (e3312f2) into

[GitHub] [arrow] xhochy commented on a change in pull request #8757: ARROW-8147: [C++] Add google-cloud-cpp to ThirdpartyToolchain

2021-02-27 Thread GitBox
xhochy commented on a change in pull request #8757: URL: https://github.com/apache/arrow/pull/8757#discussion_r584248195 ## File path: ci/conda_env_unix.yml ## @@ -19,6 +19,8 @@ autoconf ccache +google-cloud-cpp=1.20.0 Review comment: What is the reason for the

[GitHub] [arrow] josiahyan commented on pull request #8757: ARROW-8147: [C++] Add google-cloud-cpp to ThirdpartyToolchain

2021-02-27 Thread GitBox
josiahyan commented on pull request #8757: URL: https://github.com/apache/arrow/pull/8757#issuecomment-787375395 Sorry about the delay! I've rebased the commits and manually merged in the changes made in master. This is an

[GitHub] [arrow] chrystianrgs removed a comment on issue #9420: [Rust] Arrow/Parquet/Datafusion interop not possible?

2021-02-27 Thread GitBox
chrystianrgs removed a comment on issue #9420: URL: https://github.com/apache/arrow/issues/9420#issuecomment-787210056 Hi, your sequence no true, is false This is an automated message from the Apache Git Service. To respond

[GitHub] [arrow] chrystianrgs commented on issue #9420: [Rust] Arrow/Parquet/Datafusion interop not possible?

2021-02-27 Thread GitBox
chrystianrgs commented on issue #9420: URL: https://github.com/apache/arrow/issues/9420#issuecomment-787210056 Hi, your sequence no true, is false This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] lidavidm closed pull request #9593: ARROW-11717: [Integration] Fix intermittent flight integration failures with rust

2021-02-27 Thread GitBox
lidavidm closed pull request #9593: URL: https://github.com/apache/arrow/pull/9593 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] lidavidm closed pull request #9583: ARROW-11717: [Integration] report debug logs for failing Flight case

2021-02-27 Thread GitBox
lidavidm closed pull request #9583: URL: https://github.com/apache/arrow/pull/9583 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] alamb commented on pull request #9593: ARROW-11717: [Integration] Fix intermittent flight integration failures with rust

2021-02-27 Thread GitBox
alamb commented on pull request #9593: URL: https://github.com/apache/arrow/pull/9593#issuecomment-787149976 The integration test passed: https://github.com/apache/arrow/pull/9593/checks?check_run_id=1995394206 I'll manually retrigger a few times to see if there might be still

[GitHub] [arrow] alamb commented on pull request #9523: ARROW-11687: [Rust][DataFusion] RepartitionExec Hanging

2021-02-27 Thread GitBox
alamb commented on pull request #9523: URL: https://github.com/apache/arrow/pull/9523#issuecomment-787133178 @edrevo cool -- thanks. There was one other use of crossbeam in the parquet reader which I can try tomorrow morning. Thanks!

[GitHub] [arrow] alamb commented on pull request #9593: ARROW-11717: [Integration] Fix intermittent flight integration failures with rust

2021-02-27 Thread GitBox
alamb commented on pull request #9593: URL: https://github.com/apache/arrow/pull/9593#issuecomment-787131636 @lidavidm thank you for the help -- it is like you are writing the code for me :) This is an automated message

[GitHub] [arrow] alamb commented on pull request #9593: ARROW-11717: [Integration] Fix intermittent flight integration failures with rust

2021-02-27 Thread GitBox
alamb commented on pull request #9593: URL: https://github.com/apache/arrow/pull/9593#issuecomment-787131102 @lidavidm good call -- let me try something else This is an automated message from the Apache Git Service. To

[GitHub] [arrow] github-actions[bot] commented on pull request #9576: ARROW-11821: [Rust] Edit Rust README

2021-02-27 Thread GitBox
github-actions[bot] commented on pull request #9576: URL: https://github.com/apache/arrow/pull/9576#issuecomment-787130649 https://issues.apache.org/jira/browse/ARROW-11821 This is an automated message from the Apache Git

[GitHub] [arrow] alamb commented on pull request #9576: Edit Rust README

2021-02-27 Thread GitBox
alamb commented on pull request #9576: URL: https://github.com/apache/arrow/pull/9576#issuecomment-787129683 @pierwill I understand your position. Conveniently I have been working on a script to improve the experience (https://github.com/apache/arrow/pull/9594) so I am going to reopen

[GitHub] [arrow] github-actions[bot] commented on pull request #9598: ARROW-11804: [Developer] Offer to create JIRA issue

2021-02-27 Thread GitBox
github-actions[bot] commented on pull request #9598: URL: https://github.com/apache/arrow/pull/9598#issuecomment-787129095 https://issues.apache.org/jira/browse/ARROW-11804 This is an automated message from the Apache Git

[GitHub] [arrow] alamb opened a new pull request #9598: ARROW-11804: [Developer] Offer to create JIRA issue

2021-02-27 Thread GitBox
alamb opened a new pull request #9598: URL: https://github.com/apache/arrow/pull/9598 # Rationale Currently all contributors are required to make a JIRA account and do some mechanical JIRA creation to create well formed Arrow PRs. This is mindless work and people who are used to it may

[GitHub] [arrow] github-actions[bot] commented on pull request #9597: ARROW-11820: [Rust] Added macro to create native types

2021-02-27 Thread GitBox
github-actions[bot] commented on pull request #9597: URL: https://github.com/apache/arrow/pull/9597#issuecomment-787115229 https://issues.apache.org/jira/browse/ARROW-11820 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #9597: [Rust] Added macro to create native types

2021-02-27 Thread GitBox
github-actions[bot] commented on pull request #9597: URL: https://github.com/apache/arrow/pull/9597#issuecomment-787114763 Thanks for opening a pull request! Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW Then

[GitHub] [arrow] elferherrera opened a new pull request #9597: [Rust] Added macro to create native types

2021-02-27 Thread GitBox
elferherrera opened a new pull request #9597: URL: https://github.com/apache/arrow/pull/9597 This adds a macro called creative_native to avoid DRY in the creation of native types for alloc module This is an automated

[GitHub] [arrow] elferherrera commented on a change in pull request #9573: ARROW-11783: [Rust] Proposal for RFCs in Rust Arrow

2021-02-27 Thread GitBox
elferherrera commented on a change in pull request #9573: URL: https://github.com/apache/arrow/pull/9573#discussion_r584165011 ## File path: rust/rfcs/LICENSE-APACHE ## @@ -0,0 +1,201 @@ + Apache License Review comment: I dont think so. If

[GitHub] [arrow] elferherrera commented on a change in pull request #9573: ARROW-11783: [Rust] Proposal for RFCs in Rust Arrow

2021-02-27 Thread GitBox
elferherrera commented on a change in pull request #9573: URL: https://github.com/apache/arrow/pull/9573#discussion_r584164907 ## File path: rust/rfcs/-template.md ## @@ -0,0 +1,27 @@ +- Feature Name: (fill me in with a unique ident, `my_awesome_feature`) +- Start Date:

[GitHub] [arrow] elferherrera commented on a change in pull request #9573: ARROW-11783: [Rust] Proposal for RFCs in Rust Arrow

2021-02-27 Thread GitBox
elferherrera commented on a change in pull request #9573: URL: https://github.com/apache/arrow/pull/9573#discussion_r584164881 ## File path: rust/rfcs/-template.md ## @@ -0,0 +1,27 @@ +- Feature Name: (fill me in with a unique ident, `my_awesome_feature`) Review comment:

[GitHub] [arrow] nevi-me commented on a change in pull request #9573: ARROW-11783: [Rust] Proposal for RFCs in Rust Arrow

2021-02-27 Thread GitBox
nevi-me commented on a change in pull request #9573: URL: https://github.com/apache/arrow/pull/9573#discussion_r584147360 ## File path: rust/rfcs/LICENSE-APACHE ## @@ -0,0 +1,201 @@ + Apache License Review comment: Do we need a license if

[GitHub] [arrow] nealrichardson commented on a change in pull request #9591: ARROW-11729: [R] Add examples to the datasets documentation

2021-02-27 Thread GitBox
nealrichardson commented on a change in pull request #9591: URL: https://github.com/apache/arrow/pull/9591#discussion_r584147300 ## File path: r/R/dataset-write.R ## @@ -53,6 +53,17 @@ #' - `codec`: A [Codec] which will be used to compress body buffers of written #' files.

[GitHub] [arrow] codecov-io commented on pull request #9596: ARROW-11495: [Rust] Better numerical_coercion

2021-02-27 Thread GitBox
codecov-io commented on pull request #9596: URL: https://github.com/apache/arrow/pull/9596#issuecomment-787079131 # [Codecov](https://codecov.io/gh/apache/arrow/pull/9596?src=pr=h1) Report > Merging [#9596](https://codecov.io/gh/apache/arrow/pull/9596?src=pr=desc) (f2a9f34) into

[GitHub] [arrow] github-actions[bot] commented on pull request #9594: ARROW-11819: [Rust] Add link to the doc

2021-02-27 Thread GitBox
github-actions[bot] commented on pull request #9594: URL: https://github.com/apache/arrow/pull/9594#issuecomment-787077797 https://issues.apache.org/jira/browse/ARROW-11819 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #9594: ARROW-11818: [Rust] Add link to the doc

2021-02-27 Thread GitBox
github-actions[bot] commented on pull request #9594: URL: https://github.com/apache/arrow/pull/9594#issuecomment-787076648 https://issues.apache.org/jira/browse/ARROW-11818 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #9594: ARROW-11817: [Rust] Add link to the doc

2021-02-27 Thread GitBox
github-actions[bot] commented on pull request #9594: URL: https://github.com/apache/arrow/pull/9594#issuecomment-787075600 https://issues.apache.org/jira/browse/ARROW-11817 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #9594: ARROW-11816: [Rust] Add link to the doc

2021-02-27 Thread GitBox
github-actions[bot] commented on pull request #9594: URL: https://github.com/apache/arrow/pull/9594#issuecomment-787075364 https://issues.apache.org/jira/browse/ARROW-11816 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #9594: ARROW-11815: [Rust] Add link to the doc

2021-02-27 Thread GitBox
github-actions[bot] commented on pull request #9594: URL: https://github.com/apache/arrow/pull/9594#issuecomment-787075248 https://issues.apache.org/jira/browse/ARROW-11815 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #9596: ARROW-11495: [Rust] Better numerical_coercion

2021-02-27 Thread GitBox
github-actions[bot] commented on pull request #9596: URL: https://github.com/apache/arrow/pull/9596#issuecomment-787074912 https://issues.apache.org/jira/browse/ARROW-11495 This is an automated message from the Apache Git

[GitHub] [arrow] sundy-li opened a new pull request #9596: ARROW-11495: [Rust] Better numerical_coercion

2021-02-27 Thread GitBox
sundy-li opened a new pull request #9596: URL: https://github.com/apache/arrow/pull/9596 We should consider number overflow in arithmetic operator functions. ``` UIn8 + UIn8 should be UInt16, but now it's UInt8 UInt16 * UInt16 should be UInt64, but now it's UInt16 ```

[GitHub] [arrow] github-actions[bot] commented on pull request #9594: ARROW-11807: [Rust] Add link to the doc

2021-02-27 Thread GitBox
github-actions[bot] commented on pull request #9594: URL: https://github.com/apache/arrow/pull/9594#issuecomment-787073738 https://issues.apache.org/jira/browse/ARROW-11807 This is an automated message from the Apache Git

[GitHub] [arrow] lidavidm commented on pull request #9593: ARROW-11717: [Integration] Fix intermittent flight integration failures with rust

2021-02-27 Thread GitBox
lidavidm commented on pull request #9593: URL: https://github.com/apache/arrow/pull/9593#issuecomment-787070465 @alamb if I'm not mistaken, `serve().await?` will block until the server is terminated, which means that the test is now stuck. Is there any way to start a server without

[GitHub] [arrow] github-actions[bot] commented on pull request #9595: ARROW-11806: Optimize inner join creation of indices

2021-02-27 Thread GitBox
github-actions[bot] commented on pull request #9595: URL: https://github.com/apache/arrow/pull/9595#issuecomment-787064666 https://issues.apache.org/jira/browse/ARROW-11806 This is an automated message from the Apache Git

[GitHub] [arrow] Dandandan opened a new pull request #9595: ARROW-11806: Optimize inner join creation of indices

2021-02-27 Thread GitBox
Dandandan opened a new pull request #9595: URL: https://github.com/apache/arrow/pull/9595 This PR changes the way we create an array of indices for an inner join to avoid generating a null bit map. It seems currently not really ergonomic to do this with Arrow without resorting to

[GitHub] [arrow] yordan-pavlov edited a comment on pull request #9588: ARROW-11799: [Rust] fix len of string and binary arrays created from unbound iterator

2021-02-27 Thread GitBox
yordan-pavlov edited a comment on pull request #9588: URL: https://github.com/apache/arrow/pull/9588#issuecomment-787062425 @nevi-me this probably deserves its own discussion, but you are pretty close with your suggestion to avoid `ByteArray`; I have been doing quite a lot of

[GitHub] [arrow] yordan-pavlov commented on pull request #9588: ARROW-11799: [Rust] fix len of string and binary arrays created from unbound iterator

2021-02-27 Thread GitBox
yordan-pavlov commented on pull request #9588: URL: https://github.com/apache/arrow/pull/9588#issuecomment-787062425 @nevi-me this probably deserves its own discussion, but you are pretty close with your suggestion to avoid ByteArray; I have been doing quite a lot of profiling and

[GitHub] [arrow] elferherrera commented on pull request #9567: ARROW-11775: [Rust][DataFusion] Feature Flags for Dependencies

2021-02-27 Thread GitBox
elferherrera commented on pull request #9567: URL: https://github.com/apache/arrow/pull/9567#issuecomment-787061390 @seddonm1 I think you dont need to put the `#[cfg(feature)]` to all your functions. As long as you only put it where you are going to use the module. Something like this

[GitHub] [arrow] elferherrera commented on pull request #9573: ARROW-11783: [Rust] Proposal for RFCs in Rust Arrow

2021-02-27 Thread GitBox
elferherrera commented on pull request #9573: URL: https://github.com/apache/arrow/pull/9573#issuecomment-787056503 I've been looking for ways to link the mdbook that could be generated for the RFCs to the apache arrow website and to be honest, I don't think it is possible. Also, I think

[GitHub] [arrow] codecov-io commented on pull request #9594: [Rust] Add link to the doc

2021-02-27 Thread GitBox
codecov-io commented on pull request #9594: URL: https://github.com/apache/arrow/pull/9594#issuecomment-787054968 # [Codecov](https://codecov.io/gh/apache/arrow/pull/9594?src=pr=h1) Report > Merging [#9594](https://codecov.io/gh/apache/arrow/pull/9594?src=pr=desc) (b3f0bc2) into

[GitHub] [arrow] github-actions[bot] commented on pull request #9594: [Rust] Add link to the doc

2021-02-27 Thread GitBox
github-actions[bot] commented on pull request #9594: URL: https://github.com/apache/arrow/pull/9594#issuecomment-787052762 Thanks for opening a pull request! Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW Then

[GitHub] [arrow] alamb opened a new pull request #9594: [Rust] Add link to the doc

2021-02-27 Thread GitBox
alamb opened a new pull request #9594: URL: https://github.com/apache/arrow/pull/9594 This is a test PR with a minor fix, that has no JIRA issue, to automatically create the issue This is an automated message from the

[GitHub] [arrow] alamb commented on pull request #9583: ARROW-11717: [Integration] report debug logs for failing Flight case

2021-02-27 Thread GitBox
alamb commented on pull request #9583: URL: https://github.com/apache/arrow/pull/9583#issuecomment-787051349 Proposed PR here: https://github.com/apache/arrow/pull/9593 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #9593: ARROW-11717: [Integration] Fix intermittent flight integration failures with rust

2021-02-27 Thread GitBox
github-actions[bot] commented on pull request #9593: URL: https://github.com/apache/arrow/pull/9593#issuecomment-787051350 https://issues.apache.org/jira/browse/ARROW-11717 This is an automated message from the Apache Git

[GitHub] [arrow] alamb opened a new pull request #9593: ARROW-11717: [Integration] Fix intermittent flight integration failures with rust

2021-02-27 Thread GitBox
alamb opened a new pull request #9593: URL: https://github.com/apache/arrow/pull/9593 # Background Thanks to the 礪 ️ of @lidavidm https://github.com/apache/arrow/pull/9583#issuecomment-786925049 it appears that the Rust flight integration test prints `"Server listening"` before the

[GitHub] [arrow] alamb commented on pull request #9583: ARROW-11717: [Integration] report debug logs for failing Flight case

2021-02-27 Thread GitBox
alamb commented on pull request #9583: URL: https://github.com/apache/arrow/pull/9583#issuecomment-787050650 Thanks @lidavidm -- that sounds very plausible. I'll get a PR up with a proposed fix This is an automated message

[GitHub] [arrow] nevi-me commented on pull request #9588: ARROW-11799: [Rust] fix len of string and binary arrays created from unbound iterator

2021-02-27 Thread GitBox
nevi-me commented on pull request #9588: URL: https://github.com/apache/arrow/pull/9588#issuecomment-787049887 > While looking for a way to make loading array data from parquet files faster I think the biggest win with strings is if we could avoid `ByteArray` when using the Arrow

[GitHub] [arrow] alamb commented on a change in pull request #9571: ARROW-11778: [Rust] Cast from LargeUtf8 to Numerical and temporal types

2021-02-27 Thread GitBox
alamb commented on a change in pull request #9571: URL: https://github.com/apache/arrow/pull/9571#discussion_r584101216 ## File path: rust/arrow/src/compute/kernels/cast.rs ## @@ -1719,20 +1791,27 @@ mod tests { #[test] fn test_cast_string_to_timestamp() { -

[GitHub] [arrow] codecov-io commented on pull request #9592: ARROW-11803: [Rust] Parquet] Support v2 LogicalType

2021-02-27 Thread GitBox
codecov-io commented on pull request #9592: URL: https://github.com/apache/arrow/pull/9592#issuecomment-787041283 # [Codecov](https://codecov.io/gh/apache/arrow/pull/9592?src=pr=h1) Report > Merging [#9592](https://codecov.io/gh/apache/arrow/pull/9592?src=pr=desc) (bbe9e0c) into

[GitHub] [arrow] nevi-me closed pull request #9571: ARROW-11778: [Rust] Cast from LargeUtf8 to Numerical and temporal types

2021-02-27 Thread GitBox
nevi-me closed pull request #9571: URL: https://github.com/apache/arrow/pull/9571 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] nevi-me commented on a change in pull request #9592: ARROW-11803: [Rust] Parquet] Support v2 LogicalType

2021-02-27 Thread GitBox
nevi-me commented on a change in pull request #9592: URL: https://github.com/apache/arrow/pull/9592#discussion_r584092785 ## File path: rust/parquet/src/schema/types.rs ## @@ -972,18 +1011,22 @@ fn from_thrift_helper( } /// Method to convert to Thrift. -pub fn

[GitHub] [arrow] seddonm1 commented on pull request #9523: ARROW-11687: [Rust][DataFusion] RepartitionExec Hanging

2021-02-27 Thread GitBox
seddonm1 commented on pull request #9523: URL: https://github.com/apache/arrow/pull/9523#issuecomment-787039512 @edrevo that is great that removing crossbeam solves the test case and it sounds like a good plan anyway given Tokio explicitly talks about this potential issue. It

[GitHub] [arrow] github-actions[bot] commented on pull request #9592: ARROW-11803: [Rust] Parquet] Support v2 LogicalType

2021-02-27 Thread GitBox
github-actions[bot] commented on pull request #9592: URL: https://github.com/apache/arrow/pull/9592#issuecomment-787039217 https://issues.apache.org/jira/browse/ARROW-11803 This is an automated message from the Apache Git

[GitHub] [arrow] nevi-me opened a new pull request #9592: ARROW-11803: [Rust] Parquet] Support v2 LogicalType

2021-02-27 Thread GitBox
nevi-me opened a new pull request #9592: URL: https://github.com/apache/arrow/pull/9592 This implements the LogicalType from v2 of the format, by: - renaming `parquet::basic::LogicalType` to `parquet::basic::ConvertedType` to reflect the change in the spec - implementing

[GitHub] [arrow] edrevo commented on pull request #9523: ARROW-11687: [Rust][DataFusion] RepartitionExec Hanging

2021-02-27 Thread GitBox
edrevo commented on pull request #9523: URL: https://github.com/apache/arrow/pull/9523#issuecomment-787038003 @alamb, I would like to spend this weekend unblocking a PR I have in ballista which has been open for a week now, so I would appreciate if you could pick up

[GitHub] [arrow] ritchie46 commented on a change in pull request #9571: ARROW-11778: [Rust] Cast from LargeUtf8 to Numerical and temporal types

2021-02-27 Thread GitBox
ritchie46 commented on a change in pull request #9571: URL: https://github.com/apache/arrow/pull/9571#discussion_r584089152 ## File path: rust/arrow/src/compute/kernels/cast.rs ## @@ -1719,20 +1791,27 @@ mod tests { #[test] fn test_cast_string_to_timestamp() { -