[GitHub] [arrow] emkornfield commented on a change in pull request #7816: ARROW-9528: [Python] Honor tzinfo when converting from datetime

2020-08-08 Thread GitBox
emkornfield commented on a change in pull request #7816: URL: https://github.com/apache/arrow/pull/7816#discussion_r467532213 ## File path: ci/scripts/integration_spark.sh ## @@ -22,6 +22,9 @@ source_dir=${1} spark_dir=${2} spark_version=${SPARK_VERSION:-master} +# Use old

[GitHub] [arrow] kou closed pull request #7824: ARROW-9205: [Documentation] Fix typos

2020-08-08 Thread GitBox
kou closed pull request #7824: URL: https://github.com/apache/arrow/pull/7824 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [arrow] wqc200 commented on pull request #7916: ARROW-9673: [Rust] [DataFusion] Add a param "dialect" for DFParser::parse_sql

2020-08-08 Thread GitBox
wqc200 commented on pull request #7916: URL: https://github.com/apache/arrow/pull/7916#issuecomment-670973826 > @wqc200 please have a look at this, I think you reverted your changes out of the PR Yes, because i find a error in cicd, but i don't konw why, so i revert, i want to know

[GitHub] [arrow] nevi-me commented on a change in pull request #7917: ARROW-8243: [Rust] [Parquet] Serialize Arrow schema metadata

2020-08-08 Thread GitBox
nevi-me commented on a change in pull request #7917: URL: https://github.com/apache/arrow/pull/7917#discussion_r467500876 ## File path: rust/parquet/Cargo.toml ## @@ -52,4 +53,4 @@ zstd = "0.5" arrow = { path = "../arrow", version = "2.0.0-SNAPSHOT" } [features] -default =

[GitHub] [arrow] vertexclique commented on a change in pull request #7917: ARROW-8243: [Rust] [Parquet] Serialize Arrow schema metadata

2020-08-08 Thread GitBox
vertexclique commented on a change in pull request #7917: URL: https://github.com/apache/arrow/pull/7917#discussion_r467500372 ## File path: rust/parquet/Cargo.toml ## @@ -52,4 +53,4 @@ zstd = "0.5" arrow = { path = "../arrow", version = "2.0.0-SNAPSHOT" } [features] -defau

[GitHub] [arrow] wesm commented on a change in pull request #7816: ARROW-9528: [Python] Honor tzinfo when converting from datetime

2020-08-08 Thread GitBox
wesm commented on a change in pull request #7816: URL: https://github.com/apache/arrow/pull/7816#discussion_r467500217 ## File path: ci/scripts/integration_spark.sh ## @@ -22,6 +22,9 @@ source_dir=${1} spark_dir=${2} spark_version=${SPARK_VERSION:-master} +# Use old behavio

[GitHub] [arrow] nevi-me commented on a change in pull request #7319: ARROW-8289: [Rust] Parquet Arrow writer with nested support

2020-08-08 Thread GitBox
nevi-me commented on a change in pull request #7319: URL: https://github.com/apache/arrow/pull/7319#discussion_r467500252 ## File path: rust/parquet/src/arrow/arrow_writer.rs ## @@ -0,0 +1,595 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contrib

[GitHub] [arrow] emkornfield commented on pull request #7319: ARROW-8289: [Rust] Parquet Arrow writer with nested support

2020-08-08 Thread GitBox
emkornfield commented on pull request #7319: URL: https://github.com/apache/arrow/pull/7319#issuecomment-670968900 > I'm also unsure of how to test deeply nested arrays directly in the code (I had to use Spark because Arrow reader doesn't yet support that). In C++ I covered at least

[GitHub] [arrow] emkornfield commented on a change in pull request #7319: ARROW-8289: [Rust] Parquet Arrow writer with nested support

2020-08-08 Thread GitBox
emkornfield commented on a change in pull request #7319: URL: https://github.com/apache/arrow/pull/7319#discussion_r467499425 ## File path: rust/parquet/src/arrow/arrow_writer.rs ## @@ -0,0 +1,595 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more con

[GitHub] [arrow] emkornfield commented on a change in pull request #7319: ARROW-8289: [Rust] Parquet Arrow writer with nested support

2020-08-08 Thread GitBox
emkornfield commented on a change in pull request #7319: URL: https://github.com/apache/arrow/pull/7319#discussion_r467499328 ## File path: rust/parquet/src/arrow/arrow_writer.rs ## @@ -0,0 +1,595 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more con

[GitHub] [arrow] emkornfield commented on pull request #7319: ARROW-8289: [Rust] Parquet Arrow writer with nested support

2020-08-08 Thread GitBox
emkornfield commented on pull request #7319: URL: https://github.com/apache/arrow/pull/7319#issuecomment-670968236 > @emkornfield not sure if I understand this part, I'll try create a nested batch with a few levels, and have one record have the top level be nested. There are two bugs

[GitHub] [arrow] emkornfield commented on a change in pull request #7816: ARROW-9528: [Python] Honor tzinfo when converting from datetime

2020-08-08 Thread GitBox
emkornfield commented on a change in pull request #7816: URL: https://github.com/apache/arrow/pull/7816#discussion_r467498668 ## File path: ci/scripts/integration_spark.sh ## @@ -22,6 +22,9 @@ source_dir=${1} spark_dir=${2} spark_version=${SPARK_VERSION:-master} +# Use old

[GitHub] [arrow] nevi-me commented on pull request #7916: ARROW-9673: [Rust] [DataFusion] Add a param "dialect" for DFParser::parse_sql

2020-08-08 Thread GitBox
nevi-me commented on pull request #7916: URL: https://github.com/apache/arrow/pull/7916#issuecomment-670967067 @wqc200 please have a look at this, I think you reverted your changes out of the PR This is an automated message

[GitHub] [arrow] nevi-me commented on a change in pull request #7917: ARROW-8243: [Rust] [Parquet] Serialize Arrow schema metadata

2020-08-08 Thread GitBox
nevi-me commented on a change in pull request #7917: URL: https://github.com/apache/arrow/pull/7917#discussion_r467497445 ## File path: rust/parquet/Cargo.toml ## @@ -52,4 +53,4 @@ zstd = "0.5" arrow = { path = "../arrow", version = "2.0.0-SNAPSHOT" } [features] -default =

[GitHub] [arrow] vertexclique commented on a change in pull request #7917: ARROW-8243: [Rust] [Parquet] Serialize Arrow schema metadata

2020-08-08 Thread GitBox
vertexclique commented on a change in pull request #7917: URL: https://github.com/apache/arrow/pull/7917#discussion_r467496882 ## File path: rust/parquet/Cargo.toml ## @@ -52,4 +53,4 @@ zstd = "0.5" arrow = { path = "../arrow", version = "2.0.0-SNAPSHOT" } [features] -defau

[GitHub] [arrow] github-actions[bot] commented on pull request #7917: ARROW-8243: [Rust] [Parquet] Serialize Arrow schema metadata

2020-08-08 Thread GitBox
github-actions[bot] commented on pull request #7917: URL: https://github.com/apache/arrow/pull/7917#issuecomment-670959690 https://issues.apache.org/jira/browse/ARROW-8243 This is an automated message from the Apache Git Serv

[GitHub] [arrow] nevi-me commented on a change in pull request #7917: ARROW-8243: [Rust] [Parquet] Serialize Arrow schema metadata

2020-08-08 Thread GitBox
nevi-me commented on a change in pull request #7917: URL: https://github.com/apache/arrow/pull/7917#discussion_r467491490 ## File path: rust/parquet/Cargo.toml ## @@ -52,4 +53,4 @@ zstd = "0.5" arrow = { path = "../arrow", version = "2.0.0-SNAPSHOT" } [features] -default =

[GitHub] [arrow] nevi-me commented on a change in pull request #7917: ARROW-8243: [Rust] [Parquet] Serialize Arrow schema metadata

2020-08-08 Thread GitBox
nevi-me commented on a change in pull request #7917: URL: https://github.com/apache/arrow/pull/7917#discussion_r467491183 ## File path: rust/parquet/Cargo.toml ## @@ -40,6 +40,7 @@ zstd = { version = "0.5", optional = true } chrono = "0.4" num-bigint = "0.3" arrow = { path =

[GitHub] [arrow] nevi-me opened a new pull request #7917: ARROW-8243: [Rust] [Parquet] Serialize Arrow schema metadata

2020-08-08 Thread GitBox
nevi-me opened a new pull request #7917: URL: https://github.com/apache/arrow/pull/7917 This will allow preserving Arrow-specific metadata when writing or reading Parquet files created from C++ or Rust. If the schema can't be deserialised, the normal Parquet > Arrow schema conversion is

[GitHub] [arrow] wesm commented on a change in pull request #7816: ARROW-9528: [Python] Honor tzinfo when converting from datetime

2020-08-08 Thread GitBox
wesm commented on a change in pull request #7816: URL: https://github.com/apache/arrow/pull/7816#discussion_r467490629 ## File path: ci/scripts/integration_spark.sh ## @@ -22,6 +22,9 @@ source_dir=${1} spark_dir=${2} spark_version=${SPARK_VERSION:-master} +# Use old behavio

[GitHub] [arrow] emkornfield commented on pull request #7815: ARROW-9536: [Java] Miss parameters in PlasmaOutOfMemoryException.java

2020-08-08 Thread GitBox
emkornfield commented on pull request #7815: URL: https://github.com/apache/arrow/pull/7815#issuecomment-670940242 We use a script to merge PRs that squashes all commits and then merges the squashed commit closing the original PR. You can see the link to the new commit above --

[GitHub] [arrow] andygrove commented on pull request #7319: ARROW-8289: [Rust] Parquet Arrow writer with nested support

2020-08-08 Thread GitBox
andygrove commented on pull request #7319: URL: https://github.com/apache/arrow/pull/7319#issuecomment-670935213 @nevi-me Sounds good to me. Thanks. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] xrl edited a comment on pull request #4140: ARROW-5123: [Rust] Parquet derive for simple structs

2020-08-08 Thread GitBox
xrl edited a comment on pull request #4140: URL: https://github.com/apache/arrow/pull/4140#issuecomment-670934899 @nevi-me I'm happy to give you write access to the repo, it's been done now. This is an automated message from

[GitHub] [arrow] xrl commented on pull request #4140: ARROW-5123: [Rust] Parquet derive for simple structs

2020-08-08 Thread GitBox
xrl commented on pull request #4140: URL: https://github.com/apache/arrow/pull/4140#issuecomment-670934899 @nevi-me I'm happy to give you write access to the repo, it's be done now. This is an automated message from the Apach

[GitHub] [arrow] offthewall123 commented on pull request #7815: ARROW-9536: [Java] Miss parameters in PlasmaOutOfMemoryException.java

2020-08-08 Thread GitBox
offthewall123 commented on pull request #7815: URL: https://github.com/apache/arrow/pull/7815#issuecomment-670933488 > +1, thanks. Hi @emkornfield Why this PR is closed ? not merged? This is an automated message from t

[GitHub] [arrow] paddyhoran commented on pull request #7854: ARROW-9583: [Rust] Fix offsets in result of arithmetic kernels

2020-08-08 Thread GitBox
paddyhoran commented on pull request #7854: URL: https://github.com/apache/arrow/pull/7854#issuecomment-670918619 > @paddyhoran can you take a look at this PR? Yep, will do. Apologies, I was on vacation last week. Thi

[GitHub] [arrow] nevi-me commented on pull request #4140: ARROW-5123: [Rust] Parquet derive for simple structs

2020-08-08 Thread GitBox
nevi-me commented on pull request #4140: URL: https://github.com/apache/arrow/pull/4140#issuecomment-670907714 We can't push any changes to the branch because it's from an organisation `tureus`. I'd like to work on this PR, is it fine if I open a fresh one when I'm ready to submit changes?

[GitHub] [arrow] nevi-me commented on pull request #7319: ARROW-8289: [Rust] Parquet Arrow writer with nested support

2020-08-08 Thread GitBox
nevi-me commented on pull request #7319: URL: https://github.com/apache/arrow/pull/7319#issuecomment-670899165 @sunchao @andygrove (CC @wesm @kszucs @emkornfield) in the past few months we haven't had enough review bandwidth on Rust's Parquet implementation (mostly relying on Chao for non-

[GitHub] [arrow] nevi-me commented on pull request #7319: ARROW-8289: [Rust] Parquet Arrow writer with nested support

2020-08-08 Thread GitBox
nevi-me commented on pull request #7319: URL: https://github.com/apache/arrow/pull/7319#issuecomment-670878573 > ... One thing to note which is currently a bug in C++ is once rep/def levels are computed for any anything > with deep nesting (any leaf column one or more direct struct/group

[GitHub] [arrow] nevi-me commented on a change in pull request #7319: ARROW-8289: [Rust] Parquet Arrow writer with nested support

2020-08-08 Thread GitBox
nevi-me commented on a change in pull request #7319: URL: https://github.com/apache/arrow/pull/7319#discussion_r467451835 ## File path: rust/parquet/src/arrow/arrow_writer.rs ## @@ -0,0 +1,348 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contrib

[GitHub] [arrow] nevi-me commented on a change in pull request #7319: ARROW-8289: [Rust] Parquet Arrow writer with nested support

2020-08-08 Thread GitBox
nevi-me commented on a change in pull request #7319: URL: https://github.com/apache/arrow/pull/7319#discussion_r467451701 ## File path: rust/parquet/src/arrow/arrow_writer.rs ## @@ -0,0 +1,348 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contrib

[GitHub] [arrow] vertexclique commented on pull request #7894: ARROW-9631: [Rust] Make arrow not depend on flight

2020-08-08 Thread GitBox
vertexclique commented on pull request #7894: URL: https://github.com/apache/arrow/pull/7894#issuecomment-670857784 @andygrove reverted them. This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [arrow] wqc200 closed pull request #7916: ARROW-9673: [Rust] [DataFusion] Add a param "dialect" for DFParser::parse_sql

2020-08-08 Thread GitBox
wqc200 closed pull request #7916: URL: https://github.com/apache/arrow/pull/7916 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the