Re: [PR] GH-43314: [CI][Java] Delete arrow-maven-plugins from release script [arrow]

2024-07-17 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #43313: URL: https://github.com/apache/arrow/pull/43313#issuecomment-2235758048 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit b382156cfc4288584394b95cebccb425a93887d9. There were no

Re: [I] go/adbc/driver/flightsql: performance loss in go1.22.5/go1.21.12 [arrow-adbc]

2024-07-17 Thread via GitHub
lidavidm commented on issue #2021: URL: https://github.com/apache/arrow-adbc/issues/2021#issuecomment-2235744377 I eventually bisected this to a change in the Go compiler: https://github.com/golang/go/commit/3560cf0afb @zeroshade believes it may be mallocator, but a pure-Go reproducer

Re: [I] Make LocalFileSystem an Optional Feature [arrow-rs]

2024-07-17 Thread via GitHub
tustvold commented on issue #6055: URL: https://github.com/apache/arrow-rs/issues/6055#issuecomment-2235707051 People could have disabled default features and it would be a breaking change for them -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [I] [C++] Implement cast suggestions for all nested types in Concatenate [arrow]

2024-07-17 Thread via GitHub
sunby commented on issue #43316: URL: https://github.com/apache/arrow/issues/43316#issuecomment-2235698684 I would like to take a try. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] GH-43291: [C++] Expand the 'take' function tests to cover more chunked-array cases [arrow]

2024-07-17 Thread via GitHub
zanmato1984 commented on code in PR #43292: URL: https://github.com/apache/arrow/pull/43292#discussion_r1682209433 ## cpp/src/arrow/compute/kernels/vector_selection_test.cc: ## @@ -1101,67 +1102,277 @@ TEST(TestFilterMetaFunction, ArityChecking) { // -

Re: [PR] Add PartialEq to ParquetMetaData and FileMetadata [arrow-rs]

2024-07-17 Thread via GitHub
adriangb commented on PR #6082: URL: https://github.com/apache/arrow-rs/pull/6082#issuecomment-2235549693 cc @alamb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

[PR] Add PartialEq to ParquetMetaData and FileMetadata [arrow-rs]

2024-07-17 Thread via GitHub
adriangb opened a new pull request, #6082: URL: https://github.com/apache/arrow-rs/pull/6082 Prep for #6000 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] GH-43276: [Go][Parquet] Make DeltaBitPacking Encoders/Decoders Generic [arrow]

2024-07-17 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #43279: URL: https://github.com/apache/arrow/pull/43279#issuecomment-2235528995 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit cf9f3db95cac418e149a1e9fffef310fa0decbaf. There were no

Re: [PR] GH-43293: [Docs] Update code block for Installing Java Modules [arrow]

2024-07-17 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #43295: URL: https://github.com/apache/arrow/pull/43295#issuecomment-2235504792 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 0ea00f4b7ded95757b6651a1d58a525a7eb8866d. There were no

Re: [PR] Add `unencoded_byte_array_data_bytes` to `ParquetMetaData` [arrow-rs]

2024-07-17 Thread via GitHub
etseidl commented on code in PR #6068: URL: https://github.com/apache/arrow-rs/pull/6068#discussion_r1681794975 ## parquet/src/file/page_index/offset_index.rs: ## @@ -0,0 +1,50 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agre

Re: [PR] GH-38051: [Java] Remove Java 8 support [arrow]

2024-07-17 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #43139: URL: https://github.com/apache/arrow/pull/43139#issuecomment-2235328612 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 4161898d5cd7d05ad0fd9f23e1e9df9439460c98. There were no

Re: [PR] GH-17682: [Format] Add Bool8 Canonical Extension Type [arrow]

2024-07-17 Thread via GitHub
AlenkaF commented on code in PR #43234: URL: https://github.com/apache/arrow/pull/43234#discussion_r1682111536 ## docs/source/format/CanonicalExtensions.rst: ## @@ -283,6 +283,28 @@ UUID A specific UUID version is not required or guaranteed. This extension represents UU

Re: [PR] GH-39565: [C++] Do not concatenate chunked values of fixed-width types to run "array_take" [arrow]

2024-07-17 Thread via GitHub
mapleFU commented on code in PR #41700: URL: https://github.com/apache/arrow/pull/41700#discussion_r1682078031 ## cpp/src/arrow/compute/kernels/vector_selection_take_internal.cc: ## @@ -715,12 +853,244 @@ class TakeMetaFunction : public MetaFunction { return Status::NotImpl

Re: [PR] feat(csharp/src/Drivers/Apache/Spark): low latency test cases [arrow-adbc]

2024-07-17 Thread via GitHub
CurtHagenlocher commented on PR #1948: URL: https://github.com/apache/arrow-adbc/pull/1948#issuecomment-2235273687 I'm sorry; I seem to have missed this. It looks like there might be some imminent merge conflicts with #2014 (attn @birschick-bq). -- This is an automated message from the Ap

Re: [PR] feat(csharp/src/Drivers/Apache/Spark): Add support for username and password authentication [arrow-adbc]

2024-07-17 Thread via GitHub
CurtHagenlocher commented on code in PR #2014: URL: https://github.com/apache/arrow-adbc/pull/2014#discussion_r1682074817 ## csharp/src/Drivers/Apache/Spark/SparkConnection.cs: ## @@ -269,32 +271,58 @@ protected override async ValueTask CreateProtocolAsync() Tr

Re: [PR] feat(csharp/src/Drivers/Apache/Spark): Add support for username and password authentication [arrow-adbc]

2024-07-17 Thread via GitHub
CurtHagenlocher commented on code in PR #2014: URL: https://github.com/apache/arrow-adbc/pull/2014#discussion_r1682073997 ## csharp/src/Drivers/Apache/Spark/SparkConnection.cs: ## @@ -269,32 +271,58 @@ protected override async ValueTask CreateProtocolAsync() Tr

Re: [PR] feat(csharp/src/Drivers/Apache/Spark): Add support for username and password authentication [arrow-adbc]

2024-07-17 Thread via GitHub
CurtHagenlocher commented on code in PR #2014: URL: https://github.com/apache/arrow-adbc/pull/2014#discussion_r1682073568 ## csharp/src/Drivers/Apache/Spark/SparkConnection.cs: ## @@ -269,32 +271,58 @@ protected override async ValueTask CreateProtocolAsync() Tr

Re: [PR] feat(csharp/src/Drivers/Apache/Spark): Add support for username and password authentication [arrow-adbc]

2024-07-17 Thread via GitHub
CurtHagenlocher commented on code in PR #2014: URL: https://github.com/apache/arrow-adbc/pull/2014#discussion_r1682071919 ## csharp/src/Drivers/Apache/Spark/SparkConnection.cs: ## @@ -269,32 +271,58 @@ protected override async ValueTask CreateProtocolAsync() Tr

Re: [PR] GH-43266: [C#] Add LargeBinary, LargeString and LargeList array types [arrow]

2024-07-17 Thread via GitHub
adamreeve commented on code in PR #43269: URL: https://github.com/apache/arrow/pull/43269#discussion_r1682071209 ## csharp/src/Apache.Arrow/Arrays/LargeBinaryArray.cs: ## @@ -0,0 +1,156 @@ +// Licensed to the Apache Software Foundation (ASF) under one or more +// contributor li

Re: [PR] GH-39565: [C++] Do not concatenate chunked values of fixed-width types to run "array_take" [arrow]

2024-07-17 Thread via GitHub
mapleFU commented on code in PR #41700: URL: https://github.com/apache/arrow/pull/41700#discussion_r168206 ## cpp/src/arrow/compute/kernels/vector_selection_take_internal.cc: ## @@ -715,12 +853,244 @@ class TakeMetaFunction : public MetaFunction { return Status::NotImpl

Re: [PR] docs(csharp/src/Drivers/Apache/Spark): document connection properties [arrow-adbc]

2024-07-17 Thread via GitHub
CurtHagenlocher merged PR #2019: URL: https://github.com/apache/arrow-adbc/pull/2019 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@

Re: [PR] docs(csharp/src/Drivers/Apache/Spark): document connection properties [arrow-adbc]

2024-07-17 Thread via GitHub
CurtHagenlocher commented on code in PR #2019: URL: https://github.com/apache/arrow-adbc/pull/2019#discussion_r1682066287 ## csharp/src/Drivers/Apache/Spark/README.md: ## @@ -0,0 +1,84 @@ + + +# Spark Driver + +## Database and Connection Properties + +Properties should be passe

Re: [PR] docs(csharp/src/Drivers/Apache/Spark): document connection properties [arrow-adbc]

2024-07-17 Thread via GitHub
CurtHagenlocher commented on code in PR #2019: URL: https://github.com/apache/arrow-adbc/pull/2019#discussion_r1682064979 ## csharp/src/Drivers/Apache/Spark/README.md: ## @@ -0,0 +1,84 @@ + + +# Spark Driver + +## Database and Connection Properties + +Properties should be passe

Re: [PR] GH-43266: [C#] Add LargeBinary, LargeString and LargeList array types [arrow]

2024-07-17 Thread via GitHub
CurtHagenlocher commented on code in PR #43269: URL: https://github.com/apache/arrow/pull/43269#discussion_r1682050604 ## csharp/src/Apache.Arrow/Arrays/LargeBinaryArray.cs: ## @@ -0,0 +1,156 @@ +// Licensed to the Apache Software Foundation (ASF) under one or more +// contribu

Re: [I] Make LocalFileSystem an Optional Feature [arrow-rs]

2024-07-17 Thread via GitHub
jiacai2050 commented on issue #6055: URL: https://github.com/apache/arrow-rs/issues/6055#issuecomment-2235242151 Thanks for pointing out. If `fs` feature in enabled by default, this is not a breaking change for users? -- This is an automated message from the Apache Git Service. To

Re: [I] [C++] Parquet reader is unable to read LargeString columns [arrow]

2024-07-17 Thread via GitHub
assignUser commented on issue #39682: URL: https://github.com/apache/arrow/issues/39682#issuecomment-2235234378 As we have a few follow up issues for the underlying problems and the user facing issues is fixed I'll close this. -- This is an automated message from the Apache Git Service. T

Re: [I] [R] CRAN packaging checklist for version 17.0.0 [arrow]

2024-07-17 Thread via GitHub
assignUser commented on issue #43317: URL: https://github.com/apache/arrow/issues/43317#issuecomment-2235165622 Yes :) Let me know if I can help with anything! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [I] [R] CRAN packaging checklist for version 17.0.0 [arrow]

2024-07-17 Thread via GitHub
jonkeane commented on issue #43317: URL: https://github.com/apache/arrow/issues/43317#issuecomment-2235164136 @assignUser Did you use `Rscript tools/update-checksums.R ` in the past to generate the contents of https://github.com/apache/arrow/commit/12ec842516f8dc9503dc7eb7bf79286ca2468ef3 ?

Re: [PR] DO NOT MERGE [arrow]

2024-07-17 Thread via GitHub
github-actions[bot] commented on PR #43318: URL: https://github.com/apache/arrow/pull/43318#issuecomment-2235161849 Revision: 5dabe5da316f280902318109b34d2ffab41b52ab Submitted crossbow builds: [ursacomputing/crossbow @ actions-a8858c600d](https://github.com/ursacomputing/crossbow/bra

Re: [PR] DO NOT MERGE [arrow]

2024-07-17 Thread via GitHub
jonkeane commented on PR #43318: URL: https://github.com/apache/arrow/pull/43318#issuecomment-2235159447 @github-actions crossbow submit -g r -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] GH-43258: [C++][Flight] Use a Base CRTP type for the types used in RPC calls [arrow]

2024-07-17 Thread via GitHub
lidavidm commented on code in PR #43255: URL: https://github.com/apache/arrow/pull/43255#discussion_r1681965923 ## cpp/src/arrow/flight/types.h: ## @@ -159,29 +160,80 @@ struct ARROW_FLIGHT_EXPORT CertKeyPair { std::string pem_key; }; +namespace internal { + +template +st

Re: [PR] GH-43258: [C++][Flight] Use a Base CRTP type for the types used in RPC calls [arrow]

2024-07-17 Thread via GitHub
lidavidm commented on code in PR #43255: URL: https://github.com/apache/arrow/pull/43255#discussion_r1681944012 ## cpp/src/arrow/flight/types.h: ## @@ -159,29 +160,80 @@ struct ARROW_FLIGHT_EXPORT CertKeyPair { std::string pem_key; }; +namespace internal { + +template +st

Re: [I] [Swift] Add StructArray Builders [arrow]

2024-07-17 Thread via GitHub
kou commented on issue #43168: URL: https://github.com/apache/arrow/issues/43168#issuecomment-2234946282 Issue resolved by pull request 43171 https://github.com/apache/arrow/pull/43171 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] GH-43168: [Swift] Add buffer and array builders for Struct type [arrow]

2024-07-17 Thread via GitHub
kou merged PR #43171: URL: https://github.com/apache/arrow/pull/43171 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

Re: [PR] GH-43258: [C++][Flight] Use a Base CRTP type for the types used in RPC calls [arrow]

2024-07-17 Thread via GitHub
felipecrv commented on code in PR #43255: URL: https://github.com/apache/arrow/pull/43255#discussion_r1681933211 ## cpp/src/arrow/flight/types.h: ## @@ -159,29 +160,80 @@ struct ARROW_FLIGHT_EXPORT CertKeyPair { std::string pem_key; }; +namespace internal { + +template +s

Re: [PR] GH-43266: [C#] Add LargeBinary, LargeString and LargeList array types [arrow]

2024-07-17 Thread via GitHub
adamreeve commented on PR #43269: URL: https://github.com/apache/arrow/pull/43269#issuecomment-2234923766 OK thanks, I've updated the status documentation so it isn't so misleading now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] GH-43185: [C++] Suggest a cast when Concatenate fails due to offsets overflow [arrow]

2024-07-17 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #43190: URL: https://github.com/apache/arrow/pull/43190#issuecomment-2234923636 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit a1376879ced6c0bc14dcc1e27c0c23c6ad1554a9. There were no

Re: [PR] GH-17682: [Format] Add Bool8 Canonical Extension Type [arrow]

2024-07-17 Thread via GitHub
felipecrv commented on code in PR #43234: URL: https://github.com/apache/arrow/pull/43234#discussion_r1681925204 ## docs/source/format/CanonicalExtensions.rst: ## @@ -283,6 +283,28 @@ UUID A specific UUID version is not required or guaranteed. This extension represents

Re: [PR] GH-17682: [Format] Add Bool8 Canonical Extension Type [arrow]

2024-07-17 Thread via GitHub
felipecrv commented on code in PR #43234: URL: https://github.com/apache/arrow/pull/43234#discussion_r1681923324 ## docs/source/format/CanonicalExtensions.rst: ## @@ -283,6 +283,28 @@ UUID A specific UUID version is not required or guaranteed. This extension represents

Re: [PR] GH-43168: [Swift] Add buffer and array builders for Struct type [arrow]

2024-07-17 Thread via GitHub
abandy commented on PR #43171: URL: https://github.com/apache/arrow/pull/43171#issuecomment-2234840788 @kou I hope all is well. I think I have resolved all the issues. Please review again when you get a chance. -- This is an automated message from the Apache Git Service. To respond to t

Re: [PR] [Format] Add Opaque canonical extension type [arrow]

2024-07-17 Thread via GitHub
paleolimbot commented on PR #41823: URL: https://github.com/apache/arrow/pull/41823#issuecomment-2234780963 I think the extra prose is worth it given the lengthy discussion on this PR (if in several years the justification seems excessive, it could always be removed?). -- This is an auto

Re: [PR] GH-41541: [Go][Parquet] More fixes for writer performance regression [arrow]

2024-07-17 Thread via GitHub
nontan-pixiv commented on PR #42003: URL: https://github.com/apache/arrow/pull/42003#issuecomment-2234669772 > Please do not reuse GitHub issues in the future. This should have been a new GitHub issue instead of reusing the original one. Sorry for the confusion. I will create a new on

Re: [PR] Example of reading and writing parquet metadata outside the file [arrow-rs]

2024-07-17 Thread via GitHub
adriangb commented on code in PR #6081: URL: https://github.com/apache/arrow-rs/pull/6081#discussion_r1681877412 ## parquet/examples/external_metadata.rs: ## @@ -0,0 +1,114 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreemen

Re: [PR] Example of reading and writing parquet metadata outside the file [arrow-rs]

2024-07-17 Thread via GitHub
adriangb commented on code in PR #6081: URL: https://github.com/apache/arrow-rs/pull/6081#discussion_r1681877412 ## parquet/examples/external_metadata.rs: ## @@ -0,0 +1,114 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreemen

Re: [PR] GH-43296: [C++][FlightRPC] Remove Flight UCX transport [arrow]

2024-07-17 Thread via GitHub
lidavidm commented on PR #43297: URL: https://github.com/apache/arrow/pull/43297#issuecomment-2234611198 Good idea, I added a suggestion to the PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] Add support for `newlines_in_values` [arrow-rs]

2024-07-17 Thread via GitHub
connec commented on issue #6066: URL: https://github.com/apache/arrow-rs/issues/6066#issuecomment-2234596575 You're exactly right, my bad – I made an assumption from the lack of an option controlling it as a lack of support. My use case is with datafusion, and I can probably achieve w

Re: [I] Add support for `newlines_in_values` [arrow-rs]

2024-07-17 Thread via GitHub
connec closed issue #6066: Add support for `newlines_in_values` URL: https://github.com/apache/arrow-rs/issues/6066 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [I] [CI][Java] Delete arrow-maven-plugins entry in release scripts [arrow]

2024-07-17 Thread via GitHub
danepitkin commented on issue #43314: URL: https://github.com/apache/arrow/issues/43314#issuecomment-2234472122 Issue resolved by pull request 43313 https://github.com/apache/arrow/pull/43313 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] GH-43314: [CI][Java] Delete arrow-maven-plugins from release script [arrow]

2024-07-17 Thread via GitHub
danepitkin merged PR #43313: URL: https://github.com/apache/arrow/pull/43313 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.ap

Re: [PR] GH-38051: [Java] Remove Java 8 support [arrow]

2024-07-17 Thread via GitHub
danepitkin commented on PR #43139: URL: https://github.com/apache/arrow/pull/43139#issuecomment-2234463004 Whoops it wasn't a caching issue: https://github.com/apache/arrow/pull/43313 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [I] [Java] Remove Java 8 support in Arrow v18 [arrow]

2024-07-17 Thread via GitHub
danepitkin commented on issue #38051: URL: https://github.com/apache/arrow/issues/38051#issuecomment-2234453385 Whoops it wasn't a caching issue: https://github.com/apache/arrow/pull/43313 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] GH-43314: [CI][Java] Delete arrow-maven-plugins from release script [arrow]

2024-07-17 Thread via GitHub
github-actions[bot] commented on PR #43313: URL: https://github.com/apache/arrow/pull/43313#issuecomment-2234447875 :warning: GitHub issue #43314 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] GH-38051: [Java] Remove Java 8 support [arrow]

2024-07-17 Thread via GitHub
laurentgo commented on PR #43139: URL: https://github.com/apache/arrow/pull/43139#issuecomment-2234438663 > `This build will only read from the build cache, since the clean lifecycle is not part of the build invocation.` This is a common message I see all the time and I think it's a g

Re: [PR] Add `unencoded_byte_array_data_bytes` to `ParquetMetaData` [arrow-rs]

2024-07-17 Thread via GitHub
etseidl commented on PR #6068: URL: https://github.com/apache/arrow-rs/pull/6068#issuecomment-2234438190 > I also think that a round trip test (that writes data to parquet, and then reads the file back in and verifies that the metadata is written correctly (e.g `uncencoded_byte_array_data_b

Re: [PR] [Format] Add Opaque canonical extension type [arrow]

2024-07-17 Thread via GitHub
lidavidm commented on PR #41823: URL: https://github.com/apache/arrow/pull/41823#issuecomment-2234438009 I'm happy to just remove everything but even still there seems to be persistent confusion about the scope -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] [Format] Add Opaque canonical extension type [arrow]

2024-07-17 Thread via GitHub
lidavidm commented on code in PR #41823: URL: https://github.com/apache/arrow/pull/41823#discussion_r1681831954 ## docs/source/format/CanonicalExtensions.rst: ## @@ -283,6 +283,148 @@ UUID A specific UUID version is not required or guaranteed. This extension represents

Re: [PR] [Format] Add Opaque canonical extension type [arrow]

2024-07-17 Thread via GitHub
lidavidm commented on code in PR #41823: URL: https://github.com/apache/arrow/pull/41823#discussion_r1681831548 ## docs/source/format/CanonicalExtensions.rst: ## @@ -283,6 +283,148 @@ UUID A specific UUID version is not required or guaranteed. This extension represents

Re: [PR] [Format] Add Opaque canonical extension type [arrow]

2024-07-17 Thread via GitHub
lidavidm commented on PR #41823: URL: https://github.com/apache/arrow/pull/41823#issuecomment-2234435707 All the examples and extra detail is in response to these questions in this PR! I'm not sure what is expected? -- This is an automated message from the Apache Git Service. To respond t

Re: [PR] Try to clear Java's build cache [arrow]

2024-07-17 Thread via GitHub
github-actions[bot] commented on PR #43313: URL: https://github.com/apache/arrow/pull/43313#issuecomment-2234404470 Thanks for opening a pull request! If this is not a [minor PR](https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes). Could you open an issue f

[PR] Try to clear Java's build cache [arrow]

2024-07-17 Thread via GitHub
danepitkin opened a new pull request, #43313: URL: https://github.com/apache/arrow/pull/43313 Do not merge -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

Re: [PR] Add `encode_metadata` function to mirror `decode_metadata` and allow ad-hoc encoding of `ParquetMetadata` [arrow-rs]

2024-07-17 Thread via GitHub
alamb commented on PR #6000: URL: https://github.com/apache/arrow-rs/pull/6000#issuecomment-2234386677 I started on a basic example here: https://github.com/apache/arrow-rs/pull/6081 -- tomorrow I'll try and find time to try and rebase it on this PR and see if I can do what is needed --

Re: [I] API for encoding/decoding ParquetMetadata with more control [arrow-rs]

2024-07-17 Thread via GitHub
alamb commented on issue #6002: URL: https://github.com/apache/arrow-rs/issues/6002#issuecomment-2234386167 I started working on an example here: https://github.com/apache/arrow-rs/pull/6081 (and tried to describe what I think the usecase actually is). -- This is an automated message fro

Re: [I] [Go][Parquet] Delta Binary Packed encoding with null read batchsize panic [arrow]

2024-07-17 Thread via GitHub
joellubi commented on issue #43276: URL: https://github.com/apache/arrow/issues/43276#issuecomment-2234380375 Issue resolved by pull request 43279 https://github.com/apache/arrow/pull/43279 -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] GH-43276: [Go][Parquet] Make DeltaBitPacking Encoders/Decoders Generic [arrow]

2024-07-17 Thread via GitHub
joellubi merged PR #43279: URL: https://github.com/apache/arrow/pull/43279 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apac

Re: [PR] MINOR: [Java] Bump com.puppycrawl.tools:checkstyle from 8.29 to 10.17.0 in /java [arrow]

2024-07-17 Thread via GitHub
github-actions[bot] commented on PR #43311: URL: https://github.com/apache/arrow/pull/43311#issuecomment-2234371579 Revision: 6583c9266e9bda47f29547acea3bf21a63a062f3 Submitted crossbow builds: [ursacomputing/crossbow @ actions-3d2568a501](https://github.com/ursacomputing/crossbow/bra

[PR] Example of reading and writing parquet metadata outside the file [arrow-rs]

2024-07-17 Thread via GitHub
alamb opened a new pull request, #6081: URL: https://github.com/apache/arrow-rs/pull/6081 (very much a WIP) # Which issue does this PR close? Related to https://github.com/apache/arrow-rs/issues/6002 # Rationale for this change To figure out a good API we need an exam

Re: [PR] MINOR: [Java] Bump com.puppycrawl.tools:checkstyle from 8.29 to 10.17.0 in /java [arrow]

2024-07-17 Thread via GitHub
danepitkin commented on PR #43311: URL: https://github.com/apache/arrow/pull/43311#issuecomment-2234367842 @github-actions crossbow submit -g java -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[PR] MINOR: [Java] Bump com.puppycrawl.tools:checkstyle from 8.29 to 10.17.0 in /java [arrow]

2024-07-17 Thread via GitHub
danepitkin opened a new pull request, #43311: URL: https://github.com/apache/arrow/pull/43311 ### Rationale for this change Now that Java 8 is deprecated, we can bump checkstyle to latest version. ### What changes are included in this PR? * Bump checkstyle to 10.17.0

Re: [PR] Add `unencoded_byte_array_data_bytes` to `ParquetMetaData` [arrow-rs]

2024-07-17 Thread via GitHub
etseidl commented on code in PR #6068: URL: https://github.com/apache/arrow-rs/pull/6068#discussion_r1681797354 ## parquet/src/file/metadata/mod.rs: ## @@ -179,6 +184,16 @@ impl ParquetMetaData { self.offset_index.as_ref() } +/// Returns `unencoded_byte_array

Re: [PR] Add `unencoded_byte_array_data_bytes` to `ParquetMetaData` [arrow-rs]

2024-07-17 Thread via GitHub
etseidl commented on code in PR #6068: URL: https://github.com/apache/arrow-rs/pull/6068#discussion_r1681795471 ## parquet/src/file/page_index/index_reader.rs: ## @@ -81,9 +82,9 @@ pub fn read_columns_indexes( /// Return an empty vector if this row group does not contain an //

Re: [PR] Add `unencoded_byte_array_data_bytes` to `ParquetMetaData` [arrow-rs]

2024-07-17 Thread via GitHub
etseidl commented on code in PR #6068: URL: https://github.com/apache/arrow-rs/pull/6068#discussion_r1681794975 ## parquet/src/file/page_index/offset_index.rs: ## @@ -0,0 +1,50 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agre

Re: [PR] Add `unencoded_byte_array_data_bytes` to `ParquetMetaData` [arrow-rs]

2024-07-17 Thread via GitHub
etseidl commented on code in PR #6068: URL: https://github.com/apache/arrow-rs/pull/6068#discussion_r1681791064 ## parquet/src/file/page_index/index_reader.rs: ## @@ -109,7 +146,13 @@ pub fn read_pages_locations( .collect() } -pub(crate) fn decode_offset_index(data:

Re: [PR] MINOR: [Java] Bump ch.qos.logback:logback-classic from 1.3.14 to 1.4.14 in /java [arrow]

2024-07-17 Thread via GitHub
github-actions[bot] commented on PR #43310: URL: https://github.com/apache/arrow/pull/43310#issuecomment-2234350739 Revision: 6edff42f370cd0bb58b4b0242ebeb9a605a2df52 Submitted crossbow builds: [ursacomputing/crossbow @ actions-8b50bd2155](https://github.com/ursacomputing/crossbow/bra

Re: [PR] MINOR: [Java] Bump ch.qos.logback:logback-classic from 1.3.14 to 1.5.6 in /java [arrow]

2024-07-17 Thread via GitHub
danepitkin commented on PR #43310: URL: https://github.com/apache/arrow/pull/43310#issuecomment-2234346052 @github-actions crossbow submit -g java -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] [Java][Docs] Install document has a broken markup part [arrow]

2024-07-17 Thread via GitHub
kou commented on issue #43293: URL: https://github.com/apache/arrow/issues/43293#issuecomment-2234345861 Issue resolved by pull request 43295 https://github.com/apache/arrow/pull/43295 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] GH-43293: [Docs] Update code block for Installing Java Modules [arrow]

2024-07-17 Thread via GitHub
kou merged PR #43295: URL: https://github.com/apache/arrow/pull/43295 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

Re: [PR] GH-43293: [Docs] Update code block for Installing Java Modules [arrow]

2024-07-17 Thread via GitHub
kou commented on PR #43295: URL: https://github.com/apache/arrow/pull/43295#issuecomment-2234344458 http://crossbow.voltrondata.com/pr_docs/43295/java/install.html#java-compatibility -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [PR] MINOR: [Java] Bump ch.qos.logback:logback-classic from 1.3.14 to 1.5.6 in /java [arrow]

2024-07-17 Thread via GitHub
github-actions[bot] commented on PR #43310: URL: https://github.com/apache/arrow/pull/43310#issuecomment-2234330172 Revision: 63f628bd489b13c3edd15ed43cbe393ede9a3c16 Submitted crossbow builds: [ursacomputing/crossbow @ actions-d7566f83b4](https://github.com/ursacomputing/crossbow/bra

Re: [PR] MINOR: [Java] Bump ch.qos.logback:logback-classic from 1.3.14 to 1.5.6 in /java [arrow]

2024-07-17 Thread via GitHub
danepitkin commented on PR #43310: URL: https://github.com/apache/arrow/pull/43310#issuecomment-2234324817 @github-actions crossbow submit -g java -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Add `unencoded_byte_array_data_bytes` to `ParquetMetaData` [arrow-rs]

2024-07-17 Thread via GitHub
alamb commented on code in PR #6068: URL: https://github.com/apache/arrow-rs/pull/6068#discussion_r1681758137 ## parquet/src/column/writer/mod.rs: ## @@ -204,6 +204,33 @@ struct ColumnMetrics { max_column_value: Option, num_column_nulls: u64, column_distinct_count

Re: [PR] Add `unencoded_byte_array_data_bytes` to `ParquetMetaData` [arrow-rs]

2024-07-17 Thread via GitHub
etseidl commented on code in PR #6068: URL: https://github.com/apache/arrow-rs/pull/6068#discussion_r1681754797 ## parquet/src/column/writer/mod.rs: ## @@ -204,6 +204,33 @@ struct ColumnMetrics { max_column_value: Option, num_column_nulls: u64, column_distinct_cou

Re: [PR] MINOR: [Java] Bump org.apache.derby:derby from 10.14.2.0 to 10.15.2.0 in /java [arrow]

2024-07-17 Thread via GitHub
github-actions[bot] commented on PR #43309: URL: https://github.com/apache/arrow/pull/43309#issuecomment-2234308243 Revision: 1faefb9d321777ee72b2b3865429030841688e6b Submitted crossbow builds: [ursacomputing/crossbow @ actions-a7bf5b3c3f](https://github.com/ursacomputing/crossbow/bra

Re: [PR] MINOR: [Java] Bump org.apache.derby:derby from 10.14.2.0 to 10.15.2.0 in /java [arrow]

2024-07-17 Thread via GitHub
danepitkin commented on PR #43309: URL: https://github.com/apache/arrow/pull/43309#issuecomment-2234300592 @github-actions crossbow submit -g java -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] MINOR: [Java] Bump ch.qos.logback:logback-classic from 1.3.14 to 1.5.6 in /java [arrow]

2024-07-17 Thread via GitHub
github-actions[bot] commented on PR #43310: URL: https://github.com/apache/arrow/pull/43310#issuecomment-2234292629 Revision: ff6a1517a7f7316c26caa98f46890f2970dc1273 Submitted crossbow builds: [ursacomputing/crossbow @ actions-5b735d9dc6](https://github.com/ursacomputing/crossbow/bra

Re: [PR] MINOR: [Java] Bump ch.qos.logback:logback-classic from 1.3.14 to 1.5.6 in /java [arrow]

2024-07-17 Thread via GitHub
danepitkin commented on PR #43310: URL: https://github.com/apache/arrow/pull/43310#issuecomment-2234285864 @github-actions crossbow submit -g java -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[PR] MINOR: [Java] Bump ch.qos.logback:logback-classic from 1.3.14 to 1.5.6 in /java [arrow]

2024-07-17 Thread via GitHub
danepitkin opened a new pull request, #43310: URL: https://github.com/apache/arrow/pull/43310 ### Rationale for this change With Java 8 deprecated, we can bump logback to the latest version which supports Java 11. See https://github.com/apache/arrow/pull/40778/files ### What ch

Re: [PR] MINOR: [Java] Bump org.apache.derby:derby from 10.14.2.0 to 10.15.2.0 in /java [arrow]

2024-07-17 Thread via GitHub
github-actions[bot] commented on PR #43309: URL: https://github.com/apache/arrow/pull/43309#issuecomment-2234266940 Revision: d7eaf6575039efec51a0398b99a61711e47cd258 Submitted crossbow builds: [ursacomputing/crossbow @ actions-6099ab2e1a](https://github.com/ursacomputing/crossbow/bra

Re: [PR] Add `unencoded_byte_array_data_bytes` to `ParquetMetaData` [arrow-rs]

2024-07-17 Thread via GitHub
alamb commented on code in PR #6068: URL: https://github.com/apache/arrow-rs/pull/6068#discussion_r1681702498 ## parquet/src/column/writer/mod.rs: ## @@ -204,6 +204,33 @@ struct ColumnMetrics { max_column_value: Option, num_column_nulls: u64, column_distinct_count

Re: [PR] MINOR: [Java] Bump org.apache.derby:derby from 10.14.2.0 to 10.15.2.0 in /java [arrow]

2024-07-17 Thread via GitHub
danepitkin commented on PR #43309: URL: https://github.com/apache/arrow/pull/43309#issuecomment-2234263687 @github-actions crossbow submit -g java -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[PR] MINOR: [Java] Bump org.apache.derby:derby from 10.14.2.0 to 10.15.2.0 in /java [arrow]

2024-07-17 Thread via GitHub
danepitkin opened a new pull request, #43309: URL: https://github.com/apache/arrow/pull/43309 ### Rationale for this change Bump to latest version that supports Java 11. See Apache Derby support matrix here https://db.apache.org/derby/derby_downloads.html ### What changes are i

Re: [PR] MINOR: [Java] Bump org.mockito:mockito-junit-jupiter from 2.25.1 to 5.12.0 in /java [arrow]

2024-07-17 Thread via GitHub
github-actions[bot] commented on PR #43308: URL: https://github.com/apache/arrow/pull/43308#issuecomment-2234259007 Revision: 8bef9ef94fc4232ecd4033be3b775828a5604050 Submitted crossbow builds: [ursacomputing/crossbow @ actions-1fdb0faa0a](https://github.com/ursacomputing/crossbow/bra

Re: [PR] MINOR: [Java] Bump org.mockito:mockito-junit-jupiter from 2.25.1 to 5.12.0 in /java [arrow]

2024-07-17 Thread via GitHub
danepitkin commented on PR #43308: URL: https://github.com/apache/arrow/pull/43308#issuecomment-2234255499 @github-actions crossbow submit -g java -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[PR] MINOR: [Java] Bump org.mockito:mockito-junit-jupiter from 2.25.1 to 5.12.0 in /java [arrow]

2024-07-17 Thread via GitHub
danepitkin opened a new pull request, #43308: URL: https://github.com/apache/arrow/pull/43308 ### Rationale for this change Bump version now that Java 8 is deprecated. Follow up PR from https://github.com/apache/arrow/pull/39408. ### What changes are included in this PR?

[PR] refactor: Separate implementation from interface for nanoarrow_testing component [arrow-nanoarrow]

2024-07-17 Thread via GitHub
paleolimbot opened a new pull request, #561: URL: https://github.com/apache/arrow-nanoarrow/pull/561 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [PR] GH-43284: [Release] Fix variable name on post-12-bump-versions.sh script [arrow]

2024-07-17 Thread via GitHub
kou commented on PR #43294: URL: https://github.com/apache/arrow/pull/43294#issuecomment-2234247421 Ah, we need to run `current_version` before we call `update_versions`: ```diff diff --git a/dev/release/post-12-bump-versions.sh b/dev/release/post-12-bump-versions.sh index 42282

Re: [PR] GH-43284: [Release] Fix variable name on post-12-bump-versions.sh script [arrow]

2024-07-17 Thread via GitHub
kou commented on PR #43294: URL: https://github.com/apache/arrow/pull/43294#issuecomment-2234232180 Could you share the command line you tried? I want to try it too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] GH-43284: [Release] Fix variable name on post-12-bump-versions.sh script [arrow]

2024-07-17 Thread via GitHub
kou commented on PR #43294: URL: https://github.com/apache/arrow/pull/43294#issuecomment-2234230482 > I am not sure why `update_deb_package_names` is failing on the git mv here: https://github.com/apache/arrow/blob/main/dev/release/utils-prepare.sh#L239-L241 I was doing some tests locally

Re: [PR] GH-43284: [Release] Fix variable name on post-12-bump-versions.sh script [arrow]

2024-07-17 Thread via GitHub
kou commented on code in PR #43294: URL: https://github.com/apache/arrow/pull/43294#discussion_r1681696315 ## dev/release/post-12-bump-versions.sh: ## @@ -64,7 +64,7 @@ if [ ${BUMP_VERSION_POST_TAG} -gt 0 ]; then fi if [ ${BUMP_DEB_PACKAGE_NAMES} -gt 0 ] && \ - [ "${next

Re: [PR] GH-38051: [Java] Remove Java 8 support [arrow]

2024-07-17 Thread via GitHub
danepitkin commented on PR #43139: URL: https://github.com/apache/arrow/pull/43139#issuecomment-2234223165 Thank you @laurentgo ! This PR was immensely helpful to the project. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [I] [Java] Remove Java 8 support in Arrow v18 [arrow]

2024-07-17 Thread via GitHub
danepitkin commented on issue #38051: URL: https://github.com/apache/arrow/issues/38051#issuecomment-2234222596 Issue resolved by pull request 43139 https://github.com/apache/arrow/pull/43139 -- This is an automated message from the Apache Git Service. To respond to the message, please

  1   2   3   >