Re: [PR] Optimise decimal casting for infallible conversions [arrow-rs]

2025-02-19 Thread via GitHub
aweltsch commented on PR #7021: URL: https://github.com/apache/arrow-rs/pull/7021#issuecomment-2670679597 Thanks @himadripal, @parthchandra & @tustvold for taking a look. Sorry that clippy failed, I had the pre-commit hook enabled, but for some reason clippy doesn't run with `--workspace --

Re: [PR] Support div_wrapping/rem_wrapping for numeric arithmetic kernels [arrow-rs]

2025-02-19 Thread via GitHub
tustvold commented on PR #7159: URL: https://github.com/apache/arrow-rs/pull/7159#issuecomment-2670672147 The absence of these kernels is intentional - https://github.com/apache/arrow-rs/issues/2647 -- This is an automated message from the Apache Git Service. To respond to the message, pl

Re: [PR] feat: implement GetOptions extensions [arrow-rs]

2025-02-19 Thread via GitHub
waynr commented on PR #7160: URL: https://github.com/apache/arrow-rs/pull/7160#issuecomment-2670664445 By the way, it's worth mentioning that I based this PR on the tag for the `0.11.2` release because I ran into a bunch of breaking changes with respect to integer types in main branch when

[PR] fix: create_random_batch fail with timestamp types with timezone [arrow-rs]

2025-02-19 Thread via GitHub
niebayes opened a new pull request, #7162: URL: https://github.com/apache/arrow-rs/pull/7162 # Which issue does this PR close? Closes #7161 . # Rationale for this change # What changes are included in this PR? # Are there any user-facing ch

Re: [I] [Bug] The "create_random_array" function does not respect input's timezone [arrow-rs]

2025-02-19 Thread via GitHub
niebayes closed issue #6859: [Bug] The "create_random_array" function does not respect input's timezone URL: https://github.com/apache/arrow-rs/issues/6859 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[I] create_random_batch fails with timestamp types having a timezone [arrow-rs]

2025-02-19 Thread via GitHub
niebayes opened a new issue, #7161: URL: https://github.com/apache/arrow-rs/issues/7161 **Describe the bug** The arrow::util module contains a `create_random_batch` function for testing purpose. When given a schema containing a field having a timestamp type with a timezone, th

[PR] feat: implement GetOptions extensions [arrow-rs]

2025-02-19 Thread via GitHub
waynr opened a new pull request, #7160: URL: https://github.com/apache/arrow-rs/pull/7160 # Which issue does this PR close? Closes https://github.com/apache/arrow-rs/issues/7155 # Rationale for this change This is laid out in the associated issue. # Wha

Re: [I] [Python][Packaging] S3FileSystem curl error when using `localstack`-created S3 bucket or custom ca-certificate [arrow]

2025-02-19 Thread via GitHub
cceyda commented on issue #37001: URL: https://github.com/apache/arrow/issues/37001#issuecomment-2670587997 If using jupyter notebook -> https://github.com/apache/arrow/issues/40754#issuecomment-2670583950 -- This is an automated message from the Apache Git Service. To respond to the mess

Re: [I] [Python] Cannot read data if endpoint is s3 on a "secure" Minio server [arrow]

2025-02-19 Thread via GitHub
cceyda commented on issue #40754: URL: https://github.com/apache/arrow/issues/40754#issuecomment-2670583950 If using jupyter notebook you gotta add `SSL_CERT_FILE` env variable in your `kernelspec.json` (found by `jupyter kernelspec list`) formatted like ``` "env": { "SSL_CERT_F

Re: [I] [FlightSQL] Add an interface to FlightSqlService for retrieving the schema key in the request metadata [arrow-rs]

2025-02-19 Thread via GitHub
niebayes closed issue #6562: [FlightSQL] Add an interface to FlightSqlService for retrieving the schema key in the request metadata URL: https://github.com/apache/arrow-rs/issues/6562 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [I] [C++] Meson Support for Arrow [arrow]

2025-02-19 Thread via GitHub
kou commented on issue #41816: URL: https://github.com/apache/arrow/issues/41816#issuecomment-2670567706 How about enabling one test for `libarrow.so` as the next step? We can enable more tests for `libarrow.so` after it. We can enable more modules such as IPC after all tests for `lib

[PR] Support div_wrapping/rem_wrapping for numeric arithmetic kernels [arrow-rs]

2025-02-19 Thread via GitHub
wForget opened a new pull request, #7159: URL: https://github.com/apache/arrow-rs/pull/7159 # Which issue does this PR close? Closes #7158. # Rationale for this change The `div/rem` operations may also overflow, so I want to add `div_wrapping/rem_wrapping` operat

Re: [PR] [C++][Parquet] Add variant type [arrow]

2025-02-19 Thread via GitHub
wgtmac commented on code in PR #45375: URL: https://github.com/apache/arrow/pull/45375#discussion_r1962879360 ## cpp/src/parquet/types.cc: ## @@ -1619,6 +1629,22 @@ class LogicalType::Impl::Float16 final : public LogicalType::Impl::Incompatible, GENERATE_MAKE(Float16) +cla

Re: [PR] GH-45545: [C++][Parquet] Add missing includes [arrow]

2025-02-19 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #45554: URL: https://github.com/apache/arrow/pull/45554#issuecomment-2670494863 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 288a4312900ef5ec3643f36a2264567e58948f75. There were no

[PR] eat(csharp/src/Drivers/BigQuery): choose the first project ID if not specified [arrow-adbc]

2025-02-19 Thread via GitHub
davidhcoe opened a new pull request, #2541: URL: https://github.com/apache/arrow-adbc/pull/2541 - The call to BigQueryClient.CreateQueryJob fails if a project ID is not present (even though the `*detect-project-id*` value is passed). This change locates the first project ID in the list of p

Re: [I] [C++] MakeStatisticsArrayMaxApproximate test uses min not max [arrow]

2025-02-19 Thread via GitHub
kou commented on issue #45578: URL: https://github.com/apache/arrow/issues/45578#issuecomment-2670461328 Issue resolved by pull request 45579 https://github.com/apache/arrow/pull/45579 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] GH-45578: [C++] Use max not min in MakeStatisticsArrayMaxApproximate test [arrow]

2025-02-19 Thread via GitHub
kou merged PR #45579: URL: https://github.com/apache/arrow/pull/45579 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

[I] Support div_wrapping/rem_wrapping for numeric arithmetic kernels [arrow-rs]

2025-02-19 Thread via GitHub
wForget opened a new issue, #7158: URL: https://github.com/apache/arrow-rs/issues/7158 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** The `div/rem` operations may also overflow, so I want to add `div_wrapping/rem_wrapping` op

Re: [PR] GH-45584: [C++][Thirdparty] Bump zstd to v1.5.7 [arrow]

2025-02-19 Thread via GitHub
github-actions[bot] commented on PR #45585: URL: https://github.com/apache/arrow/pull/45585#issuecomment-2670413169 Revision: dad2a1b124ccb6266a5316490ae829346283e229 Submitted crossbow builds: [ursacomputing/crossbow @ actions-e95c349169](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-45584: [C++][Thirdparty] Bump zstd to v1.5.7 [arrow]

2025-02-19 Thread via GitHub
mapleFU commented on PR #45585: URL: https://github.com/apache/arrow/pull/45585#issuecomment-2670408359 @github-actions crossbow submit -g cpp -g wheel -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [I] [Parquet][C++][Python] "List index overflow" when read parquet file [arrow]

2025-02-19 Thread via GitHub
greenlaw commented on issue #33188: URL: https://github.com/apache/arrow/issues/33188#issuecomment-2670397899 Just chiming to say that I'm running into the same issue (`OSError: List index overflow`) when attempting to read parquet files generated by AWS Athena with a column containing list

Re: [PR] GH-45584: [C++][Thirdparty] Bump zstd to v1.5.7 [arrow]

2025-02-19 Thread via GitHub
github-actions[bot] commented on PR #45585: URL: https://github.com/apache/arrow/pull/45585#issuecomment-2670376916 :warning: GitHub issue #45584 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] GH-36628: [Python][Parquet] Fail when instantiating internal Parquet metadata classes [arrow]

2025-02-19 Thread via GitHub
curioustien commented on code in PR #45549: URL: https://github.com/apache/arrow/pull/45549#discussion_r1962806783 ## python/pyarrow/_parquet.pyx: ## @@ -739,6 +749,9 @@ cdef class SortingColumn: cdef class RowGroupMetaData(_Weakrefable): """Metadata for a single row group

Re: [PR] GH-45545: [C++][Parquet] Add missing includes [arrow]

2025-02-19 Thread via GitHub
amoeba commented on PR #45554: URL: https://github.com/apache/arrow/pull/45554#issuecomment-2670373651 > > it took me a sec but the issue number and PR number are not the same. Very close though. > > It's updated by kou :-) Thanks all Oh, got it now. I misinterpreted what you w

Re: [PR] fix(go/adbc/driver/snowflake): use one session for connection [arrow-adbc]

2025-02-19 Thread via GitHub
zeroshade commented on PR #2494: URL: https://github.com/apache/arrow-adbc/pull/2494#issuecomment-2669902558 Nice, I'll see if I can reproduce with the same data. Hopefully it'll be consistent and I can fix this. Thanks @davlee1972 -- This is an automated message from the Apache Git Serv

Re: [PR] [C++][Thirdparty] Bump zstd to v1.5.7 [arrow]

2025-02-19 Thread via GitHub
github-actions[bot] commented on PR #45585: URL: https://github.com/apache/arrow/pull/45585#issuecomment-2670360782 Thanks for opening a pull request! If this is not a [minor PR](https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes). Could you open an issue f

[PR] [C++][Thirdparty] Bump zstd to v1.5.7 [arrow]

2025-02-19 Thread via GitHub
mapleFU opened a new pull request, #45585: URL: https://github.com/apache/arrow/pull/45585 ### Rationale for this change Zstd now released 1.5.7: https://github.com/facebook/zstd/releases/tag/v1.5.7 . It has an optimization that it improves speed for small blocks

Re: [PR] GH-45545: [C++][Parquet] Add missing includes [arrow]

2025-02-19 Thread via GitHub
mapleFU commented on PR #45554: URL: https://github.com/apache/arrow/pull/45554#issuecomment-2670339872 > it took me a sec but the issue number and PR number are not the same. Very close though. It's updated by kou :-) Thanks all -- This is an automated message from the Apache Git

[PR] Improve arrow-json deserialization performance by 30% [arrow-rs]

2025-02-19 Thread via GitHub
mwylde opened a new pull request, #7157: URL: https://github.com/apache/arrow-rs/pull/7157 # Which issue does this PR close? Closes #7156 # Rationale for this change Described in the issue # What changes are included in this PR? This PR is made up of three

[I] Inefficiencies in the arrow-json tape implementation [arrow-rs]

2025-02-19 Thread via GitHub
mwylde opened a new issue, #7156: URL: https://github.com/apache/arrow-rs/issues/7156 We rely heavily on arrow-json, and json decoding is often a performance-sensitive part of streaming pipelines. After doing some benchmarking, I was surprised to see that arrow-json was significantly slower

Re: [PR] MINOR: Bump com.google.api.grpc:proto-google-common-protos from 2.51.0 to 2.52.0 [arrow-java]

2025-02-19 Thread via GitHub
lidavidm commented on PR #617: URL: https://github.com/apache/arrow-java/pull/617#issuecomment-2670126206 I'll fix the Dev PR check sometime, made a typo apparently -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] GH-45582: [Python] Preserve decimal32/64/256 metadata in Schema.metadata [arrow]

2025-02-19 Thread via GitHub
github-actions[bot] commented on PR #45583: URL: https://github.com/apache/arrow/pull/45583#issuecomment-2670090646 :warning: GitHub issue #45582 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

[PR] GH-45582: [Python] Preserve decimal32/64/256 metadata in Schema.metadata [arrow]

2025-02-19 Thread via GitHub
mroeschke opened a new pull request, #45583: URL: https://github.com/apache/arrow/pull/45583 ### Rationale for this change Before, these types would be interpreted as `"object"` type and therefore the `precision` and `scale` attributes of these types would not be preserved in

Re: [I] [Release] Update release scripts to handle separate Java release cycle [arrow-cookbook]

2025-02-19 Thread via GitHub
amoeba commented on issue #372: URL: https://github.com/apache/arrow-cookbook/issues/372#issuecomment-2669921685 I'll work on this today and test on https://github.com/apache/arrow-cookbook/pull/375. -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] MINOR: Bump io.netty:netty-bom from 4.1.117.Final to 4.1.118.Final [arrow-java]

2025-02-19 Thread via GitHub
lidavidm merged PR #616: URL: https://github.com/apache/arrow-java/pull/616 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apa

Re: [PR] MINOR: [Release] Update versions for 19.0.1 [arrow-cookbook]

2025-02-19 Thread via GitHub
lidavidm commented on code in PR #375: URL: https://github.com/apache/arrow-cookbook/pull/375#discussion_r1962530211 ## java/source/demo/pom.xml: ## @@ -41,7 +41,7 @@ 11 11 -18.1.0 +19.0.1 Review Comment: Seems this got messed up? -

Re: [PR] GH-620: Upgrade to Google flatbuffers 25.2.10 [arrow-java]

2025-02-19 Thread via GitHub
lidavidm merged PR #622: URL: https://github.com/apache/arrow-java/pull/622 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apa

Re: [PR] GH-55: [Gandiva] Re-enable tests [arrow-java]

2025-02-19 Thread via GitHub
lidavidm merged PR #595: URL: https://github.com/apache/arrow-java/pull/595 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apa

Re: [PR] GH-55: [Gandiva] Re-enable tests [arrow-java]

2025-02-19 Thread via GitHub
lidavidm commented on PR #595: URL: https://github.com/apache/arrow-java/pull/595#issuecomment-2669987288 Looks like that worked! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] MINOR: Bump com.google.api.grpc:proto-google-common-protos from 2.51.0 to 2.52.0 [arrow-java]

2025-02-19 Thread via GitHub
lidavidm commented on PR #617: URL: https://github.com/apache/arrow-java/pull/617#issuecomment-2669970313 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [C++][Parquet] Add variant type [arrow]

2025-02-19 Thread via GitHub
sfc-gh-nchao commented on code in PR #45375: URL: https://github.com/apache/arrow/pull/45375#discussion_r1962491542 ## cpp/src/parquet/types.cc: ## @@ -1619,6 +1629,22 @@ class LogicalType::Impl::Float16 final : public LogicalType::Impl::Incompatible, GENERATE_MAKE(Float16)

Re: [I] [C++][Parquet] Add missing array include in size_statistics.cc [arrow]

2025-02-19 Thread via GitHub
amoeba commented on issue #45545: URL: https://github.com/apache/arrow/issues/45545#issuecomment-2669894686 Issue resolved by pull request 45554 https://github.com/apache/arrow/pull/45554 -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] GH-45545: [C++][Parquet] Add missing includes [arrow]

2025-02-19 Thread via GitHub
amoeba merged PR #45554: URL: https://github.com/apache/arrow/pull/45554 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache

Re: [PR] GH-45545: [C++][Parquet] Add missing includes [arrow]

2025-02-19 Thread via GitHub
amoeba commented on PR #45554: URL: https://github.com/apache/arrow/pull/45554#issuecomment-2669896892 @mapleFU it took me a sec but the issue number and PR number are not the same. Very close though. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] GH-45566: [C++][Parquet][CMake] Remove a workaround for Windows in FindThriftAlt.cmake [arrow]

2025-02-19 Thread via GitHub
kou commented on PR #45567: URL: https://github.com/apache/arrow/pull/45567#issuecomment-2669866322 It's a good news! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [I] [Docs] inconsistency between C++ implementation and the statistics schema [arrow]

2025-02-19 Thread via GitHub
kou commented on issue #45560: URL: https://github.com/apache/arrow/issues/45560#issuecomment-2669858029 Ah, let's work on it as a separated task too. We'll be able to merge it quickly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[PR] MINOR: [Release] Update versions for 19.0.1 [arrow-cookbook]

2025-02-19 Thread via GitHub
amoeba opened a new pull request, #375: URL: https://github.com/apache/arrow-cookbook/pull/375 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] MINOR: [R] Clean up a linting issue from #45261 [arrow]

2025-02-19 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #45575: URL: https://github.com/apache/arrow/pull/45575#issuecomment-2669815103 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit eb622d927b400e00df69c5a28e7d8cf0d517e4db. There were no

Re: [PR] GH-45572: [C++][Compute] Add rank_normal function [arrow]

2025-02-19 Thread via GitHub
kou commented on code in PR #45573: URL: https://github.com/apache/arrow/pull/45573#discussion_r1962396399 ## cpp/src/arrow/util/math_internal.cc: ## @@ -0,0 +1,130 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See

Re: [PR] fix(go/adbc/driver/snowflake): use one session for connection [arrow-adbc]

2025-02-19 Thread via GitHub
davlee1972 commented on PR #2494: URL: https://github.com/apache/arrow-adbc/pull/2494#issuecomment-2669735263 Finally got my dual login to github issues resolved (for next 90 days)... Here's my test code using the NYC Taxi Trip Data. https://www.kaggle.com/datasets/elemento/nyc

Re: [PR] feat: impl `PartialEq + Eq` for `GetOptions` & `PutPayload` [arrow-rs]

2025-02-19 Thread via GitHub
waynr commented on code in PR #7152: URL: https://github.com/apache/arrow-rs/pull/7152#discussion_r1962352629 ## object_store/src/lib.rs: ## @@ -916,7 +916,7 @@ pub struct ObjectMeta { } /// Options for a get request, such as range -#[derive(Debug, Default, Clone)] +#[derive

Re: [I] go/adbc/driver/snowflake: option for session parameters [arrow-adbc]

2025-02-19 Thread via GitHub
frbvianna commented on issue #2539: URL: https://github.com/apache/arrow-adbc/issues/2539#issuecomment-2669715389 > so this would be the way to pass session parameters when you aren't using a URI at all? Am I understanding correctly? @zeroshade I think that is one way to put it. A

Re: [PR] feat: impl `PartialEq + Eq` for `GetOptions` & `PutPayload` [arrow-rs]

2025-02-19 Thread via GitHub
waynr commented on code in PR #7152: URL: https://github.com/apache/arrow-rs/pull/7152#discussion_r1962352629 ## object_store/src/lib.rs: ## @@ -916,7 +916,7 @@ pub struct ObjectMeta { } /// Options for a get request, such as range -#[derive(Debug, Default, Clone)] +#[derive

Re: [PR] GH-37876: [Format] Add list-view specification to arrow format [arrow]

2025-02-19 Thread via GitHub
adriangb commented on code in PR #37877: URL: https://github.com/apache/arrow/pull/37877#discussion_r1962346147 ## docs/source/format/Columnar.rst: ## @@ -487,6 +499,103 @@ will be represented as follows: :: |---|---|

[PR] GH-45576: [Python] Support building C++ library automatically [arrow]

2025-02-19 Thread via GitHub
mgorny opened a new pull request, #45580: URL: https://github.com/apache/arrow/pull/45580 ### What changes are included in this PR? Add a `PYARROW_BUILD_ARROW_CPP` environment variable and a corresponding `--build-arrow-cpp` option that enables automatically building the Arrow C++ li

Re: [PR] `Date` data types can cast to a time zone-specific timestamp [arrow-rs]

2025-02-19 Thread via GitHub
friendlymatthew commented on code in PR #7141: URL: https://github.com/apache/arrow-rs/pull/7141#discussion_r1962233471 ## arrow-cast/src/cast/mod.rs: ## @@ -1806,55 +1806,63 @@ pub fn cast_with_options( })?, )) } -(Date64, Time

Re: [I] GeoArrow and GeoParquet [arrow-go]

2025-02-19 Thread via GitHub
zeroshade commented on issue #292: URL: https://github.com/apache/arrow-go/issues/292#issuecomment-2669626633 I definitely plan on adding support for them, though given that they are community extensions (not canonical) I'm not sure whether it makes more sense for them to be in another repo

Re: [PR] GH-45576: [Python] Support building C++ library automatically [arrow]

2025-02-19 Thread via GitHub
github-actions[bot] commented on PR #45580: URL: https://github.com/apache/arrow/pull/45580#issuecomment-2669628027 :warning: GitHub issue #45576 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [I] [Release] 18.2.0 [arrow-java]

2025-02-19 Thread via GitHub
jbonofre commented on issue #574: URL: https://github.com/apache/arrow-java/issues/574#issuecomment-2669612066 @kou @raulcd @lidavidm here's the blog post about Arrow Java 18.2.0: https://github.com/apache/arrow-site/pull/594 As soon as it's merged/published, I will do the annouces.

Re: [PR] GH-45167: [C++] Implement Compute Equals for List Types [arrow]

2025-02-19 Thread via GitHub
WillAyd commented on code in PR #45272: URL: https://github.com/apache/arrow/pull/45272#discussion_r1962263513 ## cpp/src/arrow/compute/kernels/codegen_internal.h: ## @@ -390,6 +417,12 @@ struct UnboxScalar> { } }; +template +struct UnboxScalar> { Review Comment: I ca

Re: [I] [C++] Meson Support for Arrow [arrow]

2025-02-19 Thread via GitHub
WillAyd commented on issue #41816: URL: https://github.com/apache/arrow/issues/41816#issuecomment-2669591670 @kou what do you think some nice follows up and next steps would be here? Should we look to support options like `tests`, `ipc`, etc... or do you have something else in mind? -- T

Re: [PR] GH-45167: [C++] Implement Compute Equals for List Types [arrow]

2025-02-19 Thread via GitHub
WillAyd commented on code in PR #45272: URL: https://github.com/apache/arrow/pull/45272#discussion_r1962264419 ## cpp/src/arrow/compute/kernels/codegen_internal.h: ## @@ -322,6 +329,26 @@ struct ArrayIterator> { } }; +template +struct ArrayIterator> { + using T = typenam

Re: [PR] `Date` data types can cast to a time zone-specific timestamp [arrow-rs]

2025-02-19 Thread via GitHub
friendlymatthew commented on PR #7141: URL: https://github.com/apache/arrow-rs/pull/7141#issuecomment-2669528043 > I do have one question, is this the expected behaviour? > > In particular when casting timestamps without a timezone to timestamps with a timezone, the conversion is done

Re: [PR] GH-45570: [Python] Allow Decimal32/64Array.to_pandas [arrow]

2025-02-19 Thread via GitHub
mroeschke commented on code in PR #45571: URL: https://github.com/apache/arrow/pull/45571#discussion_r1962194551 ## python/pyarrow/tests/test_pandas.py: ## @@ -2041,6 +2041,18 @@ def test_strided_objects(self, tmpdir): df = pd.DataFrame.from_dict(data) _check_p

[I] Introduce Extensions concept to object_store::GetOptions and object_store::PutOptions [arrow-rs]

2025-02-19 Thread via GitHub
waynr opened a new issue, #7155: URL: https://github.com/apache/arrow-rs/issues/7155 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** This problem is roughly described in #7135, but essentially we are looking for a way to p

Re: [I] Introduce ObjectStore methods that take Session data [arrow-rs]

2025-02-19 Thread via GitHub
waynr closed issue #7135: Introduce ObjectStore methods that take Session data URL: https://github.com/apache/arrow-rs/issues/7135 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [I] [Python] Can't convert object-dtyped NumPy arrays containing np.datetime64 objects to Arrow [arrow]

2025-02-19 Thread via GitHub
rchowell commented on issue #40580: URL: https://github.com/apache/arrow/issues/40580#issuecomment-2669401903 Hello 👋 are there any suggested workarounds here? ```python import numpy as np import pyarrow as pa data = [np.datetime64(datetime.now(), "D") for _ in range(10)]

Re: [I] Introduce ObjectStore methods that take Session data [arrow-rs]

2025-02-19 Thread via GitHub
waynr commented on issue #7135: URL: https://github.com/apache/arrow-rs/issues/7135#issuecomment-2669396568 > Adding an extension point to GetOptions & Co (potentially PutOptions as well) sounds better. We should document that the builtin stores will basically ignore them. Okay, soun

Re: [PR] GH-44924: [R] Remove usage of cpp11's HAS_UNWIND_PROTECT [arrow]

2025-02-19 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #45261: URL: https://github.com/apache/arrow/pull/45261#issuecomment-2669361012 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 7167ed1635cf0913cb9721b00e600656cf677f85. There were no

Re: [PR] GH-39010: [Python] Introduce `maps_as_pydicts` parameter for `to_pylist`, `to_pydict`, `as_py` [arrow]

2025-02-19 Thread via GitHub
jonded94 commented on code in PR #45471: URL: https://github.com/apache/arrow/pull/45471#discussion_r1962061869 ## python/pyarrow/tests/test_scalars.py: ## @@ -786,6 +786,21 @@ def test_map(pickle_module): restored = pickle_module.loads(pickle_module.dumps(s)) assert r

Re: [PR] GH-39010: [Python] Introduce `maps_as_pydicts` parameter for `to_pylist`, `to_pydict`, `as_py` [arrow]

2025-02-19 Thread via GitHub
jonded94 commented on code in PR #45471: URL: https://github.com/apache/arrow/pull/45471#discussion_r1962058683 ## python/pyarrow/array.pxi: ## @@ -1651,16 +1651,32 @@ cdef class Array(_PandasConvertible): array = array.copy() return array -def to_pyl

Re: [PR] GH-39010: [Python] Introduce `maps_as_pydicts` parameter for `to_pylist`, `to_pydict`, `as_py` [arrow]

2025-02-19 Thread via GitHub
jonded94 commented on code in PR #45471: URL: https://github.com/apache/arrow/pull/45471#discussion_r1962059353 ## python/pyarrow/scalar.pxi: ## @@ -880,12 +1348,49 @@ cdef class MapScalar(ListScalar): for k, v in zip(arr.field(self.type.key_field.name), arr.field(self

Re: [PR] GH-39010: [Python] Introduce `maps_as_pydicts` parameter for `to_pylist`, `to_pydict`, `as_py` [arrow]

2025-02-19 Thread via GitHub
jonded94 commented on code in PR #45471: URL: https://github.com/apache/arrow/pull/45471#discussion_r1962056822 ## python/pyarrow/scalar.pxi: ## @@ -148,7 +149,26 @@ cdef class Scalar(_Weakrefable): def __reduce__(self): return scalar, (self.as_py(), self.type) -

Re: [PR] GH-39010: [Python] Introduce `maps_as_pydicts` parameter for `to_pylist`, `to_pydict`, `as_py` [arrow]

2025-02-19 Thread via GitHub
jonded94 commented on code in PR #45471: URL: https://github.com/apache/arrow/pull/45471#discussion_r1962054700 ## python/pyarrow/array.pxi: ## @@ -2286,12 +2302,28 @@ cdef class MonthDayNanoIntervalArray(Array): Concrete class for Arrow arrays of interval[MonthDayNano] typ

[PR] feat(csharp/src/Drivers/Apache): Add support for Hive ADBC Driver with unit tests [arrow-adbc]

2025-02-19 Thread via GitHub
kushaman opened a new pull request, #2540: URL: https://github.com/apache/arrow-adbc/pull/2540 1. Added support for the Hive ADBC driver with HTTP transport protocol. 2. Added new unit tests for Hive ADBC driver support. -- This is an automated message from the Apache Git Service. To re

Re: [PR] GH-45536: [Dev][R] Update code to match new linters on lintr=3.2.0 [arrow]

2025-02-19 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #45556: URL: https://github.com/apache/arrow/pull/45556#issuecomment-2669097152 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 3712a59936fdd08ca5833f8ed4928d693b5d544c. There were no

[I] Remove file-handle from object store GET operations [arrow-rs]

2025-02-19 Thread via GitHub
crepererum opened a new issue, #7154: URL: https://github.com/apache/arrow-rs/issues/7154 # What Remove [`GetResultPayload::File`](https://docs.rs/object_store/latest/object_store/enum.GetResultPayload.html#variant.File) (and the entire enum). # Why The file handle has several

Re: [PR] feat: Add arrayApproxEqualString to handle null characters in string. [arrow-go]

2025-02-19 Thread via GitHub
zeroshade commented on code in PR #291: URL: https://github.com/apache/arrow-go/pull/291#discussion_r1961938576 ## arrow/array/compare_test.go: ## @@ -725,4 +774,4 @@ func TestTableEqual(t *testing.T) { } }) } -} +} Review Comment

Re: [PR] GH-620: Upgrade to Google flatbuffers 25.2.10 [arrow-java]

2025-02-19 Thread via GitHub
github-actions[bot] commented on PR #622: URL: https://github.com/apache/arrow-java/pull/622#issuecomment-2668838389 Thank you for opening a pull request! Please label the PR with one or more of: - bug-fix - chore - dependencies - documentation - enhancement

Re: [PR] GH-45522: [Parquet][C++] Parquet GEOMETRY and GEOGRAPHY logical type implementations [arrow]

2025-02-19 Thread via GitHub
wgtmac commented on code in PR #45459: URL: https://github.com/apache/arrow/pull/45459#discussion_r1961812496 ## cpp/src/parquet/arrow/schema.cc: ## @@ -428,13 +520,20 @@ Status FieldToNode(const std::string& name, const std::shared_ptr& field, } case ArrowTypeId::EXT

[PR] GH-45578:[C++]Fix MakeStatisticsArrayMaxApproximate test case bug [arrow]

2025-02-19 Thread via GitHub
arashandishgar opened a new pull request, #45579: URL: https://github.com/apache/arrow/pull/45579 ### Rationale for this change The test was written for Min instead of max ### What changes are included in this PR? correction for MaxApproximate test case ### Are th

Re: [I] [C++] When using arrow to create hdfs link, StackOverflowError is reported [arrow]

2025-02-19 Thread via GitHub
amoeba commented on issue #45335: URL: https://github.com/apache/arrow/issues/45335#issuecomment-2669032962 Thanks for reporting back. Hopefully if someone runs into the issue this will help them. -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] docs(license): update LICENSE.txt [arrow-go]

2025-02-19 Thread via GitHub
zeroshade merged PR #290: URL: https://github.com/apache/arrow-go/pull/290 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apac

Re: [I] [R] creating arrow supported expressions [arrow]

2025-02-19 Thread via GitHub
JosiahParry commented on issue #45438: URL: https://github.com/apache/arrow/issues/45438#issuecomment-2669013217 Thanks, @thisisnic. No, I don't think this is sufficient. I am making bindings to geoarrow-rs rust crate (https://github.com/josiahParry/geoarrowrs) which works with the

Re: [I] [Python] Support building libarrow (C++) automatically as part of the Python package [arrow]

2025-02-19 Thread via GitHub
mgorny commented on issue #45576: URL: https://github.com/apache/arrow/issues/45576#issuecomment-2668952723 I explored two options so far: 1. Using `ExternalProject` in CMake — it doesn't seem feasible since the C++ build isn't configured before the build step, and I don't see any way

Re: [PR] GH-45578:[C++]Fix MakeStatisticsArrayMaxApproximate test case bug [arrow]

2025-02-19 Thread via GitHub
github-actions[bot] commented on PR #45579: URL: https://github.com/apache/arrow/pull/45579#issuecomment-2668937381 :warning: GitHub issue #45578 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] MINOR: [R] Clean up a linting issue from #45261 [arrow]

2025-02-19 Thread via GitHub
pitrou merged PR #45575: URL: https://github.com/apache/arrow/pull/45575 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache

Re: [PR] Upgrade to Google flatbuffers 25.2.10 [arrow-java]

2025-02-19 Thread via GitHub
jbonofre commented on PR #622: URL: https://github.com/apache/arrow-java/pull/622#issuecomment-2668836124 This closes #620 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[PR] GH-44615: [C++] Add possibility to extract spans/byte offsets directly for compute.extract_regex [arrow]

2025-02-19 Thread via GitHub
arashandishgar opened a new pull request, #45577: URL: https://github.com/apache/arrow/pull/45577 ### Rationale for this change ### What changes are included in this PR? ### Are these changes tested? ### Are there any user-facing changes?

Re: [I] [C++] Add possibility to extract spans/byte offsets directly for `compute.extract_regex` [arrow]

2025-02-19 Thread via GitHub
pitrou commented on issue #44615: URL: https://github.com/apache/arrow/issues/44615#issuecomment-2668802497 @arashandishgar You can definitely send a PR, it will help us comment more precisely and accurately :) -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] GH-45522: [Parquet][C++] Parquet GEOMETRY and GEOGRAPHY logical type implementations [arrow]

2025-02-19 Thread via GitHub
wgtmac commented on code in PR #45459: URL: https://github.com/apache/arrow/pull/45459#discussion_r1957011653 ## cpp/src/parquet/CMakeLists.txt: ## @@ -171,6 +171,8 @@ set(PARQUET_SRCS exception.cc file_reader.cc file_writer.cc +geometry_statistics.cc Review

Re: [I] [C++] Add possibility to extract spans/byte offsets directly for `compute.extract_regex` [arrow]

2025-02-19 Thread via GitHub
arashandishgar commented on issue #44615: URL: https://github.com/apache/arrow/issues/44615#issuecomment-2668772482 I've finished the implementation. Do I send a pull request or wait for further discussion? -- This is an automated message from the Apache Git Service. To respond to the mes

[PR] MINOR: [R] Clean up a linting issue from #45261 [arrow]

2025-02-19 Thread via GitHub
jonkeane opened a new pull request, #45575: URL: https://github.com/apache/arrow/pull/45575 ### Rationale for this change Cleanup a minor formatting issue introduced in #45261 ### What changes are included in this PR? Remove two new lines ### Are these changes test

Re: [PR] GH-45536: [Dev][R] Update code to match new linters on lintr=3.2.0 [arrow]

2025-02-19 Thread via GitHub
jonkeane commented on PR #45556: URL: https://github.com/apache/arrow/pull/45556#issuecomment-2668759727 I'll fix it though -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] GH-45536: [Dev][R] Update code to match new linters on lintr=3.2.0 [arrow]

2025-02-19 Thread via GitHub
jonkeane commented on PR #45556: URL: https://github.com/apache/arrow/pull/45556#issuecomment-2668751433 Aaaah, that's actually from https://github.com/apache/arrow/pull/45261 (confusingly the lintr update here is only for R code, and just after this was merged there _was_ a C++ change).

Re: [I] Release arrow-rs / parquet minor version `55.0.0` (Apr 2025) [arrow-rs]

2025-02-19 Thread via GitHub
alamb commented on issue #7084: URL: https://github.com/apache/arrow-rs/issues/7084#issuecomment-2668383043 > Items for the next major release: https://github.com/apache/arrow-rs/issues?q=is%3Aopen%20label%3Anext-major-release%20 > > (title should be updated `minor` -> `major`)

[I] [R] Cannot correctly handle arrow metadata derived from the R arrow package [arrow-nanoarrow]

2025-02-19 Thread via GitHub
eitsupi opened a new issue, #713: URL: https://github.com/apache/arrow-nanoarrow/issues/713 It seems to be unable to correctly handle the Arrow data headers generated from extension type such as from `package_version`? ``` r arrow::as_arrow_array( package_version(c( "1.

Re: [PR] MINOR: Bump io.netty:netty-bom from 4.1.117.Final to 4.1.118.Final [arrow-java]

2025-02-19 Thread via GitHub
jbonofre commented on PR #616: URL: https://github.com/apache/arrow-java/pull/616#issuecomment-2668620188 NB: we can update `LICENSE`/`NOTICE` by adding a commit in this PR. Unfortunately I don't have the permission to do so. @lidavidm @kou I propose to merge this PR and I will deal with `L

Re: [PR] GH-45570: [Python] Allow Decimal32/64Array.to_pandas [arrow]

2025-02-19 Thread via GitHub
pitrou commented on code in PR #45571: URL: https://github.com/apache/arrow/pull/45571#discussion_r1961716806 ## python/pyarrow/tests/test_pandas.py: ## @@ -2041,6 +2041,18 @@ def test_strided_objects(self, tmpdir): df = pd.DataFrame.from_dict(data) _check_pand

Re: [PR] GH-45536: [Dev][R] Update code to match new linters on lintr=3.2.0 [arrow]

2025-02-19 Thread via GitHub
pitrou commented on PR #45556: URL: https://github.com/apache/arrow/pull/45556#issuecomment-2668695704 I'm getting a R linter error here: https://github.com/apache/arrow/actions/runs/13413824721/job/37469723948?pr=45573 Seems it might be related to this PR. @thisisnic @jonkeane --

  1   2   >