Re: [PR] feat: Implement read support for String/Binary View types [arrow-nanoarrow]

2024-09-21 Thread via GitHub
paleolimbot commented on code in PR #596: URL: https://github.com/apache/arrow-nanoarrow/pull/596#discussion_r1769704165 ## src/nanoarrow/common/inline_array.h: ## @@ -467,52 +468,111 @@ static inline ArrowErrorCode ArrowArrayAppendDouble(struct ArrowArray* array, return NAN

Re: [PR] GH-44178: [GLib][Ruby][Flight] Allow setting CallOption timeout [arrow]

2024-09-21 Thread via GitHub
datbth commented on PR #44179: URL: https://github.com/apache/arrow/pull/44179#issuecomment-2365422452 Many thanks for commenting, @kou. Yes please work on this per your convenience. Thank you! Indeed I'm not familiar with GObject introspection. Besides, I'm not a C++ dev

Re: [PR] GH-44178: [GLib][Ruby][Flight] Allow setting CallOption timeout [arrow]

2024-09-21 Thread via GitHub
kou commented on code in PR #44179: URL: https://github.com/apache/arrow/pull/44179#discussion_r1769663320 ## c_glib/arrow-flight-glib/client.cpp: ## @@ -294,6 +294,23 @@ gaflight_call_options_foreach_header(GAFlightCallOptions *options, } } +/** + * gaflight_call_options

Re: [PR] GH-44178: [GLib][Ruby][Flight] Allow setting CallOption timeout [arrow]

2024-09-21 Thread via GitHub
github-actions[bot] commented on PR #44179: URL: https://github.com/apache/arrow/pull/44179#issuecomment-2365337077 :warning: GitHub issue #44178 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] GH-17: Add integration test CI [arrow-go]

2024-09-21 Thread via GitHub
kou commented on PR #129: URL: https://github.com/apache/arrow-go/pull/129#issuecomment-2365335802 Executed: https://github.com/apache/arrow-go/actions/runs/10969980134/job/30463325934?pr=129#step:10:6452 ```text Integration: Test: IPC: Go -> Go Integration: Test: IPC: Go

Re: [PR] Implement native support StringViewArray for `regexp_is_match` and `regexp_is_match_scalar` function, deprecate `regexp_is_match_utf8` and `regexp_is_match_utf8_scalar` [arrow-rs]

2024-09-21 Thread via GitHub
Dandandan commented on PR #6376: URL: https://github.com/apache/arrow-rs/pull/6376#issuecomment-2365314750 Thanks @tlm365 and @alamb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Implement native support StringViewArray for `regexp_is_match` and `regexp_is_match_scalar` function, deprecate `regexp_is_match_utf8` and `regexp_is_match_utf8_scalar` [arrow-rs]

2024-09-21 Thread via GitHub
Dandandan merged PR #6376: URL: https://github.com/apache/arrow-rs/pull/6376 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.ap

Re: [I] implement `regexp_is_match_utf8` and `regexp_is_match_utf8_scalar` for `StringViewArray` [arrow-rs]

2024-09-21 Thread via GitHub
Dandandan closed issue #6370: implement `regexp_is_match_utf8` and `regexp_is_match_utf8_scalar` for `StringViewArray` URL: https://github.com/apache/arrow-rs/issues/6370 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] GH-44114: [R] Add Rocky and opensuse to the allowlist for libarrow binaries [arrow]

2024-09-21 Thread via GitHub
jonkeane commented on PR #44124: URL: https://github.com/apache/arrow/pull/44124#issuecomment-2365283747 > but with `any(grepl(paste(allowlist, collapse = "|"), os))` `opensuse` should cover `opensuse-leap` and any future versions if I'm not mistaken Ah, right right. Ok I've changed i

Re: [I] Add `into_builder` methods for Arrays [arrow-rs]

2024-09-21 Thread via GitHub
ShiKaiWi commented on issue #6430: URL: https://github.com/apache/arrow-rs/issues/6430#issuecomment-2365286227 When trying to start from the `BooleanArray`, I find that the current `BooleanBufferBuilder` can't reuse the buffer from a sliced `BooleanArray` if the slicing is not aligned.

Re: [PR] GH-44114: [R] Add Rocky and opensuse to the allowlist for libarrow binaries [arrow]

2024-09-21 Thread via GitHub
jonkeane commented on code in PR #44124: URL: https://github.com/apache/arrow/pull/44124#discussion_r1769636600 ## r/tools/nixlibs.R: ## @@ -222,7 +222,7 @@ check_allowlist <- function(os, allowed = "https://raw.githubusercontent.com/apa # Try a remote allowlist so that we

Re: [PR] GH-43960: [R] fix `str_sub` binding to properly handle negative `end` values [arrow]

2024-09-21 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #44141: URL: https://github.com/apache/arrow/pull/44141#issuecomment-2365267746 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 37f62d0bc5f4d22e7194947963b445225b984558. There were no

Re: [I] Interval parsing interprets `5 day hour` wrongly [arrow-rs]

2024-09-21 Thread via GitHub
ByteBaker commented on issue #6390: URL: https://github.com/apache/arrow-rs/issues/6390#issuecomment-2365234381 I meant to ask if we plan to handle more cases in our parsing logic? To be closer to parsing abilities of Postgres? As @samuelcolvin pointed out above. -- This is an automated m

Re: [I] Interval parsing interprets `5 day hour` wrongly [arrow-rs]

2024-09-21 Thread via GitHub
alamb commented on issue #6390: URL: https://github.com/apache/arrow-rs/issues/6390#issuecomment-2365227911 > @alamb do we have a plan on integrating this inside Arrow? I am not sure what you mean by this question @ByteBaker -- This is an automated message from the Apache Git Servi

Re: [I] snowflake: cannot call GetObjects with null catalog [arrow-adbc]

2024-09-21 Thread via GitHub
davidhcoe commented on issue #2171: URL: https://github.com/apache/arrow-adbc/issues/2171#issuecomment-2365212456 I’ll try to pull together the specifics on Monday. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [I] [DISCUSSION] Proposal move `object_store` to its own github repo? [arrow-rs]

2024-09-21 Thread via GitHub
ByteBaker commented on issue #6183: URL: https://github.com/apache/arrow-rs/issues/6183#issuecomment-2365209169 I have a fair understanding of `object_store`. I can help on this too. I see that [Object Store Python](https://github.com/roeap/object-store-python/) already exists, so we

Re: [PR] perf: Faster decimal precision overflow checks [arrow-rs]

2024-09-21 Thread via GitHub
andygrove merged PR #6419: URL: https://github.com/apache/arrow-rs/pull/6419 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.ap

Re: [PR] feat: Implement read support for String/Binary View types [arrow-nanoarrow]

2024-09-21 Thread via GitHub
WillAyd commented on code in PR #596: URL: https://github.com/apache/arrow-nanoarrow/pull/596#discussion_r1769578656 ## src/nanoarrow/common/inline_array.h: ## @@ -467,52 +468,136 @@ static inline ArrowErrorCode ArrowArrayAppendDouble(struct ArrowArray* array, return NANOARR

Re: [PR] feat: Implement read support for String/Binary View types [arrow-nanoarrow]

2024-09-21 Thread via GitHub
WillAyd commented on code in PR #596: URL: https://github.com/apache/arrow-nanoarrow/pull/596#discussion_r1769577954 ## src/nanoarrow/common/inline_array.h: ## @@ -972,6 +1101,26 @@ static inline struct ArrowBufferView ArrowArrayViewGetBytesUnsafe( view.data.as_uint8 =

Re: [PR] GH-44125: [Python] Add concat_recordbatches function [arrow]

2024-09-21 Thread via GitHub
amol- commented on code in PR #44126: URL: https://github.com/apache/arrow/pull/44126#discussion_r1769567749 ## python/pyarrow/table.pxi: ## @@ -6259,6 +6259,56 @@ def concat_tables(tables, MemoryPool memory_pool=None, str promote_options="none return pyarrow_wrap_table(c_

Re: [PR] GH-44125: [Python] Add concat_recordbatches function [arrow]

2024-09-21 Thread via GitHub
amol- commented on code in PR #44126: URL: https://github.com/apache/arrow/pull/44126#discussion_r1769565767 ## python/pyarrow/table.pxi: ## @@ -6259,6 +6259,56 @@ def concat_tables(tables, MemoryPool memory_pool=None, str promote_options="none return pyarrow_wrap_table(c_

Re: [PR] GH-44125: [Python] Add concat_recordbatches function [arrow]

2024-09-21 Thread via GitHub
amol- commented on code in PR #44126: URL: https://github.com/apache/arrow/pull/44126#discussion_r1769565939 ## python/pyarrow/table.pxi: ## @@ -6259,6 +6259,56 @@ def concat_tables(tables, MemoryPool memory_pool=None, str promote_options="none return pyarrow_wrap_table(c_

Re: [PR] GH-44125: [Python] Add concat_recordbatches function [arrow]

2024-09-21 Thread via GitHub
amol- commented on code in PR #44126: URL: https://github.com/apache/arrow/pull/44126#discussion_r1769565939 ## python/pyarrow/table.pxi: ## @@ -6259,6 +6259,56 @@ def concat_tables(tables, MemoryPool memory_pool=None, str promote_options="none return pyarrow_wrap_table(c_

Re: [I] snowflake: cannot call GetObjects with null catalog [arrow-adbc]

2024-09-21 Thread via GitHub
joellubi commented on issue #2171: URL: https://github.com/apache/arrow-adbc/issues/2171#issuecomment-2365195543 Are you able to tell, either from the snowflake query log or from the driver source code if you have access, what APIs the ODBC driver is currently using to achieve this performa

Re: [I] Different numeric type may be able to compare [arrow-rs]

2024-09-21 Thread via GitHub
ByteBaker commented on issue #6357: URL: https://github.com/apache/arrow-rs/issues/6357#issuecomment-2365191313 @alamb @tustvold based on the discussion, should this be closed as not planned? -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] feat: Implement read support for String/Binary View types [arrow-nanoarrow]

2024-09-21 Thread via GitHub
WillAyd commented on code in PR #596: URL: https://github.com/apache/arrow-nanoarrow/pull/596#discussion_r1769562276 ## src/nanoarrow/common/array_test.cc: ## @@ -895,6 +895,146 @@ TEST(ArrayTest, ArrayTestAppendToLargeStringArray) { ArrowArrayRelease(&array); } +TEST(Arra

Re: [PR] feat: Implement read support for String/Binary View types [arrow-nanoarrow]

2024-09-21 Thread via GitHub
WillAyd commented on code in PR #596: URL: https://github.com/apache/arrow-nanoarrow/pull/596#discussion_r1769562077 ## src/nanoarrow/common/inline_array.h: ## @@ -467,52 +468,111 @@ static inline ArrowErrorCode ArrowArrayAppendDouble(struct ArrowArray* array, return NANOARR

Re: [I] [object-store]: Implement credential_process support for S3 [arrow-rs]

2024-09-21 Thread via GitHub
edmondop commented on issue #6422: URL: https://github.com/apache/arrow-rs/issues/6422#issuecomment-2365190065 This is new, and although related to the linked issues, it is not closed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [I] snowflake: cannot call GetObjects with null catalog [arrow-adbc]

2024-09-21 Thread via GitHub
davidhcoe commented on issue #2171: URL: https://github.com/apache/arrow-adbc/issues/2171#issuecomment-2365189726 Thanks. The metadata calls have been a consistent performance bottleneck, particularly as we look to switch from ODBC to ADBC. In ODBC, these calls are consistently less than 1

Re: [I] Interval parsing interprets `5 day hour` wrongly [arrow-rs]

2024-09-21 Thread via GitHub
ByteBaker commented on issue #6390: URL: https://github.com/apache/arrow-rs/issues/6390#issuecomment-2365189415 @alamb do we have a plan on integrating this inside Arrow? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [I] [object-store]: Implement credential_process support for S3 [arrow-rs]

2024-09-21 Thread via GitHub
ByteBaker commented on issue #6422: URL: https://github.com/apache/arrow-rs/issues/6422#issuecomment-2365186898 @alamb since the linked PR is closed, can we mark this as closed? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Add `GenericListViewArray` for `ListView` & `LargeListView` [arrow-rs]

2024-09-21 Thread via GitHub
Kikkon commented on code in PR #5723: URL: https://github.com/apache/arrow-rs/pull/5723#discussion_r1769551796 ## arrow-array/src/array/list_view_array.rs: ## @@ -0,0 +1,1025 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreem

Re: [PR] Add `GenericListViewArray` for `ListView` & `LargeListView` [arrow-rs]

2024-09-21 Thread via GitHub
Kikkon commented on PR #5723: URL: https://github.com/apache/arrow-rs/pull/5723#issuecomment-2365175453 @westonpace 😭 Apologies for the delay in processing this pull request due to work reasons. I hope you can help review it when you have time. -- This is an automated message from the A

Re: [PR] chore: add docs, part of #37 [arrow-rs]

2024-09-21 Thread via GitHub
ByteBaker commented on PR #6433: URL: https://github.com/apache/arrow-rs/pull/6433#issuecomment-2365151921 Updated the PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [I] Bulk Ingestion into Postgresql Numeric with scale [arrow-adbc]

2024-09-21 Thread via GitHub
ibnubay commented on issue #2177: URL: https://github.com/apache/arrow-adbc/issues/2177#issuecomment-2365143801 I try to do another approach to tackle insert "Decimal" data into Postgresql. Like in another issue about retrieve "Decimal" data from Postgresql, on "staging" table I put decim

Re: [I] Incorrect values for `is_null` and `is_not_null` on `UnionArray` [arrow-rs]

2024-09-21 Thread via GitHub
alamb commented on issue #6017: URL: https://github.com/apache/arrow-rs/issues/6017#issuecomment-2365142741 > I'm not sure if implementing `UnionArray::logical_nulls` is sufficient; as least for dictionary-encoded data it seems to me that `DictionaryArray` does not respect the inner values'

Re: [PR] chore: add docs, part of #37 [arrow-rs]

2024-09-21 Thread via GitHub
ByteBaker commented on code in PR #6433: URL: https://github.com/apache/arrow-rs/pull/6433#discussion_r1769530287 ## arrow-schema/src/ffi.rs: ## @@ -274,35 +282,59 @@ impl FFI_ArrowSchema { } } +/// Returns the flags of this schema. pub fn flags(&self) -

Re: [PR] chore: add docs, part of #37 [arrow-rs]

2024-09-21 Thread via GitHub
ByteBaker commented on code in PR #6433: URL: https://github.com/apache/arrow-rs/pull/6433#discussion_r1769530220 ## arrow-array/src/builder/generic_bytes_builder.rs: ## @@ -537,7 +537,7 @@ mod tests { write!(builder, "buz").unwrap(); builder.append_value("");

Re: [PR] chore: add docs, part of #37 [arrow-rs]

2024-09-21 Thread via GitHub
ByteBaker commented on code in PR #6433: URL: https://github.com/apache/arrow-rs/pull/6433#discussion_r1769530051 ## arrow-csv/src/writer.rs: ## @@ -508,7 +509,7 @@ Lorem ipsum dolor sit amet,123.564532,3,true,,00:20:34,cupcakes consectetur adipiscing elit,,2,false,2019-04-18T

Re: [PR] chore: add docs, part of #37 [arrow-rs]

2024-09-21 Thread via GitHub
alamb commented on code in PR #6433: URL: https://github.com/apache/arrow-rs/pull/6433#discussion_r1769529121 ## arrow-array/src/builder/generic_bytes_builder.rs: ## @@ -537,7 +537,7 @@ mod tests { write!(builder, "buz").unwrap(); builder.append_value("");

Re: [PR] MINOR: [Docs] Update implementation matrix for view types in arrow-rs [arrow]

2024-09-21 Thread via GitHub
alamb commented on PR #44175: URL: https://github.com/apache/arrow/pull/44175#issuecomment-2365133946 Thank you @kou -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] object_score: Support Azure Fabric OAuth Provider [arrow-rs]

2024-09-21 Thread via GitHub
alamb merged PR #6382: URL: https://github.com/apache/arrow-rs/pull/6382 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache

Re: [PR] object_score: Support Azure Fabric OAuth Provider [arrow-rs]

2024-09-21 Thread via GitHub
alamb commented on PR #6382: URL: https://github.com/apache/arrow-rs/pull/6382#issuecomment-2365132985 Thank you very much @tustvold @RobinLin666 and @roeap 🙏 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] object_score: Support Azure Fabric OAuth Provider [arrow-rs]

2024-09-21 Thread via GitHub
alamb commented on code in PR #6382: URL: https://github.com/apache/arrow-rs/pull/6382#discussion_r1769527513 ## object_store/src/azure/builder.rs: ## @@ -336,6 +344,34 @@ pub enum AzureConfigKey { /// - `disable_tagging` DisableTagging, +/// Fabric token service

Re: [I] Add `into_builder` methods for Arrays [arrow-rs]

2024-09-21 Thread via GitHub
alamb commented on issue #6430: URL: https://github.com/apache/arrow-rs/issues/6430#issuecomment-2365132687 @ShiKaiWi I would recommend: 1. Start with a single array type (perhaps BooleanArray) 2. Add an example of using the `into_builder` as a doc comment Perhaps initially star

Re: [I] warning: methods `as_any` and `next_batch` are never used in `parquet` crate [arrow-rs]

2024-09-21 Thread via GitHub
alamb closed issue #6143: warning: methods `as_any` and `next_batch` are never used in `parquet` crate URL: https://github.com/apache/arrow-rs/issues/6143 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Silence warnings that `as_any` and `next_batch` are never used [arrow-rs]

2024-09-21 Thread via GitHub
alamb merged PR #6432: URL: https://github.com/apache/arrow-rs/pull/6432 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache

Re: [PR] GH-43960: [R] fix `str_sub` binding to properly handle negative `end` values [arrow]

2024-09-21 Thread via GitHub
thisisnic merged PR #44141: URL: https://github.com/apache/arrow/pull/44141 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apa

Re: [PR] chore: add docs, part of #37 [arrow-rs]

2024-09-21 Thread via GitHub
ByteBaker commented on PR #6433: URL: https://github.com/apache/arrow-rs/pull/6433#issuecomment-2365082938 @alamb for your perusal. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] GH-43535: [C++] support the AWS S3 SSE-C encryption [arrow]

2024-09-21 Thread via GitHub
pitrou commented on PR #43601: URL: https://github.com/apache/arrow/pull/43601#issuecomment-2365069327 If someone is silly enough to report a self-signed certificate _used for testing purposes_ as a security vulnerability, then they deserve to be ignored. CPython has stored a ton of f

[PR] chore: add docs, part of #37 [arrow-rs]

2024-09-21 Thread via GitHub
ByteBaker opened a new pull request, #6433: URL: https://github.com/apache/arrow-rs/pull/6433 - add pragma `#![warn(missing_docs)]` to the following - `arrow-array` - `arrow-cast` - `arrow-csv` - `arrow-data` - `arrow-json` - `arrow-ord` - `arrow-pyarrow-int

Re: [PR] Implement native support StringViewArray for `regexp_is_match` and `regexp_is_match_scalar` function, deprecate `regexp_is_match_utf8` and `regexp_is_match_utf8_scalar` [arrow-rs]

2024-09-21 Thread via GitHub
tlm365 commented on PR #6376: URL: https://github.com/apache/arrow-rs/pull/6376#issuecomment-2365066852 > Thank you very much @tlm365 -- this looks great. > > I was reviewing this PR and I had the code checked out locally, so I took the liberty of making a few changes: > > 1. I

Re: [PR] wip [arrow]

2024-09-21 Thread via GitHub
github-actions[bot] commented on PR #44179: URL: https://github.com/apache/arrow/pull/44179#issuecomment-2365047933 Thanks for opening a pull request! If this is not a [minor PR](https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes). Could you open an issue f

[PR] wip [arrow]

2024-09-21 Thread via GitHub
datbth opened a new pull request, #44179: URL: https://github.com/apache/arrow/pull/44179 GH-44178: [GLib][Ruby][Flight] allow setting CallOption timeout ### Rationale for this change ### What changes are included in this PR? ### Are these changes

Re: [PR] Parquet: Verify 32-bit CRC checksum when decoding pages [arrow-rs]

2024-09-21 Thread via GitHub
xmakro commented on code in PR #6290: URL: https://github.com/apache/arrow-rs/pull/6290#discussion_r1769499771 ## parquet/README.md: ## @@ -82,4 +83,4 @@ The `parquet` crate provides the following features which may be enabled in your ## License -Licensed under the Apache