Re: [PR] GH-39262: [C++][Azure][FS] Add default credential auth configuration [arrow]

2023-12-18 Thread via GitHub
kou merged PR #39263: URL: https://github.com/apache/arrow/pull/39263 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

Re: [PR] GH-39262: [C++][Azure][FS] Add default credential auth configuration [arrow]

2023-12-18 Thread via GitHub
kou commented on code in PR #39263: URL: https://github.com/apache/arrow/pull/39263#discussion_r1431018421 ## cpp/src/arrow/filesystem/azurefs_test.cc: ## @@ -266,15 +263,12 @@ class AzureHierarchicalNSEnv : public AzureEnvImpl { bool WithHierarchicalNamespace() const final

Re: [I] [C++] "case_when" test failure on random union inputs [arrow]

2023-12-18 Thread via GitHub
zanmato1984 commented on issue #15192: URL: https://github.com/apache/arrow/issues/15192#issuecomment-1862252174 Looked a bit. The error message is ``` '::arrow::internal::ValidateArrayFull(output)' failed with Invalid: Union value at position 0 has invalid type id 0 ``` , assumi

Re: [PR] GH-39138: [R] Fix implicit conversion warnings [arrow]

2023-12-18 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #39250: URL: https://github.com/apache/arrow/pull/39250#issuecomment-1862184853 Thanks for your patience. Conbench analyzed the 6 benchmarking runs that have been run so far on PR commit d3b8accedae9f0866656f027c29d4a5c98b63cca. There was 1 b

Re: [I] Default datafusion.optimizer.prefer_existing_sort to true [arrow-datafusion]

2023-12-18 Thread via GitHub
mustafasrepo commented on issue #8572: URL: https://github.com/apache/arrow-datafusion/issues/8572#issuecomment-1862179527 Actually, it sets `preserve_order`, flag to `true` for the second repartition in the plan (What I call `SortPreservingRepartitionExec` in Plan V1). In this mode, durin

Re: [PR] feat(ord): Support equality of StructArray [arrow-rs]

2023-12-18 Thread via GitHub
tustvold commented on PR #5217: URL: https://github.com/apache/arrow-rs/pull/5217#issuecomment-1862172305 Correct, but the semantic of these kernels is any comparison against a null results in null output for that position -- This is an automated message from the Apache Git Service. To re

Re: [PR] GH-39138: [R] Fix implicit conversion warnings [arrow]

2023-12-18 Thread via GitHub
assignUser commented on PR #39250: URL: https://github.com/apache/arrow/pull/39250#issuecomment-1862116232 The benchmark that is through seems to show some regressions? But I also don't have practice in interpreting these: https://conbench.ursa.dev/compare/runs/e13bb0a5533349c094f88b39316be

Re: [I] non-null sub-field on nullable struct-field has wrong nullity. [arrow-datafusion]

2023-12-18 Thread via GitHub
ravikiran232 commented on issue #8507: URL: https://github.com/apache/arrow-datafusion/issues/8507#issuecomment-1862114562 Hi @alamb, what I understand after reading the issue is that I should check condition whether the parent is null || sub-fields are null. if I am wrong please correct m

Re: [PR] GH-39214: [Java] Support reproducible build [arrow]

2023-12-18 Thread via GitHub
assignUser commented on PR #39215: URL: https://github.com/apache/arrow/pull/39215#issuecomment-1862111613 How exactly does this work, it compares the current build against a reference build? How does that work for dev versions? We do have binary verification scripts so it would make

Re: [PR] Update hyper requirement from 0.14.4 to 1.0.1 [arrow-ballista]

2023-12-18 Thread via GitHub
dependabot[bot] commented on PR #914: URL: https://github.com/apache/arrow-ballista/pull/914#issuecomment-1862104600 Superseded by #940. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Update hyper requirement from 0.14.4 to 1.0.1 [arrow-ballista]

2023-12-18 Thread via GitHub
dependabot[bot] closed pull request #914: Update hyper requirement from 0.14.4 to 1.0.1 URL: https://github.com/apache/arrow-ballista/pull/914 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[PR] Update hyper requirement from 0.14.4 to 1.1.0 [arrow-ballista]

2023-12-18 Thread via GitHub
dependabot[bot] opened a new pull request, #940: URL: https://github.com/apache/arrow-ballista/pull/940 Updates the requirements on [hyper](https://github.com/hyperium/hyper) to permit the latest version. Release notes Sourced from https://github.com/hyperium/hyper/releases";>hyper

Re: [PR] Update hyper requirement from 0.14.4 to 1.1.0 [arrow-ballista]

2023-12-18 Thread via GitHub
dependabot[bot] commented on PR #940: URL: https://github.com/apache/arrow-ballista/pull/940#issuecomment-1862104575 The following labels could not be found: `auto-dependencies`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] MINOR: [Java] Bump org.apache.hadoop:hadoop-client-runtime from 3.3.2 to 3.3.6 in /java [arrow]

2023-12-18 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #39278: URL: https://github.com/apache/arrow/pull/39278#issuecomment-1862089499 After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 3943f744722031d2706d8ed99c1b06caf5429976. There were no

Re: [PR] Substrait insubquery [arrow-datafusion]

2023-12-18 Thread via GitHub
waynexia commented on code in PR #8363: URL: https://github.com/apache/arrow-datafusion/pull/8363#discussion_r1430884614 ## datafusion/substrait/tests/cases/roundtrip_logical_plan.rs: ## Review Comment: It looks like this projection is not pushdown-ed to table scan, here is

Re: [PR] feat(ord): Support equality of StructArray [arrow-rs]

2023-12-18 Thread via GitHub
my-vegetable-has-exploded commented on code in PR #5217: URL: https://github.com/apache/arrow-rs/pull/5217#discussion_r1430867949 ## arrow-ord/src/cmp.rs: ## @@ -268,23 +240,119 @@ fn compare_op(op: Op, lhs: &dyn Datum, rhs: &dyn Datum) -> Result match op {

Re: [PR] feat(ord): Support equality of StructArray [arrow-rs]

2023-12-18 Thread via GitHub
my-vegetable-has-exploded commented on PR #5217: URL: https://github.com/apache/arrow-rs/pull/5217#issuecomment-1862037770 > I think the output should be the union of all the null buffers. I think the nullbuffer for subarrays is only valid for the subarray itself. Take the Examp

Re: [PR] Port tests in `parquet.rs` to sqllogictest [arrow-datafusion]

2023-12-18 Thread via GitHub
hiltontj commented on PR #8560: URL: https://github.com/apache/arrow-datafusion/pull/8560#issuecomment-1861978956 The only test remaining that has not been ported over is `parquet_list_columns`. The test is currently ignored: ```rust #[ignore = "Test ignored, will be enabled as part o

Re: [PR] Port tests in `parquet.rs` to sqllogictest [arrow-datafusion]

2023-12-18 Thread via GitHub
hiltontj commented on code in PR #8560: URL: https://github.com/apache/arrow-datafusion/pull/8560#discussion_r1430816742 ## datafusion/sqllogictest/test_files/parquet.slt: ## @@ -0,0 +1,305 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor l

Re: [PR] GH-34865: [C++][Java][Flight RPC] Add Session management messages [arrow]

2023-12-18 Thread via GitHub
indigophox commented on PR #34817: URL: https://github.com/apache/arrow/pull/34817#issuecomment-1861968050 @lidavidm @zeroshade I guess the biggest question I have is about the cross-language integration test framework: Is there much in the way of creating Flight SQL tests (to avoid reimpl

Re: [PR] GH-37055: [C++] Optimize hash kernels for Dictionary ChunkedArrays [arrow]

2023-12-18 Thread via GitHub
js8544 commented on PR #38394: URL: https://github.com/apache/arrow/pull/38394#issuecomment-1861949250 > I don't see any changes other than the rebase. Still working on it :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] GH-39163: [C++] Add missing data copy in StreamDecoder::Consume(data) [arrow]

2023-12-18 Thread via GitHub
kou commented on code in PR #39164: URL: https://github.com/apache/arrow/pull/39164#discussion_r1430791451 ## cpp/src/arrow/ipc/message.cc: ## @@ -626,10 +626,24 @@ class MessageDecoder::MessageDecoderImpl { RETURN_NOT_OK(ConsumeMetadataLengthData(data, next_requir

Re: [PR] MINOR: [C#] Bump Grpc.Tools from 2.59.0 to 2.60.0 in /csharp [arrow]

2023-12-18 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #39283: URL: https://github.com/apache/arrow/pull/39283#issuecomment-1861913406 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 339810bae2e5f22bc6e1d81ab5a21e038e5994fa. There were no

Re: [PR] Change `ScalarValue::{List, LargeList, FixedSizedList}` to take specific types rather than `ArrayRef` [arrow-datafusion]

2023-12-18 Thread via GitHub
jayzhan211 commented on code in PR #8562: URL: https://github.com/apache/arrow-datafusion/pull/8562#discussion_r1430778346 ## datafusion/common/src/scalar.rs: ## @@ -3300,33 +3409,6 @@ mod tests { assert_eq!(result, &expected); } -#[test] Review Comment:

Re: [PR] GH-39262: [C++][Azure][FS] Add default credential auth configuration [arrow]

2023-12-18 Thread via GitHub
Tom-Newton commented on code in PR #39263: URL: https://github.com/apache/arrow/pull/39263#discussion_r1430076256 ## cpp/src/arrow/filesystem/azurefs_test.cc: ## @@ -266,15 +263,12 @@ class AzureHierarchicalNSEnv : public AzureEnvImpl { bool WithHierarchicalNamespace() const

Re: [PR] PARQUET-2411: [C++][Parquet] Allow reading dictionary without reading data via ByteArrayDictionaryRecordReader [arrow]

2023-12-18 Thread via GitHub
jp0317 commented on code in PR #39153: URL: https://github.com/apache/arrow/pull/39153#discussion_r1430772965 ## cpp/src/parquet/column_reader.h: ## @@ -368,6 +368,16 @@ class PARQUET_EXPORT RecordReader { virtual void DebugPrintState() = 0; + /// \brief Returns the dict

Re: [PR] GH-39138: [R] Fix implicit conversion warnings [arrow]

2023-12-18 Thread via GitHub
ursabot commented on PR #39250: URL: https://github.com/apache/arrow/pull/39250#issuecomment-1861896319 Benchmark runs are scheduled for commit d3b8accedae9f0866656f027c29d4a5c98b63cca. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be po

Re: [PR] GH-39138: [R] Fix implicit conversion warnings [arrow]

2023-12-18 Thread via GitHub
assignUser commented on PR #39250: URL: https://github.com/apache/arrow/pull/39250#issuecomment-1861896234 @ursabot please benchmark -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] PARQUET-2411: [C++][Parquet] Allow reading dictionary without reading data via ByteArrayDictionaryRecordReader [arrow]

2023-12-18 Thread via GitHub
jp0317 commented on code in PR #39153: URL: https://github.com/apache/arrow/pull/39153#discussion_r1430772245 ## cpp/src/parquet/column_reader.cc: ## @@ -1369,6 +1369,26 @@ class TypedRecordReader : public TypedColumnReaderImpl, return bytes_for_values; } + const voi

Re: [PR] GH-39262: [C++][Azure][FS] Add default credential auth configuration [arrow]

2023-12-18 Thread via GitHub
felipecrv commented on code in PR #39263: URL: https://github.com/apache/arrow/pull/39263#discussion_r1430771698 ## cpp/src/arrow/filesystem/azurefs_test.cc: ## @@ -266,15 +263,12 @@ class AzureHierarchicalNSEnv : public AzureEnvImpl { bool WithHierarchicalNamespace() const

Re: [PR] MINOR: [Java] Bump org.apache.maven.surefire:surefire-junit-platform from 3.2.2 to 3.2.3 in /java [arrow]

2023-12-18 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #39280: URL: https://github.com/apache/arrow/pull/39280#issuecomment-1861886618 After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 372f0a063f4b174a1ffab5f1cd037094d8da7a8a. There were no

Re: [PR] Minor: Introduce utils::hash for StructArray [arrow-datafusion]

2023-12-18 Thread via GitHub
jayzhan211 commented on code in PR #8552: URL: https://github.com/apache/arrow-datafusion/pull/8552#discussion_r1430763168 ## datafusion/common/src/hash_utils.rs: ## @@ -327,12 +354,16 @@ pub fn create_hashes<'a>( array => hash_dictionary(array, random_state, ha

Re: [PR] Minor: Introduce utils::hash for StructArray [arrow-datafusion]

2023-12-18 Thread via GitHub
jayzhan211 commented on code in PR #8552: URL: https://github.com/apache/arrow-datafusion/pull/8552#discussion_r1430762650 ## datafusion/common/src/hash_utils.rs: ## @@ -207,6 +208,32 @@ fn hash_dictionary( Ok(()) } +fn hash_struct_array( +array: &StructArray, +r

Re: [PR] GH-37059: [C++][Python] Add rolling_* functions [arrow]

2023-12-18 Thread via GitHub
felipecrv commented on PR #37060: URL: https://github.com/apache/arrow/pull/37060#issuecomment-1861874286 > @felipecrv @mapleFU Hi, since both of you have become committers (congrats btw), would you mind have another look at this PR? You've approved this before but it wasn't merged.

Re: [PR] Minor: Introduce utils::hash for StructArray [arrow-datafusion]

2023-12-18 Thread via GitHub
jayzhan211 commented on code in PR #8552: URL: https://github.com/apache/arrow-datafusion/pull/8552#discussion_r1430758472 ## datafusion/common/src/hash_utils.rs: ## @@ -207,6 +208,32 @@ fn hash_dictionary( Ok(()) } +fn hash_struct_array( +array: &StructArray, +r

Re: [I] ci: fix failing builds [arrow-adbc]

2023-12-18 Thread via GitHub
WillAyd commented on issue #1363: URL: https://github.com/apache/arrow-adbc/issues/1363#issuecomment-1861863380 > * Python typecheck appears to fail due to test_polars.py (if_exists doesn't exist anymore?) This happens in the integration testing too. I haven't looked too closely yet

Re: [PR] chore(deps): bump golang.org/x/crypto from 0.14.0 to 0.17.0 in /go/adbc [arrow-adbc]

2023-12-18 Thread via GitHub
github-actions[bot] commented on PR #1367: URL: https://github.com/apache/arrow-adbc/pull/1367#issuecomment-1861863119 :warning: Please follow the [Conventional Commits format in CONTRIBUTING.md](https://github.com/apache/arrow-adbc/blob/main/CONTRIBUTING.md) for PR titles. -- This is an

[PR] chore(deps): bump golang.org/x/crypto from 0.14.0 to 0.17.0 in /go/adbc [arrow-adbc]

2023-12-18 Thread via GitHub
dependabot[bot] opened a new pull request, #1367: URL: https://github.com/apache/arrow-adbc/pull/1367 Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.14.0 to 0.17.0. Commits https://github.com/golang/crypto/commit/9d2ee975ef9fe627bf0a6f01c1f69e8ef1d4f05d";>

Re: [PR] GH-37199: [C++] Expose a span converter for Buffer and ArraySpan [arrow]

2023-12-18 Thread via GitHub
felipecrv merged PR #38027: URL: https://github.com/apache/arrow/pull/38027 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apa

Re: [PR] MINOR: [Java] Bump io.grpc:grpc-bom from 1.59.0 to 1.60.0 in /java [arrow]

2023-12-18 Thread via GitHub
kou merged PR #39282: URL: https://github.com/apache/arrow/pull/39282 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

Re: [PR] feat(c/driver/postgresql): Support for writing DECIMAL types [arrow-adbc]

2023-12-18 Thread via GitHub
WillAyd commented on PR #1288: URL: https://github.com/apache/arrow-adbc/pull/1288#issuecomment-1861839092 > I don't think Googletest is that flexible. Either "parametrize" yourself by looping through a list of cases inside a single actual test case, or create a separate fixture that is par

Re: [I] object_store: support directly provided GCP access tokens [arrow-rs]

2023-12-18 Thread via GitHub
erikamundson closed issue #5223: object_store: support directly provided GCP access tokens URL: https://github.com/apache/arrow-rs/issues/5223 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] GH-39191: [R] throw error when `string_replace` is passed vector of values in `pattern` [arrow]

2023-12-18 Thread via GitHub
thisisnic commented on code in PR #39219: URL: https://github.com/apache/arrow/pull/39219#discussion_r1430739392 ## r/tests/testthat/test-dplyr-funcs-string.R: ## @@ -433,7 +433,24 @@ test_that("str_replace and str_replace_all", { collect(), df ) - + expect_error

[PR] test(python): Fix polars deprecationwarning in tests [arrow-adbc]

2023-12-18 Thread via GitHub
WillAyd opened a new pull request, #1366: URL: https://github.com/apache/arrow-adbc/pull/1366 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

Re: [PR] GH-39191: [R] throw error when `string_replace` is passed vector of values in `pattern` [arrow]

2023-12-18 Thread via GitHub
abfleishman commented on code in PR #39219: URL: https://github.com/apache/arrow/pull/39219#discussion_r1430737319 ## r/tests/testthat/test-dplyr-funcs-string.R: ## @@ -433,7 +433,24 @@ test_that("str_replace and str_replace_all", { collect(), df ) - + expect_err

Re: [I] object_store: support directly provided GCP access tokens [arrow-rs]

2023-12-18 Thread via GitHub
tustvold commented on issue #5223: URL: https://github.com/apache/arrow-rs/issues/5223#issuecomment-1861827798 The [https://docs.rs/object_store/latest/object_store/gcp/struct.GoogleCloudStorageBuilder.html](https://docs.rs/object_store/latest/object_store/gcp/struct.GoogleCloudStorageBuilde

Re: [PR] GH-39191: [R] throw error when `string_replace` is passed vector of values in `pattern` [arrow]

2023-12-18 Thread via GitHub
thisisnic commented on code in PR #39219: URL: https://github.com/apache/arrow/pull/39219#discussion_r1430730233 ## r/tests/testthat/test-dplyr-funcs-string.R: ## @@ -433,7 +433,24 @@ test_that("str_replace and str_replace_all", { collect(), df ) - + expect_error

[I] object_store: support directly provided GCP access tokens [arrow-rs]

2023-12-18 Thread via GitHub
erikamundson opened a new issue, #5223: URL: https://github.com/apache/arrow-rs/issues/5223 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** **Describe the solution you'd like** **Describe alternatives you've cons

Re: [PR] GH-39191: [R] throw error when `string_replace` is passed vector of values in `pattern` [arrow]

2023-12-18 Thread via GitHub
thisisnic commented on PR #39219: URL: https://github.com/apache/arrow/pull/39219#issuecomment-1861775130 > also, I think this is ready? LMK if there is more I should do? My apologies, I had a brain fart moment when I asked if we need to test them both, and was getting the input data

Re: [I] [R] error with dplyr queries with big data in other environment [arrow]

2023-12-18 Thread via GitHub
thisisnic commented on issue #39203: URL: https://github.com/apache/arrow/issues/39203#issuecomment-1861755655 Could you perhaps try it again with a dataset which isn't confidential, so you can show us all of the code and output? Can you tell us more about the environment you're working i

Re: [PR] MINOR: [R] Update NEWS.md for 14.0.2 [arrow]

2023-12-18 Thread via GitHub
thisisnic commented on code in PR #39286: URL: https://github.com/apache/arrow/pull/39286#discussion_r1430681106 ## r/NEWS.md: ## @@ -19,6 +19,13 @@ # arrow 14.0.1.9000 +## Minor improvements and fixes + +* Fixed C++ compiler warnings (#39138, #39186). +* Fixed confusing d

Re: [PR] feat(ord): Support equality of StructArray [arrow-rs]

2023-12-18 Thread via GitHub
tustvold commented on PR #5217: URL: https://github.com/apache/arrow-rs/pull/5217#issuecomment-1861740894 > So I don't handle nullbuffer for each field. I think the output should be the union of all the null buffers. I'll try to review this in the next couple of days -- This

Re: [PR] feat: support 'LargeList' in `array_pop_front` and `array_pop_back` [arrow-datafusion]

2023-12-18 Thread via GitHub
comphead commented on code in PR #8569: URL: https://github.com/apache/arrow-datafusion/pull/8569#discussion_r1430679163 ## datafusion/physical-expr/src/array_expressions.rs: ## @@ -673,18 +673,72 @@ where )?)) } +/// If from_front is true, it is array_pop_front, otherwi

Re: [PR] POC Make BloomFilter application general, add `PruningPredicate::contains` [arrow-datafusion]

2023-12-18 Thread via GitHub
alamb commented on PR #8397: URL: https://github.com/apache/arrow-datafusion/pull/8397#issuecomment-1861734431 The pieces of this PR are in, so closing this PR and focusing on merging the parts -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] POC Make BloomFilter application general, add `PruningPredicate::contains` [arrow-datafusion]

2023-12-18 Thread via GitHub
alamb closed pull request #8397: POC Make BloomFilter application general, add `PruningPredicate::contains` URL: https://github.com/apache/arrow-datafusion/pull/8397 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] GH-39163: [C++] Add missing data copy in StreamDecoder::Consume(data) [arrow]

2023-12-18 Thread via GitHub
felipecrv commented on code in PR #39164: URL: https://github.com/apache/arrow/pull/39164#discussion_r1430669809 ## cpp/src/arrow/ipc/message.cc: ## @@ -626,10 +626,24 @@ class MessageDecoder::MessageDecoderImpl { RETURN_NOT_OK(ConsumeMetadataLengthData(data, next_

Re: [PR] feat(ord): Support equality of StructArray [arrow-rs]

2023-12-18 Thread via GitHub
alamb commented on code in PR #5217: URL: https://github.com/apache/arrow-rs/pull/5217#discussion_r1430663811 ## arrow-ord/src/cmp.rs: ## @@ -702,4 +772,122 @@ mod tests { neq(&col.slice(0, col.len() - 1), &col.slice(1, col.len() - 1)).unwrap(); } + +#[test]

Re: [PR] feat: support 'LargeList' in `array_pop_front` and `array_pop_back` [arrow-datafusion]

2023-12-18 Thread via GitHub
Weijun-H commented on code in PR #8569: URL: https://github.com/apache/arrow-datafusion/pull/8569#discussion_r1430662249 ## datafusion/physical-expr/src/array_expressions.rs: ## @@ -673,18 +673,72 @@ where )?)) } +/// If from_front is true, it is array_pop_front, otherwi

Re: [I] Specialized / Pre-compiled / Prepared ScalarUDFs [arrow-datafusion]

2023-12-18 Thread via GitHub
alamb commented on issue #8051: URL: https://github.com/apache/arrow-datafusion/issues/8051#issuecomment-1861674787 > > That seems a bit heavy handed? Why not simply augment the scalar function expr node with an interior mutable "cache" cell, that is invisible to serialization? The cell ca

Re: [PR] feat: support 'LargeList' in `array_pop_front` and `array_pop_back` [arrow-datafusion]

2023-12-18 Thread via GitHub
comphead commented on code in PR #8569: URL: https://github.com/apache/arrow-datafusion/pull/8569#discussion_r1430656256 ## datafusion/physical-expr/src/array_expressions.rs: ## @@ -673,18 +673,72 @@ where )?)) } +/// If from_front is true, it is array_pop_front, otherwi

Re: [PR] MINOR: [Java] Bump org.apache:apache from 18 to 31 in /java [arrow]

2023-12-18 Thread via GitHub
lidavidm commented on PR #39279: URL: https://github.com/apache/arrow/pull/39279#issuecomment-1861664576 It looks like [Debian has at least 3.6.3](https://packages.debian.org/search?keywords=maven). CentOS might be an issue though? -- This is an automated message from the Apache Git Serv

Re: [PR] Add trait based ScalarUDF API [arrow-datafusion]

2023-12-18 Thread via GitHub
alamb commented on code in PR #8578: URL: https://github.com/apache/arrow-datafusion/pull/8578#discussion_r1430647761 ## datafusion/expr/src/udf.rs: ## @@ -124,22 +169,116 @@ impl ScalarUDF { &self.aliases } -/// Returns this function's signature (what input

Re: [PR] MINOR: [Java] Bump org.apache.hadoop:hadoop-client-runtime from 3.3.2 to 3.3.6 in /java [arrow]

2023-12-18 Thread via GitHub
kou merged PR #39278: URL: https://github.com/apache/arrow/pull/39278 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

Re: [PR] feat: support 'LargeList' in `array_pop_front` and `array_pop_back` [arrow-datafusion]

2023-12-18 Thread via GitHub
Weijun-H commented on code in PR #8569: URL: https://github.com/apache/arrow-datafusion/pull/8569#discussion_r1430649680 ## datafusion/physical-expr/src/array_expressions.rs: ## @@ -673,18 +673,72 @@ where )?)) } +/// If from_front is true, it is array_pop_front, otherwi

Re: [I] [JS] when creating a Table from Vectors, arrow always infers non-nullable Fields [arrow]

2023-12-18 Thread via GitHub
domoritz commented on issue #37983: URL: https://github.com/apache/arrow/issues/37983#issuecomment-1861646313 We merged a fix for nullable. Let me know what you think. I also sent https://github.com/apache/arrow/pull/39256 to allow customization of the schema so you can make fields nullable

Re: [PR] feat: Add C Data integration test shared library [arrow-nanoarrow]

2023-12-18 Thread via GitHub
paleolimbot commented on code in PR #337: URL: https://github.com/apache/arrow-nanoarrow/pull/337#discussion_r1430643336 ## src/nanoarrow/integration/c_data_integration.cc: ## @@ -0,0 +1,200 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributo

Re: [PR] Add trait based ScalarUDF API [arrow-datafusion]

2023-12-18 Thread via GitHub
alamb commented on code in PR #8578: URL: https://github.com/apache/arrow-datafusion/pull/8578#discussion_r1430520598 ## datafusion/expr/src/udf.rs: ## @@ -79,7 +90,11 @@ impl std::hash::Hash for ScalarUDF { } impl ScalarUDF { -/// Create a new ScalarUDF +/// Create

Re: [PR] feat: Add C Data integration test shared library [arrow-nanoarrow]

2023-12-18 Thread via GitHub
paleolimbot commented on code in PR #337: URL: https://github.com/apache/arrow-nanoarrow/pull/337#discussion_r1430642478 ## src/nanoarrow/integration/c_data_integration.cc: ## @@ -0,0 +1,200 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributo

Re: [I] [C++][Packaging] Consider stricter compiler options [arrow]

2023-12-18 Thread via GitHub
pitrou commented on issue #39026: URL: https://github.com/apache/arrow/issues/39026#issuecomment-1861612173 You can try enabling them and see what happens. But some of them may be too strict. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] MINOR: [CI] Bump actions/download-artifact from 3 to 4 [arrow]

2023-12-18 Thread via GitHub
kou commented on PR #39284: URL: https://github.com/apache/arrow/pull/39284#issuecomment-1861607465 We also need to update `actions/upload-artifact` to `v4` to use `actions/download-artifacts@v4`. -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] Bump actions/download-artifact from 3 to 4 [arrow-flight-sql-postgresql]

2023-12-18 Thread via GitHub
kou merged PR #183: URL: https://github.com/apache/arrow-flight-sql-postgresql/pull/183 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr.

Re: [PR] MINOR: [C#] Bump Grpc.Tools from 2.59.0 to 2.60.0 in /csharp [arrow]

2023-12-18 Thread via GitHub
kou merged PR #39283: URL: https://github.com/apache/arrow/pull/39283 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

Re: [PR] MINOR: [Java] Bump org.apache.maven.surefire:surefire-junit-platform from 3.2.2 to 3.2.3 in /java [arrow]

2023-12-18 Thread via GitHub
kou merged PR #39280: URL: https://github.com/apache/arrow/pull/39280 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

Re: [PR] GH-39138: [R] Fix implicit conversion warnings [arrow]

2023-12-18 Thread via GitHub
github-actions[bot] commented on PR #39250: URL: https://github.com/apache/arrow/pull/39250#issuecomment-1861566756 Revision: d3b8accedae9f0866656f027c29d4a5c98b63cca Submitted crossbow builds: [ursacomputing/crossbow @ actions-bfb08e2c71](https://github.com/ursacomputing/crossbow/bra

Re: [PR] MINOR: [Java] Bump org.apache:apache from 18 to 31 in /java [arrow]

2023-12-18 Thread via GitHub
kou commented on PR #39279: URL: https://github.com/apache/arrow/pull/39279#issuecomment-1861566779 It seems that we need to use Maven 3.6.3 or later for this: https://github.com/apache/arrow/actions/runs/7250708110/job/19751497905?pr=39279#step:8:3468 ```text Error: Failed

Re: [PR] GH-39138: [R] Fix implicit conversion warnings [arrow]

2023-12-18 Thread via GitHub
paleolimbot commented on PR #39250: URL: https://github.com/apache/arrow/pull/39250#issuecomment-1861559828 @github-actions crossbow submit --group r -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] feat: support `LargeList` in `array_element` [arrow-datafusion]

2023-12-18 Thread via GitHub
alamb merged PR #8570: URL: https://github.com/apache/arrow-datafusion/pull/8570 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

Re: [PR] Bump actions/upload-artifact from 3 to 4 [arrow-flight-sql-postgresql]

2023-12-18 Thread via GitHub
kou merged PR #184: URL: https://github.com/apache/arrow-flight-sql-postgresql/pull/184 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr.

Re: [PR] Bump actions/download-artifact from 3 to 4 [arrow-flight-sql-postgresql]

2023-12-18 Thread via GitHub
kou commented on PR #183: URL: https://github.com/apache/arrow-flight-sql-postgresql/pull/183#issuecomment-1861547814 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] feat: support `LargeList` in `array_positions` [arrow-datafusion]

2023-12-18 Thread via GitHub
alamb merged PR #8571: URL: https://github.com/apache/arrow-datafusion/pull/8571 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

Re: [PR] GH-38535: [Python] Fix S3FileSystem equals None segfault [arrow]

2023-12-18 Thread via GitHub
kou commented on PR #39276: URL: https://github.com/apache/arrow/pull/39276#issuecomment-1861534376 I'm not familiar with Python convention but we may want to return `False` instead of raising an exception for `None`. If we use exception here, `s3_file_system in [None, other_s3_file_syste

Re: [PR] Convert Null to Int32(None) for MakeArray at type coercion step [arrow-datafusion]

2023-12-18 Thread via GitHub
alamb closed pull request #7995: Convert Null to Int32(None) for MakeArray at type coercion step URL: https://github.com/apache/arrow-datafusion/pull/7995 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Move Coercion for MakeArray to `coerce_arguments_for_signature` and introduce another one for ArrayAppend [arrow-datafusion]

2023-12-18 Thread via GitHub
alamb merged PR #8317: URL: https://github.com/apache/arrow-datafusion/pull/8317 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

Re: [PR] Add support for conflicting order sensitive aggregates in `ARRAY_AGG` aggregate function [arrow-datafusion]

2023-12-18 Thread via GitHub
alamb commented on code in PR #8558: URL: https://github.com/apache/arrow-datafusion/pull/8558#discussion_r1430570883 ## datafusion/sqllogictest/test_files/groupby.slt: ## @@ -2297,8 +2296,7 @@ Projection: sales_global.country, ARRAY_AGG(sales_global.amount) ORDER BY [sales ph

Re: [PR] Add trait based ScalarUDF API [arrow-datafusion]

2023-12-18 Thread via GitHub
viirya commented on code in PR #8578: URL: https://github.com/apache/arrow-datafusion/pull/8578#discussion_r1430596344 ## datafusion/expr/src/udf.rs: ## @@ -124,22 +169,116 @@ impl ScalarUDF { &self.aliases } -/// Returns this function's signature (what input

Re: [PR] Add trait based ScalarUDF API [arrow-datafusion]

2023-12-18 Thread via GitHub
viirya commented on code in PR #8578: URL: https://github.com/apache/arrow-datafusion/pull/8578#discussion_r1430595595 ## datafusion/expr/src/udf.rs: ## @@ -124,22 +169,116 @@ impl ScalarUDF { &self.aliases } -/// Returns this function's signature (what input

Re: [PR] docs: generate Javadocs [arrow-adbc]

2023-12-18 Thread via GitHub
lidavidm merged PR #1362: URL: https://github.com/apache/arrow-adbc/pull/1362 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.a

Re: [PR] Add trait based ScalarUDF API [arrow-datafusion]

2023-12-18 Thread via GitHub
viirya commented on code in PR #8578: URL: https://github.com/apache/arrow-datafusion/pull/8578#discussion_r1430592815 ## docs/source/library-user-guide/adding-udfs.md: ## @@ -76,7 +76,8 @@ The challenge however is that DataFusion doesn't know about this function. We ne ###

Re: [PR] feat: Add C Data integration test shared library [arrow-nanoarrow]

2023-12-18 Thread via GitHub
pitrou commented on code in PR #337: URL: https://github.com/apache/arrow-nanoarrow/pull/337#discussion_r1430591211 ## src/nanoarrow/integration/c_data_integration.cc: ## @@ -0,0 +1,200 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor lic

Re: [PR] GH-39014: [Java] Add default truststore along with KeychainStore when on Mac system [arrow]

2023-12-18 Thread via GitHub
ravjotbrar commented on code in PR #39235: URL: https://github.com/apache/arrow/pull/39235#discussion_r143054 ## java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/utils/ClientAuthenticationUtils.java: ## @@ -115,6 +115,16 @@ static KeyStore g

Re: [PR] Add trait based ScalarUDF API [arrow-datafusion]

2023-12-18 Thread via GitHub
viirya commented on code in PR #8578: URL: https://github.com/apache/arrow-datafusion/pull/8578#discussion_r1430589301 ## datafusion/expr/src/udf.rs: ## @@ -124,22 +169,116 @@ impl ScalarUDF { &self.aliases } -/// Returns this function's signature (what input

Re: [PR] Add trait based ScalarUDF API [arrow-datafusion]

2023-12-18 Thread via GitHub
viirya commented on code in PR #8578: URL: https://github.com/apache/arrow-datafusion/pull/8578#discussion_r1430587982 ## datafusion/expr/src/udf.rs: ## @@ -79,7 +90,11 @@ impl std::hash::Hash for ScalarUDF { } impl ScalarUDF { -/// Create a new ScalarUDF +/// Create

Re: [PR] Add trait based ScalarUDF API [arrow-datafusion]

2023-12-18 Thread via GitHub
viirya commented on code in PR #8578: URL: https://github.com/apache/arrow-datafusion/pull/8578#discussion_r1430586476 ## datafusion/expr/src/udf.rs: ## @@ -95,6 +110,34 @@ impl ScalarUDF { } } +/// Create a new `ScalarUDF` from a `[ScalarUDFImpl]` trait obje

Re: [PR] GH-39014: [Java] Add default truststore along with KeychainStore when on Mac system [arrow]

2023-12-18 Thread via GitHub
ravjotbrar commented on code in PR #39235: URL: https://github.com/apache/arrow/pull/39235#discussion_r1430566562 ## java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/utils/ClientAuthenticationUtils.java: ## @@ -156,16 +166,9 @@ public static Inp

Re: [PR] Minor: Add some comments to scalar_udf example [arrow-datafusion]

2023-12-18 Thread via GitHub
alamb merged PR #8576: URL: https://github.com/apache/arrow-datafusion/pull/8576 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

Re: [PR] Minor: Add some comments to scalar_udf example [arrow-datafusion]

2023-12-18 Thread via GitHub
alamb commented on PR #8576: URL: https://github.com/apache/arrow-datafusion/pull/8576#issuecomment-1861412740 Thanks @viirya ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] feat: support 'LargeList' in `array_pop_front` and `array_pop_back` [arrow-datafusion]

2023-12-18 Thread via GitHub
comphead commented on code in PR #8569: URL: https://github.com/apache/arrow-datafusion/pull/8569#discussion_r1430561586 ## datafusion/physical-expr/src/array_expressions.rs: ## @@ -673,18 +673,72 @@ where )?)) } +/// If from_front is true, it is array_pop_front, otherwi

Re: [I] [C++][Packaging] Consider stricter compiler options [arrow]

2023-12-18 Thread via GitHub
assignUser commented on issue #39026: URL: https://github.com/apache/arrow/issues/39026#issuecomment-1861398325 We have also recently had issues on CRAN due to conversion warnings that we apparently don't use (#39138). Is there a reason why or should we start using them for libarrow? --

Re: [PR] Refactor TreeNode recursions [arrow-datafusion]

2023-12-18 Thread via GitHub
alamb commented on PR #7942: URL: https://github.com/apache/arrow-datafusion/pull/7942#issuecomment-1861389532 Thanks @peter-toth -- I will try and review this proposal later today or tomorrow -- This is an automated message from the Apache Git Service. To respond to the message, please

[PR] MINOR: [R] Update NEWS.md for 14.0.2 [arrow]

2023-12-18 Thread via GitHub
assignUser opened a new pull request, #39286: URL: https://github.com/apache/arrow/pull/39286 Update NEWS.md with recent changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

  1   2   3   >