Re: [PR] Add support for recursive CTEs [arrow-datafusion]

2024-01-04 Thread via GitHub
jonahgao commented on code in PR #7581: URL: https://github.com/apache/arrow-datafusion/pull/7581#discussion_r1442577152 ## datafusion/physical-plan/src/recursive_query.rs: ## @@ -0,0 +1,361 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributo

Re: [PR] Support S3 Express One Zone [arrow-rs]

2024-01-04 Thread via GitHub
Xuanwo commented on code in PR #5268: URL: https://github.com/apache/arrow-rs/pull/5268#discussion_r1442569651 ## object_store/src/aws/credential.rs: ## @@ -659,6 +685,56 @@ async fn task_credential( }) } +/// A session provider as used by S3 Express One Zone +/// +/// <

Re: [PR] [MINOR]: Add a test case for when target partition is 1, no hash repartition is added to the plan. [arrow-datafusion]

2024-01-04 Thread via GitHub
mustafasrepo commented on code in PR #8757: URL: https://github.com/apache/arrow-datafusion/pull/8757#discussion_r1442568002 ## datafusion/core/src/physical_optimizer/enforce_distribution.rs: ## @@ -927,9 +927,8 @@ fn add_hash_on_top( n_target: usize, repartition_benef

Re: [I] [C++] Use default_cpu_memory_manager() for the default memory pool in MessageDecoder [arrow]

2024-01-04 Thread via GitHub
kou commented on issue #39270: URL: https://github.com/apache/arrow/issues/39270#issuecomment-1878241526 Ah, I reuse this. This issue already has a general title. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] [C++] Use default_cpu_memory_manager() for the default memory pool in MessageDecoder [arrow]

2024-01-04 Thread via GitHub
kou commented on issue #39270: URL: https://github.com/apache/arrow/issues/39270#issuecomment-1878240686 Sorry. I was wrong. I just saw https://github.com/apache/arrow/blob/bec03856799a69bf0e6d4419ab7bc565afd070fe/cpp/src/arrow/device.cc#L101 but copy isn't used by https://github.co

[PR] [MINOR]: Add a test case for when target partition is 1, no hash repartition is added to the plan. [arrow-datafusion]

2024-01-04 Thread via GitHub
mustafasrepo opened a new pull request, #8757: URL: https://github.com/apache/arrow-datafusion/pull/8757 ## Which issue does this PR close? Closes #. ## Rationale for this change ## What changes are included in this PR? ## Are these changes

Re: [PR] Support S3 Express One Zone [arrow-rs]

2024-01-04 Thread via GitHub
cozmo commented on PR #5268: URL: https://github.com/apache/arrow-rs/pull/5268#issuecomment-1878230167 I wanted to play around with some "real world" use cases of S3 Express 1Z serving Parquet, and this seemed like a good place to start. When I got a test project up and running with this br

Re: [PR] GH-39433: [Ruby] Add support for Table.load(format: json) options [arrow]

2024-01-04 Thread via GitHub
github-actions[bot] commented on PR #39464: URL: https://github.com/apache/arrow/pull/39464#issuecomment-1878219476 :warning: GitHub issue #39433 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

[PR] GH-39433: [Ruby] Add support for Table.load(format: json) options [arrow]

2024-01-04 Thread via GitHub
kou opened a new pull request, #39464: URL: https://github.com/apache/arrow/pull/39464 ### Rationale for this change Other `format:` such as `format: :csv` accepts custom options. `format: :json` should also accept them. ### What changes are included in this PR? Use `Arr

Re: [I] Implement monotonicity for ScalarUDF [arrow-datafusion]

2024-01-04 Thread via GitHub
guojidan commented on issue #8756: URL: https://github.com/apache/arrow-datafusion/issues/8756#issuecomment-1878219174 cc @alamb, Is there any mistake in my understanding? if not I will implement this. -- This is an automated message from the Apache Git Service. To respond to the message

Re: [I] Support casting strings like '2001-01-01 01:01:01' to Date32 [arrow-rs]

2024-01-04 Thread via GitHub
gruuya commented on issue #5280: URL: https://github.com/apache/arrow-rs/issues/5280#issuecomment-1878209251 > It seems that pgsql ignores the time part. That is correct, it just throws it away. In fact it seems it tries to parse it as a timestamp first, and if it fails it erro

Re: [PR] GH-39384: [C++] Disable -Werror=attributes for Azure SDK's identity.hpp [arrow]

2024-01-04 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #39448: URL: https://github.com/apache/arrow/pull/39448#issuecomment-1878194785 After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit 7b0c6f955675c9ad309afc5f82da1623f9b13a59. There were 2

Re: [PR] object_store: full HTTP range support [arrow-rs]

2024-01-04 Thread via GitHub
tustvold merged PR #5222: URL: https://github.com/apache/arrow-rs/pull/5222 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apa

Re: [I] object_store: range request with suffix [arrow-rs]

2024-01-04 Thread via GitHub
tustvold closed issue #4611: object_store: range request with suffix URL: https://github.com/apache/arrow-rs/issues/4611 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [I] Support casting strings like '2001-01-01 01:01:01' to Date32 [arrow-rs]

2024-01-04 Thread via GitHub
doki23 commented on issue #5280: URL: https://github.com/apache/arrow-rs/issues/5280#issuecomment-1878171103 I've tried this: ``` postgres=# select '1998-11-30 23:59:59'::date; date 1998-11-30 ``` It seems that pgsql ignores the time part. -- This is

Re: [PR] feat: support `array_resize` [arrow-datafusion]

2024-01-04 Thread via GitHub
jayzhan211 commented on code in PR #8744: URL: https://github.com/apache/arrow-datafusion/pull/8744#discussion_r1442519085 ## datafusion/physical-expr/src/array_expressions.rs: ## @@ -2560,6 +2562,101 @@ pub fn array_distinct(args: &[ArrayRef]) -> Result { } } +pub fn a

Re: [PR] feat: support `array_resize` [arrow-datafusion]

2024-01-04 Thread via GitHub
jayzhan211 commented on code in PR #8744: URL: https://github.com/apache/arrow-datafusion/pull/8744#discussion_r1442517967 ## datafusion/common/src/utils.rs: ## @@ -492,6 +496,58 @@ pub fn list_ndims(data_type: &DataType) -> u64 { } } +/// Create an new empty array based

Re: [PR] Add `schema_err!` error macros with optional backtrace [arrow-datafusion]

2024-01-04 Thread via GitHub
comphead commented on PR #8620: URL: https://github.com/apache/arrow-datafusion/pull/8620#issuecomment-1878154904 > > Here the problem is backtrace-rs regression in Rust lang [rust-lang/rust#119560](https://github.com/rust-lang/rust/issues/119560) > > Wow -- super sleuth 🕵️ Tb

Re: [PR] Object_store: get_file and put_file [arrow-rs]

2024-01-04 Thread via GitHub
troychiu commented on PR #5281: URL: https://github.com/apache/arrow-rs/pull/5281#issuecomment-1878140627 Hi @tustvold, I am relatively new to Rust, and I am wondering if you could give me some suggestions on this PR. I have implemented the functionalities, but I ran into two issues.

Re: [PR] GH-39049: [C++] Use Cast() instead of CastTo() for Dictionary Scalar in test [arrow]

2024-01-04 Thread via GitHub
llama90 commented on code in PR #39362: URL: https://github.com/apache/arrow/pull/39362#discussion_r1442499272 ## cpp/src/arrow/compute/kernels/scalar_cast_dictionary.cc: ## @@ -77,17 +85,23 @@ Status CastToDictionary(KernelContext* ctx, const ExecSpan& batch, ExecResult* o

[PR] Object_store: get_file and put_file [arrow-rs]

2024-01-04 Thread via GitHub
troychiu opened a new pull request, #5281: URL: https://github.com/apache/arrow-rs/pull/5281 # Which issue does this PR close? Closes #5277. # Rationale for this change # What changes are included in this PR? # Are there any user-facing cha

Re: [PR] feat: support `array_resize` [arrow-datafusion]

2024-01-04 Thread via GitHub
Weijun-H commented on PR #8744: URL: https://github.com/apache/arrow-datafusion/pull/8744#issuecomment-1878117609 > array_concat(array_slice(array, 0, min(array_length, count)), array_repeat(default_array, max(count-array_length), 0)) I think this approach is not a better idea, as su

Re: [PR] GH-39049: [C++] Use Cast() instead of CastTo() for Dictionary Scalar in test [arrow]

2024-01-04 Thread via GitHub
kou commented on code in PR #39362: URL: https://github.com/apache/arrow/pull/39362#discussion_r1442490790 ## cpp/src/arrow/compute/kernels/scalar_cast_dictionary.cc: ## @@ -77,17 +85,23 @@ Status CastToDictionary(KernelContext* ctx, const ExecSpan& batch, ExecResult* o retu

Re: [PR] GH-38811: [R] Actually use fetched cmake on macos [arrow]

2024-01-04 Thread via GitHub
assignUser commented on PR #39453: URL: https://github.com/apache/arrow/pull/39453#issuecomment-1878108683 I'll merge & cherry pick this into the cran branch later today(friday). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] Export CMake config [arrow-nanoarrow]

2024-01-04 Thread via GitHub
assignUser commented on issue #350: URL: https://github.com/apache/arrow-nanoarrow/issues/350#issuecomment-1878106898 Yes, exporting a `nanoarrowConfig.cmake` is the 'modern CMake' way to make you lib usable by other projects. I can have a look at the patch/turn it into a pr (feel free to

Re: [PR] GH-38811: [R] Actually use fetched cmake on macos [arrow]

2024-01-04 Thread via GitHub
assignUser commented on PR #39453: URL: https://github.com/apache/arrow/pull/39453#issuecomment-1878102573 > I believe these jobs started using binaries when we overhauled the build system last release. Ah good catch! -- This is an automated message from the Apache Git Service. To

Re: [PR] GH-39049: [C++] Use Cast() instead of CastTo() for Dictionary Scalar in test [arrow]

2024-01-04 Thread via GitHub
llama90 commented on code in PR #39362: URL: https://github.com/apache/arrow/pull/39362#discussion_r1442481006 ## cpp/src/arrow/compute/kernels/scalar_cast_dictionary.cc: ## @@ -77,17 +85,23 @@ Status CastToDictionary(KernelContext* ctx, const ExecSpan& batch, ExecResult* o

Re: [PR] GH-39114: [JS] Fix Example Code [arrow]

2024-01-04 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #39442: URL: https://github.com/apache/arrow/pull/39442#issuecomment-1878076998 After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 6c3972651e2dfa874f9bc38791de329bcdd78ecd. There were no

Re: [PR] GH-39049: [C++] Use Cast() instead of CastTo() for Dictionary Scalar in test [arrow]

2024-01-04 Thread via GitHub
llama90 commented on code in PR #39362: URL: https://github.com/apache/arrow/pull/39362#discussion_r1442466191 ## cpp/src/arrow/scalar_test.cc: ## @@ -1479,11 +1479,12 @@ TEST(TestDictionaryScalar, Cast) { auto alpha = dict->IsValid(i) ? MakeScalar(dict->GetStr

Re: [PR] GH-39049: [C++] Use Cast() instead of CastTo() for Dictionary Scalar in test [arrow]

2024-01-04 Thread via GitHub
llama90 commented on code in PR #39362: URL: https://github.com/apache/arrow/pull/39362#discussion_r1442466313 ## cpp/src/arrow/compute/kernels/scalar_cast_dictionary.cc: ## @@ -77,17 +85,23 @@ Status CastToDictionary(KernelContext* ctx, const ExecSpan& batch, ExecResult* o

Re: [PR] GH-39439 [C++][Gandiva] Add regex_like [arrow]

2024-01-04 Thread via GitHub
kou commented on code in PR #39441: URL: https://github.com/apache/arrow/pull/39441#discussion_r1442459501 ## cpp/src/gandiva/regex_functions_holder.h: ## @@ -150,4 +150,27 @@ class GANDIVA_EXPORT ExtractHolder : public FunctionHolder { int32_t num_groups_pattern_; // numbe

Re: [PR] Add http(s) support to the command line [arrow-datafusion]

2024-01-04 Thread via GitHub
Jefffrey commented on code in PR #8753: URL: https://github.com/apache/arrow-datafusion/pull/8753#discussion_r1442454204 ## datafusion-cli/src/exec.rs: ## @@ -278,6 +279,9 @@ async fn create_external_table( let builder = get_gcs_object_store_builder(url, cmd)?;

Re: [PR] GH-39384: [C++] Disable -Werror=attributes for Azure SDK's identity.hpp [arrow]

2024-01-04 Thread via GitHub
kou commented on PR #39448: URL: https://github.com/apache/arrow/pull/39448#issuecomment-1878049364 We can just report how to reproduce this. We don't want to maintain workarounds. @anjakefala Could you report how to reproduce this problem to the upstream? (It's better that we can reprod

Re: [PR] PARQUET-2411: [C++][Parquet] Allow reading dictionary without reading data via ByteArrayDictionaryRecordReader [arrow]

2024-01-04 Thread via GitHub
mapleFU merged PR #39153: URL: https://github.com/apache/arrow/pull/39153 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apach

[I] Implement monotonicity for ScalarUDF [arrow-datafusion]

2024-01-04 Thread via GitHub
guojidan opened a new issue, #8756: URL: https://github.com/apache/arrow-datafusion/issues/8756 ### Is your feature request related to a problem or challenge? close #8045 . Now `ScalarUDF` is not supported `monotonicity` ### Describe the solution you'd like _No res

Re: [PR] GH-39439 [C++][Gandiva] Add regex_like [arrow]

2024-01-04 Thread via GitHub
likun61 commented on PR #39441: URL: https://github.com/apache/arrow/pull/39441#issuecomment-1877983883 > Could you enable GitHub Actions on your fork for now? > Could you enable GitHub Actions on your fork for now? I enabled my GitHub Actions , but all my actions pa

Re: [I] [Java][CI] Fix or suppress spurious errorprone warnings [arrow]

2024-01-04 Thread via GitHub
vibhatha commented on issue #39330: URL: https://github.com/apache/arrow/issues/39330#issuecomment-1877983760 @danepitkin I took a look at this and seems like there are plenty of cases we need to improve. I will work on this. Thanks for noting this. I noted this a few times when looking

Re: [PR] MINOR: [Java] Bump org.apache.derby:derby from 10.14.2.0 to 10.17.1.0 in /java [arrow]

2024-01-04 Thread via GitHub
vibhatha commented on PR #39281: URL: https://github.com/apache/arrow/pull/39281#issuecomment-1877980178 Seems like this needs a code level change. Should we go ahead? @lidavidm ```bash [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.112 s - in org.apache.

Re: [PR] MINOR: [Java] Bump org.codehaus.mojo:exec-maven-plugin from 1.6.0 to 3.1.1 in /java [arrow]

2024-01-04 Thread via GitHub
vibhatha commented on PR #39374: URL: https://github.com/apache/arrow/pull/39374#issuecomment-1877977849 > Hmm, we use newer Maven in other parts of CI. Maybe we should just bump Maven to some baseline? There are other Dependabot PRs that also assume later Maven versions +1, and ther

Re: [PR] MINOR: [Java] Bump com.google.errorprone:error_prone_core from 2.4.0 to 2.24.0 in /java [arrow]

2024-01-04 Thread via GitHub
vibhatha commented on PR #39409: URL: https://github.com/apache/arrow/pull/39409#issuecomment-1877975788 This has been updated here: https://github.com/apache/arrow/pull/39452 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] MINOR: [Java] Bump org.apache.maven.plugins:maven-plugin-plugin from 3.6.0 to 3.10.2 in /java [arrow]

2024-01-04 Thread via GitHub
vibhatha commented on PR #39412: URL: https://github.com/apache/arrow/pull/39412#issuecomment-1877974673 @lidavidm this change looks okay and seems like CIs are also passing. The failure is because of an already fixed issue here https://github.com/apache/arrow/issues/39423 After reba

[I] Use correct attribution in footer of documentation pages [arrow-datafusion]

2024-01-04 Thread via GitHub
andygrove opened a new issue, #8755: URL: https://github.com/apache/arrow-datafusion/issues/8755 ### Is your feature request related to a problem or challenge? To be inline with ASF policy, we should update the text in the footer used in the docs (e.g. https://arrow.apache.org/datafus

Re: [PR] GH-39456: [Go][Parquet] Arrow DATE64 Type Coerced to Parquet DATE Logical Type [arrow]

2024-01-04 Thread via GitHub
github-actions[bot] commented on PR #39460: URL: https://github.com/apache/arrow/pull/39460#issuecomment-1877966447 :warning: GitHub issue #39456 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

[PR] GH-39456: [Go][Parquet] Arrow DATE64 Type Coerced to Parquet DATE Logical Type [arrow]

2024-01-04 Thread via GitHub
joellubi opened a new pull request, #39460: URL: https://github.com/apache/arrow/pull/39460 ### Rationale for this change Closes: #39456 ### What changes are included in this PR? Update physical and logical type mapping from Arrow to Parquet for DATE64 t

Re: [I] Port tests in `expr.rs` to sqllogictest [arrow-datafusion]

2024-01-04 Thread via GitHub
comphead commented on issue #8201: URL: https://github.com/apache/arrow-datafusion/issues/8201#issuecomment-1877964011 Picking it, was working on another feature and started to move tests in `expr.rs` -- This is an automated message from the Apache Git Service. To respond to the message,

[I] Add trademark symbol to DataFusion logo [arrow-datafusion]

2024-01-04 Thread via GitHub
andygrove opened a new issue, #8754: URL: https://github.com/apache/arrow-datafusion/issues/8754 ### Is your feature request related to a problem or challenge? As stated in the [Apache Project Website Branding Policy](https://www.apache.org/foundation/marks/pmcs#attributions),

Re: [PR] GH-37848: [C++][Gandiva] Migrate LLVM JIT engine from MCJIT to ORC v2/LLJIT [arrow]

2024-01-04 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #39098: URL: https://github.com/apache/arrow/pull/39098#issuecomment-1877946795 After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit 83cba25017a5c3a03e47f1851f242fa284f93533. There were no

Re: [PR] GH-39449: [C++] Use default Azure credentials implicitly and support anonymous credentials explicitly [arrow]

2024-01-04 Thread via GitHub
felipecrv commented on code in PR #39450: URL: https://github.com/apache/arrow/pull/39450#discussion_r1442390120 ## cpp/src/arrow/filesystem/azurefs.h: ## @@ -92,30 +107,30 @@ struct ARROW_EXPORT AzureOptions { private: enum class CredentialKind { +kDefault, kAno

Re: [PR] PARQUET-2411: [C++][Parquet] Allow reading dictionary without reading data via ByteArrayDictionaryRecordReader [arrow]

2024-01-04 Thread via GitHub
emkornfield commented on PR #39153: URL: https://github.com/apache/arrow/pull/39153#issuecomment-1877936028 Yes, I think we can merge it if there are no objections. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] GH-39449: [C++] Use default Azure credentials implicitly and support anonymous credentials explicitly [arrow]

2024-01-04 Thread via GitHub
felipecrv commented on code in PR #39450: URL: https://github.com/apache/arrow/pull/39450#discussion_r1442385924 ## cpp/src/arrow/filesystem/azurefs.h: ## @@ -92,30 +107,30 @@ struct ARROW_EXPORT AzureOptions { private: enum class CredentialKind { +kDefault, kAno

Re: [PR] GH-39384: [C++] Disable -Werror=attributes for Azure SDK's identity.hpp [arrow]

2024-01-04 Thread via GitHub
felipecrv merged PR #39448: URL: https://github.com/apache/arrow/pull/39448 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apa

Re: [PR] GH-39384: [C++] Disable -Werror=attributes for Azure SDK's identity.hpp [arrow]

2024-01-04 Thread via GitHub
felipecrv commented on PR #39448: URL: https://github.com/apache/arrow/pull/39448#issuecomment-1877927701 > +1 > > Could you report this to the upstream? I read the code and I don't understand what is triggering the warning. -- This is an automated message from the Apache Git

Re: [PR] GH-38998: [Java] Build memory-core and memory-unsafe as JPMS modules [arrow]

2024-01-04 Thread via GitHub
jduo commented on code in PR #39011: URL: https://github.com/apache/arrow/pull/39011#discussion_r1442372645 ## java/memory/memory-core/pom.xml: ## @@ -54,6 +53,30 @@ + + error-prone-jdk11+ Review Comment: Changed the profile to be jdk11+. -- This is an

Re: [PR] GH-38998: [Java] Build memory-core and memory-unsafe as JPMS modules [arrow]

2024-01-04 Thread via GitHub
jduo commented on code in PR #39011: URL: https://github.com/apache/arrow/pull/39011#discussion_r1442372526 ## java/memory/memory-unsafe/src/test/java/org/apache/arrow/memory/unsafe/TestUnsafeAllocationManager.java: ## @@ -15,20 +15,25 @@ * limitations under the License. */

Re: [PR] GH-38998: [Java] Build memory-core and memory-unsafe as JPMS modules [arrow]

2024-01-04 Thread via GitHub
jduo commented on code in PR #39011: URL: https://github.com/apache/arrow/pull/39011#discussion_r1442372410 ## java/memory/memory-core/src/test/java/org/apache/arrow/memory/TestBaseAllocator.java: ## @@ -448,73 +443,73 @@ public ArrowBuf empty() { @Test public void testRoo

Re: [PR] GH-38998: [Java] Build memory-core and memory-unsafe as JPMS modules [arrow]

2024-01-04 Thread via GitHub
jduo commented on code in PR #39011: URL: https://github.com/apache/arrow/pull/39011#discussion_r1442372257 ## java/maven/module-info-compiler-maven-plugin/pom.xml: ## @@ -0,0 +1,130 @@ + + +http://maven.apache.org/POM/4.0.0"; + xmlns:xsi="http://www.w3.org/2001/XMLSchem

Re: [PR] GH-38998: [Java] Build memory-core and memory-unsafe as JPMS modules [arrow]

2024-01-04 Thread via GitHub
jduo commented on code in PR #39011: URL: https://github.com/apache/arrow/pull/39011#discussion_r1442367738 ## java/memory/memory-core/src/main/java/org/apache/arrow/memory/util/MemoryUtil.java: ## @@ -142,7 +142,7 @@ public Object run() { // the static fields above get i

Re: [I] Support casting strings like '2001-01-01 01:01:01' to Date32 [arrow-rs]

2024-01-04 Thread via GitHub
tustvold commented on issue #5280: URL: https://github.com/apache/arrow-rs/issues/5280#issuecomment-1877879375 What is the postgres behaviour if given a non-zero time? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[PR] refactor(c/driver/postgresql): Split postgres_copy_reader.h into reader/writer headers [arrow-adbc]

2024-01-04 Thread via GitHub
WillAyd opened a new pull request, #1432: URL: https://github.com/apache/arrow-adbc/pull/1432 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

Re: [I] Expression simplifier does not simplify `A = B AND B = A` [arrow-datafusion]

2024-01-04 Thread via GitHub
Jefffrey commented on issue #8724: URL: https://github.com/apache/arrow-datafusion/issues/8724#issuecomment-1877875752 The canonicalize part sounds good. The remove redundancy, I think is already taken care of by expr_simplifier? e.g. https://github.com/apache/arrow-dat

Re: [I] Expression simplifier does not simplify `A = B AND B = A` [arrow-datafusion]

2024-01-04 Thread via GitHub
alamb commented on issue #8724: URL: https://github.com/apache/arrow-datafusion/issues/8724#issuecomment-1877868644 I took a shot at writing up a description @Jefffrey -- let me know if that makes sense -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [I] Expression simplifier does not simplify `A = B AND B = A` [arrow-datafusion]

2024-01-04 Thread via GitHub
alamb commented on issue #8724: URL: https://github.com/apache/arrow-datafusion/issues/8724#issuecomment-1877868335 I am marking this as a good first issue but it is really a medium sized project However, I think it is well specified and the existing code is straightforward to exte

Re: [I] Should update rust to 1.72 [arrow-datafusion]

2024-01-04 Thread via GitHub
Jefffrey commented on issue #8743: URL: https://github.com/apache/arrow-datafusion/issues/8743#issuecomment-1877855154 Also, worth fixing as well places which don't use the workspace rust-version, e.g. https://github.com/apache/arrow-datafusion/blob/e5036d0e760b637724e8ac59c32924f12

Re: [PR] Convert Binary Operator `StringConcat` to Function for `array_concat`, `array_append` and `array_prepend` [arrow-datafusion]

2024-01-04 Thread via GitHub
alamb commented on PR #8636: URL: https://github.com/apache/arrow-datafusion/pull/8636#issuecomment-1877854772 cc @Veeupup as I think you were working on something similar -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] Should update rust to 1.72 [arrow-datafusion]

2024-01-04 Thread via GitHub
Jefffrey commented on issue #8743: URL: https://github.com/apache/arrow-datafusion/issues/8743#issuecomment-1877848985 > Maybe we can figure out why the CI check isn't failing and then upgrade to rust 1.72 as a MSRV CI check not failing because ahash didn't bump their own MSRV. I've

Re: [PR] GH-38772: [C++] Implement directory semantics even when the storage account doesn't support HNS [arrow]

2024-01-04 Thread via GitHub
felipecrv commented on PR #39361: URL: https://github.com/apache/arrow/pull/39361#issuecomment-1877843693 @kou done. These turned out to be Azurite-only tests that needed updating since the semantics are now different than what was assumed when the tests were written. -- This is an autom

Re: [PR] Change `ScalarValue::{List, LargeList, FixedSizedList}` to take specific types rather than `ArrayRef` [arrow-datafusion]

2024-01-04 Thread via GitHub
alamb commented on PR #8562: URL: https://github.com/apache/arrow-datafusion/pull/8562#issuecomment-1877839443 I will try and find time to review this PR later this week . Sorry for the delay -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] Minor: reduce code duplication in PruningPredicate test [arrow-datafusion]

2024-01-04 Thread via GitHub
alamb commented on PR #8441: URL: https://github.com/apache/arrow-datafusion/pull/8441#issuecomment-1877837881 Thank you @viirya and @Ted-Jiang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Minor: reduce code duplication in PruningPredicate test [arrow-datafusion]

2024-01-04 Thread via GitHub
alamb merged PR #8441: URL: https://github.com/apache/arrow-datafusion/pull/8441 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

Re: [PR] Add `schema_err!` error macros with optional backtrace [arrow-datafusion]

2024-01-04 Thread via GitHub
alamb commented on PR #8620: URL: https://github.com/apache/arrow-datafusion/pull/8620#issuecomment-1877835494 > Here the problem is backtrace-rs regression in Rust lang [rust-lang/rust#119560](https://github.com/rust-lang/rust/issues/119560) Wow -- super sleuth 🕵️ -- This is an

Re: [I] Generate an Expr from string [arrow-datafusion]

2024-01-04 Thread via GitHub
alamb commented on issue #8736: URL: https://github.com/apache/arrow-datafusion/issues/8736#issuecomment-1877834794 Making exprs from strings sounds like a good idea to me. BTW you can also use the more fluent style API to make exprs, which while still tedious is still a bit better:

Re: [PR] GH-39394: [Python] pa.array correctly convert nans from an array with from_pandas=True [arrow]

2024-01-04 Thread via GitHub
github-actions[bot] commented on PR #39458: URL: https://github.com/apache/arrow/pull/39458#issuecomment-1877814418 :warning: GitHub issue #39394 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

[PR] GH-39394: [Python] pa.array correctly convert nans from an array with from_pandas=True [arrow]

2024-01-04 Thread via GitHub
LucasG0 opened a new pull request, #39458: URL: https://github.com/apache/arrow/pull/39458 ### What changes are included in this PR? Replace nans by nulls when `from_pandas=True` and input object in an `Array`. I am not sure if there is a faster/cleaner way to do it. ### Are th

Re: [PR] GH-38772: [C++] Implement directory semantics even when the storage account doesn't support HNS [arrow]

2024-01-04 Thread via GitHub
kou commented on code in PR #39361: URL: https://github.com/apache/arrow/pull/39361#discussion_r1442282766 ## cpp/src/arrow/filesystem/azurefs.cc: ## @@ -1033,47 +1042,88 @@ class AzureFileSystem::Impl { return info; } catch (const Storage::StorageException& exceptio

Re: [PR] GH-38772: [C++] Implement directory semantics even when the storage account doesn't support HNS [arrow]

2024-01-04 Thread via GitHub
kou commented on PR #39361: URL: https://github.com/apache/arrow/pull/39361#issuecomment-1877794677 Could you check CI failures? https://github.com/apache/arrow/actions/runs/7414494286/job/20175680211?pr=39361#step:6:3814 ```text [ RUN ] TestAzuriteFileSystem.DeleteDirC

Re: [PR] DynamoDB ConditionalPut [arrow-rs]

2024-01-04 Thread via GitHub
tustvold merged PR #5247: URL: https://github.com/apache/arrow-rs/pull/5247 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apa

[I] Support casting strings like '2001-01-01 01:01:01' to Date32 [arrow-rs]

2024-01-04 Thread via GitHub
gruuya opened a new issue, #5280: URL: https://github.com/apache/arrow-rs/issues/5280 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** I'd like to be able to cast strings that represent timestamps into Date32 objects. Th

Re: [PR] GH-39114: [JS] Fix Example Code [arrow]

2024-01-04 Thread via GitHub
domoritz merged PR #39442: URL: https://github.com/apache/arrow/pull/39442 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apac

Re: [PR] DynamoDB ConditionalPut [arrow-rs]

2024-01-04 Thread via GitHub
roeap commented on PR #5247: URL: https://github.com/apache/arrow-rs/pull/5247#issuecomment-180192 @tustvold - I'll take a look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] GH-38964: [C++] Fix spelling (compute) [arrow]

2024-01-04 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #38965: URL: https://github.com/apache/arrow/pull/38965#issuecomment-1877768009 After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit ccc674c56f3473c9556a5af96dff9d156f559663. There were no

[PR] Add http(s) support to the command line [arrow-datafusion]

2024-01-04 Thread via GitHub
kcolford opened a new pull request, #8753: URL: https://github.com/apache/arrow-datafusion/pull/8753 ## Which issue does this PR close? Closes #8752 ## Rationale for this change ## What changes are included in this PR? ## Are these changes t

Re: [PR] GH-38772: [C++] Implement directory semantics even when the storage account doesn't support HNS [arrow]

2024-01-04 Thread via GitHub
felipecrv commented on code in PR #39361: URL: https://github.com/apache/arrow/pull/39361#discussion_r1442243505 ## cpp/src/arrow/filesystem/azurefs.cc: ## @@ -1288,97 +1328,81 @@ class AzureFileSystem::Impl { return ptr; } - Status CreateDir(const AzureLocation& loca

[I] Add http(s) support to the cli [arrow-datafusion]

2024-01-04 Thread via GitHub
kcolford opened a new issue, #8752: URL: https://github.com/apache/arrow-datafusion/issues/8752 ### Is your feature request related to a problem or challenge? It's low hanging fruit that counterparts like duckdb already support so we might as well. ### Describe the solution you

Re: [PR] GH-39449: [C++] Use default Azure credentials implicitly and support anonymous credentials explicitly [arrow]

2024-01-04 Thread via GitHub
kou commented on code in PR #39450: URL: https://github.com/apache/arrow/pull/39450#discussion_r1442238902 ## cpp/src/arrow/filesystem/azurefs.h: ## @@ -92,30 +107,30 @@ struct ARROW_EXPORT AzureOptions { private: enum class CredentialKind { +kDefault, kAnonymous

Re: [PR] GH-38772: [C++] Implement directory semantics even when the storage account doesn't support HNS [arrow]

2024-01-04 Thread via GitHub
felipecrv commented on PR #39361: URL: https://github.com/apache/arrow/pull/39361#issuecomment-1877717949 @Tom-Newton @kou I pushed changes based on your feedback. I would love to merge it before tomorrow's release cut. -- This is an automated message from the Apache Git Service. To respo

Re: [PR] GH-38772: [C++] Implement directory semantics even when the storage account doesn't support HNS [arrow]

2024-01-04 Thread via GitHub
felipecrv commented on code in PR #39361: URL: https://github.com/apache/arrow/pull/39361#discussion_r1442213883 ## cpp/src/arrow/filesystem/azurefs.cc: ## @@ -1645,21 +1748,93 @@ Result AzureFileSystem::GetFileInfo(const FileSelector& select) Status AzureFileSystem::CreateD

Re: [PR] DynamoDB ConditionalPut [arrow-rs]

2024-01-04 Thread via GitHub
tustvold commented on PR #5247: URL: https://github.com/apache/arrow-rs/pull/5247#issuecomment-1877708421 @roeap sorry to dump this on you, but I just realised this needs to go in the release or it'll be a major breaking change... Could you perhaps take a look? :pray: -- This is an autom

Re: [PR] GH-39163: [C++] Add missing data copy in StreamDecoder::Consume(data) [arrow]

2024-01-04 Thread via GitHub
kou commented on PR #39164: URL: https://github.com/apache/arrow/pull/39164#issuecomment-1877706289 Sorry. I forgot to reply this. I'll do it today. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] GH-39214: [Java] Support reproducible build [arrow]

2024-01-04 Thread via GitHub
jbonofre commented on code in PR #39215: URL: https://github.com/apache/arrow/pull/39215#discussion_r1442205290 ## dev/release/test-helper.rb: ## @@ -83,15 +83,60 @@ def parse_patch(patch) when /\A@@/ in_hunk = true diffs.last[:hunks] << [] - when /

Re: [PR] GH-39214: [Java] Support reproducible build [arrow]

2024-01-04 Thread via GitHub
jbonofre commented on code in PR #39215: URL: https://github.com/apache/arrow/pull/39215#discussion_r1442204426 ## dev/release/test-helper.rb: ## @@ -83,15 +83,60 @@ def parse_patch(patch) when /\A@@/ in_hunk = true diffs.last[:hunks] << [] - when /

Re: [PR] GH-38772: [C++] Implement directory semantics even when the storage account doesn't support HNS [arrow]

2024-01-04 Thread via GitHub
felipecrv commented on code in PR #39361: URL: https://github.com/apache/arrow/pull/39361#discussion_r1442203883 ## cpp/src/arrow/filesystem/azurefs.cc: ## @@ -1288,97 +1338,81 @@ class AzureFileSystem::Impl { return ptr; } - Status CreateDir(const AzureLocation& loca

Re: [PR] GH-38772: [C++] Implement directory semantics even when the storage account doesn't support HNS [arrow]

2024-01-04 Thread via GitHub
felipecrv commented on code in PR #39361: URL: https://github.com/apache/arrow/pull/39361#discussion_r1442203542 ## cpp/src/arrow/filesystem/azurefs.cc: ## @@ -1288,97 +1338,81 @@ class AzureFileSystem::Impl { return ptr; } - Status CreateDir(const AzureLocation& loca

Re: [PR] GH-38772: [C++] Implement directory semantics even when the storage account doesn't support HNS [arrow]

2024-01-04 Thread via GitHub
felipecrv commented on code in PR #39361: URL: https://github.com/apache/arrow/pull/39361#discussion_r1442193756 ## cpp/src/arrow/filesystem/azurefs.cc: ## @@ -1288,97 +1338,81 @@ class AzureFileSystem::Impl { return ptr; } - Status CreateDir(const AzureLocation& loca

Re: [PR] GH-39454: [JS] Support LargeList [arrow]

2024-01-04 Thread via GitHub
github-actions[bot] commented on PR #39457: URL: https://github.com/apache/arrow/pull/39457#issuecomment-1877679771 :warning: GitHub issue #39454 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

[PR] GH-39454: [JS] Support LargeList [arrow]

2024-01-04 Thread via GitHub
kylebarron opened a new pull request, #39457: URL: https://github.com/apache/arrow/pull/39457 ### Rationale for this change Support large list type in JS bindings. With the addition of LargeBinary and LargeUtf8, I'd like to get LargeList in as well, so that I don't have

Re: [PR] chore: temporary branch for IOx update (12-25-2023 to TBD) [arrow-datafusion]

2024-01-04 Thread via GitHub
appletreeisyellow closed pull request #8751: chore: temporary branch for IOx update (12-25-2023 to TBD) URL: https://github.com/apache/arrow-datafusion/pull/8751 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[PR] chore: temporary branch for IOx update (12-25-2023 to TBD) [arrow-datafusion]

2024-01-04 Thread via GitHub
appletreeisyellow opened a new pull request, #8751: URL: https://github.com/apache/arrow-datafusion/pull/8751 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] object_store: full HTTP range support [arrow-rs]

2024-01-04 Thread via GitHub
clbarnes commented on PR #5222: URL: https://github.com/apache/arrow-rs/pull/5222#issuecomment-1877677979 Yes, this looks good to me! Thank you so much for your patience and willingness to wade into this. -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [I] Should update rust to 1.72 [arrow-datafusion]

2024-01-04 Thread via GitHub
Dandandan commented on issue #8743: URL: https://github.com/apache/arrow-datafusion/issues/8743#issuecomment-1877675833 This is caused by a newer version of ahash, see https://github.com/tkaitchuck/aHash/issues/195 -- This is an automated message from the Apache Git Service. To respond t

Re: [PR] GH-38772: [C++] Implement directory semantics even when the storage account doesn't support HNS [arrow]

2024-01-04 Thread via GitHub
felipecrv commented on code in PR #39361: URL: https://github.com/apache/arrow/pull/39361#discussion_r1442187077 ## cpp/src/arrow/filesystem/azurefs.cc: ## @@ -1033,47 +1042,88 @@ class AzureFileSystem::Impl { return info; } catch (const Storage::StorageException& ex

  1   2   3   >