[GitHub] [arrow-rs] liurenjie1024 closed issue #4708: bug: DataType's contains method should consider decimal's precision and scale.

2023-08-16 Thread via GitHub
liurenjie1024 closed issue #4708: bug: DataType's contains method should consider decimal's precision and scale. URL: https://github.com/apache/arrow-rs/issues/4708 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [arrow] conbench-apache-arrow[bot] commented on pull request #37142: GH-37141: [GLib][FlightRPC] Add more ArrowFlight::ClientOptions properties

2023-08-16 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #37142: URL: https://github.com/apache/arrow/pull/37142#issuecomment-1681716827 After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit f3010bac94cbd588ecebd6e7839f9d56e97b1a9b. There were no

[GitHub] [arrow] github-actions[bot] commented on pull request #36616: WIP: Dummy PR to check maint-13.0.0 status

2023-08-16 Thread via GitHub
github-actions[bot] commented on PR #36616: URL: https://github.com/apache/arrow/pull/36616#issuecomment-1681676456 Revision: 0a9898410b69c2d712fbb37ce3a1e63c9c34c38d Submitted crossbow builds: [ursacomputing/crossbow @ actions-48118f4f0a](https://github.com/ursacomputing/crossbow/bra

[GitHub] [arrow] assignUser commented on pull request #36616: WIP: Dummy PR to check maint-13.0.0 status

2023-08-16 Thread via GitHub
assignUser commented on PR #36616: URL: https://github.com/apache/arrow/pull/36616#issuecomment-1681664375 @github-actions crossbow submit homebrew-r-* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] github-actions[bot] commented on pull request #36616: WIP: Dummy PR to check maint-13.0.0 status

2023-08-16 Thread via GitHub
github-actions[bot] commented on PR #36616: URL: https://github.com/apache/arrow/pull/36616#issuecomment-1681659518 Revision: 0a9898410b69c2d712fbb37ce3a1e63c9c34c38d Submitted crossbow builds: [ursacomputing/crossbow @ actions-675fc98ef4](https://github.com/ursacomputing/crossbow/bra

[GitHub] [arrow] assignUser commented on pull request #36616: WIP: Dummy PR to check maint-13.0.0 status

2023-08-16 Thread via GitHub
assignUser commented on PR #36616: URL: https://github.com/apache/arrow/pull/36616#issuecomment-1681657551 @github-actions crossbow submit java-jars -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [arrow] AlenkaF commented on issue #37009: Can't convert UTC tz-aware array to pandas using pd.ArrowDtype

2023-08-16 Thread via GitHub
AlenkaF commented on issue #37009: URL: https://github.com/apache/arrow/issues/37009#issuecomment-1681644987 The errors do seem related to the ones you listed. Could we first check the package versions? When running on pyarrow dev and pandas 2.0.0 the code seems to work for me: ```

[GitHub] [arrow-datafusion] Weijun-H opened a new pull request, #7313: feat: `array-empty`

2023-08-16 Thread via GitHub
Weijun-H opened a new pull request, #7313: URL: https://github.com/apache/arrow-datafusion/pull/7313 ## Which issue does this PR close? Closes #7290 ## Rationale for this change ## What changes are included in this PR? ## Are these changes t

[GitHub] [arrow] AlenkaF commented on a diff in pull request #37218: GH-37217: [Python] Add missing docstrings to Cython

2023-08-16 Thread via GitHub
AlenkaF commented on code in PR #37218: URL: https://github.com/apache/arrow/pull/37218#discussion_r1296667456 ## python/pyarrow/table.pxi: ## @@ -160,6 +160,17 @@ cdef class ChunkedArray(_PandasConvertible): return frombytes(result, safe=True) def format(self, *

[GitHub] [arrow-rs] liurenjie1024 opened a new issue, #4708: bug: DataType's contains method should consider decimal's precision and scale.

2023-08-16 Thread via GitHub
liurenjie1024 opened a new issue, #4708: URL: https://github.com/apache/arrow-rs/issues/4708 **Describe the bug** ```rust let a = DataType::Decimal(28, 10); let b = DataType::Decimal(20, 10); assert!(a.contains(b)); ``` The above assertation failed, but a is a su

[GitHub] [arrow] conbench-apache-arrow[bot] commented on pull request #37147: GH-36969: [R] Disable GCS by default when doing a bundled build on gcc-13

2023-08-16 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #37147: URL: https://github.com/apache/arrow/pull/37147#issuecomment-1681575349 After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit 3e7b35bacca776fbf2e3e313e32d872b73635c87. There were no

[GitHub] [arrow] kou commented on a diff in pull request #37218: GH-37217: [Python] Add missing docstrings to Cython

2023-08-16 Thread via GitHub
kou commented on code in PR #37218: URL: https://github.com/apache/arrow/pull/37218#discussion_r1296620274 ## python/pyarrow/gandiva.pyx: ## @@ -328,6 +428,17 @@ cdef class TreeExprBuilder(_Weakrefable): return Node.create(r) def make_or(self, children): +

[GitHub] [arrow-nanoarrow] paleolimbot commented on issue #244: Test discovery times out on Conda/MacOS M1

2023-08-16 Thread via GitHub
paleolimbot commented on issue #244: URL: https://github.com/apache/arrow-nanoarrow/issues/244#issuecomment-1681535614 Hmm...this popped up again on CI. I was able to reproduce the first time locally; however, I wasn't able to reproduce any subsequent time. For future me: ```bash

[GitHub] [arrow] kou commented on issue #37195: memory alignment in Flight rpc

2023-08-16 Thread via GitHub
kou commented on issue #37195: URL: https://github.com/apache/arrow/issues/37195#issuecomment-1681518598 It seems that your Flight RPC server produces non-aligned record batches. Is it implemented in C++ API? -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow-nanoarrow] paleolimbot commented on a diff in pull request #276: feat: Implement ArrowBitmapUnpackInt8Unsafe

2023-08-16 Thread via GitHub
paleolimbot commented on code in PR #276: URL: https://github.com/apache/arrow-nanoarrow/pull/276#discussion_r1296598699 ## src/nanoarrow/buffer_inline.h: ## @@ -236,6 +248,46 @@ static inline int8_t ArrowBitGet(const uint8_t* bits, int64_t i) { return (bits[i >> 3] >> (i &

[GitHub] [arrow] li-wu commented on issue #37193: Support setting keepalive grpc arguments for Arrow Flight server

2023-08-16 Thread via GitHub
li-wu commented on issue #37193: URL: https://github.com/apache/arrow/issues/37193#issuecomment-1681475963 Thank you. @lidavidm -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [arrow] conbench-apache-arrow[bot] commented on pull request #37191: GH-37179: [MATLAB] Add a test utility that creates a MATLAB `table` containing all supported types

2023-08-16 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #37191: URL: https://github.com/apache/arrow/pull/37191#issuecomment-1681465196 After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 01a9f72727c1ea80564a5a12ea64727774df4be7. There were no

[GitHub] [arrow] kou merged pull request #37167: GH-37144: [C++] Add RecordBatchFileReader::To{RecordBatches,Table}

2023-08-16 Thread via GitHub
kou merged PR #37167: URL: https://github.com/apache/arrow/pull/37167 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

[GitHub] [arrow] kou commented on a diff in pull request #37167: GH-37144: [C++] Add RecordBatchFileReader::To{RecordBatches,Table}

2023-08-16 Thread via GitHub
kou commented on code in PR #37167: URL: https://github.com/apache/arrow/pull/37167#discussion_r1296573973 ## cpp/src/arrow/table.h: ## @@ -75,7 +75,7 @@ class ARROW_EXPORT Table { /// \brief Construct a Table from a RecordBatchReader. /// - /// \param[in] reader the ar

[GitHub] [arrow] kou commented on issue #37208: [R] Arrow package install - R binary version mismatch

2023-08-16 Thread via GitHub
kou commented on issue #37208: URL: https://github.com/apache/arrow/issues/37208#issuecomment-1681457983 Could you open a PR for this suggestion? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [arrow] kou commented on a diff in pull request #37215: GH-37210: [Docs][MATLAB] Update MATLAB `README.md` to mention support for new MATLAB APIs (e.g. `RecordBatch`, `Field`, `Schema`, etc.)

2023-08-16 Thread via GitHub
kou commented on code in PR #37215: URL: https://github.com/apache/arrow/pull/37215#discussion_r1296559210 ## matlab/README.md: ## @@ -205,20 +209,417 @@ arrowArray = ] ``` +### Arrow `RecordBatch` class + + Create an Arrow `RecordBatch` from a MATLAB `table` + +```matla

[GitHub] [arrow] kou merged pull request #37211: GH-37209: [CI][Docs][MATLAB] Remove support for `MATLAB_ARROW_INTERFACE` flag from CMake build system and build new MATLAB Interface code by default

2023-08-16 Thread via GitHub
kou merged PR #37211: URL: https://github.com/apache/arrow/pull/37211 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

[GitHub] [arrow] kou commented on issue #37201: [CI][Packaging][Java] java-jars job fail on macOS aarch_64

2023-08-16 Thread via GitHub
kou commented on issue #37201: URL: https://github.com/apache/arrow/issues/37201#issuecomment-1681422983 https://gitlab.kitware.com/cmake/cmake/-/issues/25181#note_1401407 > I managed to wrap up that work quickly. I've released 3.27.3. It should be announced later today. https:

[GitHub] [arrow] kou commented on issue #37201: [CI][Packaging][Java] java-jars job fail on macOS aarch_64

2023-08-16 Thread via GitHub
kou commented on issue #37201: URL: https://github.com/apache/arrow/issues/37201#issuecomment-1681421554 How about adding `-DXCODE_VERSION=` CMake option as a workaround? ```diff diff --git a/ci/scripts/java_jni_macos_build.sh b/ci/scripts/java_jni_macos_build.sh index d66c39a37

[GitHub] [arrow] kou commented on issue #37201: [CI][Packaging][Java] java-jars job fail on macOS aarch_64

2023-08-16 Thread via GitHub
kou commented on issue #37201: URL: https://github.com/apache/arrow/issues/37201#issuecomment-1681420045 Duplicated of #37159. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [arrow-datafusion] devinjdangelo commented on a diff in pull request #7291: Support `EXPLAIN COPY`

2023-08-16 Thread via GitHub
devinjdangelo commented on code in PR #7291: URL: https://github.com/apache/arrow-datafusion/pull/7291#discussion_r1296541827 ## datafusion/sql/src/parser.rs: ## @@ -44,6 +44,35 @@ fn parse_file_type(s: &str) -> Result { Ok(s.to_uppercase()) } +/// DataFusion specific EX

[GitHub] [arrow-datafusion] devinjdangelo commented on a diff in pull request #7291: Support `EXPLAIN COPY`

2023-08-16 Thread via GitHub
devinjdangelo commented on code in PR #7291: URL: https://github.com/apache/arrow-datafusion/pull/7291#discussion_r1296540850 ## datafusion/expr/src/logical_plan/plan.rs: ## @@ -1100,15 +1100,13 @@ impl LogicalPlan { options, }) => {

[GitHub] [arrow-datafusion] devinjdangelo commented on a diff in pull request #7312: Document and `scratch` directory for sqllogictest and make test specific

2023-08-16 Thread via GitHub
devinjdangelo commented on code in PR #7312: URL: https://github.com/apache/arrow-datafusion/pull/7312#discussion_r1296534088 ## datafusion/sqllogictest/README.md: ## @@ -177,14 +177,32 @@ You can update the tests / generate expected output by passing the `--complete` cargo te

[GitHub] [arrow] conbench-apache-arrow[bot] commented on pull request #37184: GH-37182: [MATLAB] Add public `Schema` property to MATLAB `arrow.tabular.RecordBatch` class

2023-08-16 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #37184: URL: https://github.com/apache/arrow/pull/37184#issuecomment-1681348244 After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 0432aaed4deae54c6a9f82e3caee949fc9639f97. There were no

[GitHub] [arrow] conbench-apache-arrow[bot] commented on pull request #37189: GH-37187: [MATLAB] Re-implement `tfeathermex.m` tests in terms of new internal Feather Reader and Writer objects

2023-08-16 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #37189: URL: https://github.com/apache/arrow/pull/37189#issuecomment-1681342796 After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 7c8c2fb1d7ca487bb52ecd1854e3c6906f21875a. There were no

[GitHub] [arrow] jp0317 commented on a diff in pull request #36510: PARQUET-2321: [C++] allow customized buffer size when creating ArrowInputStream for a column PageReader

2023-08-16 Thread via GitHub
jp0317 commented on code in PR #36510: URL: https://github.com/apache/arrow/pull/36510#discussion_r1296490760 ## cpp/src/parquet/properties.h: ## @@ -56,15 +57,28 @@ constexpr int32_t kDefaultThriftStringSizeLimit = 100 * 1000 * 1000; // kDefaultStringSizeLimit. constexpr int

[GitHub] [arrow] jp0317 commented on pull request #36510: PARQUET-2321: [C++] allow customized buffer size when creating ArrowInputStream for a column PageReader

2023-08-16 Thread via GitHub
jp0317 commented on PR #36510: URL: https://github.com/apache/arrow/pull/36510#issuecomment-1681338986 > Let's step back a bit here. You are talking about limited memory use cases. The default buffer size is `kDefaultBufferSize = 4096 * 4`. Is there a situation where this is too large?

[GitHub] [arrow-datafusion] alamb commented on issue #6937: Improve Memory usage with large numbers of groups

2023-08-16 Thread via GitHub
alamb commented on issue #6937: URL: https://github.com/apache/arrow-datafusion/issues/6937#issuecomment-1681317933 > Doing that reduces the memory usage, but often with higher cost, which can be seen in the benchmark: Maybe we can get the performance back somehow (like make the outp

[GitHub] [arrow] lidavidm commented on pull request #36846: GH-36845: [C++][Python] Allow type promotion on `pa.concat_tables`

2023-08-16 Thread via GitHub
lidavidm commented on PR #36846: URL: https://github.com/apache/arrow/pull/36846#issuecomment-1681311609 The underlying function in C++ is used in both contexts, and was my original motivation. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [arrow] danepitkin commented on pull request #37097: GH-36730: [Python] Add support for Cython 3.0.0

2023-08-16 Thread via GitHub
danepitkin commented on PR #37097: URL: https://github.com/apache/arrow/pull/37097#issuecomment-1681304913 > > Numpydocs now correctly reports 72 cython functions are missing documentation for their parameters (Should be fixed in a separate PR IMO) > > Could you work on this before we

[GitHub] [arrow-nanoarrow] paleolimbot merged pull request #277: chore: Publish dev docs to asf-site branch instead of gh-pages

2023-08-16 Thread via GitHub
paleolimbot merged PR #277: URL: https://github.com/apache/arrow-nanoarrow/pull/277 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@a

[GitHub] [arrow] danepitkin commented on pull request #37218: GH-37217: [Python] Add missing docstrings to Cython

2023-08-16 Thread via GitHub
danepitkin commented on PR #37218: URL: https://github.com/apache/arrow/pull/37218#issuecomment-1681302600 I took a quick pass at adding docstrings. There are a lot so I didn't make them all perfect, e.g. some descriptions weren't added for functions that are self-explanatory. Let me know w

[GitHub] [arrow] github-actions[bot] commented on pull request #37218: GH-37217: [Python] Add missing docstrings to Cython

2023-08-16 Thread via GitHub
github-actions[bot] commented on PR #37218: URL: https://github.com/apache/arrow/pull/37218#issuecomment-1681300893 :warning: GitHub issue #37217 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [arrow] danepitkin opened a new pull request, #37218: GH-37217: [Python] Add missing docstrings to Cython

2023-08-16 Thread via GitHub
danepitkin opened a new pull request, #37218: URL: https://github.com/apache/arrow/pull/37218 ### Rationale for this change The Cython 3.0.0 upgrade https://github.com/apache/arrow/pull/37097 is triggering numpydoc errors for these missing docstrings. ### What changes are inclu

[GitHub] [arrow] davisusanibar closed pull request #37178: GH-37177: [Java][Docs] Enable singleton allocator

2023-08-16 Thread via GitHub
davisusanibar closed pull request #37178: GH-37177: [Java][Docs] Enable singleton allocator URL: https://github.com/apache/arrow/pull/37178 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [arrow] davisusanibar commented on pull request #37178: GH-37177: [Java][Docs] Enable singleton allocator

2023-08-16 Thread via GitHub
davisusanibar commented on PR #37178: URL: https://github.com/apache/arrow/pull/37178#issuecomment-1681283991 I am going to close this issue and incorporate your recommendation into https://github.com/apache/arrow/issues/37216 PRs -- This is an automated message from the Apache Git Servic

[GitHub] [arrow] davisusanibar commented on a diff in pull request #37178: GH-37177: [Java][Docs] Enable singleton allocator

2023-08-16 Thread via GitHub
davisusanibar commented on code in PR #37178: URL: https://github.com/apache/arrow/pull/37178#discussion_r1296438970 ## java/memory/memory-core/src/main/java/org/apache/arrow/memory/util/SingletonAllocator.java: ## @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [arrow] Fokko commented on pull request #36846: GH-36845: [C++][Python] Allow type promotion on `pa.concat_tables`

2023-08-16 Thread via GitHub
Fokko commented on PR #36846: URL: https://github.com/apache/arrow/pull/36846#issuecomment-1681277533 @lidavidm Thanks, in PyIceberg we do this by taking the field, casting them accordingly, and then turning it back into a table: https://github.com/apache/iceberg/blob/master/python/pyiceber

[GitHub] [arrow] kevingurney commented on a diff in pull request #37215: GH-37210: [Docs][MATLAB] Update MATLAB `README.md` to mention support for new MATLAB APIs (e.g. `RecordBatch`, `Field`, `Schema

2023-08-16 Thread via GitHub
kevingurney commented on code in PR #37215: URL: https://github.com/apache/arrow/pull/37215#discussion_r1296430605 ## matlab/README.md: ## @@ -27,17 +27,17 @@ This is a very early stage MATLAB interface to the Apache Arrow C++ libraries. Currently, the MATLAB interface suppo

[GitHub] [arrow] Tom-Newton commented on a diff in pull request #36835: GH-29847: [C++] Build with Azure SDK for C++

2023-08-16 Thread via GitHub
Tom-Newton commented on code in PR #36835: URL: https://github.com/apache/arrow/pull/36835#discussion_r1296429797 ## cpp/cmake_modules/FindAzure.cmake: ## @@ -0,0 +1,56 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. S

[GitHub] [arrow] Fokko commented on a diff in pull request #36846: GH-36845: [C++][Python] Allow type promotion on `pa.concat_tables`

2023-08-16 Thread via GitHub
Fokko commented on code in PR #36846: URL: https://github.com/apache/arrow/pull/36846#discussion_r1296429597 ## cpp/src/arrow/table.h: ## @@ -23,6 +23,7 @@ #include #include "arrow/chunked_array.h" // IWYU pragma: keep +#include "arrow/compute/cast.h" Review Comment: I

[GitHub] [arrow] Tom-Newton commented on pull request #36835: GH-29847: [C++] Build with Azure SDK for C++

2023-08-16 Thread via GitHub
Tom-Newton commented on PR #36835: URL: https://github.com/apache/arrow/pull/36835#issuecomment-1681264611 Looking at the current state of CI failures I think most are unrelated to my changes. Caused by this PR: C++ / ARM64 Ubuntu 20.04 C++ - definitely caused by this PR. Its failing i

[GitHub] [arrow] sgilmore10 commented on a diff in pull request #37215: GH-37210: [Docs][MATLAB] Update MATLAB `README.md` to mention support for new MATLAB APIs (e.g. `RecordBatch`, `Field`, `Schema`

2023-08-16 Thread via GitHub
sgilmore10 commented on code in PR #37215: URL: https://github.com/apache/arrow/pull/37215#discussion_r1296422147 ## matlab/README.md: ## @@ -27,17 +27,17 @@ This is a very early stage MATLAB interface to the Apache Arrow C++ libraries. Currently, the MATLAB interface suppor

[GitHub] [arrow] pitrou commented on a diff in pull request #37040: GH-36103: [C++] Initial device sync API

2023-08-16 Thread via GitHub
pitrou commented on code in PR #37040: URL: https://github.com/apache/arrow/pull/37040#discussion_r1296420386 ## cpp/src/arrow/device.h: ## @@ -98,6 +101,71 @@ class ARROW_EXPORT Device : public std::enable_shared_from_this, /// \brief Return the DeviceAllocationType of this

[GitHub] [arrow] kevingurney opened a new pull request, #37215: GH-37210: [Docs][MATLAB] Update MATLAB `README.md` to mention support for new MATLAB APIs (e.g. `RecordBatch`, `Field`, `Schema`, etc.)

2023-08-16 Thread via GitHub
kevingurney opened a new pull request, #37215: URL: https://github.com/apache/arrow/pull/37215 ### Rationale for this change Over the last few months, a number of new user-facing APIs have been added or changed in the MATLAB Interface. ## Examples: ### Construction Funct

[GitHub] [arrow] westonpace merged pull request #37214: GH-37213: [C#] Updating a reference to FlatBuffers missed due to rebase/merge conflict

2023-08-16 Thread via GitHub
westonpace merged PR #37214: URL: https://github.com/apache/arrow/pull/37214 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.ap

[GitHub] [arrow-rs] tustvold commented on pull request #4707: Add AnyDictionary Abstraction and Take ArrayRef in DictionaryArray::with_values

2023-08-16 Thread via GitHub
tustvold commented on PR #4707: URL: https://github.com/apache/arrow-rs/pull/4707#issuecomment-1681250365 Integration test failure seems unrelated, will see if sorts itself out tomorrow, if not I'll raise an upstream report -- This is an automated message from the Apache Git Service. To r

[GitHub] [arrow] westonpace commented on pull request #37214: GH-37213: [C#] Updating a reference to FlatBuffers missed due to rebase/merge conflict

2023-08-16 Thread via GitHub
westonpace commented on PR #37214: URL: https://github.com/apache/arrow/pull/37214#issuecomment-1681239764 There is still a timeout in one of the jobs but it looks like that timeout has been failing since before this bug so I'm pretty sure its unrelated. I'm going to go ahead and merge thi

[GitHub] [arrow-nanoarrow] WillAyd commented on a diff in pull request #276: feat: Implement ArrowBitsGet

2023-08-16 Thread via GitHub
WillAyd commented on code in PR #276: URL: https://github.com/apache/arrow-nanoarrow/pull/276#discussion_r1296384905 ## src/nanoarrow/buffer_test.cc: ## @@ -271,6 +271,65 @@ TEST(BitmapTest, BitmapTestElement) { EXPECT_EQ(ArrowBitGet(bitmap, 16 + 7), 0); } +TEST(BitmapTest

[GitHub] [arrow] westonpace commented on pull request #37211: GH-37209: [CI][Docs][MATLAB] Remove support for `MATLAB_ARROW_INTERFACE` flag from CMake build system and build new MATLAB Interface code

2023-08-16 Thread via GitHub
westonpace commented on PR #37211: URL: https://github.com/apache/arrow/pull/37211#issuecomment-1681202031 My mistake, I merged an old PR without thinking that I should probably rebase it first. Fix incoming in https://github.com/apache/arrow/pull/37214 -- This is an automated message fr

[GitHub] [arrow] github-actions[bot] commented on pull request #37214: GH-37213: [C#] Updating a reference to FlatBuffers missed due to rebase/merge conflict

2023-08-16 Thread via GitHub
github-actions[bot] commented on PR #37214: URL: https://github.com/apache/arrow/pull/37214#issuecomment-1681201617 :warning: GitHub issue #37213 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [arrow] westonpace opened a new pull request, #37214: GH-37213: [C#] Updating a reference to FlatBuffers missed due to rebase/merge conflict

2023-08-16 Thread via GitHub
westonpace opened a new pull request, #37214: URL: https://github.com/apache/arrow/pull/37214 ### Rationale for this change The build was broken ### What changes are included in this PR? Updated a newly introduced reference to `using FlatBuffers` to be `using Google.Flat

[GitHub] [arrow] zeroshade commented on a diff in pull request #37040: GH-36103: [C++] Initial device sync API

2023-08-16 Thread via GitHub
zeroshade commented on code in PR #37040: URL: https://github.com/apache/arrow/pull/37040#discussion_r1296369013 ## cpp/src/arrow/device.h: ## @@ -165,6 +198,17 @@ class ARROW_EXPORT MemoryManager : public std::enable_shared_from_this> ViewBuffer( const std::shared_ptr&

[GitHub] [arrow-datafusion] Dandandan commented on pull request #7192: Create a Priority Queue based Aggregation with `limit`

2023-08-16 Thread via GitHub
Dandandan commented on PR #7192: URL: https://github.com/apache/arrow-datafusion/pull/7192#issuecomment-1681193848 > SUM(x FILTER x > 100) I think this syntax is not correct. ``` SELECT SUM(x) FILTER (WHERE x >= 100) ``` should work -- This is an automated

[GitHub] [arrow-rs] tustvold commented on a diff in pull request #4707: Add AnyDictionary Abstraction and Take ArrayRef in DictionaryArray::with_values

2023-08-16 Thread via GitHub
tustvold commented on code in PR #4707: URL: https://github.com/apache/arrow-rs/pull/4707#discussion_r1296364918 ## arrow-array/src/array/dictionary_array.rs: ## @@ -930,6 +928,64 @@ where } } +/// Returns a [`AnyDictionaryArray`] if `array` is a dictionary +/// +/// Thi

[GitHub] [arrow] lidavidm commented on a diff in pull request #37178: GH-37177: [Java][Docs] Enable singleton allocator

2023-08-16 Thread via GitHub
lidavidm commented on code in PR #37178: URL: https://github.com/apache/arrow/pull/37178#discussion_r1296363779 ## java/memory/memory-core/src/main/java/org/apache/arrow/memory/util/SingletonAllocator.java: ## @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [arrow] zeroshade commented on a diff in pull request #37040: GH-36103: [C++] Initial device sync API

2023-08-16 Thread via GitHub
zeroshade commented on code in PR #37040: URL: https://github.com/apache/arrow/pull/37040#discussion_r1296362245 ## cpp/src/arrow/device.h: ## @@ -98,6 +101,71 @@ class ARROW_EXPORT Device : public std::enable_shared_from_this, /// \brief Return the DeviceAllocationType of t

[GitHub] [arrow] adamkennedy commented on a diff in pull request #37178: GH-37177: [Java][Docs] Enable singleton allocator

2023-08-16 Thread via GitHub
adamkennedy commented on code in PR #37178: URL: https://github.com/apache/arrow/pull/37178#discussion_r1296355737 ## java/memory/memory-core/src/main/java/org/apache/arrow/memory/util/GlobalAllocator.java: ## @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] [arrow-rs] tustvold commented on a diff in pull request #4707: Add AnyDictionary Abstraction and Take ArrayRef in DictionaryArray::with_values

2023-08-16 Thread via GitHub
tustvold commented on code in PR #4707: URL: https://github.com/apache/arrow-rs/pull/4707#discussion_r1296354255 ## arrow-arith/src/arity.rs: ## @@ -82,7 +82,7 @@ where { let dict_values = array.values().as_any().downcast_ref().unwrap(); let values = unary::(dict_valu

[GitHub] [arrow] adamkennedy commented on a diff in pull request #37178: GH-37177: [Java][Docs] Enable singleton allocator

2023-08-16 Thread via GitHub
adamkennedy commented on code in PR #37178: URL: https://github.com/apache/arrow/pull/37178#discussion_r1296354108 ## java/memory/memory-core/src/main/java/org/apache/arrow/memory/util/SingletonAllocator.java: ## @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [arrow] pitrou commented on a diff in pull request #37040: GH-36103: [C++] Initial device sync API

2023-08-16 Thread via GitHub
pitrou commented on code in PR #37040: URL: https://github.com/apache/arrow/pull/37040#discussion_r1296350867 ## cpp/src/arrow/device.h: ## @@ -98,6 +101,71 @@ class ARROW_EXPORT Device : public std::enable_shared_from_this, /// \brief Return the DeviceAllocationType of this

[GitHub] [arrow] pitrou commented on a diff in pull request #37040: GH-36103: [C++] Initial device sync API

2023-08-16 Thread via GitHub
pitrou commented on code in PR #37040: URL: https://github.com/apache/arrow/pull/37040#discussion_r1296349016 ## cpp/src/arrow/device.h: ## @@ -98,6 +101,71 @@ class ARROW_EXPORT Device : public std::enable_shared_from_this, /// \brief Return the DeviceAllocationType of this

[GitHub] [arrow] adamkennedy commented on a diff in pull request #37178: GH-37177: [Java][Docs] Enable singleton allocator

2023-08-16 Thread via GitHub
adamkennedy commented on code in PR #37178: URL: https://github.com/apache/arrow/pull/37178#discussion_r1296348047 ## java/memory/memory-core/src/test/java/org/apache/arrow/memory/util/GlobalAllocatorTest.java: ## @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [arrow] adamkennedy commented on a diff in pull request #37178: GH-37177: [Java][Docs] Enable singleton allocator

2023-08-16 Thread via GitHub
adamkennedy commented on code in PR #37178: URL: https://github.com/apache/arrow/pull/37178#discussion_r1296346656 ## java/memory/memory-core/src/main/java/org/apache/arrow/memory/util/GlobalAllocator.java: ## @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] [arrow] adamkennedy commented on a diff in pull request #37178: GH-37177: [Java][Docs] Enable singleton allocator

2023-08-16 Thread via GitHub
adamkennedy commented on code in PR #37178: URL: https://github.com/apache/arrow/pull/37178#discussion_r1296342646 ## docs/source/java/memory.rst: ## @@ -452,3 +452,5 @@ subset of that memory (e.g. through slicing) might be different. Also note that ArrowBuf 2 and ArrowBuf 4,

[GitHub] [arrow-rs] tustvold commented on a diff in pull request #4707: Add AnyDictionary Abstraction and Take ArrayRef in DictionaryArray::with_values

2023-08-16 Thread via GitHub
tustvold commented on code in PR #4707: URL: https://github.com/apache/arrow-rs/pull/4707#discussion_r1296339055 ## arrow-arith/src/arity.rs: ## @@ -82,7 +82,7 @@ where { let dict_values = array.values().as_any().downcast_ref().unwrap(); let values = unary::(dict_valu

[GitHub] [arrow] adamkennedy commented on a diff in pull request #37178: GH-37177: [Java][Docs] Enable singleton allocator

2023-08-16 Thread via GitHub
adamkennedy commented on code in PR #37178: URL: https://github.com/apache/arrow/pull/37178#discussion_r1296339566 ## java/memory/memory-core/src/main/java/org/apache/arrow/memory/util/GlobalAllocator.java: ## @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] [arrow-datafusion] avantgardnerio commented on pull request #7192: Create a Priority Queue based Aggregation with `limit`

2023-08-16 Thread via GitHub
avantgardnerio commented on PR #7192: URL: https://github.com/apache/arrow-datafusion/pull/7192#issuecomment-1681158630 Does anyone know how I test filters? I see in the codebase: ``` /// Optional filter expression to evaluate, one for each for /// accumulator. If presen

[GitHub] [arrow-rs] tustvold commented on a diff in pull request #4707: Add AnyDictionary Abstraction and Take ArrayRef in DictionaryArray::with_values

2023-08-16 Thread via GitHub
tustvold commented on code in PR #4707: URL: https://github.com/apache/arrow-rs/pull/4707#discussion_r1296339055 ## arrow-arith/src/arity.rs: ## @@ -82,7 +82,7 @@ where { let dict_values = array.values().as_any().downcast_ref().unwrap(); let values = unary::(dict_valu

[GitHub] [arrow] adamkennedy commented on a diff in pull request #37178: GH-37177: [Java][Docs] Enable singleton allocator

2023-08-16 Thread via GitHub
adamkennedy commented on code in PR #37178: URL: https://github.com/apache/arrow/pull/37178#discussion_r1296337266 ## docs/source/java/memory.rst: ## @@ -133,7 +133,7 @@ Development Guidelines Applications should generally: * Use the BufferAllocator interface in APIs instead

[GitHub] [arrow] zeroshade commented on a diff in pull request #37040: GH-36103: [C++] Initial device sync API

2023-08-16 Thread via GitHub
zeroshade commented on code in PR #37040: URL: https://github.com/apache/arrow/pull/37040#discussion_r1296335529 ## cpp/src/arrow/device.h: ## @@ -98,6 +101,71 @@ class ARROW_EXPORT Device : public std::enable_shared_from_this, /// \brief Return the DeviceAllocationType of t

[GitHub] [arrow] davisusanibar commented on a diff in pull request #37178: GH-37177: [Java][Docs] Enable singleton allocator

2023-08-16 Thread via GitHub
davisusanibar commented on code in PR #37178: URL: https://github.com/apache/arrow/pull/37178#discussion_r1296327949 ## java/memory/memory-core/src/main/java/org/apache/arrow/memory/util/SingletonAllocator.java: ## @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [arrow] davisusanibar commented on a diff in pull request #37178: GH-37177: [Java][Docs] Enable singleton allocator

2023-08-16 Thread via GitHub
davisusanibar commented on code in PR #37178: URL: https://github.com/apache/arrow/pull/37178#discussion_r1296327761 ## docs/source/java/memory.rst: ## @@ -133,7 +133,7 @@ Development Guidelines Applications should generally: * Use the BufferAllocator interface in APIs inste

[GitHub] [arrow] westonpace commented on pull request #37211: GH-37209: [CI][Docs][MATLAB] Remove support for `MATLAB_ARROW_INTERFACE` flag from CMake build system and build new MATLAB Interface code

2023-08-16 Thread via GitHub
westonpace commented on PR #37211: URL: https://github.com/apache/arrow/pull/37211#issuecomment-1681138733 > FYI - @CurtHagenlocher / @westonpace - it looks like the "Dev / Source Release and Merge Script" CI workflow is failing. The [logs mention C# and FlatBuffers](https://github.com/apac

[GitHub] [arrow-nanoarrow] codecov-commenter commented on pull request #277: chore: Publish dev docs to asf-site branch instead of gh-pages

2023-08-16 Thread via GitHub
codecov-commenter commented on PR #277: URL: https://github.com/apache/arrow-nanoarrow/pull/277#issuecomment-1681135867 ## [Codecov](https://app.codecov.io/gh/apache/arrow-nanoarrow/pull/277?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_t

[GitHub] [arrow-datafusion] alamb commented on pull request #7242: Support array aggregate function

2023-08-16 Thread via GitHub
alamb commented on PR #7242: URL: https://github.com/apache/arrow-datafusion/pull/7242#issuecomment-1681133109 I am sorry @jayzhan211 -- I don't think I will have time to help work on this PR for the next few days. I will try to find time this wekeend or next week but I am too busy now wi

[GitHub] [arrow] kevingurney commented on pull request #37211: GH-37209: [CI][Docs][MATLAB] Remove support for `MATLAB_ARROW_INTERFACE` flag from CMake build system and build new MATLAB Interface code

2023-08-16 Thread via GitHub
kevingurney commented on PR #37211: URL: https://github.com/apache/arrow/pull/37211#issuecomment-1681129697 FYI - @CurtHagenlocher / @westonpace - it looks like the "Dev / Source Release and Merge Script" CI workflow is failing. The [logs mention C# and FlatBuffers](https://github.com/apach

[GitHub] [arrow] kevingurney commented on issue #37210: [Docs][MATLAB] Update MATLAB `README.md` to mention support for new MATLAB APIs (e.g. `RecordBatch`, `Field`, `Schema`, etc.)

2023-08-16 Thread via GitHub
kevingurney commented on issue #37210: URL: https://github.com/apache/arrow/issues/37210#issuecomment-1681125051 Added label `Component: Documentation` since this is related to the MATLAB `README.md`, which is currently the main source of documentation for the MATLAB Interface. -- This i

[GitHub] [arrow-nanoarrow] paleolimbot opened a new pull request, #277: chore: Publish dev docs to asf-site branch instead of gh-pages

2023-08-16 Thread via GitHub
paleolimbot opened a new pull request, #277: URL: https://github.com/apache/arrow-nanoarrow/pull/277 Now that the documentation is hosted at https://arrow.apache.org/nanoarrow via the asf-site branch, the dev documentation should be there, too. -- This is an automated message from the Apa

[GitHub] [arrow] kevingurney commented on issue #37209: [CI][Docs][MATLAB] Remove support for `MATLAB_ARROW_INTERFACE` flag from CMake build system and build new MATLAB Interface code by default

2023-08-16 Thread via GitHub
kevingurney commented on issue #37209: URL: https://github.com/apache/arrow/issues/37209#issuecomment-1681120951 Added label `Component: Documentation` since resolving this involves updating the MATLAB `README.md` to no longer mention the `-D MATLAB_ARROW_INTERFACE=ON` flag. -- This is a

[GitHub] [arrow] kevingurney commented on issue #37209: [MATLAB] Remove support for `MATLAB_ARROW_INTERFACE` flag from CMake build system and build new MATLAB Interface code by default

2023-08-16 Thread via GitHub
kevingurney commented on issue #37209: URL: https://github.com/apache/arrow/issues/37209#issuecomment-1681117998 Added label `Component: Continuous Integration` since resolving this involves updating the `matlab_build.sh` CI script to no longer specify `-D MATLAB_ARROW_INTERFACE=ON`. --

[GitHub] [arrow-datafusion] alamb commented on pull request #7261: wip: Fix SortPreservingMerge OOM kill issue

2023-08-16 Thread via GitHub
alamb commented on PR #7261: URL: https://github.com/apache/arrow-datafusion/pull/7261#issuecomment-1681117900 Marking as draft to signify it is not waiting on review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[GitHub] [arrow-datafusion] alamb commented on pull request #6796: Implement basic unnest function

2023-08-16 Thread via GitHub
alamb commented on PR #6796: URL: https://github.com/apache/arrow-datafusion/pull/6796#issuecomment-1681117382 Marking as draft to signify it is not waiting on review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[GitHub] [arrow-datafusion] alamb commented on pull request #6713: Add `async` UDF example

2023-08-16 Thread via GitHub
alamb commented on PR #6713: URL: https://github.com/apache/arrow-datafusion/pull/6713#issuecomment-1681117530 Marking as draft to signify it is not waiting on review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[GitHub] [arrow-datafusion] alamb merged pull request #7311: Minor: Followup tasks for `nanvl`

2023-08-16 Thread via GitHub
alamb merged PR #7311: URL: https://github.com/apache/arrow-datafusion/pull/7311 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

[GitHub] [arrow] github-actions[bot] commented on pull request #37211: GH-37209: [MATLAB] Remove support for `MATLAB_ARROW_INTERFACE` flag from CMake build system and build new MATLAB Interface code b

2023-08-16 Thread via GitHub
github-actions[bot] commented on PR #37211: URL: https://github.com/apache/arrow/pull/37211#issuecomment-1681115317 :warning: GitHub issue #37209 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [arrow-rs] alamb commented on a diff in pull request #4707: Add AnyDictionary Abstraction and Take ArrayRef in DictionaryArray::with_values

2023-08-16 Thread via GitHub
alamb commented on code in PR #4707: URL: https://github.com/apache/arrow-rs/pull/4707#discussion_r1296301071 ## arrow-array/src/array/dictionary_array.rs: ## @@ -930,6 +928,64 @@ where } } +/// Returns a [`AnyDictionaryArray`] if `array` is a dictionary +/// +/// This c

[GitHub] [arrow] kevingurney opened a new pull request, #37211: GH-37209: [MATLAB] Remove support for `MATLAB_ARROW_INTERFACE` flag from CMake build system and build new MATLAB Interface code by defau

2023-08-16 Thread via GitHub
kevingurney opened a new pull request, #37211: URL: https://github.com/apache/arrow/pull/37211 ### Rationale for this change Now that the old Feather V1 code and associated build infrastructure has been removed (#37204), it makes sense to start building the new, experimental MATLAB I

[GitHub] [arrow-nanoarrow] paleolimbot merged pull request #272: docs: Add Roadmap to documentation

2023-08-16 Thread via GitHub
paleolimbot merged PR #272: URL: https://github.com/apache/arrow-nanoarrow/pull/272 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@a

[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #7291: Support `EXPLAIN COPY`

2023-08-16 Thread via GitHub
alamb commented on code in PR #7291: URL: https://github.com/apache/arrow-datafusion/pull/7291#discussion_r1296250896 ## datafusion/sql/src/parser.rs: ## @@ -74,7 +103,7 @@ pub struct CopyToStatement { /// The URL to where the data is heading pub target: String, /

[GitHub] [arrow-datafusion] viirya commented on pull request #6832: Update Arrow 45.0.0 And Datum Arithmetic, change Decimal Division semantics

2023-08-16 Thread via GitHub
viirya commented on PR #6832: URL: https://github.com/apache/arrow-datafusion/pull/6832#issuecomment-1681100995 > @tustvold @alamb @viirya spark has the config to control the `precision loss` : https://github.com/apache/spark/blob/2be20e54af6cdf64e8486d1910133b43665f/sql/catalyst/src/m

[GitHub] [arrow] kkraus14 commented on a diff in pull request #37040: GH-36103: [C++] Initial device sync API

2023-08-16 Thread via GitHub
kkraus14 commented on code in PR #37040: URL: https://github.com/apache/arrow/pull/37040#discussion_r1296290544 ## cpp/src/arrow/device.h: ## @@ -98,6 +101,71 @@ class ARROW_EXPORT Device : public std::enable_shared_from_this, /// \brief Return the DeviceAllocationType of th

[GitHub] [arrow] westonpace merged pull request #35699: GH-35698: [C#] Update FlatBuffers

2023-08-16 Thread via GitHub
westonpace merged PR #35699: URL: https://github.com/apache/arrow/pull/35699 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.ap

[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #7283: Implement Support for Copy To Logical and Physical plans

2023-08-16 Thread via GitHub
alamb commented on code in PR #7283: URL: https://github.com/apache/arrow-datafusion/pull/7283#discussion_r1296276120 ## datafusion/core/tests/sqllogictests/src/main.rs: ## @@ -58,10 +59,26 @@ pub async fn main() -> Result<()> { run_tests().await } +/// Sets up an empty

[GitHub] [arrow] davisusanibar commented on a diff in pull request #37178: GH-37177: [Java][Docs] Enable singleton allocator

2023-08-16 Thread via GitHub
davisusanibar commented on code in PR #37178: URL: https://github.com/apache/arrow/pull/37178#discussion_r1296273497 ## java/memory/memory-core/src/main/java/org/apache/arrow/memory/util/SingletonAllocator.java: ## @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Founda

  1   2   3   4   5   >