[GitHub] [arrow-rs] tatsuya6502 commented on issue #1829: AVX512 + simd binary and/or kernels slower than autovectorized version

2022-06-09 Thread GitBox
tatsuya6502 commented on issue #1829: URL: https://github.com/apache/arrow-rs/issues/1829#issuecomment-1152036764 > For some reason the second benchmark is always significantly slower when run together, running them separately gives the same (higher) performance and the assembly looks ident

[GitHub] [arrow-rs] tustvold commented on pull request #1827: Refactor parquet::arrow module

2022-06-09 Thread GitBox
tustvold commented on PR #1827: URL: https://github.com/apache/arrow-rs/pull/1827#issuecomment-1152033356 > Is there any plan for simplify the reader API Yes, the high level ticket is #1163. I'm hoping to pick it up soon, but large amounts of this API are public so needs some careful

[GitHub] [arrow] REASY commented on pull request #13157: ARROW-16584: [Java] Java JNI with S3 support

2022-06-09 Thread GitBox
REASY commented on PR #13157: URL: https://github.com/apache/arrow/pull/13157#issuecomment-1152033185 [AMD64 MacOS 10.15 C++](https://github.com/REASY/arrow/runs/6825355000?check_suite_focus=true#logs) has the following [tests failed:]( https://github.com/REASY/arrow/runs/6825355000?che

[GitHub] [arrow-rs] tustvold merged pull request #1831: [Minor] Clarify PageIterator Documentation

2022-06-09 Thread GitBox
tustvold merged PR #1831: URL: https://github.com/apache/arrow-rs/pull/1831 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apa

[GitHub] [arrow-rs] tustvold commented on pull request #1831: [Minor] Fix annotation

2022-06-09 Thread GitBox
tustvold commented on PR #1831: URL: https://github.com/apache/arrow-rs/pull/1831#issuecomment-1152028000 I agree the many levels of readers and iterators are deeply confusing 😅 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [arrow-rs] codecov-commenter commented on pull request #1832: speed up `substring_by_char` by about 2.5x

2022-06-09 Thread GitBox
codecov-commenter commented on PR #1832: URL: https://github.com/apache/arrow-rs/pull/1832#issuecomment-1152024365 # [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1832?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+S

[GitHub] [arrow-rs] HaoYang670 opened a new pull request, #1832: speed up `substring_by_char` by about 2.5x

2022-06-09 Thread GitBox
HaoYang670 opened a new pull request, #1832: URL: https://github.com/apache/arrow-rs/pull/1832 Signed-off-by: remzi <1371656737...@gmail.com> # Which issue does this PR close? Closes #1800. # What changes are included in this PR? Directly copy the string slice to `Buff

[GitHub] [arrow] kou commented on pull request #13359: ARROW-16794: [CI][C++][MinGW] Use debug build to reduce build time

2022-06-09 Thread GitBox
kou commented on PR #13359: URL: https://github.com/apache/arrow/pull/13359#issuecomment-1152000203 Debug build isn't fast... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [arrow] rtpsw commented on pull request #13355: ARROW-16796: [C++] Fix bad defaulting of ExecContext argument

2022-06-09 Thread GitBox
rtpsw commented on PR #13355: URL: https://github.com/apache/arrow/pull/13355#issuecomment-1151983476 > Good catch. I wonder if we should remove the default argument to bind entirely (it would look something like [westonpace@c9ae1dd](https://github.com/westonpace/arrow/commit/c9ae1dd6a0857a

[GitHub] [arrow-rs] codecov-commenter commented on pull request #1831: [Minor] Fix annotation

2022-06-09 Thread GitBox
codecov-commenter commented on PR #1831: URL: https://github.com/apache/arrow-rs/pull/1831#issuecomment-1151944150 # [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1831?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+S

[GitHub] [arrow-rs] Ted-Jiang opened a new pull request, #1831: [Minor] Fix annotation

2022-06-09 Thread GitBox
Ted-Jiang opened a new pull request, #1831: URL: https://github.com/apache/arrow-rs/pull/1831 # Which issue does this PR close? Closes #. # Rationale for this change If i'm right, `PageIterator ` should return one column in multi row group. I was mislead by th

[GitHub] [arrow-rs] kazuk commented on a diff in pull request #1798: add parquet-fromcsv (#1)

2022-06-09 Thread GitBox
kazuk commented on code in PR #1798: URL: https://github.com/apache/arrow-rs/pull/1798#discussion_r894144414 ## parquet/src/bin/parquet-fromcsv.rs: ## @@ -0,0 +1,517 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. Se

[GitHub] [arrow] kou commented on a diff in pull request #12763: ARROW-14892: [Python][C++] GCS Bindings

2022-06-09 Thread GitBox
kou commented on code in PR #12763: URL: https://github.com/apache/arrow/pull/12763#discussion_r894142479 ## dev/tasks/python-wheels/github.osx.arm64.yml: ## @@ -134,6 +134,8 @@ jobs: $PYTHON -m venv test-arm64-env source test-arm64-env/bin/activate

[GitHub] [arrow] kou commented on a diff in pull request #12763: ARROW-14892: [Python][C++] GCS Bindings

2022-06-09 Thread GitBox
kou commented on code in PR #12763: URL: https://github.com/apache/arrow/pull/12763#discussion_r894142479 ## dev/tasks/python-wheels/github.osx.arm64.yml: ## @@ -134,6 +134,8 @@ jobs: $PYTHON -m venv test-arm64-env source test-arm64-env/bin/activate

[GitHub] [arrow] kou commented on pull request #13244: ARROW-12626: [C++] Support toolchain xsimd, update toolchain version to version 8.1.0

2022-06-09 Thread GitBox
kou commented on PR #13244: URL: https://github.com/apache/arrow/pull/13244#issuecomment-1151914929 This is ready to merge. TODO after merge: * Set suitable GitHub personal access token to https://github.com/ursacomputing/crossbow/ to push built Docker images to ghcr.io/ursaco

[GitHub] [arrow] alexandreyc commented on a diff in pull request #13310: ARROW-16749: [Go] Fix pqarrow writer for null array

2022-06-09 Thread GitBox
alexandreyc commented on code in PR #13310: URL: https://github.com/apache/arrow/pull/13310#discussion_r894121447 ## go/parquet/file/column_writer_types.gen.go.tmpl: ## @@ -137,13 +137,13 @@ func (w *{{.Name}}ColumnChunkWriter) WriteBatchSpaced(values []{{.name}}, defLev

[GitHub] [arrow] alexandreyc commented on a diff in pull request #13310: ARROW-16749: [Go] Fix pqarrow writer for null array

2022-06-09 Thread GitBox
alexandreyc commented on code in PR #13310: URL: https://github.com/apache/arrow/pull/13310#discussion_r894120603 ## go/parquet/file/column_writer_types.gen.go.tmpl: ## @@ -137,13 +137,13 @@ func (w *{{.Name}}ColumnChunkWriter) WriteBatchSpaced(values []{{.name}}, defLev

[GitHub] [arrow] cyb70289 commented on a diff in pull request #13354: ARROW-16799: [C++] Create a self-pipe abstraction

2022-06-09 Thread GitBox
cyb70289 commented on code in PR #13354: URL: https://github.com/apache/arrow/pull/13354#discussion_r894088698 ## cpp/src/arrow/util/io_util.h: ## @@ -124,14 +125,46 @@ Result DeleteFile(const PlatformFilename& file_path, bool allow_not_found ARROW_EXPORT Result FileExists(co

[GitHub] [arrow-julia] codecov-commenter commented on pull request #325: allow ntasks to be 0 and determine whetheher to use threads based on nthreads rather than ntasks

2022-06-09 Thread GitBox
codecov-commenter commented on PR #325: URL: https://github.com/apache/arrow-julia/pull/325#issuecomment-1151854796 # [Codecov](https://codecov.io/gh/apache/arrow-julia/pull/325?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apach

[GitHub] [arrow-julia] baumgold opened a new pull request, #325: allow ntasks to be 0 and determine whetheher to use threads based on nthreads rather than ntasks

2022-06-09 Thread GitBox
baumgold opened a new pull request, #325: URL: https://github.com/apache/arrow-julia/pull/325 This allows configuring an unbuffered channel while still using multithreading -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [arrow] github-actions[bot] commented on pull request #13359: ARROW-16794: [CI][C++][MinGW] Use debug build to reduce build time

2022-06-09 Thread GitBox
github-actions[bot] commented on PR #13359: URL: https://github.com/apache/arrow/pull/13359#issuecomment-1151823251 :warning: Ticket **has not been started in JIRA**, please click 'Start Progress'. -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [arrow] github-actions[bot] commented on pull request #13359: ARROW-16794: [CI][C++][MinGW] Use debug build to reduce build time

2022-06-09 Thread GitBox
github-actions[bot] commented on PR #13359: URL: https://github.com/apache/arrow/pull/13359#issuecomment-1151823228 https://issues.apache.org/jira/browse/ARROW-16794 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [arrow] chenbaggio closed issue #13349: How to get c_glib debug compiled lib (can debug with src code)

2022-06-09 Thread GitBox
chenbaggio closed issue #13349: How to get c_glib debug compiled lib (can debug with src code) URL: https://github.com/apache/arrow/issues/13349 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [arrow] chenbaggio commented on issue #13349: How to get c_glib debug compiled lib (can debug with src code)

2022-06-09 Thread GitBox
chenbaggio commented on issue #13349: URL: https://github.com/apache/arrow/issues/13349#issuecomment-1151816262 thks, I have gotten the resolution ``` build_type='debug' is not work and report error should set build_type=None is ok ``` thks for your tips again!

[GitHub] [arrow] ursabot commented on pull request #13277: ARROW-16638: [Go][Parquet] Fix skipping large number of rows in boolean columns

2022-06-09 Thread GitBox
ursabot commented on PR #13277: URL: https://github.com/apache/arrow/pull/13277#issuecomment-1151792918 Benchmark runs are scheduled for baseline = b79eeb161e8a2137f6d0916965f1e1a3977012ad and contender = a4b14d3cd92f4b8e8a9cf81ff0a838356437d45a. a4b14d3cd92f4b8e8a9cf81ff0a838356437d45a is

[GitHub] [arrow] github-actions[bot] commented on pull request #13244: ARROW-12626: [C++] Support toolchain xsimd, update toolchain version to version 8.1.0

2022-06-09 Thread GitBox
github-actions[bot] commented on PR #13244: URL: https://github.com/apache/arrow/pull/13244#issuecomment-1151791200 Revision: eb97dbfd9f5620d756dc23a359973c35d4558e4b Submitted crossbow builds: [ursacomputing/crossbow @ actions-806517b300](https://github.com/ursacomputing/crossbow/bra

[GitHub] [arrow] kou commented on pull request #13244: ARROW-12626: [C++] Support toolchain xsimd, update toolchain version to version 8.1.0

2022-06-09 Thread GitBox
kou commented on PR #13244: URL: https://github.com/apache/arrow/pull/13244#issuecomment-1151789680 @github-actions crossbow submit conda-win-* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] kou merged pull request #13348: ARROW-16792: [C++][CMake] Add support for using Arrow options when Arrow is used as subproject

2022-06-09 Thread GitBox
kou merged PR #13348: URL: https://github.com/apache/arrow/pull/13348 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

[GitHub] [arrow] kou commented on pull request #13348: ARROW-16792: [C++][CMake] Add support for using Arrow options when Arrow is used as subproject

2022-06-09 Thread GitBox
kou commented on PR #13348: URL: https://github.com/apache/arrow/pull/13348#issuecomment-1151783600 > Hmm, why are there steps such as "Dump R install logs" in the conan builds? Because they reuse general Docker based test configuration: https://github.com/apache/arrow/blob/master/dev

[GitHub] [arrow] traversc opened a new issue, #13358: Bug: unable to install R package on CentOS 7

2022-06-09 Thread GitBox
traversc opened a new issue, #13358: URL: https://github.com/apache/arrow/issues/13358 Installing from CRAN fails with and without NOT_CRAN=true. One error message below. I'm unable to install on a fresh CentOS 7 installation, so very reproducible. Any way I can fix it? (I posted thi

[GitHub] [arrow-datafusion] andygrove opened a new pull request, #2714: Prepare 9.0.0 release

2022-06-09 Thread GitBox
andygrove opened a new pull request, #2714: URL: https://github.com/apache/arrow-datafusion/pull/2714 # Which issue does this PR close? Closes https://github.com/apache/arrow-datafusion/issues/2676 # Rationale for this change Monthly release # What cha

[GitHub] [arrow] save-buffer commented on pull request #13332: ARROW-16713: [C++] Pull join accumulation outside of HashJoinImpl

2022-06-09 Thread GitBox
save-buffer commented on PR #13332: URL: https://github.com/apache/arrow/pull/13332#issuecomment-1151739257 The only failures seem to be with S3FS (unrelated to hash join completely) and the tracing span thingy which is being addressed in https://github.com/apache/arrow/pull/13108 -- Thi

[GitHub] [arrow] save-buffer commented on pull request #13314: ARROW-16716: [C++] Add Benchmarks for ProjectNode

2022-06-09 Thread GitBox
save-buffer commented on PR #13314: URL: https://github.com/apache/arrow/pull/13314#issuecomment-1151731514 Interesting, yes I'll definitely take a look at this benchmark. I'd also be interested in comparing this to just using OpenMP - I may draw that up at some point as well. As of right n

[GitHub] [arrow] kou commented on issue #13349: How to get c_glib debug compiled lib (can debug with src code)

2022-06-09 Thread GitBox
kou commented on issue #13349: URL: https://github.com/apache/arrow/issues/13349#issuecomment-1151720017 How about specifying `build_type='debug'` explicitly? ```python meson = Meson(self, build_type='debug') ``` See also: https://docs.conan.io/en/latest/reference/build_h

[GitHub] [arrow] kou merged pull request #13351: ARROW-16694: [Packaging][Python] Use Mamba instead of conda to build conda environment for windows packaging jobs

2022-06-09 Thread GitBox
kou merged PR #13351: URL: https://github.com/apache/arrow/pull/13351 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

[GitHub] [arrow] mrkn commented on issue #12553: Support for Compute Functions on Nested Arrays

2022-06-09 Thread GitBox
mrkn commented on issue #12553: URL: https://github.com/apache/arrow/issues/12553#issuecomment-1151702963 > Going from pa.Tensor to np.array and back should be zero copy AFAIK. It's true. Only metadata, that is the shape and the stride arrays are copied from Arrow's Tensor to NumPy's

[GitHub] [arrow-rs] codecov-commenter commented on pull request #1830: Remove simd and avx512 bitwise kernels in favor of autovectorization

2022-06-09 Thread GitBox
codecov-commenter commented on PR #1830: URL: https://github.com/apache/arrow-rs/pull/1830#issuecomment-1151694569 # [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1830?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+S

[GitHub] [arrow] westonpace commented on issue #13353: Casting a decimal column of a table into a float type when converting a table into a pandas dataframe

2022-06-09 Thread GitBox
westonpace commented on issue #13353: URL: https://github.com/apache/arrow/issues/13353#issuecomment-1151694049 We have a cast function which can convert: ``` import pyarrow.compute as pc pc.cast(table.column('decimal_column'), pa.float64()) ``` However, I don't kno

[GitHub] [arrow-rs] jhorstmann opened a new pull request, #1830: Remove simd and avx512 bitwise kernels in favor of autovectorization

2022-06-09 Thread GitBox
jhorstmann opened a new pull request, #1830: URL: https://github.com/apache/arrow-rs/pull/1830 # Which issue does this PR close? Closes #1829. # Rationale for this change The autovectorized implementation is actually faster, allowing us to simplify the buffer cod

[GitHub] [arrow] westonpace commented on pull request #13355: ARROW-16796: [C++] Fix bad defaulting of ExecContext argument

2022-06-09 Thread GitBox
westonpace commented on PR #13355: URL: https://github.com/apache/arrow/pull/13355#issuecomment-1151679039 Good catch. I wonder if we should remove the default argument to bind entirely (it would look something like https://github.com/westonpace/arrow/commit/c9ae1dd6a0857af69e48a95ec76480f

[GitHub] [arrow-rs] jhorstmann commented on pull request #1825: Replace RawPtrBox with ScalarBuffer, reduce `unsafe` usage (#1811)

2022-06-09 Thread GitBox
jhorstmann commented on PR #1825: URL: https://github.com/apache/arrow-rs/pull/1825#issuecomment-1151670264 Looks good, nice improvement. Regarding the benchmarks, maybe the `Deref for ScalarBuffer` needs an explicit `#[inline]` annotation. Everything else in the hot methods, including the

[GitHub] [arrow] mdepero commented on pull request #13357: MINOR: [Go][Parquet] fix go parquet dictionary encoding writer property

2022-06-09 Thread GitBox
mdepero commented on PR #13357: URL: https://github.com/apache/arrow/pull/13357#issuecomment-1151657836 @zeroshade advise if this is worth creating a jira issue for, for now labeled as "minor" -- This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [arrow] lidavidm merged pull request #13352: ARROW-16554: [Java] Download Java nightlies artifacts from https://nightlies.apache.org/arrow/java/org/apache/arrow/

2022-06-09 Thread GitBox
lidavidm merged PR #13352: URL: https://github.com/apache/arrow/pull/13352 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apac

[GitHub] [arrow] ateucher commented on pull request #13336: ARROW-16783: [R] Explicit check for supported classes in arrow_dplyr_query

2022-06-09 Thread GitBox
ateucher commented on PR #13336: URL: https://github.com/apache/arrow/pull/13336#issuecomment-1151620343 Thanks @nealrichardson and @thisisnic - I think I've addressed your comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[GitHub] [arrow-rs] tustvold merged pull request #1822: Change to use `resolver v2`, test more feature flag combinations in CI, fix errors (#1630)

2022-06-09 Thread GitBox
tustvold merged PR #1822: URL: https://github.com/apache/arrow-rs/pull/1822 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apa

[GitHub] [arrow-rs] tustvold closed issue #1630: parquet does not compile with `features=["zstd"]`

2022-06-09 Thread GitBox
tustvold closed issue #1630: parquet does not compile with `features=["zstd"]` URL: https://github.com/apache/arrow-rs/issues/1630 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [arrow] chenbaggio commented on issue #13349: How to get c_glib debug compiled lib (can debug with src code)

2022-06-09 Thread GitBox
chenbaggio commented on issue #13349: URL: https://github.com/apache/arrow/issues/13349#issuecomment-1151603553 I use canon python script to build ``` import os from conans import ConanFile, CMake, tools, AutoToolsBuildEnvironment, Meson dependencies = [ "orc/1.

[GitHub] [arrow] rtpsw commented on pull request #13355: ARROW-16796: [C++] Fix bad defaulting of ExecContext argument

2022-06-09 Thread GitBox
rtpsw commented on PR #13355: URL: https://github.com/apache/arrow/pull/13355#issuecomment-1151600723 @westonpace, could you review or refer to someone? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] viirya commented on pull request #13219: ARROW-16636: [Rust] Activate several IPC integration tests for rust

2022-06-09 Thread GitBox
viirya commented on PR #13219: URL: https://github.com/apache/arrow/pull/13219#issuecomment-1151593037 @alamb Thank you. I think you can just merge this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] dominicbarnes commented on pull request #13322: ARROW-16759: [Go] update testify to get security patch for gopkg.in/yaml.v3 (v7)

2022-06-09 Thread GitBox
dominicbarnes commented on PR #13322: URL: https://github.com/apache/arrow/pull/13322#issuecomment-1151593191 @zeroshade I've rebased the PR, which seems to have included some JS changes during the process, though once I've completed there are no longer changes to JS. Once merged, I'

[GitHub] [arrow] alamb commented on pull request #13219: ARROW-16636: [Rust] Activate several IPC integration tests for rust

2022-06-09 Thread GitBox
alamb commented on PR #13219: URL: https://github.com/apache/arrow/pull/13219#issuecomment-1151583578 The arrow repo used to have a different way of merging PRs (using a script) -- let me know if you would like to give it a try or if you would like me to. 🎖️ -- This is an automated mess

[GitHub] [arrow-rs] jhorstmann commented on issue #1829: AVX512 + simd binary and/or kernels slower than autovectorized version

2022-06-09 Thread GitBox
jhorstmann commented on issue #1829: URL: https://github.com/apache/arrow-rs/issues/1829#issuecomment-1151583367 The auto-vectorized version of and/or with non-zero offsets amazingly also runs at about 50 GiB/s -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] alamb commented on pull request #13219: ARROW-16636: [Rust] Activate several IPC integration tests for rust

2022-06-09 Thread GitBox
alamb commented on PR #13219: URL: https://github.com/apache/arrow/pull/13219#issuecomment-1151582611 Amazing work @viirya ❤️ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[GitHub] [arrow-datafusion] nl5887 commented on issue #2326: Properly support arrow `Struct` types / Composite type in DataFusion

2022-06-09 Thread GitBox
nl5887 commented on issue #2326: URL: https://github.com/apache/arrow-datafusion/issues/2326#issuecomment-1151582810 This https://github.com/apache/arrow-datafusion/blob/master/datafusion/core/src/physical_plan/file_format/mod.rs#L238 is one reason of errors related to column projection. I

[GitHub] [arrow] viirya commented on pull request #13219: ARROW-16636: [Rust] Activate several IPC integration tests for rust

2022-06-09 Thread GitBox
viirya commented on PR #13219: URL: https://github.com/apache/arrow/pull/13219#issuecomment-1151578098 @alamb All tests are passed now. Please take a look. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [arrow-rs] jhorstmann opened a new issue, #1829: AVX512 + simd binary and/or kernels slower than autovectorized version

2022-06-09 Thread GitBox
jhorstmann opened a new issue, #1829: URL: https://github.com/apache/arrow-rs/issues/1829 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** Related to extending the tests for different features flags #1822, I wanted to take anot

[GitHub] [arrow] kou commented on issue #13349: How to get c_glib debug compiled lib (can debug with src code)

2022-06-09 Thread GitBox
kou commented on issue #13349: URL: https://github.com/apache/arrow/issues/13349#issuecomment-1151573695 `meson setup` uses debug build by default. Could you show your command lines to build Apache Arrow GLib? -- This is an automated message from the Apache Git Service. To respond to th

[GitHub] [arrow] ateucher commented on a diff in pull request #13336: ARROW-16783: [R] Explicit check for supported classes in arrow_dplyr_query

2022-06-09 Thread GitBox
ateucher commented on code in PR #13336: URL: https://github.com/apache/arrow/pull/13336#discussion_r893908380 ## r/R/dplyr.R: ## @@ -24,6 +24,21 @@ arrow_dplyr_query <- function(.data) { # RecordBatch, or Dataset) and the state of the user's dplyr query--things # like sel

[GitHub] [arrow] github-actions[bot] commented on pull request #13354: ARROW-16799: [C++] Create a self-pipe abstraction

2022-06-09 Thread GitBox
github-actions[bot] commented on PR #13354: URL: https://github.com/apache/arrow/pull/13354#issuecomment-1151563130 Revision: dcf11356c22772b791d843453eb9ee59d0e6bb5c Submitted crossbow builds: [ursacomputing/crossbow @ actions-9ec240e09d](https://github.com/ursacomputing/crossbow/bra

[GitHub] [arrow] pitrou commented on pull request #13354: ARROW-16799: [C++] Create a self-pipe abstraction

2022-06-09 Thread GitBox
pitrou commented on PR #13354: URL: https://github.com/apache/arrow/pull/13354#issuecomment-1151562338 @github-actions crossbow submit -g cpp -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [arrow-rs] tustvold merged pull request #1828: Update MIRI pin

2022-06-09 Thread GitBox
tustvold merged PR #1828: URL: https://github.com/apache/arrow-rs/pull/1828 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apa

[GitHub] [arrow] pitrou commented on pull request #13354: ARROW-16799: [C++] Create a self-pipe abstraction

2022-06-09 Thread GitBox
pitrou commented on PR #13354: URL: https://github.com/apache/arrow/pull/13354#issuecomment-1151551466 While most CI failures are unrelated, the test-debian-10-cpp-i386 failure is directly caused by this PR. -- This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [arrow-rs] tustvold commented on a diff in pull request #1828: Update MIRI pin

2022-06-09 Thread GitBox
tustvold commented on code in PR #1828: URL: https://github.com/apache/arrow-rs/pull/1828#discussion_r893888119 ## .github/workflows/miri.yaml: ## @@ -26,19 +26,15 @@ jobs: miri-checks: name: MIRI runs-on: ubuntu-latest -strategy: - matrix: -arch: [

[GitHub] [arrow-rs] codecov-commenter commented on pull request #1828: Update MIRI pin

2022-06-09 Thread GitBox
codecov-commenter commented on PR #1828: URL: https://github.com/apache/arrow-rs/pull/1828#issuecomment-1151541660 # [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1828?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+S

[GitHub] [arrow-rs] alamb commented on a diff in pull request #1828: Update MIRI pin

2022-06-09 Thread GitBox
alamb commented on code in PR #1828: URL: https://github.com/apache/arrow-rs/pull/1828#discussion_r893885924 ## .github/workflows/miri.yaml: ## @@ -26,19 +26,15 @@ jobs: miri-checks: name: MIRI runs-on: ubuntu-latest -strategy: - matrix: -arch: [amd

[GitHub] [arrow-rs] alamb commented on pull request #1828: Update MIRI pin

2022-06-09 Thread GitBox
alamb commented on PR #1828: URL: https://github.com/apache/arrow-rs/pull/1828#issuecomment-1151540135 If MIRI passes this one looks good to me -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] lidavidm commented on a diff in pull request #12775: ARROW-16006: [C++] Row conversion helpers and example

2022-06-09 Thread GitBox
lidavidm commented on code in PR #12775: URL: https://github.com/apache/arrow/pull/12775#discussion_r893820377 ## docs/source/cpp/examples/row_columnar_conversion.rst: ## @@ -21,7 +21,98 @@ Row to columnar conversion == +Fixed Schemas +- +

[GitHub] [arrow] github-actions[bot] commented on pull request #13354: ARROW-16799: [C++] Create a self-pipe abstraction

2022-06-09 Thread GitBox
github-actions[bot] commented on PR #13354: URL: https://github.com/apache/arrow/pull/13354#issuecomment-1151460606 Revision: ddd73b3b3849751225cda78bf051419299f71fc4 Submitted crossbow builds: [ursacomputing/crossbow @ actions-177418cea1](https://github.com/ursacomputing/crossbow/bra

[GitHub] [arrow] pitrou commented on pull request #13354: ARROW-16799: [C++] Create a self-pipe abstraction

2022-06-09 Thread GitBox
pitrou commented on PR #13354: URL: https://github.com/apache/arrow/pull/13354#issuecomment-1151455347 @github-actions crossbow submit -g cpp -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [arrow] save-buffer commented on a diff in pull request #13332: ARROW-16713: [C++] Pull join accumulation outside of HashJoinImpl

2022-06-09 Thread GitBox
save-buffer commented on code in PR #13332: URL: https://github.com/apache/arrow/pull/13332#discussion_r893823596 ## cpp/src/arrow/compute/exec/accumulation_queue.h: ## @@ -0,0 +1,54 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor licens

[GitHub] [arrow] save-buffer commented on a diff in pull request #13332: ARROW-16713: [C++] Pull join accumulation outside of HashJoinImpl

2022-06-09 Thread GitBox
save-buffer commented on code in PR #13332: URL: https://github.com/apache/arrow/pull/13332#discussion_r893823387 ## cpp/src/arrow/compute/exec/hash_join_benchmark.cc: ## @@ -124,67 +128,60 @@ class JoinBenchmark { schema_mgr_ = arrow::internal::make_unique(); Expres

[GitHub] [arrow] save-buffer commented on a diff in pull request #13332: ARROW-16713: [C++] Pull join accumulation outside of HashJoinImpl

2022-06-09 Thread GitBox
save-buffer commented on code in PR #13332: URL: https://github.com/apache/arrow/pull/13332#discussion_r893820918 ## cpp/src/arrow/compute/exec/accumulation_queue.h: ## @@ -0,0 +1,54 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor licens

[GitHub] [arrow] save-buffer commented on a diff in pull request #13332: ARROW-16713: [C++] Pull join accumulation outside of HashJoinImpl

2022-06-09 Thread GitBox
save-buffer commented on code in PR #13332: URL: https://github.com/apache/arrow/pull/13332#discussion_r893820476 ## cpp/src/arrow/compute/exec/accumulation_queue.h: ## @@ -0,0 +1,54 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor licens

[GitHub] [arrow] zeroshade commented on pull request #13322: ARROW-16759: [Go] update testify to get security patch for gopkg.in/yaml.v3 (v7)

2022-06-09 Thread GitBox
zeroshade commented on PR #13322: URL: https://github.com/apache/arrow/pull/13322#issuecomment-1151442679 @dominicbarnes Please run `go mod tidy` in the directory so that it properly updates the `go.sum` file also and add that to this PR. You can also change this to merge to master, and we

[GitHub] [arrow-rs] tustvold commented on pull request #1822: Change to use `resolver v2`, test more feature flag combinations in CI, fix errors (#1630)

2022-06-09 Thread GitBox
tustvold commented on PR #1822: URL: https://github.com/apache/arrow-rs/pull/1822#issuecomment-1151423914 Appears MIRI version is too old to supposed namespaced deps - https://github.com/rust-lang/cargo/issues/5565 Attempting to updated it in - https://github.com/apache/arrow-rs/pull/

[GitHub] [arrow-rs] tustvold opened a new pull request, #1828: Unpin MIRI nightly

2022-06-09 Thread GitBox
tustvold opened a new pull request, #1828: URL: https://github.com/apache/arrow-rs/pull/1828 # Which issue does this PR close? Closes #. # Rationale for this change The git history does not really indicate why this was pinned, so lets unpin and see what breaks. This is

[GitHub] [arrow] nealrichardson commented on a diff in pull request #13240: ARROW-16406: [Docs][R] Update documentation with new nightly location

2022-06-09 Thread GitBox
nealrichardson commented on code in PR #13240: URL: https://github.com/apache/arrow/pull/13240#discussion_r893792784 ## r/vignettes/developers/install_details.Rmd: ## @@ -100,7 +100,7 @@ The libarrow binaries will only be retrieved if you have set the environment var If foun

[GitHub] [arrow-rs] tustvold commented on pull request #1822: Change to use `resolver v2`, test more feature flag combinations in CI, fix errors (#1630)

2022-06-09 Thread GitBox
tustvold commented on PR #1822: URL: https://github.com/apache/arrow-rs/pull/1822#issuecomment-1151418759 MIRI appears to be unhappy... :eyes: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] github-actions[bot] commented on pull request #13356: ARROW-16800: [C++] RecordBatchBuilder deprecate Status APIs, add Result APIs

2022-06-09 Thread GitBox
github-actions[bot] commented on PR #13356: URL: https://github.com/apache/arrow/pull/13356#issuecomment-1151418928 https://issues.apache.org/jira/browse/ARROW-16800 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [arrow] wjones127 opened a new pull request, #13356: ARROW-16800: [C++] RecordBatchBuilder deprecate Status APIs, add Result APIs

2022-06-09 Thread GitBox
wjones127 opened a new pull request, #13356: URL: https://github.com/apache/arrow/pull/13356 Also added the class to the C++ API docs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [arrow-rs] alamb commented on pull request #1821: docs: remove experimental marker on C Stream Interface

2022-06-09 Thread GitBox
alamb commented on PR #1821: URL: https://github.com/apache/arrow-rs/pull/1821#issuecomment-1151417797 Relevant thread: https://lists.apache.org/thread/5bvk6m3y3wl0m4jdsnyhdylt1w5j288k -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [arrow-datafusion] alamb commented on issue #2712: Common Subexpression Eliminiation pass errors if run twice on some plans: Schema contains duplicate unqualified field name 'IsNull-Column-sy

2022-06-09 Thread GitBox
alamb commented on issue #2712: URL: https://github.com/apache/arrow-datafusion/issues/2712#issuecomment-1151400179 Thanks @waynexia -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [arrow] lidavidm commented on a diff in pull request #13354: ARROW-16799: [C++] Create a self-pipe abstraction

2022-06-09 Thread GitBox
lidavidm commented on code in PR #13354: URL: https://github.com/apache/arrow/pull/13354#discussion_r893771913 ## cpp/src/arrow/testing/gtest_util.cc: ## @@ -574,6 +578,40 @@ std::shared_ptr TweakValidityBit(const std::shared_ptr& array, return MakeArray(data); } +// XXX

[GitHub] [arrow-rs] tustvold commented on pull request #1822: Change to use `resolver v2`, test more feature flag combinations in CI, fix errors (#1630)

2022-06-09 Thread GitBox
tustvold commented on PR #1822: URL: https://github.com/apache/arrow-rs/pull/1822#issuecomment-1151392514 [Here](https://github.com/apache/arrow-rs/runs/6817433083?check_suite_focus=true) is an example of it restoring from a cache key (without lockfile suffix), and then publishing the lock

[GitHub] [arrow] pitrou commented on a diff in pull request #13354: ARROW-16799: [C++] Create a self-pipe abstraction

2022-06-09 Thread GitBox
pitrou commented on code in PR #13354: URL: https://github.com/apache/arrow/pull/13354#discussion_r893753515 ## cpp/src/arrow/testing/gtest_util.cc: ## @@ -574,6 +578,40 @@ std::shared_ptr TweakValidityBit(const std::shared_ptr& array, return MakeArray(data); } +// XXX cr

[GitHub] [arrow] rok commented on issue #12553: Support for Compute Functions on Nested Arrays

2022-06-09 Thread GitBox
rok commented on issue #12553: URL: https://github.com/apache/arrow/issues/12553#issuecomment-1151378310 > So now my only question is, while this seems like an optimal generalized solution for storage, how much computation is required to explode back out to the dense form in memory to do co

[GitHub] [arrow] github-actions[bot] commented on pull request #13355: ARROW-16796: [C++] Fix bad defaulting of ExecContext argument

2022-06-09 Thread GitBox
github-actions[bot] commented on PR #13355: URL: https://github.com/apache/arrow/pull/13355#issuecomment-1151379865 https://issues.apache.org/jira/browse/ARROW-16796 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [arrow] pitrou commented on a diff in pull request #13354: ARROW-16799: [C++] Create a self-pipe abstraction

2022-06-09 Thread GitBox
pitrou commented on code in PR #13354: URL: https://github.com/apache/arrow/pull/13354#discussion_r893753515 ## cpp/src/arrow/testing/gtest_util.cc: ## @@ -574,6 +578,40 @@ std::shared_ptr TweakValidityBit(const std::shared_ptr& array, return MakeArray(data); } +// XXX cr

[GitHub] [arrow] lidavidm commented on a diff in pull request #13354: ARROW-16799: [C++] Create a self-pipe abstraction

2022-06-09 Thread GitBox
lidavidm commented on code in PR #13354: URL: https://github.com/apache/arrow/pull/13354#discussion_r893714804 ## cpp/src/arrow/testing/gtest_util.cc: ## @@ -574,6 +578,40 @@ std::shared_ptr TweakValidityBit(const std::shared_ptr& array, return MakeArray(data); } +// XXX

[GitHub] [arrow] raulcd commented on pull request #13351: ARROW-16694: [Packaging][Python] Use Mamba instead of conda to build conda environment for windows packaging jobs

2022-06-09 Thread GitBox
raulcd commented on PR #13351: URL: https://github.com/apache/arrow/pull/13351#issuecomment-1151377388 @kszucs @kou this PR fixes the conda packaging jobs that have been failing for Windows for some time. Let me know what you think. -- This is an automated message from the Apache Git Serv

[GitHub] [arrow] pitrou commented on pull request #13334: ARROW-14314: [C++] Sorting dictionary array not implemented

2022-06-09 Thread GitBox
pitrou commented on PR #13334: URL: https://github.com/apache/arrow/pull/13334#issuecomment-1151374775 You would probably have to cast the indices to the concrete array type :-) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [arrow] ArianaVillegas commented on pull request #13334: ARROW-14314: [C++] Sorting dictionary array not implemented

2022-06-09 Thread GitBox
ArianaVillegas commented on PR #13334: URL: https://github.com/apache/arrow/pull/13334#issuecomment-1151372579 Got it, so which is the best way of getting the index? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [arrow] pitrou commented on pull request #13334: ARROW-14314: [C++] Sorting dictionary array not implemented

2022-06-09 Thread GitBox
pitrou commented on PR #13334: URL: https://github.com/apache/arrow/pull/13334#issuecomment-1151370079 You certainly don't want to go through `GetScalar` or string conversion as that will exhibit very bad performance. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [arrow] rtpsw opened a new pull request, #13355: ARROW-16796: [C++] Fix bad defaulting of ExecContext argument

2022-06-09 Thread GitBox
rtpsw opened a new pull request, #13355: URL: https://github.com/apache/arrow/pull/13355 See https://issues.apache.org/jira/browse/ARROW-16796 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow-rs] tustvold commented on a diff in pull request #1822: Change to use `resolver v2`, test more feature flag combinations in CI, fix errors (#1630)

2022-06-09 Thread GitBox
tustvold commented on code in PR #1822: URL: https://github.com/apache/arrow-rs/pull/1822#discussion_r893741151 ## .github/actions/setup-builder/action.yaml: ## @@ -25,6 +25,26 @@ inputs: runs: using: "composite" steps: +- name: Cache Cargo + uses: actions/cache@

[GitHub] [arrow-rs] tustvold commented on pull request #1822: Change to use `resolver v2`, test more feature flag combinations in CI, fix errors (#1630)

2022-06-09 Thread GitBox
tustvold commented on PR #1822: URL: https://github.com/apache/arrow-rs/pull/1822#issuecomment-1151364872 I had to rework the caching to allow splitting up the stages into smaller steps, without having to duplicate "export CARGO_HOME". into every step. Setting these variable at the top-leve

[GitHub] [arrow] ArianaVillegas commented on pull request #13334: ARROW-14314: [C++] Sorting dictionary array not implemented

2022-06-09 Thread GitBox
ArianaVillegas commented on PR #13334: URL: https://github.com/apache/arrow/pull/13334#issuecomment-1151363802 At the end, to avoid doing cast, I'm using stoull to get the index. Should I add test with other types of values on dictionary? -- This is an automated message from the Apache Gi

[GitHub] [arrow] rok commented on pull request #13302: ARROW-16741: [C++] Add Benchmarks for Binary Temporal Operations

2022-06-09 Thread GitBox
rok commented on PR #13302: URL: https://github.com/apache/arrow/pull/13302#issuecomment-1151352636 @iChauster here are the [temporal benchmarks for this PR](https://conbench.ursa.dev/compare/runs/85289eeb3c794766948e9065016c4717...aabd4c1c4d934cfa81256b6f772453f1/) if you're curious. --

[GitHub] [arrow] pitrou commented on pull request #13334: ARROW-14314: [C++] Sorting dictionary array not implemented

2022-06-09 Thread GitBox
pitrou commented on PR #13334: URL: https://github.com/apache/arrow/pull/13334#issuecomment-1151347327 > And another question (OOT), can we build and array from a dictionary array? You can, for example by using the take function (`take(values, indices)`). -- This is an automated mes

[GitHub] [arrow] pitrou commented on pull request #13334: ARROW-14314: [C++] Sorting dictionary array not implemented

2022-06-09 Thread GitBox
pitrou commented on PR #13334: URL: https://github.com/apache/arrow/pull/13334#issuecomment-1151344861 > And, I want to know whether it's possible to cast indices into an array of indices_type or not Ah, yes, of course. Just use `checked_cast` with the right concrete array type.

  1   2   3   >