[GitHub] [arrow] rok commented on pull request #11889: ARROW-14708: [C++] Adding missing abseil dependencies to enable static flight build

2022-03-05 Thread GitBox
rok commented on pull request #11889: URL: https://github.com/apache/arrow/pull/11889#issuecomment-1059907579 @lidavidm this now builds the repro on my laptop with a small adjustment to CMakeLists.txt. Namely last line should be: `target_link_libraries(arrow_example PRIVATE arrow_flight_

[GitHub] [arrow-rs] shanisolomon commented on a change in pull request #1346: Remove existing has_ methods for optional fields in `ColumnChunkMetaData`

2022-03-05 Thread GitBox
shanisolomon commented on a change in pull request #1346: URL: https://github.com/apache/arrow-rs/pull/1346#discussion_r820186754 ## File path: parquet/src/file/metadata.rs ## @@ -428,32 +428,21 @@ impl ColumnChunkMetaData { self.data_page_offset } -/// Retu

[GitHub] [arrow-rs] jackwener commented on issue #1390: `FixedSizeBinaryArray::try_from_sparse_iter` failed when given all Nones

2022-03-05 Thread GitBox
jackwener commented on issue #1390: URL: https://github.com/apache/arrow-rs/issues/1390#issuecomment-1059900498 Has it already been fixed? ```rust fn test_fixed_size_binary_array_from_sparse_iter() { let none_option: Option<[u8; 32]> = None; let input_arg = vec![none_

[GitHub] [arrow-datafusion] matthewmturner commented on pull request #1928: Add db benchmark script

2022-03-05 Thread GitBox
matthewmturner commented on pull request #1928: URL: https://github.com/apache/arrow-datafusion/pull/1928#issuecomment-1059894093 I have this working now. Would at least one other person be able to give it a try. I do the following from root directory of `arrow-datafusion` `

[GitHub] [arrow-julia] codecov-commenter commented on pull request #299: Bump version for release

2022-03-05 Thread GitBox
codecov-commenter commented on pull request #299: URL: https://github.com/apache/arrow-julia/pull/299#issuecomment-1059889078 # [Codecov](https://codecov.io/gh/apache/arrow-julia/pull/299?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_ter

[GitHub] [arrow-julia] quinnj commented on issue #284: Future of the Julia arrow implementation

2022-03-05 Thread GitBox
quinnj commented on issue #284: URL: https://github.com/apache/arrow-julia/issues/284#issuecomment-1059886912 Thank you @kou for all your help. I've created a PR so we can try out an initial release w/ the new process: https://github.com/apache/arrow-julia/pull/299 -- This is an automat

[GitHub] [arrow-rs] HaoYang670 commented on issue #1400: Interesting benchmark results of `min_max_helper`

2022-03-05 Thread GitBox
HaoYang670 commented on issue #1400: URL: https://github.com/apache/arrow-rs/issues/1400#issuecomment-1059886865 Also, using `copy` or `reference` seems to impact the performance of null handling loop. I get the following benchmark result: 1. using `m.iter().copied().reduce(...)`: ```

[GitHub] [arrow-julia] quinnj commented on a change in pull request #277: refactor Arrow.write to support incremental writes

2022-03-05 Thread GitBox
quinnj commented on a change in pull request #277: URL: https://github.com/apache/arrow-julia/pull/277#discussion_r820173661 ## File path: src/write.jl ## @@ -51,131 +51,213 @@ Supported keyword arguments to `Arrow.write` include: """ function write end -write(io_or_file; k

[GitHub] [arrow-julia] quinnj commented on a change in pull request #277: refactor Arrow.write to support incremental writes

2022-03-05 Thread GitBox
quinnj commented on a change in pull request #277: URL: https://github.com/apache/arrow-julia/pull/277#discussion_r820173569 ## File path: src/write.jl ## @@ -51,131 +51,213 @@ Supported keyword arguments to `Arrow.write` include: """ function write end -write(io_or_file; k

[GitHub] [arrow-julia] quinnj commented on a change in pull request #277: refactor Arrow.write to support incremental writes

2022-03-05 Thread GitBox
quinnj commented on a change in pull request #277: URL: https://github.com/apache/arrow-julia/pull/277#discussion_r820173490 ## File path: src/write.jl ## @@ -51,131 +51,213 @@ Supported keyword arguments to `Arrow.write` include: """ function write end -write(io_or_file; k

[GitHub] [arrow-julia] quinnj commented on a change in pull request #277: refactor Arrow.write to support incremental writes

2022-03-05 Thread GitBox
quinnj commented on a change in pull request #277: URL: https://github.com/apache/arrow-julia/pull/277#discussion_r820173392 ## File path: src/write.jl ## @@ -51,131 +51,213 @@ Supported keyword arguments to `Arrow.write` include: """ function write end -write(io_or_file; k

[GitHub] [arrow] lidavidm commented on a change in pull request #12571: RFC: Add inlined data to flight.

2022-03-05 Thread GitBox
lidavidm commented on a change in pull request #12571: URL: https://github.com/apache/arrow/pull/12571#discussion_r820172392 ## File path: format/Flight.proto ## @@ -301,6 +301,33 @@ message Location { */ message Ticket { bytes ticket = 1; + // Data representing some par

[GitHub] [arrow-rs] HaoYang670 commented on issue #1400: Interesting benchmark results of `min_max_helper`

2022-03-05 Thread GitBox
HaoYang670 commented on issue #1400: URL: https://github.com/apache/arrow-rs/issues/1400#issuecomment-1059882916 Strongly agree with you @jhorstmann. `fold` seems to copy all elements in the array, and `reduce` just takes the references. If the code is rewritten as ```rust if null

[GitHub] [arrow-datafusion] Jimexist merged pull request #1929: Fix JIT configs for aarch64

2022-03-05 Thread GitBox
Jimexist merged pull request #1929: URL: https://github.com/apache/arrow-datafusion/pull/1929 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gith

[GitHub] [arrow] Jimexist commented on pull request #12028: ARROW-15192: [Java] Allow use of Jackson 2.12 and higher

2022-03-05 Thread GitBox
Jimexist commented on pull request #12028: URL: https://github.com/apache/arrow/pull/12028#issuecomment-1059880764 i havent tested this locally - so if anyone would give a second opinion i'll wait for a few days before trying to merge as is -- This is an automated message from the Apache

[GitHub] [arrow-julia] quinnj commented on pull request #296: Solving #295

2022-03-05 Thread GitBox
quinnj commented on pull request #296: URL: https://github.com/apache/arrow-julia/pull/296#issuecomment-1059875570 @complyue, thanks for the contribution! I spent some time reviewing your PR and something seemed very familiar about the problem. And that's when I remembered that we already

[GitHub] [arrow-julia] quinnj opened a new pull request #298: Alternative fix for #295.

2022-03-05 Thread GitBox
quinnj opened a new pull request #298: URL: https://github.com/apache/arrow-julia/pull/298 We already have a utility defined (`OrderedChannel`) that we use when writing record batches to ensure batches get _written_ in the same order they are provided; it makes sense to use the same ut

[GitHub] [arrow-datafusion] matthewmturner commented on issue #1930: Add `ObjectStore` support via SQL

2022-03-05 Thread GitBox
matthewmturner commented on issue #1930: URL: https://github.com/apache/arrow-datafusion/issues/1930#issuecomment-1059872236 maybe my objective could be achieved with some command line options instead. for example: Default credentials ``` $ datafusion-cli --object-store s3

[GitHub] [arrow-rs] codecov-commenter edited a comment on pull request #1389: Introduce `ReadOptions` with builder API, filter row groups that satisfy all filters, and enable filter row groups by rang

2022-03-05 Thread GitBox
codecov-commenter edited a comment on pull request #1389: URL: https://github.com/apache/arrow-rs/pull/1389#issuecomment-1058957663 # [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1389?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm

[GitHub] [arrow-rs] yjshen commented on a change in pull request #1389: Introduce `ReadOptions` with builder API, filter row groups that satisfy all filters, and enable filter row groups by range.

2022-03-05 Thread GitBox
yjshen commented on a change in pull request #1389: URL: https://github.com/apache/arrow-rs/pull/1389#discussion_r820164755 ## File path: parquet/src/file/serialized_reader.rs ## @@ -127,6 +127,56 @@ pub struct SerializedFileReader { metadata: ParquetMetaData, } +/// A

[GitHub] [arrow] GavinRay97 edited a comment on issue #12570: Arrow nightly Maven releases don't seem to work

2022-03-05 Thread GitBox
GavinRay97 edited a comment on issue #12570: URL: https://github.com/apache/arrow/issues/12570#issuecomment-1059852761 Here is a Node.js script to download from the Nightlies and extract the assets into Maven repository structure: ```json { "name": "arrow-download-nightly-a

[GitHub] [arrow] ursabot edited a comment on pull request #12320: ARROW-15483: [Release] Revamp the verification scripts

2022-03-05 Thread GitBox
ursabot edited a comment on pull request #12320: URL: https://github.com/apache/arrow/pull/12320#issuecomment-1059837801 Benchmark runs are scheduled for baseline = 4ef95eb89f9202dfcd9017633cf55671d56e337f and contender = a13870e33eb8b25a1e9cee28a310c7c6cd9a4fb4. a13870e33eb8b25a1e9cee28a

[GitHub] [arrow-datafusion] yjshen commented on pull request #1929: Fix JIT configs for aarch64

2022-03-05 Thread GitBox
yjshen commented on pull request #1929: URL: https://github.com/apache/arrow-datafusion/pull/1929#issuecomment-1059861425 Thanks again for fixing this. ❤️ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [arrow] eelxpeng commented on pull request #12055: ARROW-11989: [C++][Python] Improve ChunkedArray's complexity for the access of elements

2022-03-05 Thread GitBox
eelxpeng commented on pull request #12055: URL: https://github.com/apache/arrow/pull/12055#issuecomment-1059861418 Verified the speed improvement on the [example](https://pastebin.com/pSkYHQn9) provided above. Thanks for the good work! @edponce I have one question, does the change

[GitHub] [arrow] GavinRay97 edited a comment on issue #12570: Arrow nightly Maven releases don't seem to work

2022-03-05 Thread GitBox
GavinRay97 edited a comment on issue #12570: URL: https://github.com/apache/arrow/issues/12570#issuecomment-1059852761 Here is a Node.js script to download from the Nightlies and extract the assets into Maven repository structure: ```json { "name": "arrow-download-nightly-a

[GitHub] [arrow] ursabot edited a comment on pull request #12320: ARROW-15483: [Release] Revamp the verification scripts

2022-03-05 Thread GitBox
ursabot edited a comment on pull request #12320: URL: https://github.com/apache/arrow/pull/12320#issuecomment-1059837801 Benchmark runs are scheduled for baseline = 4ef95eb89f9202dfcd9017633cf55671d56e337f and contender = a13870e33eb8b25a1e9cee28a310c7c6cd9a4fb4. a13870e33eb8b25a1e9cee28a

[GitHub] [arrow-datafusion] matthewmturner commented on a change in pull request #1928: Add db benchmark script

2022-03-05 Thread GitBox
matthewmturner commented on a change in pull request #1928: URL: https://github.com/apache/arrow-datafusion/pull/1928#discussion_r820160520 ## File path: benchmarks/db-benchmark/db-benchmark.dockerfile ## @@ -0,0 +1,54 @@ +FROM ubuntu +ARG DEBIAN_FRONTEND=noninteractive + +RUN

[GitHub] [arrow] GavinRay97 edited a comment on issue #12570: Arrow nightly Maven releases don't seem to work

2022-03-05 Thread GitBox
GavinRay97 edited a comment on issue #12570: URL: https://github.com/apache/arrow/issues/12570#issuecomment-1059852761 Here is a Node.js script to download from the Nightlies and extract the assets into Maven repository structure: ```json { "name": "arrow-download-nightly-a

[GitHub] [arrow] ursabot edited a comment on pull request #12320: ARROW-15483: [Release] Revamp the verification scripts

2022-03-05 Thread GitBox
ursabot edited a comment on pull request #12320: URL: https://github.com/apache/arrow/pull/12320#issuecomment-1059837801 Benchmark runs are scheduled for baseline = 4ef95eb89f9202dfcd9017633cf55671d56e337f and contender = a13870e33eb8b25a1e9cee28a310c7c6cd9a4fb4. a13870e33eb8b25a1e9cee28a

[GitHub] [arrow] GavinRay97 edited a comment on issue #12570: Arrow nightly Maven releases don't seem to work

2022-03-05 Thread GitBox
GavinRay97 edited a comment on issue #12570: URL: https://github.com/apache/arrow/issues/12570#issuecomment-1059852761 Here is a Node.js script to download from the Nightlies and extract the assets into Maven repository structure: ```json { "name": "arrow-download-nightly-a

[GitHub] [arrow] GavinRay97 commented on issue #12570: Arrow nightly Maven releases don't seem to work

2022-03-05 Thread GitBox
GavinRay97 commented on issue #12570: URL: https://github.com/apache/arrow/issues/12570#issuecomment-1059852761 Here is a Node.js script to download from the Nightlies and extract the assets into Maven repository structure: ```js // index.mjs // Run with: $ node index.mjs im

[GitHub] [arrow] ursabot edited a comment on pull request #12320: ARROW-15483: [Release] Revamp the verification scripts

2022-03-05 Thread GitBox
ursabot edited a comment on pull request #12320: URL: https://github.com/apache/arrow/pull/12320#issuecomment-1059837801 Benchmark runs are scheduled for baseline = 4ef95eb89f9202dfcd9017633cf55671d56e337f and contender = a13870e33eb8b25a1e9cee28a310c7c6cd9a4fb4. a13870e33eb8b25a1e9cee28a

[GitHub] [arrow-datafusion] viirya commented on a change in pull request #1921: Add timeout to can_grow_directly when waiting for the Condvar.

2022-03-05 Thread GitBox
viirya commented on a change in pull request #1921: URL: https://github.com/apache/arrow-datafusion/pull/1921#discussion_r820135554 ## File path: datafusion/src/execution/memory_manager.rs ## @@ -340,7 +341,13 @@ impl MemoryManager { } else if current < min_per_rqt

[GitHub] [arrow-datafusion] viirya closed issue #1920: Add timeout to can_grow_directly

2022-03-05 Thread GitBox
viirya closed issue #1920: URL: https://github.com/apache/arrow-datafusion/issues/1920 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsu

[GitHub] [arrow] ursabot commented on pull request #12320: ARROW-15483: [Release] Revamp the verification scripts

2022-03-05 Thread GitBox
ursabot commented on pull request #12320: URL: https://github.com/apache/arrow/pull/12320#issuecomment-1059837801 Benchmark runs are scheduled for baseline = 4ef95eb89f9202dfcd9017633cf55671d56e337f and contender = a13870e33eb8b25a1e9cee28a310c7c6cd9a4fb4. a13870e33eb8b25a1e9cee28a310c7c6

[GitHub] [arrow-datafusion] alamb merged pull request #1860: Increase default partition column type from Dict(UInt8) to Dict(UInt16)

2022-03-05 Thread GitBox
alamb merged pull request #1860: URL: https://github.com/apache/arrow-datafusion/pull/1860 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-

[GitHub] [arrow-datafusion] alamb commented on pull request #1860: Increase default partition column type from Dict(UInt8) to Dict(UInt16)

2022-03-05 Thread GitBox
alamb commented on pull request #1860: URL: https://github.com/apache/arrow-datafusion/pull/1860#issuecomment-1059837724 Filed https://github.com/apache/arrow-datafusion/issues/1931 for follow on work ; Thanks again @Igosuki and @rdettai -- This is an automated message from the Apach

[GitHub] [arrow-datafusion] alamb closed issue #1859: UInt8 isn't enough for partitioning values

2022-03-05 Thread GitBox
alamb closed issue #1859: URL: https://github.com/apache/arrow-datafusion/issues/1859 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsub

[GitHub] [arrow-datafusion] alamb opened a new issue #1931: More efficient Dictionary / constant encoding for partition values in ListingFileProvider

2022-03-05 Thread GitBox
alamb opened a new issue #1931: URL: https://github.com/apache/arrow-datafusion/issues/1931 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** The `ListingFileProvider` after https://github.com/apache/arrow-datafusion/pull/1860

[GitHub] [arrow] kou closed pull request #12320: ARROW-15483: [Release] Revamp the verification scripts

2022-03-05 Thread GitBox
kou closed pull request #12320: URL: https://github.com/apache/arrow/pull/12320 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...

[GitHub] [arrow] kou commented on pull request #12320: ARROW-15483: [Release] Revamp the verification scripts

2022-03-05 Thread GitBox
kou commented on pull request #12320: URL: https://github.com/apache/arrow/pull/12320#issuecomment-1059836982 Merged: :-) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [arrow-datafusion] thinkharderdev commented on a change in pull request #1929: Fix JIT configs for aarch64

2022-03-05 Thread GitBox
thinkharderdev commented on a change in pull request #1929: URL: https://github.com/apache/arrow-datafusion/pull/1929#discussion_r820151234 ## File path: datafusion-jit/src/jit.rs ## @@ -51,6 +52,28 @@ impl Default for JIT { module, } } + +#[cfg(t

[GitHub] [arrow-datafusion] hntd187 commented on issue #1544: Streaming support for DataFusion

2022-03-05 Thread GitBox
hntd187 commented on issue #1544: URL: https://github.com/apache/arrow-datafusion/issues/1544#issuecomment-1059835380 @matthewmturner here is a first pass are people able to start editing this doc? https://docs.google.com/document/d/1ZR171NcI_lrn7e7yVjSZFtCkNM7w2yzia_pH3b4QL_c/edit

[GitHub] [arrow-datafusion] alamb commented on pull request #1860: Increase default partition column type from Dict(UInt8) to Dict(UInt16)

2022-03-05 Thread GitBox
alamb commented on pull request #1860: URL: https://github.com/apache/arrow-datafusion/pull/1860#issuecomment-1059834753 In order to unstick this PR I plan to file a follow on ticket to add a more sophisticated handling of dictionaries and then merge this PR in as a workaround until it is

[GitHub] [arrow-rs] alamb commented on issue #1391: Arrow release 10.0.0

2022-03-05 Thread GitBox
alamb commented on issue #1391: URL: https://github.com/apache/arrow-rs/issues/1391#issuecomment-1059833775 RC is available: https://lists.apache.org/thread/0qdntovo308g1cxsj1h113b1lr6gfvhh -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [arrow-rs] sunchao commented on a change in pull request #1389: Introduce `ReadOptions` with builder API, filter row groups that satisfy all filters, and enable filter row groups by range.

2022-03-05 Thread GitBox
sunchao commented on a change in pull request #1389: URL: https://github.com/apache/arrow-rs/pull/1389#discussion_r820143715 ## File path: parquet/src/file/serialized_reader.rs ## @@ -127,6 +127,56 @@ pub struct SerializedFileReader { metadata: ParquetMetaData, } +/// A

[GitHub] [arrow-datafusion] houqp commented on a change in pull request #1928: Add db benchmark script

2022-03-05 Thread GitBox
houqp commented on a change in pull request #1928: URL: https://github.com/apache/arrow-datafusion/pull/1928#discussion_r820143271 ## File path: benchmarks/db-benchmark/db-benchmark.dockerfile ## @@ -0,0 +1,54 @@ +FROM ubuntu +ARG DEBIAN_FRONTEND=noninteractive + +RUN apt-get u

[GitHub] [arrow-datafusion] houqp commented on a change in pull request #1928: Add db benchmark script

2022-03-05 Thread GitBox
houqp commented on a change in pull request #1928: URL: https://github.com/apache/arrow-datafusion/pull/1928#discussion_r820143207 ## File path: benchmarks/db-benchmark/db-benchmark.dockerfile ## @@ -0,0 +1,54 @@ +FROM ubuntu +ARG DEBIAN_FRONTEND=noninteractive + +RUN apt-get u

[GitHub] [arrow-datafusion] houqp commented on a change in pull request #1929: Fix JIT configs for aarch64

2022-03-05 Thread GitBox
houqp commented on a change in pull request #1929: URL: https://github.com/apache/arrow-datafusion/pull/1929#discussion_r820142824 ## File path: datafusion-jit/src/jit.rs ## @@ -51,6 +52,28 @@ impl Default for JIT { module, } } + +#[cfg(target_arc

[GitHub] [arrow-datafusion] hntd187 commented on pull request #1600: Experimental commit for stream processing

2022-03-05 Thread GitBox
hntd187 commented on pull request #1600: URL: https://github.com/apache/arrow-datafusion/pull/1600#issuecomment-1059820737 Sure, I should have the docs up for that proposal shortly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [arrow-datafusion] hntd187 closed pull request #1600: Experimental commit for stream processing

2022-03-05 Thread GitBox
hntd187 closed pull request #1600: URL: https://github.com/apache/arrow-datafusion/pull/1600 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: githu

[GitHub] [arrow] ursabot edited a comment on pull request #12490: PARQUET-2131: Number values decoded DCHECKs should be exceptions

2022-03-05 Thread GitBox
ursabot edited a comment on pull request #12490: URL: https://github.com/apache/arrow/pull/12490#issuecomment-1059610557 Benchmark runs are scheduled for baseline = 6734d0faede2030e202aee5a0c7a1ace8eefd2d5 and contender = 4ef95eb89f9202dfcd9017633cf55671d56e337f. 4ef95eb89f9202dfcd9017633

[GitHub] [arrow] GavinRay97 edited a comment on issue #12570: Arrow nightly Maven releases don't seem to work

2022-03-05 Thread GitBox
GavinRay97 edited a comment on issue #12570: URL: https://github.com/apache/arrow/issues/12570#issuecomment-1059817506 I've used a regular GitHub repository as a Maven repository before, for that you have to use the "raw" URL: ```groovy repositories { maven { n

[GitHub] [arrow] GavinRay97 commented on issue #12570: Arrow nightly Maven releases don't seem to work

2022-03-05 Thread GitBox
GavinRay97 commented on issue #12570: URL: https://github.com/apache/arrow/issues/12570#issuecomment-1059817506 I've used a regular GitHub repository as a Maven repository before, for that you have to use the "raw" URL Maybe something like this might be needed for using tagged relea

[GitHub] [arrow-datafusion] matthewmturner commented on pull request #1929: Fix JIT configs for aarch64

2022-03-05 Thread GitBox
matthewmturner commented on pull request #1929: URL: https://github.com/apache/arrow-datafusion/pull/1929#issuecomment-1059814999 I noticed this locally as well. Thx for fixing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [arrow-datafusion] alamb commented on pull request #1929: Fix JIT configs for aarch64

2022-03-05 Thread GitBox
alamb commented on pull request #1929: URL: https://github.com/apache/arrow-datafusion/pull/1929#issuecomment-1059813789 cc @yjshen -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [arrow-datafusion] matthewmturner commented on pull request #1928: Add db benchmark script

2022-03-05 Thread GitBox
matthewmturner commented on pull request #1928: URL: https://github.com/apache/arrow-datafusion/pull/1928#issuecomment-1059812594 Lol. I unknowingly created a shell script that had a name conflict with a db-benchmark script which caused issue. -- This is an automated message from the A

[GitHub] [arrow-datafusion] viirya closed pull request #1921: Add timeout to can_grow_directly when waiting for the Condvar.

2022-03-05 Thread GitBox
viirya closed pull request #1921: URL: https://github.com/apache/arrow-datafusion/pull/1921 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github

[GitHub] [arrow-datafusion] viirya commented on a change in pull request #1921: Add timeout to can_grow_directly when waiting for the Condvar.

2022-03-05 Thread GitBox
viirya commented on a change in pull request #1921: URL: https://github.com/apache/arrow-datafusion/pull/1921#discussion_r820135554 ## File path: datafusion/src/execution/memory_manager.rs ## @@ -340,7 +341,13 @@ impl MemoryManager { } else if current < min_per_rqt

[GitHub] [arrow] pachadotdev edited a comment on pull request #10546: ARROW-12845: [R] [C++] S3 connections for different providers

2022-03-05 Thread GitBox
pachadotdev edited a comment on pull request #10546: URL: https://github.com/apache/arrow/pull/10546#issuecomment-1059806276 Hi @jonkeane, do I just delete the branch?. IMO this very old PR had some cool additions for the documentation. -- This is an automated message from the Apache Git

[GitHub] [arrow] pachadotdev commented on pull request #10546: ARROW-12845: [R] [C++] S3 connections for different providers

2022-03-05 Thread GitBox
pachadotdev commented on pull request #10546: URL: https://github.com/apache/arrow/pull/10546#issuecomment-1059806276 Hi @jonkeane, do I just delete the branch. IMO this very old PR had some cool additions for the documentation. -- This is an automated message from the Apache Git Service

[GitHub] [arrow-datafusion] matthewmturner commented on issue #1930: Add `ObjectStore` support via SQL

2022-03-05 Thread GitBox
matthewmturner commented on issue #1930: URL: https://github.com/apache/arrow-datafusion/issues/1930#issuecomment-1059805594 actually, im not sure how well those parameters in `register_object_store` will generalize to other `ObjectStore` besides s3. so now im not sure if a general funct

[GitHub] [arrow-datafusion] matthewmturner commented on issue #1930: Add `ObjectStore` support via SQL

2022-03-05 Thread GitBox
matthewmturner commented on issue #1930: URL: https://github.com/apache/arrow-datafusion/issues/1930#issuecomment-1059804187 @seddonm1 @yjshen @houqp FYI - in case you have thoughts on this. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [arrow-datafusion] matthewmturner opened a new issue #1930: Add `ObjectStore` support via SQL

2022-03-05 Thread GitBox
matthewmturner opened a new issue #1930: URL: https://github.com/apache/arrow-datafusion/issues/1930 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** A clear and concise description of what the problem is. Ex. I'm always frustrate

[GitHub] [arrow] ursabot edited a comment on pull request #12521: ARROW-15795: [Java] Add a getter for the timeZone in timestamp with timezone vectors

2022-03-05 Thread GitBox
ursabot edited a comment on pull request #12521: URL: https://github.com/apache/arrow/pull/12521#issuecomment-1059579237 Benchmark runs are scheduled for baseline = 348057aea798bf612eddcae42495234e5853fd76 and contender = 6734d0faede2030e202aee5a0c7a1ace8eefd2d5. 6734d0faede2030e202aee5a0

[GitHub] [arrow] vibhatha commented on pull request #12113: ARROW-14679: [R] [C++] Handle suffix argument in joins

2022-03-05 Thread GitBox
vibhatha commented on pull request #12113: URL: https://github.com/apache/arrow/pull/12113#issuecomment-1059789949 working resolving some merge conflicts -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [arrow-datafusion] yjshen commented on a change in pull request #1905: Avoid repeated `open` for one single file and simplify object reader API on the `sync` part

2022-03-05 Thread GitBox
yjshen commented on a change in pull request #1905: URL: https://github.com/apache/arrow-datafusion/pull/1905#discussion_r820119627 ## File path: datafusion/src/datasource/object_store/local.rs ## @@ -82,23 +112,12 @@ impl ObjectReader for LocalFileReader { ) }

[GitHub] [arrow] jonkeane commented on pull request #11682: ARROW-14679: [R] [C++] Handle suffix argument in joins

2022-03-05 Thread GitBox
jonkeane commented on pull request #11682: URL: https://github.com/apache/arrow/pull/11682#issuecomment-1059787534 Closing in favor of #12113 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] jonkeane closed pull request #10546: ARROW-12845: [R] [C++] S3 connections for different providers

2022-03-05 Thread GitBox
jonkeane closed pull request #10546: URL: https://github.com/apache/arrow/pull/10546 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubs

[GitHub] [arrow] lidavidm commented on pull request #12573: ARROW-15721: [Docs][FlightSQL] Add Flight/Flight SQL to subprojects

2022-03-05 Thread GitBox
lidavidm commented on pull request #12573: URL: https://github.com/apache/arrow/pull/12573#issuecomment-1059787241 CC @jduo if you'd like to take a look -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] jonkeane closed pull request #9999: ARROW-11755: [R] Add tests from dplyr/test-mutate.r

2022-03-05 Thread GitBox
jonkeane closed pull request #: URL: https://github.com/apache/arrow/pull/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr

[GitHub] [arrow] github-actions[bot] removed a comment on pull request #12573: ARROW-15721: [Docs][FlightSQL] Add Flight/Flight SQL to subprojects

2022-03-05 Thread GitBox
github-actions[bot] removed a comment on pull request #12573: URL: https://github.com/apache/arrow/pull/12573#issuecomment-1059786888 :warning: Ticket **has not been started in JIRA**, please click 'Start Progress'. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [arrow] lidavidm commented on pull request #12573: ARROW-15721: [Docs][FlightSQL] Add Flight/Flight SQL to subprojects

2022-03-05 Thread GitBox
lidavidm commented on pull request #12573: URL: https://github.com/apache/arrow/pull/12573#issuecomment-1059786910 I would like to add sequence diagrams, how do we feel about something like [sphinxcontrib-mermaid](https://pypi.org/project/sphinxcontrib-mermaid/)? -- This is an automated

[GitHub] [arrow] github-actions[bot] commented on pull request #12573: ARROW-15721: [Docs][FlightSQL] Add Flight/Flight SQL to subprojects

2022-03-05 Thread GitBox
github-actions[bot] commented on pull request #12573: URL: https://github.com/apache/arrow/pull/12573#issuecomment-1059786886 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [arrow] lidavidm opened a new pull request #12573: ARROW-15721: [Docs][FlightSQL] Add Flight/Flight SQL to subprojects

2022-03-05 Thread GitBox
lidavidm opened a new pull request #12573: URL: https://github.com/apache/arrow/pull/12573 Adds a Flight SQL documentation page that we can link to from the website. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [arrow-datafusion] matthewmturner commented on pull request #1922: Add write_csv to DataFrame

2022-03-05 Thread GitBox
matthewmturner commented on pull request #1922: URL: https://github.com/apache/arrow-datafusion/pull/1922#issuecomment-1059783928 @Jimexist what would you think about merging this as is and then tackling that point as part of convos on #1712 where there is interest in moving DataFrame fro

[GitHub] [arrow-datafusion] alamb commented on a change in pull request #1905: Avoid repeated `open` for one single file and simplify object reader API on the `sync` part

2022-03-05 Thread GitBox
alamb commented on a change in pull request #1905: URL: https://github.com/apache/arrow-datafusion/pull/1905#discussion_r820111433 ## File path: datafusion/src/datasource/object_store/local.rs ## @@ -82,23 +112,12 @@ impl ObjectReader for LocalFileReader { ) } -

[GitHub] [arrow-datafusion] thinkharderdev commented on a change in pull request #1912: Refactor the event channel

2022-03-05 Thread GitBox
thinkharderdev commented on a change in pull request #1912: URL: https://github.com/apache/arrow-datafusion/pull/1912#discussion_r820111007 ## File path: ballista/rust/scheduler/src/scheduler_server/mod.rs ## @@ -0,0 +1,142 @@ +// Licensed to the Apache Software Foundation (ASF

[GitHub] [arrow-datafusion] thinkharderdev commented on a change in pull request #1913: Refactor scheduler state mod

2022-03-05 Thread GitBox
thinkharderdev commented on a change in pull request #1913: URL: https://github.com/apache/arrow-datafusion/pull/1913#discussion_r820110604 ## File path: ballista/rust/scheduler/src/state/persistent_state.rs ## @@ -0,0 +1,312 @@ +// Licensed to the Apache Software Foundation (A

[GitHub] [arrow-datafusion] thinkharderdev opened a new pull request #1929: Fix JIT configs for aarch64

2022-03-05 Thread GitBox
thinkharderdev opened a new pull request #1929: URL: https://github.com/apache/arrow-datafusion/pull/1929 # Which issue does this PR close? Closes #. # Rationale for this change Currently tests do not pass on Apple Silicone (or any non x86_64) platform. Thi

[GitHub] [arrow] ursabot edited a comment on pull request #12488: PARQUET-2130: Fix crash in debug with non-standard key names.

2022-03-05 Thread GitBox
ursabot edited a comment on pull request #12488: URL: https://github.com/apache/arrow/pull/12488#issuecomment-1059557562 Benchmark runs are scheduled for baseline = 762bb3d64f055db72ebb61ebe0d53a929ea8cd34 and contender = 348057aea798bf612eddcae42495234e5853fd76. 348057aea798bf612eddcae42

[GitHub] [arrow-rs] codecov-commenter edited a comment on pull request #1396: Add note in contributing guideline about types of contributions

2022-03-05 Thread GitBox
codecov-commenter edited a comment on pull request #1396: URL: https://github.com/apache/arrow-rs/pull/1396#issuecomment-1058450817 # [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1396?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm

[GitHub] [arrow] github-actions[bot] commented on pull request #12572: add GitHub URL for PyPi

2022-03-05 Thread GitBox
github-actions[bot] commented on pull request #12572: URL: https://github.com/apache/arrow/pull/12572#issuecomment-1059771601 Thanks for opening a pull request! If this is not a [minor PR](https://github.com/apache/arrow/blob/master/CONTRIBUTING.md#Minor-Fixes). Could you op

[GitHub] [arrow] andriyor opened a new pull request #12572: add GitHub URL for PyPi

2022-03-05 Thread GitBox
andriyor opened a new pull request #12572: URL: https://github.com/apache/arrow/pull/12572 Warehouse now uses the project_urls provided to display links in the sidebar on [this screen](https://pypi.org/project/requests/), as well as including them in API responses to help the automation to

[GitHub] [arrow-datafusion] alamb closed issue #1908: [Ballista] Refactor ballista scheduler server

2022-03-05 Thread GitBox
alamb closed issue #1908: URL: https://github.com/apache/arrow-datafusion/issues/1908 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsub

[GitHub] [arrow-datafusion] alamb merged pull request #1911: Refactor scheduler server

2022-03-05 Thread GitBox
alamb merged pull request #1911: URL: https://github.com/apache/arrow-datafusion/pull/1911 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-

[GitHub] [arrow-datafusion] alamb commented on pull request #1911: Refactor scheduler server

2022-03-05 Thread GitBox
alamb commented on pull request #1911: URL: https://github.com/apache/arrow-datafusion/pull/1911#issuecomment-1059771423 cc @realno @liukun4515 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow-datafusion] alamb commented on pull request #1911: Refactor scheduler server

2022-03-05 Thread GitBox
alamb commented on pull request #1911: URL: https://github.com/apache/arrow-datafusion/pull/1911#issuecomment-1059771098 Rather than allow PRs to accumulate, I'll merge them in as they seem uncontroversial and have been reviewed. We can always continue reorganizing as we move forward.

[GitHub] [arrow] lidavidm commented on issue #12570: Arrow nightly Maven releases don't seem to work

2022-03-05 Thread GitBox
lidavidm commented on issue #12570: URL: https://github.com/apache/arrow/issues/12570#issuecomment-1059770864 @davisusanibar were you able to get this to work? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [arrow-datafusion] alamb commented on pull request #1898: Remove uneeded Mutex in Ballista Client

2022-03-05 Thread GitBox
alamb commented on pull request #1898: URL: https://github.com/apache/arrow-datafusion/pull/1898#issuecomment-1059770069 Rebased -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [arrow-datafusion] alamb merged pull request #1917: Rename DEVELOPERS.md to CONTRIBUTING.md and added introduction

2022-03-05 Thread GitBox
alamb merged pull request #1917: URL: https://github.com/apache/arrow-datafusion/pull/1917 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-

[GitHub] [arrow-rs] alamb merged pull request #1402: fix: Fix grpc schema hack in flight integration test

2022-03-05 Thread GitBox
alamb merged pull request #1402: URL: https://github.com/apache/arrow-rs/pull/1402 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr

[GitHub] [arrow-rs] alamb closed issue #1398: Integration Test is failing on master branch

2022-03-05 Thread GitBox
alamb closed issue #1398: URL: https://github.com/apache/arrow-rs/issues/1398 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@a

[GitHub] [arrow-rs] alamb commented on issue #180: Parquet does not support wasm32-unknown-unknown target

2022-03-05 Thread GitBox
alamb commented on issue #180: URL: https://github.com/apache/arrow-rs/issues/180#issuecomment-1059769108 Thank you for the update @kylebarron -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [arrow-datafusion] alamb commented on issue #1384: Support `register_parquet` from a list of files

2022-03-05 Thread GitBox
alamb commented on issue #1384: URL: https://github.com/apache/arrow-datafusion/issues/1384#issuecomment-1059768319 I think https://github.com/apache/arrow-datafusion/pull/1863 from @matthewmturner may start to lay some of the foundations for doing something similar (e.g. a ObjectStore b

[GitHub] [arrow] xhochy commented on pull request #12562: [Minor]: Mac m1 minor test fix [WIP]

2022-03-05 Thread GitBox
xhochy commented on pull request #12562: URL: https://github.com/apache/arrow/pull/12562#issuecomment-1059764529 This should not be necessary. These variables should be picked up automatically -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [arrow-rs] jhorstmann commented on issue #1400: Interesting benchmark results of `min_max_helper`

2022-03-05 Thread GitBox
jhorstmann commented on issue #1400: URL: https://github.com/apache/arrow-rs/issues/1400#issuecomment-1059762317 Good point, I tried some other variations and it seems to come down to the handling of references vs copying. The following two variations lead to different code: ```

[GitHub] [arrow-rs] codecov-commenter commented on pull request #1401: add support for date and timestamp prior to 1/1/1970

2022-03-05 Thread GitBox
codecov-commenter commented on pull request #1401: URL: https://github.com/apache/arrow-rs/pull/1401#issuecomment-1059761599 # [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1401?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [arrow] dragosmg commented on a change in pull request #12319: ARROW-14199 [R] bindings for format (where possible)

2022-03-05 Thread GitBox
dragosmg commented on a change in pull request #12319: URL: https://github.com/apache/arrow/pull/12319#discussion_r820098108 ## File path: r/R/dplyr-funcs-type.R ## @@ -292,3 +293,18 @@ register_bindings_type_elementwise <- function() { is_inf & !call_binding("is.na", is_i

[GitHub] [arrow] dragosmg commented on a change in pull request #12319: ARROW-14199 [R] bindings for format (where possible)

2022-03-05 Thread GitBox
dragosmg commented on a change in pull request #12319: URL: https://github.com/apache/arrow/pull/12319#discussion_r820098058 ## File path: r/tests/testthat/test-dplyr-funcs-type.R ## @@ -843,3 +843,90 @@ test_that("as.Date() converts successfully from date, timestamp, integer,

  1   2   >