Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-20 Thread Micah Kornfield
Rebasing myself seems to have fixed it. On Mon, Apr 20, 2020 at 10:50 PM Micah Kornfield wrote: > Did something go wrong with the rebase of open PRs? I'm seeing a lot of > diffs show up after rebasing off of master? > > Thanks, > Micah > > On Mon, Apr 20, 2020 at 8:59 PM Sutou Kouhei wrote: >

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-20 Thread Micah Kornfield
Did something go wrong with the rebase of open PRs? I'm seeing a lot of diffs show up after rebasing off of master? Thanks, Micah On Mon, Apr 20, 2020 at 8:59 PM Sutou Kouhei wrote: > MSYS2: Merged: https://github.com/msys2/MINGW-packages/pull/6396 > > In > "Re: [VOTE] Release Apache Arrow

Re: [C++] Revamping approach to Arrow compute kernel development

2020-04-20 Thread Sven Wagner-Boysen
Hi Wes, I think reducing temporary memory allocation is a great effort and will show great benefit in compute intensive scenarios. As we are mainly working with the Rust and Datafusion part of the Arrow project I was wondering how we could best align the concepts and implementations on that level.

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-20 Thread Sutou Kouhei
MSYS2: Merged: https://github.com/msys2/MINGW-packages/pull/6396 In "Re: [VOTE] Release Apache Arrow 0.17.0 - RC0" on Mon, 20 Apr 2020 20:09:29 -0700, Neal Richardson wrote: > Homebrew updates submitted: > https://github.com/Homebrew/homebrew-core/pull/53475 > > R packaging hopefully will

[jira] [Created] (ARROW-8538) [Packaging] Remove boost from homebrew formula

2020-04-20 Thread Neal Richardson (Jira)
Neal Richardson created ARROW-8538: -- Summary: [Packaging] Remove boost from homebrew formula Key: ARROW-8538 URL: https://issues.apache.org/jira/browse/ARROW-8538 Project: Apache Arrow Issue

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-20 Thread Neal Richardson
Homebrew updates submitted: https://github.com/Homebrew/homebrew-core/pull/53475 R packaging hopefully will go through tomorrow. Neal On Mon, Apr 20, 2020 at 7:28 PM Andy Grove wrote: > Well, I got trhe crates published, but there's a nasty workaround for users > that want to use these crates

[jira] [Created] (ARROW-8537) [C++] Performance regression from ARROW-8523

2020-04-20 Thread Yibo Cai (Jira)
Yibo Cai created ARROW-8537: --- Summary: [C++] Performance regression from ARROW-8523 Key: ARROW-8537 URL: https://issues.apache.org/jira/browse/ARROW-8537 Project: Apache Arrow Issue Type: Bug

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-20 Thread Andy Grove
Well, I got trhe crates published, but there's a nasty workaround for users that want to use these crates as a dependency and it means there is no real dependency management on the Flight protocol version. I think the answer is that we need to publish the Flight.proto as part of the arrow-flight cr

[jira] [Created] (ARROW-8536) [Rust] Failed to locate format/Flight.proto in any parent directory

2020-04-20 Thread Andy Grove (Jira)
Andy Grove created ARROW-8536: - Summary: [Rust] Failed to locate format/Flight.proto in any parent directory Key: ARROW-8536 URL: https://issues.apache.org/jira/browse/ARROW-8536 Project: Apache Arrow

Re: [Java] Memory Allocation Tips

2020-04-20 Thread Fan Liya
Hi Razvan, Arrow Java is based on off-heap memory. So it does not rely on GC. Some of the recommended best practice can be found in https://arrow.apache.org/docs/java/vector.html Best, Liya Fan On Mon, Apr 20, 2020 at 8:05 PM Razvan Chitu wrote: > Hi, > > Does the Arrow community have any ti

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-20 Thread Andy Grove
I've run into issues publishing the Rust crates and I don't think I can resolve this tonight. I am documenting the issue in https://issues.apache.org/jira/browse/ARROW-8535 On Mon, Apr 20, 2020 at 5:02 PM Krisztián Szűcs wrote: > Created a PR with updated docs. > > Conda post release task is le

[jira] [Created] (ARROW-8535) [Rust] Fix issues discovered when releasing 0.17.0

2020-04-20 Thread Andy Grove (Jira)
Andy Grove created ARROW-8535: - Summary: [Rust] Fix issues discovered when releasing 0.17.0 Key: ARROW-8535 URL: https://issues.apache.org/jira/browse/ARROW-8535 Project: Apache Arrow Issue Type:

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-20 Thread Krisztián Szűcs
Created a PR with updated docs. Conda post release task is left, it's a bit strange that the conda-forge autotick bot has not created the version bump PRs yet. I'm updating them manually tomorrow. 1. [x] rebase 2. [x] upload source 3. [x] upload binaries 4. [x] update website 5. [x] upload r

[jira] [Created] (ARROW-8534) [C++][CSV] Issue building CSV component under GCC 6.1.0

2020-04-20 Thread Ross Wolfson (Jira)
Ross Wolfson created ARROW-8534: --- Summary: [C++][CSV] Issue building CSV component under GCC 6.1.0 Key: ARROW-8534 URL: https://issues.apache.org/jira/browse/ARROW-8534 Project: Apache Arrow Is

[jira] [Created] (ARROW-8533) [Release] Don't commit doctrees in the docs post release script

2020-04-20 Thread Krisztian Szucs (Jira)
Krisztian Szucs created ARROW-8533: -- Summary: [Release] Don't commit doctrees in the docs post release script Key: ARROW-8533 URL: https://issues.apache.org/jira/browse/ARROW-8533 Project: Apache Arr

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-20 Thread Sutou Kouhei
Hi, I've uploaded C# packages. I'll also work on MSYS2: The current status is (`x` means done): 1. [x] rebase 2. [x] upload source 3. [x] upload binaries 4. [x] update website 5. [x] upload ruby gems 6. [x] upload js packages 8. [x] upload C# packages 9. [Andy] upload rust crates 10. [ ]

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-20 Thread Krisztián Szűcs
On Mon, Apr 20, 2020 at 11:17 PM Andy Grove wrote: > > Ok, I can look into this after work today (in about 3 hours). Great, thanks! The current status is (`x` means done): 1. [x] rebase 2. [x] upload source 3. [x] upload binaries 4. [x] update website 5. [x] upload ruby gems 6. [x] upload

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-20 Thread Andy Grove
Ok, I can look into this after work today (in about 3 hours). On Mon, Apr 20, 2020, 2:47 PM Krisztián Szűcs wrote: > Thanks Andy! I tried to upload the rust packages but arrow-flight, > but a version pin is missing from the package tree: > > error: all dependencies must have a version specified

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-20 Thread Krisztián Szűcs
Thanks Andy! I tried to upload the rust packages but arrow-flight, but a version pin is missing from the package tree: error: all dependencies must have a version specified when publishing. dependency `arrow-flight` does not specify a version Please upload the packages! Also added Uwe and Kou to

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-20 Thread Neal Richardson
I'm working on R and will also do Homebrew. On Mon, Apr 20, 2020 at 1:24 PM Andy Grove wrote: > You should have an invite for the arrow-flight crate. Please check > https://crates.io/me/pending-invites > > On Mon, Apr 20, 2020 at 2:10 PM Krisztián Szűcs > > wrote: > > > 1. [x] rebase > > 2. [

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-20 Thread Andy Grove
You should have an invite for the arrow-flight crate. Please check https://crates.io/me/pending-invites On Mon, Apr 20, 2020 at 2:10 PM Krisztián Szűcs wrote: > 1. [x] rebase > 2. [x] upload source > 3. [x] upload binaries > 4. [x] update website > 5. [x] upload ruby gems > 6. [x] upload j

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-20 Thread Krisztián Szűcs
1. [x] rebase 2. [x] upload source 3. [x] upload binaries 4. [x] update website 5. [x] upload ruby gems 6. [x] upload js packages 8. [ ] upload C# crates 9. [ ] upload rust crates 10. [ ] update conda recipes 11. [-] upload wheels to pypi 12. [ ] update homebrew packages 13. [-] update mave

[jira] [Created] (ARROW-8532) [C++][CSV] Add support for sentinel values.

2020-04-20 Thread Ravil Bikbulatov (Jira)
Ravil Bikbulatov created ARROW-8532: --- Summary: [C++][CSV] Add support for sentinel values. Key: ARROW-8532 URL: https://issues.apache.org/jira/browse/ARROW-8532 Project: Apache Arrow Issue

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-20 Thread Krisztián Szűcs
The VOTE carries with 7 binding +1 votes and 1 non-binding +1 vote. I'm starting the post-release tasks. I'm going to send updates to this thread to coordinate if anyone wants to help. On Mon, Apr 20, 2020 at 3:25 PM Rok Mihevc wrote: > > +1 (non-binding) > > Tested on Ubuntu 18.04. C++ and Pyth

[C++] Big-endian support

2020-04-20 Thread Antoine Pitrou
Hello, Recently some issues have been opened for big-endian support (i.e. support for big-endian *hosts*), and a couple patches submitted, thanks to Kazuaki Ishizaki. See e.g.: https://issues.apache.org/jira/browse/ARROW-8457 https://issues.apache.org/jira/browse/ARROW-8467 https://issues.apac

[NIGHTLY] Arrow Build Report for Job nightly-2020-04-20-1

2020-04-20 Thread Crossbow
Arrow Build Report for Job nightly-2020-04-20-1 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-20-1 Failed Tasks: - gandiva-jar-osx: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-20-1-travis-gandiva-jar-osx - homebrew-cpp:

[jira] [Created] (ARROW-8531) [C++] Deprecate ARROW_USE_SIMD CMake option

2020-04-20 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-8531: --- Summary: [C++] Deprecate ARROW_USE_SIMD CMake option Key: ARROW-8531 URL: https://issues.apache.org/jira/browse/ARROW-8531 Project: Apache Arrow Issue Type: Im

[jira] [Created] (ARROW-8530) [Ruby] Return dictionary value by DictionaryArray#[]

2020-04-20 Thread Yosuke Shiro (Jira)
Yosuke Shiro created ARROW-8530: --- Summary: [Ruby] Return dictionary value by DictionaryArray#[] Key: ARROW-8530 URL: https://issues.apache.org/jira/browse/ARROW-8530 Project: Apache Arrow Issue

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-20 Thread Rok Mihevc
+1 (non-binding) Tested on Ubuntu 18.04. C++ and Python builds / tests went fine. Rok On Mon, Apr 20, 2020 at 11:57 AM Krisztián Szűcs wrote: > So far we're having 7 +1 binding votes. I'm planning to close the > vote later today, I anyone else would like to verify the release > please do it to

Re: 0.17 release blog post: help needed

2020-04-20 Thread Fan Liya
I have added some Java items. Best, Liya Fan On Mon, Apr 20, 2020 at 10:49 AM Kenta Murata wrote: > I've edited Ruby and C GLib parts. > Kou and Shiro will check them later. > > 2020年4月20日(月) 11:09 Wes McKinney : > > > > I made a pass through the changelog and added a bunch of TODOs related > >

[jira] [Created] (ARROW-8529) [C++] Fix usage of NextCounts() in GetBatchWithDict[Spaced]

2020-04-20 Thread Antoine Pitrou (Jira)
Antoine Pitrou created ARROW-8529: - Summary: [C++] Fix usage of NextCounts() in GetBatchWithDict[Spaced] Key: ARROW-8529 URL: https://issues.apache.org/jira/browse/ARROW-8529 Project: Apache Arrow

[jira] [Created] (ARROW-8528) [CI][NIGHTLY:gandiva-jar-osx] gandiva osx build is failing

2020-04-20 Thread Prudhvi Porandla (Jira)
Prudhvi Porandla created ARROW-8528: --- Summary: [CI][NIGHTLY:gandiva-jar-osx] gandiva osx build is failing Key: ARROW-8528 URL: https://issues.apache.org/jira/browse/ARROW-8528 Project: Apache Arrow

[Java] Memory Allocation Tips

2020-04-20 Thread Razvan Chitu
Hi, Does the Arrow community have any tips / recommendations / best practices on how to manage Arrow memory in Java? Is there a way to rely on the GC exclusively (i.e. is there support for heap-only allocation)? Best, Razvan

[jira] [Created] (ARROW-8527) [C++][CSV] Add support for ReadOptions::skip_rows >= block_size

2020-04-20 Thread Ravil Bikbulatov (Jira)
Ravil Bikbulatov created ARROW-8527: --- Summary: [C++][CSV] Add support for ReadOptions::skip_rows >= block_size Key: ARROW-8527 URL: https://issues.apache.org/jira/browse/ARROW-8527 Project: Apache A

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-20 Thread Krisztián Szűcs
So far we're having 7 +1 binding votes. I'm planning to close the vote later today, I anyone else would like to verify the release please do it today. Thanks, Krisztian On Sun, Apr 19, 2020 at 10:59 PM Wes McKinney wrote: > > +1 (binding) > > I was able to run the verification of source and whee

[jira] [Created] (ARROW-8526) [Python] Fix non-deterministic row order failure in dataset tests

2020-04-20 Thread Krisztian Szucs (Jira)
Krisztian Szucs created ARROW-8526: -- Summary: [Python] Fix non-deterministic row order failure in dataset tests Key: ARROW-8526 URL: https://issues.apache.org/jira/browse/ARROW-8526 Project: Apache A

[jira] [Created] (ARROW-8525) [Crossbow] Add support for passing parameters to jobs from CLI

2020-04-20 Thread Krisztian Szucs (Jira)
Krisztian Szucs created ARROW-8525: -- Summary: [Crossbow] Add support for passing parameters to jobs from CLI Key: ARROW-8525 URL: https://issues.apache.org/jira/browse/ARROW-8525 Project: Apache Arro

Re: [NIGHTLY] Arrow Build Report for Job nightly-2020-04-18-1

2020-04-20 Thread Krisztián Szűcs
By simply removing cached docker images (~13GB), haskell (~8.5GB), swap and apt cache I managed [1] to increase the available free space to 31GB from 8.3GB (out of 84GB). GHA ships with a lot of preinstalled software [2] which we don't need in docker based builds. [1] https://github.com/apache/arr

[jira] [Created] (ARROW-8524) [CI] Free up space on github actions

2020-04-20 Thread Krisztian Szucs (Jira)
Krisztian Szucs created ARROW-8524: -- Summary: [CI] Free up space on github actions Key: ARROW-8524 URL: https://issues.apache.org/jira/browse/ARROW-8524 Project: Apache Arrow Issue Type: Imp

[NIGHTLY] Arrow Build Report for Job nightly-2020-04-20-0

2020-04-20 Thread Crossbow
Arrow Build Report for Job nightly-2020-04-20-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-20-0 Failed Tasks: - centos-8-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-20-0-github-centos-8-amd64 - debian-buster-a