[NOTICE] Mandatory migration of git repos to gitbox.apache.org - three weeks left!

2019-01-14 Thread Apache Infrastructure Team
Hello, arrow folks. As stated earlier in 2018, and reiterated two weeks ago, all git repositories must be migrated from the git-wip-us.apache.org URL to gitbox.apache.org, as the old service is being decommissioned. Your project is receiving this email because you still have repositories on

Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW

2019-01-14 Thread Wes McKinney
I'm working on getting ARROW-4258 merged right now. ARROW-4260 also should be fixed before another RC is cut https://issues.apache.org/jira/browse/ARROW-4260 On Mon, Jan 14, 2019 at 11:36 PM Kouhei Sutou wrote: > > Hi, > > Does your verify-release-candidate.sh include >

Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW

2019-01-14 Thread Kouhei Sutou
Hi, Does your verify-release-candidate.sh include https://github.com/apache/arrow/commit/be663c14637b2bdfef935946b6e91b6317219332 ? arrow-json-integration-test needs -DARROW_BUILD_TESTS=ON. Thanks, -- kou In "Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW" on Tue, 15 Jan 2019

Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW

2019-01-14 Thread Wes McKinney
hi Andy, We'd need to see a more complete log of the verification script run to see what's going on. If I had to venture a guess if you are working on EC2 you might be running out of /tmp space. The build produces well over a gigabyte of artifacts. If this is the case, we might add an

Re: [C++] Adding gmock to the toolchain?

2019-01-14 Thread Wes McKinney
Seems fine to me. We never decided against building it but never had a need for it (or perhaps there are some tests we might have written that could have made use of mocks) On Mon, Jan 14, 2019 at 10:42 PM Micah Kornfield wrote: > > Would people be OK if I submitted a PR for this? It is already

[C++] Adding gmock to the toolchain?

2019-01-14 Thread Micah Kornfield
Would people be OK if I submitted a PR for this? It is already packaged with googletest, so I don't think it would require downloading additional dependencies. Pros: - Don't need to create our own mocks (not terribly hard but a little boring). It would be useful to have mock Kernel

[jira] [Created] (ARROW-4264) Convert DCHECKs in that check compute/* input parameters to error statuses

2019-01-14 Thread Micah Kornfield (JIRA)
Micah Kornfield created ARROW-4264: -- Summary: Convert DCHECKs in that check compute/* input parameters to error statuses Key: ARROW-4264 URL: https://issues.apache.org/jira/browse/ARROW-4264

Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW

2019-01-14 Thread Andrew Palumbo
After following all setup and install steps from: https://cwiki.apache.org/confluence/display/ARROW/How+to+Verify+Release+Candidates apalumbo@ip-xxx-xxx-xxx-xxx:~/arrow/dev/release$ verify-release-candidate.sh source 0.12.0 2 fails with this stack trace:

Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW

2019-01-14 Thread Andrew Palumbo
Hello, I've just run the verify scipt on the 0.12.0 tag after configuring an Ubuntu machine per the instructions. I'm crashing at "c++ producing, c++ consuming". With a no such file or directory... Arrow-json-integration-test. When I was trying to build straight from that submodule there was

[jira] [Created] (ARROW-4263) [Rust] Donate DataFusion

2019-01-14 Thread Andy Grove (JIRA)
Andy Grove created ARROW-4263: - Summary: [Rust] Donate DataFusion Key: ARROW-4263 URL: https://issues.apache.org/jira/browse/ARROW-4263 Project: Apache Arrow Issue Type: Improvement

Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW

2019-01-14 Thread Kouhei Sutou
Hi, > There is a problem in Plasma + Python + CUDA: > > https://issues.apache.org/jira/browse/ARROW-4246 I've created a pull request for this: https://github.com/apache/arrow/pull/3398 Thanks, -- kou In <20190113.124329.44772517181197518@clear-code.com> "Re: [VOTE] Release Apache

Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW

2019-01-14 Thread Kouhei Sutou
Hi, I've created a pull request for verifying binaries on Bintray: https://github.com/apache/arrow/pull/3397 Thanks, -- kou In <0a69afdb-2c15-2d7d-1d56-5333bf12b...@python.org> "Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW" on Mon, 14 Jan 2019 09:34:06 +0100, Antoine Pitrou

Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW

2019-01-14 Thread Wes McKinney
Does anything need to change in the verification scripts to run the JavaScript parts of the integration tests? The build was passing when I merged the refactor. We aren't pushing any npm artifacts from this release so as long as the tests run it is OK I would be OK with merging the patch for

Re: [Rust] [DISCUSS] Donate DataFusion to Arrow project

2019-01-14 Thread Andy Grove
Wes, Thanks. Yes, I'd like to proceed with the vote as soon as you are ready. I don't think I need much time at all at this point to prepare the merge. I already have a branch of DataFusion that is building against the latest Arrow code, so it's really just a case of updating source files with

Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW

2019-01-14 Thread Krisztián Szűcs
I'm fine with cutting an RC3. ARROW-4258 seems pretty bad. If We decide to create a new RC how should We deal with the recent JS refactoring [1]? On OSX the verification worked well though, except a plasma issue. Uwe, if You have time please verify it on OSX too. [1]

[jira] [Created] (ARROW-4262) [Website] Blog post to give preview into using R and Arrow with Apache Spark

2019-01-14 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-4262: --- Summary: [Website] Blog post to give preview into using R and Arrow with Apache Spark Key: ARROW-4262 URL: https://issues.apache.org/jira/browse/ARROW-4262 Project:

Re: [Rust] [DISCUSS] Donate DataFusion to Arrow project

2019-01-14 Thread Wes McKinney
Getting the 0.12 release out is my priority right now, but it seems that there are no major objections to this code donation. @Andy -- I can kick off the vote to accept the code donation in the next few days if you'd like to proceed with that. How much time do you think it would take for you to

[jira] [Created] (ARROW-4261) [C++] CMake paths for IPC, Flight, Thrift, and Plasma don't support using Arrow as a subproject

2019-01-14 Thread Michael Vilim (JIRA)
Michael Vilim created ARROW-4261: Summary: [C++] CMake paths for IPC, Flight, Thrift, and Plasma don't support using Arrow as a subproject Key: ARROW-4261 URL: https://issues.apache.org/jira/browse/ARROW-4261

Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW

2019-01-14 Thread Antoine Pitrou
Ok, it worked, except that I got the classical test_plasma_list() failure. Regards Antoine. Le 14/01/2019 à 17:50, Wes McKinney a écrit : > How about using the conda-forge package? > > conda create -y -p $HOME/release-0.12-toolchain brotli -c conda-forge > export

[jira] [Created] (ARROW-4260) [Python] test_serialize_deserialize_pandas is failing on OSX with Xcode 6.4

2019-01-14 Thread Hatem Helal (JIRA)
Hatem Helal created ARROW-4260: -- Summary: [Python] test_serialize_deserialize_pandas is failing on OSX with Xcode 6.4 Key: ARROW-4260 URL: https://issues.apache.org/jira/browse/ARROW-4260 Project:

Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW

2019-01-14 Thread Wes McKinney
In light of the severity of ARROW-4258 (https://github.com/apache/arrow/pull/3395) and some of the other rough edges in this RC, I would suggest making some fixes today and cutting an RC3. I'm revising my vote to -0 @Krisztian -- you are the RM, so it is up to you whether you want to cancel the

Re: Reporting Travis CI Failures

2019-01-14 Thread Wes McKinney
That can certainly be reported, particularly if you see it more than once On Mon, Jan 14, 2019 at 11:18 AM Hatem Helal wrote: > > Hi Wes, > > Thanks for that - makes sense to me. > > How about this Xcode 6.4 failure? > > https://travis-ci.org/apache/arrow/jobs/479378190#L2427 > > I was hesitant

Re: Reporting Travis CI Failures

2019-01-14 Thread Hatem Helal
Hi Wes, Thanks for that - makes sense to me. How about this Xcode 6.4 failure? https://travis-ci.org/apache/arrow/jobs/479378190#L2427 I was hesitant to report this as the warning at the top of the page states that Xcode 6.4 is due to be deprecated this month. Thanks, Hatem On 1/14/19,

Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW

2019-01-14 Thread Wes McKinney
How about using the conda-forge package? conda create -y -p $HOME/release-0.12-toolchain brotli -c conda-forge export BROTLI_HOME=$HOME/release-0.12-toolchain On Mon, Jan 14, 2019 at 10:46 AM Antoine Pitrou wrote: > > > Unfortunately, Ubuntu only provides shared library versions of > Brotli

Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW

2019-01-14 Thread Antoine Pitrou
Unfortunately, Ubuntu only provides shared library versions of Brotli but we're looking for a statically-linked brotli: make[2]: *** No rule to make target '/usr/lib/x86_64-linux-gnu/libbrotlidec.a', needed by 'release/libarrow.so.12.0.0'. Stop. CMakeFiles/Makefile2:2013: recipe for target

[jira] [Created] (ARROW-4259) [Plasma] CI failure in test_plasma_tf_op

2019-01-14 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-4259: --- Summary: [Plasma] CI failure in test_plasma_tf_op Key: ARROW-4259 URL: https://issues.apache.org/jira/browse/ARROW-4259 Project: Apache Arrow Issue Type: Bug

Re: Reporting Travis CI Failures

2019-01-14 Thread Wes McKinney
hi Hatem, There are various causes of flakiness in builds (for example, conda package manager timeouts) -- we have done our best so far to manage the flakiness and will continue to need to manage it. In such cases, you can git commit --amend && git push -f to restart builds without having to make

Reporting Travis CI Failures

2019-01-14 Thread Hatem Helal
Hi everyone, I’ve had some trouble getting a clean CI build for a recent pull request and wanted to understand how this project manages CI failures. I don’t have much experience with Travis CI to draw on but my own approach so far has been to try and correlate failures on my own PR against

Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW

2019-01-14 Thread Wes McKinney
I forgot to add that I also hit known issue https://issues.apache.org/jira/browse/ARROW-3344 on Ubuntu 14.04 On Mon, Jan 14, 2019 at 9:46 AM Wes McKinney wrote: > > Can you set $BROTLI_HOME to unblock yourself? Since we have .deb > packages for Ubuntu Bionic it's an annoyance for source-builders

Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW

2019-01-14 Thread Wes McKinney
Can you set $BROTLI_HOME to unblock yourself? Since we have .deb packages for Ubuntu Bionic it's an annoyance for source-builders but not a deal breaker (IMHO) for the RC On Mon, Jan 14, 2019 at 3:24 AM Antoine Pitrou wrote: > > > Unfortunately I cannot validate the source release on Ubuntu

[jira] [Created] (ARROW-4258) [Python] Safe cast fails from numpy float64 array with nans to integer

2019-01-14 Thread Krisztian Szucs (JIRA)
Krisztian Szucs created ARROW-4258: -- Summary: [Python] Safe cast fails from numpy float64 array with nans to integer Key: ARROW-4258 URL: https://issues.apache.org/jira/browse/ARROW-4258 Project:

Re: pyarrow data type casting problem when safe=True

2019-01-14 Thread Krisztián Szűcs
Verified, issue: https://issues.apache.org/jira/browse/ARROW-4258 On Mon, Jan 14, 2019 at 12:31 AM Wes McKinney wrote: > This seems like a bug to me; I would not expect this to fail. It's too > bad it didn't get fixed in time for 0.12 > > On Thu, Jan 10, 2019 at 4:20 PM Bryan Cutler wrote: > >

Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW

2019-01-14 Thread Antoine Pitrou
Unfortunately I cannot validate the source release on Ubuntu 18.04, as brotli fails compiling with a internal compiler error (this is gcc 7.3.0): /tmp/arrow-0.12.0.BZuZO/apache-arrow-0.12.0/cpp/build/brotli_ep-prefix/src/brotli_ep/enc/compress_fragment.c: In function

Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW

2019-01-14 Thread Antoine Pitrou
Apparently the binaries verification script attempts to download from dist.apache.org? $ dev/release/verify-release-candidate.sh binaries 0.12.0 rc2 [...] + wget -P apache-arrow-0.12.0-rcrc2 --quiet --no-host-directories --cut-dirs=5 --no-parent --reject 'index.html*' --recursive