[jira] [Created] (ARROW-4756) [CI] document the procedure to update docker image for manylinux1 builds

2019-03-03 Thread Pindikura Ravindra (JIRA)
Pindikura Ravindra created ARROW-4756: - Summary: [CI] document the procedure to update docker image for manylinux1 builds Key: ARROW-4756 URL: https://issues.apache.org/jira/browse/ARROW-4756 Proj

[jira] [Created] (ARROW-4754) [CI][Java] Flaky TestAuth Flight test

2019-03-03 Thread Micah Kornfield (JIRA)
Micah Kornfield created ARROW-4754: -- Summary: [CI][Java] Flaky TestAuth Flight test Key: ARROW-4754 URL: https://issues.apache.org/jira/browse/ARROW-4754 Project: Apache Arrow Issue Type: Im

[jira] [Created] (ARROW-4755) [Java] Flight tests should use randomized server ports

2019-03-03 Thread Micah Kornfield (JIRA)
Micah Kornfield created ARROW-4755: -- Summary: [Java] Flight tests should use randomized server ports Key: ARROW-4755 URL: https://issues.apache.org/jira/browse/ARROW-4755 Project: Apache Arrow

[Rust] [DataFusion] Preferences on futures / threading crates?

2019-03-03 Thread Andy Grove
I have been working on a PoC of parallel query execution and it is working well, and I am now starting to create PRs for the various refactors necessary for this in DataFusion. I haven't been following the async/await and futures/tokio developments lately but for the PoC I used tokio-threadpool wh

[jira] [Created] (ARROW-4753) Support optionally, and as an extension, an encoding layout for text-optimized data structures

2019-03-03 Thread Edmon Begoli (JIRA)
Edmon Begoli created ARROW-4753: --- Summary: Support optionally, and as an extension, an encoding layout for text-optimized data structures Key: ARROW-4753 URL: https://issues.apache.org/jira/browse/ARROW-4753

Re: Text data structures-optimized layout in Arrow

2019-03-03 Thread Wes McKinney
Great, yes, please go ahead and open JIRA issues. That would be the appropriate place to make the development work more clearly specified Thanks On Sun, Mar 3, 2019 at 7:36 PM Edmon Begoli wrote: > > Thanks, Wes. > > _contrib_ could indeed be a good option for this. > > Unless the community obje

Re: [C++] BUILD_WARNING_LEVEL=EVERYTHING?

2019-03-03 Thread Wes McKinney
No opposition from me On Sun, Mar 3, 2019 at 10:02 PM Micah Kornfield wrote: > > I'm ok with that. I think some of the conversion warnings might be useful > (I know I've had bugs in other code that would have been caught with > them). Would people be opposed if I tried to go through and cleanup

Re: [C++] BUILD_WARNING_LEVEL=EVERYTHING?

2019-03-03 Thread Micah Kornfield
I'm ok with that. I think some of the conversion warnings might be useful (I know I've had bugs in other code that would have been caught with them). Would people be opposed if I tried to go through and cleanup the EVERYTHING warnings even if more might creep in? Thanks, Micah On Sun, Mar 3, 20

[jira] [Created] (ARROW-4752) [Rust] Add explicit SIMD vectorization for the divide kernel

2019-03-03 Thread Paddy Horan (JIRA)
Paddy Horan created ARROW-4752: -- Summary: [Rust] Add explicit SIMD vectorization for the divide kernel Key: ARROW-4752 URL: https://issues.apache.org/jira/browse/ARROW-4752 Project: Apache Arrow

Re: Text data structures-optimized layout in Arrow

2019-03-03 Thread Edmon Begoli
Thanks, Wes. _contrib_ could indeed be a good option for this. Unless the community objects, I suggest that I create a JIRA issue for this. We could use that issue for tracking and documentation of the intended purpose, design thinking, and also add as many details as possible. My team and I hav

Re: Text data structures-optimized layout in Arrow

2019-03-03 Thread Wes McKinney
hi Edmon, Since we've just added a C++ API for "extension types" this might be a place to try these out to define custom container types for text: https://github.com/apache/arrow/commit/a79cc809883192417920b501e41a0e8b63cd0ad1 I don't have a sense of where such code should go in the project and

Re: [C++] BUILD_WARNING_LEVEL=EVERYTHING?

2019-03-03 Thread Wes McKinney
I'm of the same mind as Antoine on this. I think it's useful to look at the EVERYTHING warnings periodically, but it is enough effort to keep things simultaneously building cleanly with gcc, clang, and MSVC, that I would prefer to maintain the status quo until it can be demonstrated to be a problem

Re: Boost and manylinux CI builds

2019-03-03 Thread Wes McKinney
hi Ravindra, Can we document this (just by copy-pasting what you wrote) on the wiki or someplace for future work that may touch the manylinux package builds? This might be a bit more discoverable than going through the email logs Thanks! Wes On Fri, Mar 1, 2019 at 9:58 PM Ravindra Pindikura wro

Re: Arrow Flight protocol/API questions

2019-03-03 Thread Wes McKinney
On Tue, Feb 12, 2019 at 2:44 PM David Ming Li wrote: > > Hi all, > > > > We've been evaluating Flight for our use, and we're wondering if the protocol > is still open to extensions, as having a few application-defined metadata > fields would help our use cases a lot. > > > > (Apologies if this i

Re: Flaky Travis CI builds on master

2019-03-03 Thread Kouhei Sutou
Hi, >> * Seemingly a GLib Plasma OOM >> https://travis-ci.org/apache/arrow/jobs/498906118#L3689 > > I take this: > https://issues.apache.org/jira/browse/ARROW-4704 I've confirmed that this has been fixed by https://github.com/apache/arrow/commit/95b33efbeae444bbdc7ec932e3ba0f8a319ec2d1 plasma_s

[jira] [Created] (ARROW-4751) [C++] Add pkg-config to conda_env_cpp.yml

2019-03-03 Thread Uwe L. Korn (JIRA)
Uwe L. Korn created ARROW-4751: -- Summary: [C++] Add pkg-config to conda_env_cpp.yml Key: ARROW-4751 URL: https://issues.apache.org/jira/browse/ARROW-4751 Project: Apache Arrow Issue Type: Task

[jira] [Created] (ARROW-4750) [C++] RapidJSON triggers Wclass-memaccess on GCC 8+

2019-03-03 Thread Uwe L. Korn (JIRA)
Uwe L. Korn created ARROW-4750: -- Summary: [C++] RapidJSON triggers Wclass-memaccess on GCC 8+ Key: ARROW-4750 URL: https://issues.apache.org/jira/browse/ARROW-4750 Project: Apache Arrow Issue Ty

[jira] [Created] (ARROW-4749) [Rust] RecordBatch::new() should return result instead of panicking

2019-03-03 Thread Andy Grove (JIRA)
Andy Grove created ARROW-4749: - Summary: [Rust] RecordBatch::new() should return result instead of panicking Key: ARROW-4749 URL: https://issues.apache.org/jira/browse/ARROW-4749 Project: Apache Arrow

[jira] [Created] (ARROW-4748) [Rust] [DataFusion] GROUP BY performance could be optimized

2019-03-03 Thread Andy Grove (JIRA)
Andy Grove created ARROW-4748: - Summary: [Rust] [DataFusion] GROUP BY performance could be optimized Key: ARROW-4748 URL: https://issues.apache.org/jira/browse/ARROW-4748 Project: Apache Arrow I

[jira] [Created] (ARROW-4747) [C++/PyPy] Add docker image to test against PyPy nightlies

2019-03-03 Thread Uwe L. Korn (JIRA)
Uwe L. Korn created ARROW-4747: -- Summary: [C++/PyPy] Add docker image to test against PyPy nightlies Key: ARROW-4747 URL: https://issues.apache.org/jira/browse/ARROW-4747 Project: Apache Arrow I

[jira] [Created] (ARROW-4746) [C++/Python] PyDataTime_Date wrongly casted to PyDataTime_DateTime

2019-03-03 Thread Uwe L. Korn (JIRA)
Uwe L. Korn created ARROW-4746: -- Summary: [C++/Python] PyDataTime_Date wrongly casted to PyDataTime_DateTime Key: ARROW-4746 URL: https://issues.apache.org/jira/browse/ARROW-4746 Project: Apache Arrow

Re: java/format: Windows build fails due to no flatc binary available

2019-03-03 Thread Sebastian Piu
Sounds good, I've submitted a PR on the upstream maven artifact Arrow currently uses as that seems to be the easiest way to solve it and wouldn't require changes to Arrow's codebase, if that doesn't go through I'll push a patch to the readme or pom+ant task On Sun, 3 Mar 2019 at 03:58, Micah Kornf

Re: [C++] BUILD_WARNING_LEVEL=EVERYTHING?

2019-03-03 Thread Antoine Pitrou
Hmm... There are enough warnings that need pampering in the default settings that I don't think we want to go the full length of enabling all warnings. Sometimes it's a PITA to get code to compile cleanly on all platforms. If compiler writers had a more reasonable judgement when it comes to des