[jira] [Created] (ARROW-2743) [Java] Travis CI test scripts did not catch POM file bug fixed in ARROW-2727

2018-06-25 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-2743: --- Summary: [Java] Travis CI test scripts did not catch POM file bug fixed in ARROW-2727 Key: ARROW-2743 URL: https://issues.apache.org/jira/browse/ARROW-2743 Project:

Re: Arrow and oamap [was: Re: Gandiva Initiative]

2018-06-25 Thread Jim Pivarski
Thanks, I'll let you know if I run into issues with Gandiva, and I'll pass that on to my contact with ALICE. I haven't caught up to the earlier part of this conversation yet, but the main thing that was lacking for my use case was handling arbitrary data structures and arbitrary code structures.

Re: Arrow and oamap [was: Re: Gandiva Initiative]

2018-06-25 Thread Wes McKinney
Thanks Jim, that's helpful. In the long run, we'd like to make sure that the Arrow libraries can serve as a robust dependency for computational systems like OAMap. It doesn't strike me that OAMap is a library that could be used within the Arrow project, though there are some things which could be

[jira] [Created] (ARROW-2742) [Python] Allow Table.from_batches to use Iterator of ArrowRecordBatches

2018-06-25 Thread Bryan Cutler (JIRA)
Bryan Cutler created ARROW-2742: --- Summary: [Python] Allow Table.from_batches to use Iterator of ArrowRecordBatches Key: ARROW-2742 URL: https://issues.apache.org/jira/browse/ARROW-2742 Project: Apache

Re: Running Apache Arrow on Visual Studio

2018-06-25 Thread Wes McKinney
I am not an expert in Visual Studio -- the error is caused by a missing library dependency (which I think can be fixed by appropriate linker configuration https://msdn.microsoft.com/en-us/library/ba1z7822.aspx). @Max, do you have any advice? We are using these libraries with Visual Studio to

Re: Arrow and oamap [was: Re: Gandiva Initiative]

2018-06-25 Thread Jim Pivarski
What Martin said about OAMap and ROOT is true: there's no dependence, ROOT and Arrow are both backends. What Wes said about embeddability is also right: OAMap is pure Python+Numpy and would be hard to use within a C++ framework (or Java). This has already been an issue with a potential user in

Re: Arrow and oamap [was: Re: Gandiva Initiative]

2018-06-25 Thread Martin Durant
Let me just quickly correct a couple of point, for clarity. The following module https://github.com/diana-hep/oamap/blob/master/oamap/backend/arrow.py is a proof-of-concept of oamap running directly on arrow memory - this is the original reason I raised the topic here. There are also POCs

Re: Building against arrow static library

2018-06-25 Thread Wes McKinney
hi Praveen, Are /usr/include or /usr/local/include getting added to your build paths (this could happen if you have LLVM or some other library in your system paths)? Unfortunately, when you're using conda libraries or an external toolchain at all, you have to be really strict about toolchain

Re: Minimal GCC version

2018-06-25 Thread Wes McKinney
hi Dimitri, to add some extra color: you're right that Ubuntu 14.04 LTS is gcc 4.8, and the minimum standards-compliant C++11 gcc. I think it would be a good idea to support >= 4.8 for a while longer yet so 14.04 users can compile and use Arrow out of the box. There are some drawbacks, like

Arrow and oamap [was: Re: Gandiva Initiative]

2018-06-25 Thread Wes McKinney
hi Martin, These projects are very different. Many analytic databases feature code generation (recently a lot of these use LLVM -- see Hyper, Apache Impala, and others) on the hot paths for function evaluation (e.g. for evaluating the expressions in the SELECT part or the WHERE part) -- the

[jira] [Created] (ARROW-2741) [Python] pa.array from np.datetime[D] and type=pa.date64 produces invalid results

2018-06-25 Thread Uwe L. Korn (JIRA)
Uwe L. Korn created ARROW-2741: -- Summary: [Python] pa.array from np.datetime[D] and type=pa.date64 produces invalid results Key: ARROW-2741 URL: https://issues.apache.org/jira/browse/ARROW-2741 Project:

Re: Gandiva Initiative

2018-06-25 Thread Martin Durant
I am a little surprised by the very positive reception to Gandiva (which doubtless is very useful - I know very little about it) versus when I brought up the prospect of using oamap ( https://github.com/diana-hep/oamap ) on this mailing list. oamap uses numba to compile *python* functions at

Re: Building against arrow static library

2018-06-25 Thread Praveen Kumar
Tried that did not help. I also tried to link in the boost libraries myself since it was not clear if arrow was including the transitive dependencies from the build script. That does not help as well. On Mon, Jun 25, 2018 at 7:55 PM, Dimitri Vorona < alen...@googlemail.com.invalid> wrote: > Hi,

Re: Minimal GCC version

2018-06-25 Thread Anthony Scopatz
Hi All, On behalf of conda-forge here, yes, we are in the process of transitioning to newer compilers. If you have any questions or concerns, feel free to reach out to me. Be Well Anthony On Mon, Jun 25, 2018 at 12:33 PM Dimitri Vorona wrote: > Hi Uwe, > > great, thanks! > > Cheers. > > On

Re: Minimal GCC version

2018-06-25 Thread Dimitri Vorona
Hi Uwe, great, thanks! Cheers. On Mon, Jun 25, 2018 at 6:29 PM Uwe L. Korn wrote: > Hello Dimitri, > > the minimal GCC version is currently driven by the minimal requirements of > conda-forge and the manylinux1 platform tag for Python Wheels. conda-forge > is expected to migrate soon to GCC

Re: Minimal GCC version

2018-06-25 Thread Uwe L. Korn
Hello Dimitri, the minimal GCC version is currently driven by the minimal requirements of conda-forge and the manylinux1 platform tag for Python Wheels. conda-forge is expected to migrate soon to GCC 7+. The manylinux standard will also publish a manylinux2010 tag that will upgrade their

Minimal GCC version

2018-06-25 Thread Dimitri Vorona
Hi, I wondered what is the decision process behind the minimal supported GCC version (currently 4.8)? Is it something like "the default GCC in the oldest supported LTS Ubuntu"? Or maybe there are some ASF guidelines? Cheers, Dimitri.

[jira] [Created] (ARROW-2740) [Python] Add address property to Buffer

2018-06-25 Thread Antoine Pitrou (JIRA)
Antoine Pitrou created ARROW-2740: - Summary: [Python] Add address property to Buffer Key: ARROW-2740 URL: https://issues.apache.org/jira/browse/ARROW-2740 Project: Apache Arrow Issue Type:

Re: Building against arrow static library

2018-06-25 Thread Dimitri Vorona
Hi, I'd this a similar issue some time ago, and the solution was building after a clean checkout, which I interpreted as some kind of caching issue. Generally, I've found that starting with a clean checkout and following the steps from [0] never failed for me. Hope that helps! Cheers, Dimitri.

Building against arrow static library

2018-06-25 Thread Praveen Kumar
Hi Folks, I am a newbie to cpp build/packaging. I need some help on building against the arrow static library. I tried the following 1. Using both arrow and boost from conda (latest versions 0.9 and 1.67). 2. Building arrow from source and boost from conda. 3. Built both from source. But i am

[jira] [Created] (ARROW-2739) [GLib] Use G_DECLARE_DERIVABLE_TYPE for GArrowDecimalDataType and GArrowDecimal128ArrayBuilder

2018-06-25 Thread yosuke shiro (JIRA)
yosuke shiro created ARROW-2739: --- Summary: [GLib] Use G_DECLARE_DERIVABLE_TYPE for GArrowDecimalDataType and GArrowDecimal128ArrayBuilder Key: ARROW-2739 URL: https://issues.apache.org/jira/browse/ARROW-2739

[jira] [Created] (ARROW-2738) Use Brewfile on installation process

2018-06-25 Thread Manabu Ejima (JIRA)
Manabu Ejima created ARROW-2738: --- Summary: Use Brewfile on installation process Key: ARROW-2738 URL: https://issues.apache.org/jira/browse/ARROW-2738 Project: Apache Arrow Issue Type:

Re: Running Apache Arrow on Visual Studio

2018-06-25 Thread diam5752
I add the .lib files , from ..\arrow-master\cpp\build\release\Release and \Debug and \MinSizeRel and RelWithDebInfo but still the same error. Am I missing something ?

Re: Gandiva Initiative

2018-06-25 Thread Praveen Kumar
Hi Everyone, I am Praveen, another engineer working on Gandiva. The interest and speed of engagement around this is great !!Excited to engage with you folks on this. Thx. On 2018/06/22 18:09:42, Julian Hyde wrote: > This is exciting. We have wanted to build an Arrow adapter in Calcite for >