Re: CMake refactor Heads-up

2019-03-18 Thread Uwe L. Korn
I would advise against mixing these environments. Either use only packages from defaults or from conda-forge as things like boost from defaults and boost from conda-forge can be installed at the same time but then lead to segfaults. Uwe On Mon, Mar 18, 2019, at 2:10 PM, Antoine Pitrou wrote: >

Re: CMake refactor Heads-up

2019-03-18 Thread Antoine Pitrou
Well, in the meantime I can just use the conda-forge packages. (though there are regular issues when updating packages where conda switches back and forth from Anaconda and conda-forge packages) Regards Antoine. Le 18/03/2019 à 13:59, Uwe L. Korn a écrit : > Hello Antoine, > > you're running

Re: CMake refactor Heads-up

2019-03-18 Thread Uwe L. Korn
Hello Antoine, you're running into https://github.com/ContinuumIO/anaconda-issues/issues/10731 I would rather have Anaconda fix this but we can also add alternative detection for this. I've opened https://issues.apache.org/jira/browse/ARROW-4946, I can then look into this the next hours/tomorr

Re: CMake refactor Heads-up

2019-03-18 Thread Antoine Pitrou
I see, thanks. Regards Antoine. Le 18/03/2019 à 13:57, Wes McKinney a écrit : > See https://issues.apache.org/jira/browse/ARROW-4879 > > CMake support for Flatbuffers was only very recently made available in > the 1.10.1 conda-forge package (see [1]); it will have to be fixed in > Anaconda's

Re: CMake refactor Heads-up

2019-03-18 Thread Wes McKinney
See https://issues.apache.org/jira/browse/ARROW-4879 CMake support for Flatbuffers was only very recently made available in the 1.10.1 conda-forge package (see [1]); it will have to be fixed in Anaconda's repo. In the meantime if you want to use Anaconda packages instead of conda-forge (not recomm

Re: CMake refactor Heads-up

2019-03-18 Thread Antoine Pitrou
Ah, apparently I can do it through `-Ddouble-conversion_SOURCE=BUNDLED`. Now there's another issue: the CMake configuration fails to find flatbuffers, even though I have flatbuffers 1.7.1 installed from Anaconda. CMake Error at cmake_modules/ThirdpartyToolchain.cmake:152 (find_package): By n

Re: CMake refactor Heads-up

2019-03-18 Thread Wes McKinney
hi Antoine, I have addressed this issue in the documentation patch I just did https://github.com/apache/arrow/blob/master/docs/source/developers/cpp.rst#individual-dependency-resolution passing -Ddouble-conversion_SOURCE=BUNDLED should do the trick - Wes On Mon, Mar 18, 2019 at 7:51 AM Antoi

Re: CMake refactor Heads-up

2019-03-18 Thread Antoine Pitrou
Ok, so I have a problem. I had the following line: export DOUBLE_CONVERSION_HOME= which was used to force double-conversion to be built from source despite other dependencies being taken from the Conda environment. Now it doesn't work anymore, and I haven't found how to emulate it. Regards

Re: CMake refactor Heads-up

2019-03-17 Thread Suvayu Ali
On Sat, Mar 16, 2019 at 04:31:32PM +0100, Uwe L. Korn wrote: > > > 4. AFAIU, the pyarrow build expects the libraries in > > $CMAKE_INSTALL_PREFIX/lib. This will never be accepted by a distro. I do > > realise this one is probably hard to resolve, given how the builds are > > setup at the mome

Re: CMake refactor Heads-up

2019-03-16 Thread Suvayu Ali
Hi Uwe, On Sat, Mar 16, 2019 at 04:31:32PM +0100, Uwe L. Korn wrote: > > > 2. I don't know if this is intentional, but jemalloc and rapidjson aren't > > detected on my system. > > > > Not sure if it is detected or not as this is missing from the log above. That > log lists only the bundled ve

Re: CMake refactor Heads-up

2019-03-16 Thread Uwe L. Korn
Hello, > Am 16.03.2019 um 14:30 schrieb Suvayu Ali : > > Hello Uwe, > >> On Fri, Mar 15, 2019 at 10:38:32AM -0400, Uwe L. Korn wrote: >> >> we have merged the CMake refactor yesterday >> https://github.com/apache/arrow/pull/3688 and this means that the build >> system behaves a bit different.

Re: CMake refactor Heads-up

2019-03-16 Thread Suvayu Ali
Hello Uwe, On Fri, Mar 15, 2019 at 10:38:32AM -0400, Uwe L. Korn wrote: > > we have merged the CMake refactor yesterday > https://github.com/apache/arrow/pull/3688 and this means that the build > system behaves a bit different. The main differences are: > A few more comments: 1. There was an e

Re: CMake refactor Heads-up

2019-03-16 Thread Suvayu Ali
On Sat, Mar 16, 2019 at 04:35:32AM -0400, Uwe L. Korn wrote: > Hello Suvayu, > > On Fri, Mar 15, 2019, at 10:08 PM, Suvayu Ali wrote: > > Is there a recommended way to choose between Python versions? Fedora > > repos often provide several Python versions, e.g. on F28 I have 2.7 > > (default), 3

Re: CMake refactor Heads-up

2019-03-16 Thread Uwe L. Korn
Hello Suvayu, On Fri, Mar 15, 2019, at 10:08 PM, Suvayu Ali wrote: > Is there a recommended way to choose between Python versions? Fedora > repos often provide several Python versions, e.g. on F28 I have 2.7 > (default), 3.6, and 3.7. The correct Python version was detected for > me with: >

Re: CMake refactor Heads-up

2019-03-15 Thread Suvayu Ali
On Sat, Mar 16, 2019 at 11:03:27AM +0530, Ravindra Pindikura wrote: > On Sat, Mar 16, 2019 at 2:38 AM Suvayu Ali > wrote: > > > Secondly, I was trying to compile with Gandiva enabled. But it seems the > > LLVM requirement has gone up to 7.0 (available only on F29 onwards). I had > > managed to

Re: CMake refactor Heads-up

2019-03-15 Thread Ravindra Pindikura
On Sat, Mar 16, 2019 at 2:38 AM Suvayu Ali wrote: > Hi Uwe, > > On Fri, Mar 15, 2019 at 10:38:32AM -0400, Uwe L. Korn wrote: > > > > we have merged the CMake refactor yesterday > https://github.com/apache/arrow/pull/3688 and this means that the build > system behaves a bit different. The main dif

Re: CMake refactor Heads-up

2019-03-15 Thread Suvayu Ali
Hi Uwe, On Fri, Mar 15, 2019 at 10:38:32AM -0400, Uwe L. Korn wrote: > > we have merged the CMake refactor yesterday > https://github.com/apache/arrow/pull/3688 and this means that the build > system behaves a bit different. The main differences are: That's a lot of work! Thank you very much

Re: CMake refactor Heads-up

2019-03-15 Thread Philipp Moritz
Thanks Uwe for doing this! It's amazing how well the arrow build system is working given it supports so many different configurations (which makes Uwe's refactor all the more impressive). Big thanks for everybody helping to maintain it. On Fri, Mar 15, 2019 at 8:08 AM Wes McKinney wrote: > Than

Re: CMake refactor Heads-up

2019-03-15 Thread Wes McKinney
Thanks Uwe for seeing through this massive improvement to our C++ build system! This should help package managers build the project in precisely the way that's needed to make the Arrow C++ libraries available in more places. A couple notes: * I personally keep my C++ toolchain and conda environme

CMake refactor Heads-up

2019-03-15 Thread Uwe L. Korn
Hello fellow Arrow Devs, we have merged the CMake refactor yesterday https://github.com/apache/arrow/pull/3688 and this means that the build system behaves a bit different. The main differences are: * If you're in a conda environment, we automatically detect this using the environment variable