Re: Trouble building on Mac OS Mojave
I was previously using virtualenv. On Sun, 1 Sep 2019 at 09:50, Chris Teoh wrote: > Ok thanks for your help. I will attempt to use the conda approach. > > On Sun, 1 Sep 2019 at 03:27, Wes McKinney wrote: > >> Hi Chris -- to follow up to your question also, we intend for contributors >> to build from source using the CMake configuration directly. We have found >> that conda is the easiest and most reliable way to manage the dependencies >> for C++ and Python contributors across all platforms. If you follow the >> Python contribution guide and something doesn't work please let us know so >> we can work with you to fix. >> >> Wes >> >> On Sat, Aug 31, 2019, 9:18 AM Uwe L. Korn wrote: >> >> > Hello Chris, >> > >> > as a contributor, it is often simpler to use conda to construct a local >> > development environment as outlined in >> > https://arrow.apache.org/docs/developers/python.html#using-conda >> > This is the typical environment most contributors work in. Even when not >> > using conda as a package/environment manager elsewhere, I would >> recommend >> > to use it to setup your Arrow build environment as this is the way most >> > developers do. Thus it will be easier to help you and this is the setup >> we >> > (try to) maintain best. >> > >> > Cheers >> > Uwe >> > >> > On Sat, Aug 31, 2019, at 3:48 PM, Chris Teoh wrote: >> > > Does this approach fit with potentially a contributor's workflow? I >> was >> > > looking into contributing though I'm unsure if I am doing it right. >> > > >> > > On Sat, 31 Aug 2019 at 22:22, Jeroen Ooms >> wrote: >> > > >> > > > On Sat, Aug 31, 2019 at 4:48 AM Chris Teoh >> > wrote: >> > > > > >> > > > > That being said, is there an easier way by using a Docker >> container I >> > > > could >> > > > > use to build this in? >> > > > >> > > > An easy way to install arrow on MacOS is using homebrew. To get a >> > > > precompiled version of the latest release: >> > > > >> > > > brew install apache-arrow >> > > > >> > > > Or to build the master branch from source: >> > > > >> > > >brew install apache-arrow --HEAD >> > > > >> > > > If you want to customize the configuration use "brew edit >> > > > apache-arrow" before building from source. >> > > > >> > > >> > > >> > > -- >> > > Chris >> > > >> > >> > > > -- > Chris > -- Chris
Re: Trouble building on Mac OS Mojave
Ok thanks for your help. I will attempt to use the conda approach. On Sun, 1 Sep 2019 at 03:27, Wes McKinney wrote: > Hi Chris -- to follow up to your question also, we intend for contributors > to build from source using the CMake configuration directly. We have found > that conda is the easiest and most reliable way to manage the dependencies > for C++ and Python contributors across all platforms. If you follow the > Python contribution guide and something doesn't work please let us know so > we can work with you to fix. > > Wes > > On Sat, Aug 31, 2019, 9:18 AM Uwe L. Korn wrote: > > > Hello Chris, > > > > as a contributor, it is often simpler to use conda to construct a local > > development environment as outlined in > > https://arrow.apache.org/docs/developers/python.html#using-conda > > This is the typical environment most contributors work in. Even when not > > using conda as a package/environment manager elsewhere, I would recommend > > to use it to setup your Arrow build environment as this is the way most > > developers do. Thus it will be easier to help you and this is the setup > we > > (try to) maintain best. > > > > Cheers > > Uwe > > > > On Sat, Aug 31, 2019, at 3:48 PM, Chris Teoh wrote: > > > Does this approach fit with potentially a contributor's workflow? I was > > > looking into contributing though I'm unsure if I am doing it right. > > > > > > On Sat, 31 Aug 2019 at 22:22, Jeroen Ooms > wrote: > > > > > > > On Sat, Aug 31, 2019 at 4:48 AM Chris Teoh > > wrote: > > > > > > > > > > That being said, is there an easier way by using a Docker > container I > > > > could > > > > > use to build this in? > > > > > > > > An easy way to install arrow on MacOS is using homebrew. To get a > > > > precompiled version of the latest release: > > > > > > > > brew install apache-arrow > > > > > > > > Or to build the master branch from source: > > > > > > > >brew install apache-arrow --HEAD > > > > > > > > If you want to customize the configuration use "brew edit > > > > apache-arrow" before building from source. > > > > > > > > > > > > > -- > > > Chris > > > > > > -- Chris
Re: Trouble building on Mac OS Mojave
Does this approach fit with potentially a contributor's workflow? I was looking into contributing though I'm unsure if I am doing it right. On Sat, 31 Aug 2019 at 22:22, Jeroen Ooms wrote: > On Sat, Aug 31, 2019 at 4:48 AM Chris Teoh wrote: > > > > That being said, is there an easier way by using a Docker container I > could > > use to build this in? > > An easy way to install arrow on MacOS is using homebrew. To get a > precompiled version of the latest release: > > brew install apache-arrow > > Or to build the master branch from source: > >brew install apache-arrow --HEAD > > If you want to customize the configuration use "brew edit > apache-arrow" before building from source. > -- Chris
Re: Trouble building on Mac OS Mojave
That being said, is there an easier way by using a Docker container I could
use to build this in?
On Sat, 31 Aug 2019 at 12:44, Chris Teoh wrote:
> Hey there,
>
> Brand new to Arrow here.
>
> Trying to build it following the instructions and I get errors with the
> ORC module building cpp
>
> In file included from
> /Users/test/GitHub/arrow/cpp/build/orc_ep-prefix/src/orc_ep/c++/src/wrap/orc-proto-wrapper.cc:44:
>
> /Users/test/GitHub/arrow/cpp/build/orc_ep-prefix/src/orc_ep-build/c++/src/orc_proto.pb.cc:960:145:
> error: possible misuse of comma operator here [-Werror,-Wcomma]
> static bool dynamic_init_dummy_orc_5fproto_2eproto = (
>
> ::PROTOBUF_NAMESPACE_ID::internal::AddDescriptors(_table_orc_5fproto_2eproto),
> true);
>
> ^
>
> /Users/test/GitHub/arrow/cpp/build/orc_ep-prefix/src/orc_ep-build/c++/src/orc_proto.pb.cc:960:57:
> note: cast expression to void to silence warning
> static bool dynamic_init_dummy_orc_5fproto_2eproto = (
>
> ::PROTOBUF_NAMESPACE_ID::internal::AddDescriptors(_table_orc_5fproto_2eproto),
> true);
>
> I can disable the ORC module and that part builds fine, build command is:-
>
> pushd arrow/cpp/build
>
>
>
> cmake -DPYTHON_EXECUTABLE=$VIRTUAL_ENV/bin/python
> -DCMAKE_INSTALL_PREFIX=$ARROW_HOME \
>
> -DCMAKE_INSTALL_LIBDIR=lib \
>
> -DARROW_FLIGHT=ON \
>
> -DARROW_GANDIVA=ON \
>
> -DARROW_ORC=OFF \
>
> -DARROW_PARQUET=ON \
>
> -DARROW_PYTHON=ON \
>
> -DARROW_PLASMA=ON \
>
> -DARROW_BUILD_TESTS=ON \
>
> ..
>
> make -j4
>
> make install
>
> popd
>
> then I try to build the python module:-
>
> pushd arrow/python
> export PYARROW_WITH_FLIGHT=1
> export PYARROW_WITH_GANDIVA=1
> export PYARROW_WITH_ORC=0
> export PYARROW_WITH_PARQUET=1
> python setup.py build_ext --inplace
> popd
>
> and get:-
> running build_ext
> creating build
> creating build/temp.macosx-10.14-intel-2.7
> -- Running cmake for pyarrow
> cmake -DPYTHON_EXECUTABLE=/Users/test/GitHub/pyarrow/bin/python
> -DPYARROW_BUILD_FLIGHT=on -DPYARROW_BUILD_PARQUET=on
> -DPYARROW_BOOST_USE_SHARED=on -DPYARROW_BUILD_GANDIVA=on
> -DCMAKE_BUILD_TYPE=release /Users/test/GitHub/arrow/python
> -- The C compiler identification is AppleClang 10.0.1.10010046
> -- The CXX compiler identification is AppleClang 10.0.1.10010046
> -- Check for working C compiler:
> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
> -- Check for working C compiler:
> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
> -- works
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- Detecting C compile features
> -- Detecting C compile features - done
> -- Check for working CXX compiler:
> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
> -- Check for working CXX compiler:
> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
> -- works
> -- Detecting CXX compiler ABI info
> -- Detecting CXX compiler ABI info - done
> -- Detecting CXX compile features
> -- Detecting CXX compile features - done
> -- Compiler command: env LANG=C
> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
> -v
> -- Compiler version: Apple LLVM version 10.0.1 (clang-1001.0.46.4)
> Target: x86_64-apple-darwin18.6.0
> Thread model: posix
> InstalledDir:
> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
>
> -- Compiler id: Clang
> Selected compiler clang 4.1.0svn
> -- Performing Test CXX_SUPPORTS_SSE4_2
> -- Performing Test CXX_SUPPORTS_SSE4_2 - Success
> -- Performing Test CXX_SUPPORTS_ALTIVEC
> -- Performing Test CXX_SUPPORTS_ALTIVEC - Success
> -- Performing Test CXX_SUPPORTS_ARMCRC
> -- Performing Test CXX_SUPPORTS_ARMCRC - Failed
> -- Performing Test CXX_SUPPORTS_ARMV8_CRC_CRYPTO
> -- Performing Test CXX_SUPPORTS_ARMV8_CRC_CRYPTO - Failed
> -- Arrow build warning level: PRODUCTION
> Configured for RELEASE build (set with cmake
> -DCMAKE_BUILD_TYPE={release,debug,...})
> -- Build Type: RELEASE
> -- Build output directory:
> /Users/test/GitHub/arrow/python/build/temp.macosx-10.14-intel-2.7/release
> -- Found PythonInterp: /Users/test/GitHub/pyarrow/bin/python (found
> version "2.7.10")
> -- Found PythonLibs:
> /System/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib
> -- Found NumPy: version "1.16.5"
> /Users/test/GitHub/pyarrow/lib/python2.7/site-packages/numpy/core/inc
Trouble building on Mac OS Mojave
Hey there,
Brand new to Arrow here.
Trying to build it following the instructions and I get errors with the ORC
module building cpp
In file included from
/Users/test/GitHub/arrow/cpp/build/orc_ep-prefix/src/orc_ep/c++/src/wrap/orc-proto-wrapper.cc:44:
/Users/test/GitHub/arrow/cpp/build/orc_ep-prefix/src/orc_ep-build/c++/src/orc_proto.pb.cc:960:145:
error: possible misuse of comma operator here [-Werror,-Wcomma]
static bool dynamic_init_dummy_orc_5fproto_2eproto = (
::PROTOBUF_NAMESPACE_ID::internal::AddDescriptors(_table_orc_5fproto_2eproto),
true);
^
/Users/test/GitHub/arrow/cpp/build/orc_ep-prefix/src/orc_ep-build/c++/src/orc_proto.pb.cc:960:57:
note: cast expression to void to silence warning
static bool dynamic_init_dummy_orc_5fproto_2eproto = (
::PROTOBUF_NAMESPACE_ID::internal::AddDescriptors(_table_orc_5fproto_2eproto),
true);
I can disable the ORC module and that part builds fine, build command is:-
pushd arrow/cpp/build
cmake -DPYTHON_EXECUTABLE=$VIRTUAL_ENV/bin/python
-DCMAKE_INSTALL_PREFIX=$ARROW_HOME \
-DCMAKE_INSTALL_LIBDIR=lib \
-DARROW_FLIGHT=ON \
-DARROW_GANDIVA=ON \
-DARROW_ORC=OFF \
-DARROW_PARQUET=ON \
-DARROW_PYTHON=ON \
-DARROW_PLASMA=ON \
-DARROW_BUILD_TESTS=ON \
..
make -j4
make install
popd
then I try to build the python module:-
pushd arrow/python
export PYARROW_WITH_FLIGHT=1
export PYARROW_WITH_GANDIVA=1
export PYARROW_WITH_ORC=0
export PYARROW_WITH_PARQUET=1
python setup.py build_ext --inplace
popd
and get:-
running build_ext
creating build
creating build/temp.macosx-10.14-intel-2.7
-- Running cmake for pyarrow
cmake -DPYTHON_EXECUTABLE=/Users/test/GitHub/pyarrow/bin/python
-DPYARROW_BUILD_FLIGHT=on -DPYARROW_BUILD_PARQUET=on
-DPYARROW_BOOST_USE_SHARED=on -DPYARROW_BUILD_GANDIVA=on
-DCMAKE_BUILD_TYPE=release /Users/test/GitHub/arrow/python
-- The C compiler identification is AppleClang 10.0.1.10010046
-- The CXX compiler identification is AppleClang 10.0.1.10010046
-- Check for working C compiler:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Compiler command: env LANG=C
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-v
-- Compiler version: Apple LLVM version 10.0.1 (clang-1001.0.46.4)
Target: x86_64-apple-darwin18.6.0
Thread model: posix
InstalledDir:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
-- Compiler id: Clang
Selected compiler clang 4.1.0svn
-- Performing Test CXX_SUPPORTS_SSE4_2
-- Performing Test CXX_SUPPORTS_SSE4_2 - Success
-- Performing Test CXX_SUPPORTS_ALTIVEC
-- Performing Test CXX_SUPPORTS_ALTIVEC - Success
-- Performing Test CXX_SUPPORTS_ARMCRC
-- Performing Test CXX_SUPPORTS_ARMCRC - Failed
-- Performing Test CXX_SUPPORTS_ARMV8_CRC_CRYPTO
-- Performing Test CXX_SUPPORTS_ARMV8_CRC_CRYPTO - Failed
-- Arrow build warning level: PRODUCTION
Configured for RELEASE build (set with cmake
-DCMAKE_BUILD_TYPE={release,debug,...})
-- Build Type: RELEASE
-- Build output directory:
/Users/test/GitHub/arrow/python/build/temp.macosx-10.14-intel-2.7/release
-- Found PythonInterp: /Users/test/GitHub/pyarrow/bin/python (found version
"2.7.10")
-- Found PythonLibs:
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib
-- Found NumPy: version "1.16.5"
/Users/test/GitHub/pyarrow/lib/python2.7/site-packages/numpy/core/include
-- Found PkgConfig: /usr/local/bin/pkg-config (found version "0.29.2")
-- Found the Arrow core library: /Users/test/GitHub/dist/lib/libarrow.dylib
-- Found the Arrow Python library:
/Users/test/GitHub/dist/lib/libarrow_python.dylib
-- Added shared library dependency arrow_shared:
/Users/test/GitHub/dist/lib/libarrow.dylib
-- Added shared library dependency arrow_python_shared:
/Users/test/GitHub/dist/lib/libarrow_python.dylib
-- Checking for module 'parquet'
-- No package 'parquet' found
-- Could not find the parquet library. Looked in system search paths.
CMake Error at CMakeLists.txt:417 (message):
Unable to locate Parquet libraries
-- Configuring incomplete, errors occurred!
My "dist" folder is as follows:-
dist
dist/bin
dist/bin/plasma-store-server
dist/include
