[
https://issues.apache.org/jira/browse/ARROW-955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15999644#comment-15999644
]
Devang Shah commented on ARROW-955:
-----------------------------------
Here's the output of the successful "python setup.py ..." call for the pyarrow
build (which used to fail until your fix):
{code}
(pyarrow-dev) derdo@prompt:~/repos/arrow/python$ python setup.py build_ext
--build-type=$ARROW_BUILD_TYPE
--with-parquet --with-jemalloc --inplace
running build_ext
cmake -DPYTHON_EXECUTABLE=/home/derdo/miniconda2/envs/pyarrow-dev/bin/python
-DPYARROW_BUILD_PARQUET=on
-DPYARROW_BUILD_JEMALLOC=on -DCMAKE_BUILD_TYPE=release
/home/derdo/repos/arrow/python
Configured for RELEASE build (set with cmake
-DCMAKE_BUILD_TYPE={release,debug,...})
-- Build Type: RELEASE
INFOCompiler version: Using built-in specs.
COLLECT_GCC=/usr/bin/c++
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.8/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu
4.8.4-2ubuntu1~14.04.3'
--with-bugurl=file:///usr/share/doc/gcc-4.8/README.Bugs
--enable-languages=c,c++,java,go,d,fortran,objc,obj-c++
--prefix=/usr --program-suffix=-4.8 --enable-shared --enable-linker-build-id
--libexecdir=/usr/lib
--without-included-gettext --enable-threads=posix
--with-gxx-include-dir=/usr/include/c++/4.8 --libdir=/usr/lib
--enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug
--enable-libstdcxx-time=yes
--enable-gnu-unique-object --disable-libmudflap --enable-plugin
--with-system-zlib --disable-browser-plugin
--enable-java-awt=gtk --enable-gtk-cairo
--with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64/jre --enable-java-home
--with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64
--with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.8-amd64
--with-arch-directory=amd64
--with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc
--enable-multiarch --disable-werror --with-arch-32=i686
--with-abi=m64 --with-multilib-list=m32,m64,mx32 --with-tune=generic
--enable-checking=release --build=x86_64-linux-gnu
--host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
INFOCompiler id: GNU
Selected compiler gcc 4.8.4
Using static linking for RELEASE builds
collect2 version 4.8.4
/usr/bin/ld --sysroot=/ --build-id --eh-frame-hdr -m elf_x86_64
--hash-style=gnu --as-needed -dynamic-linker
/lib64/ld-linux-x86-64.so.2 -z relro
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crt1.o
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crti.o
/usr/lib/gcc/x86_64-linux-gnu/4.8/crtbegin.o
-L/usr/lib/gcc/x86_64-linux-gnu/4.8
-L/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu
-L/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib -L/lib/x86_64-linux-gnu
-L/lib/../lib -L/usr/lib/x86_64-linux-gnu
-L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/4.8/../../.. --version
-lstdc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc
/usr/lib/gcc/x86_64-linux-gnu/4.8/crtend.o
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crtn.o
Using ld linker
-- Build output directory:
/home/derdo/repos/arrow/python/build/temp.linux-x86_64-2.7/release/
-- Searching for Python libs in
/home/derdo/miniconda2/envs/pyarrow-dev/lib64;/home/derdo/miniconda2/envs/pyarrow-dev/lib;/home/derdo/miniconda2/envs/pyarrow-dev/lib/python2.7/config
-- Looking for python2.7
-- Found Python lib /home/derdo/miniconda2/envs/pyarrow-dev/lib/libpython2.7.so
-- Searching for Python libs in
/home/derdo/miniconda2/envs/pyarrow-dev/lib64;/home/derdo/miniconda2/envs/pyarrow-dev/lib;/home/derdo/miniconda2/envs/pyarrow-dev/lib/python2.7/config
-- Looking for python2.7
-- Found Python lib /home/derdo/miniconda2/envs/pyarrow-dev/lib/libpython2.7.so
-- Found the Parquet library: /usr/local/lib/libparquet.so
-- Found the Parquet Arrow library: /usr/local/lib
-- Found the Arrow core library:
/home/derdo/miniconda2/envs/pyarrow-dev/lib/libarrow.so
-- Found the Arrow Python library:
/home/derdo/miniconda2/envs/pyarrow-dev/lib/libarrow_python.so
-- Found the Arrow jemalloc library:
/home/derdo/miniconda2/envs/pyarrow-dev/lib/libarrow_jemalloc.so
Added shared library dependency arrow:
/home/derdo/miniconda2/envs/pyarrow-dev/lib/libarrow.so
Added shared library dependency arrow_python:
/home/derdo/miniconda2/envs/pyarrow-dev/lib/libarrow_python.so
Added shared library dependency parquet_arrow:
/usr/local/lib/libparquet_arrow.so
Added shared library dependency arrow_jemalloc:
/home/derdo/miniconda2/envs/pyarrow-dev/lib/libarrow_jemalloc.so
-- Configuring done
-- Generating done
-- Build files have been written to:
/home/derdo/repos/arrow/python/build/temp.linux-x86_64-2.7
make
Scanning dependencies of target _parquet_pyx
[ 4%] Compiling Cython CXX source for _parquet...
[ 4%] Built target _parquet_pyx
Scanning dependencies of target _parquet
[ 8%] Building CXX object CMakeFiles/_parquet.dir/_parquet.cxx.o
[ 12%] Linking CXX shared module release/_parquet.so
[ 12%] Built target _parquet
Scanning dependencies of target _error_pyx
[ 16%] Compiling Cython CXX source for _error...
[ 16%] Built target _error_pyx
Scanning dependencies of target _error
[ 20%] Building CXX object CMakeFiles/_error.dir/_error.cxx.o
[ 25%] Linking CXX shared module release/_error.so
[ 25%] Built target _error
Scanning dependencies of target _jemalloc_pyx
[ 29%] Compiling Cython CXX source for _jemalloc...
[ 29%] Built target _jemalloc_pyx
Scanning dependencies of target _jemalloc
[ 33%] Building CXX object CMakeFiles/_jemalloc.dir/_jemalloc.cxx.o
[ 37%] Linking CXX shared module release/_jemalloc.so
[ 37%] Built target _jemalloc
Scanning dependencies of target _table_pyx
[ 41%] Compiling Cython CXX source for _table...
[ 41%] Built target _table_pyx
Scanning dependencies of target _table
[ 45%] Building CXX object CMakeFiles/_table.dir/_table.cxx.o
[ 50%] Linking CXX shared module release/_table.so
[ 50%] Built target _table
Scanning dependencies of target _config_pyx
[ 54%] Compiling Cython CXX source for _config...
[ 54%] Built target _config_pyx
Scanning dependencies of target _config
[ 58%] Building CXX object CMakeFiles/_config.dir/_config.cxx.o
[ 62%] Linking CXX shared module release/_config.so
[ 62%] Built target _config
Scanning dependencies of target _memory_pyx
[ 66%] Compiling Cython CXX source for _memory...
[ 66%] Built target _memory_pyx
Scanning dependencies of target _memory
[ 70%] Building CXX object CMakeFiles/_memory.dir/_memory.cxx.o
[ 75%] Linking CXX shared module release/_memory.so
[ 75%] Built target _memory
Scanning dependencies of target _array_pyx
[ 79%] Compiling Cython CXX source for _array...
[ 79%] Built target _array_pyx
Scanning dependencies of target _array
[ 83%] Building CXX object CMakeFiles/_array.dir/_array.cxx.o
[ 87%] Linking CXX shared module release/_array.so
[ 87%] Built target _array
Scanning dependencies of target _io_pyx
[ 91%] Compiling Cython CXX source for _io...
[ 91%] Built target _io_pyx
Scanning dependencies of target _io
[ 95%] Building CXX object CMakeFiles/_io.dir/_io.cxx.o
[100%] Linking CXX shared module release/_io.so
[100%] Built target _io
('Moving built C-extension', 'release/_array.so', 'to build path',
'/home/derdo/repos/arrow/python/pyarrow/_array.so')
('Moving built C-extension', 'release/_config.so', 'to build path',
'/home/derdo/repos/arrow/python/pyarrow/_config.so')
('Moving built C-extension', 'release/_error.so', 'to build path',
'/home/derdo/repos/arrow/python/pyarrow/_error.so')
('Moving built C-extension', 'release/_io.so', 'to build path',
'/home/derdo/repos/arrow/python/pyarrow/_io.so')
('Moving built C-extension', 'release/_jemalloc.so', 'to build path',
'/home/derdo/repos/arrow/python/pyarrow/_jemalloc.so')
('Moving built C-extension', 'release/_memory.so', 'to build path',
'/home/derdo/repos/arrow/python/pyarrow/_memory.so')
('Moving built C-extension', 'release/_parquet.so', 'to build path',
'/home/derdo/repos/arrow/python/pyarrow/_parquet.so')
('Moving built C-extension', 'release/_table.so', 'to build path',
'/home/derdo/repos/arrow/python/pyarrow/_table.so')
{code}
> [Docs] Guide for building Python from source on Ubuntu 14.04 LTS without conda
> ------------------------------------------------------------------------------
>
> Key: ARROW-955
> URL: https://issues.apache.org/jira/browse/ARROW-955
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
> Environment: Ubuntu - 3.19.0-80-generic #88~14.04.1-Ubuntu
> Python 2.7.6
> Reporter: Devang Shah
>
> I built pyarrow, arrow, and parquet-cpp from source - so that I could use the
> new read_row_group() interface and in general, have access to the latest
> versions. I ran into many issues during the build but was ultimately
> successful (notes below). However, I am not able to import pyarrow.parquet
> due to the following issue:
> >>import pyarrow.parquet
> Traceback (most recent call last):
> File "", line 1, in
> File "pyarrow/init.py", line 28, in
> import pyarrow._config
> ImportError: No module named _config
> This is similar to an issue reported in github/conda-forge/pyarrow-feedstock,
> where also I posted this...but I think this forum is more direct and
> appropriate - so re-posting here.
> I used instructions at https://arrow.apache.org/docs/python/install.html to
> build arrow/cpp, parquet-cpp, and then pyarrow, with the following deviations
> (I view them as possibly bugs in the instructions):
> arrow/cpp build:
> export ARROW_HOME=$HOME/local
> I had to specify -DARROW_PYTHON=on and -DPARQUET_ARROW=ON to the cmake
> command (besides the -DCMAKE_INSTALL_PREFIX=$ARROW_HOME)
> parquet-cpp build:
> export ARROW_HOME=$HOME/local
> cmake -DARROW_HOME=$HOME/local -DPARQUET_ARROW_LINKAGE=static
> -DPARQUET_ARROW=ON .
> make
> sudo make install ----> this installs parquet libs in the std systems
> location (/usr/local/lib) so that the pyarrow build (see below) can find the
> parquet libs
> pyarrow build:
> export ARROW_HOME=$HOME/local (not a deviation; just repeating here)
> export LD_LIBRARY_PATH=$HOME/local/lib:$HOME/parquet4/parquet-cpp/build/latest
> sudo python setup.py build_ext --with-parquet --with-jemalloc
> --build-type=release install
> sudo python setup.py install
> (sudo is needed to install in /usr/local/lib/python2.7/dist-packages )
> These are the steps and modifications to the instructions needed for me to
> build the pyarrow.parquet package. However, when I now try to import the
> package I get the error specified above.
> Maybe I did something wrong in my steps which I kind of put together by
> searching for these issues...but really can't tell what. It took me almost a
> whole day to get to the point where I can build pyarrow and parquet, and now
> I can't use what I built.
> Any comments, help appreciated! Thanks in advance.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
