[ https://issues.apache.org/jira/browse/ARROW-955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15999598#comment-15999598 ]
Devang Shah commented on ARROW-955: ----------------------------------- Yes! Thanks a million. However, a couple of tests fail: (code) -- Found the Parquet library: /usr/local/lib/libparquet.so -- Found the Parquet Arrow library: /usr/local/lib -- Found the Arrow core library: /home/derdo/miniconda2/envs/pyarrow-dev/lib/libarrow.so -- Found the Arrow Python library: /home/derdo/miniconda2/envs/pyarrow-dev/lib/libarrow_python.so -- Found the Arrow jemalloc library: /home/derdo/miniconda2/envs/pyarrow-dev/lib/libarrow_jemalloc.so Added shared library dependency arrow: /home/derdo/miniconda2/envs/pyarrow-dev/lib/libarrow.so Added shared library dependency arrow_python: /home/derdo/miniconda2/envs/pyarrow-dev/lib/libarrow_python.so Added shared library dependency parquet_arrow: /usr/local/lib/libparquet_arrow.so Added shared library dependency arrow_jemalloc: /home/derdo/miniconda2/envs/pyarrow-dev/lib/libarrow_jemalloc.so -- Configuring done -- Generating done -- Build files have been written to: /home/derdo/repos/arrow/python/build/temp.linux-x86_64-2.7 make Scanning dependencies of target _parquet_pyx [ 4%] Compiling Cython CXX source for _parquet... [ 4%] Built target _parquet_pyx Scanning dependencies of target _parquet [ 8%] Building CXX object CMakeFiles/_parquet.dir/_parquet.cxx.o [ 12%] Linking CXX shared module release/_parquet.so [ 12%] Built target _parquet Scanning dependencies of target _error_pyx [ 16%] Compiling Cython CXX source for _error... [ 16%] Built target _error_pyx Scanning dependencies of target _error [ 20%] Building CXX object CMakeFiles/_error.dir/_error.cxx.o [ 25%] Linking CXX shared module release/_error.so [ 25%] Built target _error Scanning dependencies of target _jemalloc_pyx [ 29%] Compiling Cython CXX source for _jemalloc... [ 29%] Built target _jemalloc_pyx Scanning dependencies of target _jemalloc [ 33%] Building CXX object CMakeFiles/_jemalloc.dir/_jemalloc.cxx.o [ 37%] Linking CXX shared module release/_jemalloc.so [ 37%] Built target _jemalloc Scanning dependencies of target _table_pyx [ 41%] Compiling Cython CXX source for _table... [ 41%] Built target _table_pyx Scanning dependencies of target _table [ 45%] Building CXX object CMakeFiles/_table.dir/_table.cxx.o [ 50%] Linking CXX shared module release/_table.so [ 50%] Built target _table Scanning dependencies of target _config_pyx [ 54%] Compiling Cython CXX source for _config... [ 54%] Built target _config_pyx Scanning dependencies of target _config [ 58%] Building CXX object CMakeFiles/_config.dir/_config.cxx.o [ 62%] Linking CXX shared module release/_config.so [ 62%] Built target _config Scanning dependencies of target _memory_pyx [ 66%] Compiling Cython CXX source for _memory... [ 66%] Built target _memory_pyx Scanning dependencies of target _memory [ 70%] Building CXX object CMakeFiles/_memory.dir/_memory.cxx.o [ 75%] Linking CXX shared module release/_memory.so [ 75%] Built target _memory Scanning dependencies of target _array_pyx [ 79%] Compiling Cython CXX source for _array... [ 79%] Built target _array_pyx Scanning dependencies of target _array [ 83%] Building CXX object CMakeFiles/_array.dir/_array.cxx.o [ 87%] Linking CXX shared module release/_array.so [ 87%] Built target _array Scanning dependencies of target _io_pyx [ 91%] Compiling Cython CXX source for _io... [ 91%] Built target _io_pyx Scanning dependencies of target _io [ 95%] Building CXX object CMakeFiles/_io.dir/_io.cxx.o [100%] Linking CXX shared module release/_io.so [100%] Built target _io ('Moving built C-extension', 'release/_array.so', 'to build path', '/home/derdo/repos/arrow/python/pyarrow/_array.so') ('Moving built C-extension', 'release/_config.so', 'to build path', '/home/derdo/repos/arrow/python/pyarrow/_config.so') ('Moving built C-extension', 'release/_error.so', 'to build path', '/home/derdo/repos/arrow/python/pyarrow/_error.so') ('Moving built C-extension', 'release/_io.so', 'to build path', '/home/derdo/repos/arrow/python/pyarrow/_io.so') ('Moving built C-extension', 'release/_jemalloc.so', 'to build path', '/home/derdo/repos/arrow/python/pyarrow/_jemalloc.so') ('Moving built C-extension', 'release/_memory.so', 'to build path', '/home/derdo/repos/arrow/python/pyarrow/_memory.so') ('Moving built C-extension', 'release/_parquet.so', 'to build path', '/home/derdo/repos/arrow/python/pyarrow/_parquet.so') ('Moving built C-extension', 'release/_table.so', 'to build path', '/home/derdo/repos/arrow/python/pyarrow/_table.so') (pyarrow-dev) derdo@prompt:~/repos/arrow/python$ py.test pyarrow =========================== test session starts =========================== platform linux2 -- Python 2.7.13, pytest-3.0.7, py-1.4.33, pluggy-0.4.0 rootdir: /home/derdo/repos/arrow/python, inifile: collected 210 items pyarrow/tests/test_array.py ........... pyarrow/tests/test_convert_builtin.py ...................... pyarrow/tests/test_convert_pandas.py ..........................x.... pyarrow/tests/test_feather.py ......................x.... pyarrow/tests/test_hdfs.py sssssssssssssss pyarrow/tests/test_io.py .................. pyarrow/tests/test_ipc.py .......... pyarrow/tests/test_jemalloc.py .. pyarrow/tests/test_parquet.py ...................... pyarrow/tests/test_scalars.py .......... pyarrow/tests/test_schema.py ............. pyarrow/tests/test_table.py ............. pyarrow/tests/test_tensor.py ................ ============ 193 passed, 15 skipped, 2 xfailed in 1.27 seconds ============ > [Docs] Guide for building Python from source on Ubuntu 14.04 LTS without conda > ------------------------------------------------------------------------------ > > Key: ARROW-955 > URL: https://issues.apache.org/jira/browse/ARROW-955 > Project: Apache Arrow > Issue Type: Improvement > Components: Python > Environment: Ubuntu - 3.19.0-80-generic #88~14.04.1-Ubuntu > Python 2.7.6 > Reporter: Devang Shah > > I built pyarrow, arrow, and parquet-cpp from source - so that I could use the > new read_row_group() interface and in general, have access to the latest > versions. I ran into many issues during the build but was ultimately > successful (notes below). However, I am not able to import pyarrow.parquet > due to the following issue: > >>import pyarrow.parquet > Traceback (most recent call last): > File "", line 1, in > File "pyarrow/init.py", line 28, in > import pyarrow._config > ImportError: No module named _config > This is similar to an issue reported in github/conda-forge/pyarrow-feedstock, > where also I posted this...but I think this forum is more direct and > appropriate - so re-posting here. > I used instructions at https://arrow.apache.org/docs/python/install.html to > build arrow/cpp, parquet-cpp, and then pyarrow, with the following deviations > (I view them as possibly bugs in the instructions): > arrow/cpp build: > export ARROW_HOME=$HOME/local > I had to specify -DARROW_PYTHON=on and -DPARQUET_ARROW=ON to the cmake > command (besides the -DCMAKE_INSTALL_PREFIX=$ARROW_HOME) > parquet-cpp build: > export ARROW_HOME=$HOME/local > cmake -DARROW_HOME=$HOME/local -DPARQUET_ARROW_LINKAGE=static > -DPARQUET_ARROW=ON . > make > sudo make install ----> this installs parquet libs in the std systems > location (/usr/local/lib) so that the pyarrow build (see below) can find the > parquet libs > pyarrow build: > export ARROW_HOME=$HOME/local (not a deviation; just repeating here) > export LD_LIBRARY_PATH=$HOME/local/lib:$HOME/parquet4/parquet-cpp/build/latest > sudo python setup.py build_ext --with-parquet --with-jemalloc > --build-type=release install > sudo python setup.py install > (sudo is needed to install in /usr/local/lib/python2.7/dist-packages ) > These are the steps and modifications to the instructions needed for me to > build the pyarrow.parquet package. However, when I now try to import the > package I get the error specified above. > Maybe I did something wrong in my steps which I kind of put together by > searching for these issues...but really can't tell what. It took me almost a > whole day to get to the point where I can build pyarrow and parquet, and now > I can't use what I built. > Any comments, help appreciated! Thanks in advance. -- This message was sent by Atlassian JIRA (v6.3.15#6346)