[jira] [Comment Edited] (ARROW-909) libjemalloc.so.2: cannot open shared object file:
[ https://issues.apache.org/jira/browse/ARROW-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15989684#comment-15989684 ] Abdul Rahman edited comment on ARROW-909 at 4/29/17 1:48 AM: - [~wesmckinn] Thanks. I noticed arrow/cpp does have jemalloc in the build folder, but doesnt load them in system libraries. was (Author: abdulrahman004): [~wesmckinn] Thanks. I noticed arrow/cpp does have jemalloc in the build folder, but doesnt load them in system libraries.f > libjemalloc.so.2: cannot open shared object file: > -- > > Key: ARROW-909 > URL: https://issues.apache.org/jira/browse/ARROW-909 > Project: Apache Arrow > Issue Type: Bug > Environment: linux centos >Reporter: Abdul Rahman > Labels: pyarrow > > >>> import pyarrow > Traceback (most recent call last): > File "", line 1, in > File > "/home/default/src/venv/lib/python2.7/site-packages/pyarrow-0.2.1.dev244+g14bec24-py2.7-linux-x86_64.egg/pyarrow/__init__.py", > line 28, in > import pyarrow._config > ImportError: libjemalloc.so.2: cannot open shared object file: No such file > or directory > $LD_LIBRARY_PATH has libarrow_jemalloc.a along with other libraries including > libarrow.so, libparquet.so, libparquet_arrow.so. Pyarrow was built using > with-jemalloc and parquet-cpp was cmake-d with > -DPARQUET_ARROW=ON > Also, noticed that arrow/python documentation has been cleaned up with the > installation instructions having the coda approach only .Is this the only > supported way going forward ? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ARROW-909) libjemalloc.so.2: cannot open shared object file:
[ https://issues.apache.org/jira/browse/ARROW-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15989684#comment-15989684 ] Abdul Rahman commented on ARROW-909: [~wesmckinn] Thanks. I noticed arrow/cpp does have jemalloc in the build folder, but doesnt load them in system libraries.f > libjemalloc.so.2: cannot open shared object file: > -- > > Key: ARROW-909 > URL: https://issues.apache.org/jira/browse/ARROW-909 > Project: Apache Arrow > Issue Type: Bug > Environment: linux centos >Reporter: Abdul Rahman > Labels: pyarrow > > >>> import pyarrow > Traceback (most recent call last): > File "", line 1, in > File > "/home/default/src/venv/lib/python2.7/site-packages/pyarrow-0.2.1.dev244+g14bec24-py2.7-linux-x86_64.egg/pyarrow/__init__.py", > line 28, in > import pyarrow._config > ImportError: libjemalloc.so.2: cannot open shared object file: No such file > or directory > $LD_LIBRARY_PATH has libarrow_jemalloc.a along with other libraries including > libarrow.so, libparquet.so, libparquet_arrow.so. Pyarrow was built using > with-jemalloc and parquet-cpp was cmake-d with > -DPARQUET_ARROW=ON > Also, noticed that arrow/python documentation has been cleaned up with the > installation instructions having the coda approach only .Is this the only > supported way going forward ? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ARROW-881) [Python] Reconstruct Pandas DataFrame indexes using custom_metadata
[ https://issues.apache.org/jira/browse/ARROW-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phillip Cloud updated ARROW-881: Summary: [Python] Reconstruct Pandas DataFrame indexes using custom_metadata (was: Reconstruct Pandas DataFrame indexes using custom_metadata) > [Python] Reconstruct Pandas DataFrame indexes using custom_metadata > --- > > Key: ARROW-881 > URL: https://issues.apache.org/jira/browse/ARROW-881 > Project: Apache Arrow > Issue Type: Bug >Reporter: Phillip Cloud >Assignee: Phillip Cloud > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ARROW-909) libjemalloc.so.2: cannot open shared object file:
[ https://issues.apache.org/jira/browse/ARROW-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15989528#comment-15989528 ] Wes McKinney commented on ARROW-909: I opened ARROW-913 to make the user experience with jemalloc a bit more pleasant (only complain when you try to use the extension) > libjemalloc.so.2: cannot open shared object file: > -- > > Key: ARROW-909 > URL: https://issues.apache.org/jira/browse/ARROW-909 > Project: Apache Arrow > Issue Type: Bug > Environment: linux centos >Reporter: Abdul Rahman > Labels: pyarrow > > >>> import pyarrow > Traceback (most recent call last): > File "", line 1, in > File > "/home/default/src/venv/lib/python2.7/site-packages/pyarrow-0.2.1.dev244+g14bec24-py2.7-linux-x86_64.egg/pyarrow/__init__.py", > line 28, in > import pyarrow._config > ImportError: libjemalloc.so.2: cannot open shared object file: No such file > or directory > $LD_LIBRARY_PATH has libarrow_jemalloc.a along with other libraries including > libarrow.so, libparquet.so, libparquet_arrow.so. Pyarrow was built using > with-jemalloc and parquet-cpp was cmake-d with > -DPARQUET_ARROW=ON > Also, noticed that arrow/python documentation has been cleaned up with the > installation instructions having the coda approach only .Is this the only > supported way going forward ? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ARROW-913) [Python] Only link jemalloc to the Cython extension where it's needed
Wes McKinney created ARROW-913: -- Summary: [Python] Only link jemalloc to the Cython extension where it's needed Key: ARROW-913 URL: https://issues.apache.org/jira/browse/ARROW-913 Project: Apache Arrow Issue Type: Improvement Components: Python Reporter: Wes McKinney see https://github.com/apache/arrow/blob/master/python/CMakeLists.txt#L319 If the user does not have jemalloc installed, it would be better to defer the library load failure until they try to use the jemalloc-based memory allocator. See e.g. https://issues.apache.org/jira/browse/ARROW-909 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ARROW-912) [Python] Account for multiarch systems in development.rst
Wes McKinney created ARROW-912: -- Summary: [Python] Account for multiarch systems in development.rst Key: ARROW-912 URL: https://issues.apache.org/jira/browse/ARROW-912 Project: Apache Arrow Issue Type: Improvement Components: Python Reporter: Wes McKinney Some systems will install libraries in lib64 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ARROW-902) [C++] offline build support
[
https://issues.apache.org/jira/browse/ARROW-902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15989525#comment-15989525
]
Wes McKinney commented on ARROW-902:
Good point; we have to add {{CMAKE_INSTALL_LIBDIR}} in conda recipes for this
reason. I opened https://issues.apache.org/jira/browse/ARROW-912 to fix this in
the documentation
> [C++] offline build support
> ---
>
> Key: ARROW-902
> URL: https://issues.apache.org/jira/browse/ARROW-902
> Project: Apache Arrow
> Issue Type: Wish
>Reporter: Luke Higgins
>Priority: Minor
> Attachments: CMakeLists.txt
>
>
> Could there be a flag or similar to make offline building of the project
> easier (or maybe that is in place and I don't see it)?
> Something that perhaps in CMakeLists.txt that has the option for the URL to
> be a file:// path?
> This would help me in that I could download all relevant tarballs and have
> the same build happen offline or online.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (ARROW-909) libjemalloc.so.2: cannot open shared object file:
[
https://issues.apache.org/jira/browse/ARROW-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15989459#comment-15989459
]
Wes McKinney commented on ARROW-909:
hi [~abdulrahman004]. Do you have the jemalloc libraries installed in your
system? {{libjemalloc}} must be in your library loader path; having
{{libarrow_jemalloc.so}} is not enough because this is only a binding to
Arrow's memory allocator. So either install jemalloc on your system or simply
don't build that extension. We have provided for building jemalloc in the C++
libraries' thirdparty stack to facilitate cross-platform packaging (e.g.
manylinux1 for Python packages). The expectation is that it will be separately
installed on the system where it's deployed.
> Also, noticed that arrow/python documentation has been cleaned up with the
> installation instructions having the coda approach only .Is this the only
> supported way going forward ?
I guess we have to define "supported". It's hard for us to be on the hook to
debug build issues on any arbitrary platform. I added the conda build because
that's reasonably portable, and doesn't involve platform specific details (like
Linux package managers, or Homebrew). It would be great if you or other could
contribute reproducible build instructions for other platforms and package
managers. I opened https://issues.apache.org/jira/browse/ARROW-911 to cover
this. It would be good to have a build guide for macOS, Ubuntu 14.04 and 16.04,
and CentOS/RedHat 6 and/or 7. Since most of the dependencies build out of the
box, there will be minimal pre-requisites to install from apt-get/yum/Homebrew.
Patches welcome.
> libjemalloc.so.2: cannot open shared object file:
> --
>
> Key: ARROW-909
> URL: https://issues.apache.org/jira/browse/ARROW-909
> Project: Apache Arrow
> Issue Type: Bug
> Environment: linux centos
>Reporter: Abdul Rahman
> Labels: pyarrow
>
> >>> import pyarrow
> Traceback (most recent call last):
> File "", line 1, in
> File
> "/home/default/src/venv/lib/python2.7/site-packages/pyarrow-0.2.1.dev244+g14bec24-py2.7-linux-x86_64.egg/pyarrow/__init__.py",
> line 28, in
> import pyarrow._config
> ImportError: libjemalloc.so.2: cannot open shared object file: No such file
> or directory
> $LD_LIBRARY_PATH has libarrow_jemalloc.a along with other libraries including
> libarrow.so, libparquet.so, libparquet_arrow.so. Pyarrow was built using
> with-jemalloc and parquet-cpp was cmake-d with
> -DPARQUET_ARROW=ON
> Also, noticed that arrow/python documentation has been cleaned up with the
> installation instructions having the coda approach only .Is this the only
> supported way going forward ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (ARROW-911) [Python] Expand development.rst with build instructions without conda
Wes McKinney created ARROW-911: -- Summary: [Python] Expand development.rst with build instructions without conda Key: ARROW-911 URL: https://issues.apache.org/jira/browse/ARROW-911 Project: Apache Arrow Issue Type: Improvement Components: Python Reporter: Wes McKinney There should be sufficient detail to install on at least OS X and Linux using the built-in thirdparty build toolchain https://github.com/wesm/arrow/blob/ee5cb2ad171f0f4c7673f2937dc226d62aad972c/python/README.md -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ARROW-910) [C++] Write 0-length EOS indicator at end of stream
Wes McKinney created ARROW-910: -- Summary: [C++] Write 0-length EOS indicator at end of stream Key: ARROW-910 URL: https://issues.apache.org/jira/browse/ARROW-910 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Wes McKinney Fix For: 0.3.0 While not required, this will make the C++ StreamWriter more consistent with the Java one -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ARROW-902) [C++] offline build support
[ https://issues.apache.org/jira/browse/ARROW-902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15988787#comment-15988787 ] Luke Higgins commented on ARROW-902: so on the thirdparty toolchain doc, development.rst, I had to do the following for the build: export ARROW_HOME=/[blah blah]/envs/pyarrow-dev/ export LD_LIBRARY_PATH=/[blah blah]/envs/pyarrow-dev/lib64/ > [C++] offline build support > --- > > Key: ARROW-902 > URL: https://issues.apache.org/jira/browse/ARROW-902 > Project: Apache Arrow > Issue Type: Wish >Reporter: Luke Higgins >Priority: Minor > Attachments: CMakeLists.txt > > > Could there be a flag or similar to make offline building of the project > easier (or maybe that is in place and I don't see it)? > Something that perhaps in CMakeLists.txt that has the option for the URL to > be a file:// path? > This would help me in that I could download all relevant tarballs and have > the same build happen offline or online. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ARROW-909) libjemalloc.so.2: cannot open shared object file:
Abdul Rahman created ARROW-909: -- Summary: libjemalloc.so.2: cannot open shared object file: Key: ARROW-909 URL: https://issues.apache.org/jira/browse/ARROW-909 Project: Apache Arrow Issue Type: Bug Environment: linux centos Reporter: Abdul Rahman >>> import pyarrow Traceback (most recent call last): File "", line 1, in File "/home/default/src/venv/lib/python2.7/site-packages/pyarrow-0.2.1.dev244+g14bec24-py2.7-linux-x86_64.egg/pyarrow/__init__.py", line 28, in import pyarrow._config ImportError: libjemalloc.so.2: cannot open shared object file: No such file or directory $LD_LIBRARY_PATH has libarrow_jemalloc.a along with other libraries including libarrow.so, libparquet.so, libparquet_arrow.so. Pyarrow was built using with-jemalloc and parquet-cpp was cmake-d with -DPARQUET_ARROW=ON Also, noticed that arrow/python documentation has been cleaned up with the installation instructions having the coda approach only .Is this the only supported way going forward ? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
