[ https://issues.apache.org/jira/browse/ARROW-13053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17385581#comment-17385581 ]
Ryan Williams edited comment on ARROW-13053 at 7/22/21, 3:27 PM: ----------------------------------------------------------------- I was able to {{pip install}} a version of pyarrow by creating a branch from 4.0.1 and cherry-picking in the fix from this issue: {code:bash} git clone https://github.com/apache/arrow.git && cd arrow git checkout apache-arrow-4.0.1 git cherry-pick 0e9285b28 pip install -e python {code} You can install [my branch|https://github.com/runsascoded/arrow/tree/macos-4.0.1] directly via: {code:bash} pip install pyarrow@git+https://github.com/runsascoded/arrow.git@macos-4.0.1#subdirectory=python {code} It seems to work on Python 3.9.5, but I get a linker error trying to actually {{import pyarrow}} in Python 3.8.10: {code:none} $ python -c 'import pyarrow' Traceback (most recent call last): File "<string>", line 1, in <module> File "/Users/ryan/.pyenv/versions/ctbk-3.8.10/lib/python3.8/site-packages/pyarrow/__init__.py", line 63, in <module> import pyarrow.lib as _lib ImportError: dlopen(/Users/ryan/.pyenv/versions/ctbk-3.8.10/lib/python3.8/site-packages/pyarrow/lib.cpython-38-darwin.so, 2): Symbol not found: __Py_FatalErrorFunc Referenced from: /opt/homebrew/opt/apache-arrow/lib/libarrow_python.400.dylib Expected in: flat namespace in /opt/homebrew/opt/apache-arrow/lib/libarrow_python.400.dylib {code} I'm not sure what the problem is there. Note also that trying to install the upstream 5.0.0 release tag fails because Homebrew only offers 4.0.1, and you'll run into compile errors (and of course, the vanilla 4.0.1 release doesn't include the fix from this issue). was (Author: rdub): I was able to {{pip install}} a version of pyarrow by creating a branch from 4.0.1 and cherry-picking in the fix from this issue: {code:bash} git clone https://github.com/apache/arrow.git && cd arrow git checkout apache-arrow-4.0.1 git cherry-pick 0e9285b28 pip install -e python {code} You can install [my branch|https://github.com/runsascoded/arrow/tree/macos-4.0.1] directly via: {code:bash} pip install pyarrow@git+https://github.com/runsascoded/arrow.git@macos-4.0.1#subdirectory=python {code} I did notice a linker issue trying to actually {{import pyarrow}} one time when I installed it: {code:none} $ python -c 'import pyarrow' Traceback (most recent call last): File "<string>", line 1, in <module> File "/Users/ryan/.pyenv/versions/ctbk-3.8.10/lib/python3.8/site-packages/pyarrow/__init__.py", line 63, in <module> import pyarrow.lib as _lib ImportError: dlopen(/Users/ryan/.pyenv/versions/ctbk-3.8.10/lib/python3.8/site-packages/pyarrow/lib.cpython-38-darwin.so, 2): Symbol not found: __Py_FatalErrorFunc Referenced from: /opt/homebrew/opt/apache-arrow/lib/libarrow_python.400.dylib Expected in: flat namespace in /opt/homebrew/opt/apache-arrow/lib/libarrow_python.400.dylib {code} But I'm unable to reproduce that right now; not sure what happened. Note also that trying to install the upstream 5.0.0 release tag fails because Homebrew only offers 4.0.1, and you'll run into compile errors (and of course, the vanilla 4.0.1 release doesn't include the fix from this issue). > [Python] Build fails on MacOS Big Sur using homebrewed Arrow libraries > ---------------------------------------------------------------------- > > Key: ARROW-13053 > URL: https://issues.apache.org/jira/browse/ARROW-13053 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Affects Versions: 4.0.1 > Environment: MacOS BigSur 11.4 (Apple Silicon) > Python 3.9.5 > apache-arrow 4.0.1 (via Homebrew) > Reporter: Dorian Kind > Assignee: Dorian Kind > Priority: Major > Labels: build-failure, pull-request-available > Fix For: 5.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > When installing pyarrow 4.0.1 from source, the install step fails with > {{error: can't copy 'build/lib.macosx-11.3-arm64-3.9/pyarrow/include/arrow': > doesn't exist or not a regular file}} > because the headers directory > {{build/lib.macosx-11.3-arm64-3.9/pyarrow/include/arrow}} > is a relative symlink to {{../Cellar/apache-arrow/4.0.1/include/arrow}} > I believe this is caused by the build system including the header files from > {{/opt/homebrew/include/arrow}}, which is the above symlink: > {\{ls -hl /opt/homebrew/include/arrow}} > {{lrwxr-xr-x 1 dki admin 42B Jun 8 15:35 /opt/homebrew/include/arrow -> > ../Cellar/apache-arrow/4.0.1/include/arrow}} > I was able work around this issue by modifying line 334 in {{CMakeLists.txt}} > from > {{#Always bundle includes}} > {{file(COPY ${ARROW_INCLUDE_DIR}/arrow DESTINATION > ${BUILD_OUTPUT_ROOT_DIRECTORY}/include)}} > to > {{#Always bundle includes}} > {{get_filename_component(REAL_ARROW_INCLUDE_DIR "${ARROW_INCLUDE_DIR}/arrow" > REALPATH)}} > {{file(COPY ${}}{{REAL_ARROW_INCLUDE_DIR}}{{} DESTINATION > ${BUILD_OUTPUT_ROOT_DIRECTORY}/include)}} > But I'm not familiar with CMake, so maybe there is a more appropriate way to > fix this. -- This message was sent by Atlassian Jira (v8.3.4#803005)