``` $ conda create -n fresh_env python=3.7
$ conda activate fresh_env $ conda install -c conda-forge pyarrow==3.0.0 $ python >>> import pyarrow as pa >>> pa.__version__ '3.0.0' >>> pa.__file__ '/ home/xander/anaconda3/envs/fresh_env/lib/python3.7/site-packages/pyarrow/__init__.py ( http://home/xander/anaconda3/envs/fresh_env/lib/python3.7/site-packages/pyarrow/__init__.py ) ' >>> import pyarrow.plasma >>> import pyarrow.gandiva Traceback (most recent call last): File "<stdin>", line 1, in <module> ModuleNotFoundError: No module named 'pyarrow.gandiva' ``` Same result in a fresh conda environment. The library is also still missing from the installed location: ``` $ l /home/xander/anaconda3/envs/fresh_env/lib/python3.7/site-packages/pyarrow/ | grep gandiva -rw-rw-r-- 3 xander xander 17K Jan 18 14:00 gandiva.pyx ``` And a correction to what I wrote earlier, I'm on Ubuntu 18.04: ``` $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 18.04.5 LTS Release: 18.04 Codename: bionic $ conda --version conda 4.10.0 ``` I do have several arrow packages installed via apt, but I don't think that's interfering: ``` apache-arrow-archive-keyring/bionic,now 3.0.0-1 all [installed] gir1.2-arrow-1.0/bionic,now 3.0.0-1 amd64 [installed,automatic] libarrow-dev/bionic,now 3.0.0-1 amd64 [installed] libarrow-glib-dev/bionic,now 3.0.0-1 amd64 [installed,automatic] libarrow-glib300/bionic,now 3.0.0-1 amd64 [installed,automatic] libarrow300/bionic,now 3.0.0-1 amd64 [installed] ``` Thanks for the help, Xander On Tue, Apr 13, 2021 at 1:25 PM, Wes McKinney < [email protected] > wrote: > > It looks to me like you have the wheel installed, not the conda package. > Can you reproduce this on Ubuntu from a fresh conda environment? > > On Tue, Apr 13, 2021 at 3:16 PM Xander Dunn < xander@ xander. ai ( > [email protected] ) > wrote: > > >> Typo. The issue remains present. From my Ubuntu machine just now: >> >> ``` >> >> $ python >> >> >>> import pyarrow as pa >> >> >>> print(pa.__file__) >> >> / home/ xander/ anaconda3/ envs/ plutus_model/ lib/ python3. 7/ >> site-packages/ >> pyarrow/ __init__. py ( >> http://home/xander/anaconda3/envs/plutus_model/lib/python3.7/site-packages/pyarrow/__init__.py >> ) >> >> >>> import pyarrow.plasma >> >> >>> import pyarrow.gandiva >> >> Traceback (most recent call last): >> >> File "<stdin>", line 1, in <module> >> >> ModuleNotFoundError: No module named 'pyarrow.gandiva' >> >> ``` >> >> >> >> The .py I'm executing on both machines is identical. Works on mac. Not >> found on Ubuntu. >> >> >> >> >> On Tue, Apr 13, 2021 at 1:01 PM, Micah Kornfield < emkornfield@ gmail. com >> ( [email protected] ) > wrote: >> >>> Hi Xander, >>> Was there autocorrect on this e-mail? the second example shows "gondiva" >>> not "gandiva" >>> >>> >>> -Micah >>> >>> On Tue, Apr 13, 2021 at 12:59 PM Xander Dunn < xander@ xander. ai ( >>> [email protected] ) > wrote: >>> >>> >>>> On my local macOS 11.2.3: >>>> >>>> ``` >>>> >>>> $ python --version >>>> >>>> Python 3.7.10 >>>> >>>> $ pip --version >>>> >>>> pip 21.0.1 from >>>> /usr/local/anaconda3/envs/my_model/lib/python3.7/site-packages/pip (python >>>> 3.7) >>>> >>>> $ pip list | grep pyarrow >>>> >>>> pyarrow 3.0.0 >>>> >>>> $ which python >>>> >>>> /usr/local/anaconda3/envs/my_model/bin/python >>>> >>>> $ python >>>> >>>> >>> import pyarrow as pa >>>> >>>> >>> print(pa.__file__) >>>> >>>> / usr/ local/ anaconda3/ envs/ my_model/ lib/ python3. 7/ site-packages/ >>>> pyarrow/ >>>> __init__. py ( >>>> http://usr/local/anaconda3/envs/my_model/lib/python3.7/site-packages/pyarrow/__init__.py >>>> ) >>>> >>>> >>> import pyarrow.plasma >>>> >>>> >>> import pyarrow.gandiva as ga >>>> >>>> >>> print(ga.__file__) >>>> >>>> / usr/ local/ anaconda3/ envs/ my_model/ lib/ python3. 7/ site-packages/ >>>> pyarrow/ >>>> gandiva. cpython-37m-darwin. so ( >>>> http://usr/local/anaconda3/envs/my_model/lib/python3.7/site-packages/pyarrow/gandiva.cpython-37m-darwin.so >>>> ) >>>> >>>> ``` >>>> >>>> >>>> >>>> On my Ubuntu 14.04 instance: >>>> >>>> ``` >>>> >>>> $ python --version >>>> >>>> Python 3.7.10 >>>> >>>> $ pip --version >>>> >>>> pip 21.0.1 from >>>> /home/xander/anaconda3/envs/my_model/lib/python3.7/site-packages/pip >>>> (python 3.7) >>>> >>>> $ pip list | grep pyarrow >>>> >>>> pyarrow 3.0.0 >>>> >>>> $ which python >>>> >>>> /home/xander/anaconda3/envs/my_model/bin/python >>>> >>>> $ python >>>> >>>> >>> import pyarrow as pa >>>> >>>> >>> print(pa.__file__) >>>> >>>> / home/ xander/ anaconda3/ envs/ my_model/ lib/ python3. 7/ site-packages/ >>>> pyarrow/ __init__. py ( >>>> http://home/xander/anaconda3/envs/my_model/lib/python3.7/site-packages/pyarrow/__init__.py >>>> ) >>>> >>>> >>> import pyarrow.plasma >>>> >>>> >>> import pyarrow.gondiva >>>> >>>> Traceback (most recent call last): >>>> >>>> File "<stdin>", line 1, in <module> >>>> >>>> ModuleNotFoundError: No module named 'pyarrow.gondiva' >>>> >>>> ``` >>>> >>>> You can see that pyarrow.gondiva is found on mac but not on Ubuntu. Same >>>> Python version. Same pyarrow version. I installed both of them with `conda >>>> install -c conda-forge pyarrow==3.0.0`. >>>> >>>> >>>> >>>> On Mac, I see the expected Cython file and library: >>>> >>>> ``` >>>> >>>> $ l >>>> /usr/local/anaconda3/envs/my_model/lib/python3.7/site-packages/pyarrow/ | >>>> grep gandiva >>>> >>>> -rwxrwxr-x 2 xander staff 221K Apr 1 12:44 gandiva. >>>> cpython-37m-darwin. >>>> so ( http://gandiva.cpython-37m-darwin.so/ ) >>>> >>>> -rw-rw-r-- 2 xander staff 17K Jan 18 14:00 gandiva.pyx >>>> >>>> ``` >>>> >>>> >>>> >>>> On Ubuntu, I see only the Cython file: >>>> >>>> ``` >>>> >>>> $ l ~/anaconda3/envs/my_model/lib/python3.7/site-packages/pyarrow/ | grep >>>> gandiva >>>> >>>> -rw-rw-r-- 1 xander xander 17K Apr 13 12:28 gandiva.pyx >>>> >>>> ``` >>>> >>>> >>>> >>>> Is this expected? Should I be able to import pyarrow.gandiva on Ubuntu? >>>> Everything is run on Ubuntu so if I make use of pyarrow.gandiva I'll need >>>> to figure out how to call it. >>>> >>>> >>>> >>>> It's mentioned here that it was removed from Python wheels but should >>>> still be available in the conda install: https:/ / issues. apache. org/ >>>> jira/ >>>> browse/ ARROW-10154 ( https://issues.apache.org/jira/browse/ARROW-10154 ). >>>> I'm not finding it in my Ubuntu conda install. >>>> >>>> >>>> >>>> Thanks, >>>> >>>> Xander >>>> >>> >>> >> >> > >
