[ https://issues.apache.org/jira/browse/ARROW-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Philipp Moritz resolved ARROW-2872. ----------------------------------- Resolution: Fixed Issue resolved by pull request 2279 [https://github.com/apache/arrow/pull/2279] > [Python] Add pytest mark to opt into TensorFlow-related unit tests > ------------------------------------------------------------------ > > Key: ARROW-2872 > URL: https://issues.apache.org/jira/browse/ARROW-2872 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Reporter: Wes McKinney > Assignee: Wes McKinney > Priority: Major > Labels: pull-request-available > Fix For: 0.10.0 > > Time Spent: 20m > Remaining Estimate: 0h > > After pulling in ARROW-1744, I hit this error: > {code} > pyarrow/tests/test_plasma_tf_op.py::test_plasma_tf_op FAILED > [ 82%] > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured stdout > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > TensorFlow version: 1.8.0 > Compiling Plasma TensorFlow Op... > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured stderr > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > +++ dirname /home/wesm/code/arrow/python/pyarrow/tensorflow/build.sh > ++ cd /home/wesm/code/arrow/python/pyarrow/tensorflow > ++ pwd > + PYARROW_TENSORFLOW_DIR=/home/wesm/code/arrow/python/pyarrow/tensorflow > ++ python -c 'import tensorflow as tf; print(" > ".join(tf.sysconfig.get_compile_flags()))' > + > TF_CFLAGS='-I/home/wesm/miniconda/envs/arrow-dev/lib/python3.6/site-packages/tensorflow/include > -D_GLIBCXX_USE_CXX11_ABI=0' > ++ python -c 'import tensorflow as tf; print(" > ".join(tf.sysconfig.get_link_flags()))' > + > TF_LFLAGS='-L/home/wesm/miniconda/envs/arrow-dev/lib/python3.6/site-packages/tensorflow > -ltensorflow_framework' > ++ uname > + '[' Linux == Darwin ']' > + NDEBUG=-DNDEBUG > ++ pkg-config --cflags --libs plasma arrow arrow-python > + g++ -std=c++11 -g -shared > /home/wesm/code/arrow/python/pyarrow/tensorflow/plasma_op.cc -o > /home/wesm/code/arrow/python/pyarrow/tensorflow/plasma_op.so -DNDEBUG > -I/home/wesm/local/include > -I/home/wesm/miniconda/envs/arrow-dev/include/python3.6m > -I/home/wesm/local/include -L/home/wesm/local/lib -lplasma -larrow_python > -larrow -fPIC > -I/home/wesm/miniconda/envs/arrow-dev/lib/python3.6/site-packages/tensorflow/include > -D_GLIBCXX_USE_CXX11_ABI=0 > -L/home/wesm/miniconda/envs/arrow-dev/lib/python3.6/site-packages/tensorflow > -ltensorflow_framework -O2 > /home/wesm/code/arrow/python/pyarrow/tensorflow/plasma_op.cc:33:10: fatal > error: arrow/adapters/tensorflow/convert.h: No such file or directory > #include "arrow/adapters/tensorflow/convert.h" > ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > compilation terminated. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > use_gpu = False > @pytest.mark.plasma > def test_plasma_tf_op(use_gpu=False): > import pyarrow.plasma as plasma > > > plasma.build_plasma_tensorflow_op() > pyarrow/tests/test_plasma_tf_op.py:89: > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ _ _ _ > pyarrow/plasma.py:56: in build_plasma_tensorflow_op > subprocess.check_call(["bash", script_path]) > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ _ _ _ > popenargs = (['bash', > '/home/wesm/code/arrow/python/pyarrow/tensorflow/build.sh'],) > kwargs = {}, retcode = 1 > cmd = ['bash', '/home/wesm/code/arrow/python/pyarrow/tensorflow/build.sh'] > def check_call(*popenargs, **kwargs): > """Run command with arguments. Wait for command to complete. If > the exit code was zero then return, otherwise raise > CalledProcessError. The CalledProcessError object will have the > return code in the returncode attribute. > > The arguments are the same as for the call function. Example: > > check_call(["ls", "-l"]) > """ > retcode = call(*popenargs, **kwargs) > if retcode: > cmd = kwargs.get("args") > if cmd is None: > cmd = popenargs[0] > > raise CalledProcessError(retcode, cmd) > E subprocess.CalledProcessError: Command '['bash', > '/home/wesm/code/arrow/python/pyarrow/tensorflow/build.sh']' returned > non-zero exit status 1. > ../../../miniconda/envs/arrow-dev/lib/python3.6/subprocess.py:291: > CalledProcessError > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > /home/wesm/miniconda/envs/arrow-dev/lib/python3.6/subprocess.py(291)check_call() > -> raise CalledProcessError(retcode, cmd) > {code} > I misdiagnosed the problem originally (which was my stale Arrow install), but > I think it would be good to have a separate pytest mark for > TensorFlow-related unit tests since they are also time consuming. The unit > test in question now has both the plasma and tensorflow marks -- This message was sent by Atlassian JIRA (v7.6.3#76005)