Avro 1.53 doesn't have this issue? Does it use python-snappy? Russell Jurney twitter.com/rjurney russell.jur...@gmail.com datasyndrome.com
On Dec 23, 2011, at 7:05 PM, Ken Krugler <kkrugler_li...@transpac.com> wrote: I installed brew, then ran 'brew install snappy', which worked. But 'sudo python setup.py install' still fails, though with a different problem (I was getting the same compilation errors as below, now I get a broken pipe error). running install running bdist_egg running egg_info writing requirements to avro.egg-info/requires.txt writing avro.egg-info/PKG-INFO writing top-level names to avro.egg-info/top_level.txt writing dependency_links to avro.egg-info/dependency_links.txt reading manifest file 'avro.egg-info/SOURCES.txt' writing manifest file 'avro.egg-info/SOURCES.txt' installing library code to build/bdist.macosx-10.6-universal/egg running install_lib running build_py creating build/bdist.macosx-10.6-universal/egg creating build/bdist.macosx-10.6-universal/egg/avro copying build/lib/avro/__init__.py -> build/bdist.macosx-10.6-universal/egg/avro copying build/lib/avro/datafile.py -> build/bdist.macosx-10.6-universal/egg/avro copying build/lib/avro/io.py -> build/bdist.macosx-10.6-universal/egg/avro copying build/lib/avro/ipc.py -> build/bdist.macosx-10.6-universal/egg/avro copying build/lib/avro/protocol.py -> build/bdist.macosx-10.6-universal/egg/avro copying build/lib/avro/schema.py -> build/bdist.macosx-10.6-universal/egg/avro copying build/lib/avro/tool.py -> build/bdist.macosx-10.6-universal/egg/avro copying build/lib/avro/txipc.py -> build/bdist.macosx-10.6-universal/egg/avro byte-compiling build/bdist.macosx-10.6-universal/egg/avro/__init__.py to __init__.pyc byte-compiling build/bdist.macosx-10.6-universal/egg/avro/datafile.py to datafile.pyc byte-compiling build/bdist.macosx-10.6-universal/egg/avro/io.py to io.pyc byte-compiling build/bdist.macosx-10.6-universal/egg/avro/ipc.py to ipc.pyc byte-compiling build/bdist.macosx-10.6-universal/egg/avro/protocol.py to protocol.pyc byte-compiling build/bdist.macosx-10.6-universal/egg/avro/schema.py to schema.pyc byte-compiling build/bdist.macosx-10.6-universal/egg/avro/tool.py to tool.pyc byte-compiling build/bdist.macosx-10.6-universal/egg/avro/txipc.py to txipc.pyc creating build/bdist.macosx-10.6-universal/egg/EGG-INFO installing scripts to build/bdist.macosx-10.6-universal/egg/EGG-INFO/scripts running install_scripts running build_scripts creating build/bdist.macosx-10.6-universal/egg/EGG-INFO/scripts copying build/scripts-2.6/avro -> build/bdist.macosx-10.6-universal/egg/EGG-INFO/scripts changing mode of build/bdist.macosx-10.6-universal/egg/EGG-INFO/scripts/avro to 755 copying avro.egg-info/PKG-INFO -> build/bdist.macosx-10.6-universal/egg/EGG-INFO copying avro.egg-info/SOURCES.txt -> build/bdist.macosx-10.6-universal/egg/EGG-INFO copying avro.egg-info/dependency_links.txt -> build/bdist.macosx-10.6-universal/egg/EGG-INFO copying avro.egg-info/requires.txt -> build/bdist.macosx-10.6-universal/egg/EGG-INFO copying avro.egg-info/top_level.txt -> build/bdist.macosx-10.6-universal/egg/EGG-INFO zip_safe flag not set; analyzing archive contents... creating 'dist/avro-1.6.1-py2.6.egg' and adding 'build/bdist.macosx-10.6-universal/egg' to it removing 'build/bdist.macosx-10.6-universal/egg' (and everything under it) Processing avro-1.6.1-py2.6.egg Removing /Library/Python/2.6/site-packages/avro-1.6.1-py2.6.egg Copying avro-1.6.1-py2.6.egg to /Library/Python/2.6/site-packages avro 1.6.1 is already the active version in easy-install.pth Installing avro script to /usr/local/bin Installed /Library/Python/2.6/site-packages/avro-1.6.1-py2.6.egg Processing dependencies for avro==1.6.1 Searching for python-snappy Reading http://pypi.python.org/simple/python-snappy/ Reading http://github.com/andrix/python-snappy Best match: python-snappy 0.3.2 Downloading http://pypi.python.org/packages/source/p/python-snappy/python-snappy-0.3.2.tar.gz#md5=94ec3eb54a780fac3b15a6c141af973f Processing python-snappy-0.3.2.tar.gz Running python-snappy-0.3.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-jARda8/python-snappy-0.3.2/egg-dist-tmp-DDcMTW cc1plus: warning: command line option "-Wstrict-prototypes" is valid for C/ObjC but not for C++ snappymodule.cc:41: warning: ‘_state’ defined but not used /usr/libexec/gcc/powerpc-apple-darwin10/4.2.1/as: assembler (/usr/bin/../libexec/gcc/darwin/ppc/as or /usr/bin/../local/libexec/gcc/darwin/ppc/as) for architecture ppc not installed Installed assemblers are: /usr/bin/../libexec/gcc/darwin/x86_64/as for architecture x86_64 /usr/bin/../libexec/gcc/darwin/i386/as for architecture i386 cc1plus: warning: command line option "-Wstrict-prototypes" is valid for C/ObjC but not for C++ snappymodule.cc:41: warning: ‘_state’ defined but not used snappymodule.cc:246: fatal error: error writing to -: Broken pipe compilation terminated. cc1plus: warning: command line option "-Wstrict-prototypes" is valid for C/ObjC but not for C++ snappymodule.cc:41: warning: ‘_state’ defined but not used lipo: can't open input file: /var/tmp//cc1RAU1Q.out (No such file or directory) error: Setup script exited with error: command 'gcc-4.2' failed with exit status 1 Note that I have to run as sudo, otherwise I get: Installing avro script to /usr/local/bin error: /usr/local/bin/avro: Permission denied I've added my vote to https://issues.apache.org/jira/browse/AVRO-981, and a comment about why I think it's good to fix this issue. In the meantime, we're going to use Avro 1.5.3 Thanks, -- Ken On Dec 14, 2011, at 7:07pm, Russell Jurney wrote: I installed the snappy macport, and I still can't get snappy-python to build. I'll try switching to brew. On Wed, Dec 14, 2011 at 5:46 PM, Joe Crobak <joec...@gmail.com> wrote: > It appears that the python-snappy library assumes that the snappy library > has already been installed. I saw similar errors, but once I installed > snappy (I use homebrew, so 'brew install snappy') python-snappy installs > fine. > > HTH, > Joe > > > On Wed, Dec 14, 2011 at 8:23 PM, Russell Jurney > <russell.jur...@gmail.com>wrote: > >> I am unable to build Avro for Python on OS X 10.6.8 because python-snappy >> fails to build. Is there a way around this? >> >> I wrote a post here >> http://datasyndrome.com/post/13707537045/booting-the-analytics-application-events-rubyabout >> how to use Avro with Pig and Ruby. The code is here: >> https://github.com/rjurney/Booting-the-Analytics-Application I am >> trying to create Python instructions that parallel the Ruby ones. >> >> I've tried multiple install methods, and I am unable to build >> snappy-python, so the avro build fails. This is the error message: >> >> Installed >> /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/avro-_AVRO_VERSION_-py2.6.egg >> Processing dependencies for avro==-AVRO-VERSION- >> Searching for python-snappy >> Reading http://pypi.python.org/simple/python-snappy/ >> Reading http://github.com/andrix/python-snappy >> Best match: python-snappy 0.3.2 >> Downloading >> http://pypi.python.org/packages/source/p/python-snappy/python-snappy-0.3.2.tar.gz#md5=94ec3eb54a780fac3b15a6c141af973f >> Processing python-snappy-0.3.2.tar.gz >> Running python-snappy-0.3.2/setup.py -q bdist_egg --dist-dir >> /tmp/easy_install-0KEFAv/python-snappy-0.3.2/egg-dist-tmp-1BPj3j >> cc1plus: warning: command line option "-Wstrict-prototypes" is valid for >> C/ObjC but not for C++ >> snappymodule.cc:31:22: error: snappy-c.h: No such file or directory >> snappymodule.cc: In function ‘PyObject* snappy__compress(PyObject*, >> PyObject*)’: >> snappymodule.cc:62: error: ‘snappy_status’ was not declared in this scope >> snappymodule.cc:62: error: expected `;' before ‘status’ >> snappymodule.cc:75: error: ‘snappy_max_compressed_length’ was not >> declared in this scope >> snappymodule.cc:79: error: ‘status’ was not declared in this scope >> snappymodule.cc:79: error: ‘snappy_compress’ was not declared in this >> scope >> snappymodule.cc:81: error: ‘SNAPPY_OK’ was not declared in this scope >> snappymodule.cc: In function ‘PyObject* snappy__uncompress(PyObject*, >> PyObject*)’: >> snappymodule.cc:107: error: ‘snappy_status’ was not declared in this scope >> snappymodule.cc:107: error: expected `;' before ‘status’ >> snappymodule.cc:120: error: ‘status’ was not declared in this scope >> snappymodule.cc:120: error: ‘snappy_uncompressed_length’ was not declared >> in this scope >> snappymodule.cc:121: error: ‘SNAPPY_OK’ was not declared in this scope >> snappymodule.cc:128: error: ‘snappy_uncompress’ was not declared in this >> scope >> snappymodule.cc:129: error: ‘SNAPPY_OK’ was not declared in this scope >> snappymodule.cc: In function ‘PyObject* >> snappy__is_valid_compressed_buffer(PyObject*, PyObject*)’: >> snappymodule.cc:151: error: ‘snappy_status’ was not declared in this scope >> snappymodule.cc:151: error: expected `;' before ‘status’ >> snappymodule.cc:156: error: ‘status’ was not declared in this scope >> snappymodule.cc:156: error: ‘snappy_validate_compressed_buffer’ was not >> declared in this scope >> snappymodule.cc:157: error: ‘SNAPPY_OK’ was not declared in this scope >> snappymodule.cc: At global scope: >> snappymodule.cc:41: warning: ‘_state’ defined but not used >> error: Setup script exited with error: command '/usr/bin/gcc-4.2' failed >> with exit status 1 >> >> -- >> > -------------------------- Ken Krugler http://www.scaleunlimited.com custom big data solutions & training Hadoop, Cascading, Mahout & Solr
