You're right, it looks like the thrift client isn't threadsafe. I'll take a look at whether theres a thrift knob to generate threadsafe client code. There shouldn't be any other reason that the Hypertable thrift client shouldn't be thread safe.
-Sanjit On Thu, Jul 1, 2010 at 2:48 AM, Sreejith K <[email protected]> wrote: > Hi Sanjit, > > So, the only way to use ThriftClient in a multithreaded application is to > instantiate the client in all the threads. Any plans to make it thread-safe > ? Just curious to know why it is behaving like this. > > -- > Regards, > > Sreejith K > > > > On Thu, Jul 1, 2010 at 9:52 AM, Sreejith K <[email protected]> wrote: > >> Thanks for the reply Sanjit. That seems to have solved the issue. >> >> -- >> Regards, >> >> Sreejith K >> >> >> >> On Thu, Jul 1, 2010 at 12:29 AM, Sanjit Jhala <[email protected]> wrote: >> >>> Hi Sreejith, >>> >>> Instead of sharing the ThriftClient across all the threads, try creating >>> one ThriftClient per thread (ie 'push "client = ThriftClient('localhost', >>> 38080)" ' into the 'Get' method in your program), >>> >>> -Sanjit >>> >>> >>> >>> On Wed, Jun 30, 2010 at 12:45 AM, Sreejith K <[email protected]>wrote: >>> >>>> Hi, >>>> >>>> Actually the failed open_scanner/next_cells calls are being called from >>>> newly spawned threads. I wrote a sample program to reproduce this. See the >>>> program and its output. >>>> >>>> Hypertable content: http://pastie.org/1024681 >>>> Program: http://pastie.org/1024685 >>>> Output: http://pastie.org/1024684 >>>> >>>> >>>> -- >>>> Regards, >>>> >>>> Sreejith K >>>> >>>> >>>> >>>> On Wed, Jun 30, 2010 at 8:05 AM, Sanjit Jhala <[email protected]>wrote: >>>> >>>>> And, I assume you're running the ThriftBroker from this 0.9.3.3 build ? >>>>> Do you see this error on every open_scanner/next_cells call or some >>>>> specific >>>>> one ? >>>>> >>>>> -Sanjit >>>>> >>>>> >>>>> On Tue, Jun 29, 2010 at 6:32 AM, Sreejith K <[email protected]>wrote: >>>>> >>>>>> Hi Sanjit, >>>>>> >>>>>> I used thrift-0.3.0-rc4 ( >>>>>> http://people.apache.org/~bryanduxbury/thrift-0.3.0-rc4.tar.gz<http://people.apache.org/%7Ebryanduxbury/thrift-0.3.0-rc4.tar.gz>) >>>>>> for building the binary packages from hypertable 0.9.3.3 source. I >>>>>> installed >>>>>> thrift python bindings from thrift-0.3.0-rc4 to >>>>>> /usr/local/lib/python2.6/dist-packages/thrift. The cmake output confirmed >>>>>> the thrift version as 0.3.0 as shown below >>>>>> >>>>>> s...@codedesk:~/build/hypertable$ cmake -DPACKAGE_THRIFTBROKER= >>>>>> -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=Release ~/src/hypertable >>>>>> -DJAVA_INCLUDE_PATH=/usr/lib/jvm/java-6-sun-1.6.0.20/include >>>>>> -DJAVA_INCLUDE_PATH2=/usr/lib/jvm/java-6-sun-1.6.0.20/include/linux >>>>>> -DHADOOP_INCLUDE_PATH=/home/sree/src/hadoop-0.20.2/src/c++/install/include >>>>>> -DHADOOP_LIB_PATH=/home/sree/src/hadoop-0.20.2/src/c++/install/lib >>>>>> -DPHPTHRIFT_ROOT=/home/sree/src/thrift-0.3.0/lib/php/src >>>>>> -- Use thread library: -lpthread >>>>>> -- Looking for required boost libraries... >>>>>> -- GCC version: 4.4.1 >>>>>> -- Boost version: 1_38 >>>>>> -- Boost thread lib: /usr/lib/libboost_thread-mt.so >>>>>> -- Boost program options lib: /usr/lib/libboost_program_options-mt.so >>>>>> -- Boost filesystem lib: /usr/lib/libboost_filesystem-mt.so >>>>>> -- Boost iostreams lib: /usr/lib/libboost_iostreams-mt.so >>>>>> -- Boost python lib: /usr/lib/libboost_python-mt.so >>>>>> -- Boost system lib: /usr/lib/libboost_system-mt.so >>>>>> -- Boost lib dir: /usr/lib >>>>>> -- Found BerkeleyDB: /usr/local/lib/libdb_cxx.so >>>>>> -- Berkeley DB version: 4.8.26 >>>>>> -- Found RRDtool: /usr/lib/librrd.so >>>>>> -- RRDtool version: 1.3.1 >>>>>> -- Found Readline libraries: >>>>>> /usr/lib/libreadline.so;/usr/lib/libncurses.so >>>>>> -- Found SIGAR: /usr/local/lib/libsigar-x86-linux.so;dl >>>>>> -- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) >>>>>> -- Found Tcmalloc: /usr/local/lib/libtcmalloc_minimal.so >>>>>> -- version: 1.5 >>>>>> -- Not Found Hoard: Hoard_LIBRARY-NOTFOUND >>>>>> -- Did not find ceph libraries >>>>>> -- Found Ant: Apache Ant version 1.7.1 compiled on October 19 2009 >>>>>> -- Javac: javac 1.6.0_20 >>>>>> -- Found libevent: /usr/lib/libevent.so >>>>>> -- Found thrift: >>>>>> /usr/local/lib/libthrift.so;/usr/local/lib/libthriftnb.so >>>>>> -- compiler: Thrift version 0.3.0 >>>>>> -- Thrift for ruby not found. ThriftBroker support for ruby will be >>>>>> disabled >>>>>> -- Found thrift for php: /home/sree/src/thrift-0.3.0/lib/php/src >>>>>> -- Found thrift for perl >>>>>> -- Found thrift for python >>>>>> -- Java headers found at: /usr/lib/jvm/java-6-sun-1.6.0.20/include >>>>>> -- Hadoop includes located at: >>>>>> /home/sree/src/hadoop-0.20.2/src/c++/install/include >>>>>> -- Hadoop libraries located at: >>>>>> /home/sree/src/hadoop-0.20.2/src/c++/install/lib >>>>>> -- Configuring done >>>>>> -- Generating done >>>>>> -- Build files have been written to: /home/sree/build/hypertable >>>>>> >>>>>> The only strange thing I noticed was the version for thrift bindings >>>>>> for python. It says 0.1 >>>>>> >>>>>> s...@codedesk:~/build/hypertable$ cat >>>>>> /usr/local/lib/python2.6/dist-packages/Thrift-0.1.egg-info >>>>>> Metadata-Version: 1.0 >>>>>> Name: Thrift >>>>>> Version: 0.1 >>>>>> Summary: Thrift Python Libraries >>>>>> Home-page: http://incubator.apache.org/thrift/ >>>>>> Author: ['Thrift Developers'] >>>>>> Author-email: ['[email protected]'] >>>>>> License: Apache License 2.0 >>>>>> Description: UNKNOWN >>>>>> Platform: UNKNOWN >>>>>> >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> >>>>>> Sreejith K >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Jun 29, 2010 at 12:19 AM, Sanjit Jhala <[email protected]>wrote: >>>>>> >>>>>>> Are the ThriftBroker and ThriftClient code generated by the same >>>>>>> version of the code and Thrift? From the error message it sounds like >>>>>>> theres >>>>>>> a mismatch in the protocol. >>>>>>> >>>>>>> -Sanjit >>>>>>> >>>>>>> On Mon, Jun 28, 2010 at 1:06 AM, Sreejith K >>>>>>> <[email protected]>wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I'm using latest Hypertable thriftclient for Python from version >>>>>>>> 0.9.3.3 (built dynamically with Thrift package from >>>>>>>> http://people.apache.org/~bryanduxbury/thrift-0.3.0-rc4.tar.gz<http://people.apache.org/%7Ebryanduxbury/thrift-0.3.0-rc4.tar.gz>). >>>>>>>> But when using open_scanner and next_cells api calls its raising >>>>>>>> TApplicationException and sometimes EOFError. >>>>>>>> >>>>>>>> TApplicationException: next_cells failed: unknown result >>>>>>>> TApplicationException: open_scanner failed: unknown result >>>>>>>> >>>>>>>> Unfortunately I'm unable to reproduce this error using a sample >>>>>>>> script. What could be the reason for thriftclient to raise such errors >>>>>>>> ? >>>>>>>> >>>>>>>> -- >>>>>>>> Regards, >>>>>>>> >>>>>>>> Sreejith K >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Hypertable Development" group. >>>>>>>> To post to this group, send email to >>>>>>>> [email protected]. >>>>>>>> To unsubscribe from this group, send email to >>>>>>>> [email protected]<hypertable-dev%[email protected]> >>>>>>>> . >>>>>>>> For more options, visit this group at >>>>>>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Hypertable Development" group. >>>>>>> To post to this group, send email to [email protected] >>>>>>> . >>>>>>> To unsubscribe from this group, send email to >>>>>>> [email protected]<hypertable-dev%[email protected]> >>>>>>> . >>>>>>> For more options, visit this group at >>>>>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Hypertable Development" group. >>>>>> To post to this group, send email to [email protected]. >>>>>> To unsubscribe from this group, send email to >>>>>> [email protected]<hypertable-dev%[email protected]> >>>>>> . >>>>>> For more options, visit this group at >>>>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Hypertable Development" group. >>>>> To post to this group, send email to [email protected]. >>>>> To unsubscribe from this group, send email to >>>>> [email protected]<hypertable-dev%[email protected]> >>>>> . >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Hypertable Development" group. >>>> To post to this group, send email to [email protected]. >>>> To unsubscribe from this group, send email to >>>> [email protected]<hypertable-dev%[email protected]> >>>> . >>>> For more options, visit this group at >>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Hypertable Development" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]<hypertable-dev%[email protected]> >>> . >>> For more options, visit this group at >>> http://groups.google.com/group/hypertable-dev?hl=en. >>> >> >> > -- > You received this message because you are subscribed to the Google Groups > "Hypertable Development" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<hypertable-dev%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/hypertable-dev?hl=en. > -- You received this message because you are subscribed to the Google Groups "Hypertable Development" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en.
