Thanks for the information Sanjit. -- Regards,
Sreejith K On Thu, Jul 1, 2010 at 8:09 PM, Sanjit Jhala <[email protected]> wrote: > Looking at this > post<http://mail-archives.apache.org/mod_mbox/incubator-thrift-user/200911.mbox/%[email protected]%3e>it > looks like Thrift expects thread safety to be implemented in the > application layer. So if you protect the client calls with a mutex that > would work too and shouldn't have any more of a performance hit than if the > locks were pushed into the generated thrift code. > > -Sanjit > > > On Thu, Jul 1, 2010 at 7:26 AM, Sanjit Jhala <[email protected]> wrote: > >> You're right, it looks like the thrift client isn't threadsafe. I'll take >> a look at whether theres a thrift knob to generate threadsafe client code. >> There shouldn't be any other reason that the Hypertable thrift client >> shouldn't be thread safe. >> >> -Sanjit >> >> >> >> On Thu, Jul 1, 2010 at 2:48 AM, Sreejith K <[email protected]> wrote: >> >>> Hi Sanjit, >>> >>> So, the only way to use ThriftClient in a multithreaded application is to >>> instantiate the client in all the threads. Any plans to make it thread-safe >>> ? Just curious to know why it is behaving like this. >>> >>> -- >>> Regards, >>> >>> Sreejith K >>> >>> >>> >>> On Thu, Jul 1, 2010 at 9:52 AM, Sreejith K <[email protected]>wrote: >>> >>>> Thanks for the reply Sanjit. That seems to have solved the issue. >>>> >>>> -- >>>> Regards, >>>> >>>> Sreejith K >>>> >>>> >>>> >>>> On Thu, Jul 1, 2010 at 12:29 AM, Sanjit Jhala <[email protected]>wrote: >>>> >>>>> Hi Sreejith, >>>>> >>>>> Instead of sharing the ThriftClient across all the threads, try >>>>> creating one ThriftClient per thread (ie 'push "client = ThriftClient( >>>>> 'localhost', 38080)" ' into the 'Get' method in your program), >>>>> >>>>> -Sanjit >>>>> >>>>> >>>>> >>>>> On Wed, Jun 30, 2010 at 12:45 AM, Sreejith K <[email protected]>wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Actually the failed open_scanner/next_cells calls are being called >>>>>> from newly spawned threads. I wrote a sample program to reproduce this. >>>>>> See >>>>>> the program and its output. >>>>>> >>>>>> Hypertable content: http://pastie.org/1024681 >>>>>> Program: http://pastie.org/1024685 >>>>>> Output: http://pastie.org/1024684 >>>>>> >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> >>>>>> Sreejith K >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Jun 30, 2010 at 8:05 AM, Sanjit Jhala <[email protected]>wrote: >>>>>> >>>>>>> And, I assume you're running the ThriftBroker from this 0.9.3.3 build >>>>>>> ? Do you see this error on every open_scanner/next_cells call or some >>>>>>> specific one ? >>>>>>> >>>>>>> -Sanjit >>>>>>> >>>>>>> >>>>>>> On Tue, Jun 29, 2010 at 6:32 AM, Sreejith K >>>>>>> <[email protected]>wrote: >>>>>>> >>>>>>>> Hi Sanjit, >>>>>>>> >>>>>>>> I used thrift-0.3.0-rc4 ( >>>>>>>> http://people.apache.org/~bryanduxbury/thrift-0.3.0-rc4.tar.gz<http://people.apache.org/%7Ebryanduxbury/thrift-0.3.0-rc4.tar.gz>) >>>>>>>> for building the binary packages from hypertable 0.9.3.3 source. I >>>>>>>> installed >>>>>>>> thrift python bindings from thrift-0.3.0-rc4 to >>>>>>>> /usr/local/lib/python2.6/dist-packages/thrift. The cmake output >>>>>>>> confirmed >>>>>>>> the thrift version as 0.3.0 as shown below >>>>>>>> >>>>>>>> s...@codedesk:~/build/hypertable$ cmake -DPACKAGE_THRIFTBROKER= >>>>>>>> -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=Release ~/src/hypertable >>>>>>>> -DJAVA_INCLUDE_PATH=/usr/lib/jvm/java-6-sun-1.6.0.20/include >>>>>>>> -DJAVA_INCLUDE_PATH2=/usr/lib/jvm/java-6-sun-1.6.0.20/include/linux >>>>>>>> -DHADOOP_INCLUDE_PATH=/home/sree/src/hadoop-0.20.2/src/c++/install/include >>>>>>>> -DHADOOP_LIB_PATH=/home/sree/src/hadoop-0.20.2/src/c++/install/lib >>>>>>>> -DPHPTHRIFT_ROOT=/home/sree/src/thrift-0.3.0/lib/php/src >>>>>>>> -- Use thread library: -lpthread >>>>>>>> -- Looking for required boost libraries... >>>>>>>> -- GCC version: 4.4.1 >>>>>>>> -- Boost version: 1_38 >>>>>>>> -- Boost thread lib: /usr/lib/libboost_thread-mt.so >>>>>>>> -- Boost program options lib: >>>>>>>> /usr/lib/libboost_program_options-mt.so >>>>>>>> -- Boost filesystem lib: /usr/lib/libboost_filesystem-mt.so >>>>>>>> -- Boost iostreams lib: /usr/lib/libboost_iostreams-mt.so >>>>>>>> -- Boost python lib: /usr/lib/libboost_python-mt.so >>>>>>>> -- Boost system lib: /usr/lib/libboost_system-mt.so >>>>>>>> -- Boost lib dir: /usr/lib >>>>>>>> -- Found BerkeleyDB: /usr/local/lib/libdb_cxx.so >>>>>>>> -- Berkeley DB version: 4.8.26 >>>>>>>> -- Found RRDtool: /usr/lib/librrd.so >>>>>>>> -- RRDtool version: 1.3.1 >>>>>>>> -- Found Readline libraries: >>>>>>>> /usr/lib/libreadline.so;/usr/lib/libncurses.so >>>>>>>> -- Found SIGAR: /usr/local/lib/libsigar-x86-linux.so;dl >>>>>>>> -- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) >>>>>>>> -- Found Tcmalloc: /usr/local/lib/libtcmalloc_minimal.so >>>>>>>> -- version: 1.5 >>>>>>>> -- Not Found Hoard: Hoard_LIBRARY-NOTFOUND >>>>>>>> -- Did not find ceph libraries >>>>>>>> -- Found Ant: Apache Ant version 1.7.1 compiled on October 19 2009 >>>>>>>> -- Javac: javac 1.6.0_20 >>>>>>>> -- Found libevent: /usr/lib/libevent.so >>>>>>>> -- Found thrift: >>>>>>>> /usr/local/lib/libthrift.so;/usr/local/lib/libthriftnb.so >>>>>>>> -- compiler: Thrift version 0.3.0 >>>>>>>> -- Thrift for ruby not found. ThriftBroker support for ruby will be >>>>>>>> disabled >>>>>>>> -- Found thrift for php: /home/sree/src/thrift-0.3.0/lib/php/src >>>>>>>> -- Found thrift for perl >>>>>>>> -- Found thrift for python >>>>>>>> -- Java headers found at: /usr/lib/jvm/java-6-sun-1.6.0.20/include >>>>>>>> -- Hadoop includes located at: >>>>>>>> /home/sree/src/hadoop-0.20.2/src/c++/install/include >>>>>>>> -- Hadoop libraries located at: >>>>>>>> /home/sree/src/hadoop-0.20.2/src/c++/install/lib >>>>>>>> -- Configuring done >>>>>>>> -- Generating done >>>>>>>> -- Build files have been written to: /home/sree/build/hypertable >>>>>>>> >>>>>>>> The only strange thing I noticed was the version for thrift bindings >>>>>>>> for python. It says 0.1 >>>>>>>> >>>>>>>> s...@codedesk:~/build/hypertable$ cat >>>>>>>> /usr/local/lib/python2.6/dist-packages/Thrift-0.1.egg-info >>>>>>>> Metadata-Version: 1.0 >>>>>>>> Name: Thrift >>>>>>>> Version: 0.1 >>>>>>>> Summary: Thrift Python Libraries >>>>>>>> Home-page: http://incubator.apache.org/thrift/ >>>>>>>> Author: ['Thrift Developers'] >>>>>>>> Author-email: ['[email protected]'] >>>>>>>> License: Apache License 2.0 >>>>>>>> Description: UNKNOWN >>>>>>>> Platform: UNKNOWN >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Regards, >>>>>>>> >>>>>>>> Sreejith K >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Jun 29, 2010 at 12:19 AM, Sanjit Jhala >>>>>>>> <[email protected]>wrote: >>>>>>>> >>>>>>>>> Are the ThriftBroker and ThriftClient code generated by the same >>>>>>>>> version of the code and Thrift? From the error message it sounds like >>>>>>>>> theres >>>>>>>>> a mismatch in the protocol. >>>>>>>>> >>>>>>>>> -Sanjit >>>>>>>>> >>>>>>>>> On Mon, Jun 28, 2010 at 1:06 AM, Sreejith K <[email protected] >>>>>>>>> > wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> I'm using latest Hypertable thriftclient for Python from version >>>>>>>>>> 0.9.3.3 (built dynamically with Thrift package from >>>>>>>>>> http://people.apache.org/~bryanduxbury/thrift-0.3.0-rc4.tar.gz<http://people.apache.org/%7Ebryanduxbury/thrift-0.3.0-rc4.tar.gz>). >>>>>>>>>> But when using open_scanner and next_cells api calls its raising >>>>>>>>>> TApplicationException and sometimes EOFError. >>>>>>>>>> >>>>>>>>>> TApplicationException: next_cells failed: unknown result >>>>>>>>>> TApplicationException: open_scanner failed: unknown result >>>>>>>>>> >>>>>>>>>> Unfortunately I'm unable to reproduce this error using a sample >>>>>>>>>> script. What could be the reason for thriftclient to raise such >>>>>>>>>> errors ? >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Regards, >>>>>>>>>> >>>>>>>>>> Sreejith K >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>>> Groups "Hypertable Development" group. >>>>>>>>>> To post to this group, send email to >>>>>>>>>> [email protected]. >>>>>>>>>> To unsubscribe from this group, send email to >>>>>>>>>> [email protected]<hypertable-dev%[email protected]> >>>>>>>>>> . >>>>>>>>>> For more options, visit this group at >>>>>>>>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "Hypertable Development" group. >>>>>>>>> To post to this group, send email to >>>>>>>>> [email protected]. >>>>>>>>> To unsubscribe from this group, send email to >>>>>>>>> [email protected]<hypertable-dev%[email protected]> >>>>>>>>> . >>>>>>>>> For more options, visit this group at >>>>>>>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Hypertable Development" group. >>>>>>>> To post to this group, send email to >>>>>>>> [email protected]. >>>>>>>> To unsubscribe from this group, send email to >>>>>>>> [email protected]<hypertable-dev%[email protected]> >>>>>>>> . >>>>>>>> For more options, visit this group at >>>>>>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Hypertable Development" group. >>>>>>> To post to this group, send email to [email protected] >>>>>>> . >>>>>>> To unsubscribe from this group, send email to >>>>>>> [email protected]<hypertable-dev%[email protected]> >>>>>>> . >>>>>>> For more options, visit this group at >>>>>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Hypertable Development" group. >>>>>> To post to this group, send email to [email protected]. >>>>>> To unsubscribe from this group, send email to >>>>>> [email protected]<hypertable-dev%[email protected]> >>>>>> . >>>>>> For more options, visit this group at >>>>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Hypertable Development" group. >>>>> To post to this group, send email to [email protected]. >>>>> To unsubscribe from this group, send email to >>>>> [email protected]<hypertable-dev%[email protected]> >>>>> . >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>>> >>>> >>>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Hypertable Development" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]<hypertable-dev%[email protected]> >>> . >>> For more options, visit this group at >>> http://groups.google.com/group/hypertable-dev?hl=en. >>> >> >> > -- > You received this message because you are subscribed to the Google Groups > "Hypertable Development" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<hypertable-dev%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/hypertable-dev?hl=en. > -- You received this message because you are subscribed to the Google Groups "Hypertable Development" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en.
