I remember some discussion about making the accelerated the new default and ripping the old one entirely out. The argument was sth. along "why should I use the old one at all if I can use accel?" and that sounds logical to me. Obviously nobody did it, so maybe it's time to complete that now.
I dont know if theres a ticket, but "accelerated" gives a good search term ;-) Have fun, JensG ________________________________ Von: Jim King Gesendet: 17.06.2016 21:20 An: dev@thrift.apache.org Betreff: RE: Performance issue Is there a Jira ticket for this? -----Original Message----- From: glennpierce.w...@gmail.com [mailto:glennpierce.w...@gmail.com] On Behalf Of Glenn Pierce Sent: Thursday, June 16, 2016 3:52 PM To: dev@thrift.apache.org Subject: Re: Performance issue Wow changed to TBinaryProtocolAccelerated and it went down to 1.5 seconds. Something very wrong with the default python implementation. On 16 June 2016 at 20:34, Glenn Pierce <glenn.pie...@carnegosystems.com> wrote: > The important profile lines are > > ncalls <http://carneab4.memset.net:8000/?sort=calls> tottime > <http://carneab4.memset.net:8000/?sort=time> percall cumtime > <http://carneab4.memset.net:8000/?sort=cumulative> percall filename > <http://carneab4.memset.net:8000/?sort=module>:lineno > <http://carneab4.memset.net:8000/?sort=nfl>(function) > 1 0.006 0.006 963.335 963.335 > ./ThriftTestServer.py:3(<module> > <http://carneab4.memset.net:8000/?func_name=%3Cmodule%3E>) > 1 0.000 0.000 963.304 963.304 > /usr/lib64/python2.7/site-packages/thrift/server/TServer.py:102(serve > <http://carneab4.memset.net:8000/?func_name=serve>) > 2 0.000 0.000 963.303 481.652 > /usr/lib64/python2.7/site-packages/thrift/transport/TSocket.py:188(accept > <http://carneab4.memset.net:8000/?func_name=accept>) > 2 0.000 0.000 963.303 481.652 > /usr/lib64/python2.7/socket.py:201(accept > <http://carneab4.memset.net:8000/?func_name=accept>) > 2 963.303 481.652 963.303 481.652 {method 'accept' of > '_socket.socket' objects} > > > > On 16 June 2016 at 20:01, Glenn Pierce > <glenn.pie...@carnegosystems.com> > wrote: > >> That code is only called once. >> All the time is spent after execution leaves my server function. So >> it is somewhere in the generated thrift code. >> >> Just to be certain I generated the test data at the start of the >> program like >> >> results = [('2016-05-20T14:01:01+0000', [22222])] * 60000 >> >> But I get the same reults. 2 minutes twenty seconds. >> >> My python server process runs at over 100% cpu. >> >> >> Thanks >> >> >> >> >> >> On 16 June 2016 at 19:23, Randy Abernethy <r...@apache.org> wrote: >> >>> Hi Glenn, >>> >>> While two minutes is a little surprising, I suspect opening/closng >>> the file every time the getData() method is called will be >>> problematic. File open operations are usually an order of magnitude >>> more expensive than file read or write operations. Try opening the >>> file in the handler constructor, buffering it, closing the file and >>> then responding to getData requests using the buffer. >>> >>> -Randy >>> >>> >>> On Thu, Jun 16, 2016 at 9:08 AM, Glenn Pierce < >>> glenn.pie...@carnegosystems.com> wrote: >>> >>> > Hi I wonder if someone could give advice on a performance issue I have. >>> > >>> > I have written a test case in Python >>> > >>> > The server is >>> > >>> > class TestServerThriftHandler(object): >>> > >>> > def getData(self): >>> > with open('data.json', 'r') as f: >>> > results = ujson.loads(f.read()) >>> > results = [(r['timestamp'], r['values']) for r in >>> results] >>> > print "size of list", len(results) >>> > data = [ThriftTestGroup(timestamp=row[0], >>> > values=[float(x) if x is >>> > not >>> None >>> > else float('NaN') for x in row[1]]) for row in results] >>> > >>> > return data >>> > >>> > >>> > if __name__ == "__main__": >>> > handler = TestServerThriftHandler() >>> > processor = ThriftTestService.Processor(handler) >>> > transport = TSocket.TServerSocket("127.0.0.1", 7777) >>> > tfactory = TTransport.TBufferedTransportFactory() >>> > pfactory = TBinaryProtocol.TBinaryProtocolFactory() >>> > server = TServer.TThreadedServer(processor, transport, >>> > tfactory, >>> > pfactory) >>> > print 'Starting server' >>> > server.serve() >>> > >>> > >>> > The interface is >>> > >>> > struct ThriftTestGroup { >>> > >>> > 1: required Timestamp timestamp; >>> > 2: required list<double> values; } >>> > >>> > service ThriftTestService { >>> > >>> > list<ThriftTestGroup> getData() throws ( 1:TestException ex); } >>> > >>> > >>> > Basically getData() loads some example 58773 entries into a list from a >>> > file. >>> > Each entry is a string representing the time like >>> > 2016-05-20T13:58:59+0000 and one or more double values. >>> > In this test there is only one double value. >>> > >>> > My client on the same host calls this method and it takes 2 minutes 14 >>> > seconds ? >>> > My code loading the data in the server in instant. >>> > >>> > Does anyone have an idea why this is so slow ? >>> > >>> > Any advice would be great. >>> > >>> > Thanks >>> > >>> >> >> > --------------------------------------------------------------------------------------- PRIVACY STATEMENT: This message is a PRIVATE communication. This message and all attachments are a private communication sent by SimpliVity and are considered to be confidential or protected by privilege. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the information contained in or attached to this message is strictly prohibited. Please notify the sender of the delivery error by replying to this message, and then delete it from your system. ---------------------------------------------------------------------------------------
