Yeah increasing read time out to 60 seconds worked out for me! I am putting 1000 objects with putAll with 7 functional indexes being created synchronously!
Thanks, Dharam - Dharam Thacker On Tue, Dec 20, 2016 at 12:39 AM, Dan Smith <[email protected]> wrote: > You might consider upping the read-timeout on the client to see if that > causes the issue to go away. The default read-timeout is 10 seconds. So if > it's really just an issue of the server being slow and not responding in > time a larger read timeout would help. > > How much data are you putting with this putAll? > > -Dan > > On Mon, Dec 19, 2016 at 10:02 AM, Thacker, Dharam < > [email protected]> wrote: > >> Not sure if this is related with past bug as per link? There are few more >> archive mails matching to same exception for jiras worked out in past! >> >> >> >> http://markmail.org/message/oefrls7bdof776t7?q=ServerConnect >> ivityException&page=2 >> >> >> >> I have 7 functional indexes on server side for same region due to which >> insertion happens slow, but that works fine if I mark region just as >> REPLICATE. >> >> >> >> Regards, >> >> Dharam >> >> >> >> *From:* Thacker, Dharam >> *Sent:* Monday, December 19, 2016 10:51 PM >> >> *To:* [email protected] >> *Subject:* RE: Socket timeout after making regions persistent in async >> way >> >> >> >> Yeah forgot to mention that it’s the only client connecting to server to >> load region shown below! No high io/cpu consuming activity running on >> server side! >> >> >> >> *From:* Thacker, Dharam >> *Sent:* Monday, December 19, 2016 10:46 PM >> *To:* [email protected] >> *Subject:* RE: Socket timeout after making regions persistent in async >> way >> >> >> >> Yeah sure! >> >> >> >> Client config looks simple. I have not enabled redundant copies. More >> region is of type REPLICATE_PERSISTENT on server side. Server is just >> initialized with –Xms = 20G and –Xmx = 30G with ConcMarkSweep GC along with >> few more tuning parameters which works descent though! >> >> >> >> *Server logs:* >> >> >> >> [warn 2016/12/19 22:42:13.254 IST <ServerConnection on port 40404 Thread >> 2> tid=0x5b] Server connection from >> [identity(XXXXX(11700:loner):63652:3b290e18,connection=2; >> port=63657]: Unexpected *IOException*: >> >> *java.net.SocketException*: Software caused connection abort: socket >> write error >> >> at java.net.SocketOutputStream.socketWrite0(*Native Method*) >> >> at java.net.SocketOutputStream.socketWrite( >> *SocketOutputStream.java:109*) >> >> at java.net.SocketOutputStream.write(*SocketOutputStream.java:153* >> ) >> >> at org.apache.geode.internal.cache.tier.sockets.Message.flushBu >> ffer(*Message.java:649*) >> >> at org.apache.geode.internal.cache.tier.sockets.Message.sendBytes( >> *Message.java:627*) >> >> at org.apache.geode.internal.cache.tier.sockets.ChunkedMessage. >> sendChunk(*ChunkedMessage.java:314*) >> >> at org.apache.geode.internal.cache.tier.sockets.ChunkedMessage. >> sendChunk(*ChunkedMessage.java:322*) >> >> at org.apache.geode.internal.cache.tier.sockets.command.PutAll8 >> 0.writeReply(*PutAll80.java:388*) >> >> at org.apache.geode.internal.cache.tier.sockets.command.PutAll8 >> 0.cmdExecute(*PutAll80.java:339*) >> >> at org.apache.geode.internal.cache.tier.sockets.BaseCommand. >> execute(*BaseCommand.java:147*) >> >> at org.apache.geode.internal.cache.tier.sockets.ServerConnectio >> n.doNormalMsg(*ServerConnection.java:783*) >> >> at org.apache.geode.internal.cache.tier.sockets.ServerConnectio >> n.doOneMessage(*ServerConnection.java:913*) >> >> at org.apache.geode.internal.cache.tier.sockets.ServerConnectio >> n.run(*ServerConnection.java:1180*) >> >> at java.util.concurrent.ThreadPoolExecutor.runWorker( >> *ThreadPoolExecutor.java:1142*) >> >> at java.util.concurrent.ThreadPoolExecutor$Worker.run( >> *ThreadPoolExecutor.java:617*) >> >> at org.apache.geode.internal.cache.tier.sockets.AcceptorImpl$1$ >> 1.run(*AcceptorImpl.java:546*) >> >> at java.lang.Thread.run(*Thread.java:745*) >> >> >> >> >> >> *Client config:* >> >> <gfe:pool id="event-geode-pool" subscription-enabled="true" >> server-group="EVENT_GRP"> >> >> <gfe:localtor...> >> >> </gfe:pool> >> >> >> >> <gfe:client-region id="Event" >> >> cache-ref="event-cache" >> >> pool-name="event-geode-pool" >> >> shortcut="PROXY" >> >> key-constraint="java.lang.String" >> >> value-constraint="a.m.n.Event"> >> >> </gfe:client-region> >> >> >> >> >> >> <gfe:cq-listener-container id="cqListenerContainer" >> cache="event-cache"> >> >> <gfe:listener ref="eventListener" >> query="SELECT * FROM /Event WHERE status='PENDING'"/> >> >> </gfe:cq-listener-container> >> >> >> >> <bean id="eventListener" class="a.m.n.cqlisteners.MyEve >> ntListener"/> >> >> >> >> Thanks, >> >> Dharam >> >> *From:* Udo Kohlmeyer [mailto:[email protected] >> <[email protected]>] >> *Sent:* Monday, December 19, 2016 10:26 PM >> *To:* [email protected] >> *Subject:* Re: Socket timeout after making regions persistent in async >> way >> >> >> >> Hi there Dharam, >> >> It is possible that persistence has an effect on the performance of the >> system. But generally socket timeouts occur due to other issues like GC or >> system load. >> >> What is the server load around the time of the disconnection? Have you >> enabled redundant copies for your cq? >> >> Could you please provide your client configuration for this problem. >> >> Maybe even a snippet of the server logs around the time of the client >> disconnection. (a minute either side of the client disconnection is a good >> start). >> >> --Udo >> >> >> >> On 12/19/16 05:22, Thacker, Dharam wrote: >> >> Hi, >> >> >> >> Has anyone faced below exceptions while cqlistener thread is working on? >> It started coming once after I changes regions to be persistent. Am I >> missing anything related to socket timeout? >> >> >> >> <geode:disk-store id=*"my_diskstore"* cache-ref=*"geodeCache"* >> >> compaction-threshold=*"50"* >> >> auto-compact=*"false"* >> >> allow-force-compaction=*"true"* >> >> max-oplog-size=*"512"* >> >> queue-size=*"10000"* >> >> time-interval=*"500"* >> >> write-buffer-size=*"65536"* >> >> disk-usage-warning-percentage=*"80"* >> >> disk-usage-critical-percentage=*"98"*> >> >> <geode:disk-dir location=*"mydata"* max-size=*"25600"*/> >> >> </geode:disk-store> >> >> >> >> It tells me that server is unreachable but bouncing client connects to >> server again but after some interval of activity with server, again throws >> back below exception! >> >> >> >> [warn 2016/12/19 18:43:22.608 IST <cqListenerContainer-1> tid=0x6c] Pool >> unexpected socket timed out on client connection=Pooled Connection to >> XXXX:40404: Connection[XXXX:40404]@1743344121). Server unreachable: >> could not connect after 1 attempts >> >> >> >> [cqListenerContainer-1] ERROR o.s.d.g.l.a.ContinuousQueryListenerAdapter >> Listener execution failed... >> >> *org.apache.geode.cache.client.ServerConnectivityException*: Pool >> unexpected socket timed out on client connection=Pooled Connection to >> XXXX:40404: Connection[XXXX:40404]@1743344121). Server unreachable: >> could not connect after 1 attempts >> >> at org.apache.geode.cache.client.internal.OpExecutorImpl.handle >> Exception(*OpExecutorImpl.java:819*) >> >> at org.apache.geode.cache.client.internal.OpExecutorImpl.handle >> Exception(*OpExecutorImpl.java:603*) >> >> at org.apache.geode.cache.client.internal.OpExecutorImpl.execute( >> *OpExecutorImpl.java:173*) >> >> at org.apache.geode.cache.client.internal.OpExecutorImpl.execute( >> *OpExecutorImpl.java:110*) >> >> at org.apache.geode.cache.client.internal.PoolImpl.execute( >> *PoolImpl.java:697*) >> >> at org.apache.geode.cache.client.internal.PutAllOp.execute( >> *PutAllOp.java:110*) >> >> at org.apache.geode.cache.client.internal.ServerRegionProxy.put >> All(*ServerRegionProxy.java:643*) >> >> at org.apache.geode.internal.cache.LocalRegion.basicPutAll( >> *LocalRegion.java:10166*) >> >> at org.apache.geode.internal.cache.LocalRegion.putAll( >> *LocalRegion.java:10101*) >> >> at org.apache.geode.internal.cache.LocalRegion.putAll( >> *LocalRegion.java:10113*) >> >> at org.springframework.data.gemfire.GemfireTemplate.putAll( >> *GemfireTemplate.java:201*) >> >> ... 25 common frames omitted >> >> >> >> Thanks, >> >> Dharam >> >> >> >> This email is confidential and subject to important disclaimers and >> conditions including on offers for the purchase or sale of securities, >> accuracy and completeness of information, viruses, confidentiality, legal >> privilege, and legal entity disclaimers, available at >> http://www.jpmorgan.com/pages/disclosures/email >> >> >> >> This email is confidential and subject to important disclaimers and >> conditions including on offers for the purchase or sale of securities, >> accuracy and completeness of information, viruses, confidentiality, legal >> privilege, and legal entity disclaimers, available at >> http://www.jpmorgan.com/pages/disclosures/email >> >> This email is confidential and subject to important disclaimers and >> conditions including on offers for the purchase or sale of securities, >> accuracy and completeness of information, viruses, confidentiality, legal >> privilege, and legal entity disclaimers, available at >> http://www.jpmorgan.com/pages/disclosures/email >> >> This email is confidential and subject to important disclaimers and >> conditions including on offers for the purchase or sale of securities, >> accuracy and completeness of information, viruses, confidentiality, legal >> privilege, and legal entity disclaimers, available at >> http://www.jpmorgan.com/pages/disclosures/email >> > >
