You might consider upping the read-timeout on the client to see if that causes the issue to go away. The default read-timeout is 10 seconds. So if it's really just an issue of the server being slow and not responding in time a larger read timeout would help.
How much data are you putting with this putAll? -Dan On Mon, Dec 19, 2016 at 10:02 AM, Thacker, Dharam < [email protected]> wrote: > Not sure if this is related with past bug as per link? There are few more > archive mails matching to same exception for jiras worked out in past! > > > > http://markmail.org/message/oefrls7bdof776t7?q= > ServerConnectivityException&page=2 > > > > I have 7 functional indexes on server side for same region due to which > insertion happens slow, but that works fine if I mark region just as > REPLICATE. > > > > Regards, > > Dharam > > > > *From:* Thacker, Dharam > *Sent:* Monday, December 19, 2016 10:51 PM > > *To:* [email protected] > *Subject:* RE: Socket timeout after making regions persistent in async way > > > > Yeah forgot to mention that it’s the only client connecting to server to > load region shown below! No high io/cpu consuming activity running on > server side! > > > > *From:* Thacker, Dharam > *Sent:* Monday, December 19, 2016 10:46 PM > *To:* [email protected] > *Subject:* RE: Socket timeout after making regions persistent in async way > > > > Yeah sure! > > > > Client config looks simple. I have not enabled redundant copies. More > region is of type REPLICATE_PERSISTENT on server side. Server is just > initialized with –Xms = 20G and –Xmx = 30G with ConcMarkSweep GC along with > few more tuning parameters which works descent though! > > > > *Server logs:* > > > > [warn 2016/12/19 22:42:13.254 IST <ServerConnection on port 40404 Thread > 2> tid=0x5b] Server connection from > [identity(XXXXX(11700:loner):63652:3b290e18,connection=2; > port=63657]: Unexpected *IOException*: > > *java.net.SocketException*: Software caused connection abort: socket > write error > > at java.net.SocketOutputStream.socketWrite0(*Native Method*) > > at java.net.SocketOutputStream.socketWrite( > *SocketOutputStream.java:109*) > > at java.net.SocketOutputStream.write(*SocketOutputStream.java:153*) > > at org.apache.geode.internal.cache.tier.sockets.Message. > flushBuffer(*Message.java:649*) > > at org.apache.geode.internal.cache.tier.sockets.Message.sendBytes( > *Message.java:627*) > > at org.apache.geode.internal.cache.tier.sockets. > ChunkedMessage.sendChunk(*ChunkedMessage.java:314*) > > at org.apache.geode.internal.cache.tier.sockets. > ChunkedMessage.sendChunk(*ChunkedMessage.java:322*) > > at org.apache.geode.internal.cache.tier.sockets.command. > PutAll80.writeReply(*PutAll80.java:388*) > > at org.apache.geode.internal.cache.tier.sockets.command. > PutAll80.cmdExecute(*PutAll80.java:339*) > > at org.apache.geode.internal.cache.tier.sockets. > BaseCommand.execute(*BaseCommand.java:147*) > > at org.apache.geode.internal.cache.tier.sockets. > ServerConnection.doNormalMsg(*ServerConnection.java:783*) > > at org.apache.geode.internal.cache.tier.sockets. > ServerConnection.doOneMessage(*ServerConnection.java:913*) > > at org.apache.geode.internal.cache.tier.sockets. > ServerConnection.run(*ServerConnection.java:1180*) > > at java.util.concurrent.ThreadPoolExecutor.runWorker( > *ThreadPoolExecutor.java:1142*) > > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > *ThreadPoolExecutor.java:617*) > > at org.apache.geode.internal.cache.tier.sockets. > AcceptorImpl$1$1.run(*AcceptorImpl.java:546*) > > at java.lang.Thread.run(*Thread.java:745*) > > > > > > *Client config:* > > <gfe:pool id="event-geode-pool" subscription-enabled="true" > server-group="EVENT_GRP"> > > <gfe:localtor...> > > </gfe:pool> > > > > <gfe:client-region id="Event" > > cache-ref="event-cache" > > pool-name="event-geode-pool" > > shortcut="PROXY" > > key-constraint="java.lang.String" > > value-constraint="a.m.n.Event"> > > </gfe:client-region> > > > > > > <gfe:cq-listener-container id="cqListenerContainer" > cache="event-cache"> > > <gfe:listener ref="eventListener" > query="SELECT * FROM /Event WHERE status='PENDING'"/> > > </gfe:cq-listener-container> > > > > <bean id="eventListener" class="a.m.n.cqlisteners. > MyEventListener"/> > > > > Thanks, > > Dharam > > *From:* Udo Kohlmeyer [mailto:[email protected] > <[email protected]>] > *Sent:* Monday, December 19, 2016 10:26 PM > *To:* [email protected] > *Subject:* Re: Socket timeout after making regions persistent in async way > > > > Hi there Dharam, > > It is possible that persistence has an effect on the performance of the > system. But generally socket timeouts occur due to other issues like GC or > system load. > > What is the server load around the time of the disconnection? Have you > enabled redundant copies for your cq? > > Could you please provide your client configuration for this problem. > > Maybe even a snippet of the server logs around the time of the client > disconnection. (a minute either side of the client disconnection is a good > start). > > --Udo > > > > On 12/19/16 05:22, Thacker, Dharam wrote: > > Hi, > > > > Has anyone faced below exceptions while cqlistener thread is working on? > It started coming once after I changes regions to be persistent. Am I > missing anything related to socket timeout? > > > > <geode:disk-store id=*"my_diskstore"* cache-ref=*"geodeCache"* > > compaction-threshold=*"50"* > > auto-compact=*"false"* > > allow-force-compaction=*"true"* > > max-oplog-size=*"512"* > > queue-size=*"10000"* > > time-interval=*"500"* > > write-buffer-size=*"65536"* > > disk-usage-warning-percentage=*"80"* > > disk-usage-critical-percentage=*"98"*> > > <geode:disk-dir location=*"mydata"* max-size=*"25600"*/> > > </geode:disk-store> > > > > It tells me that server is unreachable but bouncing client connects to > server again but after some interval of activity with server, again throws > back below exception! > > > > [warn 2016/12/19 18:43:22.608 IST <cqListenerContainer-1> tid=0x6c] Pool > unexpected socket timed out on client connection=Pooled Connection to > XXXX:40404: Connection[XXXX:40404]@1743344121). Server unreachable: could > not connect after 1 attempts > > > > [cqListenerContainer-1] ERROR o.s.d.g.l.a.ContinuousQueryListenerAdapter > Listener execution failed... > > *org.apache.geode.cache.client.ServerConnectivityException*: Pool > unexpected socket timed out on client connection=Pooled Connection to > XXXX:40404: Connection[XXXX:40404]@1743344121). Server unreachable: could > not connect after 1 attempts > > at org.apache.geode.cache.client.internal.OpExecutorImpl. > handleException(*OpExecutorImpl.java:819*) > > at org.apache.geode.cache.client.internal.OpExecutorImpl. > handleException(*OpExecutorImpl.java:603*) > > at org.apache.geode.cache.client.internal.OpExecutorImpl.execute( > *OpExecutorImpl.java:173*) > > at org.apache.geode.cache.client.internal.OpExecutorImpl.execute( > *OpExecutorImpl.java:110*) > > at org.apache.geode.cache.client.internal.PoolImpl.execute( > *PoolImpl.java:697*) > > at org.apache.geode.cache.client.internal.PutAllOp.execute( > *PutAllOp.java:110*) > > at org.apache.geode.cache.client.internal.ServerRegionProxy.putAll( > *ServerRegionProxy.java:643*) > > at org.apache.geode.internal.cache.LocalRegion.basicPutAll( > *LocalRegion.java:10166*) > > at org.apache.geode.internal.cache.LocalRegion.putAll( > *LocalRegion.java:10101*) > > at org.apache.geode.internal.cache.LocalRegion.putAll( > *LocalRegion.java:10113*) > > at org.springframework.data.gemfire.GemfireTemplate.putAll( > *GemfireTemplate.java:201*) > > ... 25 common frames omitted > > > > Thanks, > > Dharam > > > > This email is confidential and subject to important disclaimers and > conditions including on offers for the purchase or sale of securities, > accuracy and completeness of information, viruses, confidentiality, legal > privilege, and legal entity disclaimers, available at > http://www.jpmorgan.com/pages/disclosures/email > > > > This email is confidential and subject to important disclaimers and > conditions including on offers for the purchase or sale of securities, > accuracy and completeness of information, viruses, confidentiality, legal > privilege, and legal entity disclaimers, available at > http://www.jpmorgan.com/pages/disclosures/email > > This email is confidential and subject to important disclaimers and > conditions including on offers for the purchase or sale of securities, > accuracy and completeness of information, viruses, confidentiality, legal > privilege, and legal entity disclaimers, available at > http://www.jpmorgan.com/pages/disclosures/email > > This email is confidential and subject to important disclaimers and > conditions including on offers for the purchase or sale of securities, > accuracy and completeness of information, viruses, confidentiality, legal > privilege, and legal entity disclaimers, available at > http://www.jpmorgan.com/pages/disclosures/email >
