Yeah forgot to mention that it's the only client connecting to server to load region shown below! No high io/cpu consuming activity running on server side!
From: Thacker, Dharam Sent: Monday, December 19, 2016 10:46 PM To: [email protected] Subject: RE: Socket timeout after making regions persistent in async way Yeah sure! Client config looks simple. I have not enabled redundant copies. More region is of type REPLICATE_PERSISTENT on server side. Server is just initialized with -Xms = 20G and -Xmx = 30G with ConcMarkSweep GC along with few more tuning parameters which works descent though! Server logs: [warn 2016/12/19 22:42:13.254 IST <ServerConnection on port 40404 Thread 2> tid=0x5b] Server connection from [identity(XXXXX(11700:loner):63652:3b290e18,connection=2; port=63657]: Unexpected IOException: java.net.SocketException: Software caused connection abort: socket write error at java.net.SocketOutputStream.socketWrite0(Native Method) at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109) at java.net.SocketOutputStream.write(SocketOutputStream.java:153) at org.apache.geode.internal.cache.tier.sockets.Message.flushBuffer(Message.java:649) at org.apache.geode.internal.cache.tier.sockets.Message.sendBytes(Message.java:627) at org.apache.geode.internal.cache.tier.sockets.ChunkedMessage.sendChunk(ChunkedMessage.java:314) at org.apache.geode.internal.cache.tier.sockets.ChunkedMessage.sendChunk(ChunkedMessage.java:322) at org.apache.geode.internal.cache.tier.sockets.command.PutAll80.writeReply(PutAll80.java:388) at org.apache.geode.internal.cache.tier.sockets.command.PutAll80.cmdExecute(PutAll80.java:339) at org.apache.geode.internal.cache.tier.sockets.BaseCommand.execute(BaseCommand.java:147) at org.apache.geode.internal.cache.tier.sockets.ServerConnection.doNormalMsg(ServerConnection.java:783) at org.apache.geode.internal.cache.tier.sockets.ServerConnection.doOneMessage(ServerConnection.java:913) at org.apache.geode.internal.cache.tier.sockets.ServerConnection.run(ServerConnection.java:1180) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.geode.internal.cache.tier.sockets.AcceptorImpl$1$1.run(AcceptorImpl.java:546) at java.lang.Thread.run(Thread.java:745) Client config: <gfe:pool id="event-geode-pool" subscription-enabled="true" server-group="EVENT_GRP"> <gfe:localtor...> </gfe:pool> <gfe:client-region id="Event" cache-ref="event-cache" pool-name="event-geode-pool" shortcut="PROXY" key-constraint="java.lang.String" value-constraint="a.m.n.Event"> </gfe:client-region> <gfe:cq-listener-container id="cqListenerContainer" cache="event-cache"> <gfe:listener ref="eventListener" query="SELECT * FROM /Event WHERE status='PENDING'"/> </gfe:cq-listener-container> <bean id="eventListener" class="a.m.n.cqlisteners.MyEventListener"/> Thanks, Dharam From: Udo Kohlmeyer [mailto:[email protected]] Sent: Monday, December 19, 2016 10:26 PM To: [email protected]<mailto:[email protected]> Subject: Re: Socket timeout after making regions persistent in async way Hi there Dharam, It is possible that persistence has an effect on the performance of the system. But generally socket timeouts occur due to other issues like GC or system load. What is the server load around the time of the disconnection? Have you enabled redundant copies for your cq? Could you please provide your client configuration for this problem. Maybe even a snippet of the server logs around the time of the client disconnection. (a minute either side of the client disconnection is a good start). --Udo On 12/19/16 05:22, Thacker, Dharam wrote: Hi, Has anyone faced below exceptions while cqlistener thread is working on? It started coming once after I changes regions to be persistent. Am I missing anything related to socket timeout? <geode:disk-store id="my_diskstore" cache-ref="geodeCache" compaction-threshold="50" auto-compact="false" allow-force-compaction="true" max-oplog-size="512" queue-size="10000" time-interval="500" write-buffer-size="65536" disk-usage-warning-percentage="80" disk-usage-critical-percentage="98"> <geode:disk-dir location="mydata" max-size="25600"/> </geode:disk-store> It tells me that server is unreachable but bouncing client connects to server again but after some interval of activity with server, again throws back below exception! [warn 2016/12/19 18:43:22.608 IST <cqListenerContainer-1> tid=0x6c] Pool unexpected socket timed out on client connection=Pooled Connection to XXXX:40404: Connection[XXXX:40404]@1743344121). Server unreachable: could not connect after 1 attempts [cqListenerContainer-1] ERROR o.s.d.g.l.a.ContinuousQueryListenerAdapter Listener execution failed... org.apache.geode.cache.client.ServerConnectivityException: Pool unexpected socket timed out on client connection=Pooled Connection to XXXX:40404: Connection[XXXX:40404]@1743344121). Server unreachable: could not connect after 1 attempts at org.apache.geode.cache.client.internal.OpExecutorImpl.handleException(OpExecutorImpl.java:819) at org.apache.geode.cache.client.internal.OpExecutorImpl.handleException(OpExecutorImpl.java:603) at org.apache.geode.cache.client.internal.OpExecutorImpl.execute(OpExecutorImpl.java:173) at org.apache.geode.cache.client.internal.OpExecutorImpl.execute(OpExecutorImpl.java:110) at org.apache.geode.cache.client.internal.PoolImpl.execute(PoolImpl.java:697) at org.apache.geode.cache.client.internal.PutAllOp.execute(PutAllOp.java:110) at org.apache.geode.cache.client.internal.ServerRegionProxy.putAll(ServerRegionProxy.java:643) at org.apache.geode.internal.cache.LocalRegion.basicPutAll(LocalRegion.java:10166) at org.apache.geode.internal.cache.LocalRegion.putAll(LocalRegion.java:10101) at org.apache.geode.internal.cache.LocalRegion.putAll(LocalRegion.java:10113) at org.springframework.data.gemfire.GemfireTemplate.putAll(GemfireTemplate.java:201) ... 25 common frames omitted Thanks, Dharam This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email
