The stacktrace says it is an exception coming back from server side. Client
is just logging it out here.
at
org.apache.geode.cache.client.internal.OpExecutorImpl.handleException(OpExecutorImpl.java:669)
at org.apache.geode.cache.client.internal.OpExecutorImpl.
*handleException*(OpExecutorImpl.java:623)
at org.apache.geode.cache.client.internal.OpExecutorImpl.
*executeOnServer*(OpExecutorImpl.java:385)
On Thu, 7 Sep 2017 at 11:52 PM, Dharam Thacker <[email protected]>
wrote:
> Yeah I have enabled debug logs in client and server both today. I am
> trying to reproduce it.
>
> For me it's coming at geode client application though.
>
> Thanks,
> Dharam
>
>
> On Sep 7, 2017 9:18 PM, "Xiawei Zhang" <[email protected]> wrote:
>
> By any chance you have debug log enabled on the server side? I remember I
> encountered a similar problem before, and it turns out the real exception
> was logged at debug level on the server side.
>
> -Xiawei
>
> On Thu, 7 Sep 2017 at 12:30 AM, Thacker, Dharam <
> [email protected]> wrote:
>
>> Hello Anthony,
>>
>> Thank you for guidance on that!
>>
>> I am sending you output of '-verbose:class' along with JVM GC args I have
>> used for spring boot geode client app.
>>
>> -Xms4g -Xmx4g -XX:+UseG1GC -XX:+ParallelRefProcEnabled
>> -XX:+UseStringDeduplication -XX:+PrintFlagsFinal -XX:+PrintReferenceGC
>> -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps
>> -XX:+PrintAdaptiveSizePolicy -XX:+HeapDumpOnOutOfMemoryError
>> -XX:HeapDumpPath=/apps/logs/heap_dump
>> -Xloggc:/local/apps/ravenapps/logs/gc/event-publisher/jvm-gc-%t.log
>> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=10M
>> -Dcom.sun.management.jmxremote
>> -Dcom.sun.management.jmxremote.authenticate=false
>> -Dcom.sun.management.jmxremote.ssl=false -client -XX:+AlwaysPreTouch
>> -XX:+UnlockDiagnosticVMOptions -XX:+G1SummarizeConcMark -Xss10M
>> -XX:InitiatingHeapOccupancyPercent=40 -XX:ParallelGCThreads=20
>> -XX:ConcGCThreads=5
>>
>> Thanks,
>> Dharam
>>
>> -----Original Message-----
>> From: Anthony Baker [mailto:[email protected]]
>> Sent: Wednesday, September 06, 2017 8:44 PM
>> To: [email protected]
>> Subject: Re: NoClassDefFoundError on client shutdown
>>
>> You might trying adding '-verbose:class’ to your client startup options
>> to help troubleshoot this issue.
>>
>> Anthony
>>
>>
>> > On Sep 6, 2017, at 7:41 AM, Thacker, Dharam <
>> [email protected]> wrote:
>> >
>> > Hi Anthony,
>> >
>> > Versions:
>> >
>> > Apache Geode: 1.1.1
>> > Spring Boot: 1.5.2.RELEASE
>> > Spring Data Geode: 1.0.0.INCUBATING-RELEASE [Within Maven POM Apache
>> > Geode jars have been overridden using version 1.1.1 – Attached
>> > pom.xml]
>> >
>> >
>> > [Using config level logging in geode client logs]
>> >
>> > Class path: /dev/lib/event-publisher.jar [It’s a fat UBER jar using
>> > spring boot maven plugin] Library Path:
>> > /usr/java/packages/lib/amd64
>> > /usr/lib64
>> > /lib64
>> > /lib
>> > /usr/lib
>> >
>> >
>> > Thanks & Regards,
>> > Dharam
>> > From: Anthony Baker [mailto:[email protected]]
>> > Sent: Wednesday, September 06, 2017 7:51 PM
>> > To: [email protected]
>> > Subject: Re: NoClassDefFoundError on client shutdown
>> >
>> > What is your classpath? What versions of Spring Boot and Geode are you
>> using on the client and server?
>> >
>> > Anthony
>> >
>> > On Sep 5, 2017, at 10:35 PM, Thacker, Dharam <
>> [email protected]> wrote:
>> >
>> > Hi Team,
>> >
>> > I am observing few “NoClassDefFoundError” errors on unexpected
>> termination of client under medium load.
>> >
>> > Could you help me to verify? Looks like a bug where it’s not able to
>> find expected classes in available runtime.
>> >
>> > [error 2017/09/05 06:29:57.677 EDT <poolTimer-DEFAULT-17374>
>> > tid=0xa9830] Unexpected error in pool task
>> > <org.apache.geode.cache.client.internal.LiveServerPinger$PingTask@5ba4
>> > 4553>
>> > java.lang.NoClassDefFoundError:
>> org/apache/geode/internal/cache/tier/BatchException
>> > at
>> org.apache.geode.cache.client.internal.OpExecutorImpl.handleException(OpExecutorImpl.java:669)
>> > at
>> org.apache.geode.cache.client.internal.OpExecutorImpl.handleException(OpExecutorImpl.java:623)
>> > at
>> org.apache.geode.cache.client.internal.OpExecutorImpl.executeOnServer(OpExecutorImpl.java:385)
>> > at
>> org.apache.geode.cache.client.internal.OpExecutorImpl.executeOn(OpExecutorImpl.java:341)
>> > at
>> org.apache.geode.cache.client.internal.PoolImpl.executeOn(PoolImpl.java:777)
>> > at
>> org.apache.geode.cache.client.internal.PingOp.execute(PingOp.java:36)
>> > at
>> org.apache.geode.cache.client.internal.LiveServerPinger$PingTask.run2(LiveServerPinger.java:90)
>> > at
>> org.apache.geode.cache.client.internal.PoolImpl$PoolTask.run(PoolImpl.java:1291)
>> > at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > at
>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>> > at
>> org.apache.geode.internal.ScheduledThreadPoolExecutorWithKeepAlive$DelegatingScheduledFuture.run(ScheduledThreadPoolExecutorWithKeepAlive.java:257)
>> > at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > at java.lang.Thread.run(Thread.java:745)
>> >
>> > [info 2017/09/05 06:30:06.327 EDT <Distributed system shutdown hook>
>> > tid=0x21] VM is exiting - shutting down distributed system
>> >
>> > [info 2017/09/05 06:30:06.330 EDT <Distributed system shutdown hook>
>> tid=0x21] GemFireCache[id = 260620222; isClosing = true; isShutDownAll =
>> false; created = Wed Aug 16 07:22:16 EDT 2017; server = false; copyOnRead =
>> false; lockLease = 120; lockTimeout = 60]: Now closing.
>> >
>> > [warning 2017/09/05 06:30:06.333 EDT <Distributed system shutdown
>> > hook> tid=0x21] Exception trying to close cache
>> > java.lang.NoClassDefFoundError:
>> org/apache/geode/internal/cache/partitioned/RegionAdvisor
>> > at
>> org.apache.geode.internal.cache.GemFireCacheImpl.close(GemFireCacheImpl.java:2177)
>> > at
>> org.apache.geode.distributed.internal.InternalDistributedSystem.disconnect(InternalDistributedSystem.java:1314)
>> > at
>> org.apache.geode.distributed.internal.InternalDistributedSystem$6.run(InternalDistributedSystem.java:2283)
>> > at java.lang.Thread.run(Thread.java:745)
>> > Caused by: java.lang.ClassNotFoundException:
>> org.apache.geode.internal.cache.partitioned.RegionAdvisor
>> > at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>> > at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>> > at
>> org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:94)
>> > at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>> > ... 4 more
>> >
>> > Thanks,
>> > Dharam
>> > This message is confidential and subject to terms at:
>> http://www.jpmorgan.com/emaildisclaimer including on confidentiality,
>> legal privilege, viruses and monitoring of electronic messages. If you are
>> not the intended recipient, please delete this message and notify the
>> sender immediately. Any unauthorized use is strictly prohibited.
>> >
>> > This message is confidential and subject to terms at:
>> http://www.jpmorgan.com/emaildisclaimer including on confidentiality,
>> legal privilege, viruses and monitoring of electronic messages. If you are
>> not the intended recipient, please delete this message and notify the
>> sender immediately. Any unauthorized use is strictly prohibited.
>> >
>> > <pom.xml>
>>
>>
>> This message is confidential and subject to terms at:
>> http://www.jpmorgan.com/emaildisclaimer including on confidentiality,
>> legal privilege, viruses and monitoring of electronic messages. If you are
>> not the intended recipient, please delete this message and notify the
>> sender immediately. Any unauthorized use is strictly prohibited.
>>
>
>
