On Fri, Nov 4, 2022 at 10:05 PM Clebert Suconic <clebert.suco...@gmail.com> wrote:
> More importantly.. that goes without saying... If we make this happen, > I am looking for contributors to share the project with me :) > > On Fri, Nov 4, 2022 at 4:51 PM Clebert Suconic > <clebert.suco...@gmail.com> wrote: > > > > I have a pet project that I have kept on hold for many years... but I > > see a need in Artemis... and perhaps I could use this as a push to > > make it happen.. > > > > https://github.com/clebertsuconic/d-sect > > > > > > The project is a JVMTI C Agent, that I can use to fetch objects, > > references and do some reports. It's a lot simpler to write tests to > > avoid memory leaks with this.. and you can do things like this: > > > > @Test > > void test() { > > > > > > ... do your test thing.. and then. > > > > > > JVMTIInterface jvmti = new JVMTIInterface(); > > Object[] objects = jvmti.getAllObjects("mypackage.MyClass"); > > > > Assert.assertEquals("My class is leaking", objects.length); > > > > } > > > > There are reporting methods that will show what would eventually be > > holding the references. > > > > > > What would be a good way to consume this in ActiveMQ Artemis? We > > should add a few tests like this but I'm not sure how we would consume > > the library in there? > Isn't there any such library already? Looking for it, I indeed had trouble finding anything for Java. In .NET/C# world, there is dotMemory Unit, https://www.nuget.org/packages/JetBrains.DotMemoryUnit/, (some custom EULA, not opensource). There are some usage examples in an announcement blog, https://blog.jetbrains.com/dotnet/2015/03/04/unit-testing-and-memory-profiling-can-they-be-combined/, and in the docs https://www.jetbrains.com/help/dotmemory-unit/Introduction.html. dotMemory.Check(memory => Assert.That(memory.GetObjects(where => where.Type.Is<Goo>()).ObjectsCount, Is.EqualTo(0))); In JavaScript world, there seem to be some research efforts such as http://plasma-umass.org/BLeak/, https://cacm.acm.org/magazines/2020/11/248223-bleak/abstract, which requires writing UI tests that navigate in a webapp in a loop, and compare objects in heap snapshots after going through the loop several times. There is also https://www.npmjs.com/package/memlab from Facebook, and looks like many more. For Android Java/Kotlin, there is https://square.github.io/leakcanary/, (ASL-2.0). Written in Kotlin, not sure if it has a reasonable Java API. They focus on Android, and the main mode of operation is to instrument the application during e2e testing (and possibly also in production deployment). It has knowledge of Android application structure to detect disposed-off application screens that are stuck in memory for some reason. https://square.github.io/leakcanary/fundamentals-how-leakcanary-works/. There is some support for usage on the JVM (not Android) https://square.github.io/leakcanary/recipes/#detecting-leaks-in-jvm-applications. They have assertNoLeaks() test rule which apparently works only on android, and they have a ticket wanting to port over this JVM implementation https://github.com/cashapp/paparazzi/pull/431/files#diff-7ea5ca018df762ee2392a3ca3afe479a5e9212ab7c7bb36c582331deef6448edR20. This seems to trigger heap dumps into a file on disk and then uses the Shark library to analyze the dump and do assertions on object counts: Class.forName("com.sun.management.HotSpotDiagnosticMXBean") .getMethod("dumpHeap", String::class.java, Boolean::class.javaPrimitiveType) .invoke(mBean, fileName, LIVE) } And I also found https://github.com/centic9/commons-test/blob/master/src/main/java/org/dstadler/commons/testing/MemoryLeakVerifier.java, which works by keeping a list of weak references to the objects that are supposed to be gc'd at the end of the test, and the final check consists of verifying that all the weak references have been nulled. (This does not look practical in case the objects are some internal objects not visible to the test, of course.) Some people on StackOverflow dislike the JVM Tool Interface API approach, claiming it is slow, and of course it requires writing in C/C++, https://stackoverflow.com/questions/67148994/jvm-getting-the-largest-objects-in-the-heap-programmatically. There they suggest using OpenJDK-specific api in JMX to get object counts. Looking for existing JVMTI agents, I found https://github.com/JetBrains/debugger-memory-agent (ASL-2.0), which is supposed to provide the Memory view feature in IntelliJ, https://www.jetbrains.com/help/idea/analyze-objects-in-the-jvm-heap.html, Looks like IntelliJ and Eclipse need this functionality for their debugger, so there can hopefully be an opportunity to reuse a maintained implementation. I was hopeful about https://github.com/GoogleCloudPlatform/cloud-debug-java/tree/main/src/agent but they don't appear to focus on inspecting the heap, strangely. There was something called JBoss Profiler JVMTI 2, https://anonsvn.jboss.org/repos/jbossprofiler/branches/JBossProfiler2/native/src/main/native/agent.cpp, but it looks dead. Exposed API for the debugger-memory-agent in previous paragraph is https://github.com/JetBrains/debugger-memory-agent/blob/b36a6d807da89d330206067ccc772f1487ca8312/test_data/proxy/src/com/intellij/memory/agent/MemoryAgent.java#L65-L98 Btw, while searching for all the above, I found something they call Lincheck, https://github.com/Kotlin/kotlinx-lincheck, and https://blog.jetbrains.com/kotlin/2021/02/how-we-test-concurrent-primitives-in-kotlin-coroutines/. It is able to run a test multiple times, changing thread interleaving, and assert that the result is the same every time. It seems to be useful for small tests only (individual classes, not entire programs). The blogs show Kotlin, but the library should be just fine testing Java classes. -- Mit freundlichen Grüßen / Kind regards Jiri Daněk
