1) We are in the process of migrating to Flink 1.11. But it is going to take a while before we can make everything work with the latest version. Meanwhile since this is happening in production I am trying to solve this. 2) Finalizae class is pointing to org.apache.flink.streaming.runtime.tasks.SystemProcessingTimeService . This class has a finalize method. I have attached spreadsheet ( *Object-explorer.csv*) to give you a high level view 3) The difference between working cluster and NON working cluster is only on Beam and Flink. Hardware, Input message rate, Application jars, Kafka are all the same between those 2 clusters. Working cluster was with Flink 1.4 and Beam 2.4.0
Any insights into this will help me to debug further Thanks, Josson On Thu, Sep 3, 2020 at 3:34 AM Piotr Nowojski <pnowoj...@apache.org> wrote: > Hi, > > Have you tried using a more recent Flink version? 1.8.x is no longer > supported, and latest versions might not have this issue anymore. > > Secondly, have you tried backtracking those references to the Finalizers? > Assuming that Finalizer is indeed the class causing problems. > > Also it may well be a non Flink issue [1]. > > Best regards, > Piotrek > > [1] https://issues.apache.org/jira/browse/KAFKA-8546 > > czw., 3 wrz 2020 o 04:47 Josson Paul <jossonp...@gmail.com> napisał(a): > >> Hi All, >> >> *ISSUE* >> ------ >> Flink application runs for sometime and suddenly the CPU shoots up and >> touches the peak, POD memory reaches to the peak, GC count increases, >> Old-gen spaces reach close to 100%. Full GC doesn't clean up heap space. At >> this point I stopped sending the data and cancelled the Flink Jobs. Still >> the Old-Gen space doesn't come down. I took a heap dump and can see that >> lot of Objects in the java.lang.Finalizer class. I have attached the >> details in a word document. I do have the heap dump but it is close to 2GB >> of compressed size. Is it safe to upload somewhere and share it here?. >> >> This issue doesn't happen in Flink: 1.4.0 and Beam: release-2.4.0 >> >> *WORKING CLUSTER INFO* (Flink: 1.4.0 and Beam: release-2.4.0) >> ---------------------------------------------------- >> >> Application reads from Kafka and does aggregations and writes into Kafka. >> Application has 5 minutes windows. Application uses Beam constructs to >> build the pipeline. To read and write we use Beam connectors. >> >> Flink version: 1.4.0 >> Beam version: release-2.4.0 >> Backend State: State backend is in the Heap and check pointing happening >> to the distributed File System. >> >> No of task Managers: 1 >> Heap: 6.4 GB >> CPU: 4 Cores >> Standalone cluster deployment on a Kubernetes pod >> >> *NOT WORKING CLUSTER INFO* (Flink version: 1.8.3 and Beam version: >> release-2.15.0) >> ---------- >> Application details are same as above >> >> *No change in application and the rate at which data is injected. But >> change in Flink and Beam versions* >> >> >> Flink version: 1.8.3 >> Beam version: release-2.15.0 >> Backend State: State backend is in the Heap and check pointing happening >> to the distributed File System. >> >> No of task Managers: 1 >> Heap: 6.5 GB >> CPU: 4 Cores >> >> Deployment: Standalone cluster deployment on a Kubernetes pod >> >> My Observations >> ------------- >> >> 1) CPU flame graph shows that in the working version, the cpu time on GC >> is lesser compared to non-working version (Please see the attached Flame >> Graph. *CPU-flame-WORKING.svg* for working cluster and >> *CPU-flame-NOT-working.svg*) >> >> 2) I have attached the flame graph for native memory MALLOC calls when >> the issue was happening. Please find the attached SVG image ( >> *malloc-NOT-working.svg*). The POD memory peaks when this issue happens. >> For me, it looks like the GC process is requesting a lot of native memory. >> >> 3) When the issue is happening the GC cpu usage is very high. Please see >> the flame graph (*CPU-graph-at-issuetime.svg*) >> >> Note: SVG file can be opened using any browser and it is clickable while >> opened. >> -- >> Thanks >> Josson >> > -- Thanks Josson
"Name","Retained Size","Shallow Size","Level"
"java.lang.ref.Finalizer","4860933776","88","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860933672","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860933632","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860933080","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860932528","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860931976","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860931424","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860930872","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860930320","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860927352","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860927312","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860927272","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860926448","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860926408","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860925008","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860924968","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860924928","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860924104","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860924064","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860915152","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860915112","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860915072","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860915032","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860914992","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860914440","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860914400","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860914360","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860914320","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860914280","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860914240","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860914200","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860914160","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860914120","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860914080","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860914040","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860913928","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860913888","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860913848","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860913736","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860913624","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860913584","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860913544","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860913504","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860913464","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860913424","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860913384","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860913344","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860913304","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860913264","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860913224","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860913184","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860913144","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860913104","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860913064","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860913024","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912984","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912944","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912904","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912864","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912824","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912784","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912744","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912704","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912664","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912624","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912584","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912544","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912504","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912464","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912424","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912384","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912344","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912304","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912264","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912224","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912184","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912144","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912104","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912064","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860912024","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911984","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911944","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911904","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911864","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911824","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911784","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911744","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911704","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911664","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911624","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911584","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911544","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911504","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911464","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911424","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911384","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911344","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911304","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911264","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911224","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911184","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911144","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911104","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911064","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860911024","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910984","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910944","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910904","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910864","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910824","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910784","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910744","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910704","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910664","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910624","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910584","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910544","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910504","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910464","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910424","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910384","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910344","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910304","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910264","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910224","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910184","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910144","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910104","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910064","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860910024","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909984","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909944","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909904","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909864","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909824","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909784","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909744","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909704","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909664","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909624","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909584","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909544","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909504","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909464","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909424","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909384","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909344","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909304","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909264","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909224","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909184","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909144","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909104","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909064","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860909024","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908984","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908944","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908904","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908864","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908824","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908784","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908744","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908704","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908664","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908624","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908584","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908544","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908504","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908464","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908424","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908384","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908344","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908304","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908264","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908224","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908184","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908144","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908104","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908064","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860908024","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907984","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907944","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907904","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907864","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907824","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907784","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907744","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907704","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907664","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907624","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907584","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907544","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907504","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907464","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907424","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907384","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907344","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907304","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907264","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907224","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907184","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860907144","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860906448","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860905752","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860904968","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860904184","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860904144","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860904104","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860904064","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860903952","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860903840","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860903728","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860903688","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860903648","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860903608","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860903568","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860903528","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860902992","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860902952","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860902912","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860902872","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860902832","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860902792","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860902752","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860902712","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860902672","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860902632","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860902592","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860902552","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860902512","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860902472","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4860896944","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4854574256","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4854568728","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4847429560","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4847429520","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4847429480","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4847427360","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4847425680","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","4469176736","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","2795955632","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","2795953512","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","2416112440","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","2416110760","40","1"
"[Pending Finalization] org.apache.flink.streaming.runtime.tasks.SystemProcessingTimeService","1671899912","32","1"
"[Pending Finalization] org.apache.flink.streaming.runtime.tasks.OneInputStreamTask","1671899848","88","1"
"[Pending Finalization] org.apache.beam.runners.flink.translation.wrappers.streaming.WindowDoFnOperator","1670567504","280","1"
"[Pending Finalization] org.apache.flink.streaming.runtime.tasks.SystemProcessingTimeService","1659607152","32","1"
"[Pending Finalization] org.apache.flink.streaming.runtime.tasks.OneInputStreamTask","1659607088","88","1"
"[Pending Finalization] org.apache.beam.runners.flink.translation.wrappers.streaming.WindowDoFnOperator","1658336656","280","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.HeapKeyedStateBackend","1507069128","96","1"
"[Pending Finalization] java.util.HashMap size = 8","1507068184","48","1"
"[Pending Finalization] java.util.HashMap$Node[16]","1507068136","80","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.HeapKeyedStateBackend","1495946168","96","1"
"[Pending Finalization] java.util.HashMap size = 8","1495945224","48","1"
"[Pending Finalization] java.util.HashMap$Node[16]","1495945176","80","1"
"[Pending Finalization] java.util.HashMap$Node (""buf"" → CopyOnWriteStateTable) chunk size = 1","1480900592","32","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","1480900560","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[524288]","1480900208","2097168","1"
"[Pending Finalization] java.util.HashMap$Node (""buf"" → CopyOnWriteStateTable) chunk size = 1","1469961072","32","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","1469961040","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[524288]","1469960688","2097168","1"
"[Pending Finalization] java.lang.ref.Finalizer","756503208","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","756501240","40","1"
"[Pending Finalization] org.apache.flink.streaming.runtime.tasks.SystemProcessingTimeService","379840672","32","1"
"[Pending Finalization] org.apache.flink.streaming.runtime.tasks.OneInputStreamTask","379840608","88","1"
"[Pending Finalization] java.lang.ref.Finalizer","379739208","40","1"
"[Pending Finalization] java.lang.ref.Finalizer","379737240","40","1"
"[Pending Finalization] org.apache.flink.streaming.runtime.tasks.SystemProcessingTimeService","379693792","32","1"
"[Pending Finalization] org.apache.flink.streaming.runtime.tasks.OneInputStreamTask","379693728","88","1"
"[Pending Finalization] org.apache.beam.runners.flink.translation.wrappers.streaming.WindowDoFnOperator","378795152","280","1"
"[Pending Finalization] org.apache.beam.runners.flink.translation.wrappers.streaming.WindowDoFnOperator","378795008","280","1"
"[Pending Finalization] org.apache.flink.streaming.runtime.tasks.SystemProcessingTimeService","376773408","32","1"
"[Pending Finalization] org.apache.flink.streaming.runtime.tasks.OneInputStreamTask","376773344","88","1"
"[Pending Finalization] org.apache.flink.streaming.runtime.tasks.SystemProcessingTimeService","376751520","32","1"
"[Pending Finalization] org.apache.flink.streaming.runtime.tasks.OneInputStreamTask","376751456","88","1"
"[Pending Finalization] org.apache.beam.runners.flink.translation.wrappers.streaming.WindowDoFnOperator","375863328","280","1"
"[Pending Finalization] org.apache.beam.runners.flink.translation.wrappers.streaming.WindowDoFnOperator","375863208","280","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.HeapKeyedStateBackend","291809512","96","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.HeapKeyedStateBackend","291809368","96","1"
"[Pending Finalization] java.util.HashMap size = 8","291808568","48","1"
"[Pending Finalization] java.util.HashMap$Node[16]","291808520","80","1"
"[Pending Finalization] java.util.HashMap size = 8","291808424","48","1"
"[Pending Finalization] java.util.HashMap$Node[16]","291808376","80","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.HeapKeyedStateBackend","289484312","96","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.HeapKeyedStateBackend","289484192","96","1"
"[Pending Finalization] java.util.HashMap size = 8","289483368","48","1"
"[Pending Finalization] java.util.HashMap$Node[16]","289483320","80","1"
"[Pending Finalization] java.util.HashMap size = 8","289483248","48","1"
"[Pending Finalization] java.util.HashMap$Node[16]","289483200","80","1"
"[Pending Finalization] java.util.HashMap$Node (""buf"" → CopyOnWriteStateTable) chunk size = 1","271996416","32","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","271996384","64","1"
"[Pending Finalization] java.util.HashMap$Node (""buf"" → CopyOnWriteStateTable) chunk size = 1","271996272","32","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","271996240","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[131072]","271996088","524304","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[131072]","271995944","524304","1"
"[Pending Finalization] java.util.HashMap$Node (""buf"" → CopyOnWriteStateTable) chunk size = 1","269808096","32","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","269808064","64","1"
"[Pending Finalization] java.util.HashMap$Node (""buf"" → CopyOnWriteStateTable) chunk size = 1","269807976","32","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","269807944","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[131072]","269807768","524304","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[131072]","269807648","524304","1"
"com.google.common.cache.LocalCache$LocalManualCache","82603968","16","1"
"com.google.common.cache.LocalCache","82603952","128","1"
"com.google.common.cache.LocalCache$Segment[4]","82602656","32","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.HeapPriorityQueueSet","61478064","40","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.HeapPriorityQueueSet","61076464","40","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","31908848","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[32768]","31908608","131088","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","31695648","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[32768]","31695408","131088","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.HeapPriorityQueueSet","28417536","40","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.HeapPriorityQueueSet","28417536","40","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.HeapPriorityQueueSet","28236256","40","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.HeapPriorityQueueSet","28236256","40","1"
"[Pending Finalization] java.util.HashMap$Node (""watermark-holds"" → CopyOnWriteStateTable) chunk size = 1","20807616","32","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","20807584","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[32768]","20807304","131088","1"
"com.google.common.cache.LocalCache$Segment","20704560","80","1"
"com.google.common.cache.LocalCache$Segment","20681384","80","1"
"[Pending Finalization] java.util.HashMap$Node (""watermark-holds"" → CopyOnWriteStateTable) chunk size = 1","20660272","32","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","20660240","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[32768]","20659960","131088","1"
"com.google.common.cache.LocalCache$Segment","20633472","80","1"
"com.google.common.cache.LocalCache$Segment","20583208","80","1"
"[Pending Finalization] java.util.HashMap[64]","19586672","272","1"
"[Pending Finalization] java.util.HashMap[64]","19474352","272","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","17837936","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","17837936","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[32768]","17837696","131088","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[32768]","17837696","131088","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","17708720","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","17708720","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[32768]","17708480","131088","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[32768]","17708480","131088","1"
"[Pending Finalization] java.util.HashMap$Node (""watermark-holds"" → CopyOnWriteStateTable) chunk size = 1","10699376","32","1"
"[Pending Finalization] java.util.HashMap$Node (""watermark-holds"" → CopyOnWriteStateTable) chunk size = 1","10699376","32","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","10699344","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","10699344","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[32768]","10699064","131088","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[32768]","10699064","131088","1"
"[Pending Finalization] java.util.HashMap$Node (""watermark-holds"" → CopyOnWriteStateTable) chunk size = 1","10622656","32","1"
"[Pending Finalization] java.util.HashMap$Node (""watermark-holds"" → CopyOnWriteStateTable) chunk size = 1","10622656","32","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","10622624","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","10622624","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[32768]","10622344","131088","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[32768]","10622344","131088","1"
"[Pending Finalization] java.util.HashMap[64]","9753200","272","1"
"[Pending Finalization] java.util.HashMap[64]","9753200","272","1"
"[Pending Finalization] java.util.HashMap[64]","9697840","272","1"
"[Pending Finalization] java.util.HashMap[64]","9697840","272","1"
"[Pending Finalization] org.apache.flink.streaming.runtime.tasks.SystemProcessingTimeService","7138352","32","1"
"[Pending Finalization] org.apache.flink.streaming.runtime.tasks.StoppableSourceStreamTask","7138288","88","1"
"org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$ParentFirstClassLoader (3)","6270460","80","1"
"java.util.Vector size = 2,229","6218084","32","1"
"java.lang.Object[2560]","6218052","10256","1"
"[Pending Finalization] org.apache.flink.streaming.runtime.tasks.SystemProcessingTimeService","6148704","32","1"
"[Pending Finalization] org.apache.flink.streaming.runtime.tasks.StoppableSourceStreamTask","6148640","88","1"
"[Pending Finalization] java.util.HashMap$Node (""closed"" → CopyOnWriteStateTable) chunk size = 1","6045376","32","1"
"[Pending Finalization] java.util.HashMap$Node (""closed"" → CopyOnWriteStateTable) chunk size = 1","6045376","32","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","6045344","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","6045344","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[131072]","6045080","524304","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[131072]","6045080","524304","1"
"[Pending Finalization] java.util.HashMap$Node (""closed"" → CopyOnWriteStateTable) chunk size = 1","6004016","32","1"
"[Pending Finalization] java.util.HashMap$Node (""closed"" → CopyOnWriteStateTable) chunk size = 1","6004016","32","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","6003984","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","6003984","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[131072]","6003720","524304","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[131072]","6003720","524304","1"
"org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$ParentFirstClassLoader (2)","5819028","80","1"
"java.util.Vector size = 2,117","5767580","32","1"
"java.lang.Object[2560]","5767548","10256","1"
"com.cisco.tesseract.model.trap.Trap org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$ParentFirstClassLoader (2)","4851696","88","1"
"com.cisco.tesseract.model.trap.Trap org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$ParentFirstClassLoader (3)","4851696","88","1"
"[Pending Finalization] org.apache.flink.streaming.api.operators.StoppableStreamSource","4290088","104","1"
"[Pending Finalization] org.apache.beam.runners.flink.translation.wrappers.streaming.io.UnboundedSourceWrapper","4285512","64","1"
"[Pending Finalization] java.util.ArrayList size = 1","4275280","24","1"
"[Pending Finalization] java.lang.Object[10]","4275256","56","1"
"[Pending Finalization] org.apache.beam.sdk.io.kafka.KafkaUnboundedReader","4275200","112","1"
"sun.misc.Launcher$AppClassLoader","3938045","88","1"
"com.google.protobuf.GeneratedMessageV3$FieldAccessorTable","3684408","32","1"
"com.google.protobuf.GeneratedMessageV3$FieldAccessorTable","3684408","32","1"
"com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$FieldAccessor[3612]","3684360","14464","1"
"com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$FieldAccessor[3612]","3684360","14464","1"
"[Unreachable] int[914092] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...}","3656384","3656384","1"
"[Pending Finalization] java.util.HashMap$Node (""closed"" → CopyOnWriteStateTable) chunk size = 1","3574912","32","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","3574880","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[65536]","3574616","262160","1"
"[Pending Finalization] java.util.HashMap$Node (""closed"" → CopyOnWriteStateTable) chunk size = 1","3550096","32","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","3550064","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[65536]","3549800","262160","1"
"[Pending Finalization] org.apache.flink.streaming.api.operators.StoppableStreamSource","3320128","104","1"
"[Pending Finalization] org.apache.beam.runners.flink.translation.wrappers.streaming.io.UnboundedSourceWrapper","3315552","64","1"
"[Pending Finalization] java.util.ArrayList size = 1","3305320","24","1"
"[Pending Finalization] java.lang.Object[10]","3305296","56","1"
"[Pending Finalization] org.apache.beam.sdk.io.kafka.KafkaUnboundedReader","3305240","112","1"
"[Pending Finalization] java.util.HashMap$Node (""pane"" → CopyOnWriteStateTable) chunk size = 1","3034096","32","1"
"[Pending Finalization] java.util.HashMap$Node (""pane"" → CopyOnWriteStateTable) chunk size = 1","3034096","32","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","3034064","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","3034064","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[131072]","3033800","524304","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[131072]","3033800","524304","1"
"[Pending Finalization] java.util.HashMap$Node (""pane"" → CopyOnWriteStateTable) chunk size = 1","3015296","32","1"
"[Pending Finalization] java.util.HashMap$Node (""pane"" → CopyOnWriteStateTable) chunk size = 1","3015296","32","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","3015264","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","3015264","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[131072]","3015000","524304","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[131072]","3015000","524304","1"
"java.util.Vector size = 10,128","2775221","32","1"
"java.lang.Object[10240]","2775189","40976","1"
"[Unreachable] int[687816] = {0, -179650375, -179650379, -179650383, -179650387, 0, 0, 0, -179650391, -179650395, 0, 0, 0, 0, 0, 0, 0, 0, -1422160397, 7, -134202672, -1737518409, -967695518, -972006206, 0, 0, -1422160365, 7, -134202672, 114038, -967695560, -972006204, 0, 0, -1422160333, 7, -134202672, 1497985842, -967695592, -972006202, 0, 0, -1422160301, 7, -134202672, 1349527473, -967695539, -972006200, 0, 0, ...}","2751280","2751280","1"
"[Unreachable] int[687816] = {3145786, 3801141, 4587568, 4325434, 52, 0, 21, 0, -134209146, 17096, 21, 0, -134217022, -178601820, 0, 0, 17, 0, -134217665, 4, 6881380, 6684774, 21, 0, -134217022, -178601814, 0, 0, 17, 0, -134217665, 9, 3014712, 3014712, 3276849, 3014704, 48, 0, -1771057391, 60, -134217022, -178601806, 0, 0, 17, 0, -134217665, 14, 6881347, 6488179, ...}","2751280","2751280","1"
"[Unreachable] int[687816] = {0, 0, -174407476, -174407480, 0, -174407488, 0, -174407492, -174407496, -174407500, -174407504, 0, 0, 0, -174407508, -174407512, 0, 0, 0, 0, 0, 0, 0, 0, -1364872773, 7, -134202672, -1737518409, -967695518, -972006206, 0, 0, -1364872741, 7, -134202672, 114038, -967695560, -972006204, 0, 0, -1364872709, 7, -134202672, 1497985842, -967695592, -972006202, 0, 0, -1364872677, 7, ...}","2751280","2751280","1"
"[Unreachable] int[687816] = {-133228084, 0, 935835887, 372, -1365591853, 7, -134202672, -952205174, -173358942, -173358939, 0, 0, -1365591821, 7, -134217022, -961538245, -952219724, 0, -1362475141, 7, -132198465, -173358935, -173359029, -173358932, -945625300, 0, -1362475109, 7, -134217022, -961538228, -224723864, 0, -1362475085, 7, -133228084, 0, 935745887, 372, -1404716125, 7, -132929463, 929886313, -173358923, 0, -173358902, -837629980, -989711646, 137375, -539261408, 0, ...}","2751280","2751280","1"
"[Unreachable] int[687816] = {-1360869173, 7, -132198465, -172310368, -172310365, -172310353, -945625300, 0, -1360869141, 7, -134217022, -723787206, -194247671, 0, -1360869117, 7, -132200752, -969444398, -172310362, 0, -1360869093, 7, -133222208, -172310359, -172310356, 0, -1360869069, 7, -133228084, 0, 935565888, 372, -1360869045, 7, -133228084, 0, 935865888, 372, -1360869021, 7, -133228084, 0, 935865887, 372, -1359890709, 7, -133075918, 0, 0, -174207785, ...}","2751280","2751280","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.HeapPriorityQueueElement[638677]","2554728","2554728","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.HeapPriorityQueueElement[638677]","2554728","2554728","1"
"[Pending Finalization] org.apache.kafka.clients.consumer.KafkaConsumer","2267264","96","1"
"[Pending Finalization] org.apache.kafka.clients.consumer.KafkaConsumer","2263448","96","1"
"[Pending Finalization] org.apache.kafka.clients.consumer.internals.Fetcher","2106296","112","1"
"[Pending Finalization] org.apache.kafka.clients.consumer.internals.Fetcher","2106296","112","1"
"[Pending Finalization] org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords","2098184","72","1"
"[Pending Finalization] org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords","2098184","72","1"
"[Pending Finalization] byte[2097267] = {0, 0, 25, -112, 0, 0, 0, 0, 0, 0, 108, -19, 112, 6, 0, 0, 0, 1, 0, 15, 87, 105, 114, 101, 108, 101, 115, 115, 67, 108, 105, 101, 110, 116, 115, 0, 0, 0, 2, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, ...}","2097288","2097288","1"
"[Unreachable] byte[2097267] = {0, 0, 24, -42, 0, 0, 0, 0, 0, 0, 9, -93, 25, 110, 0, 0, 0, 1, 0, 15, 87, 105, 114, 101, 108, 101, 115, 115, 67, 108, 105, 101, 110, 116, 115, 0, 0, 0, 2, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, ...}","2097288","2097288","1"
"[Pending Finalization] byte[2097267] = {0, 0, 24, -40, 0, 0, 0, 0, 0, 0, 9, -93, 25, 110, 0, 0, 0, 1, 0, 15, 87, 105, 114, 101, 108, 101, 115, 115, 67, 108, 105, 101, 110, 116, 115, 0, 0, 0, 2, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, ...}","2097288","2097288","1"
"[Unreachable] byte[2097267] = {0, 0, 24, -44, 0, 0, 0, 0, 0, 0, 9, -93, 25, 110, 0, 0, 0, 1, 0, 15, 87, 105, 114, 101, 108, 101, 115, 115, 67, 108, 105, 101, 110, 116, 115, 0, 0, 0, 2, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, ...}","2097288","2097288","1"
"[Unreachable] byte[2097267] = {0, 0, 25, -110, 0, 0, 0, 0, 0, 0, 108, -19, 112, 6, 0, 0, 0, 1, 0, 15, 87, 105, 114, 101, 108, 101, 115, 115, 67, 108, 105, 101, 110, 116, 115, 0, 0, 0, 2, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, ...}","2097288","2097288","1"
"[Unreachable] byte[2097267] = {0, 0, 24, -39, 0, 0, 0, 0, 0, 0, 9, -93, 25, 110, 0, 0, 0, 1, 0, 15, 87, 105, 114, 101, 108, 101, 115, 115, 67, 108, 105, 101, 110, 116, 115, 0, 0, 0, 2, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, ...}","2097288","2097288","1"
"[Unreachable] int[503240] = {6357051, 7602275, 7733353, 5439589, 6357108, 6619252, 6946892, 7733345, 3080289, 6357100, 6750318, 4325423, 7274607, 6619244, 7209057, 6357051, 5046384, 6488161, 6946892, 7733345, 3080289, 6357100, 6750318, 5439535, 7471220, 7209065, 3866727, 7536738, 6881395, 4980836, 6357098, 6357110, 7077935, 7209057, 3080295, 7602259, 6881394, 6750318, 6488123, 6881388, 7209061, 5046388, 6488161, 6553665, 7471204, 7536741, 4980851, 6357098, 6357110, 7077935, ...}","2012976","2012976","1"
"[Unreachable] int[486832] = {-56837488, 469917062, 1668507971, 946167663, 979579962, 302134369, 775433784, 774910513, 1678246448, 40265321, 33561494, 976236578, 809119792, 892353094, 977612858, 149060, 2, 1128666626, 842150472, 942822966, 823394883, 892220727, 774909488, 33686066, 842477850, 775500334, 775435057, 1208104499, 1647393072, 1701065267, 1664103725, 1630809444, 942487608, 758462004, 808794419, 1630822711, 946026086, -1665495038, 39683836, 1936278300, 1700753251, 1664367160, 36987194, 942553106, 808595758, 134361134, 1717987684, 1807874, 808460802, 976236602, ...}","1947344","1947344","1"
"[Unreachable] int[467320] = {-967695511, -972006176, 0, 0, 37, 0, -134202672, -1075952106, -967695497, -972006174, 0, 0, 37, 0, -134202672, 909925332, -967695532, -972006172, 0, 0, 37, 0, -134202672, 110622674, -967695469, -972006220, 0, 0, 37, 0, -134202672, 107854, -967695553, -972006170, -785617066, 0, 37, 0, -134202672, 109643470, -967695623, -972006168, 0, 0, 37, 0, -134202672, 3601293, -967695504, -972006166, ...}","1869296","1869296","1"
"[Unreachable] int[447374] = {-56886320, 469917062, 1668507971, 946167663, 979579962, 302134369, 775433784, 774910513, 1678246448, 40265321, 33692566, 807535104, 808466992, 977678394, 893007152, 1160985157, 100804610, -1677590526, 200592808, 514, 0, 33554432, 1212368406, 909259314, 1127756374, 925963266, 808791605, 841887790, 925964802, 959589938, 942879022, 37106478, 825831496, 1681011554, 808791397, 875390051, 758659169, 892744760, 892941101, 878786352, 1664509537, -70254024, 1569127579, 1766005762, 1601135475, 876230757, 878787171, 775426562, 842083896, 36711984, ...}","1789512","1789512","1"
"[Pending Finalization] java.util.HashMap$Node (""pane"" → CopyOnWriteStateTable) chunk size = 1","1768144","32","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","1768112","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[65536]","1767848","262160","1"
"[Pending Finalization] java.util.HashMap$Node (""pane"" → CopyOnWriteStateTable) chunk size = 1","1756864","32","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable","1756832","64","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.CopyOnWriteStateTable$StateTableEntry[65536]","1756568","262160","1"
"[Unreachable] int[344178] = {3538993, 3801122, 2883632, 3211298, 2228279, 3145786, 2228268, 3670065, 3801122, 2883634, 3211298, 2228281, 3145786, 2228268, 3145778, 3801122, 2883632, 3276834, 2228273, 2228282, 2883618, 3276834, 2228274, 2228282, 2883618, 3276834, 2228275, 2228282, 2883618, 3276834, 2228276, 2228282, 2883618, 3276834, 2228277, 2228282, 7602287, 6619240, 5374066, 7274607, 4391028, 7667809, 6619251, 2883618, 3276834, 2228278, 3211322, 3735605, 3473464, 3735607, ...}","1376728","1376728","1"
"[Unreachable] int[327146] = {-134217022, -579149897, -1267389599, 0, 2050706571, 7, -134202672, 169477904, -865243730, -865096298, 0, 0, 2050706603, 7, -134217022, -655107904, 169479434, 0, 2050706707, 7, -134202672, -952205174, -865243723, -865253160, 0, 0, 2050706739, 7, -134217022, -961538245, -952219724, 0, 2050706843, 7, -134202672, -1838208060, -865243716, -865243713, 0, 0, 2050706875, 7, -134217022, -805396799, -1838171733, 0, 2050360171, 7, -132198465, -865243709, ...}","1308600","1308600","1"
"[Unreachable] int[318108] = {36711984, 1718182920, -1953627546, 570556418, 809119792, 1177565744, 976564282, 1144669491, -1682570683, 33867496, -957759252, 33620484, -187848044, -1178992117, -1193541118, -1748499966, -738051126, 37473004, 268211156, 1175847424, 842156099, 844510770, 335692600, 775239473, 808333365, 33698350, 925964802, 959589938, 942879022, 37106478, 825831496, 1681011554, 808791397, 875390051, 758659169, 892744760, 892941101, 878786352, 1664509537, -1764752840, 1569127571, 1766005762, 1601135475, 876230757, 878787171, 775426562, 842083896, 36711984, 1718182920, -1953627546, ...}","1272448","1272448","1"
"org.eclipse.jetty.servlet.ServletContextHandler","1199024","224","1"
"org.eclipse.jetty.servlet.ServletHandler","1196768","160","1"
"[Unreachable] int[296398] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...}","1185608","1185608","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.HeapPriorityQueueElement[283857]","1135448","1135448","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.HeapPriorityQueueElement[283857]","1135448","1135448","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.HeapPriorityQueueElement[283857]","1135448","1135448","1"
"[Pending Finalization] org.apache.flink.runtime.state.heap.HeapPriorityQueueElement[283857]","1135448","1135448","1"
"org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$ParentFirstClassLoader (1)","1127193","80","1"
"java.util.Vector size = 2,474","1080401","32","1"
"java.lang.Object[2560]","1080369","10256","1"
"[Pending Finalization] org.apache.beam.sdk.io.kafka.KafkaUnboundedReader$PartitionState","1027064","56","1"
"[Pending Finalization] java.util.Collections$UnmodifiableCollection$1","1026880","24","1"
"[Pending Finalization] java.util.ArrayList size = 80","1026800","24","1"
"[Pending Finalization] java.lang.Object[109]","1026776","456","1"
"[Unreachable] int[242588] = {-865374540, -865374537, -865374534, -966781000, -865374540, -865374540, -865374540, -865374540, -865374540, -865374531, -865374529, -865374529, -865374527, -865374527, -865374527, -865374540, -865374524, -865374540, -966780959, -865374521, -865374527, -865374527, -865374527, -865374527, -865374527, 0, 1705271171, 7, -134217022, -969360453, -854863572, 0, 1705271195, 7, -134217022, -945340366, -1181192002, 0, 1705271219, 7, -134209146, -45, 1705271235, 7, -134209146, 19, 1705271251, 7, -134209146, 0, ...}","970368","970368","1"
"redis.clients.jedis.JedisCluster","916080","24","1"
"redis.clients.jedis.JedisSlotBasedConnectionHandler","916056","16","1"
"redis.clients.jedis.JedisClusterInfoCache","916040","56","1"
"java.util.HashMap size = 16,384","915520","48","1"
"java.util.HashMap$Node[32768]","915472","131088","1"
"org.eclipse.jetty.servlet.ServletContextHandler","906368","224","1"
"org.eclipse.jetty.servlet.ServletHandler","904288","160","1"
"[Unreachable] int[225036] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...}","900160","900160","1"
"org.eclipse.jetty.servlet.ServletHolder","897264","144","1"
"[Pending Finalization] org.apache.beam.sdk.io.kafka.KafkaUnboundedReader$PartitionState","896848","56","1"
"org.glassfish.jersey.servlet.ServletContainer","896752","40","1"
"[Pending Finalization] java.util.Collections$UnmodifiableCollection$1","896664","24","1"
"[Pending Finalization] java.util.ArrayList size = 71","896584","24","1"
"[Pending Finalization] java.lang.Object[73]","896560","312","1"
"org.glassfish.jersey.server.ApplicationHandler","896328","32","1"
"[Unreachable] int[221946] = {-865537347, -865537335, -945625300, 0, 1834045667, 7, -134217022, -540615191, -172083167, 0, 1834045691, 7, -132200752, -969444398, -865537344, 0, 1834045715, 7, -133222208, -865537341, -865537338, 0, 1834045739, 7, -133228084, 0, 935445888, 372, 1834045763, 7, -133228084, 0, 935745888, 372, 1834045787, 7, -133228084, 0, 935895887, 372, -1609209045, 7, -132235413, 275349, 935625888, 372, -865611158, -865537328, 724742315, 7, ...}","887800","887800","1"
"[Pending Finalization] org.apache.flink.configuration.Configuration","864552","16","1"
"[Pending Finalization] org.apache.flink.configuration.Configuration","864552","16","1"
"[Pending Finalization] java.util.HashMap size = 24","864536","48","1"
"[Pending Finalization] java.util.HashMap size = 24","864536","48","1"
"[Pending Finalization] java.util.HashMap$Node[64]","864488","272","1"
"[Pending Finalization] java.util.HashMap$Node[64]","864488","272","1"
"sun.misc.Launcher$ExtClassLoader","849824","80","1"
"[Pending Finalization] org.apache.beam.sdk.io.kafka.KafkaUnboundedReader$PartitionState","836208","56","1"
"[Unreachable] byte[836107] = {-84, -19, 0, 5, 115, 114, 0, 17, 106, 97, 118, 97, 46, 117, 116, 105, 108, 46, 72, 97, 115, 104, 77, 97, 112, 5, 7, -38, -63, -61, 22, 96, -47, 3, 0, 2, 70, 0, 10, 108, 111, 97, 100, 70, 97, 99, 116, 111, 114, 73, ...}","836128","836128","1"
"[Pending Finalization] java.util.Collections$UnmodifiableCollection$1","836024","24","1"
"[Pending Finalization] java.util.ArrayList size = 8","835944","24","1"
"[Pending Finalization] java.lang.Object[10]","835920","56","1"
"[Pending Finalization] java.util.HashMap$Node (""chainedTaskConfig_"" → byte[]) chunk size = 1","831064","32","1"
"[Pending Finalization] java.util.HashMap$Node (""chainedTaskConfig_"" → byte[]) chunk size = 1","831064","32","1"
"[Pending Finalization] byte[830988] = {-84, -19, 0, 5, 115, 114, 0, 17, 106, 97, 118, 97, 46, 117, 116, 105, 108, 46, 72, 97, 115, 104, 77, 97, 112, 5, 7, -38, -63, -61, 22, 96, -47, 3, 0, 2, 70, 0, 10, 108, 111, 97, 100, 70, 97, 99, 116, 111, 114, 73, ...}","831008","831008","1"
"[Pending Finalization] byte[830988] = {-84, -19, 0, 5, 115, 114, 0, 17, 106, 97, 118, 97, 46, 117, 116, 105, 108, 46, 72, 97, 115, 104, 77, 97, 112, 5, 7, -38, -63, -61, 22, 96, -47, 3, 0, 2, 70, 0, 10, 108, 111, 97, 100, 70, 97, 99, 116, 111, 114, 73, ...}","831008","831008","1"
"org.apache.flink.runtime.io.network.buffer.NetworkBufferPool","801112","40","1"
"java.util.concurrent.ArrayBlockingQueue size = 0","800704","48","1"
"java.lang.Object[15395]","800560","61600","1"
"java.util.concurrent.ConcurrentHashMap size = 13,408","781920","64","1"
"java.util.concurrent.ConcurrentHashMap$Node[32768]","781856","131088","1"
"java.util.concurrent.ConcurrentHashMap size = 13,334","771352","64","1"
"java.util.concurrent.ConcurrentHashMap$Node[32768]","771288","131088","1"
"java.io.ObjectStreamClass$Caches","761824","96","1"
"[Unreachable] int[180312] = {0, 43, 43, 0, 256, -836853801, 1594006019, 7, -134217491, 43, 976236578, 809119792, 892353094, 977613114, 892352325, 892810086, 825438258, 1667457847, 1664102456, 1697854009, 6434872, 0, 1594006083, 7, -132198465, -836853789, -836853786, -836853774, -945625300, 0, 1594006115, 7, -134217022, -669299731, -199788797, 0, 1594006139, 7, -132200752, -969444398, -836853783, 0, 1594006163, 7, -133222208, -836853780, -836853777, 0, 1594006187, 7, ...}","721264","721264","1"
"[Unreachable] byte[703911] = {-84, -19, 0, 5, 115, 114, 0, 17, 106, 97, 118, 97, 46, 117, 116, 105, 108, 46, 72, 97, 115, 104, 77, 97, 112, 5, 7, -38, -63, -61, 22, 96, -47, 3, 0, 2, 70, 0, 10, 108, 111, 97, 100, 70, 97, 99, 116, 111, 114, 73, ...}","703928","703928","1"
"com.cisco.tesseract.collectors.schema.sdk.SchemaClientLocator sun.misc.Launcher$AppClassLoader","675384","80","1"
"and more...","","","1"
