Setting DEBUG logging for 'org.apache.metron.profiler' should show you very detailed information about what is happening with the Profiler in your topology.
On Fri, Mar 6, 2020 at 7:18 AM Gonçalo Pedras <[email protected]> wrote: > Hi, > > I’m experiencing some GC overhead limits in my profiler. The profiler > works fine while testing in Stellar enviorment, although it fails most of > the time as a running topology. My profiler has 1 profile only. > > My profile has the following configuration: > > · 5 integer counters, when It finds a specific string it adds to > the respective counter. > > · Unique list of strings, when It finds a new string it adds to > the list. > > · A fixed string field (stays the same throughout the profiler > duration). > > My profile flushes: > > · Profiler: a hardcoded Json map of the results of the counters. > Ex: {\”counter1\”: variable1, \”counter2\”: variable2 ..} > > · Triage: the integer counters, the fixed string value and the > size of the unique string list. > > The only reason I have a fixed value on the profile is that I don’t want > to enrich the same data twice, so I just might pass it through the > profiler. > > Is there any other way to debug the profiler while running as a topology? > Because I can’t seem to understand why it ends in error and the storm logs > don’t help much. I’ve tweaked some options like the profile period, time to > live and worker childopts. > > > > Thanks >
