Hi, I am seeing below issues with Metron profiler.
1) Profiler is writing the duplicate data to Hbase. For example I have inserted few records couple of days back to my test sensor topic and written a profile that counts the number of records. I see that profiler is still storing the same count continuously in to Hbase. As I observe when I see the WARN message below in profiler log, I see the profiler inserts data to Hbase that number of times (I am not sure if there is any relation) a. 2019-03-07 15:56:28.419 o.a.m.p.DefaultMessageDistributor pool-17-thread-1 [WARN] Profile expired from active cache; profile=demo_user_activity_count_ar, entity=Test1User5 2) I am storing the STAT objects to Hbase and trying to calculate the bin in another profiler by pulling Stats from Hbase. I see below exception after a while a. 2019-03-11 11:24:51.039 o.a.m.p.DefaultProfileBuilder pool-18-thread-1 [ERROR] Bad 'update' expression: error='Unable to parse: if EXISTS(s) then STATS_BIN(s, activity_count, 'DECILE') else -1 due to: Your bins must be non-decreasing with relevant variables s=org.apache.metron.statistics.OnlineStatisticsProvider@808c9b26,activity_count=24', expr='if EXISTS(s) then STATS_BIN(s, activity_count, 'DECILE') else -1', profile='demo_user_activity_bin_ar', entity='Test2User4', variables-available='[period.start, period, enrichmentsplitterbolt.splitter.end.ts, bin, profile, enrichmentsplitterbolt.splitter.begin.ts, is_alert, source.type, threatintelsplitterbolt.splitter.end.ts, s, activity_count, threatinteljoinbolt.joiner.ts, enrichmentjoinbolt.joiner.ts, period.end, threatintelsplitterbolt.splitter.begin.ts, entity, timestamp]' b. org.apache.metron.stellar.dsl.ParseException: Unable to parse: if EXISTS(s) then STATS_BIN(s, activity_count, 'DECILE') else -1 due to: Your bins must be non-decreasing with relevant variables s=org.apache.metron.statistics.OnlineStatisticsProvider@808c9b26,activity_count=24<mailto:s=org.apache.metron.statistics.OnlineStatisticsProvider@808c9b26,activity_count=24> I would like to understand what does it meant by "[WARN] Profile expired from active cache; profile=demo_user_activity_count_ar, entity=Test1User5", when can this happen and how to resolve this? I would also like to understand how to resolve the issue "[ERROR] Bad 'update' expression: error='Unable to parse: if EXISTS(s) then STATS_BIN(s, activity_count, 'DECILE') else -1 due to: Your bins must be non-decreasing with relevant variables s=org.apache.metron.statistics.OnlineStatisticsProvider@808c9b26,activity_count=24', expr='if EXISTS(s) then STATS_BIN(s, activity_count, 'DECILE') else -1', profile='demo_user_activity_bin_ar', entity='Test2User4', variables-available='[period.start, period, enrichmentsplitterbolt.splitter.end.ts, bin, profile, enrichmentsplitterbolt.splitter.begin.ts, is_alert, source.type, threatintelsplitterbolt.splitter.end.ts, s, activity_count, threatinteljoinbolt.joiner.ts, enrichmentjoinbolt.joiner.ts, period.end, threatintelsplitterbolt.splitter.begin.ts, entity, timestamp]'" Looking forward for some help from Metron team or community to understand and resolve the above issues. Thanking you. Thanks, Anil.
