[
https://issues.apache.org/jira/browse/NIFI-6801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yolanda M. Davis updated NIFI-6801:
-----------------------------------
Description:
Connections should have Status Analytics Model instances that are unique for
each connection such that calculations for the model can be referred to or
updated when new observations are available. Currently the model appears to be
shared across connection objects which leads to erroneous results, especially
when a connection does not have enough observations to update the model (since
the older values of a previous connection may still be present).
This behavior can be seen when debugging is enabled for analytics and noting
differences between a Timer Driven Thread execution of predictions vs the NiFi
Web Server thread predictions for multiple connections (a flow with one
connection would not exhibit this issue). The Time Driven thread may not
retrieve enough connection status observations to refresh the model, and when
that occurs, it may still contain calculations from a previous model and use
that for predictions. Observers may see behavior where web server and timer
driven predictions only have similar or matching predictions for one connection
but others may be mismatched, such as in the below (where connection BBBB has
similar prediction between threads but AAAA, and CCCC do not):
{code:java}
2019-10-22 08:01:34,539 INFO [NiFi Web Server-23]
o.a.nifi.reporting.StandardEventAccess >>>> AAAA: predicted time to COUNT
backpressure = 00:44:13 / 2653416
2019-10-22 08:01:34,539 INFO [NiFi Web Server-23]
o.a.nifi.reporting.StandardEventAccess >>>> AAAA: predicted time to BYTES
backpressure = 01:09:13 / 4153511
2019-10-22 08:01:34,540 INFO [NiFi Web Server-23]
o.a.nifi.reporting.StandardEventAccess >>>> CCCC: predicted time to COUNT
backpressure = 00:32:31 / 1951609
2019-10-22 08:01:34,540 INFO [NiFi Web Server-23]
o.a.nifi.reporting.StandardEventAccess >>>> CCCC: predicted time to BYTES
backpressure = 00:32:28 / 1948504
2019-10-22 08:01:34,541 INFO [NiFi Web Server-23]
o.a.nifi.reporting.StandardEventAccess >>>> BBBB: predicted time to COUNT
backpressure = 07:29:14 / 26954837
2019-10-22 08:01:34,541 INFO [NiFi Web Server-23]
o.a.nifi.reporting.StandardEventAccess >>>> BBBB: predicted time to BYTES
backpressure = 05:24:56 / 19496954
2019-10-22 08:02:02,767 INFO [Timer-Driven Process Thread-2]
o.a.nifi.reporting.StandardEventAccess >>>> AAAA: predicted time to COUNT
backpressure = 04:08:36 / 14916150
2019-10-22 08:02:02,767 INFO [Timer-Driven Process Thread-2]
o.a.nifi.reporting.StandardEventAccess >>>> AAAA: predicted time to BYTES
backpressure = 04:17:49 / 15469243
2019-10-22 08:02:02,769 INFO [Timer-Driven Process Thread-2]
o.a.nifi.reporting.StandardEventAccess >>>> CCCC: predicted time to COUNT
backpressure = 01:13:42 / 4422829
2019-10-22 08:02:02,769 INFO [Timer-Driven Process Thread-2]
o.a.nifi.reporting.StandardEventAccess >>>> CCCC: predicted time to BYTES
backpressure = 00:57:15 / 3435109
2019-10-22 08:02:02,770 INFO [Timer-Driven Process Thread-2]
o.a.nifi.reporting.StandardEventAccess >>>> BBBB: predicted time to COUNT
backpressure = 07:28:46 / 26926608
2019-10-22 08:02:02,770 INFO [Timer-Driven Process Thread-2]
o.a.nifi.reporting.StandardEventAccess >>>> BBBB: predicted time to BYTES
backpressure = 05:24:28 / 19468725
{code}
A workaround for this is to increase the nifi.analytics.query.interval
property or decrease the nifi.components.status.snapshot.frequency property to
ensure obtaining enough observations for a refresh. However an appropriate fix
is to ensure model instances are unique for each connection and to ensure that,
by default, all threads will obtain enough observations for predictions.
was:
Connections should have Status Analytics Model instances that are unique for
each connection such that calculations for the model can be referred to or
updated when new observations are available. Currently the model appears to be
shared across connection objects which leads to erroneous results, especially
when a connection does not have enough observations to update the model (since
the older values of a previous connection may still be present).
This behavior can be seen when debugging is enabled for analytics and noting
differences between a Timer Driven Thread execution of predictions vs the NiFi
Web Server thread predictions for multiple connections (a flow with one
connection would not exhibit this issue). The Time Driven thread may not
retrieve enough connection status observations to refresh the model, and when
that occurs, it may still contain calculations from a previous model and use
that for predictions. Observers may see behavior where web server and timer
driven predictions only have similar or matching predictions for one connection
but others may be mismatched, such as in the below (where connection BBBB has
similar prediction between threads but AAAA, and CCCC do not):
{code:java}
2019-10-22 08:01:34,539 INFO [NiFi Web Server-23]
o.a.nifi.reporting.StandardEventAccess >>>> AAAA: predicted time to COUNT
backpressure = 00:44:13 / 2653416
2019-10-22 08:01:34,539 INFO [NiFi Web Server-23]
o.a.nifi.reporting.StandardEventAccess >>>> AAAA: predicted time to BYTES
backpressure = 01:09:13 / 4153511
2019-10-22 08:01:34,540 INFO [NiFi Web Server-23]
o.a.nifi.reporting.StandardEventAccess >>>> CCCC: predicted time to COUNT
backpressure = 00:32:31 / 1951609
2019-10-22 08:01:34,540 INFO [NiFi Web Server-23]
o.a.nifi.reporting.StandardEventAccess >>>> CCCC: predicted time to BYTES
backpressure = 00:32:28 / 1948504
2019-10-22 08:01:34,541 INFO [NiFi Web Server-23]
o.a.nifi.reporting.StandardEventAccess >>>> BBBB: predicted time to COUNT
backpressure = 07:29:14 / 26954837
2019-10-22 08:01:34,541 INFO [NiFi Web Server-23]
o.a.nifi.reporting.StandardEventAccess >>>> BBBB: predicted time to BYTES
backpressure = 05:24:56 / 19496954
2019-10-22 08:02:02,767 INFO [Timer-Driven Process Thread-2]
o.a.nifi.reporting.StandardEventAccess >>>> AAAA: predicted time to COUNT
backpressure = 04:08:36 / 14916150
2019-10-22 08:02:02,767 INFO [Timer-Driven Process Thread-2]
o.a.nifi.reporting.StandardEventAccess >>>> AAAA: predicted time to BYTES
backpressure = 04:17:49 / 15469243
2019-10-22 08:02:02,769 INFO [Timer-Driven Process Thread-2]
o.a.nifi.reporting.StandardEventAccess >>>> CCCC: predicted time to COUNT
backpressure = 01:13:42 / 4422829
2019-10-22 08:02:02,769 INFO [Timer-Driven Process Thread-2]
o.a.nifi.reporting.StandardEventAccess >>>> CCCC: predicted time to BYTES
backpressure = 00:57:15 / 3435109
2019-10-22 08:02:02,770 INFO [Timer-Driven Process Thread-2]
o.a.nifi.reporting.StandardEventAccess >>>> BBBB: predicted time to COUNT
backpressure = 07:28:46 / 26926608
2019-10-22 08:02:02,770 INFO [Timer-Driven Process Thread-2]
o.a.nifi.reporting.StandardEventAccess >>>> BBBB: predicted time to BYTES
backpressure = 05:24:28 / 19468725
{code}
A workaround for this is to increase the nifi.analytics.query.interval
property or decrease the nifi.components.status.snapshot.frequency property to
ensure obtaining enough observations for a refresh. However an appropriate fix
is to ensure model instances are unique for each connection and to ensure that
by default timer driven threads will obtain enough observations for predictions.
> Connection analytics models should be distinct for connections
> --------------------------------------------------------------
>
> Key: NIFI-6801
> URL: https://issues.apache.org/jira/browse/NIFI-6801
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.10.0
> Reporter: Yolanda M. Davis
> Assignee: Yolanda M. Davis
> Priority: Major
> Fix For: 1.11.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Connections should have Status Analytics Model instances that are unique for
> each connection such that calculations for the model can be referred to or
> updated when new observations are available. Currently the model appears to
> be shared across connection objects which leads to erroneous results,
> especially when a connection does not have enough observations to update the
> model (since the older values of a previous connection may still be present).
> This behavior can be seen when debugging is enabled for analytics and noting
> differences between a Timer Driven Thread execution of predictions vs the
> NiFi Web Server thread predictions for multiple connections (a flow with one
> connection would not exhibit this issue). The Time Driven thread may not
> retrieve enough connection status observations to refresh the model, and when
> that occurs, it may still contain calculations from a previous model and use
> that for predictions. Observers may see behavior where web server and timer
> driven predictions only have similar or matching predictions for one
> connection but others may be mismatched, such as in the below (where
> connection BBBB has similar prediction between threads but AAAA, and CCCC do
> not):
> {code:java}
> 2019-10-22 08:01:34,539 INFO [NiFi Web Server-23]
> o.a.nifi.reporting.StandardEventAccess >>>> AAAA: predicted time to COUNT
> backpressure = 00:44:13 / 2653416
> 2019-10-22 08:01:34,539 INFO [NiFi Web Server-23]
> o.a.nifi.reporting.StandardEventAccess >>>> AAAA: predicted time to BYTES
> backpressure = 01:09:13 / 4153511
> 2019-10-22 08:01:34,540 INFO [NiFi Web Server-23]
> o.a.nifi.reporting.StandardEventAccess >>>> CCCC: predicted time to COUNT
> backpressure = 00:32:31 / 1951609
> 2019-10-22 08:01:34,540 INFO [NiFi Web Server-23]
> o.a.nifi.reporting.StandardEventAccess >>>> CCCC: predicted time to BYTES
> backpressure = 00:32:28 / 1948504
> 2019-10-22 08:01:34,541 INFO [NiFi Web Server-23]
> o.a.nifi.reporting.StandardEventAccess >>>> BBBB: predicted time to COUNT
> backpressure = 07:29:14 / 26954837
> 2019-10-22 08:01:34,541 INFO [NiFi Web Server-23]
> o.a.nifi.reporting.StandardEventAccess >>>> BBBB: predicted time to BYTES
> backpressure = 05:24:56 / 19496954
> 2019-10-22 08:02:02,767 INFO [Timer-Driven Process Thread-2]
> o.a.nifi.reporting.StandardEventAccess >>>> AAAA: predicted time to COUNT
> backpressure = 04:08:36 / 14916150
> 2019-10-22 08:02:02,767 INFO [Timer-Driven Process Thread-2]
> o.a.nifi.reporting.StandardEventAccess >>>> AAAA: predicted time to BYTES
> backpressure = 04:17:49 / 15469243
> 2019-10-22 08:02:02,769 INFO [Timer-Driven Process Thread-2]
> o.a.nifi.reporting.StandardEventAccess >>>> CCCC: predicted time to COUNT
> backpressure = 01:13:42 / 4422829
> 2019-10-22 08:02:02,769 INFO [Timer-Driven Process Thread-2]
> o.a.nifi.reporting.StandardEventAccess >>>> CCCC: predicted time to BYTES
> backpressure = 00:57:15 / 3435109
> 2019-10-22 08:02:02,770 INFO [Timer-Driven Process Thread-2]
> o.a.nifi.reporting.StandardEventAccess >>>> BBBB: predicted time to COUNT
> backpressure = 07:28:46 / 26926608
> 2019-10-22 08:02:02,770 INFO [Timer-Driven Process Thread-2]
> o.a.nifi.reporting.StandardEventAccess >>>> BBBB: predicted time to BYTES
> backpressure = 05:24:28 / 19468725
> {code}
> A workaround for this is to increase the nifi.analytics.query.interval
> property or decrease the nifi.components.status.snapshot.frequency property
> to ensure obtaining enough observations for a refresh. However an appropriate
> fix is to ensure model instances are unique for each connection and to ensure
> that, by default, all threads will obtain enough observations for predictions.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
