David Handermann created NIP-19:
-----------------------------------
Summary: Add Support for Gauges to Processor Metrics
Key: NIP-19
URL: https://issues.apache.org/jira/browse/NIP-19
Project: NiFi Improvement Proposal
Issue Type: New Feature
Reporter: David Handermann
Assignee: David Handermann
h2. Motivation
The framework has supported Counters as a general method for tracking selected
behaviors in Processors since project inception. Although Counters can be used
for a number of scenarios, they are limited to reporting values that accumulate
over time.
Adding support for Gauges would enable Processors to report values at a given
point in time. Gauges support tracking for numbers such as the processing rate
of an operation, or the size of temporary buffers used for an operation, among
other metrics.
h2. Scope
The initial scope involves adding a new method to the {{ProcessSession}}
interface in the public Apache NiFi API. The initial implementation should
include updates to the standard implementations of {{ProcessSession}} in the
framework, along with changes to the {{StatusHistoryRepository}} and other
framework components to capture and process Gauges.
Subsequent implementation could include updates to the user interface to
present captured Gauge metrics. User interface changes may involve additional
design consideration for optimal presentation in the NiFi application. For
these reasons, any changes to the user interface should be evaluated after the
initial NiFi API and framework level changes.
h2. Compatibility
As an addition to the {{ProcessSession}} interface, support for Gauges does not
present any potential break in backward compatibility, and thus does not
require a new major version of either the Apache NiFi API or the Apache NiFi
framework.
h2. Description
The implementation strategy consists of NiFi API additions, NiFi framework
updates, NiFi REST API updates, and NiFi UI changes.
h3. NiFi API
Adding support for Gauges to Processor metrics begins with targeted updates to
the NiFi API, adding a method to the {{ProcessSession}} interface. The method
should accept a name, a number, and flag indicating commit timing.
{{void recordGauge(String name, double value, CommitTiming commitTiming);}}
The {{CommitTiming}} argument is a new {{enum}} that should have the following
values:
* NOW
* SESSION_COMMITTED
The {{CommitTiming}} enumeration follows the pattern of the Boolean
{{immediate}} flag in the existing {{adjustCounter}} method, but using an
{{enum}} provides clearer semantics when writing and reading code for
implementing Processors.
h3. NiFi Framework
Implementation in the NiFi framework involves implementing the new
{{ProcessSession}} method in both standard classes and the {{nifi-mock}}
library. The {{StandardProcessSession}} is the primary point of implementation,
requiring new instance variables for storing recored Gauge values analogous to
current Maps for counters.
Additional framework changes include updates to the {{RepositoryContext}}
interface, the addition of an intermediate {{GaugeRepository}} analogous to the
{{{}CounterRepository{}}}, and potential changes to the
{{{}StatusHistoryRepository{}}}. As framework interfaces, these changes can be
coordinated within the scope of implementation for general framework updates.
Framework implementation may include the introduction of new application
properties for configuring settings specific to Gauges.
h3. NiFi REST API
Following basic implementation at the framework level, a new {{GaugesResource}}
should be created, analogous to the current {{{}CountersResource{}}}, providing
REST API methods for listing current Gauges and values.
h3. NiFi UI
Potential NiFi user interface changes include integrating Gauges with Status
History information. With certain Node information such as CPU and memory
already rendered based on gauges, any new implementation should follow the same
general pattern.
Unlike Counters that show an accumulated number, rendering a simple list of
Gauges could limit the presentation to the most recent value recorded.
h2. Verification
Changes to the {{ProcessSession}} interface in NiFi API are self-contained.
Updates to the NiFi framework and other components should follow standard unit
test patterns. As a framework feature, Gauges would also be a candidate for
system testing.
h2. Alternatives
Possible alternatives include custom implementation in each Processor that
requires the ability to record metrics using Gauges. This might work for a
small number of Processors, but would not be suitable as a pattern for
implementation across the Apache NiFi project.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
