[
https://issues.apache.org/jira/browse/NIP-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18052479#comment-18052479
]
Mark Bean commented on NIP-19:
------------------------------
My only concern is any increase in CPU overhead and/or storage requirements as
mandatory for this feature. If it is minimal impact and/or it can be turned
off, that is preferred.
> Add Support for Gauges to Processor Metrics
> -------------------------------------------
>
> Key: NIP-19
> URL: https://issues.apache.org/jira/browse/NIP-19
> Project: NiFi Improvement Proposal
> Issue Type: New Feature
> Reporter: David Handermann
> Assignee: David Handermann
> Priority: Major
>
> h2. Motivation
> The framework has supported Counters as a general method for tracking
> selected behaviors in Processors since project inception. Although Counters
> can be used for a number of scenarios, they are limited to reporting values
> that accumulate over time.
> Adding support for Gauges would enable Processors to report values at a given
> point in time. Gauges support tracking for numbers such as the processing
> rate of an operation, or the size of temporary buffers used for an operation,
> among other metrics.
> h2. Scope
> The initial scope involves adding a new method to the {{ProcessSession}}
> interface in the public Apache NiFi API. The initial implementation should
> include updates to the standard implementations of {{ProcessSession}} in the
> framework, along with changes to the {{StatusHistoryRepository}} and other
> framework components to capture and process Gauges.
> Subsequent implementation could include updates to the user interface to
> present captured Gauge metrics. User interface changes may involve additional
> design consideration for optimal presentation in the NiFi application. For
> these reasons, any changes to the user interface should be evaluated after
> the initial NiFi API and framework level changes.
> h2. Compatibility
> As an addition to the {{ProcessSession}} interface, support for Gauges does
> not present any potential break in backward compatibility, and thus does not
> require a new major version of either the Apache NiFi API or the Apache NiFi
> framework.
> h2. Description
> The implementation strategy consists of NiFi API additions, NiFi framework
> updates, NiFi REST API updates, and NiFi UI changes.
> h3. NiFi API
> Adding support for Gauges to Processor metrics begins with targeted updates
> to the NiFi API, adding a method to the {{ProcessSession}} interface. The
> method should accept a name, a number, and flag indicating commit timing.
> {{void recordGauge(String name, double value, CommitTiming commitTiming);}}
> The {{CommitTiming}} argument is a new {{enum}} that should have the
> following values:
> * NOW
> * SESSION_COMMITTED
> The {{CommitTiming}} enumeration follows the pattern of the Boolean
> {{immediate}} flag in the existing {{adjustCounter}} method, but using an
> {{enum}} provides clearer semantics when writing and reading code for
> implementing Processors.
> h3. NiFi Framework
> Implementation in the NiFi framework involves implementing the new
> {{ProcessSession}} method in both standard classes and the {{nifi-mock}}
> library. The {{StandardProcessSession}} is the primary point of
> implementation, requiring new instance variables for storing recored Gauge
> values analogous to current Maps for counters.
> Additional framework changes include updates to the {{RepositoryContext}}
> interface, the addition of an intermediate {{GaugeRepository}} analogous to
> the {{{}CounterRepository{}}}, and potential changes to the
> {{{}StatusHistoryRepository{}}}. As framework interfaces, these changes can
> be coordinated within the scope of implementation for general framework
> updates.
> Framework implementation may include the introduction of new application
> properties for configuring settings specific to Gauges.
> h3. NiFi REST API
> Following basic implementation at the framework level, a new
> {{GaugesResource}} should be created, analogous to the current
> {{{}CountersResource{}}}, providing REST API methods for listing current
> Gauges and values.
> h3. NiFi UI
> Potential NiFi user interface changes include integrating Gauges with Status
> History information. With certain Node information such as CPU and memory
> already rendered based on gauges, any new implementation should follow the
> same general pattern.
> Unlike Counters that show an accumulated number, rendering a simple list of
> Gauges could limit the presentation to the most recent value recorded.
> h2. Verification
> Changes to the {{ProcessSession}} interface in NiFi API are self-contained.
> Updates to the NiFi framework and other components should follow standard
> unit test patterns. As a framework feature, Gauges would also be a candidate
> for system testing.
> h2. Alternatives
> Possible alternatives include custom implementation in each Processor that
> requires the ability to record metrics using Gauges. This might work for a
> small number of Processors, but would not be suitable as a pattern for
> implementation across the Apache NiFi project.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
