Matt Burgess created NIP-2:
------------------------------
Summary: Preview capability for processors
Key: NIP-2
URL: https://issues.apache.org/jira/browse/NIP-2
Project: NiFi Improvement Proposal
Issue Type: New Feature
Reporter: Matt Burgess
*Motivation*
The ability to "send" a sample FlowFile through a processor in an existing
flow, without being associated with the flow itself (provenance, I/O metrics,
e.g.), rather by executing the configured processor using the provided input
and returning the output could be highly beneficial for Flow Designers. It's
somewhat analogous to the "Run Once" feature but aims to operate outside the
actual flow framework so as not to potentially contaminate the various NiFi
repositories keeping track of the flow of "real" data.
*Scope*
This would involve several changes and additions to the API and codebase, see
the Description section for details.
*Compatibility*
This feature would be backwards-compatible as is it is new and can only be
executed on a flow in a NiFi instance that supports this capability.
*Description*
This capability would provide a user to test a processor already in the flow
during design before starting the flow with "real" data which can help debug
flows while preventing data loss during development.
Tasks:
* REST API call(s) to POST data to endpoint(s) which respond with machine
readable output (JSON, XML, e.g.)
* An annotation on processors such as Previewable, perhaps the SideEffectFree
annotation suffices but you cannot extend Java annotations so SideEffectFree
processors may just be a good starting point
* An established format for both the POST data and the output data (content
and attributes, multiple FlowFiles, relationships, etc.)
* UI changes to include:
-- Context-sensitive (right-click) menu option to Preview
-- Dialog to paste/upload/drag the input content
-- Inject parameters (perhaps reuse the Verification capability here as prudent)
-- Leverage the existing Content Viewer as prudent to show the output (both
content and attributes in one place would be preferable, but code reuse in the
UI helps with consistency and maintainability)
* Documentation
-- Add a section to the processors' documentation if they are Previewable, a
common description could be provided to avoid the need for custom UIs
-- Add a section to the User Guide to describe how to preview a Previewable
processor and how to interpret the results
-- Add a section as necessary to the Admin guide for allowing/disallowing
Preview (see the Authorization section below)
* Authorization
-- Add a policy to allow/prevent previewing a processor (optional for this NIP
as this is intended to be a "sandbox" operation)
*Verification*
As processors are augmented with the Previewable annotation, the pull requests
should include unit tests and the overall documentation should describe how to
use the Preview capability on a Previewable processor.
*Alternatives*
On development systems (or in development process groups), the GenerateXYZ
processors can be used to inject data into the processor and the UI or the REST
API can be used to inspect the output data from the connection. This can be
quite cumbersome and requires a separate NiFi system or process group for
testing flow development, then moving the tested flow into the target NiFi
system.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)