Matt Burgess created NIP-2:
------------------------------

             Summary: Preview capability for processors
                 Key: NIP-2
                 URL: https://issues.apache.org/jira/browse/NIP-2
             Project: NiFi Improvement Proposal
          Issue Type: New Feature
            Reporter: Matt Burgess


*Motivation*
The ability to "send" a sample FlowFile through a processor in an existing 
flow, without being associated with the flow itself (provenance, I/O metrics, 
e.g.), rather by executing the configured processor using the provided input 
and returning the output could be highly beneficial for Flow Designers. It's 
somewhat analogous to the "Run Once" feature but aims to operate outside the 
actual flow framework so as not to potentially contaminate the various NiFi 
repositories keeping track of the flow of "real" data.

*Scope*
This would involve several changes and additions to the API and codebase, see 
the Description section for details.

*Compatibility*
This feature would be backwards-compatible as is it is new and can only be 
executed on a flow in a NiFi instance that supports this capability.

*Description*
This capability would provide a user to test a processor already in the flow 
during design before starting the flow with "real" data which can help debug 
flows while preventing data loss during development.

Tasks:
* REST API call(s) to POST data to endpoint(s) which respond with machine 
readable output (JSON, XML, e.g.)
*  An annotation on processors such as Previewable, perhaps the SideEffectFree 
annotation suffices but you cannot extend Java annotations so SideEffectFree 
processors may just be a good starting point
*  An established format for both the POST data and the output data (content 
and attributes, multiple FlowFiles, relationships, etc.)
*  UI changes to include:
-- Context-sensitive (right-click) menu option to Preview
-- Dialog to paste/upload/drag the input content
-- Inject parameters (perhaps reuse the Verification capability here as prudent)
-- Leverage the existing Content Viewer as prudent to show the output (both 
content and attributes in one place would be preferable, but code reuse in the 
UI helps with consistency and maintainability)

* Documentation
-- Add a section to the processors' documentation if they are Previewable, a 
common description could be provided to avoid the need for custom UIs
-- Add a section to the User Guide to describe how to preview a Previewable 
processor and how to interpret the results
-- Add a section as necessary to the Admin guide for allowing/disallowing 
Preview (see the Authorization section below)

* Authorization
-- Add a policy to allow/prevent previewing a processor (optional for this NIP 
as this is intended to be a "sandbox" operation)

*Verification*
As processors are augmented with the Previewable annotation, the pull requests 
should include unit tests and the overall documentation should describe how to 
use the Preview capability on a Previewable processor.

*Alternatives*
On development systems (or in development process groups), the GenerateXYZ 
processors can be used to inject data into the processor and the UI or the REST 
API can be used to inspect the output data from the connection. This can be 
quite cumbersome and requires a separate NiFi system or process group for 
testing flow development, then moving the tested flow into the target NiFi 
system.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to