Matt Foley created NIFI-1842:
--------------------------------
Summary: implement GetWithCamel and PutWithCamel processors, as
selectable-endpoint connectors
Key: NIFI-1842
URL: https://issues.apache.org/jira/browse/NIFI-1842
Project: Apache NiFi
Issue Type: Improvement
Components: Extensions
Reporter: Matt Foley
While the SpringContextProcessor (NIFI-1571) provides an excellent basis for
running Camel, it still requires the user to program Camel. In light of the
fact that Camel provides some 80+ protocol-aware endpoints (see the list at
https://github.com/apache/camel/tree/master/components), at least half of which
are probably useful to NiFi and not yet available as NiFi-native processors, I
propose the following:
Create processors named GetWithCamel and PutWithCamel, that allow:
* configuration-time selection from a subset of Camel-supported endpoints
* and then providing the additional config values required by the selected
endpoint, probably in an XML file
These processors will of course be based on SpringContextProcessor, but
hardwired for Camel and with added configuration glue in the UI to allow
selecting a Camel endpoint without having to write Camel or Spring code. This
won't be possible for all the endpoints, but will work for many useful ones.
DESIGN ISSUES to be resolved (Thanks to [~joewitt],
[[email protected]], [~ozhurakousky], for raising these items):
* We want to add this feature without requiring that big blocks of Camel or
Spring dependency code get sucked in to NiFi deployments that don't require
them. Appropriate bundle isolation (at NiFi and possibly Camel levels), plus
use of dynamic class loaders, are clearly part of the solution, but this
strategy needs to be elaborated. The upcoming NiFi Extension Registry may also
be part of the solution.
* We want to avoid run-time, or even configuration-time, dependency resolution.
While maven is a widely acknowledged utility for dependency resolution, it is
not allowable in all environments. Resolving all dependencies into a metadata
form at build time is preferable, and I believe achievable. A dependency
resolver capable of downloading dependencies from maven repositories at config
time, if not present in classpath, may still be useful as an option in some
environments. The upcoming NiFi Extension Registry may also be part of the
solution, if trusted in deployment environments.
* The provenance-reporting feature of NiFi must be supported. How to integrate
this with Camel end-points needs design work. It may need to be different for
different end-points.
* Effective code review will require patience, since Spring and Camel expertise
may be less available in the community. Maintainability of the new processors
must also be considered, given this limitation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
