[
https://issues.apache.org/jira/browse/NIFI-5519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess resolved NIFI-5519.
--------------------------------
Resolution: Won't Fix
I think that's why this Jira has sat around for so long, most ListXYZ
processors are true source processors because they mostly deal with filesystems
which are hierarchical and can be recursed for a single overall resource
(top-level folder, etc.) Although databases tend to be less hierarchical, I
think between the Category, Schema, and Table Name properties being able to
match on wildcards, we can cover all tables for a single overall resource (the
top-level database). The usual convention is to have multiple instances of the
processor for each top-level resource (i.e. database connection).
Having said that, if we want to leverage ControllerServiceLookups to allow a
single instance to perform more diverse "work", we may want to look at another
set of processors that better support swapping out controller services based on
attribute values.
> Allow ListDatabaseTables to accept incoming connections
> -------------------------------------------------------
>
> Key: NIFI-5519
> URL: https://issues.apache.org/jira/browse/NIFI-5519
> Project: Apache NiFi
> Issue Type: Wish
> Reporter: Matt Burgess
> Assignee: Jim Steinebrey
> Priority: Major
> Time Spent: 1h
> Remaining Estimate: 0h
>
> As of [NIFI-5229|https://issues.apache.org/jira/browse/NIFI-5229],
> DBCPConnectionPoolLookup allows the dynamic selection of a DBCPConnectionPool
> by name. This allows processors who are to perform the same work on multiple
> databases to be able to do so by providing individual flow files upstream
> with the database.name attribute set.
> However ListDatabaseTables does not accept incoming connections, so you
> currently need 1 DBCPConnectionPool per database, plus 1 ListDatabaseTables
> per database, each using a corresponding DBCPConnectionPool. It would be nice
> if ListDatabaseTables could accept incoming connection(s), if only to provide
> attributes for selecting the DBCPConnectionPool.
> I propose the behavior be like other processors that can generate data with
> or without an incoming connection (such as GenerateTableFetch, see
> [NIFI-2881|https://issues.apache.org/jira/browse/NIFI-2881] for details). In
> general that means if there is an incoming non-loop connection, it becomes
> more "event-driven" in the sense that it will not execute if there is no
> FlowFile on which to work. If there is no incoming connection, then it would
> run as it always has, on its Run Schedule and with State Management, so as
> not to re-list the same tables every time it executes.
> However with an incoming connection and an available FlowFile, the behavior
> could be that all tables for that database are listed, meaning that processor
> state would not be updated nor queried, making it fully "event-driven". If
> the tables for a database are not to be re-listed, the onus would be on the
> upstream flow to not send a flow file for that database. This is not a
> requirement, just a suggestion; it could be more flexible by honoring
> processor state if the Refresh Interval is non-zero, but I think that adds
> too much complexity for the user, for little payoff.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
