Shot in the dark here, but what you try to do is create a custom connection pool service that uses dynamic properties to build a "pool of connection pools." You could then use the property names as hints for where to send the queries.
On Wed, Apr 25, 2018 at 6:19 AM Rishab Prasad <rishabprasad...@gmail.com> wrote: > Hi, > > Basically, there are 'n' number of databases that we are dealing with. We > need to fetch the data from the source database into HDFS. Now since we are > dealing with many databases, the source database is not static and changes > every now and then. And every time the source database changes we manually > need to change the value for the connection parameters in > DBCPConnectionPool. Now, people suggest that for 'n' databases create 'n' > connections for each database, but that is not possible because 'n' is a > big number and creating that many connections in DBCPConnectionPool is not > possible. So we were looking for a way where we can specify all the > connection parameters in a file present in our local system and then make > the DBCPConnectionPool controller service to read the values from the file. > In that way we can simply change the value in the file present in the local > system. No need to alter anything in the dataflow. But it turns out that > FlowFile attributes are not available to the controller services as the > expression language is evaluated at the time of service enable. > > So can you suggest a way where I can achieve my requirement (except > 'variable.registry' ) ? I am looking to develop a custom controller service > that can serve the requirement but how do I make the flowfile attributes > available to the service? >
