Andrew,
You are right that it is a common request and desire and the
dev/stagin/prod lifecycle you mention makes sense and is also common.
In the upcoming nifi 1.0 release a variable registry has been
introduced which will allow you to do exactly what you show above for
any property of any component (processor or controller service) that
supports expression language. Several more processors have been
updated to be able to take advantage of this but many more need to be.
We can and should also keep making progress here well beyond what is
in the upcoming 1.0 release. We should make the variable registry
editable through the UI and it will need to support sensitive
properties. The work in 1.0 lays that foundation nicely. We should
also be able to support things like cluster wide scope vs node
specific scope. Cluster wide scope examples would be things like
passwords for external services and node specific scope would be
things like directory locations on the local file system.
Also aiding the lifecycle you mention is that in the 1.0 line we now
have support for deterministic template export. This means you can
save templates into git and use normal diffing tools and get far more
predictable results which also makes it easier to see what has
changed. We should also go further there to include graphical diffs
(of the flow graph itself not just some text representation). This
deterministic template support combined with the variable registry
means they're more portable and more useful. Again, here too we want
to do much more.
We just kicked off a vote to release the 1.0-Beta and once that is out
it would be great if you could give that a shot and see if that helps
you make progress.
Thanks
Joe
On Fri, Aug 5, 2016 at 6:23 PM, Andrew Serff <andrew.se...@acesinc.net> wrote:
> I have what I’m hoping is a simple question that I just can’t find the
> answer to. We run our NiFi in development, integration and production
> environments and in each environment we have different login properties
> needed for things like Mongo processors, JMS processors, etc. The
> username/password and url to log into mongo for instance is different in our
> dev environment and our productions environment. We have many of these
> processors within our flow and when we move it from one environment to
> another, we have to open the flow file and update all those processors. We
> can do this brute force in the flow.xml file, but it’s still a very manual
> process going from one environment to another.
>
> Is there a way to externalize these kinds of properties from the flow.xml?
> I’ve tried putting them into the nifi.conf and access them with something
> like ${mongo.username}, but that doesn’t work. I don’t think the Mongo
> Processor supports expression language, but I’ve also tried this in
> something like the ListenTCP processor that does and it still doesn’t work.
>
> It seems like this has to be a very common problem, so I hope I’ve just
> missed it somewhere in the docs. I can’t find anything that would help
> besides a ControllerService, but the processors would need to support it and
> the ones out of the box don’t seem to. I also don’t see how you’d pull
> properties from the nifi.conf file into those Controller Services either.
>
> If anyone could help us make out deployments more streamlined, that would be
> great!
>
> Thanks!
> Andrew
