Re: Variable / Nifi-EL for RPG Endpoint?

2018-09-06 Thread Bryan Bende
That makes sense, not having connectivity definitely introduces some additional work. The approach you could take is to have a NiFi Registry instance per environment [1] and then use the NiFi CLI [2] to export and import between them. You are right that the RPG and secrets would have to be applie

Re: Variable / Nifi-EL for RPG Endpoint?

2018-09-06 Thread Jon Logan
Yeah, I meant to say NiFi Registry but substituted in Docker instead! Part of the reason we've approached NiFi registry with caution is the lack of connectivity between installations. Would the recommended approach be shipping a registry containing the flows desired, and then shipping the NiFi cont

Re: Variable / Nifi-EL for RPG Endpoint?

2018-09-06 Thread Bryan Bende
Sorry Jon, I made the assumption that you were talking about saving a flow to NiFi Registry from say you dev environment and then importing into another environment and figuring out how to deal with the RPG URL. The process I described is how RPG URLs are handled with versioned flows in NiFi Regis

Re: Variable / Nifi-EL for RPG Endpoint?

2018-09-06 Thread Andy LoPresto
I think Bryan was referring to Apache NiFi Registry [1] which is the complementary standalone application which hosts version controlled flow snippets. [1] https://nifi.apache.org/registry.html Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.co

Re: Variable / Nifi-EL for RPG Endpoint?

2018-09-06 Thread Jon Logan
Thanks Bryan -- Is that in terms of Docker Registry? We don't currently use Docker Registry, but our plan was to bundle a flow.gz into a Docker image directly. I'm not sure if maybe we're approaching this problem in a weird way? How do most people handle multiple instances in different environments

Re: Variable / Nifi-EL for RPG Endpoint?

2018-09-06 Thread Bryan Bende
Jon, The RPG URL is treated similar to sensitive properties or parent controller services, meaning that after deploying your flow you can issue an update to the RPG to set the URL to given environment's value and that change will not be considered a change as far as version control and will be ret

Variable / Nifi-EL for RPG Endpoint?

2018-09-06 Thread Jon Logan
We are running into a slight issue when we wish to migrate between NiFi cluster instances...the RPG endpoint in our flow seems to disallow setting of a variable, which makes our flow have to be specific to the specific environment it's being deployed in. It seems odd that it won't let us use an exp

Status of "event-driven" scheduling

2018-09-06 Thread Joe Percivall
Hey everyone, The dataflow I'm running has one main flow and a couple other disjoint process groups. Within that main flow, there are sections which aren't used very often. In trying to optimize things, I looked into the guidance we have on the "event-driven" scheduling type. There doesn't appear