Re: Service controller list NiFi 0.x to 1.x

[Russell Bateman]

It's because product planning can't be done overnight. It was a fight 
(months ago) to get higher approval to move to1.x. 1.1.2was the current 
release when approval was given. We'll plan to move ahead to 1.2.xin a 
few months motivated by the next release. In development, I tend to keep 
up, but product moves behind. As soon as my code for this release is 
locked, I'll start a new project and move up--months ahead of everyone else.

We're not clustering yet. I think the make-over of the authentication 
files and this issue were really the only ones that gave us pause in 
making the move. I pushed hard because I'm tired of going back and forth 
between the UI changes. However, as principally a write of custom 
processors and a controller service or two, not much has really changed 
for me.

NiFi's a great framework!

Russ

On 05/16/2017 03:37 PM, Andy LoPresto wrote:
Russell,

Is there a reason you are planning your migration to 1.1.2 and not 
1.2.0? There are a number of significant feature and bug fixes that 
went into 1.2.0 that will likely make your experience much better.


Andy LoPresto
alopre...@apache.org <mailto:alopre...@apache.org>
/alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com>/
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On May 16, 2017, at 2:30 PM, Russell Bateman <r...@windofkeltia.com 
<mailto:r...@windofkeltia.com>> wrote:

We're still testing our migration from 0.7.2 to 1.1.2 which is 
imminent. We still don't know how much of this is going to have to be 
reconfigured. Our down-streamers tend to be on the grouchy side, so 
we're testing to see what's going to irritate them. Were it not for 
the magic ids, I'd feel safe writing a Python script to run through 
their flow tarballs fixing the controller-setting impedance 
mismatches. Who knows; maybe there will be nothing to change although 
we know for certain of a /DBConnectionPool/ property change.

I will come back here in a few hours or tomorrow with an update on 
how it's going. I'll try to keep the noise down because what may 
alarm us shouldn't be a cause for alarm for everyone.

Thanks for the replies,

Russ

On 05/16/2017 02:47 PM, Michael Moser wrote:
Hi Russ,

In NiFi 1.2.0, there are 3 Reporting Tasks
(SiteToSiteBulletionReportingTask, SiteToSiteProvenanceReportingTask,
and SiteToSiteStatusReportingTask) that have the capability of using a
Controller Service (StandardSSLContextService).  So yes, the Global ->
Controller Settings has a limited set of use cases, but they are very
important.  This becomes even more true considering the extensibility
of NiFi components.  Who can predict what new reporting tasks might be
created to meet end user's needs, and what controller services those
tasks might need?

This is definitely high on the list of things that catch NiFi users
off guard, as they transition from the 0.x versions into the 1.x
versions.  I can say that once I got used to using the Operate Palette
(especially for defining controller services!) then it didn't seem as
bad.

I'll see what I can add to the 0.x to 1.0.0 Migration Guide [1] to
call this out specifically.

-- Mike

[1] -https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance



On Tue, May 16, 2017 at 3:53 PM, Russell Bateman<r...@windofkeltia.com>  wrote:
Thanks, Andrew!

(From [1]:) "This means that the service will be available to all Processors
and Controller Services defined in that Process Group and below."

In my experience, this isn't true. If I create a controller via the General
menu in the very root of my NiFi canvas, configure its name in Settings,
calling it Jack, then I create a new process group, then configure a new
processor in that group, when I try to configure to use the controller, Jack
is not among the options.

In order for the statement above to be true, I have to create it via the
gear icon in the Operate menu/palette.

Is this not what you see?

So, beginning sometime in 1.x, the Controller Settings option in the General
menu became useless, even at the top level except for "all ReportingTasks
and services defined in the Controller Settings." But, when would any
reporting task or other service defined be able to benefit? Never if I'm any
judge.

I think this is much more than a mere documentation issue. I wonder if
removing the Controller Services... option from the General menu would not
be the most important thing to do (even before documenting the gear icon in
the Operate menu).

Russ


On 05/16/2017 10:05 AM, Andrew Lim wrote:

Hi Russell,

Thanks for your question.

Yes, working with Controller Services has definitely changed in 1.x compared
to 0.x NiFi.  Matt Gilman wrote a nice article about how Controller Service
scoping was updated in 1.x with the introduction of Multi-Tenant
Authorization and also discusses the recent improvements made in NiFi 1.2 to
alleviate some of the user confusion around scoping [1].   If you would like
to see further details, the parent Jira for the improvements can be found
here [2].

I think there is opportunity to improve the Apache documentation we have
around this functionality, so I just filed a new Jira [3].

Let us know if you have any more questions.

Thanks,

Drew

[1]
https://community.hortonworks.com/articles/90259/understanding-controller-service-availability-in-a.html
[2]https://issues.apache.org/jira/browse/NIFI-3128
[3]https://issues.apache.org/jira/browse/NIFI-3911



On May 16, 2017, at 11:28 AM, Russell Bateman<r...@windofkeltia.com>  wrote:

It appears to me that that, unlike what happened in NiFi 0.x, in 1.x when I
look at controller services via the General menu -> Controller Services,
what I see is totally different from what I see when I configure controller
services for a processor.

If I use the General menu to set up my controller services, I do not see nor
am I given the option of using them in particular for processors I'm
configuring. Instead, I appear to get a "Process Group Configuration and a
list of controller services which are not the ones I'm looking for (because
when I set them up, I gave them "special" names or renamed names I could
recognize apart from any other use).

Note: I'm more of a processor and controller service author than an
experienced user of NiFi, so I may just be hopelessly confused.

My question is what's the point of being able to configure controller
services "globally" or "generally" if you can't reach them when you need
them?

Please confirm that I'm not just smoking funny weed and that this is
different, in fact, from how it worked in 0.7.1.

Thanks.