Re: Clarification Regarding custom controller service & AWS-CREDENTIAL-CONTROLER-SERVICE
Thanks Joe and i am extremely sorry being impatient(bcoz today i have a solution meeting to give the details). I have gone through the nifi developer guide But i am not able to understand how puts3Object processor is getting my custom awscrdentailProvider controller dependency.as i have never given this dependency in pom.And from the document its saying it is dynamically its loading the implementation of the controller service by the class loader. whenever you have time please explain, how NIFI-AWS-PROCESSOR service is getting the dependency of AWS-CRDENTAIL-CONTROLER custom controller. Already the pom of AWS-CRDENTAIL-CONTROLER custom controller contain dependency of custom NIFI-AWS-PROCESSOR . Regards, Sanjeet On Mon, Oct 21, 2019 at 8:12 AM Joe Witt wrote: > Sanjeet > > Your e-mail was sent 10 hours ago and is during the weekend. Please be > patient. > > I would do two nars. The controller service nar and the processor nar > which depends on it. This way you can have other processor nars that also > depend on that controller service nar if necessary. > > Thanks > > On Sun, Oct 20, 2019 at 10:26 PM sanjeet rath > wrote: > >> Hi Team, >> >> Please clarify my trailed mail query. >> >> >> On Sun, 20 Oct, 2019, 9:35 PM sanjeet rath, >> wrote: >> >>> Hi Team, >>> >>> I have a requirement of building a custom >>> controller service(AWS-CREDENTIAL-CONTROLER-SERVICE) and custom processor >>> service(NIFI-AWS-SERVICE).for puts3Object. >>> Already code changes are done for both of it. and i am building 2 >>> separate nars . >>> The custom controller service(AWS-CREDENTIAL-CONTROLER-SERVICE) nar >>> contain dependency of custom processor service(NIFI-AWS-SERVICE). >>> >>> So for puts3Object processor, it should use my custom controller service. >>> >>> My question is, Should i build 2 separate nars , 1 for custom processor >>> service(NIFI-AWS-SERVICE) and 1 for custom >>> controller service(AWS-CREDENTIAL-CONTROLER-SERVICE). and putting both the >>> nars , in lib folder to make nifi work. >>> >>> Or should i build only 1 nar file for custom >>> controller service(AWS-CREDENTIAL-CONTROLER-SERVICE) as it has a pom >>> dependency of custom processor service(NIFI-AWS-SERVICE). >>> >>> Please clarify. >>> >>> Regards, >>> Sanjeet >>> >>> >>> >>> >>> >>> -- >>> Sanjeet Kumar Rath, >>> mob- +91 8777577470 >>> >>> -- Sanjeet Kumar Rath, mob- +91 8777577470
Re: Clarification Regarding custom controller service & AWS-CREDENTIAL-CONTROLER-SERVICE
Sanjeet Your e-mail was sent 10 hours ago and is during the weekend. Please be patient. I would do two nars. The controller service nar and the processor nar which depends on it. This way you can have other processor nars that also depend on that controller service nar if necessary. Thanks On Sun, Oct 20, 2019 at 10:26 PM sanjeet rath wrote: > Hi Team, > > Please clarify my trailed mail query. > > > On Sun, 20 Oct, 2019, 9:35 PM sanjeet rath, > wrote: > >> Hi Team, >> >> I have a requirement of building a custom >> controller service(AWS-CREDENTIAL-CONTROLER-SERVICE) and custom processor >> service(NIFI-AWS-SERVICE).for puts3Object. >> Already code changes are done for both of it. and i am building 2 >> separate nars . >> The custom controller service(AWS-CREDENTIAL-CONTROLER-SERVICE) nar >> contain dependency of custom processor service(NIFI-AWS-SERVICE). >> >> So for puts3Object processor, it should use my custom controller service. >> >> My question is, Should i build 2 separate nars , 1 for custom processor >> service(NIFI-AWS-SERVICE) and 1 for custom >> controller service(AWS-CREDENTIAL-CONTROLER-SERVICE). and putting both the >> nars , in lib folder to make nifi work. >> >> Or should i build only 1 nar file for custom >> controller service(AWS-CREDENTIAL-CONTROLER-SERVICE) as it has a pom >> dependency of custom processor service(NIFI-AWS-SERVICE). >> >> Please clarify. >> >> Regards, >> Sanjeet >> >> >> >> >> >> -- >> Sanjeet Kumar Rath, >> mob- +91 8777577470 >> >>
Re: Clarification Regarding custom controller service & AWS-CREDENTIAL-CONTROLER-SERVICE
Hi Team, Please clarify my trailed mail query. On Sun, 20 Oct, 2019, 9:35 PM sanjeet rath, wrote: > Hi Team, > > I have a requirement of building a custom > controller service(AWS-CREDENTIAL-CONTROLER-SERVICE) and custom processor > service(NIFI-AWS-SERVICE).for puts3Object. > Already code changes are done for both of it. and i am building 2 separate > nars . > The custom controller service(AWS-CREDENTIAL-CONTROLER-SERVICE) nar > contain dependency of custom processor service(NIFI-AWS-SERVICE). > > So for puts3Object processor, it should use my custom controller service. > > My question is, Should i build 2 separate nars , 1 for custom processor > service(NIFI-AWS-SERVICE) and 1 for custom > controller service(AWS-CREDENTIAL-CONTROLER-SERVICE). and putting both the > nars , in lib folder to make nifi work. > > Or should i build only 1 nar file for custom > controller service(AWS-CREDENTIAL-CONTROLER-SERVICE) as it has a pom > dependency of custom processor service(NIFI-AWS-SERVICE). > > Please clarify. > > Regards, > Sanjeet > > > > > > -- > Sanjeet Kumar Rath, > mob- +91 8777577470 > >
Clarification Regarding custom controller service & AWS-CREDENTIAL-CONTROLER-SERVICE
Hi Team, I have a requirement of building a custom controller service(AWS-CREDENTIAL-CONTROLER-SERVICE) and custom processor service(NIFI-AWS-SERVICE).for puts3Object. Already code changes are done for both of it. and i am building 2 separate nars . The custom controller service(AWS-CREDENTIAL-CONTROLER-SERVICE) nar contain dependency of custom processor service(NIFI-AWS-SERVICE). So for puts3Object processor, it should use my custom controller service. My question is, Should i build 2 separate nars , 1 for custom processor service(NIFI-AWS-SERVICE) and 1 for custom controller service(AWS-CREDENTIAL-CONTROLER-SERVICE). and putting both the nars , in lib folder to make nifi work. Or should i build only 1 nar file for custom controller service(AWS-CREDENTIAL-CONTROLER-SERVICE) as it has a pom dependency of custom processor service(NIFI-AWS-SERVICE). Please clarify. Regards, Sanjeet -- Sanjeet Kumar Rath, mob- +91 8777577470
RE: Custom Controller Service
Thank you, Mark! It worked as great per your suggestion.
Thanks
Kumiko
From: Mark Payne [mailto:marka...@hotmail.com]
Sent: Friday, June 17, 2016 12:26 PM
To: users@nifi.apache.org
Cc: Ki Kang <ki.k...@ds-iq.com>; Kevin Verhoeven <kevin.verhoe...@ds-iq.com>
Subject: Re: Custom Controller Service
Hi Kumiko,
I would recommend that in your OnEnabled method that you just create a
ScheduledExecutorService and
schedule the task to occur every 24 hours or however you'd like. Then, in your
OnDisabled method call
shutdown on that ScheduledExecutorService.
A word to the wise, though - NiFi tends to have quite a few threads running, so
for debugging purposes
it is easier to see what's going on if you provide a ThreadFactory to your
executor so that you can name
your threads appropriately. Something like:
Executors.newScheduledThreadPool(3, new ThreadFactory() {
@Override
public Thread newThread(Runnable r) {
final Thread t = Executors.defaultThreadFactory().newThread(r);
t.setDaemon(true);
t.setName("Refresh Azure DataLake Token");
return t;
}
});
I hope this helps!
Thanks
-Mark
On Jun 17, 2016, at 3:00 PM, Kumiko Yada
<kumiko.y...@ds-iq.com<mailto:kumiko.y...@ds-iq.com>> wrote:
Hello,
I wrote the Custom Controller Service to authenticate the Azure Datalake Store,
and I’d like to refresh a token every 24 hours. What is the best way to
authenticate every 24 hours? Should I loop the authenticate code every 24
hours in OnEnabled?
Thanks
Kumiko
Re: Custom Controller Service
We could potentially use a Reporting Task for this, actually. Not really the
intent
of the Reporting Task capability, but it does provide all the necessary
lifecycle constructs
and is generally easier to implement than Controller Services.
-Mark
> On Jun 17, 2016, at 3:29 PM, Joe Witt <joe.w...@gmail.com> wrote:
>
> ...i wonder if we've done the same ourselves or given similar advice
> to others. This feels like something the framework should make
> available as an expected lifecycle point. I could see it making sense
> for such cases that we would allow 'OnScheduled' to be on controller
> services and let the user schedule how often that should get invoked
> and just never let them set it to 0 in the case of a controller
> service.
>
> On Fri, Jun 17, 2016 at 3:25 PM, Mark Payne <marka...@hotmail.com> wrote:
>> Hi Kumiko,
>>
>> I would recommend that in your OnEnabled method that you just create a
>> ScheduledExecutorService and
>> schedule the task to occur every 24 hours or however you'd like. Then, in
>> your OnDisabled method call
>> shutdown on that ScheduledExecutorService.
>>
>> A word to the wise, though - NiFi tends to have quite a few threads running,
>> so for debugging purposes
>> it is easier to see what's going on if you provide a ThreadFactory to your
>> executor so that you can name
>> your threads appropriately. Something like:
>>
>> Executors.newScheduledThreadPool(3, new ThreadFactory() {
>>@Override
>>public Thread newThread(Runnable r) {
>>final Thread t =
>> Executors.defaultThreadFactory().newThread(r);
>>t.setDaemon(true);
>>t.setName("Refresh Azure DataLake Token");
>>return t;
>> }
>> });
>>
>> I hope this helps!
>>
>> Thanks
>> -Mark
>>
>>
>> On Jun 17, 2016, at 3:00 PM, Kumiko Yada <kumiko.y...@ds-iq.com> wrote:
>>
>> Hello,
>>
>> I wrote the Custom Controller Service to authenticate the Azure Datalake
>> Store, and I’d like to refresh a token every 24 hours. What is the best way
>> to authenticate every 24 hours? Should I loop the authenticate code every
>> 24 hours in OnEnabled?
>>
>> Thanks
>> Kumiko
>>
>>
Re: Custom Controller Service
...i wonder if we've done the same ourselves or given similar advice
to others. This feels like something the framework should make
available as an expected lifecycle point. I could see it making sense
for such cases that we would allow 'OnScheduled' to be on controller
services and let the user schedule how often that should get invoked
and just never let them set it to 0 in the case of a controller
service.
On Fri, Jun 17, 2016 at 3:25 PM, Mark Payne <marka...@hotmail.com> wrote:
> Hi Kumiko,
>
> I would recommend that in your OnEnabled method that you just create a
> ScheduledExecutorService and
> schedule the task to occur every 24 hours or however you'd like. Then, in
> your OnDisabled method call
> shutdown on that ScheduledExecutorService.
>
> A word to the wise, though - NiFi tends to have quite a few threads running,
> so for debugging purposes
> it is easier to see what's going on if you provide a ThreadFactory to your
> executor so that you can name
> your threads appropriately. Something like:
>
> Executors.newScheduledThreadPool(3, new ThreadFactory() {
> @Override
> public Thread newThread(Runnable r) {
> final Thread t =
> Executors.defaultThreadFactory().newThread(r);
> t.setDaemon(true);
> t.setName("Refresh Azure DataLake Token");
> return t;
> }
> });
>
> I hope this helps!
>
> Thanks
> -Mark
>
>
> On Jun 17, 2016, at 3:00 PM, Kumiko Yada <kumiko.y...@ds-iq.com> wrote:
>
> Hello,
>
> I wrote the Custom Controller Service to authenticate the Azure Datalake
> Store, and I’d like to refresh a token every 24 hours. What is the best way
> to authenticate every 24 hours? Should I loop the authenticate code every
> 24 hours in OnEnabled?
>
> Thanks
> Kumiko
>
>
Re: Custom Controller Service
Hi Kumiko,
I would recommend that in your OnEnabled method that you just create a
ScheduledExecutorService and
schedule the task to occur every 24 hours or however you'd like. Then, in your
OnDisabled method call
shutdown on that ScheduledExecutorService.
A word to the wise, though - NiFi tends to have quite a few threads running, so
for debugging purposes
it is easier to see what's going on if you provide a ThreadFactory to your
executor so that you can name
your threads appropriately. Something like:
Executors.newScheduledThreadPool(3, new ThreadFactory() {
@Override
public Thread newThread(Runnable r) {
final Thread t = Executors.defaultThreadFactory().newThread(r);
t.setDaemon(true);
t.setName("Refresh Azure DataLake Token");
return t;
}
});
I hope this helps!
Thanks
-Mark
> On Jun 17, 2016, at 3:00 PM, Kumiko Yada <kumiko.y...@ds-iq.com> wrote:
>
> Hello,
>
> I wrote the Custom Controller Service to authenticate the Azure Datalake
> Store, and I’d like to refresh a token every 24 hours. What is the best way
> to authenticate every 24 hours? Should I loop the authenticate code every 24
> hours in OnEnabled?
>
> Thanks
> Kumiko
Custom Controller Service
Hello, I wrote the Custom Controller Service to authenticate the Azure Datalake Store, and I'd like to refresh a token every 24 hours. What is the best way to authenticate every 24 hours? Should I loop the authenticate code every 24 hours in OnEnabled? Thanks Kumiko
