Re: How to enable debug logging per processor vs. system wide

2015-09-04 Thread Matt Gilman 
Rick,

The logging is configured in /conf/logback.xml. In that file you
can specify the fully qualified class name for the processor you want to
enable specific logging for. For instance,



This would enable DEBUG level logging for every GetFile in your flow.

Matt

On Fri, Sep 4, 2015 at 4:48 PM, Rick Braddy  wrote:

> I know there's a way to enable DEBUG logging... thought I saw it somewhere
> in the docs, but just researched and the Developer's Guide says to use
> ProcessorLog for DEBUG, but doesn't say how to enable debug logging (or
> what the logging levels are, how to change logging levels system-wide or
> per processor).
>
> Any suggestions?
>

How to enable debug logging per processor vs. system wide

2015-09-04 Thread Rick Braddy 
I know there's a way to enable DEBUG logging... thought I saw it somewhere in 
the docs, but just researched and the Developer's Guide says to use 
ProcessorLog for DEBUG, but doesn't say how to enable debug logging (or what 
the logging levels are, how to change logging levels system-wide or per 
processor).

Any suggestions?

Re: POM dependency failures

2015-09-04 Thread Bryan Bende 
Glad you got it working!

-Bryan

On Fri, Sep 4, 2015 at 3:20 PM, Rick Braddy  wrote:

> That's what it was. Resolution was simple.
>
> ln -s ~/.m2 /root
>
> Now I can build, update and restart Nifi service as root and the
> dependencies are found in both my local account and root.
>
> Not ideal, but what's a developer to do...
>
> Thanks for the assist.
> Rick
>
> -Original Message-
> From: Rick Braddy [mailto:rbra...@softnas.com]
> Sent: Friday, September 04, 2015 1:03 PM
> To: dev@nifi.apache.org
> Subject: RE: POM dependency failures
>
> I may know what the problem is...
>
> I decided to run the build as 'root' user so my rebuild script would be
> able to copy over my NAR bundle and restart Nifi service... so it appears
> that since the Maven repository is associated with my local home directory,
> that may be why it's failing.
>
> I will create a link and see if that resolves the issue.
>
> Thanks for the pointer.
>
> Rick
>
> -Original Message-
> From: Rick Braddy [mailto:rbra...@softnas.com]
> Sent: Friday, September 04, 2015 1:00 PM
> To: dev@nifi.apache.org
> Subject: RE: POM dependency failures
>
> Bryan,
>
> Yes.  The 0.3.0-SNAPSHOT folders are there for each Nifi subsystem folder.
>
> root@rick-dev:~/.m2/repository/org/apache/nifi# ls
> nifi  nifi-external  nifi-mock
> nifi-processor-utils
> nifi-api  nifi-framework-bundle  nifi-nar-bundles
>  nifi-provenance-repository-bundle
> nifi-commons  nifi-maven-archetypes  nifi-nar-maven-plugin
>
> ./nifi-provenance-repository-bundle/0.3.0-SNAPSHOT
> ./nifi-maven-archetypes/0.3.0-SNAPSHOT
> ./nifi-mock/0.3.0-SNAPSHOT
> ./nifi/0.3.0-SNAPSHOT
> ./nifi-external/0.3.0-SNAPSHOT
> ./nifi-processor-utils/0.3.0-SNAPSHOT
> ./nifi-nar-bundles/0.3.0-SNAPSHOT
> ./nifi-commons/0.3.0-SNAPSHOT
> ./nifi-api/0.3.0-SNAPSHOT
> ./nifi-framework-bundle/0.3.0-SNAPSHOT
>
> Rick
>
> -Original Message-
> From: Bryan Bende [mailto:bbe...@gmail.com]
> Sent: Friday, September 04, 2015 12:54 PM
> To: dev@nifi.apache.org
> Subject: Re: POM dependency failures
>
> Rick,
>
> Can you check in your local Maven repository to see if the 0.3.0-SNAPSHOTs
> are in there?  It is typically in HOME/.m2
>
> As one example ~/.m2/repository/org/apache/nifi/nifi-api/  should have a
> sub-directory for 0.3.0-SNAPSHOT.
>
> -Bryan
>
> On Fri, Sep 4, 2015 at 2:50 PM, Rick Braddy  wrote:
>
> > Yes. I pulled source from Nifi on Github[1] and built it locally,
> > which seemed to work okay at first.
> >
> > Is there a better process to use?
> >
> > [1] https://github.com/apache/nifi
> >
> >
> > -Original Message-
> > From: Aldrin Piri [mailto:aldrinp...@gmail.com]
> > Sent: Friday, September 04, 2015 12:46 PM
> > To: dev@nifi.apache.org
> > Subject: Re: POM dependency failures
> >
> > Rick,
> >
> > Did you do a build (mvn install) of the 0.3.0 repository on the same
> > machine you are doing this development on?  We do not, as a project,
> > publish any snapshots and this would explain the issues in not being
> > able to locate the associate poms.
> >
> > On Fri, Sep 4, 2015 at 1:39 PM, Rick Braddy  wrote:
> >
> > > Hi,
> > >
> > > Does anyone know why these dependencies would suddenly start failing
> > > or how to go about resolving?
> > >
> > > [WARNING] The POM for org.apache.nifi:nifi-api:jar:0.3.0-SNAPSHOT is
> > > missing, no dependency information available [WARNING] The POM for
> > > org.apache.nifi:nifi-processor-utils:jar:0.3.0-SNAPSHOT is missing,
> > > no dependency information available [WARNING] The POM for
> > > org.apache.nifi:nifi-mock:jar:0.3.0-SNAPSHOT is missing, no
> > > dependency information available
> > >
> > > These errors occur during Maven build in a separate NAR bundle
> > > project created using Bryan's Customer Processor post here<
> > > http://bryanbende.com/development/2015/02/04/custom-processors-for-a
> > > pa
> > > che-nifi/>
> > > for developing custom processors.
> > >
> > > This Maven project had been building okay yesterday and just
> > > suddenly stopped working today (and I don't believe I changed any POM
> files)...
> > >
> > > Thanks
> > > Rick
> > >
> >
>

RE: POM dependency failures

2015-09-04 Thread Rick Braddy 
That's what it was. Resolution was simple.

ln -s ~/.m2 /root

Now I can build, update and restart Nifi service as root and the dependencies 
are found in both my local account and root.

Not ideal, but what's a developer to do...

Thanks for the assist.
Rick

-Original Message-
From: Rick Braddy [mailto:rbra...@softnas.com] 
Sent: Friday, September 04, 2015 1:03 PM
To: dev@nifi.apache.org
Subject: RE: POM dependency failures

I may know what the problem is...

I decided to run the build as 'root' user so my rebuild script would be able to 
copy over my NAR bundle and restart Nifi service... so it appears that since 
the Maven repository is associated with my local home directory, that may be 
why it's failing.

I will create a link and see if that resolves the issue.

Thanks for the pointer.

Rick

-Original Message-
From: Rick Braddy [mailto:rbra...@softnas.com]
Sent: Friday, September 04, 2015 1:00 PM
To: dev@nifi.apache.org
Subject: RE: POM dependency failures

Bryan,

Yes.  The 0.3.0-SNAPSHOT folders are there for each Nifi subsystem folder.

root@rick-dev:~/.m2/repository/org/apache/nifi# ls
nifi  nifi-external  nifi-mock  nifi-processor-utils
nifi-api  nifi-framework-bundle  nifi-nar-bundles   
nifi-provenance-repository-bundle
nifi-commons  nifi-maven-archetypes  nifi-nar-maven-plugin

./nifi-provenance-repository-bundle/0.3.0-SNAPSHOT
./nifi-maven-archetypes/0.3.0-SNAPSHOT
./nifi-mock/0.3.0-SNAPSHOT
./nifi/0.3.0-SNAPSHOT
./nifi-external/0.3.0-SNAPSHOT
./nifi-processor-utils/0.3.0-SNAPSHOT
./nifi-nar-bundles/0.3.0-SNAPSHOT
./nifi-commons/0.3.0-SNAPSHOT
./nifi-api/0.3.0-SNAPSHOT
./nifi-framework-bundle/0.3.0-SNAPSHOT

Rick

-Original Message-
From: Bryan Bende [mailto:bbe...@gmail.com]
Sent: Friday, September 04, 2015 12:54 PM
To: dev@nifi.apache.org
Subject: Re: POM dependency failures

Rick,

Can you check in your local Maven repository to see if the 0.3.0-SNAPSHOTs are 
in there?  It is typically in HOME/.m2

As one example ~/.m2/repository/org/apache/nifi/nifi-api/  should have a 
sub-directory for 0.3.0-SNAPSHOT.

-Bryan

On Fri, Sep 4, 2015 at 2:50 PM, Rick Braddy  wrote:

> Yes. I pulled source from Nifi on Github[1] and built it locally, 
> which seemed to work okay at first.
>
> Is there a better process to use?
>
> [1] https://github.com/apache/nifi
>
>
> -Original Message-
> From: Aldrin Piri [mailto:aldrinp...@gmail.com]
> Sent: Friday, September 04, 2015 12:46 PM
> To: dev@nifi.apache.org
> Subject: Re: POM dependency failures
>
> Rick,
>
> Did you do a build (mvn install) of the 0.3.0 repository on the same 
> machine you are doing this development on?  We do not, as a project, 
> publish any snapshots and this would explain the issues in not being 
> able to locate the associate poms.
>
> On Fri, Sep 4, 2015 at 1:39 PM, Rick Braddy  wrote:
>
> > Hi,
> >
> > Does anyone know why these dependencies would suddenly start failing 
> > or how to go about resolving?
> >
> > [WARNING] The POM for org.apache.nifi:nifi-api:jar:0.3.0-SNAPSHOT is 
> > missing, no dependency information available [WARNING] The POM for 
> > org.apache.nifi:nifi-processor-utils:jar:0.3.0-SNAPSHOT is missing, 
> > no dependency information available [WARNING] The POM for 
> > org.apache.nifi:nifi-mock:jar:0.3.0-SNAPSHOT is missing, no 
> > dependency information available
> >
> > These errors occur during Maven build in a separate NAR bundle 
> > project created using Bryan's Customer Processor post here< 
> > http://bryanbende.com/development/2015/02/04/custom-processors-for-a
> > pa
> > che-nifi/>
> > for developing custom processors.
> >
> > This Maven project had been building okay yesterday and just 
> > suddenly stopped working today (and I don't believe I changed any POM 
> > files)...
> >
> > Thanks
> > Rick
> >
>

RE: POM dependency failures

2015-09-04 Thread Rick Braddy 
I may know what the problem is...

I decided to run the build as 'root' user so my rebuild script would be able to 
copy over my NAR bundle and restart Nifi service... so it appears that since 
the Maven repository is associated with my local home directory, that may be 
why it's failing.

I will create a link and see if that resolves the issue.

Thanks for the pointer.

Rick

-Original Message-
From: Rick Braddy [mailto:rbra...@softnas.com] 
Sent: Friday, September 04, 2015 1:00 PM
To: dev@nifi.apache.org
Subject: RE: POM dependency failures

Bryan,

Yes.  The 0.3.0-SNAPSHOT folders are there for each Nifi subsystem folder.

root@rick-dev:~/.m2/repository/org/apache/nifi# ls
nifi  nifi-external  nifi-mock  nifi-processor-utils
nifi-api  nifi-framework-bundle  nifi-nar-bundles   
nifi-provenance-repository-bundle
nifi-commons  nifi-maven-archetypes  nifi-nar-maven-plugin

./nifi-provenance-repository-bundle/0.3.0-SNAPSHOT
./nifi-maven-archetypes/0.3.0-SNAPSHOT
./nifi-mock/0.3.0-SNAPSHOT
./nifi/0.3.0-SNAPSHOT
./nifi-external/0.3.0-SNAPSHOT
./nifi-processor-utils/0.3.0-SNAPSHOT
./nifi-nar-bundles/0.3.0-SNAPSHOT
./nifi-commons/0.3.0-SNAPSHOT
./nifi-api/0.3.0-SNAPSHOT
./nifi-framework-bundle/0.3.0-SNAPSHOT

Rick

-Original Message-
From: Bryan Bende [mailto:bbe...@gmail.com]
Sent: Friday, September 04, 2015 12:54 PM
To: dev@nifi.apache.org
Subject: Re: POM dependency failures

Rick,

Can you check in your local Maven repository to see if the 0.3.0-SNAPSHOTs are 
in there?  It is typically in HOME/.m2

As one example ~/.m2/repository/org/apache/nifi/nifi-api/  should have a 
sub-directory for 0.3.0-SNAPSHOT.

-Bryan

On Fri, Sep 4, 2015 at 2:50 PM, Rick Braddy  wrote:

> Yes. I pulled source from Nifi on Github[1] and built it locally, 
> which seemed to work okay at first.
>
> Is there a better process to use?
>
> [1] https://github.com/apache/nifi
>
>
> -Original Message-
> From: Aldrin Piri [mailto:aldrinp...@gmail.com]
> Sent: Friday, September 04, 2015 12:46 PM
> To: dev@nifi.apache.org
> Subject: Re: POM dependency failures
>
> Rick,
>
> Did you do a build (mvn install) of the 0.3.0 repository on the same 
> machine you are doing this development on?  We do not, as a project, 
> publish any snapshots and this would explain the issues in not being 
> able to locate the associate poms.
>
> On Fri, Sep 4, 2015 at 1:39 PM, Rick Braddy  wrote:
>
> > Hi,
> >
> > Does anyone know why these dependencies would suddenly start failing 
> > or how to go about resolving?
> >
> > [WARNING] The POM for org.apache.nifi:nifi-api:jar:0.3.0-SNAPSHOT is 
> > missing, no dependency information available [WARNING] The POM for 
> > org.apache.nifi:nifi-processor-utils:jar:0.3.0-SNAPSHOT is missing, 
> > no dependency information available [WARNING] The POM for 
> > org.apache.nifi:nifi-mock:jar:0.3.0-SNAPSHOT is missing, no 
> > dependency information available
> >
> > These errors occur during Maven build in a separate NAR bundle 
> > project created using Bryan's Customer Processor post here< 
> > http://bryanbende.com/development/2015/02/04/custom-processors-for-a
> > pa
> > che-nifi/>
> > for developing custom processors.
> >
> > This Maven project had been building okay yesterday and just 
> > suddenly stopped working today (and I don't believe I changed any POM 
> > files)...
> >
> > Thanks
> > Rick
> >
>

RE: POM dependency failures

2015-09-04 Thread Rick Braddy 
Bryan,

Yes.  The 0.3.0-SNAPSHOT folders are there for each Nifi subsystem folder.

root@rick-dev:~/.m2/repository/org/apache/nifi# ls
nifi  nifi-external  nifi-mock  nifi-processor-utils
nifi-api  nifi-framework-bundle  nifi-nar-bundles   
nifi-provenance-repository-bundle
nifi-commons  nifi-maven-archetypes  nifi-nar-maven-plugin

./nifi-provenance-repository-bundle/0.3.0-SNAPSHOT
./nifi-maven-archetypes/0.3.0-SNAPSHOT
./nifi-mock/0.3.0-SNAPSHOT
./nifi/0.3.0-SNAPSHOT
./nifi-external/0.3.0-SNAPSHOT
./nifi-processor-utils/0.3.0-SNAPSHOT
./nifi-nar-bundles/0.3.0-SNAPSHOT
./nifi-commons/0.3.0-SNAPSHOT
./nifi-api/0.3.0-SNAPSHOT
./nifi-framework-bundle/0.3.0-SNAPSHOT

Rick

-Original Message-
From: Bryan Bende [mailto:bbe...@gmail.com] 
Sent: Friday, September 04, 2015 12:54 PM
To: dev@nifi.apache.org
Subject: Re: POM dependency failures

Rick,

Can you check in your local Maven repository to see if the 0.3.0-SNAPSHOTs are 
in there?  It is typically in HOME/.m2

As one example ~/.m2/repository/org/apache/nifi/nifi-api/  should have a 
sub-directory for 0.3.0-SNAPSHOT.

-Bryan

On Fri, Sep 4, 2015 at 2:50 PM, Rick Braddy  wrote:

> Yes. I pulled source from Nifi on Github[1] and built it locally, 
> which seemed to work okay at first.
>
> Is there a better process to use?
>
> [1] https://github.com/apache/nifi
>
>
> -Original Message-
> From: Aldrin Piri [mailto:aldrinp...@gmail.com]
> Sent: Friday, September 04, 2015 12:46 PM
> To: dev@nifi.apache.org
> Subject: Re: POM dependency failures
>
> Rick,
>
> Did you do a build (mvn install) of the 0.3.0 repository on the same 
> machine you are doing this development on?  We do not, as a project, 
> publish any snapshots and this would explain the issues in not being 
> able to locate the associate poms.
>
> On Fri, Sep 4, 2015 at 1:39 PM, Rick Braddy  wrote:
>
> > Hi,
> >
> > Does anyone know why these dependencies would suddenly start failing 
> > or how to go about resolving?
> >
> > [WARNING] The POM for org.apache.nifi:nifi-api:jar:0.3.0-SNAPSHOT is 
> > missing, no dependency information available [WARNING] The POM for 
> > org.apache.nifi:nifi-processor-utils:jar:0.3.0-SNAPSHOT is missing, 
> > no dependency information available [WARNING] The POM for 
> > org.apache.nifi:nifi-mock:jar:0.3.0-SNAPSHOT is missing, no 
> > dependency information available
> >
> > These errors occur during Maven build in a separate NAR bundle 
> > project created using Bryan's Customer Processor post here< 
> > http://bryanbende.com/development/2015/02/04/custom-processors-for-a
> > pa
> > che-nifi/>
> > for developing custom processors.
> >
> > This Maven project had been building okay yesterday and just 
> > suddenly stopped working today (and I don't believe I changed any POM 
> > files)...
> >
> > Thanks
> > Rick
> >
>

Re: POM dependency failures

2015-09-04 Thread Bryan Bende 
Rick,

Can you check in your local Maven repository to see if the 0.3.0-SNAPSHOTs
are in there?  It is typically in HOME/.m2

As one example ~/.m2/repository/org/apache/nifi/nifi-api/  should have a
sub-directory for 0.3.0-SNAPSHOT.

-Bryan

On Fri, Sep 4, 2015 at 2:50 PM, Rick Braddy  wrote:

> Yes. I pulled source from Nifi on Github[1] and built it locally, which
> seemed to work okay at first.
>
> Is there a better process to use?
>
> [1] https://github.com/apache/nifi
>
>
> -Original Message-
> From: Aldrin Piri [mailto:aldrinp...@gmail.com]
> Sent: Friday, September 04, 2015 12:46 PM
> To: dev@nifi.apache.org
> Subject: Re: POM dependency failures
>
> Rick,
>
> Did you do a build (mvn install) of the 0.3.0 repository on the same
> machine you are doing this development on?  We do not, as a project,
> publish any snapshots and this would explain the issues in not being able
> to locate the associate poms.
>
> On Fri, Sep 4, 2015 at 1:39 PM, Rick Braddy  wrote:
>
> > Hi,
> >
> > Does anyone know why these dependencies would suddenly start failing
> > or how to go about resolving?
> >
> > [WARNING] The POM for org.apache.nifi:nifi-api:jar:0.3.0-SNAPSHOT is
> > missing, no dependency information available [WARNING] The POM for
> > org.apache.nifi:nifi-processor-utils:jar:0.3.0-SNAPSHOT is missing, no
> > dependency information available [WARNING] The POM for
> > org.apache.nifi:nifi-mock:jar:0.3.0-SNAPSHOT is missing, no dependency
> > information available
> >
> > These errors occur during Maven build in a separate NAR bundle project
> > created using Bryan's Customer Processor post here<
> > http://bryanbende.com/development/2015/02/04/custom-processors-for-apa
> > che-nifi/>
> > for developing custom processors.
> >
> > This Maven project had been building okay yesterday and just suddenly
> > stopped working today (and I don't believe I changed any POM files)...
> >
> > Thanks
> > Rick
> >
>

RE: POM dependency failures

2015-09-04 Thread Rick Braddy 
Yes. I pulled source from Nifi on Github[1] and built it locally, which seemed 
to work okay at first. 

Is there a better process to use?

[1] https://github.com/apache/nifi


-Original Message-
From: Aldrin Piri [mailto:aldrinp...@gmail.com] 
Sent: Friday, September 04, 2015 12:46 PM
To: dev@nifi.apache.org
Subject: Re: POM dependency failures

Rick,

Did you do a build (mvn install) of the 0.3.0 repository on the same machine 
you are doing this development on?  We do not, as a project, publish any 
snapshots and this would explain the issues in not being able to locate the 
associate poms.

On Fri, Sep 4, 2015 at 1:39 PM, Rick Braddy  wrote:

> Hi,
>
> Does anyone know why these dependencies would suddenly start failing 
> or how to go about resolving?
>
> [WARNING] The POM for org.apache.nifi:nifi-api:jar:0.3.0-SNAPSHOT is 
> missing, no dependency information available [WARNING] The POM for 
> org.apache.nifi:nifi-processor-utils:jar:0.3.0-SNAPSHOT is missing, no 
> dependency information available [WARNING] The POM for 
> org.apache.nifi:nifi-mock:jar:0.3.0-SNAPSHOT is missing, no dependency 
> information available
>
> These errors occur during Maven build in a separate NAR bundle project 
> created using Bryan's Customer Processor post here< 
> http://bryanbende.com/development/2015/02/04/custom-processors-for-apa
> che-nifi/>
> for developing custom processors.
>
> This Maven project had been building okay yesterday and just suddenly 
> stopped working today (and I don't believe I changed any POM files)...
>
> Thanks
> Rick
>

Re: POM dependency failures

2015-09-04 Thread Aldrin Piri 
Rick,

Did you do a build (mvn install) of the 0.3.0 repository on the same
machine you are doing this development on?  We do not, as a project,
publish any snapshots and this would explain the issues in not being able
to locate the associate poms.

On Fri, Sep 4, 2015 at 1:39 PM, Rick Braddy  wrote:

> Hi,
>
> Does anyone know why these dependencies would suddenly start failing or
> how to go about resolving?
>
> [WARNING] The POM for org.apache.nifi:nifi-api:jar:0.3.0-SNAPSHOT is
> missing, no dependency information available
> [WARNING] The POM for
> org.apache.nifi:nifi-processor-utils:jar:0.3.0-SNAPSHOT is missing, no
> dependency information available
> [WARNING] The POM for org.apache.nifi:nifi-mock:jar:0.3.0-SNAPSHOT is
> missing, no dependency information available
>
> These errors occur during Maven build in a separate NAR bundle project
> created using Bryan's Customer Processor post here<
> http://bryanbende.com/development/2015/02/04/custom-processors-for-apache-nifi/>
> for developing custom processors.
>
> This Maven project had been building okay yesterday and just suddenly
> stopped working today (and I don't believe I changed any POM files)...
>
> Thanks
> Rick
>

POM dependency failures

2015-09-04 Thread Rick Braddy 
Hi,

Does anyone know why these dependencies would suddenly start failing or how to 
go about resolving?

[WARNING] The POM for org.apache.nifi:nifi-api:jar:0.3.0-SNAPSHOT is missing, 
no dependency information available
[WARNING] The POM for org.apache.nifi:nifi-processor-utils:jar:0.3.0-SNAPSHOT 
is missing, no dependency information available
[WARNING] The POM for org.apache.nifi:nifi-mock:jar:0.3.0-SNAPSHOT is missing, 
no dependency information available

These errors occur during Maven build in a separate NAR bundle project created 
using Bryan's Customer Processor post 
here
 for developing custom processors.

This Maven project had been building okay yesterday and just suddenly stopped 
working today (and I don't believe I changed any POM files)...

Thanks
Rick

Re: Facing problem from ExecuteSQL processor

2015-09-04 Thread Aldrin Piri 
Shubham,

One more note.  While I was typing up the last email, it appears a ticket
was created to alter ExecuteSQL to carry out the functionality I think you
are looking for.  It can be located here:
https://issues.apache.org/jira/browse/NIFI-932

On Fri, Sep 4, 2015 at 1:25 PM, Aldrin Piri  wrote:

> Shubham,
>
> ExecuteSQL requires an input flowfile to cause processing to occur.  Based
> off your message, it sounds like this is the only instance and thus there
> is no action occurring.  This is certainly unclear on our behalf in terms
> of documentation on the processor.  By design, this processor would receive
> input input and use its attributes in conjunction with expression language
> [1] to create "dynamic" queries.
>
> If you were interested in running a static query, you can use
> GenerateFlowFile to provide input and trigger the processor to execute.
>
> [1]
> http://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html
>
>
>
> On Fri, Sep 4, 2015 at 7:40 AM, Bryan Bende  wrote:
>
>> Hi Shubham,
>>
>> I believe images usually get stripped out of the emails, can you upload
>> the
>> image somewhere and provide a link?
>>
>> Generally a processor will transfer data on a relationship, and a
>> relationship is connected to another processor, or the relationship can be
>> auto-terminated which means the data has reached a point where you don't
>> want to do anything else with it.
>> For testing, you may want to connect your ExecuteSQL processor to a
>> PutFile
>> processor that writes to a directory on your local machine, this way you
>> can see what is getting produced. Once you know it is working then you can
>> reconfigure the flow how you like.
>>
>> Thanks,
>>
>> Bryan
>>
>> On Fri, Sep 4, 2015 at 4:33 AM, Shubham Bansal <
>> shubham.ban...@whishworks.com> wrote:
>>
>> > Hi,
>> >
>> > I am using ExecuteSQL processor in my flow. Infact , that is the only
>> > processor I have added to the flow. I have added DBCPConnectionPool
>> service
>> > to this processor. I am attaching screenshots for the configuration done
>> > for these. There is no error on the connection side but I am not able to
>> > fetch any data. Where exactly will this data go ? Please help me with
>> this.
>> >
>> > Thanks,
>> >
>> > --
>> >
>> >
>> >
>> > *Shubham Bansal*, Software Engineer,
>> >
>> > 4th Floor, Pioneer Towers, Madhapur, Hyderabad
>> > *M: **+91 9059328793 <%2B91%20%C2%A09059328793> *
>> > *W: *ww w.whishworks.com
>> > 
>> >
>> >   
>> > 
>> > 
>> >
>> > The contents of this e-mail are confidential and for the exclusive use
>> of
>> > the intended recipient. If you receive this e-mail in error please
>> delete
>> > it from your system immediately and notify us either by e-mail or
>> > telephone. You should not copy, forward or otherwise disclose the
>> content
>> > of the e-mail. The views expressed in this communication may not
>> > necessarily be the view held by WHISHWORKS.
>>
>
>

Re: Facing problem from ExecuteSQL processor

2015-09-04 Thread Aldrin Piri 
Shubham,

ExecuteSQL requires an input flowfile to cause processing to occur.  Based
off your message, it sounds like this is the only instance and thus there
is no action occurring.  This is certainly unclear on our behalf in terms
of documentation on the processor.  By design, this processor would receive
input input and use its attributes in conjunction with expression language
[1] to create "dynamic" queries.

If you were interested in running a static query, you can use
GenerateFlowFile to provide input and trigger the processor to execute.

[1]
http://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html



On Fri, Sep 4, 2015 at 7:40 AM, Bryan Bende  wrote:

> Hi Shubham,
>
> I believe images usually get stripped out of the emails, can you upload the
> image somewhere and provide a link?
>
> Generally a processor will transfer data on a relationship, and a
> relationship is connected to another processor, or the relationship can be
> auto-terminated which means the data has reached a point where you don't
> want to do anything else with it.
> For testing, you may want to connect your ExecuteSQL processor to a PutFile
> processor that writes to a directory on your local machine, this way you
> can see what is getting produced. Once you know it is working then you can
> reconfigure the flow how you like.
>
> Thanks,
>
> Bryan
>
> On Fri, Sep 4, 2015 at 4:33 AM, Shubham Bansal <
> shubham.ban...@whishworks.com> wrote:
>
> > Hi,
> >
> > I am using ExecuteSQL processor in my flow. Infact , that is the only
> > processor I have added to the flow. I have added DBCPConnectionPool
> service
> > to this processor. I am attaching screenshots for the configuration done
> > for these. There is no error on the connection side but I am not able to
> > fetch any data. Where exactly will this data go ? Please help me with
> this.
> >
> > Thanks,
> >
> > --
> >
> >
> >
> > *Shubham Bansal*, Software Engineer,
> >
> > 4th Floor, Pioneer Towers, Madhapur, Hyderabad
> > *M: **+91 9059328793 <%2B91%20%C2%A09059328793> *
> > *W: *ww w.whishworks.com
> > 
> >
> >   
> > 
> > 
> >
> > The contents of this e-mail are confidential and for the exclusive use of
> > the intended recipient. If you receive this e-mail in error please delete
> > it from your system immediately and notify us either by e-mail or
> > telephone. You should not copy, forward or otherwise disclose the content
> > of the e-mail. The views expressed in this communication may not
> > necessarily be the view held by WHISHWORKS.
>

Re: Facing problem from ExecuteSQL processor

2015-09-04 Thread Bryan Bende 
Hi Shubham,

I believe images usually get stripped out of the emails, can you upload the
image somewhere and provide a link?

Generally a processor will transfer data on a relationship, and a
relationship is connected to another processor, or the relationship can be
auto-terminated which means the data has reached a point where you don't
want to do anything else with it.
For testing, you may want to connect your ExecuteSQL processor to a PutFile
processor that writes to a directory on your local machine, this way you
can see what is getting produced. Once you know it is working then you can
reconfigure the flow how you like.

Thanks,

Bryan

On Fri, Sep 4, 2015 at 4:33 AM, Shubham Bansal <
shubham.ban...@whishworks.com> wrote:

> Hi,
>
> I am using ExecuteSQL processor in my flow. Infact , that is the only
> processor I have added to the flow. I have added DBCPConnectionPool service
> to this processor. I am attaching screenshots for the configuration done
> for these. There is no error on the connection side but I am not able to
> fetch any data. Where exactly will this data go ? Please help me with this.
>
> Thanks,
>
> --
>
>
>
> *Shubham Bansal*, Software Engineer,
>
> 4th Floor, Pioneer Towers, Madhapur, Hyderabad
> *M: **+91  9059328793 <%2B91%20%C2%A09059328793> *
> *W: *ww w.whishworks.com
> 
>
>   
> 
> 
>
> The contents of this e-mail are confidential and for the exclusive use of
> the intended recipient. If you receive this e-mail in error please delete
> it from your system immediately and notify us either by e-mail or
> telephone. You should not copy, forward or otherwise disclose the content
> of the e-mail. The views expressed in this communication may not
> necessarily be the view held by WHISHWORKS.

Facing problem from ExecuteSQL processor

2015-09-04 Thread Shubham Bansal 
Hi,

I am using ExecuteSQL processor in my flow. Infact , that is the only
processor I have added to the flow. I have added DBCPConnectionPool service
to this processor. I am attaching screenshots for the configuration done
for these. There is no error on the connection side but I am not able to
fetch any data. Where exactly will this data go ? Please help me with this.

Thanks,

-- 



*Shubham Bansal*, Software Engineer,

4th Floor, Pioneer Towers, Madhapur, Hyderabad
*M: **+91  9059328793 *
*W: *ww w.whishworks.com


  



-- 
The contents of this e-mail are confidential and for the exclusive use of 
the intended recipient. If you receive this e-mail in error please delete 
it from your system immediately and notify us either by e-mail or 
telephone. You should not copy, forward or otherwise disclose the content 
of the e-mail. The views expressed in this communication may not 
necessarily be the view held by WHISHWORKS.

Re: Resetting counters whilst clustered disconnects nodes

2015-09-04 Thread Matt Gilman 
Tommy,

Thanks for the great write up! I've replicated the issue of the node 
disconnecting using the steps you've provided. I've created a JIRA for the 
issue [1]. For the other concern, that is how it's currently designed to work. 
The 'run on primary node only' applies when a node is part of a cluster. If a 
node is disconnected from a cluster and a processor is configured with that 
scheduling strategy the processor will run as though it's timer driven.

We should have the counters issue addressed for the upcoming 0.3.0 release.

Thanks!

Matt

[1] https://issues.apache.org/jira/browse/NIFI-926

> On Thu, Sep 3, 2015 at 5:14 AM, tommy.yard...@baesystems.com 
>  wrote:
> Hi,
> 
> I have a three machine setup (1 NCM + 2 Nodes) running 0.2.0-incubating and 
> observed the following:
> 
> 
> 1.   Resetting counters can result in the MCN disconnecting a node
> 
> 2.   The node that is disconnected begins processing FlowFiles
> 
> Description:
> 
> My clustered NiFi is running a single pipeline containing 3 processors. While 
> the pipeline is running, resetting counters will result in any nodes which 
> are not processing anything (i.e. are not contributing to the count) to 
> disconnect. The node can then be reconnected via the UI. Looking at the stats 
> it appears the pipeline then began running on the disconnected node, as well 
> as the single remaining connected node. This has been tested using custom 
> processors as well as standard processors.
> 
> Steps to Replicate:
> 
> 
> 1.   Create cluster with 2 nodes + 1 MCN (2 nodes for processing are 
> needed or the problem won't appear)
> 
> 2.   Add GenerateFlowFile processor:
> 
> a.   Scheduling: Change Scheduling strategy to 'On primary node'
> 
> b.  Properties: Change File Size to '10B' (say)
> 
> 3.   Add HashAttribute processor:
> 
> a.   Properties: Change Key to 'hash.value'
> 
> 4.   Add DetectDuplicate processor:
> 
> a.   Properties: Under Distributed Cache Service add a 
> 'DistributedMapCacheClientService'
> 
>i.  For 
> the Client Service Add Server name to 'localhost' under properties
> 
>  ii.  Enable 
> The Client Service
> 
> iii.  Add a 
> DistrubtedMapCacheServer under the Controller Services
> 
>iv.  Enable 
> the Cache Server
> 
>  v.  Exit 
> NiFi Flow Settings
> 
> 5.   Connect all 3 processors on success
> 
> 6.   Auto-terminate all options for DetectDuplicate
> 
> 7.   Run all processors and wait for ~10seconds or so
> 
> 8.   Open counters tab and refresh to make sure counters > 0
> 
> 9.   Reset one of the counters
> 
> Note: I'm specifically using the DetectDuplicate processor in this example 
> because it contains a custom counter.
> 
> This should then disconnect the node that was not active (node that was not 
> selected to be the primary). Even though the GenerateFlowFile processor is 
> scheduled to run on the primary node the disconnected node begins to emit 
> FlowFiles.
> 
> The following Warning was pulled from the MCNs logs:
> 
> 2015-09-02 10:40:16,750 WARN [NiFi Web Server-149] 
> o.a.n.c.manager.impl.WebClusterManager One or more nodes failed to process 
> URI 
> 'http://localhost:8082/nifi-api/controller/counters/2207ea22-0d4a-389d-b746-82e568c6228d'.
>   Requesting each node to disconnect from cluster.
> 
> I'm interested in knowing if this is expected behaviour or if I should open a 
> JIRA ticket (2 perhaps).
> 
> Thanks,
> Tommy
> Please consider the environment before printing this email. This message 
> should be regarded as confidential. If you have received this email in error 
> please notify the sender and destroy it immediately. Statements of intent 
> shall only become binding when confirmed in hard copy by an authorised 
> signatory. The contents of this email may relate to dealings with other 
> companies under the control of BAE Systems Applied Intelligence Limited, 
> details of which can be found at 
> http://www.baesystems.com/Businesses/index.htm.