RE: Overwriting attribute in GetMongo

2018-09-28 Thread Rajesh Biswas 
Hello Mike,
We fixed the issue by altering the code suggested by Mark in the JIRA

Added below line in GetMongo.java file's 301 line

flowFile = (input == null) ? session.create() : session.create(input);

After that we checked the functionality and we are able to union the attributes 
from other processors with GetMongo processor.

Please let us know if the solution is correct.

Thanks and Regards,
Rajesh Biswas | +91 9886433461 | www.bridgera.com


-Original Message-
From: Mike Thomsen [mailto:mikerthom...@gmail.com]
Sent: Friday, September 28, 2018 3:38 AM
To: dev@nifi.apache.org
Subject: Re: Overwriting attribute in GetMongo

Looks like it was already done in NIFI-5334. That is part of the current
work in progress and will be available whenever 1.8 is released.

Thanks,

Mike

On Thu, Sep 27, 2018 at 10:49 AM Rajesh Biswas 
wrote:

> Hello Mike,
> So as of now it cannot be done?
> Is there any other way to achieve the same result.
> If you will have JIRA ticket, when we can expect the fix?
>
> Thanks and Regards,
> Rajesh Biswas | +91 9886433461 | www.bridgera.com
>
>
> -Original Message-
> From: Mike Thomsen [mailto:mikerthom...@gmail.com]
> Sent: Thursday, September 27, 2018 8:17 PM
> To: dev@nifi.apache.org
> Subject: Re: Overwriting attribute in GetMongo
>
> Ok. So it's actually ListenHTTP -> GetMongo, not straight to PutMongo.
> That'll require a Jira ticket. I'll write one up. I'll do it as an optional
> parameter.
>
> On Thu, Sep 27, 2018 at 10:43 AM Rajesh Biswas  >
> wrote:
>
> > Hello Mike,
> > Sorry to miss it again, let me attach a Doc file
> >
> >
> > Thanks and Regards,
> > Rajesh Biswas | +91 9886433461 | www.bridgera.com
> >
> >
> > -Original Message-
> > From: Mike Thomsen [mailto:mikerthom...@gmail.com]
> > Sent: Thursday, September 27, 2018 8:09 PM
> > To: dev@nifi.apache.org
> > Subject: Re: Overwriting attribute in GetMongo
> >
> > Didn't make it that time either. Please just describe the relationship.
> >
> > Thanks,
> >
> > Mike
> >
> > On Thu, Sep 27, 2018 at 10:36 AM Rajesh Biswas <
> rajesh.bis...@bridgera.com
> > >
> > wrote:
> >
> > > Hello Shiva,
> > >
> > > Please find 3 images attached
> > >
> > > 1. NifI Flow
> > > 2.Configuration for ListenHTTP Processor
> > > 3.Configuration for PutMongo Processor
> > >
> > >
> > > Thanks and Regards,
> > > Rajesh Biswas | +91 9886433461 | www.bridgera.com
> > >
> > >
> > > -Original Message-
> > > From: Sivaprasanna [mailto:sivaprasanna...@gmail.com]
> > > Sent: Thursday, September 27, 2018 7:46 PM
> > > To: dev@nifi.apache.org
> > > Subject: Re: Overwriting attribute in GetMongo
> > >
> > > Rajesh, I don’t think the images were uploaded properly. If possible,
> > > please export the flow as a template and attach it here.
> > >
> > > -
> > > Sivaprasanna
> > >
> > > On Thu, 27 Sep 2018 at 7:31 PM, Rajesh Biswas <
> > rajesh.bis...@bridgera.com>
> > > wrote:
> > >
> > > > Please find the flow,
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Below is the configuration for ListenHTTP processor
> > > >
> > > >
> > > >
> > > > Below is the configuration for PutMongo processor
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Thanks and Regards,
> > > >
> > > > Rajesh Biswas | +91 9886433461 | www.bridgera.com
> > > >
> > > >
> > > >
> > > > -Original Message-
> > > > From: Mike Thomsen [mailto:mikerthom...@gmail.com]
> > > > Sent: Thursday, September 27, 2018 7:08 PM
> > > > To: dev@nifi.apache.org
> > > > Subject: Re: Overwriting attribute in GetMongo
> > > >
> > > >
> > > >
> > > > Where does GetMongo sit relative to the connected ListenHTTP and
> > PutMongo
> > > >
> > > > processors? That's the critical part that's missing here in helping
> > you.
> > > >
> > > >
> > > >
> > > > Thanks,
> > > >
> > > >
> > > >
> > > > Mike
> > > >
> > > >
> > > >
> > > > On Thu, Sep 27, 2018 at 9:20 AM Rajesh Biswas <
> > > rajesh.bis...@bridgera.com>
> > > >
> > > > wrote:
> > > >
> > > >
> > > >
> > > > > Hello NiFi Team,
> > > >
> > > > >
> > > >
> > > > > We are using ListenHTTP processor to read HTTP input from client
> and
> > we
> > > >
> > > > > have
> > > >
> > > > > PutMongo processor after that.
> > > >
> > > > >
> > > >
> > > > > We have seen the attributes we are getting from ListenHTTP is
> > > overwritten
> > > >
> > > > > when we call GetMongo collection.
> > > >
> > > > >
> > > >
> > > > > Would you please suggest how to preserve existing request
> attributes
> > > and
> > > >
> > > > > add
> > > >
> > > > > the document attributes feteched thru GetMongo processor.
> > > >
> > > > >
> > > >
> > > > >
> > > >
> > > > >
> > > >
> > > > > Thanks and Regards,
> > > >
> > > > >
> > > >
> > > > > Rajesh Biswas | +91 9886433461 |  
> > > >
> > > > > www.bridgera.com
> > > >
> > > > >
> > > >
> > > > >
> > > >
> > > > >
> > > >
> > > > >
> > > >
> > > > >
> > > >
> > > > > ---
> > > >
> > > > > This email has been checked for viruses by Avast antivirus
> software.
> > > >
> > > >

Re: Overwriting attribute in GetMongo

2018-09-28 Thread Mike Thomsen 
That should be enough for this particular case.

On Fri, Sep 28, 2018 at 6:08 AM Rajesh Biswas 
wrote:

> Hello Mike,
> We fixed the issue by altering the code suggested by Mark in the JIRA
>
> Added below line in GetMongo.java file's 301 line
>
> flowFile = (input == null) ? session.create() : session.create(input);
>
> After that we checked the functionality and we are able to union the
> attributes from other processors with GetMongo processor.
>
> Please let us know if the solution is correct.
>
> Thanks and Regards,
> Rajesh Biswas | +91 9886433461 | www.bridgera.com
>
>
> -Original Message-
> From: Mike Thomsen [mailto:mikerthom...@gmail.com]
> Sent: Friday, September 28, 2018 3:38 AM
> To: dev@nifi.apache.org
> Subject: Re: Overwriting attribute in GetMongo
>
> Looks like it was already done in NIFI-5334. That is part of the current
> work in progress and will be available whenever 1.8 is released.
>
> Thanks,
>
> Mike
>
> On Thu, Sep 27, 2018 at 10:49 AM Rajesh Biswas  >
> wrote:
>
> > Hello Mike,
> > So as of now it cannot be done?
> > Is there any other way to achieve the same result.
> > If you will have JIRA ticket, when we can expect the fix?
> >
> > Thanks and Regards,
> > Rajesh Biswas | +91 9886433461 | www.bridgera.com
> >
> >
> > -Original Message-
> > From: Mike Thomsen [mailto:mikerthom...@gmail.com]
> > Sent: Thursday, September 27, 2018 8:17 PM
> > To: dev@nifi.apache.org
> > Subject: Re: Overwriting attribute in GetMongo
> >
> > Ok. So it's actually ListenHTTP -> GetMongo, not straight to PutMongo.
> > That'll require a Jira ticket. I'll write one up. I'll do it as an
> optional
> > parameter.
> >
> > On Thu, Sep 27, 2018 at 10:43 AM Rajesh Biswas <
> rajesh.bis...@bridgera.com
> > >
> > wrote:
> >
> > > Hello Mike,
> > > Sorry to miss it again, let me attach a Doc file
> > >
> > >
> > > Thanks and Regards,
> > > Rajesh Biswas | +91 9886433461 | www.bridgera.com
> > >
> > >
> > > -Original Message-
> > > From: Mike Thomsen [mailto:mikerthom...@gmail.com]
> > > Sent: Thursday, September 27, 2018 8:09 PM
> > > To: dev@nifi.apache.org
> > > Subject: Re: Overwriting attribute in GetMongo
> > >
> > > Didn't make it that time either. Please just describe the relationship.
> > >
> > > Thanks,
> > >
> > > Mike
> > >
> > > On Thu, Sep 27, 2018 at 10:36 AM Rajesh Biswas <
> > rajesh.bis...@bridgera.com
> > > >
> > > wrote:
> > >
> > > > Hello Shiva,
> > > >
> > > > Please find 3 images attached
> > > >
> > > > 1. NifI Flow
> > > > 2.Configuration for ListenHTTP Processor
> > > > 3.Configuration for PutMongo Processor
> > > >
> > > >
> > > > Thanks and Regards,
> > > > Rajesh Biswas | +91 9886433461 | www.bridgera.com
> > > >
> > > >
> > > > -Original Message-
> > > > From: Sivaprasanna [mailto:sivaprasanna...@gmail.com]
> > > > Sent: Thursday, September 27, 2018 7:46 PM
> > > > To: dev@nifi.apache.org
> > > > Subject: Re: Overwriting attribute in GetMongo
> > > >
> > > > Rajesh, I don’t think the images were uploaded properly. If possible,
> > > > please export the flow as a template and attach it here.
> > > >
> > > > -
> > > > Sivaprasanna
> > > >
> > > > On Thu, 27 Sep 2018 at 7:31 PM, Rajesh Biswas <
> > > rajesh.bis...@bridgera.com>
> > > > wrote:
> > > >
> > > > > Please find the flow,
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Below is the configuration for ListenHTTP processor
> > > > >
> > > > >
> > > > >
> > > > > Below is the configuration for PutMongo processor
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Thanks and Regards,
> > > > >
> > > > > Rajesh Biswas | +91 9886433461 | www.bridgera.com
> > > > >
> > > > >
> > > > >
> > > > > -Original Message-
> > > > > From: Mike Thomsen [mailto:mikerthom...@gmail.com]
> > > > > Sent: Thursday, September 27, 2018 7:08 PM
> > > > > To: dev@nifi.apache.org
> > > > > Subject: Re: Overwriting attribute in GetMongo
> > > > >
> > > > >
> > > > >
> > > > > Where does GetMongo sit relative to the connected ListenHTTP and
> > > PutMongo
> > > > >
> > > > > processors? That's the critical part that's missing here in helping
> > > you.
> > > > >
> > > > >
> > > > >
> > > > > Thanks,
> > > > >
> > > > >
> > > > >
> > > > > Mike
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Sep 27, 2018 at 9:20 AM Rajesh Biswas <
> > > > rajesh.bis...@bridgera.com>
> > > > >
> > > > > wrote:
> > > > >
> > > > >
> > > > >
> > > > > > Hello NiFi Team,
> > > > >
> > > > > >
> > > > >
> > > > > > We are using ListenHTTP processor to read HTTP input from client
> > and
> > > we
> > > > >
> > > > > > have
> > > > >
> > > > > > PutMongo processor after that.
> > > > >
> > > > > >
> > > > >
> > > > > > We have seen the attributes we are getting from ListenHTTP is
> > > > overwritten
> > > > >
> > > > > > when we call GetMongo collection.
> > > > >
> > > > > >
> > > > >
> > > > > > Would you please suggest how to preserve existing request
> > attributes
> > > > and
> > > > >
> > > > > > add
> > > > >
> >

Re: POJO to Record

2018-09-28 Thread Matt Burgess 
Moving users list to BCC and adding devs list, as this question is
better suited for the devs.

Ed,

Does XMLReader not parse your XML? I guess you'd need to know the
schema so if you don't (especially if it is dynamic) then you might be
faced with a custom processor as you described.

Take a look at DataTypeUtils, it has methods toRecord() and
convertRecordFieldtoObject() which helps you convert from POJO
<->Record. The JoltTransformRecord processor uses these, so you can
have a look at that code for an example.

Regards,
Matt

On Fri, Sep 28, 2018 at 11:28 AM Ed J  wrote:
>
> Hi,
> I'm ingesting some XML with an overly complicated structure and I've build a 
> custom java DOM parser to wrangle it into a saner form. (Is that frowned upon 
> - should I just use the built-in processors to wrangle it?)
>
> So my question is, I've parsed the XML into a simple POJO, how do I get that 
> pojo into the next processor as a 'Record'.  My custom nifi processor's 
> onTrigger looks like this:
>
> MyParser parser = new MyParser();
> MyPojo pojo = parser.parse(flowFileContents);
>
> // TODO: convert pojo to record
> Record myrecord = pojo.toRecord(); // how to do this?
> session.transfer(myrecord, REL_SUCCESS)
>
> I'm guessing I could convert the pojo to JSON, write that out to the next 
> stage, and then use a JsonRecordReader to convert the JSON into a record.  
> However, if I can go straight from pojo to record, that seems more efficient 
> - just not sure how to go about it.  Thanks.
>