Re: Reading the incoming flowfile "twice"

2020-03-31 Thread Mark Payne
Sorry, typo. ‘modified’ should have been ‘output’. > On Mar 31, 2020, at 2:44 PM, Russell Bateman wrote: > > (Oh, I see where *out*comes from, but not *modified*.) > > On 3/31/20 12:35 PM, Russell Bateman wrote: >> Wait, where is *modified*from? >> >> Thanks >> >> On 3/31/20 12:24 PM, Mark

Re: Reading the incoming flowfile "twice"

2020-03-31 Thread Russell Bateman
Let's see... Does this fix the typos the way you intended? public void onTrigger( final ProcessContext context, final ProcessSession session ) throws ProcessException {   FlowFile original = session.get(); if( original == null ) { context.yield(); return; }   FlowFile output   =

Re: Reading the incoming flowfile "twice"

2020-03-31 Thread Russell Bateman
(Oh, I see where *out*comes from, but not *modified*.) On 3/31/20 12:35 PM, Russell Bateman wrote: Wait, where is *modified*from? Thanks On 3/31/20 12:24 PM, Mark Payne wrote: Russ, OK, so then I think the pattern you’d want to follow would be something like this: FlowFile original =

Re: Reading the incoming flowfile "twice"

2020-03-31 Thread Russell Bateman
And, also *out*? On 3/31/20 12:35 PM, Russell Bateman wrote: Wait, where is *modified*from? Thanks On 3/31/20 12:24 PM, Mark Payne wrote: Russ, OK, so then I think the pattern you’d want to follow would be something like this: FlowFile original = session.get(); if (flowFile == null) {

Re: Reading the incoming flowfile "twice"

2020-03-31 Thread Russell Bateman
Wait, where is *modified*from? Thanks On 3/31/20 12:24 PM, Mark Payne wrote: Russ, OK, so then I think the pattern you’d want to follow would be something like this: FlowFile original = session.get(); if (flowFile == null) { return; } FlowFile output = session.create(original); //

Re: Reading the incoming flowfile "twice"

2020-03-31 Thread Mark Payne
Russ, OK, so then I think the pattern you’d want to follow would be something like this: FlowFile original = session.get(); if (flowFile == null) { return; } FlowFile output = session.create(original); // Begin writing to ‘output flowfile' output = session.write(modified, new

Re: Reading the incoming flowfile "twice"

2020-03-31 Thread Russell Bateman
Mark, Thanks for getting back. My steps are: 1. Read the "first half" of the input stream copying it to the output stream. This is because I need to preserve the exact form of it (spacing, indentation, lines, etc.) without change whatsoever. If I 2. Reopen the stream from the beginning with

Re: Reading the incoming flowfile "twice"

2020-03-31 Thread Otto Fowler
Oh, sorry I did not understand that you needed to do that On March 31, 2020 at 11:22:20, Russell Bateman (r...@windofkeltia.com) wrote: Yes, I though of that, but there's no way to insert completing XML structure into the input stream ahead of (). SAX will choke if I just start feeding it the

Re: Reading the incoming flowfile "twice"

2020-03-31 Thread Mark Payne
Russ, As far as I can tell, this is working exactly as expected. To verify, I created a simple Integration test, as well, which I attached below. Let me outline what I *think* you’re trying to do here and please correct me if I’m wrong: 1. Read the content of the FlowFile. (Via session.read)

Re: Reading the incoming flowfile "twice"

2020-03-31 Thread Russell Bateman
Yes, I though of that, but there's no way to insert completing XML structure into the input stream ahead of (). SAX will choke if I just start feeding it the flowfile where I left off from copying up to . On 3/30/20 8:25 PM, Otto Fowler wrote: Can I ask why you would consume the whole stream

Re: Reading the incoming flowfile "twice"

2020-03-30 Thread Otto Fowler
Can I ask why you would consume the whole stream when doing the non-sax part? If you consume the stream right up to the sax part ( the stream POS is at the start of the xml ) then you can just pass the stream to sax as is can’t you? On March 30, 2020 at 16:23:27, Russell Bateman

Re: Reading the incoming flowfile "twice"

2020-03-30 Thread Russell Bateman
If I haven't worn out my welcome, here is the simplified code that should demonstrate either that I have miscoded your suggestions or that the API doesn't in fact work as advertised. First, the output. The code, both JUnit test and processor are attached and the files are pretty small. Much

Re: Reading the incoming flowfile "twice"

2020-03-29 Thread Russell Bateman
No, this test file is tiny. The real thing is usually megabytes in size. On Sun, Mar 29, 2020 at 3:15 AM Mike Thomsen wrote: > If these files are only a few MB at the most, you can also just export them > to a ByteArrayOutputStream. Just a thought. > > On Sun, Mar 29, 2020 at 12:16 AM Russell

Re: Reading the incoming flowfile "twice"

2020-03-29 Thread Joe Witt
Russell I recommend writing very simple code that does two successive read/write operations on basic data so you can make sure the api work/as expected. Then add the xml bits. Thanks On Sun, Mar 29, 2020 at 5:15 AM Mike Thomsen wrote: > If these files are only a few MB at the most, you can

Re: Reading the incoming flowfile "twice"

2020-03-29 Thread Mike Thomsen
If these files are only a few MB at the most, you can also just export them to a ByteArrayOutputStream. Just a thought. On Sun, Mar 29, 2020 at 12:16 AM Russell Bateman wrote: > Joe and Mike, > > Sadly, I was not able to get very far on this. It seems that the extend > to which I copy the first

Re: Reading the incoming flowfile "twice"

2020-03-28 Thread Russell Bateman
Joe and Mike, Sadly, I was not able to get very far on this. It seems that the extend to which I copy the first half of the contents of the input stream, I lose what comes after when I try to read again, basically, the second half comprising the and elements which I was hoping to SAX-parse.

Re: Reading the incoming flowfile "twice"

2020-03-28 Thread Mike Thomsen
I prefer to avoid the callbacks and do this: InputStream is = session.read(flowFile); some code is.close(); On Fri, Mar 27, 2020 at 5:22 PM Russell Bateman wrote: > Joe, > > Ah, thanks. I think I have learned a lot about what's going on down > inside session.read/write()today. I don't

Re: Reading the incoming flowfile "twice"

2020-03-27 Thread Russell Bateman
Joe, Ah, thanks. I think I have learned a lot about what's going on down inside session.read/write()today. I don't have to stand on my head. For completeness if anyone else looks for this answer, here's my code amended: public void onTrigger( final ProcessContext context, final

Re: Reading the incoming flowfile "twice"

2020-03-27 Thread Joe Witt
you should be able to call write as many times as you need. just keep using the resulting flowfile reference into the next call. On Fri, Mar 27, 2020 at 5:06 PM Russell Bateman wrote: > Mike, > > Many thanks for responding. Do you mean to say that all I have to do is > something like this? > >

Re: Reading the incoming flowfile "twice"

2020-03-27 Thread Russell Bateman
Mike, Many thanks for responding. Do you mean to say that all I have to do is something like this? public void onTrigger( final ProcessContext context, final ProcessSession session ) throws ProcessException {   FlowFile flowfile = session.get();   ...   // this is will

Re: Reading the incoming flowfile "twice"

2020-03-27 Thread Mike Thomsen
session.read(FlowFile) just gives you an InputStream. You should be able to rerun that as many times as you want provided you properly close it. On Fri, Mar 27, 2020 at 11:25 AM Russell Bateman wrote: > In my custom processor, I'm using a SAX parser to process an incoming > flowfile that's in

Reading the incoming flowfile "twice"

2020-03-27 Thread Russell Bateman
In my custom processor, I'm using a SAX parser to process an incoming flowfile that's in XML. Except that, this particular XML is in essence two different files and I would like to split, read and process the first "half", which starts a couple of lines (XML elements) into the file) not using