[jira] [Created] (NIFI-6829) RethinkDB processors require Username and Password

[Nicholas Carenza (Jira)]

Nicholas Carenza created NIFI-6829:
--

 Summary: RethinkDB processors require Username and Password
 Key: NIFI-6829
 URL: https://issues.apache.org/jira/browse/NIFI-6829
 Project: Apache NiFi
  Issue Type: Bug
  Components: Extensions
Affects Versions: 1.9.2
Reporter: Nicholas Carenza


The RethinkDB suite of processor all require username and password. They are 
marked as optional but if you don't provide them then the processor fails to 
initialize.

It's probably because in the AbstractRethinkDBProcessor the `makeConnection` 
method doesn't have a case for making a connection in the absence of username 
and password.

When you run the processor in this case you get an unhelpful RuntimeError with 
no message and a NullPointerException .

[https://github.com/apache/nifi/blob/ea9b0db2f620526c8dd0db595cf8b44c3ef835be/nifi-nar-bundles/nifi-rethinkdb-bundle/nifi-rethinkdb-processors/src/main/java/org/apache/nifi/processors/rethinkdb/AbstractRethinkDBProcessor.java#L199-L203]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (NIFI-3229) When a queue contains only Penalized FlowFile's the next processor Tasks/Time statistics becomes extremely large

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708944#comment-16708944
 ] 

Nicholas Carenza commented on NIFI-3229:


What's next for this?

> When a queue contains only Penalized FlowFile's the next processor Tasks/Time 
> statistics becomes extremely large
> 
>
> Key: NIFI-3229
> URL: https://issues.apache.org/jira/browse/NIFI-3229
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Dmitry Lukyanov
>Assignee: Peter Wicks
>Priority: Minor
> Attachments: flow.xml.gz, nifi-stats.png, nifi-stats2.png
>
>
> fetchfile on `not.found` produces penalized flow file
> in this case i'm expecting the next processor will do one task execution when 
> flow file penalize time over.
> but according to stats it executes approximately 1-6 times.
> i understand that it could be a feature but stats became really unclear...
> maybe there should be two columns? 
> `All Task/Times` and `Committed Task/Times`



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (NIFI-3463) ExecuteSQL Errors on first use after long period of unuse

[Nicholas Carenza (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Carenza resolved NIFI-3463.

Resolution: Not A Bug

> ExecuteSQL Errors on first use after long period of unuse
> -
>
> Key: NIFI-3463
> URL: https://issues.apache.org/jira/browse/NIFI-3463
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Nicholas Carenza
>Priority: Trivial
>  Labels: ExecuteSQL
>
> I have a flow that runs ~ once a month and it gets this error for the first 
> query each time it runs.
> ExecuteSQL[id=61f3179d-34f3-1986-ee27-134f425dfa18] Unable to execute SQL 
> select query SELECT blah blah blah,claim=StandardContentClaim 
> [resourceClaim=StandardResourceClaim[id=1486767996789-3396773, 
> container=default, section=165], offset=619840, 
> length=89131409],offset=57849490,name=filename,size=1566] due to 
> org.apache.nifi.processor.exception.ProcessException: 
> com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet 
> successfully received from the server was 1,401,254,731 milliseconds ago.  
> The last packet sent successfully to the server was 1,401,254,731 
> milliseconds ago. is longer than the server configured value of 
> 'wait_timeout'. You should consider either expiring and/or testing connection 
> validity before use in your application, increasing the server configured 
> values for client timeouts, or using the Connector/J connection property 
> 'autoReconnect=true' to avoid this problem.; routing to failure: 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3463) ExecuteSQL Errors on first use after long period of unuse

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519609#comment-16519609
 ] 

Nicholas Carenza commented on NIFI-3463:


I don't recall. I know I use that field now but this was so long ago. I 
probably wasn't using it then.

> ExecuteSQL Errors on first use after long period of unuse
> -
>
> Key: NIFI-3463
> URL: https://issues.apache.org/jira/browse/NIFI-3463
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Nicholas Carenza
>Priority: Trivial
>  Labels: ExecuteSQL
>
> I have a flow that runs ~ once a month and it gets this error for the first 
> query each time it runs.
> ExecuteSQL[id=61f3179d-34f3-1986-ee27-134f425dfa18] Unable to execute SQL 
> select query SELECT blah blah blah,claim=StandardContentClaim 
> [resourceClaim=StandardResourceClaim[id=1486767996789-3396773, 
> container=default, section=165], offset=619840, 
> length=89131409],offset=57849490,name=filename,size=1566] due to 
> org.apache.nifi.processor.exception.ProcessException: 
> com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet 
> successfully received from the server was 1,401,254,731 milliseconds ago.  
> The last packet sent successfully to the server was 1,401,254,731 
> milliseconds ago. is longer than the server configured value of 
> 'wait_timeout'. You should consider either expiring and/or testing connection 
> validity before use in your application, increasing the server configured 
> values for client timeouts, or using the Connector/J connection property 
> 'autoReconnect=true' to avoid this problem.; routing to failure: 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (NIFI-5185) Convenient "Run Once" button for scheduled processors

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-5185:
--

 Summary: Convenient "Run Once" button for scheduled processors
 Key: NIFI-5185
 URL: https://issues.apache.org/jira/browse/NIFI-5185
 Project: Apache NiFi
  Issue Type: Improvement
Reporter: Nicholas Carenza


I have a stateful cron processor that runs once a day. Sometimes though I would 
like to trigger it to run on-demand. I could change it to use different 
scheduling temporarily. I would just like a more convenient way to do this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3753) ListenBeats: Compressed beats packets may cause: Error decoding Beats frame: Error decompressing frame: invalid distance too far back

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469614#comment-16469614
 ] 

Nicholas Carenza commented on NIFI-3753:


Thanks John, Unfortunately when i set max_bulk_size to 0, nothing happens at 
all. If i set it to 1, lines get sent albeit very slowly and i/o timeouts occur 
constantly. Filebeat v5.6.6 and Nifi v1.3

> ListenBeats: Compressed beats packets may cause: Error decoding Beats  frame: 
> Error decompressing  frame: invalid distance too far back
> ---
>
> Key: NIFI-3753
> URL: https://issues.apache.org/jira/browse/NIFI-3753
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Andre F de Miranda
>Priority: Critical
>
> 017-04-28 02:03:37,153 ERROR [pool-106-thread-1] 
> o.a.nifi.processors.beats.List
> enBeats
> org.apache.nifi.processors.beats.frame.BeatsFrameException: Error decoding 
> Beats
>  frame: Error decompressing  frame: invalid distance too far back
> at 
> org.apache.nifi.processors.beats.frame.BeatsDecoder.process(BeatsDeco
> der.java:123) ~[nifi-beats-processors-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.nifi.processors.beats.handler.BeatsSocketChannelHandler.pr
> ocessBuffer(BeatsSocketChannelHandler.java:71) 
> ~[nifi-beats-processors-1.2.0-SNA
> PSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.nifi.processor.util.listen.handler.socket.StandardSocketCh
> annelHandler.run(StandardSocketChannelHandler.java:76) 
> [nifi-processor-utils-1.2
> .0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
> java:1142) [na:1.8.0_131]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_131]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
> Caused by: org.apache.nifi.processors.beats.frame.BeatsFrameException: Error 
> decompressing  frame: invalid distance too far back
> at 
> org.apache.nifi.processors.beats.frame.BeatsDecoder.processPAYLOAD(BeatsDecoder.java:292)
>  ~[nifi-beats-processors-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.nifi.processors.beats.frame.BeatsDecoder.process(BeatsDecoder.java:103)
>  ~[nifi-beats-processors-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> ... 5 common frames omitted
> Caused by: java.util.zip.ZipException: invalid distance too far back
> at 
> java.util.zip.InflaterInputStream.read(InflaterInputStream.java:164) 
> ~[na:1.8.0_131]
> at java.io.FilterInputStream.read(FilterInputStream.java:107) 
> ~[na:1.8.0_131]
> at 
> org.apache.nifi.processors.beats.frame.BeatsDecoder.processPAYLOAD(BeatsDecoder.java:277)
>  ~[nifi-beats-processors-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> ... 6 common frames omitted



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5126) QueryDatabaseTable state key leads to unexpected behavior when table name changes

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458997#comment-16458997
 ] 

Nicholas Carenza commented on NIFI-5126:


Yea so if there had been a way for me to pre-populate the state that would have 
also worked.

> QueryDatabaseTable state key leads to unexpected behavior when table name 
> changes
> -
>
> Key: NIFI-5126
> URL: https://issues.apache.org/jira/browse/NIFI-5126
> Project: Apache NiFi
>  Issue Type: Bug
> Environment: Nifi 1.3.0
>Reporter: Nicholas Carenza
>Priority: Major
> Attachments: image-2018-04-26-10-36-45-879.png
>
>
> I renamed a table in my database and updated my QueryDatabaseTable to match 
> thinking it would resume from where it left off but the state variable name 
> changed with along with the table name property.
> !image-2018-04-26-10-36-45-879.png!
> So it ended up querying the full table all over again. Can we add some 
> configuration to control this behavior or remove the table name from the 
> state variable by default? Since the processor can only query from one table 
> anyways, I am not sure why the table name needs to be saved to state.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-5126) QueryDatabaseTable state key leads to unexpected behavior when table name changes

[Nicholas Carenza (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Carenza updated NIFI-5126:
---
Description: 
I renamed a table in my database and updated my QueryDatabaseTable to match 
thinking it would resume from where it left off but the state variable name 
changed with along with the table name property.

!image-2018-04-26-10-36-45-879.png!

So it ended up querying the full table all over again. Can we add some 
configuration to control this behavior or remove the table name from the state 
variable by default? Since the processor can only query from one table anyways, 
I am not sure why the table name needs to be saved to state.

  was:
I renamed a table in my database and updated my QueryDatabaseTable to match 
thinking it would resume from where it left off but the state variable name 
changed with along with the table name property.

!image-2018-04-26-10-32-55-718.png!

So it ended up querying the full table all over again. Can we add some 
configuration to control this behavior or remove the table name from the state 
variable by default? Since the processor can only query from one table anyways, 
I am not sure why the table name needs to be saved to state.


> QueryDatabaseTable state key leads to unexpected behavior when table name 
> changes
> -
>
> Key: NIFI-5126
> URL: https://issues.apache.org/jira/browse/NIFI-5126
> Project: Apache NiFi
>  Issue Type: Bug
> Environment: Nifi 1.3.0
>Reporter: Nicholas Carenza
>Priority: Major
> Attachments: image-2018-04-26-10-36-45-879.png
>
>
> I renamed a table in my database and updated my QueryDatabaseTable to match 
> thinking it would resume from where it left off but the state variable name 
> changed with along with the table name property.
> !image-2018-04-26-10-36-45-879.png!
> So it ended up querying the full table all over again. Can we add some 
> configuration to control this behavior or remove the table name from the 
> state variable by default? Since the processor can only query from one table 
> anyways, I am not sure why the table name needs to be saved to state.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-5126) QueryDatabaseTable state key leads to unexpected behavior when table name changes

[Nicholas Carenza (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Carenza updated NIFI-5126:
---
Attachment: image-2018-04-26-10-36-45-879.png

> QueryDatabaseTable state key leads to unexpected behavior when table name 
> changes
> -
>
> Key: NIFI-5126
> URL: https://issues.apache.org/jira/browse/NIFI-5126
> Project: Apache NiFi
>  Issue Type: Bug
> Environment: Nifi 1.3.0
>Reporter: Nicholas Carenza
>Priority: Major
> Attachments: image-2018-04-26-10-36-45-879.png
>
>
> I renamed a table in my database and updated my QueryDatabaseTable to match 
> thinking it would resume from where it left off but the state variable name 
> changed with along with the table name property.
> !image-2018-04-26-10-36-45-879.png!
> So it ended up querying the full table all over again. Can we add some 
> configuration to control this behavior or remove the table name from the 
> state variable by default? Since the processor can only query from one table 
> anyways, I am not sure why the table name needs to be saved to state.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (NIFI-5126) QueryDatabaseTable state key leads to unexpected behavior when table name changes

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-5126:
--

 Summary: QueryDatabaseTable state key leads to unexpected behavior 
when table name changes
 Key: NIFI-5126
 URL: https://issues.apache.org/jira/browse/NIFI-5126
 Project: Apache NiFi
  Issue Type: Bug
 Environment: Nifi 1.3.0
Reporter: Nicholas Carenza
 Attachments: image-2018-04-26-10-36-45-879.png

I renamed a table in my database and updated my QueryDatabaseTable to match 
thinking it would resume from where it left off but the state variable name 
changed with along with the table name property.

!image-2018-04-26-10-32-55-718.png!

So it ended up querying the full table all over again. Can we add some 
configuration to control this behavior or remove the table name from the state 
variable by default? Since the processor can only query from one table anyways, 
I am not sure why the table name needs to be saved to state.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3753) ListenBeats: Compressed beats packets may cause: Error decoding Beats frame: Error decompressing frame: invalid distance too far back

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430958#comment-16430958
 ] 

Nicholas Carenza commented on NIFI-3753:


[~jsmith1] did turning off compression and setting bulk max size to 0 in beats 
work for you and you are able to use ListenBeats now in Nifi in it's current 
state?

> ListenBeats: Compressed beats packets may cause: Error decoding Beats  frame: 
> Error decompressing  frame: invalid distance too far back
> ---
>
> Key: NIFI-3753
> URL: https://issues.apache.org/jira/browse/NIFI-3753
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Andre F de Miranda
>Priority: Critical
>
> 017-04-28 02:03:37,153 ERROR [pool-106-thread-1] 
> o.a.nifi.processors.beats.List
> enBeats
> org.apache.nifi.processors.beats.frame.BeatsFrameException: Error decoding 
> Beats
>  frame: Error decompressing  frame: invalid distance too far back
> at 
> org.apache.nifi.processors.beats.frame.BeatsDecoder.process(BeatsDeco
> der.java:123) ~[nifi-beats-processors-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.nifi.processors.beats.handler.BeatsSocketChannelHandler.pr
> ocessBuffer(BeatsSocketChannelHandler.java:71) 
> ~[nifi-beats-processors-1.2.0-SNA
> PSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.nifi.processor.util.listen.handler.socket.StandardSocketCh
> annelHandler.run(StandardSocketChannelHandler.java:76) 
> [nifi-processor-utils-1.2
> .0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
> java:1142) [na:1.8.0_131]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_131]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
> Caused by: org.apache.nifi.processors.beats.frame.BeatsFrameException: Error 
> decompressing  frame: invalid distance too far back
> at 
> org.apache.nifi.processors.beats.frame.BeatsDecoder.processPAYLOAD(BeatsDecoder.java:292)
>  ~[nifi-beats-processors-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.nifi.processors.beats.frame.BeatsDecoder.process(BeatsDecoder.java:103)
>  ~[nifi-beats-processors-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> ... 5 common frames omitted
> Caused by: java.util.zip.ZipException: invalid distance too far back
> at 
> java.util.zip.InflaterInputStream.read(InflaterInputStream.java:164) 
> ~[na:1.8.0_131]
> at java.io.FilterInputStream.read(FilterInputStream.java:107) 
> ~[na:1.8.0_131]
> at 
> org.apache.nifi.processors.beats.frame.BeatsDecoder.processPAYLOAD(BeatsDecoder.java:277)
>  ~[nifi-beats-processors-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> ... 6 common frames omitted



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3753) ListenBeats: Compressed beats packets may cause: Error decoding Beats frame: Error decompressing frame: invalid distance too far back

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137600#comment-16137600
 ] 

Nicholas Carenza commented on NIFI-3753:


I am getting this error as well. Any update on this issue?

> ListenBeats: Compressed beats packets may cause: Error decoding Beats  frame: 
> Error decompressing  frame: invalid distance too far back
> ---
>
> Key: NIFI-3753
> URL: https://issues.apache.org/jira/browse/NIFI-3753
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Andre F de Miranda
>Priority: Critical
>
> 017-04-28 02:03:37,153 ERROR [pool-106-thread-1] 
> o.a.nifi.processors.beats.List
> enBeats
> org.apache.nifi.processors.beats.frame.BeatsFrameException: Error decoding 
> Beats
>  frame: Error decompressing  frame: invalid distance too far back
> at 
> org.apache.nifi.processors.beats.frame.BeatsDecoder.process(BeatsDeco
> der.java:123) ~[nifi-beats-processors-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.nifi.processors.beats.handler.BeatsSocketChannelHandler.pr
> ocessBuffer(BeatsSocketChannelHandler.java:71) 
> ~[nifi-beats-processors-1.2.0-SNA
> PSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.nifi.processor.util.listen.handler.socket.StandardSocketCh
> annelHandler.run(StandardSocketChannelHandler.java:76) 
> [nifi-processor-utils-1.2
> .0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
> java:1142) [na:1.8.0_131]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_131]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
> Caused by: org.apache.nifi.processors.beats.frame.BeatsFrameException: Error 
> decompressing  frame: invalid distance too far back
> at 
> org.apache.nifi.processors.beats.frame.BeatsDecoder.processPAYLOAD(BeatsDecoder.java:292)
>  ~[nifi-beats-processors-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.nifi.processors.beats.frame.BeatsDecoder.process(BeatsDecoder.java:103)
>  ~[nifi-beats-processors-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> ... 5 common frames omitted
> Caused by: java.util.zip.ZipException: invalid distance too far back
> at 
> java.util.zip.InflaterInputStream.read(InflaterInputStream.java:164) 
> ~[na:1.8.0_131]
> at java.io.FilterInputStream.read(FilterInputStream.java:107) 
> ~[na:1.8.0_131]
> at 
> org.apache.nifi.processors.beats.frame.BeatsDecoder.processPAYLOAD(BeatsDecoder.java:277)
>  ~[nifi-beats-processors-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> ... 6 common frames omitted



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (NIFI-4291) See logs for individual processors and process groups

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-4291:
--

 Summary: See logs for individual processors and process groups
 Key: NIFI-4291
 URL: https://issues.apache.org/jira/browse/NIFI-4291
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core UI
Reporter: Nicholas Carenza
Priority: Minor


An handy option on a processor's context menu would be to show logs from that 
processor. It's already built in to the log viewer to filter by processor id.

Same goes for processor groups.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4101) Inter-process group portals

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16058069#comment-16058069
 ] 

Nicholas Carenza commented on NIFI-4101:


What if this were implemented as a purely visual feature using existing 
relationships but instead of there being a line connecting the processors, they 
each had just a stub and, going with [~alopresto]'s idea, clicking on the 
relationship would reveal the path. THis could be available in the context menu 
for the relationship.

> Inter-process group portals
> ---
>
> Key: NIFI-4101
> URL: https://issues.apache.org/jira/browse/NIFI-4101
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Nicholas Carenza
>Priority: Minor
>  Labels: ui
> Attachments: Drawing.jpeg, Screen Shot 2017-06-21 at 3.15.17 PM.png, 
> Screen Shot 2017-06-21 at 3.16.31 PM.png, Screen Shot 2017-06-21 at 9.59.22 
> AM.png
>
>
> I'd like to be able to create something like ports to transport flowfiles 
> around a process group to avoid having criss-crossed relationship lines like 
> in the attached screenshot.
> I drew something that represents what I'm talking about and attached a 
> sketch. I recreated the flow from the screenshot using portals.
> Portals would kind of be like funnels in that they don't have functionality 
> of their own and have a queue for each relationship attached to them.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3654) PutKinesisFirehose Needs A Separate Outgoing Relationship for Retryable Errors

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15948069#comment-15948069
 ] 

Nicholas Carenza commented on NIFI-3654:


the processor does write failure attributes that i can use

aws.kinesis.firehose.error.message
aws.kinesis.firehose.error.code

> PutKinesisFirehose Needs A Separate Outgoing Relationship for Retryable Errors
> --
>
> Key: NIFI-3654
> URL: https://issues.apache.org/jira/browse/NIFI-3654
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Nicholas Carenza
>Priority: Minor
>
> Just like the InvokeHttp processor understands what kinds of errors are worth 
> retrying and which are not, so should this processor.
> For example: the error when a file is over the max size of 1000kb is 
> unrecoverable and there is no point in retrying so output to failure. Server 
> unavailable, output to retry... and maybe administrative yield?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3654) PutKinesisFirehose Needs A Separate Outgoing Relationship for Retryable Errors

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-3654:
--

 Summary: PutKinesisFirehose Needs A Separate Outgoing Relationship 
for Retryable Errors
 Key: NIFI-3654
 URL: https://issues.apache.org/jira/browse/NIFI-3654
 Project: Apache NiFi
  Issue Type: Improvement
Reporter: Nicholas Carenza
Priority: Minor


Just like the InvokeHttp processor understands what kinds of errors are worth 
retrying and which are not, so should this processor.

For example: the error when a file is over the max size of 1000kb is 
unrecoverable and there is no point in retrying so output to failure. Server 
unavailable, output to retry... and maybe administrative yield?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3646) Show named relationship icons on processor hover

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-3646:
--

 Summary: Show named relationship icons on processor hover
 Key: NIFI-3646
 URL: https://issues.apache.org/jira/browse/NIFI-3646
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core UI
Reporter: Nicholas Carenza
Priority: Minor
 Attachments: nifi-connect-rel.png

When hovering over a processor, instead of showing a single add-connection 
element in the middle, show an array of connections. I attached an image 
example.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3606) MonitorAcitivity to track activity based on attribute

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-3606:
--

 Summary: MonitorAcitivity to track activity based on attribute
 Key: NIFI-3606
 URL: https://issues.apache.org/jira/browse/NIFI-3606
 Project: Apache NiFi
  Issue Type: Improvement
Reporter: Nicholas Carenza
Priority: Minor


Just like ControlRate can throttle based on attribute values, MonitorActivity 
should be able to monitor based on attribute values.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (NIFI-3573) PutKinesisFirehose Sends Batches Greater Than "Max message buffer size"

[Nicholas Carenza (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Carenza updated NIFI-3573:
---
Description: 
Firehose has some hard limits and I get these errors every so often even though 
I set "Max message buffer size" to 4MB.

InvalidArgumentException: Records size exceeds 4 MB limit (Service: 
AmazonKinesisFirehose; Status Code: 400; Error Code: InvalidArgumentException; 

http://docs.aws.amazon.com/firehose/latest/dev/limits.html

> PutKinesisFirehose Sends Batches Greater Than "Max message buffer size"
> ---
>
> Key: NIFI-3573
> URL: https://issues.apache.org/jira/browse/NIFI-3573
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Nicholas Carenza
>
> Firehose has some hard limits and I get these errors every so often even 
> though I set "Max message buffer size" to 4MB.
> InvalidArgumentException: Records size exceeds 4 MB limit (Service: 
> AmazonKinesisFirehose; Status Code: 400; Error Code: 
> InvalidArgumentException; 
> http://docs.aws.amazon.com/firehose/latest/dev/limits.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (NIFI-3573) PutKinesisFirehose Sends Batches Greater Than "Max message buffer size"

[Nicholas Carenza (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Carenza updated NIFI-3573:
---
Summary: PutKinesisFirehose Sends Batches Greater Than "Max message buffer 
size"  (was: PutKi)

> PutKinesisFirehose Sends Batches Greater Than "Max message buffer size"
> ---
>
> Key: NIFI-3573
> URL: https://issues.apache.org/jira/browse/NIFI-3573
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Nicholas Carenza
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (NIFI-3573) PutKinesisFirehose Sends Batches Greater Than "Max message buffer size"

[Nicholas Carenza (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Carenza updated NIFI-3573:
---
Priority: Minor  (was: Major)

> PutKinesisFirehose Sends Batches Greater Than "Max message buffer size"
> ---
>
> Key: NIFI-3573
> URL: https://issues.apache.org/jira/browse/NIFI-3573
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Nicholas Carenza
>Priority: Minor
>
> Firehose has some hard limits and I get these errors every so often even 
> though I set "Max message buffer size" to 4MB.
> InvalidArgumentException: Records size exceeds 4 MB limit (Service: 
> AmazonKinesisFirehose; Status Code: 400; Error Code: 
> InvalidArgumentException; 
> http://docs.aws.amazon.com/firehose/latest/dev/limits.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3573) PutKinesisFirehose Sends Batches Greater Than "Max message buffer size"

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901929#comment-15901929
 ] 

Nicholas Carenza commented on NIFI-3573:


sorry, hit enter and it submitted before even finishing the title

> PutKinesisFirehose Sends Batches Greater Than "Max message buffer size"
> ---
>
> Key: NIFI-3573
> URL: https://issues.apache.org/jira/browse/NIFI-3573
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Nicholas Carenza
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3573) PutKi

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-3573:
--

 Summary: PutKi
 Key: NIFI-3573
 URL: https://issues.apache.org/jira/browse/NIFI-3573
 Project: Apache NiFi
  Issue Type: Bug
Reporter: Nicholas Carenza






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3456) UserAgentParser Processor

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895266#comment-15895266
 ] 

Nicholas Carenza commented on NIFI-3456:


Anyone have experiment with http://uadetector.sourceforge.net/usage.html or 
have suggestions for a library to use?

> UserAgentParser Processor
> -
>
> Key: NIFI-3456
> URL: https://issues.apache.org/jira/browse/NIFI-3456
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Nicholas Carenza
>Priority: Minor
>
> Should work just like GeoEnrichIP. Given an attribute name to find the 
> user_agent string, parses it from each flowfile and adds additional 
> attributes for parsed fields.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3456) UserAgentParser Processor

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895248#comment-15895248
 ] 

Nicholas Carenza commented on NIFI-3456:


Starting work on this

> UserAgentParser Processor
> -
>
> Key: NIFI-3456
> URL: https://issues.apache.org/jira/browse/NIFI-3456
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Nicholas Carenza
>Priority: Minor
>
> Should work just like GeoEnrichIP. Given an attribute name to find the 
> user_agent string, parses it from each flowfile and adds additional 
> attributes for parsed fields.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3238) ListenLumberjack should support the *beat protocol

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890689#comment-15890689
 ] 

Nicholas Carenza commented on NIFI-3238:


I would really like to see this is next release

> ListenLumberjack should support the *beat protocol
> --
>
> Key: NIFI-3238
> URL: https://issues.apache.org/jira/browse/NIFI-3238
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Andre F de Miranda
>Assignee: Andre F de Miranda
>
> ListenLumberjack currently only supports v1 of the Lumberjack protocol. This 
> version has been deprecated in favor of v2, which is used on *beat (e.g. 
> filebeat, packetbeat, etc) edge components of the ELK  stack.
> We should consider deprecating ListenLumberjack or to extend it to handle 
> both v1 and v2 of the protocol.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (NIFI-3508) InvokeHTTP send request body on PATCH if PROP_SEND_BODY

[Nicholas Carenza (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Carenza updated NIFI-3508:
---
Resolution: Implemented
Status: Resolved  (was: Patch Available)

> InvokeHTTP send request body on PATCH if PROP_SEND_BODY
> ---
>
> Key: NIFI-3508
> URL: https://issues.apache.org/jira/browse/NIFI-3508
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Nicholas Carenza
>Priority: Minor
>  Labels: InvokeHTTP, processor
>
> I am dealing with an API that updates documents using the PATCH request 
> method and I need to send a request body but InvokeHTTP will only send a 
> request body if the request method is POST or PUT without any way to override.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3508) InvokeHTTP send request body on PATCH if PROP_SEND_BODY

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-3508:
--

 Summary: InvokeHTTP send request body on PATCH if PROP_SEND_BODY
 Key: NIFI-3508
 URL: https://issues.apache.org/jira/browse/NIFI-3508
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: Nicholas Carenza
Priority: Minor


I am dealing with an API that updates documents using the PATCH request method 
and I need to send a request body but InvokeHTTP will only send a request body 
if the request method is POST or PUT without any way to override.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3464) Claim Lock on Flowfile modifications

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-3464:
--

 Summary: Claim Lock on Flowfile modifications
 Key: NIFI-3464
 URL: https://issues.apache.org/jira/browse/NIFI-3464
 Project: Apache NiFi
  Issue Type: Improvement
Reporter: Nicholas Carenza
Priority: Minor
 Attachments: operate-lock.png

I am getting ready to introduce Nifi to my team and opening up the UI to them. 
One thing that concerns me is the chance that two of us might try to edit the 
same processor or related processors while the other is working on it.

It would be nice if there was a way to indicate in the UI to other clients that 
you are going to modify a certain processor/processor group or the entire flow.

This might be an enforced lock or just a warning/suggestion for other users. I 
imagine just having a button on the screen, maybe in the Operate window since 
it already switches context based on your selection (see image attachment).

It's probably enough to implement a lock as a warning. It will visually 
indicate if someone has locked it and warn if you try to change it anyways. you 
don't even need to keep track of who locked it and any user could perform the 
unlock. A more advanced implementation might add lock/unlock access privileges. 
For me, it would be enough to communicate intention to modify a particular 
processor or group through the UI.

One scenario I image is: some downstream processor starts failing, buffers 
start filling, errors start logging, slack starts putting and more than one 
person notice and attempt to triage.

Then of course there are actual locking mechanisms with which come a lot more 
complications.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3463) ExecuteSQL Errors on first use after long period of unuse

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-3463:
--

 Summary: ExecuteSQL Errors on first use after long period of unuse
 Key: NIFI-3463
 URL: https://issues.apache.org/jira/browse/NIFI-3463
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Nicholas Carenza
Priority: Trivial


I have a flow that runs ~ once a month and it gets this error for the first 
query each time it runs.

ExecuteSQL[id=61f3179d-34f3-1986-ee27-134f425dfa18] Unable to execute SQL 
select query SELECT blah blah blah,claim=StandardContentClaim 
[resourceClaim=StandardResourceClaim[id=1486767996789-3396773, 
container=default, section=165], offset=619840, 
length=89131409],offset=57849490,name=filename,size=1566] due to 
org.apache.nifi.processor.exception.ProcessException: 
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet 
successfully received from the server was 1,401,254,731 milliseconds ago.  The 
last packet sent successfully to the server was 1,401,254,731 milliseconds ago. 
is longer than the server configured value of 'wait_timeout'. You should 
consider either expiring and/or testing connection validity before use in your 
application, increasing the server configured values for client timeouts, or 
using the Connector/J connection property 'autoReconnect=true' to avoid this 
problem.; routing to failure: 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3462) Show Process Group hierarchy in Summary tables

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-3462:
--

 Summary: Show Process Group hierarchy in Summary tables
 Key: NIFI-3462
 URL: https://issues.apache.org/jira/browse/NIFI-3462
 Project: Apache NiFi
  Issue Type: Improvement
Reporter: Nicholas Carenza
Priority: Minor


I would find the summary pages more useful if I could see/filter by/group by 
the  process group hierarchy.

The simplest implementation would add Process Group column with values like: 
/some-process-group/some-nested-process-group.

A more advanced implementation would add a select list to filter processors by 
group:

Process Group ▼
-
> /
> /top-level-group-a
> /top-level-group-a/next-level-group-1
> /top-level-group-a/next-level-group-2

A further implementation would have nested collapsible sections grouped by 
process group.




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3460) Statsd Reporting Task

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-3460:
--

 Summary: Statsd Reporting Task
 Key: NIFI-3460
 URL: https://issues.apache.org/jira/browse/NIFI-3460
 Project: Apache NiFi
  Issue Type: New Feature
Reporter: Nicholas Carenza
Priority: Minor


I would like to be able to report metrics to Statsd.

https://github.com/etsy/statsd.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (NIFI-3460) Statsd Reporting Task

[Nicholas Carenza (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Carenza updated NIFI-3460:
---
Labels: reporting_task  (was: )

> Statsd Reporting Task
> -
>
> Key: NIFI-3460
> URL: https://issues.apache.org/jira/browse/NIFI-3460
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Nicholas Carenza
>Priority: Minor
>  Labels: reporting_task
>
> I would like to be able to report metrics to Statsd.
> https://github.com/etsy/statsd.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (NIFI-3460) Statsd Reporting Task

[Nicholas Carenza (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Carenza updated NIFI-3460:
---
Component/s: Core Framework

> Statsd Reporting Task
> -
>
> Key: NIFI-3460
> URL: https://issues.apache.org/jira/browse/NIFI-3460
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Nicholas Carenza
>Priority: Minor
>  Labels: reporting_task
>
> I would like to be able to report metrics to Statsd.
> https://github.com/etsy/statsd.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3438) Ability to link to a view in Nifi (process group, processor, etc...)

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858930#comment-15858930
 ] 

Nicholas Carenza commented on NIFI-3438:


[~andrewmlim] I think we can close this as duplicate of NIFI-3035.

> Ability to link to a view in Nifi (process group, processor, etc...)
> 
>
> Key: NIFI-3438
> URL: https://issues.apache.org/jira/browse/NIFI-3438
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Reporter: Nicholas Carenza
>Priority: Minor
>
> I would like to be able to share a link to a particular view in the Nifi UI. 
> For example:
> #/ might just bring me to that processor, center it in view and 
> reset the zoom. If it is a process group, it might go to it an bring all 
> processors into view.
> #//config might then also bring up the configuration for that 
> processor
> #//config/properties might then also set the tab to 'properties'



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3451) PutKinesisFirehose MAX_WAIT_TIME property for batch processing

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-3451:
--

 Summary: PutKinesisFirehose MAX_WAIT_TIME property for batch 
processing
 Key: NIFI-3451
 URL: https://issues.apache.org/jira/browse/NIFI-3451
 Project: Apache NiFi
  Issue Type: Improvement
Reporter: Nicholas Carenza
Priority: Trivial


Add MAX_WAIT_TIME property to PutKinesisFirehose for batch processing.

If the processor has not reached BATCH_SIZE or MAX_MESSAGE_BUFFER_SIZE_MB in 
MAX_WAIT_TIME, then it will send the batch as-is.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (NIFI-3444) PutKinesisFirehose Optional message demarcator

[Nicholas Carenza (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Carenza updated NIFI-3444:
---
Summary: PutKinesisFirehose Optional message demarcator  (was: 
PutKenesisFirehose Optional message demarcator)

> PutKinesisFirehose Optional message demarcator
> --
>
> Key: NIFI-3444
> URL: https://issues.apache.org/jira/browse/NIFI-3444
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Nicholas Carenza
>Priority: Trivial
>
> When using PutKenesisFirehose, flowfile content will be written as-is 
> byte-for-byte to the firehose stream with no demarcation. For me at least, I 
> expected there to be new lines in between but I can certainly understand this 
> not being the default.
> I have seen many processors with options for message demarcating. Is that an 
> option others might find useful on this processor and does it belong there?
> For now, I am using ReplaceText with '$' as the search and '' as 
> the replace.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3444) PutKenesisFirehose Optional message demarcator

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-3444:
--

 Summary: PutKenesisFirehose Optional message demarcator
 Key: NIFI-3444
 URL: https://issues.apache.org/jira/browse/NIFI-3444
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: Nicholas Carenza
Priority: Trivial


When using PutKenesisFirehose, flowfile content will be written as-is 
byte-for-byte to the firehose stream with no demarcation. For me at least, I 
expected there to be new lines in between but I can certainly understand this 
not being the default.

I have seen many processors with options for message demarcating. Is that an 
option others might find useful on this processor and does it belong there?

For now, I am using ReplaceText with '$' as the search and '' as 
the replace.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3437) KafkaConsumer pause/resume

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15854475#comment-15854475
 ] 

Nicholas Carenza commented on NIFI-3437:


[~joewitt] Apologies I am not very experienced with Kafka, or Nifi for that 
matter. I was trying to find something in the ConsumeKafka processor that kept 
track of state. I read up on the Kafka documentation a bit and I think I 
understand the processor works with Kafka to keep track of its offset. Let me 
see if I have this right:

The processor commits its latest consumed offset to Kafka at least every 
MAX_UNCOMMITTED_TIME and it's Kafka that keeps a record based on the consumer's 
group id. So when I stop and start the processor, as long as it has the same 
group id, Kafka will start sending messages starting from the last committed 
offset.

https://kafka.apache.org/documentation/#distributionimpl

> KafkaConsumer pause/resume
> --
>
> Key: NIFI-3437
> URL: https://issues.apache.org/jira/browse/NIFI-3437
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Nicholas Carenza
>Priority: Minor
>
> Could the KafkaConsumer processor keep track of it's position so that if it 
> is stopped and started again it can attempt to resume consuming from that 
> position?
> If I have to restart nifi or rename the consumer processor I want to make 
> sure I don't miss any messages.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (NIFI-3437) KafkaConsumer pause/resume

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15854475#comment-15854475
 ] 

Nicholas Carenza edited comment on NIFI-3437 at 2/6/17 6:13 PM:


[~joewitt] Apologies I am not very experienced with Kafka, or Nifi for that 
matter. I was trying to find something in the ConsumeKafka processor that kept 
track of state. I read up on the Kafka documentation a bit and I think I 
understand how the processor works with Kafka to keep track of its offset. Let 
me see if I have this right:

The processor commits its latest consumed offset to Kafka at least every 
MAX_UNCOMMITTED_TIME and it's Kafka that keeps a record based on the consumer's 
group id. So when I stop and start the processor, as long as it has the same 
group id, Kafka will start sending messages starting from the last committed 
offset.

https://kafka.apache.org/documentation/#distributionimpl


was (Author: ncarenza):
[~joewitt] Apologies I am not very experienced with Kafka, or Nifi for that 
matter. I was trying to find something in the ConsumeKafka processor that kept 
track of state. I read up on the Kafka documentation a bit and I think I 
understand the processor works with Kafka to keep track of its offset. Let me 
see if I have this right:

The processor commits its latest consumed offset to Kafka at least every 
MAX_UNCOMMITTED_TIME and it's Kafka that keeps a record based on the consumer's 
group id. So when I stop and start the processor, as long as it has the same 
group id, Kafka will start sending messages starting from the last committed 
offset.

https://kafka.apache.org/documentation/#distributionimpl

> KafkaConsumer pause/resume
> --
>
> Key: NIFI-3437
> URL: https://issues.apache.org/jira/browse/NIFI-3437
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Nicholas Carenza
>Priority: Minor
>
> Could the KafkaConsumer processor keep track of it's position so that if it 
> is stopped and started again it can attempt to resume consuming from that 
> position?
> If I have to restart nifi or rename the consumer processor I want to make 
> sure I don't miss any messages.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (NIFI-3437) KafkaConsumer pause/resume

[Nicholas Carenza (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Carenza updated NIFI-3437:
---
Priority: Minor  (was: Major)

> KafkaConsumer pause/resume
> --
>
> Key: NIFI-3437
> URL: https://issues.apache.org/jira/browse/NIFI-3437
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Nicholas Carenza
>Priority: Minor
>
> Could the KafkaConsumer processor keep track of it's position so that if it 
> is stopped and started again it can attempt to resume consuming from that 
> position?
> If I have to restart nifi or rename the consumer processor I want to make 
> sure I don't miss any messages.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3438) Ability to link to a view in Nifi (process group, processor, etc...)

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-3438:
--

 Summary: Ability to link to a view in Nifi (process group, 
processor, etc...)
 Key: NIFI-3438
 URL: https://issues.apache.org/jira/browse/NIFI-3438
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core UI
Reporter: Nicholas Carenza
Priority: Minor


I would like to be able to share a link to a particular view in the Nifi UI. 

For example:
#/ might just bring me to that processor, center it in view and 
reset the zoom. If it is a process group, it might go to it an bring all 
processors into view.
#//config might then also bring up the configuration for that 
processor
#//config/properties might then also set the tab to 'properties'



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3437) KafkaConsumer pause/resume

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-3437:
--

 Summary: KafkaConsumer pause/resume
 Key: NIFI-3437
 URL: https://issues.apache.org/jira/browse/NIFI-3437
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: Nicholas Carenza


Could the KafkaConsumer processor keep track of it's position so that if it is 
stopped and started again it can attempt to resume consuming from that position?

If I have to restart nifi or rename the consumer processor I want to make sure 
I don't miss any messages.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (NIFI-3430) PutSQL Errors attempting to coerce date string to timestamp

[Nicholas Carenza (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Carenza updated NIFI-3430:
---
Priority: Minor  (was: Major)

> PutSQL Errors attempting to coerce date string to timestamp
> ---
>
> Key: NIFI-3430
> URL: https://issues.apache.org/jira/browse/NIFI-3430
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Nicholas Carenza
>Priority: Minor
>  Labels: convertjsontosql, putsql
>
> I am attempting to write a flat json object to my postgresql database using 
> ConvertJSONToSQL followed by PutSQL. My json has a string date field in ISO 
> 8601 format which postgres can accept as a timestamp. The column in the 
> database is of type 'timestamp without time zone'.
> org.apache.nifi.processor.exception.ProcessException: The value of the 
> sql.args.33.value is '2017-02-01T23:12:21+00:00', which cannot be converted 
> to a timestamp
> Is PutSQL trying to do some preprocessing here that it should just let the 
> database handle?
> select  cast('2017-02-01T23:12:21+00:00' as timestamp without time zone);
>   timestamp  
> -
>  2017-02-01 23:12:21
> (1 row)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (NIFI-3430) PutSQL Errors attempting to coerce date string to timestamp

[Nicholas Carenza (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Carenza updated NIFI-3430:
---
Issue Type: Improvement  (was: Bug)

> PutSQL Errors attempting to coerce date string to timestamp
> ---
>
> Key: NIFI-3430
> URL: https://issues.apache.org/jira/browse/NIFI-3430
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Nicholas Carenza
>  Labels: convertjsontosql, putsql
>
> I am attempting to write a flat json object to my postgresql database using 
> ConvertJSONToSQL followed by PutSQL. My json has a string date field in ISO 
> 8601 format which postgres can accept as a timestamp. The column in the 
> database is of type 'timestamp without time zone'.
> org.apache.nifi.processor.exception.ProcessException: The value of the 
> sql.args.33.value is '2017-02-01T23:12:21+00:00', which cannot be converted 
> to a timestamp
> Is PutSQL trying to do some preprocessing here that it should just let the 
> database handle?
> select  cast('2017-02-01T23:12:21+00:00' as timestamp without time zone);
>   timestamp  
> -
>  2017-02-01 23:12:21
> (1 row)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (NIFI-3429) ConvertJSONToSQL Does not work with table identifiers that need to be quoted

[Nicholas Carenza (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Carenza updated NIFI-3429:
---
Priority: Minor  (was: Major)

> ConvertJSONToSQL Does not work with table identifiers that need to be quoted
> 
>
> Key: NIFI-3429
> URL: https://issues.apache.org/jira/browse/NIFI-3429
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Nicholas Carenza
>Priority: Minor
>
> I have a table identifer that needs to be quoted. If i supple the table 
> identifer unquoted in the properties, the processor can convert the json to 
> sql but the output sql doesn't have the table name quoted even if I select 
> the option Quote Identifiers because, true to the description, it only 
> applies to column identifiers. If I supply the table identifier quoted in the 
> properties, then the processor fails completely because it can't find the 
> table.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (NIFI-3428) Processor Instances (Shared Settings)

[Nicholas Carenza (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Carenza updated NIFI-3428:
---
Priority: Minor  (was: Major)

> Processor Instances (Shared Settings)
> -
>
> Key: NIFI-3428
> URL: https://issues.apache.org/jira/browse/NIFI-3428
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Nicholas Carenza
>Priority: Minor
>  Labels: processor
>
> I have a lot of duplicated processors that I would like to keep in sync with 
> each other. Could we discuss a method of sharing configuration parameters 
> amongst multiple processors of the same kind such that they can all be 
> updated at once?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Issue Comment Deleted] (NIFI-3430) PutSQL Errors attempting to coerce date string to timestamp

[Nicholas Carenza (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Carenza updated NIFI-3430:
---
Comment: was deleted

(was: https://github.com/apache/nifi/pull/1463)

> PutSQL Errors attempting to coerce date string to timestamp
> ---
>
> Key: NIFI-3430
> URL: https://issues.apache.org/jira/browse/NIFI-3430
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Nicholas Carenza
>  Labels: convertjsontosql, putsql
>
> I am attempting to write a flat json object to my postgresql database using 
> ConvertJSONToSQL followed by PutSQL. My json has a string date field in ISO 
> 8601 format which postgres can accept as a timestamp. The column in the 
> database is of type 'timestamp without time zone'.
> org.apache.nifi.processor.exception.ProcessException: The value of the 
> sql.args.33.value is '2017-02-01T23:12:21+00:00', which cannot be converted 
> to a timestamp
> Is PutSQL trying to do some preprocessing here that it should just let the 
> database handle?
> select  cast('2017-02-01T23:12:21+00:00' as timestamp without time zone);
>   timestamp  
> -
>  2017-02-01 23:12:21
> (1 row)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (NIFI-3430) PutSQL Errors attempting to coerce date string to timestamp

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15850224#comment-15850224
 ] 

Nicholas Carenza edited comment on NIFI-3430 at 2/2/17 5:40 PM:


refactoring to leverage the existing sql.args.n.format attribute and 
java.time.format.DateTimeFormatter


was (Author: ncarenza):
refactoring to leverage the existing sql.args.n.format attribute

> PutSQL Errors attempting to coerce date string to timestamp
> ---
>
> Key: NIFI-3430
> URL: https://issues.apache.org/jira/browse/NIFI-3430
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Nicholas Carenza
>  Labels: convertjsontosql, putsql
>
> I am attempting to write a flat json object to my postgresql database using 
> ConvertJSONToSQL followed by PutSQL. My json has a string date field in ISO 
> 8601 format which postgres can accept as a timestamp. The column in the 
> database is of type 'timestamp without time zone'.
> org.apache.nifi.processor.exception.ProcessException: The value of the 
> sql.args.33.value is '2017-02-01T23:12:21+00:00', which cannot be converted 
> to a timestamp
> Is PutSQL trying to do some preprocessing here that it should just let the 
> database handle?
> select  cast('2017-02-01T23:12:21+00:00' as timestamp without time zone);
>   timestamp  
> -
>  2017-02-01 23:12:21
> (1 row)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3430) PutSQL Errors attempting to coerce date string to timestamp

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849258#comment-15849258
 ] 

Nicholas Carenza commented on NIFI-3430:


https://github.com/apache/nifi/pull/1463

> PutSQL Errors attempting to coerce date string to timestamp
> ---
>
> Key: NIFI-3430
> URL: https://issues.apache.org/jira/browse/NIFI-3430
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Nicholas Carenza
>  Labels: convertjsontosql, putsql
>
> I am attempting to write a flat json object to my postgresql database using 
> ConvertJSONToSQL followed by PutSQL. My json has a string date field in ISO 
> 8601 format which postgres can accept as a timestamp. The column in the 
> database is of type 'timestamp without time zone'.
> org.apache.nifi.processor.exception.ProcessException: The value of the 
> sql.args.33.value is '2017-02-01T23:12:21+00:00', which cannot be converted 
> to a timestamp
> Is PutSQL trying to do some preprocessing here that it should just let the 
> database handle?
> select  cast('2017-02-01T23:12:21+00:00' as timestamp without time zone);
>   timestamp  
> -
>  2017-02-01 23:12:21
> (1 row)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (NIFI-3430) PutSQL Errors attempting to coerce date string to timestamp

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849233#comment-15849233
 ] 

Nicholas Carenza edited comment on NIFI-3430 at 2/2/17 1:27 AM:


Apparently it only supports a single string date format hardcoded into the 
processor:
`SimpleDateFormat dateFormat = new SimpleDateFormat("-MM-dd HH:mm:ss.SSS");`


was (Author: ncarenza):
Apprently it only supports a single string date format hardcoded into the 
processor:
`SimpleDateFormat dateFormat = new SimpleDateFormat("-MM-dd HH:mm:ss.SSS");`

> PutSQL Errors attempting to coerce date string to timestamp
> ---
>
> Key: NIFI-3430
> URL: https://issues.apache.org/jira/browse/NIFI-3430
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Nicholas Carenza
>  Labels: convertjsontosql, putsql
>
> I am attempting to write a flat json object to my postgresql database using 
> ConvertJSONToSQL followed by PutSQL. My json has a string date field in ISO 
> 8601 format which postgres can accept as a timestamp. The column in the 
> database is of type 'timestamp without time zone'.
> org.apache.nifi.processor.exception.ProcessException: The value of the 
> sql.args.33.value is '2017-02-01T23:12:21+00:00', which cannot be converted 
> to a timestamp
> Is PutSQL trying to do some preprocessing here that it should just let the 
> database handle?
> select  cast('2017-02-01T23:12:21+00:00' as timestamp without time zone);
>   timestamp  
> -
>  2017-02-01 23:12:21
> (1 row)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3429) ConvertJSONToSQL Does not work with table identifiers that need to be quoted

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849232#comment-15849232
 ] 

Nicholas Carenza commented on NIFI-3429:


Apprently it only supports a single string date format hardcoded into the 
processor:

`SimpleDateFormat dateFormat = new SimpleDateFormat("-MM-dd HH:mm:ss.SSS");`

> ConvertJSONToSQL Does not work with table identifiers that need to be quoted
> 
>
> Key: NIFI-3429
> URL: https://issues.apache.org/jira/browse/NIFI-3429
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Nicholas Carenza
>
> I have a table identifer that needs to be quoted. If i supple the table 
> identifer unquoted in the properties, the processor can convert the json to 
> sql but the output sql doesn't have the table name quoted even if I select 
> the option Quote Identifiers because, true to the description, it only 
> applies to column identifiers. If I supply the table identifier quoted in the 
> properties, then the processor fails completely because it can't find the 
> table.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3430) PutSQL Errors attempting to coerce date string to timestamp

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849233#comment-15849233
 ] 

Nicholas Carenza commented on NIFI-3430:


Apprently it only supports a single string date format hardcoded into the 
processor:
`SimpleDateFormat dateFormat = new SimpleDateFormat("-MM-dd HH:mm:ss.SSS");`

> PutSQL Errors attempting to coerce date string to timestamp
> ---
>
> Key: NIFI-3430
> URL: https://issues.apache.org/jira/browse/NIFI-3430
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Nicholas Carenza
>  Labels: convertjsontosql, putsql
>
> I am attempting to write a flat json object to my postgresql database using 
> ConvertJSONToSQL followed by PutSQL. My json has a string date field in ISO 
> 8601 format which postgres can accept as a timestamp. The column in the 
> database is of type 'timestamp without time zone'.
> org.apache.nifi.processor.exception.ProcessException: The value of the 
> sql.args.33.value is '2017-02-01T23:12:21+00:00', which cannot be converted 
> to a timestamp
> Is PutSQL trying to do some preprocessing here that it should just let the 
> database handle?
> select  cast('2017-02-01T23:12:21+00:00' as timestamp without time zone);
>   timestamp  
> -
>  2017-02-01 23:12:21
> (1 row)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Issue Comment Deleted] (NIFI-3429) ConvertJSONToSQL Does not work with table identifiers that need to be quoted

[Nicholas Carenza (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Carenza updated NIFI-3429:
---
Comment: was deleted

(was: Apprently it only supports a single string date format hardcoded into the 
processor:

`SimpleDateFormat dateFormat = new SimpleDateFormat("-MM-dd 
HH:mm:ss.SSS");`)

> ConvertJSONToSQL Does not work with table identifiers that need to be quoted
> 
>
> Key: NIFI-3429
> URL: https://issues.apache.org/jira/browse/NIFI-3429
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Nicholas Carenza
>
> I have a table identifer that needs to be quoted. If i supple the table 
> identifer unquoted in the properties, the processor can convert the json to 
> sql but the output sql doesn't have the table name quoted even if I select 
> the option Quote Identifiers because, true to the description, it only 
> applies to column identifiers. If I supply the table identifier quoted in the 
> properties, then the processor fails completely because it can't find the 
> table.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3430) PutSQL Errors attempting to coerce date string to timestamp

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-3430:
--

 Summary: PutSQL Errors attempting to coerce date string to 
timestamp
 Key: NIFI-3430
 URL: https://issues.apache.org/jira/browse/NIFI-3430
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Nicholas Carenza


I am attempting to write a flat json object to my postgresql database using 
ConvertJSONToSQL followed by PutSQL. My json has a string date field in ISO 
8601 format which postgres can accept as a timestamp. The column in the 
database is of type 'timestamp without time zone'.

org.apache.nifi.processor.exception.ProcessException: The value of the 
sql.args.33.value is '2017-02-01T23:12:21+00:00', which cannot be converted to 
a timestamp

Is PutSQL trying to do some preprocessing here that it should just let the 
database handle?

select  cast('2017-02-01T23:12:21+00:00' as timestamp without time zone);
  timestamp  
-
 2017-02-01 23:12:21
(1 row)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3429) ConvertJSONToSQL Does not work with table identifiers that need to be quoted

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-3429:
--

 Summary: ConvertJSONToSQL Does not work with table identifiers 
that need to be quoted
 Key: NIFI-3429
 URL: https://issues.apache.org/jira/browse/NIFI-3429
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Nicholas Carenza


I have a table identifer that needs to be quoted. If i supple the table 
identifer unquoted in the properties, the processor can convert the json to sql 
but the output sql doesn't have the table name quoted even if I select the 
option Quote Identifiers because, true to the description, it only applies to 
column identifiers. If I supply the table identifier quoted in the properties, 
then the processor fails completely because it can't find the table.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3428) Processor Instances (Shared Settings)

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-3428:
--

 Summary: Processor Instances (Shared Settings)
 Key: NIFI-3428
 URL: https://issues.apache.org/jira/browse/NIFI-3428
 Project: Apache NiFi
  Issue Type: New Feature
  Components: Core Framework
Reporter: Nicholas Carenza


I have a lot of duplicated processors that I would like to keep in sync with 
each other. Could we discuss a method of sharing configuration parameters 
amongst multiple processors of the same kind such that they can all be updated 
at once?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3351) Manual Review Processor/Queue

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15824457#comment-15824457
 ] 

Nicholas Carenza commented on NIFI-3351:


Hey Joseph,

I don't see a way to modify messages from a queue. I could use the 
RouteOnAttribute processor following a way to manually set an attribute on a 
flowfile. I am imagining a use case like manually reviewing posted content on a 
form and labeling it as ok or not. The automatic expiration feature would 
actually _send_ the flowfile to the default relationship instead of deleting it 
like the queue expiration setting.

I hope that clarifies the features I am after.

- Nick

> Manual Review Processor/Queue
> -
>
> Key: NIFI-3351
> URL: https://issues.apache.org/jira/browse/NIFI-3351
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Nicholas Carenza
>Priority: Minor
>  Labels: processor, queue
> Attachments: nifi-manual-queue.png
>
>
> I'd like to direct certain flowfiles to a manual review queue in certain 
> failure cases where I can view each flowfile just like I can when listing a 
> queue and then choose one of the relationships to send it to.
> Features:
> - Create any number of outgoing relationships
> - Review items selectively from the queue
> - Built-in Expiration relationship for automatically processing flowfiles 
> after a given time or buffer size or flowfile count.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-3351) Manual Review Processor/Queue

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-3351:
--

 Summary: Manual Review Processor/Queue
 Key: NIFI-3351
 URL: https://issues.apache.org/jira/browse/NIFI-3351
 Project: Apache NiFi
  Issue Type: New Feature
  Components: Core Framework
Reporter: Nicholas Carenza
Priority: Minor
 Attachments: nifi-manual-queue.png

I'd like to direct certain flowfiles to a manual review queue in certain 
failure cases where I can view each flowfile just like I can when listing a 
queue and then choose one of the relationships to send it to.

Features:
- Create any number of outgoing relationships
- Review items selectively from the queue
- Built-in Expiration relationship for automatically processing flowfiles after 
a given time or buffer size or flowfile count.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (NIFI-3280) PostHTTP Option to write response to attribute or flowfile content

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805858#comment-15805858
 ] 

Nicholas Carenza edited comment on NIFI-3280 at 1/6/17 9:44 PM:


You are correct. I didn't notice the InvokeHttp processor already had this 
capability.


was (Author: ncarenza):
You are correct. I didn't notice that processor already had this capability.

> PostHTTP Option to write response to attribute or flowfile content
> --
>
> Key: NIFI-3280
> URL: https://issues.apache.org/jira/browse/NIFI-3280
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Nicholas Carenza
>Priority: Minor
>
> Currently there doesn't seem to be a way to access anything about the 
> response of an HTTP request from the PostHTTP processor other than whether it 
> failed or succeeded.
> I would like to be able to:
>  - store the response body in an attribute or replace the flowfile content
>  - store the status code in an attribute



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3280) PostHTTP Option to write response to attribute or flowfile content

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805858#comment-15805858
 ] 

Nicholas Carenza commented on NIFI-3280:


You are correct. I didn't notice that processor already had this capability.

> PostHTTP Option to write response to attribute or flowfile content
> --
>
> Key: NIFI-3280
> URL: https://issues.apache.org/jira/browse/NIFI-3280
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Nicholas Carenza
>Priority: Minor
>
> Currently there doesn't seem to be a way to access anything about the 
> response of an HTTP request from the PostHTTP processor other than whether it 
> failed or succeeded.
> I would like to be able to:
>  - store the response body in an attribute or replace the flowfile content
>  - store the status code in an attribute



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-3280) PostHTTP Option to write response to attribute or flowfile content

[Nicholas Carenza (JIRA)]

Nicholas Carenza created NIFI-3280:
--

 Summary: PostHTTP Option to write response to attribute or 
flowfile content
 Key: NIFI-3280
 URL: https://issues.apache.org/jira/browse/NIFI-3280
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: Nicholas Carenza
Priority: Minor


Currently there doesn't seem to be a way to access anything about the response 
of an HTTP request from the PostHTTP processor other than whether it failed or 
succeeded.

I would like to be able to:
 - store the response body in an attribute or replace the flowfile content
 - store the status code in an attribute



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3279) Upgrade jolt to version 0.1.0

[Nicholas Carenza (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15799002#comment-15799002
 ] 

Nicholas Carenza commented on NIFI-3279:


Hi [~pvillard], thanks. I missed that dependency bump from that PR.

> Upgrade jolt to version 0.1.0
> -
>
> Key: NIFI-3279
> URL: https://issues.apache.org/jira/browse/NIFI-3279
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Nicholas Carenza
>Priority: Minor
>  Labels: jolt, processor
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Jolt 0.0.22 and beyond offer a new modify-* class of operations that are very 
> useful for transforming documents.
> http://jolt-demo.appspot.com/#modify-listFunctions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)