Re: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

2018-10-30 Thread James Srinivasan
Apologies if I've missed this in the discussion so far - we use the InvokeHTTP processor a lot, and the invokehttp.java.exception.message attribute is really handy diving into why things have failed without having to match up logs with flow files (from a system with hundreds of processors making th

RE: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

2018-10-30 Thread Peter Wicks (pwicks)
Sorry for the delayed response, I've been traveling. Responses in order: Matt, Right now our work around is to keep retrying errors, usually with a penalty or control rate processor. The problem is that we don't know why it failed, and thus don't know if retry is the correct option. I have not

Re: Using record path API to remove field values

2018-10-30 Thread Bryan Bende
I'm not sure that removing a field from record path makes sense since technically removing the field all together is a different schema. You could easily make two versions of the schema, one with all fields, and a second with a subset, then just use any record processor with a reader using schema

Re: Using record path API to remove field values

2018-10-30 Thread Mike Thomsen
Something more aggressive than that. I have a new ES bulk operation processor that uses the record API to specify operations. I want to be able to remove the record paths that relate to things like the index and type. In StandardFieldValue it also looks like we ignore nulls altogether: @Overr

Re: Hi

2018-10-30 Thread Milan Das
I mean ParseEvtx processor. On 10/30/18, 7:37 AM, "Milan Das" wrote: Did you try WinEventLogProcessor ? Thanks, Milan Das On 10/29/18, 11:17 PM, "find" <907611...@qq.com> wrote: Hi, Sorry to disturb you.I want to parse a .evtx(a window event log file

Re: Hi

2018-10-30 Thread Otto Fowler
https://community.hortonworks.com/articles/58493/parsing-evtx-files-with-apache-nifi.html On October 29, 2018 at 23:17:27, find (907611...@qq.com) wrote: Hi, Sorry to disturb you.I want to parse a .evtx(a window event log file) file using nifi(https://github.com/apache/nifi),but I just don't kno

Re: Using record path API to remove field values

2018-10-30 Thread Pierre Villard
Hey Mike, In the JSON Record Set Writer you have a "Suppress Null Values" parameter to do just that. Are you thinking about something else? Pierre Le mar. 30 oct. 2018 à 13:44, Mike Thomsen a écrit : > Is there an example of how to use the record path API to remove an element > from a record?

Using record path API to remove field values

2018-10-30 Thread Mike Thomsen
Is there an example of how to use the record path API to remove an element from a record? Ideally, if the field is nullable, I'd like to be able to completely remove the value rather than send a null value (to Elastic in this case). Thanks, Mike

Re: Hi

2018-10-30 Thread Milan Das
Did you try WinEventLogProcessor ? Thanks, Milan Das On 10/29/18, 11:17 PM, "find" <907611...@qq.com> wrote: Hi, Sorry to disturb you.I want to parse a .evtx(a window event log file) file using nifi(https://github.com/apache/nifi),but I just don't know how to use it(specific codes to