Hi,
I will test your idea to use RouteOnContent with a length check before the
ParseCEF processor.
Since all message needs to be parsed, even if the msg field exceed 1023
character, I will probably have to save the content of the msg field
temporarily, truncate the msg field in the message, pars
Hi, I`m quite sure, I have verified it in a small test Flow. A propert to set
the do_validate flag to false would be helpful in this case. Regards, Lj
På fre, jan. 2019 klockan 0:05, Otto Fowler skrev:
#yiv4068205678 body{font-family:Helvetica, Arial;font-size:13px;}Can I ask how
you are
Hi Felix,
As you noticed, the CEF standard indeed states that 1023 characters is the
limit and I such when I wrote the underlying parser (ParCEFone) I enforced
a length check and failed the parsing in case of exceeding that value.
Could I just truncate? Possibly... But what to do when the vendor
Otto,
The parser should fail in that case (and many others).
https://github.com/fluenda/ParCEFone/blob/master/src/main/java/com/fluenda/parcefone/event/CefRev23.java#L327
Cheers
On Fri, Jan 4, 2019 at 10:05 AM Otto Fowler wrote:
> Can I ask how you are sure it is the message size that is caus
Hi all,
I have a job consist of following steps: first consuming data from kafka,
and then packing data every 5 minutes into one file, finally put the packed
file into hdfs.
I use the [MergeContent] processor to accomplish the “packing” step. The
properties of MergeContent I configured is list belo
Hi Jianan
As you have “Minimum Number of Entries: 1” it is normal that you can see merges
with only one flowfile. In my opinion the “Minimum Number of Entries” is
stronger than the “Max Bin Age” (first is written bold and second not).
Additionally it is called “Max Bin Age” and not “Bin Age”. S
Hi Josef,
Thanks for reply. In my opinion the “Minimum Number of Entries” is should
not and can not stronger than the “Max Bin Age”. Suppose I have only ONE
flowfile from datasource put into MergeContent processor, and I set
"Minimum Number of Entries" = 2, then this ONE flowfile will never coming
Hi Jianan
I just say that as soon as “Minimum Number of Entries” is reached the flow can
be flushed out, and further if the minimum number isn’t reached I would expect
that the “Max Bin Age” takes place. Have you tried that?
Cheers Josef
From: Jianan Zhang
Reply-To: "users@nifi.apache.org"
D
Yeah, I went through the code after my mail last night. Would having
validation optional not be a valid setting? How often would the output
feeding this flow be invalid for some other reason?
On January 4, 2019 at 04:33:54, Andre (andre-li...@fucs.org) wrote:
Otto,
The parser should fail in
Josef,
Thanks for the info! There are a few things to consider here. Firstly, you said
that you are using NiFi 1.8.0.
Are you using the new Load Balancing capability? I.e., do you have any
Connections configured to balance
load across your cluster? And if so, are you load-balancing any 0-byte fi
The inner workings of MergeContent is certainly a FAQ. This message [1] to
the users list from a long time ago may help. I think it's still accurate.
[1] -
https://lists.apache.org/thread.html/5ab5d9d0bcd0eef8ace391d00f5f5678427bee4b2fbf1e48d78ea8c8@1445464430@%3Cusers.nifi.apache.org%3E
Regard
Mark,
Yes we are using Load Balancing capability and we do that after the ListSFTP
processor, so yes we loadbalance 0-Byte files. Seems that we probably facing
your Bug here.
Thanks a lot for explaining in detail what happens regarding the
flowfile/content repo in NiFi.
Additionally we have s
Josef,
OK, thanks for confirming. My suspicion is that the Load-Balancing bug is what
is biting you, and that
when you tried to replicate with the CompressContent in a simple case, you may
have just been experiencing
the "cleanup lag" related to the way that the repositories interact with one
a
Otto,
I considered that but never implemented it in NiFi as I had concerns people
would have the incorrect assumption the parser can help on those cases.
>From the top of my head, I have seen type mismatch, length overflows,
malformed CEF headers (the pipe delimited section of the CEF message). I
Please avoid sending more copies of the question. Hopefully someone
familiar with the processors in question will be available in time.
Thanks
On Fri, Jan 4, 2019 at 9:14 PM Luis Carmona wrote:
> Hi everyone,
>
> Im having a strange behavior with Wait / Notify mechanism. Attached is
> the im
I'm sorry,
got messages from nifi-users saying "UNCHECKED", and reading about understood
the message did not get trough.
Thanks for letting me know.
LC
De: "Joe Witt"
Para: "users"
Enviados: Viernes, 4 de Enero 2019 23:23:02
Asunto: Re: Wait/Notify inconsistent behavior
Please avoi
thanks for letting us know. the lists can be a bit awkward from a user
experience pov. no worries
On Fri, Jan 4, 2019, 9:26 PM Luis Carmona I'm sorry,
>
> got messages from nifi-users saying "UNCHECKED", and reading about
> understood the message did not get trough.
>
> Thanks for letting me kn
17 matches
Mail list logo