Patch available at https://issues.apache.org/jira/browse/NIFI-5761
<https://www.google.com/url?q=https://issues.apache.org/jira/browse/NIFI-5761&sa=D&source=hangouts&ust=1540669801593000&usg=AFQjCNHVVj1whk8fb1Z3Ic51w2TdwvrC1Q>
On Thu, 18 Oct 2018 at 13:33 Juan Pablo Gardella <
gardellajuanpa...@gmail.com> wrote:
> Exactly, I know it is not an issue in the replace text code, but it
> happens inside it. If we are using the ReplaceText in multiple places, it
> increases the flow design complexity. We need to evaluate all expressions
> before sending to the processor, to be sure will not fail in ReplaceText
> processor.
>
> Notice it's impossible if you have to process content dynamically. I would
> happy to file a ticket and the patch as I mention.
>
> Juan
>
>
>
> On Thu, 18 Oct 2018 at 13:18 Shawn Weeks <swe...@weeksconsulting.us>
> wrote:
>
>> I understand the issue now, I’m not sure that a failure of ReplaceText is
>> the best place to catch this though. The reason I’m not sure it’s the best
>> place is what happens if there are multiple failures because you had
>> multiple expressions, just having them all routed to the same failure
>> wouldn’t help you make decisions on what to do with a single attribute.
>> Perhaps a better solution would be to use a RouteOnAttribute to check if
>> the attributes match a certain pattern before sending them to ReplaceText.
>> A possible expression could be
>> “${actualSettlementDate:matches('[0-9]{2}/[0-9]{2}/[0-9]{4}')}” however
>> that would not catch things that look like dates but aren’t valid.
>>
>>
>>
>> Thanks
>>
>> Shawn Weeks
>>
>>
>>
>> *From:* Juan Pablo Gardella <gardellajuanpa...@gmail.com>
>>
>> *Sent:* Thursday, October 18, 2018 11:03 AM
>>
>>
>> *To:* users@nifi.apache.org
>> *Subject:* Re: [EXT] ReplaceText cannot consume messages if Regex does
>> not match
>>
>>
>>
>> At *search value*:(?s)(^.*$)
>>
>>
>>
>> At *Replacement value*:
>>
>>
>> *<?xml version="1.0" encoding="UTF-8"?>
>> <SettleDate>${actualSettlementDate:toDate('MM/dd/yyyy'):format("yyyy-MM-dd'T'00:00:00.0000000")}</SettleDate>*
>>
>>
>>
>> The actualSettlementDate is a flowfile attribute. The problem is the
>> replacement value is evaluated inside the processor and the *toDate *method
>> fails.
>>
>>
>>
>> Hope it's clear now.
>>
>>
>>
>>
>>
>> On Thu, 18 Oct 2018 at 12:51 Shawn Weeks <swe...@weeksconsulting.us>
>> wrote:
>>
>> I’m still trying to understand your actual issue, can your provide a
>> screenshot of the ReplaceText config like the attached, I need to see
>> exactly where you’re putting the expression. A template would also be
>> really helpful.
>>
>>
>>
>> Thanks
>>
>> Shawn Weeks
>>
>>
>>
>> *From:* Juan Pablo Gardella <gardellajuanpa...@gmail.com>
>>
>> *Sent:* Thursday, October 18, 2018 10:45 AM
>>
>>
>> *To:* users@nifi.apache.org
>> *Subject:* Re: [EXT] ReplaceText cannot consume messages if Regex does
>> not match
>>
>>
>>
>> At ReplaceText
>> <https://raw.githubusercontent.com/apache/nifi/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceText.java>processor
>> we have:
>>
>>
>>
>> [image: image.png]
>>
>> As you can see, only if *StackOverflowError *is raised during the
>> evaluation, the flowfile is send to failure relationship. I would like to
>> update the code to use Exception or NifiExpressionFailedException (if it
>> exits).
>>
>>
>>
>> Juan
>>
>>
>>
>> On Thu, 18 Oct 2018 at 12:33 Shawn Weeks <swe...@weeksconsulting.us>
>> wrote:
>>
>> What processor are you defining your expression in? I also may be
>> misunderstanding the problem because I don’t see any regular expressions
>> anywhere. Can you create a sample workflow showing your issue so I can take
>> a look at it.
>>
>>
>>
>> Thanks
>>
>> Shawn Weeks
>>
>>
>>
>> *From:* Juan Pablo Gardella <gardellajuanpa...@gmail.com>
>> *Sent:* Thursday, October 18, 2018 10:27 AM
>> *To:* users@nifi.apache.org
>> *Subject:* Re: [EXT] ReplaceText cannot consume messages if Regex does
>> not match
>>
>>
>>
>> No, it's not a valid date. I would like if it an error happens, I would
>> like to throw the flowfile to failure and continue.
>>
>>
>>
>> On Thu, 18 Oct 2018 at 12:19 Shawn Weeks <swe...@weeksconsulting.us>
>> wrote:
>>
>> Any expression language syntax has to be correct or the processor won’t
>> run. I’m not sure there is any way to work around that except to explicitly
>> check that the value you are trying to evaluate is valid. Is the attribute
>> “tradeDate” coming from the contents of a flow file or is it defined
>> somewhere else. Can you ensure it is a valid date in that format before
>> hand?
>>
>>
>>
>> Thanks
>>
>> Shawn Weeks
>>
>>
>>
>> *From:* Juan Pablo Gardella <gardellajuanpa...@gmail.com>
>>
>> *Sent:* Thursday, October 18, 2018 10:13 AM
>>
>>
>> *To:* users@nifi.apache.org
>> *Subject:* Re: [EXT] ReplaceText cannot consume messages if Regex does
>> not match
>>
>>
>>
>> Hi, the error is not in the processor itself. It's in the expression used
>> against flowfile attributes. For example inside the text, I have:
>>
>>
>>
>>
>> <EventDate>${tradeDate:toDate('MM/dd/yyyy'):format("yyyy-MM-dd'T'00:00:00.0000000")}</EventDate>
>>
>> And that is the root issue. If it's unable to convert it, the flow cannot
>> be consumed. How can I evaluate attributes in a non-blocker way?
>>
>>
>>
>> Juan
>>
>>
>>
>> On Thu, 18 Oct 2018 at 12:07 Shawn Weeks <swe...@weeksconsulting.us>
>> wrote:
>>
>> Where is your expression? That’s not the entire configuration for that
>> processor.
>>
>>
>>
>> Thanks
>>
>> Shawn Weeks
>>
>>
>>
>> *From:* Juan Pablo Gardella <gardellajuanpa...@gmail.com>
>> *Sent:* Thursday, October 18, 2018 10:03 AM
>> *To:* users@nifi.apache.org
>> *Subject:* Re: [EXT] ReplaceText cannot consume messages if Regex does
>> not match
>>
>>
>>
>> Configuration:
>>
>> Replacement Strategy: Always replace
>>
>> EvaluationMode: Entire text
>>
>>
>>
>>
>>
>> On Thu, 18 Oct 2018 at 12:01 Juan Pablo Gardella <
>> gardellajuanpa...@gmail.com> wrote:
>>
>> Hortonworks nifi based on 1.5.0:
>>
>>
>>
>> Configuration:
>>
>> Thanks
>>
>>
>>
>> On Thu, 18 Oct 2018 at 11:56 Peter Wicks (pwicks) <pwi...@micron.com>
>> wrote:
>>
>> Hi Juan,
>>
>>
>>
>> What version of NiFi are you running on?
>>
>> What mode are you running ReplaceText in, all text or line by line?
>>
>> Other settings that might be important? What’s your RegEx look like (if
>> your able to share).
>>
>>
>>
>> --Peter
>>
>>
>>
>>
>>
>> *From:* Juan Pablo Gardella [mailto:gardellajuanpa...@gmail.com]
>> *Sent:* Thursday, October 18, 2018 8:53 AM
>> *To:* users@nifi.apache.org
>> *Subject:* [EXT] ReplaceText cannot consume messages if Regex does not
>> match
>>
>>
>>
>> Hi all,
>>
>>
>>
>> I'm seeing that ReplaceText is not able to consume messages that does not
>> match regex. It keeps all the messages in the input queue instead of
>> sending them to failure relationship. Is this the intended behavior or I
>> have to file a ticket in order to be fixed? In that way, the processor is
>> not able to process bad messages and converts in the bottleneck of a flow
>>
>>
>>
>> Juan
>>
>>
