Exactly, I know it is not an issue in the replace text code, but it happens
inside it. If we are using the ReplaceText in multiple places, it increases
the flow design complexity. We need to evaluate all expressions before
sending to the processor, to be sure will not fail in ReplaceText
processor.
Notice it's impossible if you have to process content dynamically. I would
happy to file a ticket and the patch as I mention.
Juan
On Thu, 18 Oct 2018 at 13:18 Shawn Weeks <swe...@weeksconsulting.us> wrote:
> I understand the issue now, I’m not sure that a failure of ReplaceText is
> the best place to catch this though. The reason I’m not sure it’s the best
> place is what happens if there are multiple failures because you had
> multiple expressions, just having them all routed to the same failure
> wouldn’t help you make decisions on what to do with a single attribute.
> Perhaps a better solution would be to use a RouteOnAttribute to check if
> the attributes match a certain pattern before sending them to ReplaceText.
> A possible expression could be
> “${actualSettlementDate:matches('[0-9]{2}/[0-9]{2}/[0-9]{4}')}” however
> that would not catch things that look like dates but aren’t valid.
>
>
>
> Thanks
>
> Shawn Weeks
>
>
>
> *From:* Juan Pablo Gardella <gardellajuanpa...@gmail.com>
>
> *Sent:* Thursday, October 18, 2018 11:03 AM
>
>
> *To:* users@nifi.apache.org
> *Subject:* Re: [EXT] ReplaceText cannot consume messages if Regex does
> not match
>
>
>
> At *search value*:(?s)(^.*$)
>
>
>
> At *Replacement value*:
>
>
> *<?xml version="1.0" encoding="UTF-8"?>
> <SettleDate>${actualSettlementDate:toDate('MM/dd/yyyy'):format("yyyy-MM-dd'T'00:00:00.0000000")}</SettleDate>*
>
>
>
> The actualSettlementDate is a flowfile attribute. The problem is the
> replacement value is evaluated inside the processor and the *toDate *method
> fails.
>
>
>
> Hope it's clear now.
>
>
>
>
>
> On Thu, 18 Oct 2018 at 12:51 Shawn Weeks <swe...@weeksconsulting.us>
> wrote:
>
> I’m still trying to understand your actual issue, can your provide a
> screenshot of the ReplaceText config like the attached, I need to see
> exactly where you’re putting the expression. A template would also be
> really helpful.
>
>
>
> Thanks
>
> Shawn Weeks
>
>
>
> *From:* Juan Pablo Gardella <gardellajuanpa...@gmail.com>
>
> *Sent:* Thursday, October 18, 2018 10:45 AM
>
>
> *To:* users@nifi.apache.org
> *Subject:* Re: [EXT] ReplaceText cannot consume messages if Regex does
> not match
>
>
>
> At ReplaceText
> <https://raw.githubusercontent.com/apache/nifi/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceText.java>processor
> we have:
>
>
>
> [image: image.png]
>
> As you can see, only if *StackOverflowError *is raised during the
> evaluation, the flowfile is send to failure relationship. I would like to
> update the code to use Exception or NifiExpressionFailedException (if it
> exits).
>
>
>
> Juan
>
>
>
> On Thu, 18 Oct 2018 at 12:33 Shawn Weeks <swe...@weeksconsulting.us>
> wrote:
>
> What processor are you defining your expression in? I also may be
> misunderstanding the problem because I don’t see any regular expressions
> anywhere. Can you create a sample workflow showing your issue so I can take
> a look at it.
>
>
>
> Thanks
>
> Shawn Weeks
>
>
>
> *From:* Juan Pablo Gardella <gardellajuanpa...@gmail.com>
> *Sent:* Thursday, October 18, 2018 10:27 AM
> *To:* users@nifi.apache.org
> *Subject:* Re: [EXT] ReplaceText cannot consume messages if Regex does
> not match
>
>
>
> No, it's not a valid date. I would like if it an error happens, I would
> like to throw the flowfile to failure and continue.
>
>
>
> On Thu, 18 Oct 2018 at 12:19 Shawn Weeks <swe...@weeksconsulting.us>
> wrote:
>
> Any expression language syntax has to be correct or the processor won’t
> run. I’m not sure there is any way to work around that except to explicitly
> check that the value you are trying to evaluate is valid. Is the attribute
> “tradeDate” coming from the contents of a flow file or is it defined
> somewhere else. Can you ensure it is a valid date in that format before
> hand?
>
>
>
> Thanks
>
> Shawn Weeks
>
>
>
> *From:* Juan Pablo Gardella <gardellajuanpa...@gmail.com>
>
> *Sent:* Thursday, October 18, 2018 10:13 AM
>
>
> *To:* users@nifi.apache.org
> *Subject:* Re: [EXT] ReplaceText cannot consume messages if Regex does
> not match
>
>
>
> Hi, the error is not in the processor itself. It's in the expression used
> against flowfile attributes. For example inside the text, I have:
>
>
>
>
> <EventDate>${tradeDate:toDate('MM/dd/yyyy'):format("yyyy-MM-dd'T'00:00:00.0000000")}</EventDate>
>
> And that is the root issue. If it's unable to convert it, the flow cannot
> be consumed. How can I evaluate attributes in a non-blocker way?
>
>
>
> Juan
>
>
>
> On Thu, 18 Oct 2018 at 12:07 Shawn Weeks <swe...@weeksconsulting.us>
> wrote:
>
> Where is your expression? That’s not the entire configuration for that
> processor.
>
>
>
> Thanks
>
> Shawn Weeks
>
>
>
> *From:* Juan Pablo Gardella <gardellajuanpa...@gmail.com>
> *Sent:* Thursday, October 18, 2018 10:03 AM
> *To:* users@nifi.apache.org
> *Subject:* Re: [EXT] ReplaceText cannot consume messages if Regex does
> not match
>
>
>
> Configuration:
>
> Replacement Strategy: Always replace
>
> EvaluationMode: Entire text
>
>
>
>
>
> On Thu, 18 Oct 2018 at 12:01 Juan Pablo Gardella <
> gardellajuanpa...@gmail.com> wrote:
>
> Hortonworks nifi based on 1.5.0:
>
>
>
> Configuration:
>
> Thanks
>
>
>
> On Thu, 18 Oct 2018 at 11:56 Peter Wicks (pwicks) <pwi...@micron.com>
> wrote:
>
> Hi Juan,
>
>
>
> What version of NiFi are you running on?
>
> What mode are you running ReplaceText in, all text or line by line?
>
> Other settings that might be important? What’s your RegEx look like (if
> your able to share).
>
>
>
> --Peter
>
>
>
>
>
> *From:* Juan Pablo Gardella [mailto:gardellajuanpa...@gmail.com]
> *Sent:* Thursday, October 18, 2018 8:53 AM
> *To:* users@nifi.apache.org
> *Subject:* [EXT] ReplaceText cannot consume messages if Regex does not
> match
>
>
>
> Hi all,
>
>
>
> I'm seeing that ReplaceText is not able to consume messages that does not
> match regex. It keeps all the messages in the input queue instead of
> sending them to failure relationship. Is this the intended behavior or I
> have to file a ticket in order to be fixed? In that way, the processor is
> not able to process bad messages and converts in the bottleneck of a flow
>
>
>
> Juan
>
>
