I understand the issue now, I’m not sure that a failure of ReplaceText is the best place to catch this though. The reason I’m not sure it’s the best place is what happens if there are multiple failures because you had multiple expressions, just having them all routed to the same failure wouldn’t help you make decisions on what to do with a single attribute. Perhaps a better solution would be to use a RouteOnAttribute to check if the attributes match a certain pattern before sending them to ReplaceText. A possible expression could be “${actualSettlementDate:matches('[0-9]{2}/[0-9]{2}/[0-9]{4}')}” however that would not catch things that look like dates but aren’t valid.
Thanks Shawn Weeks From: Juan Pablo Gardella <gardellajuanpa...@gmail.com> Sent: Thursday, October 18, 2018 11:03 AM To: users@nifi.apache.org Subject: Re: [EXT] ReplaceText cannot consume messages if Regex does not match At search value:(?s)(^.*$) At Replacement value: <?xml version="1.0" encoding="UTF-8"?> <SettleDate>${actualSettlementDate:toDate('MM/dd/yyyy'):format("yyyy-MM-dd'T'00:00:00.0000000")}</SettleDate> The actualSettlementDate is a flowfile attribute. The problem is the replacement value is evaluated inside the processor and the toDate method fails. Hope it's clear now. On Thu, 18 Oct 2018 at 12:51 Shawn Weeks <swe...@weeksconsulting.us<mailto:swe...@weeksconsulting.us>> wrote: I’m still trying to understand your actual issue, can your provide a screenshot of the ReplaceText config like the attached, I need to see exactly where you’re putting the expression. A template would also be really helpful. Thanks Shawn Weeks From: Juan Pablo Gardella <gardellajuanpa...@gmail.com<mailto:gardellajuanpa...@gmail.com>> Sent: Thursday, October 18, 2018 10:45 AM To: users@nifi.apache.org<mailto:users@nifi.apache.org> Subject: Re: [EXT] ReplaceText cannot consume messages if Regex does not match At ReplaceText <https://raw.githubusercontent.com/apache/nifi/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceText.java> processor we have: [image.png] As you can see, only if StackOverflowError is raised during the evaluation, the flowfile is send to failure relationship. I would like to update the code to use Exception or NifiExpressionFailedException (if it exits). Juan On Thu, 18 Oct 2018 at 12:33 Shawn Weeks <swe...@weeksconsulting.us<mailto:swe...@weeksconsulting.us>> wrote: What processor are you defining your expression in? I also may be misunderstanding the problem because I don’t see any regular expressions anywhere. Can you create a sample workflow showing your issue so I can take a look at it. Thanks Shawn Weeks From: Juan Pablo Gardella <gardellajuanpa...@gmail.com<mailto:gardellajuanpa...@gmail.com>> Sent: Thursday, October 18, 2018 10:27 AM To: users@nifi.apache.org<mailto:users@nifi.apache.org> Subject: Re: [EXT] ReplaceText cannot consume messages if Regex does not match No, it's not a valid date. I would like if it an error happens, I would like to throw the flowfile to failure and continue. On Thu, 18 Oct 2018 at 12:19 Shawn Weeks <swe...@weeksconsulting.us<mailto:swe...@weeksconsulting.us>> wrote: Any expression language syntax has to be correct or the processor won’t run. I’m not sure there is any way to work around that except to explicitly check that the value you are trying to evaluate is valid. Is the attribute “tradeDate” coming from the contents of a flow file or is it defined somewhere else. Can you ensure it is a valid date in that format before hand? Thanks Shawn Weeks From: Juan Pablo Gardella <gardellajuanpa...@gmail.com<mailto:gardellajuanpa...@gmail.com>> Sent: Thursday, October 18, 2018 10:13 AM To: users@nifi.apache.org<mailto:users@nifi.apache.org> Subject: Re: [EXT] ReplaceText cannot consume messages if Regex does not match Hi, the error is not in the processor itself. It's in the expression used against flowfile attributes. For example inside the text, I have: <EventDate>${tradeDate:toDate('MM/dd/yyyy'):format("yyyy-MM-dd'T'00:00:00.0000000")}</EventDate> And that is the root issue. If it's unable to convert it, the flow cannot be consumed. How can I evaluate attributes in a non-blocker way? Juan On Thu, 18 Oct 2018 at 12:07 Shawn Weeks <swe...@weeksconsulting.us<mailto:swe...@weeksconsulting.us>> wrote: Where is your expression? That’s not the entire configuration for that processor. Thanks Shawn Weeks From: Juan Pablo Gardella <gardellajuanpa...@gmail.com<mailto:gardellajuanpa...@gmail.com>> Sent: Thursday, October 18, 2018 10:03 AM To: users@nifi.apache.org<mailto:users@nifi.apache.org> Subject: Re: [EXT] ReplaceText cannot consume messages if Regex does not match Configuration: Replacement Strategy: Always replace EvaluationMode: Entire text On Thu, 18 Oct 2018 at 12:01 Juan Pablo Gardella <gardellajuanpa...@gmail.com<mailto:gardellajuanpa...@gmail.com>> wrote: Hortonworks nifi based on 1.5.0: Configuration: Thanks On Thu, 18 Oct 2018 at 11:56 Peter Wicks (pwicks) <pwi...@micron.com<mailto:pwi...@micron.com>> wrote: Hi Juan, What version of NiFi are you running on? What mode are you running ReplaceText in, all text or line by line? Other settings that might be important? What’s your RegEx look like (if your able to share). --Peter From: Juan Pablo Gardella [mailto:gardellajuanpa...@gmail.com<mailto:gardellajuanpa...@gmail.com>] Sent: Thursday, October 18, 2018 8:53 AM To: users@nifi.apache.org<mailto:users@nifi.apache.org> Subject: [EXT] ReplaceText cannot consume messages if Regex does not match Hi all, I'm seeing that ReplaceText is not able to consume messages that does not match regex. It keeps all the messages in the input queue instead of sending them to failure relationship. Is this the intended behavior or I have to file a ticket in order to be fixed? In that way, the processor is not able to process bad messages and converts in the bottleneck of a flow Juan