Wow. Thanks.
This seems unnecessarily complicated using the “Advanced” feature, given it’s a standard boolean evaluation I’m trying to do. But I’m glad to hear there’s some way to do it. You know, assuming NiFi is written in Java (which I presume it is, if the custom processors are as well), when the day comes where you realize your Expression Language is simply too limited and quirky for anyone to stand any longer (that day is HERE for me, now!), I would humbly suggest you guys make use of Java 8 Nashorn. Then you could allow NiFi developers to embed arbitrary JavaScript in your little window, rather than your crazy, exists-no-where-else-on-earth E.L. Then one could simply create expressions representing “a and b are not empty or c is not empty” without the complexity of the E.L. itself, or forcing one to use the Advanced dialog, but using JavaScript, which is almost the world’s 2nd most used programming language at this point. Paul From: Matthew Clarke [mailto:[email protected]] Sent: Wednesday, March 23, 2016 12:26 PM To: Paul Nahay Cc: [email protected] Subject: Re: notEmpty Paul, The expression you have above ( ${allAttributes('x','y'):isEmpty():not():or(${z:isEmpty():not()})} ) will not logically result in the same outcome as ( ${allAttributes('x','y','z'):isEmpty():not()} ). I understand you realize this, but I just want to clarify it for the rest of the community reading this thread. Assuming x and/or y are empty and z is not empty, your expression will result in a true. Your understanding of how this would be parsed is correct. assume x is blank, y = 1, and z = 2. so the expression would evaluate to (attribute x is not empty (false) or attribute z is not empty (true) ) and (attribute y is not empty (true) or attribute z is not empty (true)) which breaks down further to (false or true = true) and (true or true = true) which finally results in (true and true = true). So as you can see as long as z is not empty, your expression will resolve to true. So you are correct that z would get evaluated for every attribute declared in the allAttributes() function. To avoid evaluating z multiple times in this particular use case, you would want to make use of the "Advanced" capability of the updateAttribute processor. When you open the configuration window for the UpdateAttribute processor, you will notice an "Advanced" button in the lower right corner. This well open the advanced configuration UI for the UpdateAttribute. Here you can essentially build if/then logic. In your case you would add two new "rules" and set the "FlowFile policy" to "use original" rule 1 would check to see if z is not empty. The condition must evaluate to "true" before the associated actions are applied. The action could set a new attribute with a value of true rule 2 would check to see if AllAttributes(x,y,a,b,c,etc) are not empty. if evaluated to true, the action would set the "same" attribute used in rule 1 to true. So essentially you are saying if this or if this set this. If both rules resolve to true and policy is set to use original, the last rule evaluated to true will set the attribute. The else part of the if/then/else logic here is achieved outside of the "Advanced" UI. There you could declare the "same" attribute again but set it to a value of false. The way this works.... If an attribute is declared both inside and outside the "Advanced" UI, the value set outside the "advanced" UI will only be applied to the FlowFile if it was not set by one of the rules. This process would result in evaluating z only once for each FlowFile. You can then use this newly created attribute later in your dataflow for additional logic checks, routing, etc... More information on the updateAttribute "Advanced" UI can be found here: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.attributes.UpdateAttribute/additionalDetails.html Matt On Wed, Mar 23, 2016 at 10:28 AM, Paul Nahay <[email protected]> wrote: OK, I am still confused about how to implement this boolean expression with the EL: (attributes x and y are both not empty) or (attribute z is not empty) If I do this: ${allAttributes('x','y'):isEmpty():not():or(${z:isEmpty():not()})} from my understanding of your documentation about the allAttributes() function, then the EL above is doing this (due to ":or()" coming right after ":not()"): (attribute x is not empty or attribute z is not empty) and (attribute y is not empty or attribute z is not empty) I realize logically this ends up being the same thing in the end, however, the "z is not empty" part gets evaluated TWICE, or as many times as there are parameters passed to allAttributes() (i.e., if we had instead "allAttributes('1','2','3','4','5'), then testing z for non-emptiness occurs five times, not once). Is there any way to actually express in the NiFi EL my boolean at the top of my message above? Paul -----Original Message----- From: Matthew Clarke Sent: Mar 22, 2016 8:17 PM To: Paul Nahay Subject: Re: notEmpty Paul, While i do agree that the addition of a notEmpty function would be nice, I was simply offering an alternative solution to your particular problem. The solution I provided does indeed work. I built a test flow to verify it. I created some test files with the following attributes on them: Property: Value: attr1 attr2 attr3 attr4 1 attr5 2 attr6 3 Those files were sent to a routeOnAttribute processor where I used that expression language statement to find route where all attributes were not empty. RouteOnAttribute: Property: Value: AllAttributesNotEmpty ${allAttributes('attr1','attr2','attr3'):isEmpty():not()} <--- I changed what attributes to look at in each test. I then varied what attribute I was looking at using the AllAttributes function. First I looked at attr1, attr2, and attr3 <--- files routed to unmatched as expected Second I looked at attr3, attr4, and attr5 <--- again all files routed to unmatched since one of the attributes (attr3) was empty Third I looked at attr4, attr5, and attr6 <--- All files routed to the AllAttributesNotEmpty relationship as expected. Perhaps I did not follow your use case correctly, but the above is what I believed you were looking for. Matt On Tue, Mar 22, 2016 at 6:39 PM, Paul Nahay <[email protected]> wrote: Thanks for responding, however... Your expression below evaluates to true if IT IS NOT THE CASE THAT ALL THREE ATTRIBUTES ARE EMPTY. But I need an expression that evalutes to true if IT IS THE CASE THAT ALL THREE ATTRIBUTES ARE NOT EMPTY. There are two entirely different things. Thus, if attr1 is set (thus is “not empty”), but the others are not (thus they “are empty”), your expression evaluates to true. But I need to ensure that NONE of the three “are empty”. I want an expression that in the case given above evaluates to false. Actually, I know how to make an expression that does what I need, however, it is more confusing than it needs to be. The simpler expression that one would desire to write REQUIRES that you offered a “notEmpty” function, thus would be expressible as: ${allAttributes('attr1','attr2','attr3'):notEmpty()} Again, this gives true only if all three attributes are not empty. The expression you suggest gives true if even only ONE of the attributes is not empty! The lack of “notEmpty()” seems a very grave omission on your part. And, you have no comment about the errors in your documentation for “isEmpty” and “allAttributes” that I told you? --Paul Nahay From: Matthew Clarke [mailto:[email protected]] Sent: Tuesday, March 22, 2016 11:07 AM To: [email protected]; Paul Nahay Subject: Re: notEmpty Paul, You can achieve what you are trying to do by using the not function. let assume the attributes you want to check to make sure they have a vlaue set are attr1, attr2, and attr3. The expression would be ${allAttributes('attr1','attr2','attr3'):isEmpty():not()} Thanks, Matt On Tue, Mar 22, 2016 at 9:46 AM, Paul Nahay <[email protected]> wrote: https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#isempty Why don't you have function "notEmpty"? This would be useful when combined with "allAttributes". I can't see any way, with your current set of functions, to determine that all of a set of attributes are not empty, using the "allAttributes" function. You have "isNull" and "notNull", why don't you have "notEmpty"? Paul Nahay [email protected]
