Thanks for your answer.
After playing around with your example, I think I'm starting to
understand how it works.
in the example, I've defined a property "foo" with a value of
"foovalue1" within the GenerateFlowFile processor. Inside the
LookupAttribute processor, I've changed the value of the property "bar"
into "${foo}.1". This will create correctly an attribute "bar" with the
lookup value of "barvalue1".
-What I'm still struggling with is to understand where the "dot one"
comes from. Is this from the column index needed for multi-column tables?
-How often will the CSV controller service refresh when a file is
changed? I think it would be handy if this could be done based on a
schedule.
You asked for more feedback regarding the csv controller service. Coming
from another platform (Splunk) where lookup tables are used extensively
for enriching and filtering, following features would be useful :-)
-Being able to select multiple columns (input attributes) at the same
time as a criteria for the lookup
-Being able to select one or more columns as output attributes
-Wildcard/Regex matching of values
-CaseInsensitive matching of values
-CIDR matching for IP networks
Probably larger CSV files should be indexed and cached in memory for
reasonable performance.
Thanks
Mika>
On 04/16/2017 09:06 PM, Joey Frazee wrote:
Mika,
The values for the dynamic properties are the names of the lookup keys
themselves. On top of that for the CSV lookup table they’re indexed for
multi-column tables, so for your example you want to add a dynamic property bar
with the value foo.1 (see [1] for a template). The reason ${foo} doesn’t work
is twofold: (1) there’s no foo attribute yet so ${foo} is empty and so there’s
no key, and (2) it’s not indexed. I suppose this indicates the documentation
isn’t sufficient or that the CSV controller service is just confusing and
should work differently.
You might be wondering why the property values are the lookup keys. The answer
is that it’s very useful for use cases where you’re using FlowFile attributes
or contents to specify what the keys are. For example, you could imagine a flow
that is:
[GetHTTP to fetch some user profile JSON from a WS] -> [EvaluateJsonPath to
extract a zipcode attribute, let’s say 10453] -> [LookupAttribute to enrich the
user profile by doing a lookup for the key ${zipcode}, which is 10453, and then
stuffing the result, NYC, in the attribute location]
For your example, the key isn’t being generated from an existing attribute so
you don’t need an Expression Language expression.
As for it being official, I’ve been working on NIFI-3404 [2, 3] which I’ll
probably PR this week and then we’ll see what people think. I actually wasn’t
sure I was going to include the CSV controller service so your email couldn’t
have come at a better time.
If you have any feedback on the behavior of the CSV controller service, I’m all
ears.
1. https://gist.github.com/jfrazee/a3b5558882b45228f768ef8dabb9ef54
2. https://issues.apache.org/jira/browse/NIFI-3404
3. https://github.com/jfrazee/nifi/tree/NIFI-3404
On Apr 16, 2017, at 1:50 AM, Mika Borner <[email protected]> wrote:
Hi
I'm struggling with the LookupAttribute Processor [1].
My flowfile has an attribute "foo" with the value "foovalue1". I want to create a new
attribute "bar", that matches foo's value (=>barvalue1). Therefore I have defined a csv lookup
table like this:
foo,bar
foovalue1,barvalue1
foovalue2,barvalue2
Therefore I'm creating a new dynamic property "bar" with the value "${foo}".
Unfortunately this does not work. Any hints?
I think this processor is very useful. Will it be integrated into an official
Nifi release anytime?
Thanks
Mika>
[1]
https://github.com/jfrazee/nifi-lookup-service/tree/file-based-lookup-service
