Re: Lookup services for cleanup and other tasks

2019-04-24 Thread Mike Thomsen
Andy, Without going into specifics, we have been using this on a client project where the cleanup needs are "non-trivial" and often require writing custom lookup services to repair fields and things like that. So far, it seems to be working very well for this sort of advanced cleanup need. Before

Re: Lookup services for cleanup and other tasks

2019-01-21 Thread Andy LoPresto
Mike, This looks useful to me as an offering, but also very complex and almost abstracting an arbitrary DSL out. One of the patterns of NiFi has been the Unix tool model of each component doing a single activity and separating responsibilities between “black box” processors. This seems a bit

Lookup services for cleanup and other tasks

2019-01-21 Thread Mike Thomsen
A lot of the data we get is very dirty and in need of cleanup to such a point that UpdateRecord cannot handle it. So we use lookup services and LookupRecord to do things like normalize certain fields that are often wildly off from what we need. I wrote this processor over the weekend, and am