Surely the caching should make this effectively an in memory lookup. Does the stellar enrichment function not use the same clientside caching as the Hbase bolt?
Simon > On 19 Jun 2017, at 06:21, Casey Stella <ceste...@gmail.com> wrote: > > In order to do that, the easiest thing to do is to create a stellar function > to load and do in-memory lookups. > >> On Sun, Jun 18, 2017 at 11:48 PM, Ali Nazemian <alinazem...@gmail.com> wrote: >> Hi all, >> >> We are using Metron HBase enrichment for a few use cases, but we have >> noticed the achievable throughput is not very great. I was wondering whether >> there is a way to load the external enrichment data in-memory and use it >> with normal Stellar enrichments. In our use cases, the number of rows in the >> external enrichments that we are dealing with is less than a 100k and it is >> a static list, so it is feasible to load them in-memory and use that for the >> enrichment. However, I am not sure how that would be achievable from the >> Metron capabilities. >> >> Regards, >> Ali >
