I need to check it in a long run. Perhaps my test scenario was not long
enough to hit the cache frequently.
On Tue, Jun 20, 2017 at 12:21 AM, Ryan Merriman wrote:
> "It would be nice if our enrichment cache mechanism reported hit/miss
> stats or something."
>
> +1 to that. We do this in the Joi
"It would be nice if our enrichment cache mechanism reported hit/miss stats
or something."
+1 to that. We do this in the JoinBolt already so it's an easy thing to
add.
On Mon, Jun 19, 2017 at 9:04 AM, Otto Fowler
wrote:
> Are you sure it is a cache miss problem? It could be that the HBASE
> e
Are you sure it is a cache miss problem? It could be that the HBASE
enrichment component could just be optimized.
It would be nice if our enrichment cache mechanism reported hit/miss stats
or something.
On June 19, 2017 at 09:58:25, Ali Nazemian (alinazem...@gmail.com) wrote:
I have already inc
I have already increased the cache value. However, clearly, the HBase
enrichment is a bottleneck for the entire enrichment topology. The average
latency for that part is about 30-40 ms, but the latency for other parts in
total is less than 0.5 ms.
On Mon, Jun 19, 2017 at 11:45 PM, Ryan Merriman w
We already front HBase enrichments with a cache (actually all enrichments
have a cache in front, see the GenericEnrichmentBolt class). You probably
just need to increase the cache size. This setting is in the enrichment
flux file at $METRON_HOME/flux/enrichment/remote.yaml. Look for the
"withMax
The stellar enrichments do cache results. It caches at the bolt level, so
it will associate an input message with computed output.
On Mon, Jun 19, 2017 at 6:28 AM, Simon Elliston Ball <
si...@simonellistonball.com> wrote:
> Surely the caching should make this effectively an in memory lookup. Doe
Surely the caching should make this effectively an in memory lookup. Does the
stellar enrichment function not use the same clientside caching as the Hbase
bolt?
Simon
> On 19 Jun 2017, at 06:21, Casey Stella wrote:
>
> In order to do that, the easiest thing to do is to create a stellar funct
That said, I think it'd be really cool to have a set of stellar functions
to interact with reference data stored in MapDB (http://www.mapdb.org/)
which would get localized similar to the geo enrichment stellar functions
for those small-data cases.
On Mon, Jun 19, 2017 at 6:21 AM, Casey Stella wro
In order to do that, the easiest thing to do is to create a stellar
function to load and do in-memory lookups.
On Sun, Jun 18, 2017 at 11:48 PM, Ali Nazemian
wrote:
> Hi all,
>
> We are using Metron HBase enrichment for a few use cases, but we have
> noticed the achievable throughput is not very
Hi all,
We are using Metron HBase enrichment for a few use cases, but we have
noticed the achievable throughput is not very great. I was wondering
whether there is a way to load the external enrichment data in-memory and
use it with normal Stellar enrichments. In our use cases, the number of
rows
10 matches
Mail list logo