Re: Define a function that can be used in Stellar

What we need to have is a function to produce entity row_key based on device format and tenant. We can put all the configurations in enrichment config, but it makes it very complex because we will have different logic for different customers. I was thinking of storing that logic somewhere and

Re: Define a function that can be used in Stellar

I think if we understand the use case, we may be able to think of a more general set of functionality for stellar to meet this and other cases. Will this configuration change? Do you need to track that change without reloading? How *much* is in the configuration? Do we want people putting

Re: Define a function that can be used in Stellar

We use a guava cache to cache the data for 24 hours. You can see how it's done here: https://github.com/apache/metron/blob/master/metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/stellar/ObjectGet.java We also do something like this in GEO_GET as well, but it's a bit

Re: Define a function that can be used in Stellar

I forgot we added OBJECT_GET. How does the caching work on that? Simn > On 2 Feb 2018, at 14:33, Nick Allen wrote: > > There are many functions that use the global configuration. For example, > GET_GEO in org.apache.metron.enrichment.stellar.GeoEnrichmentFunctions. >

Re: Define a function that can be used in Stellar

Is there any Stellar function already been implemented in Metron that has a config file associated with it? I am trying to get an idea of how it works. On 3 Feb. 2018 00:44, "Simon Elliston Ball" wrote: > Depends how you write the function class, but most likely,

Re: HBase enrichment vs Stellar enrichment for HBase look up

Great. In my opinion, the HBase one is a little bit confusing and too limited, so deprecating that might be a good idea. On 2 Feb. 2018 21:49, "Simon Elliston Ball" wrote: > There shouldn’t be. Both run through the same kind of bolt-side caching, > so you should be

Re: Define a function that can be used in Stellar

Does it mean every time the function gets called it will load the config, but if I use the global one it will only read it one time and it will be available in memory? On 2 Feb. 2018 21:53, "Simon Elliston Ball" wrote: > Shouldn’t be. The one this I would point out

Re: Apache Metron functions implementation

Hi Helder, It is very much possible, and very easy to create your own functions and models on top of Metron. There are two main ways in which you would do this, depending on the type of use case you’re looking at. Metron uses a language called Stellar as part of the enrichment stage (and

Apache Metron functions implementation

Hello, I am a student currently finishing my master degree and for my final work I am proposing to make a security analytics tool. I will want to make it on Apache Metron framework but I have some questions: - Is it possible to implement my own functions ? ( I will want to have clustering and

Re: Define a function that can be used in Stellar

Shouldn’t be. The one this I would point out though is that you don’t necessarily know which supervisor you will be running from, so pulling from HDFS would make sense. That said, the performance implications are probably not great. A good option here would be to have the config available in

Re: HBase enrichment vs Stellar enrichment for HBase look up

There shouldn’t be. Both run through the same kind of bolt-side caching, so you should be able to use the Stellar version, and in fact that’s the general direction the project is heading. We haven’t quite deprecated the plain HBase Bolt… but Stellar is definitely the preferred option. Simon