Shouldn’t be. The one this I would point out though is that you don’t necessarily know which supervisor you will be running from, so pulling from HDFS would make sense. That said, the performance implications are probably not great. A good option here would be to have the config available in the global config for example and refer to that, since most instances of stellar apply global config to their context.
Simon > On 2 Feb 2018, at 07:14, Ali Nazemian <[email protected]> wrote: > > Will be any problem if the Stellar function we want to implement need to load > an external config file? > > Cheers, > Ali > > On Thu, Jan 18, 2018 at 4:58 PM, Ali Nazemian <[email protected] > <mailto:[email protected]>> wrote: > Thanks, All. > > Yes, Nick. It is highly related to our use case and the way that we are going > to enrich events with assets and vulnerability properties. It is not a > general case at all. > > Cheers, > Ali > > On Thu, Jan 18, 2018 at 5:43 AM, Matt Foley <[email protected] > <mailto:[email protected]>> wrote: > Besides the example code Simon mentioned at > https://github.com/apache/metron/tree/master/metron-stellar/stellar-3rd-party-example > > <https://github.com/apache/metron/tree/master/metron-stellar/stellar-3rd-party-example> > , > there is some documentation at > http://metron.apache.org/current-book/metron-stellar/stellar-common/3rdPartyStellar.html > > <http://metron.apache.org/current-book/metron-stellar/stellar-common/3rdPartyStellar.html> > > > From: Nick Allen <[email protected] <mailto:[email protected]>> > Reply-To: "[email protected] <mailto:[email protected]>" > <[email protected] <mailto:[email protected]>> > Date: Wednesday, January 17, 2018 at 4:46 AM > To: "[email protected] <mailto:[email protected]>" > <[email protected] <mailto:[email protected]>> > Subject: Re: Define a function that can be used in Stellar > > > > > > > > If something we have already does not fit the bill, I would recommend > creating that function in Java. Since you described it as "a bit complex" > and "the logic would be complicated" I don't see any value in defining > something like this in Stellar with named functions. > > > > Best > > > > > > > > > > On Wed, Jan 17, 2018 at 7:38 AM Simon Elliston Ball > <[email protected] <mailto:[email protected]>> wrote: > > Have you looked at the recent TLSH functions in Stellar? We already have that > for similarity preserving hashes. > > > > Simon > > > > > On 17 Jan 2018, at 12:35, Ali Nazemian <[email protected] > <mailto:[email protected]>> wrote: > > It is a bit complex. We want to create a function that accepts a list of > arguments for an asset and generate an asset identifier that can be used as a > row_key for the enrichment store. The logic would be complicated, though. We > may need to include some sort of similarity aware hash function as a part of > this custom function. > > > > On Wed, Jan 17, 2018 at 10:32 PM, Nick Allen <[email protected] > <mailto:[email protected]>> wrote: > > Ali - Can you describe the logic that you are trying to perform? That would > be useful as a use case to help drive a discussion around creating named > functions in Stellar. > > > > > > > > > > On Wed, Jan 17, 2018 at 6:29 AM Ali Nazemian <[email protected] > <mailto:[email protected]>> wrote: > > Thanks, Simon. We have already got a script to deal with classpath management > for the parsers. We should be able to use it for this extension as well. > > > > Yeah, I agree. It will be much easier to define functions on the fly and use > them afterwards. It could be defined as Lambda or custom function. > > > > Regards, > > Ali > > > > > > > > On Wed, Jan 17, 2018 at 9:42 PM, Simon Elliston Ball > <[email protected] <mailto:[email protected]>> wrote: > > https://github.com/apache/metron/tree/master/metron-stellar/stellar-3rd-party-example > > <https://github.com/apache/metron/tree/master/metron-stellar/stellar-3rd-party-example> > gives good details on how to add a stellar function. > > > > Stellar will pick up an annotated function on its class path, so to add > function there is no need to rebuild metron module, but you do need your > modules on the classpath, and, pending 777, to deal with things like class > path clash in your dependencies. > > > > Another idea worth discussion on the dev list is probably the notion of > defining stellar functions in stellar, which would be a much simpler solution > than custom java functions if you can already express you logic in stellar. > > > > Simon > > > > > > > On 17 Jan 2018, at 10:37, Ali Nazemian <[email protected] > <mailto:[email protected]>> wrote: > > > > Hi Simon, > > > > Yes, that is exactly what we are looking for. Is there any example regarding > adding a Stellar function in Java? Hopefully, we don't need to rebuild the > corresponding modules for this? > > > > Cheers, > > Ali > > > > On Wed, Jan 17, 2018 at 8:40 PM, Simon Elliston Ball > <[email protected] <mailto:[email protected]>> wrote: > > At present you can certainly create custom stellar functions in Java. I’m > guessing however that what you’re looking to do is create a kind of function > that combines a number of stellar functions to avoid repetition, or to ensure > consistency of certain parameters for example. Is that what you’re looking > for? Maybe some sort of syntax to create a named stellar function similar to > the way we create lambdas? > > Simon > > > > On 17 Jan 2018, at 07:25, Ali Nazemian <[email protected] > > <mailto:[email protected]>> wrote: > > > > Hi all, > > > > Is there any way that we can define a function that can be used rather than > > duplicating a logic multiple times? > > > > Cheers, > > Ali > > > > > > > -- > > A.Nazemian > > > > > > > > > -- > > A.Nazemian > > > > > > > -- > > A.Nazemian > > > > > -- > A.Nazemian > > > > -- > A.Nazemian
