Re: Metron MaaS Issue

Sorry for the late reply. Try adding: import sys,os sys.path.append(os.getcwd()) On Fri, Dec 13, 2019 at 11:53 PM Hema malini wrote: > Hi, > > I am not sure whether i am facing an issue or it's a bug . I try to deploy > the sample Maas script in metron it works perfectly. In the model >

Re: MAP Data structure in Stellar to store key/value pairs

Hi Anil, Stefan is quite correct about initializing map objects in stellar. I would point out that, given you're using a multiset, you could also initialize your data structure with MULTISET_INIT() and interact with it via MULTISET_ADD(), similar to the geographic outliers use-case (we do this

Re: [ANNOUNCE] Apache Metron release 0.7.0

+1 to that!! On Mon, Dec 17, 2018 at 13:16 Michael Miklavcic wrote: > And a big thanks to Justin Leet for being our release manager. Great work > Justin! > > On Mon, Dec 17, 2018 at 10:07 AM Justin Leet wrote: > >> Hi all, >> >> I’m pleased to announce the release of Metron 0.7.0! There's been

Re: Indexing topology keep crashing

Two questions: 1. How much memory are you giving the workers for the indexing topology? 2. how large are the messages you're sending through? On Thu, Sep 13, 2018 at 2:00 PM Vets, Laurens wrote: > Hello list, > > I've installed OS updates on my Metron 0.4.2 yesterday, restarted all > nodes and

Good press for Metron!

https://www.darkreading.com/endpoint/oh-no-not-another-security-product/a/d-id/1332453

Re: CEF Parser not Indexing data via Nifi (SysLogs)

So, I would really love to see METRON-1453 go in, because I'd love to decouple syslog parsing (very common) from generic grok. On Fri, Jul 20, 2018 at 10:26 AM Otto Fowler wrote: > Metron does not have a generic Syslog Parser. > > Nifi has Syslog parsing ( either Records or standard Processor

Re: CEF Parser not Indexing data via Nifi (SysLogs)

I just want to pile in here and recommend taking a look at the parser chaining use-case, which is a walk-through of pulling in firewall logs over syslog using grok ( https://github.com/apache/metron/tree/master/use-cases/parser_chaining). Unfortunately this is in master and yet in a release, but

Re: [ANNOUNCE] Apache Metron release 0.5.0

Great job all! This was a big release with a lot of good stuff. I especially like the performance improvements :) Casey On Fri, Jun 8, 2018 at 8:54 AM Justin Leet wrote: > Hi All, > > I’m happy to announce the release of Metron 0.5.0! Everyone has put in a > lot of working into

Re: Parse Exception at SplitterBolt in Profiler Topology: 2018-04-27 10:57:16.575 o.a.m.p.b.ProfileSplitterBolt Thread-6-splitterBolt-executor[7 7] [ERROR] Unexpected failure: message='null'

That exception appears to me to be a problem in parsing the message coming into the profiler as opposed to having trouble parsing the profiler config. That list of integers are the raw characters in the message. It may be worthwhile to try to take the array of integers and try to turn them into

Re: Define a function that can be used in Stellar

We use a guava cache to cache the data for 24 hours. You can see how it's done here: https://github.com/apache/metron/blob/master/metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/stellar/ObjectGet.java We also do something like this in GEO_GET as well, but it's a bit

Re: Metron User Community Meeting Call

I can't wait! This is going to be really cool :) On Fri, Jan 26, 2018 at 5:25 PM, James Sirota wrote: > Yeah very interested in the presentation as well > > 26.01.2018, 15:15, "Simon Elliston Ball" : > > This is going to be a really exciting

Re: Stellar on another platform?

Yeah, what otto said :) I'd just add one thing, stellar really requires nothing more than: 1. existing inside of a JVM environment. We use it inside of storm and mapreduce, but it could be used inside of spark or whatever 2. Have a VariableResolver implementation which could map your

Re: [ALL] List Replies

+1, if it doesn't happen on the list, it doesn't happen in Apache. On Wed, Jan 17, 2018 at 6:55 AM, Otto Fowler wrote: > The goal of the user list is to foster the Apache Metron community by > allowing for common discussion of the uses and application of Apache >

Re: Motivations for using Apache Storm?

At the time, we chose storm because of a few reasons: - Metron inherited its codebase from OpenSOC, which chose Storm as it predated flink and spark streaming, the two other major contenders in the hadoop stack - Storm was battle tested at the time and, at least then, we had some

Re: Full Dev -> Heartbeat issues

I haven't seen that one. I spun one up from master on Friday and it seemed ok. Sorry, "works for me!" isn't super helpful, but it may be relevant since master is close to 0.4.2 :) On Mon, Jan 8, 2018 at 11:11 AM, Otto Fowler wrote: > I just started up full dev from

Re: [ANNOUNCE] Apache Metron Release 0.4.1

Fantastic! I'm really proud of this release and a great job was done by Matt and the community for getting this out! On Tue, Sep 19, 2017 at 1:24 PM, Frank Horsfall < frankhorsf...@cunet.carleton.ca> wrote: > Congrats guys! > > > > Frank > > > > > > *From:* zeo...@gmail.com

Re: Apache Metron and STIX

At the moment, we are dependent upon the Stix library from Mitre, which is Stix 1.x. The schemata that we support are https://github.com/STIXProject/java-stix/tree/v1.2.0.2/src/main/resources/schemas On Fri, Aug 18, 2017 at 1:26 PM, Ahmed Shah wrote: > Hello, > >

Re: Offset lag tool?

It's part of kafka, actually. You can find it documented at https://cwiki.apache.org/confluence/display/KAFKA/System+Tools#SystemTools-ConsumerOffsetChecker On Mon, Aug 14, 2017 at 11:32 AM, Laurens Vets wrote: > From the Performance-tuning-guide.md: "You will find the

Re: MaaS and Metron Architecture talks at DataWorks Summit SJ 2017

Ok, those talks are added. On Thu, Aug 3, 2017 at 3:44 PM, Casey Stella <ceste...@gmail.com> wrote: > Absolutely! > > On Thu, Aug 3, 2017 at 3:41 PM, Justin Leet <justinjl...@gmail.com> wrote: > >> Could we put these up on the wiki page for tech talks in the

Re: MaaS and Metron Architecture talks at DataWorks Summit SJ 2017

tent. > > https://cwiki.apache.org/confluence/display/METRON/Tech+Talks > > On Thu, Aug 3, 2017 at 10:32 AM, Casey Stella <ceste...@gmail.com> wrote: > >> The Videos of talks that Simon Ball and I gave at DataWorks Summit are >> now up and on youtube: >> >>

MaaS and Metron Architecture talks at DataWorks Summit SJ 2017

The Videos of talks that Simon Ball and I gave at DataWorks Summit are now up and on youtube: * Solving Cyber at Scale (business-level track) - https://www.youtube.com/watch?v=zVdRhwfum4Q * Model as a Service (technical track) - https://www.youtube.com/watch?v=LkrOKvyAc0s * Metron Architecture

Re: Possible Stellar bug

Ok, I think what you've found here is a bug in the REPL. I take it that what you're looking for is JOIN( ['a', 'b'], '\\') == 'a\b' right? That is a valid stellar expression, BUT because the REPL seems to be trying to interpret the \\ before it gets to stellar, it's borking something. When I

Re: FW: STIX extractor problem.

.main(RunJar.java:148) > > these exeption below was after my own changes on original code 0.4.0 , > sorry. > rgds > az > > From: Casey Stella [mailto:ceste...@gmail.com] > Sent: Wednesday, July 26, 2017 11:56 AM > To: user@metron.apache.org > Cc: u...@metron.incubato

Re: Treat Triage boost aggregation

3 >> https://issues.apache.org/jira/browse/METRON-685 >> >> >> Thanks >> >> >> >> On Thu, Jun 22, 2017 at 9:31 AM, Casey Stella <ceste...@gmail.com> wrote: >> >>> That's correct that it's the last step. Honestly, the threat tri

Re: Treat Triage boost aggregation

That's correct that it's the last step. Honestly, the threat triage functions were added prior to Stellar really being a thing. We should allow arbitrary stellar statements in there rather than a fixed approach, so it's pluggable. On Thu, Jun 22, 2017 at 3:50 AM, Ali Nazemian

Re: Metron in-memory enrichment

ry lookup. Does > the stellar enrichment function not use the same clientside caching as the > Hbase bolt? > > Simon > > On 19 Jun 2017, at 06:21, Casey Stella <ceste...@gmail.com> wrote: > > In order to do that, the easiest thing to do is to create a stellar >

Re: Metron in-memory enrichment

That said, I think it'd be really cool to have a set of stellar functions to interact with reference data stored in MapDB (http://www.mapdb.org/) which would get localized similar to the geo enrichment stellar functions for those small-data cases. On Mon, Jun 19, 2017 at 6:21 AM, Casey Stella

Re: Metron in-memory enrichment

In order to do that, the easiest thing to do is to create a stellar function to load and do in-memory lookups. On Sun, Jun 18, 2017 at 11:48 PM, Ali Nazemian wrote: > Hi all, > > We are using Metron HBase enrichment for a few use cases, but we have > noticed the

Re: Kafka spout error in the new HCP product

Yeah, I've seen the same issue. It appears that the storm-kafka-client in versions < 1.1 has significant throughput problems. We saw a 10x speedup in moving to the 1.1 version. There is a PR out for this currently: https://github.com/apache/metron/pull/584 Casey On Tue, May 16, 2017 at 4:26

No longer incubating, but newly hatched!

Hi All, Some of you know this already and some of you might not, but as of the last ASF board meeting we became a top level project with me serving as the Vice President of Apache Metron. The good people at the ASF press office scheduled some press early this morning. - NASDAQ GlobeNewswire

Re: High percentage of failed/timed out tuples after performance tuning!

Seeing anything in the storm logs for the workers? On Fri, Apr 21, 2017 at 07:41 Ali Nazemian wrote: > Hi all, > > After I tried to tune the Metron performance I have noticed the rate of > failure for the indexing/enrichment topologies are very high (about 95%). > However,