Re: Metron Not Reading From Kafka?

2018-09-19 Thread David McGinnis
Sirota wrote: > Was this flag set by default in the Metron install? Or is that something > that you enabled? > > > 14.09.2018, 13:52, "David McGinnis" : > > All, > > So I think we've found the root cause of this issue,, so I figured I'd > report back. It ap

Re: Metron Not Reading From Kafka?

2018-09-14 Thread David McGinnis
oing through the error and invalid topics. Also worth turning up the logging in storm for the parser topologies for a bit (via storm ui) on org.apache.metron. Simon On Fri, 17 Aug 2018 at 16:07, David McGinnis mailto:mcginn...@avalonconsult.com>> wrote: All, We have a Metron 0.4.3 installation

Re: Metron Not Reading From Kafka?

2018-09-11 Thread David McGinnis
It's a CSV parser made to parse firewall logs. On Tue, Sep 11, 2018 at 10:25 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > What parser is this? > > On Tue, Sep 11, 2018, 9:20 PM David McGinnis > wrote: > >> I think that makes sense. Just so you k

Re: Metron Not Reading From Kafka?

2018-09-11 Thread David McGinnis
and HDFS, respectively > > You've already validated data exists in your parser topic, but that's the > beginning of the chain. Validate data shows up in enrichments. If not, > there's an error in your parser. If it does, check topic indexing topic... > You get the idea. You're looking for the kink

Re: Metron Not Reading From Kafka?

2018-09-11 Thread David McGinnis
, but no where does it say we have to, so this is just confusing. On Tue, Sep 11, 2018 at 5:19 PM James Sirota wrote: > Did you mix and match versions of Metron? Looks like you are having a > classpath issue here > > > 11.09.2018, 12:01, "David McGinnis" : > > Sorr

Re: Metron Not Reading From Kafka?

2018-09-11 Thread David McGinnis
r.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 14 more On Fri, Aug 17, 2018 at 10:21 AM David McGinnis wrote: >

Re: Metron Not Reading From Kafka?

2018-08-17 Thread David McGinnis
topologies picking any of this up? Is there > anything in the ES error index? > > The other thing worth doing is upgrading 0.4.3 is a very very old version > and a lot of changes have come in since then. > > Simon > > On 17 August 2018 at 16:07, David McGinnis >

Metron Not Reading From Kafka?

2018-08-17 Thread David McGinnis
the jmxmetrics JAR file. Metron reports that the topic is found, and does not tell us that the topic is not emitting, so we suspect it sees the data in there. Do you all have any ideas on where we can look to determine the cause of this issue, or things to try? Thanks! -- David McGinnis Staff Hadoop

Re: metron / spot comparison

2018-04-23 Thread David McGinnis
of Metron vs Spot? I couldn't find an existing answer, but if one exists > I'd appreciate a pointer. > > Happy to provide more background on my use case and priorities if that > would help. > > Thanks, > Tom > -- David McGinnis Staff Hadoop Consultant | Avalon Consulting,

Re: Alerts Not Being Generated?

2018-03-01 Thread David McGinnis
ckup new index types in the alerts ui, there may be > issues with caching. > > Simon > > > On 1 Mar 2018, at 15:46, David McGinnis <mcginn...@avalonconsult.com> > wrote: > > All, > > I am following the instructions located here for creating a parser which > dete

Alerts Not Being Generated?

2018-03-01 Thread David McGinnis
familiar with the process of going from enrichments to alerts UI at this point. Thanks! -- David McGinnis Staff Hadoop Consultant | Avalon Consulting, LLC <http://www.avalonconsult.com/>M: (513) 439-0082 LinkedIn <http://www.linkedin.com/company/avalon-consulting-llc> | G

Re: Metron Indexing Service Failing Shortly After Start

2018-02-27 Thread David McGinnis
pip: > name: requests > version: 2.6.1 > > I would try installing that exact version with pip. Also, have you tried > installing that dependency? > > On Tue, Feb 27, 2018 at 1:14 PM, David McGinnis < > mcginn...@avalonconsult.com> wrote: > >>

Re: Metron Indexing Service Failing Shortly After Start

2018-02-27 Thread David McGinnis
? Using pip should > also install the necessary dependencies. Either way everything should > still work, you'll just have to manually start/stop indexing since you > won't ever be able to stop it. > > You are correct about HDP 2.6. > > On Tue, Feb 27, 2018 at 12

Re: Metron Indexing Service Failing Shortly After Start

2018-02-27 Thread David McGinnis
to support that version of HDP. On Tue, Feb 27, 2018 at 12:12 PM, David McGinnis < mcginn...@avalonconsult.com> wrote: > *Simon:* > > There are some errors from processing showing up in the indexing topic, > but no data is making it through without restarting Metron completely

Re: Metron Indexing Service Failing Shortly After Start

2018-02-27 Thread David McGinnis
in kafka? > > Anything in the logs for the indexing topology in Storm UI? > > Master isn’t always the best place to start, might be worth sticking to a > released build to kick the tyres. > > Simon > > On 27 Feb 2018, at 17:38, David McGinnis <mcginn...@avalo

Re: Logging Into Metron UIs On Kerberized Cluster?

2018-02-22 Thread David McGinnis
rest logs? Is this a production install? > > On Feb 21, 2018, at 5:53 PM, David McGinnis <mcginn...@avalonconsult.com> > wrote: > > OK, did this, added a simple username/password combo, as well as an entry > in authorities with ROLE_USER, and restarted the two UIs as well as the &

Re: Logging Into Metron UIs On Kerberized Cluster?

2018-02-21 Thread David McGinnis
metron-interface/metron-rest/ > src/main/resources/schema-mysql.sql. > > On Wed, Feb 21, 2018 at 5:15 PM, David McGinnis < > mcginn...@avalonconsult.com> wrote: > >> I had originally created a database named metron and pointed the JDBC >> connection string to that. I

Re: Logging Into Metron UIs On Kerberized Cluster?

2018-02-21 Thread David McGinnis
at 1:49 PM, Ryan Merriman <merrim...@gmail.com> wrote: > Did you create the metronrest database and permission it? The tables > should get created automatically when you start REST. > > On Wed, Feb 21, 2018 at 1:46 PM, David McGinnis < > mcginn...@avalonconsult.com>

Re: Logging Into Metron UIs On Kerberized Cluster?

2018-02-21 Thread David McGinnis
tication > > <https://github.com/apache/metron/tree/master/metron-interface/metron-rest#authentication> > > Let us know if you hit any issues. > > On Wed, Feb 21, 2018 at 9:44 AM, David McGinnis < > mcginn...@avalonconsult.com> wrote: > >> All, >> &g