Re: Invite for Merton slack channel

[Marcus Persson]

Hi,

Please add me aswell.

mar...@marcuspe.se

Thanks alot!

On 2019/10/08 10:23:24, Sanket Sharma  wrote: 
> Hi,
> 
> Can you please add me to the slack channel?
> 
> Best regards,
> Sanket
> 
> From: Otto Fowler 
> Sent: Wednesday, August 21, 2019 11:16 PM
> To: Wan Nabe ; user@metron.apache.org 
> 
> Subject: Re: Invite for Merton slack channel
> 
> 
> Done, join the metron channel
> 
> 
> 
> 
> On August 21, 2019 at 00:54:08, Wan Nabe 
> (wanna...@ymail.com) wrote:
> 
> Hi,
> 
> Please add me to the channel.
> 
> 
> Thank you & Regards,
> Hans
> 
> On 14 Aug 2019, at 17:55, R K Sharma 
> mailto:rksu...@gmail.com>> wrote:
> 
> Hi,
>  Could you please add me to Metron Slack channel ?
> 
> Regards
> Rinkesh Sharma
> 
> On Tue, Aug 6, 2019 at 8:53 PM Otto Fowler 
> mailto:ottobackwa...@gmail.com>> wrote:
> 
> sure, give it a sec
> 
> 
> 
> 
> On August 6, 2019 at 10:09:36, Thiago Rahal Disposti 
> (thiago.ra...@kryptus.com) wrote:
> 
> 
> Can you please add me ?
> 
> thiago.ra...@kryptus.com
> 
> 
> Thanks.
> 
> Thiago Rahal
> 
> 
> On Thu, Jul 18, 2019 at 10:44 PM Otto Fowler 
> mailto:ottobackwa...@gmail.com>> wrote:
> 
> Both of you are all set, join the metron slack channel
> 
> 
> 
> 
> On July 18, 2019 at 20:15:33, Aman Diwakar 
> (aman.diwa...@gmail.com) wrote:
> 
> Me too please
> 
> On Thu, Jul 18, 2019, 12:32 PM Satish Abburi 
> mailto:satish.abb...@sstech.us>> wrote:
> 
> 
> 
> Can you please add me also. Thanks.
> 
> 
> 
> satish.abb...@sstech.us
> 
> 
> 
> From: "zeo...@gmail.com" 
> mailto:zeo...@gmail.com>>
> Reply-To: "user@metron.apache.org" 
> mailto:user@metron.apache.org>>
> Date: Tuesday, July 9, 2019 at 2:31 AM
> To: "user@metron.apache.org" 
> mailto:user@metron.apache.org>>
> Subject: Re: Invite for Merton slack channel
> 
> 
> 
> You got it.  Sent
> 
> Jon Zeolla
> 
> 
> 
> On Tue, Jul 9, 2019, 12:55 AM Rendi 7936 
> mailto:rendi.7...@gmail.com>> wrote:
> 
> Good Morning,
> 
> Hi there
> 
> Can i join Apache Metron Slack Channel ?
> My e-mail is rendi.7...@gmail.com
> 
> On 2019/07/08 13:29:42, "z...@gmail.com" wrote:
> > Done>
> >
> > - Jon Zeolla>
> > zeo...@gmail.com>
> >
> >
> > On Mon, Jul 8, 2019 at 9:18 AM Srikanth Nagarajan >
> > wrote:>
> >
> > > Hi>
> > >>
> > > I would appreciate an invite to the Metron slack channel .>
> > >>
> > > Thank you>
> > > Srikanth>
> > >>
> > > __>
> > > *Srikanth Nagarajan *>
> > > Principal>
> > > *Gandiva Networks Inc*>
> > > *732.690.1884 <732.690.1884>* Mobile>
> > > s...@gandivanetworks.com>
> > > www.gandivanetworks.com>
> > >>
> >
> 

Apache Metron production deployment

[marcus]

Hello,

How are you using Metron in a production environment?
I have checked around and my conclusion is that the Ambari-solution
should not be used in production environment and just for poc/testing.

I want to run Metron with Hadoop on CentOS 7 or 8, If you have other
recomendation I can change OS.


Thanks alot in advance!

Best Regards
Marcus

Re: Apache Metron production deployment

[Simon Elliston Ball]

Everyone I know of running metron at scale in production uses the ambari
based install method through a distribution, running on centos 6 on HDP
2.6.5 and for the new feature branch centos 7 on top of HDP 3.1.4.

Simon

On Tue, 29 Oct 2019 at 12:54,  wrote:

> Hello,
>
> How are you using Metron in a production environment?
> I have checked around and my conclusion is that the Ambari-solution
> should not be used in production environment and just for poc/testing.
>
> I want to run Metron with Hadoop on CentOS 7 or 8, If you have other
> recomendation I can change OS.
>
>
> Thanks alot in advance!
>
> Best Regards
> Marcus
>
-- 
--
simon elliston ball
@sireb

Re: Apache Metron production deployment

[Eric Jacksch]

We unfortunately gave up after trying several approaches to getting
Metron running in AWS. I'm disappointed -- I think Metron has huge
potential.

I suspect those who are using it have established development systems
and that there are some undocumented prerequisites. If anyone on the
Metron team has time, just try to deploy it in AWS using a freshly
spun up EC2 instance as your build/deploy machine and the issues will
rapidly become evident.

Regards,
Eric

On Tue, 29 Oct 2019 at 08:54,  wrote:
>
> Hello,
>
> How are you using Metron in a production environment?
> I have checked around and my conclusion is that the Ambari-solution
> should not be used in production environment and just for poc/testing.
>
> I want to run Metron with Hadoop on CentOS 7 or 8, If you have other
> recomendation I can change OS.
>
>
> Thanks alot in advance!
>
> Best Regards
> Marcus



-- 
Eric Jacksch, CPP, CISM, CISSP
e...@jacksch.com
Twitter: @EricJacksch
https://SecurityShelf.com

Re: Apache Metron production deployment

[Marcus Persson]

I got a bit unsure after reading on the Metron-github under metron-deployment 
"https://github.com/apache/metron/tree/master/metron-deployment"; because it 
states following "If you want to run a proof-of-concept to see how Apache 
Metron can benefit your organization, then this is the way to do it."

On 2019/10/29 13:07:33, Simon Elliston Ball  
wrote: 
> Everyone I know of running metron at scale in production uses the ambari
> based install method through a distribution, running on centos 6 on HDP
> 2.6.5 and for the new feature branch centos 7 on top of HDP 3.1.4.
> 
> Simon
> 
> On Tue, 29 Oct 2019 at 12:54,  wrote:
> 
> > Hello,
> >
> > How are you using Metron in a production environment?
> > I have checked around and my conclusion is that the Ambari-solution
> > should not be used in production environment and just for poc/testing.
> >
> > I want to run Metron with Hadoop on CentOS 7 or 8, If you have other
> > recomendation I can change OS.
> >
> >
> > Thanks alot in advance!
> >
> > Best Regards
> > Marcus
> >
> -- 
> --
> simon elliston ball
> @sireb
> 

Re: Apache Metron production deployment

[Marcus Persson]

then its just not me that have had problems... BUt my problem was best way of 
running it on CentOS..

On 2019/10/29 13:22:08, Eric Jacksch  wrote: 
> We unfortunately gave up after trying several approaches to getting
> Metron running in AWS. I'm disappointed -- I think Metron has huge
> potential.
> 
> I suspect those who are using it have established development systems
> and that there are some undocumented prerequisites. If anyone on the
> Metron team has time, just try to deploy it in AWS using a freshly
> spun up EC2 instance as your build/deploy machine and the issues will
> rapidly become evident.
> 
> Regards,
> Eric
> 
> On Tue, 29 Oct 2019 at 08:54,  wrote:
> >
> > Hello,
> >
> > How are you using Metron in a production environment?
> > I have checked around and my conclusion is that the Ambari-solution
> > should not be used in production environment and just for poc/testing.
> >
> > I want to run Metron with Hadoop on CentOS 7 or 8, If you have other
> > recomendation I can change OS.
> >
> >
> > Thanks alot in advance!
> >
> > Best Regards
> > Marcus
> 
> 
> 
> -- 
> Eric Jacksch, CPP, CISM, CISSP
> e...@jacksch.com
> Twitter: @EricJacksch
> https://SecurityShelf.com
> 

Re: Apache Metron production deployment

[Eric Jacksch]

I thought I may have just missed something, but one of my customer's
DevOps team worked on it for three days and couldn't get it going
either.

On Tue, 29 Oct 2019 at 09:32, Marcus Persson  wrote:
>
> then its just not me that have had problems... BUt my problem was best way of 
> running it on CentOS..
>
> On 2019/10/29 13:22:08, Eric Jacksch  wrote:
> > We unfortunately gave up after trying several approaches to getting
> > Metron running in AWS. I'm disappointed -- I think Metron has huge
> > potential.
> >
> > I suspect those who are using it have established development systems
> > and that there are some undocumented prerequisites. If anyone on the
> > Metron team has time, just try to deploy it in AWS using a freshly
> > spun up EC2 instance as your build/deploy machine and the issues will
> > rapidly become evident.
> >
> > Regards,
> > Eric
> >
> > On Tue, 29 Oct 2019 at 08:54,  wrote:
> > >
> > > Hello,
> > >
> > > How are you using Metron in a production environment?
> > > I have checked around and my conclusion is that the Ambari-solution
> > > should not be used in production environment and just for poc/testing.
> > >
> > > I want to run Metron with Hadoop on CentOS 7 or 8, If you have other
> > > recomendation I can change OS.
> > >
> > >
> > > Thanks alot in advance!
> > >
> > > Best Regards
> > > Marcus
> >
> >
> >
> > --
> > Eric Jacksch, CPP, CISM, CISSP
> > e...@jacksch.com
> > Twitter: @EricJacksch
> > https://SecurityShelf.com
> >



-- 
Eric Jacksch, CPP, CISM, CISSP
e...@jacksch.com
Twitter: @EricJacksch
https://SecurityShelf.com

Re: Apache Metron production deployment

[Nick Allen]

Have you opened a JIRA or sent an email describing what problems that
you've run into?

On Tue, Oct 29, 2019 at 9:35 AM Eric Jacksch  wrote:

> I thought I may have just missed something, but one of my customer's
> DevOps team worked on it for three days and couldn't get it going
> either.
>
> On Tue, 29 Oct 2019 at 09:32, Marcus Persson  wrote:
> >
> > then its just not me that have had problems... BUt my problem was best
> way of running it on CentOS..
> >
> > On 2019/10/29 13:22:08, Eric Jacksch  wrote:
> > > We unfortunately gave up after trying several approaches to getting
> > > Metron running in AWS. I'm disappointed -- I think Metron has huge
> > > potential.
> > >
> > > I suspect those who are using it have established development systems
> > > and that there are some undocumented prerequisites. If anyone on the
> > > Metron team has time, just try to deploy it in AWS using a freshly
> > > spun up EC2 instance as your build/deploy machine and the issues will
> > > rapidly become evident.
> > >
> > > Regards,
> > > Eric
> > >
> > > On Tue, 29 Oct 2019 at 08:54,  wrote:
> > > >
> > > > Hello,
> > > >
> > > > How are you using Metron in a production environment?
> > > > I have checked around and my conclusion is that the Ambari-solution
> > > > should not be used in production environment and just for
> poc/testing.
> > > >
> > > > I want to run Metron with Hadoop on CentOS 7 or 8, If you have other
> > > > recomendation I can change OS.
> > > >
> > > >
> > > > Thanks alot in advance!
> > > >
> > > > Best Regards
> > > > Marcus
> > >
> > >
> > >
> > > --
> > > Eric Jacksch, CPP, CISM, CISSP
> > > e...@jacksch.com
> > > Twitter: @EricJacksch
> > > https://SecurityShelf.com
> > >
>
>
>
> --
> Eric Jacksch, CPP, CISM, CISSP
> e...@jacksch.com
> Twitter: @EricJacksch
> https://SecurityShelf.com
>

Re: Threat Intel hailataxii

[Simon Elliston Ball]

Looks to me like your discovery server is not working properly, hence the 
failure message. This could be a temporary connectivity issue, but if it’s 
repeatable I would look into your opentaxii config. 

Simon 

> On 29 Oct 2019, at 13:23, Thiago Rahal Disposti  
> wrote:
> 
> 
> Anyone knows what's going on with the Hail a Taxii server?
> 
> I getting a service temporarily unavailable response for more than 3 weeks 
> now.
> 
> 
> 
> 
> 
> Thanks,
> Thiago Rahal

Re: Apache Metron production deployment

[Simon Elliston Ball]

I would recommend against using the AWS deploy method on the github. It’s not 
really that well maintained, and the Ambari method is definitely the preferred 
at present, but then I tend to use a distro to install, or full dev if it’s 
just for local testing.

Simon

> On 29 Oct 2019, at 13:35, Eric Jacksch  wrote:
> 
> I thought I may have just missed something, but one of my customer's
> DevOps team worked on it for three days and couldn't get it going
> either.
> 
>> On Tue, 29 Oct 2019 at 09:32, Marcus Persson  wrote:
>> 
>> then its just not me that have had problems... BUt my problem was best way 
>> of running it on CentOS..
>> 
>>> On 2019/10/29 13:22:08, Eric Jacksch  wrote:
>>> We unfortunately gave up after trying several approaches to getting
>>> Metron running in AWS. I'm disappointed -- I think Metron has huge
>>> potential.
>>> 
>>> I suspect those who are using it have established development systems
>>> and that there are some undocumented prerequisites. If anyone on the
>>> Metron team has time, just try to deploy it in AWS using a freshly
>>> spun up EC2 instance as your build/deploy machine and the issues will
>>> rapidly become evident.
>>> 
>>> Regards,
>>> Eric
>>> 
>>> On Tue, 29 Oct 2019 at 08:54,  wrote:
 
 Hello,
 
 How are you using Metron in a production environment?
 I have checked around and my conclusion is that the Ambari-solution
 should not be used in production environment and just for poc/testing.
 
 I want to run Metron with Hadoop on CentOS 7 or 8, If you have other
 recomendation I can change OS.
 
 
 Thanks alot in advance!
 
 Best Regards
 Marcus
>>> 
>>> 
>>> 
>>> --
>>> Eric Jacksch, CPP, CISM, CISSP
>>> e...@jacksch.com
>>> Twitter: @EricJacksch
>>> https://SecurityShelf.com
>>> 
> 
> 
> 
> -- 
> Eric Jacksch, CPP, CISM, CISSP
> e...@jacksch.com
> Twitter: @EricJacksch
> https://SecurityShelf.com

Re: Apache Metron production deployment

[Eric Jacksch]

I did earlier on this list, but wasn't able to make any progress.

Someone who knows the product well really should try as I suggested --
start from a clean slate in AWS, create an EC2 instance, and try to
deploy using the provided scripts. It's so badly broken that a
security guy with 25+ years of *nix experience AND an experienced
devops team who spend their life in AWS with K8S, Docker, etc,
couldn't get it built.

On Tue, 29 Oct 2019 at 09:38, Nick Allen  wrote:
>
> Have you opened a JIRA or sent an email describing what problems that you've 
> run into?
>
> On Tue, Oct 29, 2019 at 9:35 AM Eric Jacksch  wrote:
>>
>> I thought I may have just missed something, but one of my customer's
>> DevOps team worked on it for three days and couldn't get it going
>> either.
>>
>> On Tue, 29 Oct 2019 at 09:32, Marcus Persson  wrote:
>> >
>> > then its just not me that have had problems... BUt my problem was best way 
>> > of running it on CentOS..
>> >
>> > On 2019/10/29 13:22:08, Eric Jacksch  wrote:
>> > > We unfortunately gave up after trying several approaches to getting
>> > > Metron running in AWS. I'm disappointed -- I think Metron has huge
>> > > potential.
>> > >
>> > > I suspect those who are using it have established development systems
>> > > and that there are some undocumented prerequisites. If anyone on the
>> > > Metron team has time, just try to deploy it in AWS using a freshly
>> > > spun up EC2 instance as your build/deploy machine and the issues will
>> > > rapidly become evident.
>> > >
>> > > Regards,
>> > > Eric
>> > >
>> > > On Tue, 29 Oct 2019 at 08:54,  wrote:
>> > > >
>> > > > Hello,
>> > > >
>> > > > How are you using Metron in a production environment?
>> > > > I have checked around and my conclusion is that the Ambari-solution
>> > > > should not be used in production environment and just for poc/testing.
>> > > >
>> > > > I want to run Metron with Hadoop on CentOS 7 or 8, If you have other
>> > > > recomendation I can change OS.
>> > > >
>> > > >
>> > > > Thanks alot in advance!
>> > > >
>> > > > Best Regards
>> > > > Marcus
>> > >
>> > >
>> > >
>> > > --
>> > > Eric Jacksch, CPP, CISM, CISSP
>> > > e...@jacksch.com
>> > > Twitter: @EricJacksch
>> > > https://SecurityShelf.com
>> > >
>>
>>
>>
>> --
>> Eric Jacksch, CPP, CISM, CISSP
>> e...@jacksch.com
>> Twitter: @EricJacksch
>> https://SecurityShelf.com



-- 
Eric Jacksch, CPP, CISM, CISSP
e...@jacksch.com
Twitter: @EricJacksch
https://SecurityShelf.com

Re: Apache Metron production deployment

[Prashant Bhalesain]

1. Use hortonworks-ansible script available on github to build the base HDP
cluster
2. Install ambari  metron pack and/or elastic pack.
3. Install metron services

You should have a working cluster. I would not recommend the same beyond
POC/demo though

On Tue, 29 Oct 2019 at 13:43, Eric Jacksch  wrote:

> I did earlier on this list, but wasn't able to make any progress.
>
> Someone who knows the product well really should try as I suggested --
> start from a clean slate in AWS, create an EC2 instance, and try to
> deploy using the provided scripts. It's so badly broken that a
> security guy with 25+ years of *nix experience AND an experienced
> devops team who spend their life in AWS with K8S, Docker, etc,
> couldn't get it built.
>
> On Tue, 29 Oct 2019 at 09:38, Nick Allen  wrote:
> >
> > Have you opened a JIRA or sent an email describing what problems that
> you've run into?
> >
> > On Tue, Oct 29, 2019 at 9:35 AM Eric Jacksch  wrote:
> >>
> >> I thought I may have just missed something, but one of my customer's
> >> DevOps team worked on it for three days and couldn't get it going
> >> either.
> >>
> >> On Tue, 29 Oct 2019 at 09:32, Marcus Persson 
> wrote:
> >> >
> >> > then its just not me that have had problems... BUt my problem was
> best way of running it on CentOS..
> >> >
> >> > On 2019/10/29 13:22:08, Eric Jacksch  wrote:
> >> > > We unfortunately gave up after trying several approaches to getting
> >> > > Metron running in AWS. I'm disappointed -- I think Metron has huge
> >> > > potential.
> >> > >
> >> > > I suspect those who are using it have established development
> systems
> >> > > and that there are some undocumented prerequisites. If anyone on the
> >> > > Metron team has time, just try to deploy it in AWS using a freshly
> >> > > spun up EC2 instance as your build/deploy machine and the issues
> will
> >> > > rapidly become evident.
> >> > >
> >> > > Regards,
> >> > > Eric
> >> > >
> >> > > On Tue, 29 Oct 2019 at 08:54,  wrote:
> >> > > >
> >> > > > Hello,
> >> > > >
> >> > > > How are you using Metron in a production environment?
> >> > > > I have checked around and my conclusion is that the
> Ambari-solution
> >> > > > should not be used in production environment and just for
> poc/testing.
> >> > > >
> >> > > > I want to run Metron with Hadoop on CentOS 7 or 8, If you have
> other
> >> > > > recomendation I can change OS.
> >> > > >
> >> > > >
> >> > > > Thanks alot in advance!
> >> > > >
> >> > > > Best Regards
> >> > > > Marcus
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Eric Jacksch, CPP, CISM, CISSP
> >> > > e...@jacksch.com
> >> > > Twitter: @EricJacksch
> >> > > https://SecurityShelf.com
> >> > >
> >>
> >>
> >>
> >> --
> >> Eric Jacksch, CPP, CISM, CISSP
> >> e...@jacksch.com
> >> Twitter: @EricJacksch
> >> https://SecurityShelf.com
>
>
>
> --
> Eric Jacksch, CPP, CISM, CISSP
> e...@jacksch.com
> Twitter: @EricJacksch
> https://SecurityShelf.com
>
-- 
Sent from Gmail Mobile

Re: Apache Metron production deployment

[Nick Allen]

To echo Simon's comments, you should NOT use the automated deployment
mechanism described at *metron-deployment/amazon-ec2/README.md*.  This has
not been maintained, deploys an unsecure cluster by default, and is not the
preferred installation method.  This installation path pre-dates the Ambari
MPack that provides a much simpler, universal installation mechanism.

The preferred installation path with AWS is to just spin-up your EC2 nodes,
install Ambari with Metron's MPack, and use Ambari to deploy Metron.  This
is the same installation path for bare metal, a private cloud, AWS, etc.

On Tue, Oct 29, 2019 at 9:43 AM Eric Jacksch  wrote:

> I did earlier on this list, but wasn't able to make any progress.
>
> Someone who knows the product well really should try as I suggested --
> start from a clean slate in AWS, create an EC2 instance, and try to
> deploy using the provided scripts. It's so badly broken that a
> security guy with 25+ years of *nix experience AND an experienced
> devops team who spend their life in AWS with K8S, Docker, etc,
> couldn't get it built.
>
> On Tue, 29 Oct 2019 at 09:38, Nick Allen  wrote:
> >
> > Have you opened a JIRA or sent an email describing what problems that
> you've run into?
> >
> > On Tue, Oct 29, 2019 at 9:35 AM Eric Jacksch  wrote:
> >>
> >> I thought I may have just missed something, but one of my customer's
> >> DevOps team worked on it for three days and couldn't get it going
> >> either.
> >>
> >> On Tue, 29 Oct 2019 at 09:32, Marcus Persson 
> wrote:
> >> >
> >> > then its just not me that have had problems... BUt my problem was
> best way of running it on CentOS..
> >> >
> >> > On 2019/10/29 13:22:08, Eric Jacksch  wrote:
> >> > > We unfortunately gave up after trying several approaches to getting
> >> > > Metron running in AWS. I'm disappointed -- I think Metron has huge
> >> > > potential.
> >> > >
> >> > > I suspect those who are using it have established development
> systems
> >> > > and that there are some undocumented prerequisites. If anyone on the
> >> > > Metron team has time, just try to deploy it in AWS using a freshly
> >> > > spun up EC2 instance as your build/deploy machine and the issues
> will
> >> > > rapidly become evident.
> >> > >
> >> > > Regards,
> >> > > Eric
> >> > >
> >> > > On Tue, 29 Oct 2019 at 08:54,  wrote:
> >> > > >
> >> > > > Hello,
> >> > > >
> >> > > > How are you using Metron in a production environment?
> >> > > > I have checked around and my conclusion is that the
> Ambari-solution
> >> > > > should not be used in production environment and just for
> poc/testing.
> >> > > >
> >> > > > I want to run Metron with Hadoop on CentOS 7 or 8, If you have
> other
> >> > > > recomendation I can change OS.
> >> > > >
> >> > > >
> >> > > > Thanks alot in advance!
> >> > > >
> >> > > > Best Regards
> >> > > > Marcus
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Eric Jacksch, CPP, CISM, CISSP
> >> > > e...@jacksch.com
> >> > > Twitter: @EricJacksch
> >> > > https://SecurityShelf.com
> >> > >
> >>
> >>
> >>
> >> --
> >> Eric Jacksch, CPP, CISM, CISSP
> >> e...@jacksch.com
> >> Twitter: @EricJacksch
> >> https://SecurityShelf.com
>
>
>
> --
> Eric Jacksch, CPP, CISM, CISSP
> e...@jacksch.com
> Twitter: @EricJacksch
> https://SecurityShelf.com
>

Re: Apache Metron production deployment

[Michael Miklavcic]

Also agreed on Nick's deployment comments. Deploying on AWS manually is
fairly trivial. The Ansible scripts have not been touched in months (years,
even). You could also install Hadoop services manually (Big Top, for
instance), but YMMV. Most of our testing to this point has been on Ambari,
however we're looking to move away from directly depending on it in the
near future.

On Tue, Oct 29, 2019 at 8:30 AM Nick Allen  wrote:

> To echo Simon's comments, you should NOT use the automated deployment
> mechanism described at *metron-deployment/amazon-ec2/README.md*.  This
> has not been maintained, deploys an unsecure cluster by default, and is not
> the preferred installation method.  This installation path pre-dates the
> Ambari MPack that provides a much simpler, universal installation mechanism.
>
> The preferred installation path with AWS is to just spin-up your EC2
> nodes, install Ambari with Metron's MPack, and use Ambari to deploy
> Metron.  This is the same installation path for bare metal, a private
> cloud, AWS, etc.
>
> On Tue, Oct 29, 2019 at 9:43 AM Eric Jacksch  wrote:
>
>> I did earlier on this list, but wasn't able to make any progress.
>>
>> Someone who knows the product well really should try as I suggested --
>> start from a clean slate in AWS, create an EC2 instance, and try to
>> deploy using the provided scripts. It's so badly broken that a
>> security guy with 25+ years of *nix experience AND an experienced
>> devops team who spend their life in AWS with K8S, Docker, etc,
>> couldn't get it built.
>>
>> On Tue, 29 Oct 2019 at 09:38, Nick Allen  wrote:
>> >
>> > Have you opened a JIRA or sent an email describing what problems that
>> you've run into?
>> >
>> > On Tue, Oct 29, 2019 at 9:35 AM Eric Jacksch  wrote:
>> >>
>> >> I thought I may have just missed something, but one of my customer's
>> >> DevOps team worked on it for three days and couldn't get it going
>> >> either.
>> >>
>> >> On Tue, 29 Oct 2019 at 09:32, Marcus Persson 
>> wrote:
>> >> >
>> >> > then its just not me that have had problems... BUt my problem was
>> best way of running it on CentOS..
>> >> >
>> >> > On 2019/10/29 13:22:08, Eric Jacksch  wrote:
>> >> > > We unfortunately gave up after trying several approaches to getting
>> >> > > Metron running in AWS. I'm disappointed -- I think Metron has huge
>> >> > > potential.
>> >> > >
>> >> > > I suspect those who are using it have established development
>> systems
>> >> > > and that there are some undocumented prerequisites. If anyone on
>> the
>> >> > > Metron team has time, just try to deploy it in AWS using a freshly
>> >> > > spun up EC2 instance as your build/deploy machine and the issues
>> will
>> >> > > rapidly become evident.
>> >> > >
>> >> > > Regards,
>> >> > > Eric
>> >> > >
>> >> > > On Tue, 29 Oct 2019 at 08:54,  wrote:
>> >> > > >
>> >> > > > Hello,
>> >> > > >
>> >> > > > How are you using Metron in a production environment?
>> >> > > > I have checked around and my conclusion is that the
>> Ambari-solution
>> >> > > > should not be used in production environment and just for
>> poc/testing.
>> >> > > >
>> >> > > > I want to run Metron with Hadoop on CentOS 7 or 8, If you have
>> other
>> >> > > > recomendation I can change OS.
>> >> > > >
>> >> > > >
>> >> > > > Thanks alot in advance!
>> >> > > >
>> >> > > > Best Regards
>> >> > > > Marcus
>> >> > >
>> >> > >
>> >> > >
>> >> > > --
>> >> > > Eric Jacksch, CPP, CISM, CISSP
>> >> > > e...@jacksch.com
>> >> > > Twitter: @EricJacksch
>> >> > > https://SecurityShelf.com
>> >> > >
>> >>
>> >>
>> >>
>> >> --
>> >> Eric Jacksch, CPP, CISM, CISSP
>> >> e...@jacksch.com
>> >> Twitter: @EricJacksch
>> >> https://SecurityShelf.com
>>
>>
>>
>> --
>> Eric Jacksch, CPP, CISM, CISSP
>> e...@jacksch.com
>> Twitter: @EricJacksch
>> https://SecurityShelf.com
>>
>

Re: Apache Metron production deployment

[Marcus Persso]

Thanks Michael!
So what is your recommendation if we should set it up in a production 
environment? We will run it on our own servers and not in the cloud.

Do you have a plan that you can when in near future your will move away from 
ambari direct dependancy? 1 month 6 months or something?

Br
Marcus

Den 29 okt. 2019 19:25, kI 19:25, Michael Miklavcic 
 skrev:
>Also agreed on Nick's deployment comments. Deploying on AWS manually is
>fairly trivial. The Ansible scripts have not been touched in months
>(years,
>even). You could also install Hadoop services manually (Big Top, for
>instance), but YMMV. Most of our testing to this point has been on
>Ambari,
>however we're looking to move away from directly depending on it in the
>near future.
>
>On Tue, Oct 29, 2019 at 8:30 AM Nick Allen  wrote:
>
>> To echo Simon's comments, you should NOT use the automated deployment
>> mechanism described at *metron-deployment/amazon-ec2/README.md*.
>This
>> has not been maintained, deploys an unsecure cluster by default, and
>is not
>> the preferred installation method.  This installation path pre-dates
>the
>> Ambari MPack that provides a much simpler, universal installation
>mechanism.
>>
>> The preferred installation path with AWS is to just spin-up your EC2
>> nodes, install Ambari with Metron's MPack, and use Ambari to deploy
>> Metron.  This is the same installation path for bare metal, a private
>> cloud, AWS, etc.
>>
>> On Tue, Oct 29, 2019 at 9:43 AM Eric Jacksch 
>wrote:
>>
>>> I did earlier on this list, but wasn't able to make any progress.
>>>
>>> Someone who knows the product well really should try as I suggested
>--
>>> start from a clean slate in AWS, create an EC2 instance, and try to
>>> deploy using the provided scripts. It's so badly broken that a
>>> security guy with 25+ years of *nix experience AND an experienced
>>> devops team who spend their life in AWS with K8S, Docker, etc,
>>> couldn't get it built.
>>>
>>> On Tue, 29 Oct 2019 at 09:38, Nick Allen  wrote:
>>> >
>>> > Have you opened a JIRA or sent an email describing what problems
>that
>>> you've run into?
>>> >
>>> > On Tue, Oct 29, 2019 at 9:35 AM Eric Jacksch 
>wrote:
>>> >>
>>> >> I thought I may have just missed something, but one of my
>customer's
>>> >> DevOps team worked on it for three days and couldn't get it going
>>> >> either.
>>> >>
>>> >> On Tue, 29 Oct 2019 at 09:32, Marcus Persson 
>>> wrote:
>>> >> >
>>> >> > then its just not me that have had problems... BUt my problem
>was
>>> best way of running it on CentOS..
>>> >> >
>>> >> > On 2019/10/29 13:22:08, Eric Jacksch  wrote:
>>> >> > > We unfortunately gave up after trying several approaches to
>getting
>>> >> > > Metron running in AWS. I'm disappointed -- I think Metron has
>huge
>>> >> > > potential.
>>> >> > >
>>> >> > > I suspect those who are using it have established development
>>> systems
>>> >> > > and that there are some undocumented prerequisites. If anyone
>on
>>> the
>>> >> > > Metron team has time, just try to deploy it in AWS using a
>freshly
>>> >> > > spun up EC2 instance as your build/deploy machine and the
>issues
>>> will
>>> >> > > rapidly become evident.
>>> >> > >
>>> >> > > Regards,
>>> >> > > Eric
>>> >> > >
>>> >> > > On Tue, 29 Oct 2019 at 08:54,  wrote:
>>> >> > > >
>>> >> > > > Hello,
>>> >> > > >
>>> >> > > > How are you using Metron in a production environment?
>>> >> > > > I have checked around and my conclusion is that the
>>> Ambari-solution
>>> >> > > > should not be used in production environment and just for
>>> poc/testing.
>>> >> > > >
>>> >> > > > I want to run Metron with Hadoop on CentOS 7 or 8, If you
>have
>>> other
>>> >> > > > recomendation I can change OS.
>>> >> > > >
>>> >> > > >
>>> >> > > > Thanks alot in advance!
>>> >> > > >
>>> >> > > > Best Regards
>>> >> > > > Marcus
>>> >> > >
>>> >> > >
>>> >> > >
>>> >> > > --
>>> >> > > Eric Jacksch, CPP, CISM, CISSP
>>> >> > > e...@jacksch.com
>>> >> > > Twitter: @EricJacksch
>>> >> > > https://SecurityShelf.com
>>> >> > >
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Eric Jacksch, CPP, CISM, CISSP
>>> >> e...@jacksch.com
>>> >> Twitter: @EricJacksch
>>> >> https://SecurityShelf.com
>>>
>>>
>>>
>>> --
>>> Eric Jacksch, CPP, CISM, CISSP
>>> e...@jacksch.com
>>> Twitter: @EricJacksch
>>> https://SecurityShelf.com
>>>
>>

Re: Apache Metron production deployment

[Michael Miklavcic]

No specific plans set in motion yet - we'd need a solid alternative
developed first, which does not yet exist. Ideally, some of these
peripheral concerns (like install) could be pluggable. That way, vendors
can easily snap in their own solutions. That being said, I'd expect that
Ambari will still be available as in install option, but you might be
grabbing it from a specific vendor's OSS repository, or even as a
sub-project of Metron. You might expect to see a core install experience in
Apache similar to many other OSS projects where there is reference
documentation for platform install options, and then a fairly rigorous set
of docs for installing Metron manually.

On Tue, Oct 29, 2019 at 12:51 PM Marcus Persso  wrote:

> Thanks Michael!
> So what is your recommendation if we should set it up in a production
> environment? We will run it on our own servers and not in the cloud.
>
> Do you have a plan that you can when in near future your will move away
> from ambari direct dependancy? 1 month 6 months or something?
>
> Br
> Marcus
> Den 29 okt. 2019, kI 19:25, Michael Miklavcic 
> skrev:
>>
>> Also agreed on Nick's deployment comments. Deploying on AWS manually is
>> fairly trivial. The Ansible scripts have not been touched in months (years,
>> even). You could also install Hadoop services manually (Big Top, for
>> instance), but YMMV. Most of our testing to this point has been on Ambari,
>> however we're looking to move away from directly depending on it in the
>> near future.
>>
>> On Tue, Oct 29, 2019 at 8:30 AM Nick Allen < n...@nickallen.org> wrote:
>>
>>> To echo Simon's comments, you should NOT use the automated deployment
>>> mechanism described at *metron-deployment/amazon-ec2/README.md*.  This
>>> has not been maintained, deploys an unsecure cluster by default, and is not
>>> the preferred installation method.  This installation path pre-dates the
>>> Ambari MPack that provides a much simpler, universal installation
>>> mechanism.
>>>
>>> The preferred installation path with AWS is to just spin-up your EC2
>>> nodes, install Ambari with Metron's MPack, and use Ambari to deploy
>>> Metron.  This is the same installation path for bare metal, a private
>>> cloud, AWS, etc.
>>>
>>> On Tue, Oct 29, 2019 at 9:43 AM Eric Jacksch < e...@jacksch.com> wrote:
>>>
 I did earlier on this list, but wasn't able to make any progress.

 Someone who knows the product well really should try as I suggested --
 start from a clean slate in AWS, create an EC2 instance, and try to
 deploy using the provided scripts. It's so badly broken that a
 security guy with 25+ years of *nix experience AND an experienced
 devops team who spend their life in AWS with K8S, Docker, etc,
 couldn't get it built.

 On Tue, 29 Oct 2019 at 09:38, Nick Allen < n...@nickallen.org> wrote:
 >
 > Have you opened a JIRA or sent an email describing what problems that
 you've run into?
 >
 > On Tue, Oct 29, 2019 at 9:35 AM Eric Jacksch < e...@jacksch.com>
 wrote:
 >>
 >> I thought I may have just missed something, but one of my customer's
 >> DevOps team worked on it for three days and couldn't get it going
 >> either.
 >>
 >> On Tue, 29 Oct 2019 at 09:32, Marcus Persson < mar...@marcuspe.se>
 wrote:
 >> >
 >> > then its just not me that have had problems... BUt my problem was
 best way of running it on CentOS..
 >> >
 >> > On 2019/10/29 13:22:08, Eric Jacksch < e...@jacksch.com> wrote:
 >> > > We unfortunately gave up after trying several approaches to
 getting
 >> > > Metron running in AWS. I'm disappointed -- I think Metron has
 huge
 >> > > potential.
 >> > >
 >> > > I suspect those who are using it have established development
 systems
 >> > > and that there are some undocumented prerequisites. If anyone on
 the
 >> > > Metron team has time, just try to deploy it in AWS using a
 freshly
 >> > > spun up EC2 instance as your build/deploy machine and the issues
 will
 >> > > rapidly become evident.
 >> > >
 >> > > Regards,
 >> > > Eric
 >> > >
 >> > > On Tue, 29 Oct 2019 at 08:54, < mar...@marcuspe.se> wrote:
 >> > > >
 >> > > > Hello,
 >> > > >
 >> > > > How are you using Metron in a production environment?
 >> > > > I have checked around and my conclusion is that the
 Ambari-solution
 >> > > > should not be used in production environment and just for
 poc/testing.
 >> > > >
 >> > > > I want to run Metron with Hadoop on CentOS 7 or 8, If you have
 other
 >> > > > recomendation I can change OS.
 >> > > >
 >> > > >
 >> > > > Thanks alot in advance!
 >> > > >
 >> > > > Best Regards
 >> > > > Marcus
 >> > >
 >> > >
 >> > >
 >> > > --
 >> > > Eric Jacksch, CPP, CISM, CISSP
 >> > > e...@jacksch.com
 >> > > Twitter: @EricJacksch
 >> > > https://SecurityShelf.c