Hi all,
I need a layman example of how to deploy my python or R model on metron. Do
I have to develop the model separately and then deploy? I want to write the
model on the go using the snort data collected in hdfs. Maybe I am
over-expecting here but correct me if I am wrong.
This
http://metron.ap
on Elliston Ball (
> si...@simonellistonball.com) wrote:
>
> Yes. Consider a zeppelin notebook, or kibana dashboard for this.
>
> If you want to use these values for detection, consider building a profile
> based on the stats objects (see the profiler section of the documentation
> under analytics.
>
> Simon
>
> > On 6 Dec 2017, at 07:42, Syed Hammad Tahir wrote:
>
> >
> > Hi,
> >
> > Can I setup custom visualization to show lets say the peak netrwork
> usage traffic in a certain time?
> >
> > Regards.
>
>
>
Hi,
Can I setup custom visualization to show lets say the peak netrwork usage
traffic in a certain time?
Regards.
>
> 21.11.2017, 04:44, "Simon Elliston Ball" :
>
> Use MaaS:
> http://metron.apache.org/current-book/metron-analytics/
> metron-maas-service/index.html
>
>
> On 21 Nov 2017, at 11:43, Syed Hammad Tahir wrote:
>
> HI all,
>
> I have succesfully push
Hi guys,
Now that I am ready to work on my research problem and start working on
metron, I need to see a use case where a POC has been developed using
metron. Just need to get familiar with what we can potentially do on this
platform.
Regards.
HI all,
I have succesfully pushed real snort logs in to metron, now I need to apply
a machine learning or data science algorithm on it. How could I do that? I
want to code in python/R and then apply it in metron.
Regards.
ANd I dint load anything. It was supposed to be loaded during installation?
My installation is ambari based single node VM install on ubuntu host.
On Fri, Nov 17, 2017 at 3:55 PM, Syed Hammad Tahir
wrote:
> Here you go, the error part of the log is in the attachment.
>
> On Fri, Nov 17
a-management/index.html#
> GeoLite2_Loader
>
> Also, we can’t really see the error from screenshots, please send log
> entries.
>
> Simon
>
> On 17 Nov 2017, at 07:11, Syed Hammad Tahir wrote:
>
> Hi all, I am starting it again. Last one got a bit messy
>
> Ok,
Hi, I re deployed single node ambari based metron cluster and this time with
ansibleSkipTags= 'quick_dev' and now monit and sersor stubs are gone.
I run sudo service monit status and it says monit: unrecognized service
all the datanode
> service on it through Ambari.
>
>
> Regards,
>
> Aaron
> --
> *From:* Syed Hammad Tahir
> *Sent:* Thursday, November 16, 2017 5:47:49 AM
> *To:* user@metron.apache.org
> *Subject:* HDFS SIze
>
> HI,
>
> I ther
HI,
I there anyway I could alot more space to hdfs? I am redeploying single
node based ambari Metron cluster
Regards.
ok, Doing it.
On Mon, Nov 13, 2017 at 3:07 PM, zeo...@gmail.com wrote:
> Can you restart storm and give it another shot?
>
> Jon
>
> On Mon, Nov 13, 2017, 00:30 Syed Hammad Tahir
> wrote:
>
>> hi, This problem still persists guys .
>>
>> On Thu, Nov
hi, This problem still persists guys .
On Thu, Nov 9, 2017 at 11:13 PM, Syed Hammad Tahir
wrote:
> Any solution to these issues guys?
>
> On Thu, Nov 9, 2017 at 6:01 AM, Syed Hammad Tahir
> wrote:
>
>> I have attached the output of this dump
>>
>> /usr/metro
Any solution to these issues guys?
On Thu, Nov 9, 2017 at 6:01 AM, Syed Hammad Tahir
wrote:
> I have attached the output of this dump
>
> /usr/metron/0.4.1/bin/zk_load_configs.sh -z node1:2181 -m DUMP
>
>
>
> On Thu, Nov 9, 2017 at 12:06 AM, zeo...@gmail.com
> wrote:
&g
> On Wed, Nov 8, 2017 at 1:49 PM Syed Hammad Tahir
> wrote:
>
>> This is the script/command i used
>>
>> sudo cat snort.out |
>> /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh
>> --broker-list node1:6667 --topic snort
>>
>> On
Fri, Nov 3, 2017 at 12:19 PM Syed Hammad Tahir
> wrote:
>
>>
>> -- Forwarded message --
>> From: Syed Hammad Tahir
>> Date: Fri, Nov 3, 2017 at 5:07 PM
>> Subject: Re: Snort Logs
>> To: Otto Fowler
>>
>>
>> NVM, I hav
-- Forwarded message --
From: Syed Hammad Tahir
Date: Fri, Nov 3, 2017 at 5:07 PM
Subject: Re: Snort Logs
To: Otto Fowler
NVM, I have installed the elastic search head. Now where do I go in this to
find out why I cant see the snort logs in kibana dashboard, pushed to snort
And how do I install elasticsearch head on the vagrant VM?
How do I increase vagrant vm`s RAM. I have plenty of RAM to allocate to it.
[image: Inline image 1]
the logs I sent earlier. Look into the
> snort output options - may require you rerun snort, depending on your
> situation
>
> Jon
>
> On Mon, Oct 30, 2017, 06:53 Syed Hammad Tahir
> wrote:
>
>> Yes, I have converted them to text but those logs are simply captured
>>
eed text logs. Here's an example of some properly formatted logs -
> https://raw.githubusercontent.com/apache/metron/master/metron-
> deployment/roles/sensor-stubs/files/snort.out
>
> Jon
>
> On Mon, Oct 30, 2017, 01:34 Syed Hammad Tahir
> wrote:
>
>> I have found th
h I said:
>
> It should be in /usr/hdp/current/kafka-broker/bin/ or similar (from
> memory) on node1, assuming you are running full dev.
>
> Jon
>
>
> Jon
>
> On Fri, Oct 27, 2017 at 6:25 AM Syed Hammad Tahir
> wrote:
>
>> snort logs are in tcp dump
pt
>
> Jon
>
> On Fri, Oct 27, 2017, 02:15 Syed Hammad Tahir
> wrote:
>
>> Hello everyone,
>>
>> I have run snort independently on vagrant ssh and dumped the logs in
>> tcpdump format. Now I want to bring them to metron to play with them a bit.
>>
Hello everyone,
I have run snort independently on vagrant ssh and dumped the logs in
tcpdump format. Now I want to bring them to metron to play with them a bit.
Some of you already replied me with some solutions but thats lost in the
inbox somewhere and engulfed by the elasticsearhc issue that I h
d/ffmkiejjmecolpfloofpjologoblkegm
>>
>> Just plugin the address http://node1:9200/ and hit connect. I believe
>> our default status is "yellow." But that should be sufficient.
>>
>> I also second Simon's comments about reading up on Elasticsearch.
>
shutting down. Find the
> elastic processes, kill them, and start it up again.
>
>
> On 25 Oct 2017, at 13:15, Syed Hammad Tahir wrote:
>
> Just gave the command but its stuck here. I restart it earleir via ambari
> after changing heapsize. Now doing it via console
>
>
roblem is that it is not
> responding. I assume you have tried restarting elastic.
>
> On 25 Oct 2017, at 13:12, Syed Hammad Tahir wrote:
>
> It shows healthy
>
>
> But when I click in any quick link it shows this
>
>
>
> On Wed, Oct 25, 2017 at 5:07 PM, Simon
search, kafka, hadoop (hdfs in particular) and Linux. Our docs will assume
> you have at least some familiarity with those technologies.
>
> Simon
>
> On 25 Oct 2017, at 11:40, Syed Hammad Tahir wrote:
>
> Sorry, I didnt understand. Which baremetal guide should I look into? And I
com> wrote:
> Its a bug reported in metron,
>
> Look into barematel guide, Turn Red to green Cluster google it.
>
> On Oct 25, 2017 1:21 PM, "Syed Hammad Tahir" wrote:
>
>> SHould I do it from here? If yes then please guide me how to
>>
>> [image: I
gt;
>
> > On 25 Oct 2017, at 09:16, Syed Hammad Tahir
> wrote:
> >
> > When I try to open node1:5000 I see this.
> >
> >
> >
> > What could be the problem and its solution?
>
>
When I try to open node1:5000 I see this.
[image: Inline image 1]
What could be the problem and its solution?
connected snort with external source ?
> (Metron Snort ?)
>
> On Tue, Oct 24, 2017 at 8:27 PM, Nick Allen wrote:
>
>> Take a look at `kafka-console-producer.sh`, which is installed as part of
>> Kafka.
>>
>> On Tue, Oct 24, 2017 at 2:11 AM, Syed Hammad Tahir
>
Where do I find this file kafka-console-producer.sh?
On Tue, Oct 24, 2017 at 8:27 PM, Nick Allen wrote:
> Take a look at `kafka-console-producer.sh`, which is installed as part of
> Kafka.
>
> On Tue, Oct 24, 2017 at 2:11 AM, Syed Hammad Tahir
> wrote:
>
>> Ok, I hav
Ok, I have fixed everything on my own. Now that I have snort logs saved in
a file, I need to get them to metron. Can anyone help me on that?
On Mon, Oct 23, 2017 at 3:44 PM, Syed Hammad Tahir
wrote:
> yes nut I am a bit confused here. Let me ask them as well then.
>
> On Mon, Oct 23,
t; out to their community (https://snort.org/community), as they have more
> expertise in this area.
>
> Jon
>
> On Mon, Oct 23, 2017, 03:52 Syed Hammad Tahir
> wrote:
>
>> Hi guys,
>>
>> I tried to add another network interface in order to bridge it to LAN. I
Hi guys,
I tried to add another network interface in order to bridge it to LAN. I
tried to do it on virtualbox vm settings and when i did vagrant up after
that, there was no bridged interface. Can anyone help me on this?
On Sun, Oct 22, 2017 at 11:44 AM, Syed Hammad Tahir
wrote:
> Ok, thank
t;
> On 2017-10-20 00:32, Syed Hammad Tahir wrote:
>
>> I have installed the snort manually. Now I need help with :
>>
>> 1- Capturing the data of my lan and dumping it via snort :Snort cant see
>> the traffic outside vagrant vm, how do I make it see that traffic?
>
Help guys !!!
On Fri, Oct 20, 2017 at 12:32 PM, Syed Hammad Tahir
wrote:
> I have installed the snort manually. Now I need help with :
>
> 1- Capturing the data of my lan and dumping it via snort :Snort cant see
> the traffic outside vagrant vm, how do I make it see that traffic?
&
I have installed the snort manually. Now I need help with :
1- Capturing the data of my lan and dumping it via snort :Snort cant see
the traffic outside vagrant vm, how do I make it see that traffic?
2- Making a kafka topic to push those saved logs in metron for preprocessing
3- Applying a basic
I did all of that and then did vagrant up again. Snort is still not
installed. Will I have to vagrant destroy and then vagrant up again in
order for it to work?
On Thu, Oct 19, 2017 at 8:58 PM, Syed Hammad Tahir
wrote:
> would I need to vagrant destroy and then vagrant up again after this
//github.com/apache/metron/blob/master/metron-deployment/vagrant/full-dev-platform/Vagrantfile#L20>)
> to be exactly the following:
>
> ansibleSkipTags='quick_dev'
>
> Jon
>
> On Thu, Oct 19, 2017 at 7:59 AM Syed Hammad Tahir
> wrote:
>
>> Should
ng to do
> here is change a config value.
>
> Simon
>
> On 19 Oct 2017, at 11:46, Syed Hammad Tahir wrote:
>
> Ran it without -i swtich, gives this:
>
>
>
> On Thu, Oct 19, 2017 at 2:56 PM, zeo...@gmail.com
> wrote:
>
>> The sed command is falling.
how Ansible roles are defined, just start at
> the main.yml, then follow through each of the other files as they are
> included. It is pretty readable once you get use to the layout.
>
> On Tue, Oct 17, 2017 at 12:05 PM, Syed Hammad Tahir
> wrote:
>
>> Ok, Now I get
ttps://github.com/apache/metron/tree/master/metron-
> deployment/roles/sensor-stubs
>
>
>
> On Tue, Oct 17, 2017 at 10:16 AM, Syed Hammad Tahir
> wrote:
>
>> yes,, but when i do snort -v in vagrant ssh console it says snort isnt
>> installed where as it can be see
with it. :)
>
>
> On Tue, Oct 17, 2017 at 4:00 AM, Syed Hammad Tahir
> wrote:
>
>> And I am sorry about one confusion but isnt snort builtin into the metron
>> framework? If so then cant we access that snort and do the tasks you
>> mentioned earlier?
>>
>&
And I am sorry about one confusion but isnt snort builtin into the metron
framework? If so then cant we access that snort and do the tasks you
mentioned earlier?
On Tue, Oct 17, 2017 at 11:39 AM, Syed Hammad Tahir
wrote:
> Hi,
>
> Thanks for the support. Can it be performed both on d
called Snort where each
> message is a log line of the Snort file. Does that make sense?
>
> Thanks,
> James
>
>
> 11.10.2017, 23:08, "Syed Hammad Tahir" :
>
> You mean that I must start snort from terminal by doing snort -v and then
> push it to kafka topic
setup Snort on your own and push the output into a kafka
> topic (most likely using NiFi). From there on you can use the output of
> Snort in Metron.
>
>
> 10.10.2017, 00:48, "Syed Hammad Tahir" :
>
> Hi,
>
> Can I use snort in packet capture mode with met
Hi,
Can I use snort in packet capture mode with metron? By default it works in
IDS mode only.
Regards.
ver,
> I believe you can find something here:
> https://cwiki.apache.org/confluence/display/METRON/Metron+Architecture
>
> If not exact answer you will the enough idea to do R&D to achieve your
> goals.
>
> On 5 October 2017 at 13:43, Syed Hammad Tahir
> wrote:
>
&g
ou can use python kind of
> language to apply different modelling techniques on your data.
>
> Cheers,
> Umesh Kaushik
> 9620023458
>
> Sent from mobile device, kindly ignore the typographical errors.
>
> On 05-Oct-2017 10:55 AM, "Syed Hammad Tahir" wrote:
>
> 4 - The snort generated data would be indexed in Elasticsearch and/or
> stored on HDFS, depending on how you configured the system
>
> Thanks,
> James
>
>
> 04.10.2017, 03:23, "Syed Hammad Tahir" :
>
> Hi all,
>
> Now that I have installed metron (sing
Hi all,
Now that I have installed metron (single node installation on ubuntu
machine), I want to do some initial testing on snort data. I have a few
questions regarding this:
1- In how many configurations can I use snort with metron (for ex packet
capture in sniffing mode etc)?
2- How can I chan
Hi, After installing all the services, I put them on start since yesterday.
It took all the resources and I couldnt do anything. THe power outage
caused system, to restart so that process was interrupted. Now when I try
to start all services again I get this error:
[image: Inline image 1]
WHat services are necessary to run metron?
[image: Inline image 1]
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=68718548
Does this installaion guide work any more?
I am trying to to it on my 32gb ram ubuntu PC. Please let me know if there
are any changes to be made in this.
WHat do I do now?
[image: Inline image 1]
;
> Simon
>
>
> On 28 Sep 2017, at 11:38, Syed Hammad Tahir wrote:
>
> Ok, I guess it failed to install package: hadoop_2_5_3_0_37-yarn
> If I do it succesfully then should I do vagrant provision again or
> anything else?
>
> On Thu, Sep 28, 2017 at 3:32 PM, Simon Ell
y.
>
> On 28 Sep 2017, at 11:29, Syed Hammad Tahir wrote:
>
> My internet connection seems to be ok but to remove the doubt, is there
> any way to install the failed package manually? From where do I get the
> python script it ran before failure. The script which tries to downloa
ssor Speed: 3158.087 MHz
Processor Speed: 3114.001 MHz
Processor Speed: 2981.933 MHz
Processor Speed: 2458.770 MHz
Total Physical Processors: 4
Total cores: 16
Disk information:
/dev/sda1 268G 21G 234G 9% /
This CPU appears to support virtualization
On Wed, Sep 27, 2017 at 1:06 PM, Sye
yes, which one should I pursue in order to find the issue?
On Wed, Sep 27, 2017 at 12:50 PM, tkg_cangkul wrote:
> what alert that you see on ambari? there are 24 alert on your screenshot
> below.
>
>
> On 27/09/17 13:50, Syed Hammad Tahir wrote:
>
> Ambari server and a
Ambari server and agent both are running
On Wed, Sep 27, 2017 at 11:49 AM, tkg_cangkul wrote:
> Maybe you can check the ambari-agent service first from the terminal.
> If it stopped, just start it manually and then you can check the ambari
> again.
>
> On 27/09/17 13:16, Syed Ham
k what part the cluster deploy failed at.
>
>
> Regards,
>
> Aaron
>
>
> From: Syed Hammad Tahir
> Sent: Wednesday, 27 September, 06:28
> Subject: Installation Issues
> To: user@metron.apache.org
> Cc: Muhammad Umar Janjua
>
>
> Ok, Re-did every thing again and
our best bet is to requisition
> some server grade hardware from your university to test metron even if it's
> just the dev version.
>
>
>
> On Tue, Sep 26, 2017 at 9:50 AM, Syed Hammad Tahir
> wrote:
>
>> Hello everyone, any Idea how I can resolve this?
>>
>> [image: Inline image 1]
>>
>
>
Hello everyone, any Idea how I can resolve this?
[image: Inline image 1]
Metron testing). I don't
> recall the specifics of your system, are you making sure you have over 8GB
> *free* when you start spinning this up?
>
> Jon
>
> On Mon, Sep 25, 2017, 03:25 Syed Hammad Tahir
> wrote:
>
>> But this guide says that 8gb ram is requir
platform
> for Metron. I would strongly recommend going for something cloud based.
>
> I would also consider using the mpack method on an existing ambari, and
> avoiding the ansible method, that will be a little less brittle.
>
> Simon
>
>
> > On 25 Sep 2017, at 06:49
Any fix for this?
[image: Inline image 2]
Hello everyone,
I have been trying to install metron for over 2 weeks already and I havent
got any success so far.
I am doing it on my core i5 machine and have followed this guide so far:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=68718548
Please help me as its getting qui
Please help, I cant even find any folder named ambari in log files
[image: Inline image 1]
This is when I do vagrant provision
etc.)? Are they behind network
> firewalls or NAT, or are they exposed? Are they shared machines or one
> primary user each? If there are any internet exposed services, what are
> they?
>
> Jon
>
> On Wed, Sep 20, 2017, 06:50 Syed Hammad Tahir
> wrote:
>
>> Actu
d work to be done on the data. I would focus
> on setting up the sensors (custom IDS, snort) and then either gather
> metrics and scope Metron or just spin it up by default/with whatever you
> have and see how it works.
>
> Jon
>
> On Wed, Sep 20, 2017, 06:23 Syed Hammad Tahir
&
d is it, and (3) what are you planning to do with the data (profiling,
> MaaS, enrichments, etc.)?
>
> Jon
>
> On Wed, Sep 20, 2017, 04:04 Syed Hammad Tahir
> wrote:
>
>> Hello,
>>
>> What would be the system required in order to run metron and analyzy a
&g
Hello,
What would be the system required in order to run metron and analyzy a LAN
environment of almost 100 nodes using single node full development
depoloyment.
Regards.
Fri, Sep 15, 2017 at 10:01 AM, Syed Hammad Tahir
> wrote:
>
>> Can it be done on ubuntu or some other linux distribution or MAC OS is a
>> must?
>>
>>
>> On Fri, Sep 15, 2017 at 7:37 PM, Ryan Merriman
>> wrote:
>>
>>> Instructions for sta
rm#deploy-metron.
>
> On Fri, Sep 15, 2017 at 9:26 AM, Syed Hammad Tahir
> wrote:
>
>> yes but that link just states the prerequisites and not a guide, can I
>> find a full guide somewhere to install the full dev environment?
>>
>> On Fri, Sep 15, 2017 at 7:10 PM, Ry
n a desktop), our full dev environment might be a better
> approach for you and get you up and running faster:
> https://github.com/apache/metron/tree/master/metron-
> deployment/vagrant/full-dev-platform.
>
> Ryan
>
> On Fri, Sep 15, 2017 at 9:06 AM, Syed Hammad Tahir
> wrote:
etron+REST
>>
>> Thanks,
>> Venkatesh
>>
>> On Fri, Sep 15, 2017 at 3:27 PM, Khurram Ahmed
>> wrote:
>>
>>> My experience was extremely painful and I gave up shifting to a server
>>> machine with loads of RAM and processing power.
>&
up shifting to a server
> machine with loads of RAM and processing power.
>
> On Sep 15, 2017 2:51 PM, "Syed Hammad Tahir" wrote:
>
>> Even a basic VM install wont work? It says that 8Gb ram might work.
>>
>> https://cwiki.apache.org/confluence/display/METRO
u need server grade machines for Metron to work reliably.
>
> On Sep 15, 2017 2:41 PM, "Syed Hammad Tahir" wrote:
>
> My PC is core i5, 8GB RAM and a few hundred GBs of disk space. It doesnt
> have any OS as I will install it as per the recommendations in the guide.
>
ormation, which operating system your PC.
>
> Thanks,
> Venkatesh
>
> On Fri, Sep 15, 2017 at 2:57 PM, Syed Hammad Tahir
> wrote:
>
>> Hello,
>>
>> I need a guide to install metron on my PC from scratch.
>>
>> Regards
>>
>
>
Hello,
I need a guide to install metron on my PC from scratch.
Regards
Thankyou. I will start with the VM and will ask if I need any further
assistance.
On Thursday, September 7, 2017, zeo...@gmail.com wrote:
> When I say sensors I'm referring to tools that would feed into Metron like
> bro, yaf, snort, etc.
>
> Jon
>
> On Thu, Sep 7, 2017,
into Metron, and to know
> that you need to set up the sensors and get the network traffic first.
>
> Jon
>
> On Thu, Sep 7, 2017, 00:40 Syed Hammad Tahir > wrote:
>
>> Hi,
>>
>> What I wanted to do with this is the following:
>>
>> 1- Gather Net
our case you don't seem
> interested in PCAP, which means you _may_ be able to get away with
> something in EC2 or similar.
>
> Jon
>
> On Wed, Sep 6, 2017 at 6:41 AM Syed Hammad Tahir
> wrote:
>
>> Hello,
>>
>> Thankyou for answering my call to help
Jon
>
> On Wed, Sep 6, 2017, 01:59 Syed Hammad Tahir wrote:
>
>> Hello,
>>
>> I intend to use Apache Metron framework for the analysis of our local
>> area network. What is the best way to get started? Which installation is
>> most suitable for me as listed i
Hello,
I intend to use Apache Metron framework for the analysis of our local area
network. What is the best way to get started? Which installation is most
suitable for me as listed in the following link:
https://cwiki.apache.org/confluence/display/METRON/Installation
Kindly help me with this.
Re
87 matches
Mail list logo
