Re: PCAP on dashboard

[tkg_cangkul]

yes i've try to run other MR job there and it's succeed.
there is no problem when i run other MR job on my cluster.

On 08/06/17 02:13, Nick Allen wrote:
Are you able to run any MR jobs in your environment?  You could test 
that by using one of the Hadoop examples, if you don't normally run 
anything else.



$ find /usr/hdp/ -name "*hadoop*example*.jar"
/usr/hdp/2.5.3.0-37/hadoop-mapreduce/hadoop-mapreduce-examples.jar

/usr/hdp/2.5.3.0-37/hadoop-mapreduce/hadoop-mapreduce-examples-2.7.3.2.5.3.0-37.jar

$ yarn jar
/usr/hdp/2.5.3.0-37/hadoop-mapreduce/hadoop-mapreduce-examples.jar
pi 16 1000
Number of Maps  = 16
Samples per Map = 1000
Wrote input for Map #0
Wrote input for Map #1
...
Job Finished in 34.842 seconds
Estimated value of Pi is 3.1425


On Wed, Jun 7, 2017 at 1:00 PM, tkg_cangkul > wrote:


I'm using metron 0.3.0 and i run on my cluster machines.
this is the command that i run :

/usr/metron/0.3.0/bin/pcap_query.sh query -st "20160617" -df
"MMdd" -bop "/apps/metron/out" --query "ip_src_addr ==
'10.1.80.220' and ip_src_port == '6667' and ip_dst_addr ==
'10.1.80.221' and ip_dst_port == '42885' and protocol == '6'"

i also have try to using pcap_query.sh with fixed option but i
still got the message Could not initialize class
java.net.NetworkInterface

Best Regards,

On 07/06/17 20:19, Nick Allen wrote:
You need to provide more information about your environment. 
What version of Metron?  How are you running Metron (metal, VM,

cloud)?  What command did you run?

On Wed, Jun 7, 2017 at 3:27 AM, tkg_cangkul
mailto:yuza.ras...@gmail.com>> wrote:

hi Nick,

thx for your reply.
Ok, so i can use Zeppelin for this pcap dashboard.
is there any way to connected the kibana to Zeppelin? i mean
is there any way to select the pcap data to the kibana ui if
i embed it in zeppelin?

By the way, i've tried to use pcap_query tool. but i've found
some error message below :



any suggestion for that?

Best Regards,


On 29/05/17 20:49, Nick Allen wrote:

Right now it is stored in HDFS and then retrieved with the
pcap_query tool.  The pcap_query tool can also be embedded
in a Zeppelin Notebook.  Of course with this data is in
HDFS, you can integrate with it using your tool of choice.

If you have use cases in mind, please feel free to share.

On Fri, May 26, 2017 at 6:47 AM, tkg_cangkul
mailto:yuza.ras...@gmail.com>> wrote:

hi,

i'm trying to using PCAP on metron. i'm using pycapa now
and i've success to store it into kafka and hdfs.
So, what's the main function of PCAP on metron? can i
show it to the dashboard? or it's just stored in hdfs
only and read it by CLI.?


Need Your Advice,


Best Regards,

Tkg_Cangkul

Re: PCAP on dashboard

[Nick Allen]

Are you able to run any MR jobs in your environment?  You could test that
by using one of the Hadoop examples, if you don't normally run anything
else.


$ find /usr/hdp/ -name "*hadoop*example*.jar"
/usr/hdp/2.5.3.0-37/hadoop-mapreduce/hadoop-mapreduce-examples.jar
/usr/hdp/2.5.3.0-37/hadoop-mapreduce/hadoop-mapreduce-examples-2.7.3.2.5.3.0-37.jar

$ yarn jar
/usr/hdp/2.5.3.0-37/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 16
1000
Number of Maps  = 16
Samples per Map = 1000
Wrote input for Map #0
Wrote input for Map #1
...
Job Finished in 34.842 seconds
Estimated value of Pi is 3.1425


On Wed, Jun 7, 2017 at 1:00 PM, tkg_cangkul  wrote:

> I'm using metron 0.3.0 and i run on my cluster machines.
> this is the command that i run :
>
> /usr/metron/0.3.0/bin/pcap_query.sh query -st "20160617" -df "MMdd"
> -bop "/apps/metron/out" --query "ip_src_addr == '10.1.80.220' and
> ip_src_port == '6667' and ip_dst_addr == '10.1.80.221' and ip_dst_port ==
> '42885' and protocol == '6'"
>
> i also have try to using pcap_query.sh with fixed option but i still got
> the message Could not initialize class java.net.NetworkInterface
>
> Best Regards,
>
> On 07/06/17 20:19, Nick Allen wrote:
>
> You need to provide more information about your environment.  What version
> of Metron?  How are you running Metron (metal, VM, cloud)?  What command
> did you run?
>
> On Wed, Jun 7, 2017 at 3:27 AM, tkg_cangkul  wrote:
>
>> hi Nick,
>>
>> thx for your reply.
>> Ok, so i can use Zeppelin for this pcap dashboard.
>> is there any way to connected the kibana to Zeppelin? i mean is there any
>> way to select the pcap data to the kibana ui if i embed it in zeppelin?
>>
>> By the way, i've tried to use pcap_query tool. but i've found some error
>> message below :
>>
>>
>>
>> any suggestion for that?
>>
>> Best Regards,
>>
>>
>> On 29/05/17 20:49, Nick Allen wrote:
>>
>> Right now it is stored in HDFS and then retrieved with the pcap_query
>> tool.  The pcap_query tool can also be embedded in a Zeppelin Notebook.  Of
>> course with this data is in HDFS, you can integrate with it using your tool
>> of choice.
>>
>> If you have use cases in mind, please feel free to share.
>>
>> On Fri, May 26, 2017 at 6:47 AM, tkg_cangkul 
>> wrote:
>>
>>> hi,
>>>
>>> i'm trying to using PCAP on metron. i'm using pycapa now and i've
>>> success to store it into kafka and hdfs.
>>> So, what's the main function of PCAP on metron? can i show it to the
>>> dashboard? or it's just stored in hdfs only and read it by CLI.?
>>>
>>>
>>> Need Your Advice,
>>>
>>>
>>> Best Regards,
>>>
>>> Tkg_Cangkul
>>>
>>
>>
>>
>
>

Re: PCAP on dashboard

[tkg_cangkul]

I'm using metron 0.3.0 and i run on my cluster machines.
this is the command that i run :

/usr/metron/0.3.0/bin/pcap_query.sh query -st "20160617" -df "MMdd" 
-bop "/apps/metron/out" --query "ip_src_addr == '10.1.80.220' and 
ip_src_port == '6667' and ip_dst_addr == '10.1.80.221' and ip_dst_port 
== '42885' and protocol == '6'"


i also have try to using pcap_query.sh with fixed option but i still got 
the message Could not initialize class java.net.NetworkInterface


Best Regards,

On 07/06/17 20:19, Nick Allen wrote:
You need to provide more information about your environment.  What 
version of Metron?  How are you running Metron (metal, VM, cloud)? 
What command did you run?


On Wed, Jun 7, 2017 at 3:27 AM, tkg_cangkul > wrote:


hi Nick,

thx for your reply.
Ok, so i can use Zeppelin for this pcap dashboard.
is there any way to connected the kibana to Zeppelin? i mean is
there any way to select the pcap data to the kibana ui if i embed
it in zeppelin?

By the way, i've tried to use pcap_query tool. but i've found some
error message below :



any suggestion for that?

Best Regards,


On 29/05/17 20:49, Nick Allen wrote:

Right now it is stored in HDFS and then retrieved with the
pcap_query tool.  The pcap_query tool can also be embedded in a
Zeppelin Notebook.  Of course with this data is in HDFS, you can
integrate with it using your tool of choice.

If you have use cases in mind, please feel free to share.

On Fri, May 26, 2017 at 6:47 AM, tkg_cangkul
mailto:yuza.ras...@gmail.com>> wrote:

hi,

i'm trying to using PCAP on metron. i'm using pycapa now and
i've success to store it into kafka and hdfs.
So, what's the main function of PCAP on metron? can i show it
to the dashboard? or it's just stored in hdfs only and read
it by CLI.?


Need Your Advice,


Best Regards,

Tkg_Cangkul

Re: Install Metron 0.4.0 on CentOS 7 with MySQL (MariaDB) for Metron REST.

[Laurens Vets]

I'm updating the document as we speak (sensors etc...). Should be ready 
today or tomorrow.


On 2017-06-07 01:00, Dima Kovalyov wrote:

Thank you for the detailed manual Laurens,

Have you had a change to post it somewhere?
I would like to edit it and fix few points, like:

There is no need to install mariadb-server on all nodes, just on the
Metron UI node:
yum install mariadb-server
mysql-connector-java.jar package should be renamed to just
mysql-connector-java
Structure for Metron UI MySQL tables can be found here:
./metron-interface/metron-rest/target/classes/schema-mysql.sql

etc.

- Dima


On 05/12/2017 02:14 AM, Laurens Vets wrote:

On 2017-05-11 12:13, Earl Hinkle wrote:

So can it be used with out kibana working? Is that because of the
mariadb config? Also, the .md extension what app would this be opened
with?
I think that the Kibana stuff is due to my (weird?) test setup. You 
can

just open that file in a text editor or the markdown editor of your
choice :)


Sent with AquaMail for Android
http://www.aqua-mail.com


On May 11, 2017 8:27:24 AM Laurens Vets  wrote:


Hi Earl,

See attached. The document is in Markdown format. There's still a
couple
of things which aren't working as expected like Kibana... I'll try 
to

keep the document updated with my findings.

-Laurens


I would like to get the directions u used to get your installation
running.

Sent with AquaMail for Android
http://www.aqua-mail.com


On May 8, 2017 3:08:48 PM Laurens Vets  wrote:


Hi list,

I'm not sure where to post this, but I've got a simple document
which
explains installing Metron 0.4.0. I've been trying to install 
Metron

0.4.0 in 3 VMs the past couple of days and with the help of Ryan,
Jon
&
Otto succeeded today.
I've got Metron 0.4.0 installed on CentOS 7 with a MariaDB 
database

for
the REST interface.

What or where would be the best place to post this?

Re: Help needed in assign a Task

[zeo...@gmail.com]

I'm unable to find you in the Assignee list.

Jon

On Wed, Jun 7, 2017 at 10:10 AM RaghuMitra Kandikonda <
raghumitra@gmail.com> wrote:

> Hi All,
>
> Can anyone help me in assigning  METRON-988
>  to me?.
>
> My username is 'raghumitra'.
>
> -Raghu
>
-- 

Jon

Re: Help needed in assign a Task

[Michael Miklavcic]

I tried to assign this to you, but Jira says it cannot be assigned to you.
I'm not sure if this is a perms issue or something else?

On Wed, Jun 7, 2017 at 8:10 AM, RaghuMitra Kandikonda <
raghumitra@gmail.com> wrote:

> Hi All,
>
> Can anyone help me in assigning  METRON-988
>  to me?.
>
> My username is 'raghumitra'.
>
> -Raghu
>

Help needed in assign a Task

[RaghuMitra Kandikonda]

Hi All,

Can anyone help me in assigning  METRON-988
 to me?.

My username is 'raghumitra'.

-Raghu

Re: PCAP on dashboard

[Nick Allen]

You need to provide more information about your environment.  What version
of Metron?  How are you running Metron (metal, VM, cloud)?  What command
did you run?

On Wed, Jun 7, 2017 at 3:27 AM, tkg_cangkul  wrote:

> hi Nick,
>
> thx for your reply.
> Ok, so i can use Zeppelin for this pcap dashboard.
> is there any way to connected the kibana to Zeppelin? i mean is there any
> way to select the pcap data to the kibana ui if i embed it in zeppelin?
>
> By the way, i've tried to use pcap_query tool. but i've found some error
> message below :
>
>
>
> any suggestion for that?
>
> Best Regards,
>
>
> On 29/05/17 20:49, Nick Allen wrote:
>
> Right now it is stored in HDFS and then retrieved with the pcap_query
> tool.  The pcap_query tool can also be embedded in a Zeppelin Notebook.  Of
> course with this data is in HDFS, you can integrate with it using your tool
> of choice.
>
> If you have use cases in mind, please feel free to share.
>
> On Fri, May 26, 2017 at 6:47 AM, tkg_cangkul 
> wrote:
>
>> hi,
>>
>> i'm trying to using PCAP on metron. i'm using pycapa now and i've success
>> to store it into kafka and hdfs.
>> So, what's the main function of PCAP on metron? can i show it to the
>> dashboard? or it's just stored in hdfs only and read it by CLI.?
>>
>>
>> Need Your Advice,
>>
>>
>> Best Regards,
>>
>> Tkg_Cangkul
>>
>
>
>

Re: Install Metron 0.4.0 on CentOS 7 with MySQL (MariaDB) for Metron REST.

[Kevin Waterson]

How about here?

On 9 May 2017 8:08 am, "Laurens Vets"  wrote:

> Hi list,
>
> I'm not sure where to post this, but I've got a simple document which
> explains installing Metron 0.4.0. I've been trying to install Metron 0.4.0
> in 3 VMs the past couple of days and with the help of Ryan, Jon & Otto
> succeeded today.
> I've got Metron 0.4.0 installed on CentOS 7 with a MariaDB database for
> the REST interface.
>
> What or where would be the best place to post this?
>

Re: Install Metron 0.4.0 on CentOS 7 with MySQL (MariaDB) for Metron REST.

[Dima Kovalyov]

Thank you for the detailed manual Laurens,

Have you had a change to post it somewhere?
I would like to edit it and fix few points, like:
> There is no need to install mariadb-server on all nodes, just on the
> Metron UI node:
> yum install mariadb-server
> mysql-connector-java.jar package should be renamed to just
> mysql-connector-java
> Structure for Metron UI MySQL tables can be found here:
> ./metron-interface/metron-rest/target/classes/schema-mysql.sql
etc.

- Dima


On 05/12/2017 02:14 AM, Laurens Vets wrote:
> On 2017-05-11 12:13, Earl Hinkle wrote:
>> So can it be used with out kibana working? Is that because of the
>> mariadb config? Also, the .md extension what app would this be opened
>> with?
> I think that the Kibana stuff is due to my (weird?) test setup. You can 
> just open that file in a text editor or the markdown editor of your 
> choice :)
>
>> Sent with AquaMail for Android
>> http://www.aqua-mail.com
>>
>>
>> On May 11, 2017 8:27:24 AM Laurens Vets  wrote:
>>
>>> Hi Earl,
>>>
>>> See attached. The document is in Markdown format. There's still a 
>>> couple
>>> of things which aren't working as expected like Kibana... I'll try to
>>> keep the document updated with my findings.
>>>
>>> -Laurens
>>>
 I would like to get the directions u used to get your installation
 running.

 Sent with AquaMail for Android
 http://www.aqua-mail.com


 On May 8, 2017 3:08:48 PM Laurens Vets  wrote:

> Hi list,
>
> I'm not sure where to post this, but I've got a simple document 
> which
> explains installing Metron 0.4.0. I've been trying to install Metron
> 0.4.0 in 3 VMs the past couple of days and with the help of Ryan, 
> Jon
> &
> Otto succeeded today.
> I've got Metron 0.4.0 installed on CentOS 7 with a MariaDB database
> for
> the REST interface.
>
> What or where would be the best place to post this?

Re: PCAP on dashboard

[tkg_cangkul]

hi Nick,

thx for your reply.
Ok, so i can use Zeppelin for this pcap dashboard.
is there any way to connected the kibana to Zeppelin? i mean is there 
any way to select the pcap data to the kibana ui if i embed it in zeppelin?


By the way, i've tried to use pcap_query tool. but i've found some error 
message below :




any suggestion for that?

Best Regards,

On 29/05/17 20:49, Nick Allen wrote:
Right now it is stored in HDFS and then retrieved with the pcap_query 
tool. The pcap_query tool can also be embedded in a Zeppelin 
Notebook.  Of course with this data is in HDFS, you can integrate with 
it using your tool of choice.


If you have use cases in mind, please feel free to share.

On Fri, May 26, 2017 at 6:47 AM, tkg_cangkul > wrote:


hi,

i'm trying to using PCAP on metron. i'm using pycapa now and i've
success to store it into kafka and hdfs.
So, what's the main function of PCAP on metron? can i show it to
the dashboard? or it's just stored in hdfs only and read it by CLI.?


Need Your Advice,


Best Regards,

Tkg_Cangkul