Re: Metron Profiler 0.3.0: HbaseBolt not storing data to HBase Instance

Looks like you’ve set the profile to purge (expires) every 30 ms, and your period is set to 30 minutes, so the data is being expired long before it has a change to write. Simon > On 12 Jul 2017, at 06:17, Krishna Dhanekula > wrote: > > I have an problem where profiled data is not storing t

RE: Metron Profiler 0.3.0: HbaseBolt not storing data to HBase Instance

Thanks Simon for replying. This is my ‘profiler.properties’ file content. TTL is 30 minutes and for every 30 seconds I am flushing to Hbase. profiler.workers=1 profiler.executors=0 profiler.input.topic=indexing profiler.period.duration=1 profiler.period.duration.units=MINUTES profiler.ttl=30 pro

Re: Metron Profiler 0.3.0: HbaseBolt not storing data to HBase Instance

In your config for the individual profile though, you expire the content every 30ms (per the zip file) Simon > On 12 Jul 2017, at 12:20, Krishna Dhanekula > wrote: > > Thanks Simon for replying. > > This is my ‘profiler.properties’ file content. TTL is 30 minutes and for > every 30 secon

metron profiler hbase storage issue

Hello Team, We have installed metron profiler with the help of this link: https://metron.apache.org/current-book/metron-analytics/metron-profiler/index.html Attached my profiler.properties and profiler.json files, remote.yaml and worker.log files in metron-artifacts.zip. Once I place the followi

Re: metron profiler hbase storage issue

Have you been able to follow these instructions successfully? [1] I would start there and follow them exactly. Only then try and create your own profiles. I would suggest you include settings in main email body rather than an attachment. [1] http://metron.apache.org/current-book/metron-analytic

Re: Metron Profiler 0.3.0: HbaseBolt not storing data to HBase Instance

Hi Krishna - I did not look at your Profile definition. Please include it in the body of your email the next time you might need help. I am a little hesitant to open attachments on a public mailing list. My apologies. Keep in mind there are two places for what you might call 'settings'. -

Re: metron profiler hbase storage issue

As I answered earlier today, your profile in this zip file is set to expire data after 30 milliseconds, so you will not be writing any data. > On 12 Jul 2017, at 13:40, Dima Kovalyov wrote: > > Attached artifacts. > > On 07/12/2017 07:39 PM, Dima Kovalyov wrote: >> Hello Team, >> >> We have

Re: metron profiler hbase storage issue

The 'expire' field is actually in days not milliseconds. So that should not be a problem here. On Wed, Jul 12, 2017 at 8:50 AM, Simon Elliston Ball < si...@simonellistonball.com> wrote: > As I answered earlier today, your profile in this zip file is set to > expire data after 30 milliseconds, so

Re: Metron Profiler 0.3.0: HbaseBolt not storing data to HBase Instance

Quick correction for posterity... The 'expire' field is actually in days not milliseconds. So that should not be a problem here. On Wed, Jul 12, 2017 at 8:02 AM, Simon Elliston Ball < si...@simonellistonball.com> wrote: > In your config for the individual profile though, you expire the content

RE: Metron Profiler 0.3.0: HbaseBolt not storing data to HBase Instance

Under section ‘Creating Profiles’ it’s mentioned as milli secs. In Example1 it’s mentioned as 30 days. Now I removed the entire field and from ‘profiler.json’ so that it profiled data never expires. I pushed and configuration and tested from kafka. I still don’t see the data storing to Hbase.

Re: Metron REST fails to start with Ambari, works via CLI

Does this error happen on start or restart? The bug I fixed only happened on restart. On Tue, Jul 11, 2017 at 5:13 PM, Nick Allen wrote: > Yes, I have seen this. It is a bug. I believe Ryan submitted a fix in one > of his open PRs. He can chime in with the exact one. > > On Jul 11, 2017 6:08 P

Presentation

Hello, My name is Antonio Perez and I work as a developer in GMV. We joined the mailing list of users and developers because in our company we are currently developing a project called PROTECTIVE for the European Commission. In this project we have joined several companies from Europe (3 NRENs

Re: Metron Profiler 0.3.0: HbaseBolt not storing data to HBase Instance

> Under section ‘Creating Profiles’ it’s mentioned as milli secs. In Example1 it’s mentioned as 30 days. Yes, there is an inconsistency in the README. The sentence in the table is wrong, the rest of the docs are correct. Wrong: https://github.com/apache/metron/tree/master/metron-analytics/metron

Re: Presentation

First off, welcome to the community! Your project sounds quite exciting with such a wide range of partner organizations. It sounds like you have existing telemetry data, already JSON objects, which you'd like to ingest into Metron. I'd suggest starting with the JSONMapParser and using fieldTransfo

Re: Metron REST fails to start with Ambari, works via CLI

I'm encountering the Ambari restart bug on the 0.4.0 release branch. Ryan, can you let me know which PR/JIRA has the fix for it; I'd like to test it out. Thanks! -Kyle On Wed, Jul 12, 2017 at 9:48 AM, Ryan Merriman wrote: > Does this error happen on start or restart? The bug I fixed only happ

Re: Metron REST fails to start with Ambari, works via CLI

The fix was part of METRON-990. The exact fix was a one line change in /metron-interface/metron-rest/src/main/scripts/metron-rest: line 148: "$0 start" was changed to "$0 start $2" Ryan On Wed, Jul 12, 2017 at 10:06 AM, Kyle Richardson wrote: > I'm encountering the Ambari restart bug on the

Re: Metron REST fails to start with Ambari, works via CLI

Found it. It seems this change didn't make it in the 0.4.0 release :( I'll update my documentation. On 2017-07-12 08:48, Ryan Merriman wrote: > The fix was part of METRON-990. The exact fix was a one line change in > /metron-interface/metron-rest/src/main/scripts/metron-rest: > > line 148:

RE: Metron Profiler 0.3.0: HbaseBolt not storing data to HBase Instance

When I run this command bin/zk_load_configs.sh -m PUSH -i config/zookeeper/ -z krishna-3:2181 (Krishna-3 is my node1 name) [root@krishna-4 0.3.1]# bin/zk_load_configs.sh -m PUSH -i config/zookeeper/ -z krishna-3:2181 log4j:WARN No appenders could be found for logger (org.apache.curator.framewor

Re: Metron Profiler 0.3.0: HbaseBolt not storing data to HBase Instance

If the Profiler definitions were uploaded to Zookeeper correctly, you should be able to see them when using the DUMP option. See example below. [root@node1 0.4.0]# bin/zk_load_configs.sh -m DUMP -z node1:2181 ... PROFILER Config: profiler { "profiles": [ { "profile": "test", "

RE: Metron Profiler 0.3.0: HbaseBolt not storing data to HBase Instance

When I run DUMP I can see everything else under /config/zookeeper except the profile (profiler.json) created by me. From: Nick Allen [n...@nickallen.org] Sent: Wednesday, July 12, 2017 2:44 PM To: user@metron.apache.org Subject: Re: Metron Profiler 0.3.0: HbaseBol

RE: Metron Profiler 0.3.0: HbaseBolt not storing data to HBase Instance

That's your problem then. You are not getting your profile definitions into Zookeeper. Send us a copy of your terminal session showing the location and content of your profiler.json along with the command and ouput of you running the upload script. On Jul 12, 2017 8:41 PM, "Krishna Dhanekula"

RE: Metron Profiler 0.3.0: HbaseBolt not storing data to HBase Instance

Here’s the screenshot with the profiler content and the PUSH command output. [cid:image001.png@01D2FBAF.F0EA4820] Regards, Balakrishna Dhanekula [http://www.sstech.org/images/logo.jpg]