Hi Krishna - I did not look at your Profile definition. Please include it in the body of your email the next time you might need help. I am a little hesitant to open attachments on a public mailing list. My apologies.
Keep in mind there are two places for what you might call 'settings'. - There are global settings for the "Profiler" (aka profiler.properties). These are settings that apply globally to the Storm topology that consumes profile definitions and generates profile data. - There are also 'settings' that are unique to each Profile. These are contained in what I call the Profile definition. This defines how each individual Profile produces data. Based on Simon's feedback, it seems that you are confusing some of these settings. Simon is referring to the 'expires' field in your Profile definition that is documented here [1]. Please see the following links for more information. If anything is not clear in these links, please let us know. - Profile Definitions [2] - Configuring the Profiler [3] -- [1] http://metron.apache.org/current-book/metron-analytics/metron-profiler/index.html#expires [2] http://metron.apache.org/current-book/metron-analytics/metron-profiler/index.html#Creating_Profiles [3] http://metron.apache.org/current-book/metron-analytics/metron-profiler/index.html#Configuring_the_Profiler On Wed, Jul 12, 2017 at 8:02 AM, Simon Elliston Ball < si...@simonellistonball.com> wrote: > In your config for the individual profile though, you expire the content > every 30ms (per the zip file) > > Simon > > > > > On 12 Jul 2017, at 12:20, Krishna Dhanekula <krishna.dhanek...@sstech.us> > wrote: > > Thanks Simon for replying. > > This is my ‘profiler.properties’ file content. TTL is 30 minutes and for > every 30 seconds I am flushing to Hbase. > > profiler.workers=1 > profiler.executors=0 > profiler.input.topic=indexing > profiler.period.duration=1 > profiler.period.duration.units=MINUTES > profiler.ttl=30 > profiler.ttl.units=MINUTES > profiler.hbase.salt.divisor=1000 > profiler.hbase.table=profiler > profiler.hbase.column.family=P > profiler.hbase.batch=1 > profiler.hbase.flush.interval.seconds=30 > > ##### Kafka ##### > > kafka.zk=10.10.110.184:2181 > kafka.broker=10.10.110.184:6667 > kafka.start=WHERE_I_LEFT_OFF > > > > *Regards,* > > *Balakrishna* > > > *From:* Simon Elliston Ball [mailto:si...@simonellistonball.com > <si...@simonellistonball.com>] > *Sent:* Wednesday, July 12, 2017 3:28 PM > *To:* user@metron.apache.org > *Subject:* Re: Metron Profiler 0.3.0: HbaseBolt not storing data to HBase > Instance > > Looks like you’ve set the profile to purge (expires) every 30 ms, and your > period is set to 30 minutes, so the data is being expired long before it > has a change to write. > > Simon > > > > On 12 Jul 2017, at 06:17, Krishna Dhanekula <krishna.dhanek...@sstech.us> > wrote: > > I have an problem where profiled data is not storing to Hbase instance: > > Installed metron profiler in ‘HDP-2.5.5.0-157’ with the help of this link: > > https://metron.apache.org/current-book/metron-analytics/ > metron-profiler/index.html > > Attached my profiler.properties and profiler.json files, remote.yaml and > worker.log files. > > Once I place the following message into kafka topic 'indexing' > > [{"ip_src_addr":"10.0.0.1","protocol":"HTTPS","length":" > 10","bytes_in":"234"},{"ip_src_addr":"10.0.0.2"," > protocol":"HTTP","length":"20","bytes_in":"390"},{"ip_src_ > addr":"10.0.0.3","protocol":"DNS","length":"30","bytes_in":"560"}] > > It's picked by splitterBolt and passed till HBaseBolt. But data is not > getting stored to Hbase. > > Attached metron-artifacts.zip > > Attached storm-ui screenshot.storm-ui-screen.png > > Please suggest what am I missing here. > > Thanks in advance. > > Attachments: > storm-ui-screen.png (17.4 kB) > metron-artifacts.zip (36.6 kB) > > > Regards, > Balakrishna > <metron_artifacts.zip><storm_ui_screen.png> > > >
