Hi Ashutosh, The sink will not create the table or column family. Make sure you have the table and column family. Also please make sure you have HADOOP_HOME/HADOOP_PREFIX and HBASE_HOME set correctly(or they are in your class path).
Thanks Hari -- Hari Shreedharan On Thursday, June 21, 2012 at 12:52 AM, ashutosh(오픈플랫폼개발팀) wrote: > Hi, > > I have used and followed the same steps which is mentioned in below mails to > get start with the hbasesink. But agent is not storing any data into hbase. I > added the hbase-site.xml in $CLASSPATH variable to pick the hbase > information. Even I am able to connect to the hbase server from that agent > machine. > > Now, I am unable to understand and troubleshoot this problem. Seeking advice > from the community members... > > ---------------------------------------- > ---------------------------------------- > Thanks & Regards, > Ashutosh Sharma > ---------------------------------------- > > -----Original Message----- > From: Mohammad Tariq [mailto:donta...@gmail.com] > Sent: Friday, June 15, 2012 9:02 AM > To: flume-user@incubator.apache.org (mailto:flume-user@incubator.apache.org) > Subject: Re: Hbase-sink behavior > > Thank you so much Hari for the valuable response..I'll follow the guidelines > provided by you. > > Regards, > Mohammad Tariq > > > On Fri, Jun 15, 2012 at 5:26 AM, Hari Shreedharan <hshreedha...@cloudera.com > (mailto:hshreedha...@cloudera.com)> wrote: > > Hi Mohammad, > > > > My answers are inline. > > > > -- > > Hari Shreedharan > > > > On Thursday, June 14, 2012 at 4:47 PM, Mohammad Tariq wrote: > > > > Hello list, > > > > I am trying to use hbase-sink to collect data from a local file and > > dump it into an Hbase table..But there are a few things I am not able > > to understand and need some guidance. > > > > This is the content of my conf file : > > > > hbase-agent.sources = tail > > hbase-agent.sinks = sink1 > > hbase-agent.channels = ch1 > > hbase-agent.sources.tail.type = exec > > hbase-agent.sources.tail.command = tail -F /home/mohammad/demo.txt > > hbase-agent.sources.tail.channels = ch1 hbase-agent.sinks.sink1.type = > > org.apache.flume.sink.hbase.HBaseSink > > hbase-agent.sinks.sink1.channel = ch1 > > hbase-agent.sinks.sink1.table = test3 > > hbase-agent.sinks.sink1.columnFamily = testing > > hbase-agent.sinks.sink1.column = foo > > hbase-agent.sinks.sink1.serializer = > > org.apache.flume.sink.hbase.SimpleHbaseEventSerializer > > hbase-agent.sinks.sink1.serializer.payloadColumn = col1 > > hbase-agent.sinks.sink1.serializer.incrementColumn = col1 > > hbase-agent.sinks.sink1.serializer.keyType = timestamp > > hbase-agent.sinks.sink1.serializer.rowPrefix = 1 > > hbase-agent.sinks.sink1.serializer.suffix = timestamp > > hbase-agent.channels.ch1.type=memory > > > > Right now I am taking just some simple text from a file which has > > following content - > > > > value1 > > value2 > > value3 > > value4 > > value5 > > value6 > > > > And my Hbase table looks like - > > > > hbase(main):217:0> scan 'test3' > > ROW COLUMN+CELL > > 11339716704561 column=testing:col1, > > timestamp=1339716707569, value=value1 > > 11339716704562 column=testing:col1, > > timestamp=1339716707571, value=value4 > > 11339716846594 column=testing:col1, > > timestamp=1339716849608, value=value2 > > 11339716846595 column=testing:col1, > > timestamp=1339716849610, value=value1 > > 11339716846596 column=testing:col1, > > timestamp=1339716849611, value=value6 > > 11339716846597 column=testing:col1, > > timestamp=1339716849614, value=value6 > > 11339716846598 column=testing:col1, > > timestamp=1339716849615, value=value5 > > 11339716846599 column=testing:col1, > > timestamp=1339716849615, value=value6 > > incRow column=testing:col1, > > timestamp=1339716849677, value=\x00\x00\x00\x00\x00\x00\x00\x1C > > 9 row(s) in 0.0580 seconds > > > > Now I have following questions - > > > > 1- Why the timestamp value is different from the row key?(I was trying > > to make "1+timestamp" as the rowkey) > > > > The value shown by hbase shell as timestamp is the time at which the > > value was inserted into Hbase, while the value inserted by Flume is > > the timestamp at which the sink read the event from the channel. > > Depending on how long the network and HBase takes, these timestamps > > can vary. If you want 1+timestamp as row key then you should configure it: > > > > hbase-agent.sinks.sink1.serializer.rowPrefix = 1+ This prefix is > > appended as-is to the suffix you choose. > > > > 2- Although I am not using "incRow", it stills appear in the table > > with some value. Why so and what is this value?? > > > > The SimpleHBaseEventSerializer is only an example class. For custom > > use cases you can write your own serializer by implementing > > HbaseEventSerializer. In this case, you have specified > > incrementColumn, which causes an increment on the column specified. > > Simply don't specify that config and that row will not appear. > > > > 3- How can avoid the last row?? > > > > See above. > > > > > > I am still in the learning phase so please pardon my ignorance..Many thanks. > > > > No problem. Much of this is documented > > here: > > https://builds.apache.org/job/flume-trunk/site/apidocs/index.html > > > > > > > > Regards, > > Mohammad Tariq > > > > > > 이 메일은 지정된 수취인만을 위해 작성되었으며, 중요한 정보나 저작권을 포함하고 있을 수 있습니다. 어떠한 권한 없이, 본 문서에 포함된 > 정보의 전부 또는 일부를 무단으로 제3자에게 공개, 배포, 복사 또는 사용하는 것을 엄격히 금지합니다. 만약, 본 메일이 잘못 전송된 > 경우, 발신인 또는 당사에 알려주시고, 본 메일을 즉시 삭제하여 주시기 바랍니다. > This E-mail may contain confidential information and/or copyright material. > This email is intended for the use of the addressee only. If you receive this > email by mistake, please either delete it without reproducing, distributing > or retaining copies thereof or notify the sender immediately. > >
