HBASE performance highly dependant on query & row key format. Can you share few rowkeys, query format? also what is encoding you are using?
On Thu, May 24, 2018 at 8:38 PM, Kang Minwoo <[email protected]> wrote: > 5B logs a day? > => Yes, 5B/day > > You store a single log into a single cell? > => No, I store more than 50 cells because with extra information. > > The log in the hbase cell is compressed or you have enabled compression on > the hbase column family? > => enabled compression on the hbase column family > > What is your test environment like? Is it a single server? How are you > running your compare? > => about 100 region server, and I tested hive sql vs hbase scan from same > origin data. > (origin data -> put hbase, convert orc file -> test hive sql vs hbase scan) > > > Best regards, > Minwoo Kang > > ________________________________________ > 보낸 사람: Stack <[email protected]> 대신 [email protected] < > [email protected]> > 보낸 날짜: 2018년 5월 24일 목요일 01:33 > 받는 사람: Hbase-User > 제목: Re: How to improve HBase read performance. > > On Wed, May 16, 2018 at 7:30 PM, Kang Minwoo <[email protected]> > wrote: > > > Here is information. > > > > store about 5 billion a day. > > > > > 5B logs a day? > > > > > size of each log is about 4k. > > > > You store a single log into a single cell? > > > > > compression by snappy. > > > > > The log in the hbase cell is compressed or you have enabled compression on > the hbase column family? > > > What is your test environment like? Is it a single server? How are you > running your compare? > > Thanks, > M > > > > > > Best regards, > > Minwoo Kang > > > > ________________________________________ > > 보낸 사람: Ted Yu <[email protected]> > > 보낸 날짜: 2018년 5월 17일 목요일 00:01 > > 받는 사람: [email protected] > > 제목: Re: How to improve HBase read performance. > > > > bq. store a lot of logs in HBase > > > > Kang: > > Can you tell us a bit more about how you store (and access) the log(s) - > > size of each log, whether log is encoded in hbase ? > > > > ORC is columnar format while hbase uses different format. > > > > Thanks > > > > On Wed, May 16, 2018 at 6:41 AM, Marcell Ortutay <[email protected]> > > wrote: > > > > > This ticket: https://issues.apache.org/jira/browse/HBASE-20459 was > fixed > > > in > > > the latest version of HBase, upgrading to latest may help with > > performance > > > > > > On Wed, May 16, 2018 at 3:55 AM, Kang Minwoo <[email protected]> > > > wrote: > > > > > > > Hi, Users. > > > > > > > > I store a lot of logs in HBase. > > > > However, the reading speed of the log is too slow compared to the > Hive > > > ORC > > > > file. > > > > I know that HBase is slow compared to the Hive ORC file. > > > > The problem is that it is too slow. > > > > HBase is about 6 times slower. > > > > > > > > Is there a good way to speed up HBase's reading speed? > > > > Should I put a lot of servers? > > > > > > > > I am using HBase 1.2.6. > > > > > > > > Best regards, > > > > Minwoo Kang > > > > > > > > > >
