BigDataDayLA 2015

Hi Everyone, We have BigDataDayLA coming up on 27/6/2015 Los Angeles, CA. We are looking for speakers on NoSql track, so we thought of reaching HBase and Phoenix user groups to see if anyone is interested. You can submit your talks here

Re: HBase stucks from time to time

Here is disk stats. Sadness appeared ad 12.30 - 13.00 https://www.dropbox.com/s/lj4r8o10buv1n2o/Screenshot%202015-04-22%2020.48.18.png?dl=0 2015-04-22 20:41 GMT+02:00 Serega Sheypak : > Here is an image > > 2015-04-22 20:40 GMT+02:00 Serega Sheypak : > >> Here are datanode logs from 5.9.41.237

Re: HBase stucks from time to time

Here is an image 2015-04-22 20:40 GMT+02:00 Serega Sheypak : > Here are datanode logs from 5.9.41.237 , > regionserver logs were from5.9.41.237 also > > EQUEST_SHORT_CIRCUIT_FDS, blockid: 1078130838, srvID: > 659e6be2-8d98-458b-94bc-3bcdbb51750

Re: HBase stucks from time to time

Here are datanode logs from 5.9.41.237 , regionserver logs were from5.9.41.237 also EQUEST_SHORT_CIRCUIT_FDS, blockid: 1078130838, srvID: 659e6be2-8d98-458b-94bc-3bcdbb517508, success: true 2015-04-22 12:46:17,154 INFO org.apache.hadoop.hdfs.ser

Re: HBase stucks from time to time

Did you send the logs from one of those 3 servers? [5.9.41.237:50010, 5.9.77.105:50010, 5.9.73.19:50010] Sound like something is slowing done everything. Can you extract DN logs for the same time? Do you have any tool monitoring the disks and network latency over time? If not, can you run iostat

Re: HBase stucks from time to time

The image didn't come through. FYI On Wed, Apr 22, 2015 at 11:23 AM, Serega Sheypak wrote: > > major compaction runs daily. > > >What else do you see in the RS logs? > no error, only *Slow sync cost * > > >How iostat looks like? > please see image. 12.00 - 12.30 is a time when reading/writing s

Re: HBase stucks from time to time

> major compaction runs daily. >What else do you see in the RS logs? no error, only *Slow sync cost * >How iostat looks like? please see image. 12.00 - 12.30 is a time when reading/writing stopped [image: Встроенное изображение 1] >Can you share the logs around the time this occurs? 2015-04-22

Re: HBase stucks from time to time

Serega: How often is major compaction run in your cluster ? Have you configured offpeak compaction ? See related parameters in: http://hbase.apache.org/book.html#compaction.parameters Cheers On Wed, Apr 22, 2015 at 10:39 AM, Serega Sheypak wrote: > Hi, we have 10 nodes cluster running HBase 0.

Re: HBase stucks from time to time

Hi Serega, How iostat looks like? are you seeing high utilization (above 60% across the drives) or long await write times when its writing to the WAL? What else do you see in the RS logs? cheers, esteban. -- Cloudera, Inc. On Wed, Apr 22, 2015 at 10:39 AM, Serega Sheypak wrote: > Hi, we

Re: HBase stucks from time to time

Hi Serega, can also be because of garbage collection. Can you share the logs around the time this occurs? Compacting regions will not block HBase to get writes and to write in the WAL. But might impact your storage system regarding IOs. JM 2015-04-22 13:39 GMT-04:00 Serega Sheypak : > Hi, we h

HBase stucks from time to time

Hi, we have 10 nodes cluster running HBase 0.98 CDH 5.2.1 Sometimes HBase stucks. We have several apps constantly writing/reading data from it. Sometimes we see that apps response time dramatically increases. It means that app spends seconds to read/write from/to HBase. in 99% of time it takes 20ms

Re: scan startrow and stoprow

Character '*' precedes character '0' Probably that was why some rows were not returned. How long is the third component (number) normally ? Cheers > On Apr 22, 2015, at 3:56 AM, Sachin wrote: > > My rowkey format is uniqueid|timestamp|randomnumber. > > I want to retrieve data from hbase by

Re: scan startrow and stoprow

If u want data of timeStamp2 also, (All 6 rows as shown in eg: above) then you have to put timeStamp3 in stop row.. The stop row is exclusive. startRow : aabb|timeStamp1| stopRow is : aabb|timeStamp3 -Anoop- On Wed, Apr 22, 2015 at 5:09 PM, Shahab Yunus wrote: > I see that you are already

scan startrow and stoprow

My rowkey format is uniqueid|timestamp|randomnumber. I want to retrieve data from hbase by using scanner with java api where my startRow : aabb|timeStamp1| stopRow is : aabb|timeStamp2|any_number timeStamp1 and timeStamp2 are time ranges. So I want to fetch all values in between above timestam

Re: scan startrow and stoprow

I see that you are already using partial key scan approach. When you say that you want data from the timeStamp2 as well, you want all the rows from timeStamp2? Also, how are you actually setting the stopRow? What are you providing for the *anyNumber field when you are setting stopRow? e.g. if you

scan startrow and stoprow

My rowkey format is uniqueid|timestamp|randomnumber. I want to retrieve data from hbase by using scanner with java api where my startRow : aabb|timeStamp1| stopRow is : aabb|timeStamp2|*any_number timeStamp1 and timeStamp2 are time ranges. So I want to fetch all values in between above timest