Multiple RegionServer on same Node

Hi, Is there a way to start multiple region servers on same node ? Thanks, Rahul

Re: Multiple RegionServer on same Node

; FYI > > On Mon, Oct 19, 2015 at 3:16 PM, rahul malviya <malviyarahul2...@gmail.com > > > wrote: > > > Hi, > > > > Is there a way to start multiple region servers on same node ? > > > > Thanks, > > Rahul > > >

Simultaneous MR on a Snapshot

Hi, I am trying to run simultaneous two MR jobs on same sanpshot of a hbase table but I am getting exception before the job starts. Do we support simultaneous MR on sanpshots ? Exception : Exception in thread main java.io.IOException: java.util.concurrent.ExecutionException:

Re: Simultaneous MR on a Snapshot

:58 AM, rahul gidwani rahul.gidw...@gmail.com wrote: hi rahul, i believe the problem lies in the TableSnapshotInputFormat. You always have to clone the snapshot no matter what. On Tue, Jul 21, 2015 at 9:28 AM, rahul malviya malviyarahul2...@gmail.com wrote: Hi, I

Re: Simultaneous MR on a Snapshot

, rahul malviya malviyarahul2...@gmail.com wrote: Hi, I am trying to run simultaneous two MR jobs on same sanpshot of a hbase table but I am getting exception before the job starts. Do we support simultaneous MR on sanpshots ? Exception : Exception in thread main

Fix Number of Regions per Node ?

Hi, Is it possible to configure HBase to have only fix number of regions per node per table in hbase. For example node1 serves 2 regions, node2 serves 3 regions etc for any table created ? Thanks, Rahul

Re: Fix Number of Regions per Node ?

only(not on node basis). It would be best to avoid having diff in hardware in cluster machines. Thanks, Anil Gupta On Wed, Jun 17, 2015 at 5:12 PM, rahul malviya malviyarahul2...@gmail.com wrote: Hi, Is it possible to configure HBase to have only fix number of regions per node

Re: Response Too Slow in RegionServer Logs

http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired I would check your gc logs for long gc pauses. On Fri, May 29, 2015 at 11:38 AM, rahul malviya malviyarahul2...@gmail.com wrote: Hi All, In our cluster region server logs are filled with response too slow message

Response Too Slow in RegionServer Logs

Hi All, In our cluster region server logs are filled with response too slow message. This is causing jobs to slow down. How can I debug what is the reason for this slowness. We have enabled short circuit reads and region server has 27GB RAM. Here is a trace when regionserver starts. Thu Aug 14

Re: Response Too Slow in RegionServer Logs

, May 29, 2015 at 11:49 AM, rahul malviya malviyarahul2...@gmail.com wrote: This is a sample from gc log file. At the end I see long gc pauses. Is there a way I can tune this ? 2015-04-29T22:46:12.387+: 98061.660: [GC2015-04-29T22:46:12.387+: 98061.661: [ParNew: 572757K

Major Compacting Multiple Regions in same Region Server

Hi, I have observered major compaction on our hbase cluster it seems it compacts only one region at a time per region server. So I wanted to know if there is parameter to set to make it multi threaded so that parallelly multiple regions can get compacted on same region server. I looked at this

Re: Major Compacting Multiple Regions in same Region Server

I will try that. Thanks Ted. Rahul On Fri, May 15, 2015 at 9:44 AM, Ted Yu yuzhih...@gmail.com wrote: Have you tried giving hbase.regionserver.thread.compaction.large value larger than 1 ? Cheers On Fri, May 15, 2015 at 9:11 AM, rahul malviya malviyarahul2...@gmail.com wrote: Hi

MR against snapshot causes High CPU usage on Datanodes

Hi, I have recently started running MR on hbase snapshots but when the MR is running there is pretty high CPU usage on datanodes and I start seeing IO wait message in datanode logs and as soon I kill the MR on Snapshot everything come back to normal. What could be causing this ? I am running

Re: MR against snapshot causes High CPU usage on Datanodes

Yes. On Wed, May 13, 2015 at 9:40 AM, Ted Yu yuzhih...@gmail.com wrote: Have you enabled short circuit read ? Cheers On Wed, May 13, 2015 at 9:37 AM, rahul malviya malviyarahul2...@gmail.com wrote: Hi, I have recently started running MR on hbase snapshots but when the MR

Re: MR against snapshot causes High CPU usage on Datanodes

Short circuit read can cause it to spike CPU and IO wait issues ? Rahul On Wed, May 13, 2015 at 9:41 AM, rahul malviya malviyarahul2...@gmail.com wrote: Yes. On Wed, May 13, 2015 at 9:40 AM, Ted Yu yuzhih...@gmail.com wrote: Have you enabled short circuit read ? Cheers On Wed, May 13

Re: MR against snapshot causes High CPU usage on Datanodes

. On May 13, 2015, at 11:41 AM, rahul malviya malviyarahul2...@gmail.com wrote: Yes. On Wed, May 13, 2015 at 9:40 AM, Ted Yu yuzhih...@gmail.com wrote: Have you enabled short circuit read ? Cheers On Wed, May 13, 2015 at 9:37 AM, rahul malviya malviyarahul2...@gmail.com wrote

Re: MR against snapshot causes High CPU usage on Datanodes

local mappers? You load/data equally distributed? Your disk, cpu ratio looks ok. Sent from my iPhone On May 13, 2015, at 10:12 AM, rahul malviya malviyarahul2...@gmail.com wrote: *The High CPU may be WAIT IOs, which would mean that you’re cpu is waiting for reads from the local disks

Re: How to delete regions from a table

The steps in this email thread might be useful too. This does not require disabling table. http://mail-archives.apache.org/mod_mbox/hbase-user/201501.mbox/%3cca+vo6iukca-es0-auy+jfi4nhgmvkxb1t_ogg8kdpwnxxov...@mail.gmail.com%3E Thanks, Rahul On Mon, May 11, 2015 at 2:06 AM, Gaurav Agarwal

Re: How to Restore the block locality of a RegionServer ?

Data locality if completely restored after major compaction. Thanks everyone. Rahul On Sat, May 9, 2015 at 9:36 PM, rahul malviya malviyarahul2...@gmail.com wrote: Thanks for suggestion. I will try this and update the thread if the data locality is restored. Thanks, Rahul On Sat, May 9

Re: How to Restore the block locality of a RegionServer ?

to create an ‘auto restart’ script. Taking down HBase completely (probably including zookeeper) and do a full restart would probably fix the issue of data locality. On May 9, 2015, at 5:05 PM, rahul malviya malviyarahul2...@gmail.com wrote: Hi, My HBase cluster went

How to Restore the block locality of a RegionServer ?

Hi, My HBase cluster went through a rough patch recently where lot of region server started dying because of sudden increase in amount of data being funneled to the HBase cluster and we have to place a auto start script for regionservers. After this all my data locality is lost which does not

MapReduce on Sanpshots

Hi All, I have been MapReduce on HBase for a while now and this has been really successful for me. But I have some MRs jobs in my pipeline which scan every row on hbase. So I was wondering I move this MR job to run on snapshot that will reduce the load on RSs and HBase direct operations mostly

Re: MapReduce on Sanpshots

Thanks Ted. Seems to answer my question. Rahul On May 8, 2015, at 9:04 AM, Ted Yu yuzhih...@gmail.com wrote: Please see Release Note of HBASE-8369 Cheers On Fri, May 8, 2015 at 8:16 AM, Rahul Malviya malviyarahul2...@gmail.com wrote: Hi All, I have been MapReduce on HBase

Re: Snapshot Timing Out on a Large HBase Table

:40, rahul malviya malviyarahul2...@gmail.com a écrit : We have 16000 regions with approx 70 regions per regionserver. Thanks, Rahul On Fri, May 8, 2015 at 11:25 AM, Jean-Marc Spaggiari jean-m...@spaggiari.org wrote: How many regions do you have on this table, how many RS

Re: Snapshot Timing Out on a Large HBase Table

We have 16000 regions with approx 70 regions per regionserver. Thanks, Rahul On Fri, May 8, 2015 at 11:25 AM, Jean-Marc Spaggiari jean-m...@spaggiari.org wrote: How many regions do you have on this table, how many RS and how many regions per RS? JM 2015-05-08 14:21 GMT-04:00 Rahul

Re: Snapshot Timing Out on a Large HBase Table

regions per RS? JM 2015-05-08 14:21 GMT-04:00 Rahul Malviya malviyarahul2...@gmail.com: Hi, I want to run MR job on snapshot of large HBase table but I am getting timeout errors while doing so. Can you tell me what I am doing wrong ? I am using hbase-0.98.6

Re: MapReduce on Sanpshots

Currently we have only one user accessing all the big data stack - hdfs/hbase etc ... so security is not a issue for us but thanks for stressing on this and I will keep this in mind when we move to secured cluster. Apart from security there are no other concerns ? Thanks, Rahul On Fri, May 8,

Snapshot Timing Out on a Large HBase Table

Hi, I want to run MR job on snapshot of large HBase table but I am getting timeout errors while doing so. Can you tell me what I am doing wrong ? I am using hbase-0.98.6-cdh5.2.0. hbase-site.xml property namehbase.snapshot.master.timeoutMillis/name value180/value /property

Re: Snapshot Timing Out on a Large HBase Table

namehbase.snapshot.master.timeout.millis/name value180/value /property Thanks, Rahul On Fri, May 8, 2015 at 12:53 PM, rahul malviya malviyarahul2...@gmail.com wrote: Hi Ted, I tired doing this multiple time and this is what I see. Looks like some property settings for my timeout are not good and shell gets