Re: Configuring Hadoop, HBase and Hive Cluster

2012-11-12 Thread Yanbo Liang
I recommend you to deploy master node of HDFS, MR, HBase in different servers which can get better performance. An example scenario is: 1, Deploy zookeeper on each server or the server1, server2, server3, and they make up a zookeeper cluster of odd numbers. 2, Deploy HDFS NameNode, backup NN, MR

Re: How to monitor total disk access in HBase?

2012-11-12 Thread Otis Gospodnetic
Hi, I was just trawling HBase JIRA for this info after one happy user of SPM for HBase asked if we could show read requests separately from write requests. See https://issues.apache.org/jira/browse/HBASE-4989 https://issues.apache.org/jira/browse/HBASE-3647 There is also https://issues.apache.or

RE: scan is slower after bulk load

2012-11-12 Thread Bijieshan
I think one possible reason is block caching. Have you turned the block caching off during scanning? Regards, Jieshan From: Mohammad Tariq [donta...@gmail.com] Sent: Tuesday, November 13, 2012 1:04 To: user@hbase.apache.org Subject: Re: scan is slower af

Re: Configuring Hadoop, HBase and Hive Cluster

2012-11-12 Thread Dalia Sobhy
I do advise you to use Cloudera Manager its a very simple and opensource cluster configuration software.. A good design is to run zookeeper on node1, node2, another node alone Sent from my iPhone On 2012-11-13, at 2:04 AM, "Hakan Bogay" wrote: > Hi, > > I am a newbie to Hadoop, HBase and Hiv

Configuring Hadoop, HBase and Hive Cluster

2012-11-12 Thread Hakan Bogay
Hi, I am a newbie to Hadoop, HBase and Hive. I installed Hadoop, HBase and Hive in pseudodistributed mode and everything works fine. Now I am planning to set up an simple Hadoop Cluster (5 nodes) with Hive, HBase and ZooKeeper. I´ve read several documentations and instructions before but i could n

Re: About -D parameters

2012-11-12 Thread Jean-Marc Spaggiari
Oh! I see! Super, thanks a lot for all this information! Very clear to me now! Thanks, JM 2012/11/12, Shrijeet Paliwal : > Inline. > > On Mon, Nov 12, 2012 at 5:45 AM, Jean-Marc Spaggiari < > jean-m...@spaggiari.org> wrote: > >> Oups, sorry about that :( Pressed the send button to quickly :( >>

Re: About -D parameters

2012-11-12 Thread Shrijeet Paliwal
Inline. On Mon, Nov 12, 2012 at 5:45 AM, Jean-Marc Spaggiari < jean-m...@spaggiari.org> wrote: > Oups, sorry about that :( Pressed the send button to quickly :( > > So, question is. > > On the RowCounter MapReduce we can pass > "-Dhbase.client.scanner.caching=100". But I'm not able to find > anyw

Re: scan is slower after bulk load

2012-11-12 Thread Mohammad Tariq
may be because bulk load writes to the same region thus putting the entire load on a single region server. Regards, Mohammad Tariq On Mon, Nov 12, 2012 at 9:15 PM, Michael Segel wrote: > Just a guess... have you done any compactions on the table post bulk load? > > On Nov 12, 2012, at 8:44

Re: How to monitor total disk access in HBase?

2012-11-12 Thread PG
Thanks, for the suggestions. Will do instrumentation based on JMX. Regards, Yun On Nov 12, 2012, at 12:59 AM, Stack wrote: > On Sat, Nov 10, 2012 at 7:03 AM, yun peng wrote: >> Hi, I want to profile the # of disk access (both random and sequential) >> issued from HBase (into HDFS). For disk rea

Re: scan is slower after bulk load

2012-11-12 Thread Michael Segel
Just a guess... have you done any compactions on the table post bulk load? On Nov 12, 2012, at 8:44 AM, Marcos Ortiz wrote: > Regards, Amit. > Did you tuned the RegionServer where you has that data range hosted? > Why do you say that scans are slower after a bulk load? > Did you test it before

Re: scan is slower after bulk load

2012-11-12 Thread Marcos Ortiz
Regards, Amit. Did you tuned the RegionServer where you has that data range hosted? Why do you say that scans are slower after a bulk load? Did you test it before bulk load? HBase version? On 11/12/2012 09:39 AM, Amit Sela wrote: Hi all, Anyone has any idea why scanning over specific range in

Re: About -D parameters

2012-11-12 Thread Jean-Marc Spaggiari
Oups, sorry about that :( Pressed the send button to quickly :( So, question is. On the RowCounter MapReduce we can pass "-Dhbase.client.scanner.caching=100". But I'm not able to find anywhere where it's parsed I only found that on the HTable and it's read from the configuration. On my MapReduc

About -D parameters

2012-11-12 Thread Jean-Marc Spaggiari
Hi, I'm building a MapReduce and I have some question about the -D parameters.

Re: scan filtering column familly return wrong cell

2012-11-12 Thread Damien Hardy
I don't know if HBase shell scan command use ColumnCountGetFilter. The absence of compaction could explain the 2 same cell displayed. But when I filter on one colfam, I get only 1 cell ... from the wrong colfam (like if the cell is stored in the wrong HFile) ... When I add clone of my KeyValues in

Re: Why Regionserver is not serving when I set the WAL trigger?

2012-11-12 Thread yonghu
The problem is caused by the code, as I create the configuration by myself. The correct way is to use getConfiguration() method. regards! Yong On Sat, Nov 10, 2012 at 4:12 PM, ramkrishna vasudevan wrote: > Sorry i am not very sure if there is any link between the coprocessor and > region not on