Re: [ANNOUNCE] New HBase committer Baiqiang Zhao

Congratulations 👍 On Sun, Jul 11, 2021, 05:57 Bharath Vissapragada wrote: > Congratulations Baiqiang. > > On Sat, Jul 10, 2021 at 12:18 PM Nick Dimiduk wrote: > > > Hi everyone, > > > > On behalf of the Apache HBase PMC I am pleased to announce that Baiqiang > > Zhao has accepted the PMC's invi

Re: HBase Scan consumes high cpu

I would suggest you to look your design again, its wide table approach putting so many column against single rowkey. You can put you time serease data in row wize also. Thanks Manjeet singh On Wed, 18 Sep 2019, 22:23 Solvannan R M, wrote: > Hi Ram, > > Thanks for your support! We wil

Re: Query on rowkey distribution || Does RS and number of Region related with each other

F' } alter 'TEST_TABLE_KeyPrefixRegionSplitPolicy', {NAME => 'si', COMPRESSION => 'SNAPPY' #3 Currently I am consdring it and want to distribute data only based on rowkey create 'TEST_TABLE','si',{ NAME => 'si', COMPR

Re: Query on rowkey distribution || Does RS and number of Region related with each other

. Question is simple if I put salt (which can be any arbit char or genrated hash any thing) at the begning of the rowkey why my data not getting distributed Please note this is not pre splitted table. Thanks Manjeet Singh On Thu, Aug 30, 2018 at 9:11 PM Josh Elser wrote: > As I've been t

Query on rowkey distribution || Does RS and number of Region related with each other

Hi All, I have two Question *Question 1 : * I want to understand how rowkey distribution happen if I create my table with out applying any policy but opting prefix salting. Example I have rowkey like SALT_ID_DayStartTimestamp_DayEndTimeStamp_IDTimeStamp So it will look like as below *_99_1

Re: difference - filterlist with rowfilters vs multiget

n can improve performance. Thanks Manjeet singh On Wed, 15 Aug 2018, 14:37 Biplob Biswas, wrote: > Hi, > > During our implementation for fetching multiple records from an HBase > table, we came across a discussion regarding the best way to get records > out. > > The first i

Re: Query for OldWals and use of WAl for Hbase indexer

Hi Reid You suggest me to directly delete OldWals by using Hdfs command "hdfs -rm" Question is : It's really safe and I will not lose any data and Indexing from the system. Thanks Manjeet Singh On Thu, Jul 12, 2018 at 12:09 PM, Manjeet Singh wrote: > Hi > > I

Re: Query for OldWals and use of WAl for Hbase indexer

Hi I have created HBASE-20877 <https://issues.apache.org/jira/browse/HBASE-20877> for the same, request you to please move it into active sprint. Thanks Manjeet Singh On Thu, Jul 12, 2018 at 7:42 AM, Reid Chan wrote: > oldWals are supposed to be cleaned in master background chor

Re: Query for OldWals and use of WAl for Hbase indexer

I have one more question If solr is having its own data mean its maintaining data in their shards and hbase is maintaining in data folder... Why still oldWals need? Thanks Manjeet singh On Wed, 11 Jul 2018, 23:19 Manjeet Singh, wrote: > Thanks Sean for your reply > > I still

Re: Query for OldWals and use of WAl for Hbase indexer

sue due to this? Thanks Manjeet Singh On Wed, Jul 11, 2018 at 9:33 PM, Sean Busbey wrote: > Presuming you're using the Lily indexer[1], yes it relies on hbase's > built in cross-cluster replication. > > The replication system stores WALs until it can successfully send them &

Query for OldWals and use of WAl for Hbase indexer

5.2 T /hbase/data 42 126 /hbase/hbase.id 7 21 /hbase/hbase.version 7.2 T 21.6 T /hbase/oldWALs Thanks Manjeet Singh

Re: Incremental snapshot export

Hi All Does incremental snapshot export from source cluster to destination cluster is available..? Can anyone suggest me the best way to export/import entire table from source cluster to destination cluster (its live system) Thanks Manjeet singh On Thu, 19 Jan 2017, 06:31 Neelesh, wrote

Re: what is the role of ProtoBuffer while creating of instance of Table/Htable

on creation of Table object. In both case we are using Hbase Native API if Hbase internally using protobuff 2.5.0 so why my project 1 is working fine this question is comes to my mind Thanks Manjeet Singh -- Forwarded message -- From: Stack Date: Tue, Dec 5, 2017 at 3:36 AM

what is the role of ProtoBuffer while creating of instance of Table/Htable

this issue using protobuff 2.4.1 version only Thanks Manjeet Singh

Re: hbase data migration from one cluster to another cluster on different versions

org.apache.hadoop.hbase.mapreduce.Import test_table /data/ExportedFiles It's worked and data migrated. Thanks all Manjeet Singh sudo -u hdfs hbase -Dhbase.import.version=0.94 org.apache.hadoop.hbase.mapreduce.Import test_table /data/ExportedFiles On Thu, Oct 26, 2017 at 12:15 PM, anil gupta wrote: &

Re: hbase data migration from one cluster to another cluster on different versions

p 2.0 ) First I get HDFS files to local linux and use scp command to put them into destination cluster. Thanks Manjeet Singh On Thu, Oct 26, 2017 at 10:26 AM, Manjeet Singh wrote: > Hi Yung, > > First thanks for reply > The link provided by you is for upgrading the Hbase version and probl

Re: hbase data migration from one cluster to another cluster on different versions

(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs( UserGroupInformation.java:1693) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) can you please elaborate more about "Is the environment ready for the upgrade?" Thanks Manjeet Singh

Fwd: hbase data migration from one cluster to another cluster on different versions

org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) can anyone suggest how to migrate data? Thanks Manjeet Singh Hi All, I have query regarding hbase data migration from one cluster to another cluster in same N/W, but with a different version of hbase one is 0.94.27 (source cluster hbase) and

hbase data migration from one cluster to another cluster on different versions

: attempt_201710131340_0355_m_02_0, Status : FAILED Task attempt_201710131340_0355_m_02_0 failed to report status for 600 seconds. Killing! Thanks Manjeet Singh

Re: Please congratulate our new PMC Chair Misty Stanley-Jones

Congratulations Misty On 22 Sep 2017 12:39 am, "Andrew Purtell" wrote: > At today's meeting of the Board, Special Resolution B changing the HBase > project Chair to Misty Stanley-Jones was passed unanimously. > > Please join me in congratulating Misty on her new role! > > ​(If you need any help

Re: How to improve filter based scan

In fuzzyrowkey filter I must have to know Rowley right ... But I know only application name and want all subscribe On 22 Sep 2017 12:36 am, "Manjeet Singh" wrote: > Salt is one char and generated on subscriber number, we are using pre > splitting each salt is representing one

Re: How to improve filter based scan

Salt is one char and generated on subscriber number, we are using pre splitting each salt is representing one region On 22 Sep 2017 12:33 am, "Ted Yu" wrote: > Have you looked at FuzzyRowFilter (assuming salt is of fixed width) ? > > On Thu, Sep 21, 2017 at 11:5

Re: How to improve filter based scan

One more question does in hbase performance is vary based on filter? And does two or more filter can improve read scan or it increases read time? On 22 Sep 2017 12:29 am, "Manjeet Singh" wrote: > Hi > > We are using hbase 1.2 version > Our usecase is to get all the subsc

Re: How to improve filter based scan

Hi We are using hbase 1.2 version Our usecase is to get all the subscribe who have used xyz application for this we have rk as below Salt_subscriber_application_usecasenumber Example #__facebook_50 Thanks Manjeet singh On 22 Sep 2017 12:19 am, "Ted Yu" wrote: > bq. ho

How to improve filter based scan

ll me how can I take my scan in 1 or 2 second? I have tried page filter caching setmaxresult etc. Thanks Manjeet Singh

Re: [ANNOUNCE] New Apache HBase committer Ashu Pachauri

Congrats Ashu!! On Mon, Jun 19, 2017 at 1:09 PM, Karan Mehta wrote: > Congratulations Ashu!! > > On Mon, Jun 19, 2017 at 12:22 AM Anoop John wrote: > > > Congrats and welcome... > > > > -Anoop- > > > > On Mon, Jun 19, 2017 at 11:44 AM, Huaxiang Sun > wrote: > > > Congratulations! > > > > > > >

Re: Request region coprocessor endpoints from C++

You can use google protobuff to call java API frpm c++ , but for this you must have the Java code which will intract with Hbase. On Mon, May 22, 2017 at 9:55 PM, Cheyenne Forbes < cheyenne.osanu.for...@gmail.com> wrote: > Is there anyway I could attempt to do it? > > Regards, > Cheyenne O Forbes

Re: How to avoid update blocking

Hi all I am still looking if anyone can help? Thanks Manjeet On 7 Feb 2017 10:45, "Manjeet Singh" wrote: > > > How to avoid read/write update blocking in hbase and on what condition > Hbase block read/write? > > > > what should the frequency of Major compacti

How to avoid update blocking

How to avoid read/write update blocking in hbase and on what condition Hbase block read/write? what should the frequency of Major compaction ( I have daily data insert on average is 18 GB and 48 GB max, its with SNAPPY and FAST_DIFF) a. should it be on daily basis? b. should it be in weekend

Re: Hbase data version performance impact

> Take a look at TestVisibilityLabelsWithDeletes# > testDeleteColumnWithLatestTimeStampUsingMultipleVersions > around line 1368. > > FYI > > On Tue, Jan 31, 2017 at 9:58 AM, Manjeet Singh > > wrote: > > > Hi All > > > > can anyone tell me what is the Impact of using Hbase d

Hbase data version performance impact

Hi All can anyone tell me what is the Impact of using Hbase data version Q1 what is the impact if I have 1000 version set or what if its 1 Q2 Is it good idea to put non default version in different column family? Q3 Does version support to get specific version value? as showing below I wa

Re: Hbase Thrift | Support to call DAO written in java from C++/C#

I have read that tutorial but cross platform things not mentioned their On 24 Jan 2017 05:10, "Josh Elser" wrote: > Would strongly suggest you read the Apache Thrift documentation. > > Manjeet Singh wrote: > >> Thanks Josh for writing >> >> I have DAL writ

Re: Hbase Thrift | Support to call DAO written in java from C++/C#

or c++ (client) and server which will responsible to call my DAL PSI am using Java 8 Hbase 1.1.2 Thanks Manjeet On Mon, Jan 23, 2017 at 10:26 PM, Josh Elser wrote: > > > Manjeet Singh wrote: > >> >> I come to know about thrift but as per my understanding on thri

Hbase Thrift | Support to call DAO written in java from C++/C#

Hi All Can any buddy help me to call my java DAO (which is written in java) from c++ or c# code I come to know about thrift but as per my understanding on thrift client and server should be on same language thrift only give direct access to other language (correct me if I am wrong) if above stat

Re: How to store/get video files and image files in hbase?

above url only missing part is HColumnDescriptor hcd = new HColumnDescriptor(“f”); hcd.setMobEnabled(true); hcd.setMobThreshold(102400L); please any buddy tell if its ok? Thanks Manjeet On Sun, Jan 15, 2017 at 4:34 PM, Manjeet Singh wrote: > It could be 5 MB 15 MB and 50 MB differ

why Hbase indexer required REPLICATION_SCOPE => '1'

Hi All can any buddy tell why Hbase indexer required REPLICATION_SCOPE => '1' ? what its impact on cluster? does it take extra space? whats its internal working? - https://in.linkedin.com/in/manjeet-singh-b32023a <https://www.linkedin.com/public-profile/settings?tr

Re: Storing XML file in Hbase

I have same question but its related to storing vedio files more than 10 MB. Questions are: (1) how can I increase 10 MOB size? (2) what is the performance impact does any buddy have stats? (3) what is the best recommended size of MOB? (4) HColumnDescriptor hc = new HColumnDescriptor(“f”);

Re: How to store/get video files and image files in hbase?

t enough ? > > Cheers > > On Thu, Jan 12, 2017 at 9:33 AM, Manjeet Singh > > wrote: > > > Hi All, > > > > can any buddy help me to know How to store video files and image files in > > hbase? > > > > I have seen lots of blogs but not get exact e

Hbase Replication || Impact on cluster || Storage

Hi All, I have question regarding Hbase replication The clusters participating in replication can be of different sizes. The master cluster relies on randomization to attempt to balance the stream of replication on the slave clusters. It is expected that the slave cluster has storage capacity to h

How to store/get video files and image files in hbase?

Hi All, can any buddy help me to know How to store video files and image files in hbase? I have seen lots of blogs but not get exact example step by step. if any buddy able to share Java native code so it will be quite helpful Thanks Manjeet

Re: Hbase Support for C#/.Net

ant to you when looking for "best results"? #1 and > #2 have been what I've used in the past and both have worked quite well. > > -Dima > > On Mon, Jan 9, 2017 at 1:06 AM, Manjeet Singh > wrote: > > > Hi All, > > > > I have to find which is

Re: RPC calls in hbase

Hi.. Based on this little information Check you rowkey design along with what splitting policy you are using. Thanks Manjeet On Mon, Jan 9, 2017 at 3:20 PM, Rajeshkumar J wrote: > we have 7000 regions in which we only have data in 20 regions. Whether this > may be the reason for this? > >

Hbase Support for C#/.Net

Hi All, I have to find which is the best way to query on Hbase will give best result options are as below if any one can help 1. REST API 2. Using Thrift: 1. HBase and Thrift in .NET C# Tutorial http://pawelrychlicki.pl/Article/Details/52/hbase-and-thrift-in-net-c-sharp-tuto

Re: hbase monitoring SNMP

: > SNMP trap goes through UDP port. > > To my knowledge, hbase doesn't support SNMP natively. > > Suggest polling vendor's mailing list. > > On Thu, Jan 5, 2017 at 3:22 AM, Manjeet Singh > wrote: > > > Hi All > > > > We are using Cloudera Enterpri

Re: Major compaction questions

Hi Ganesh, As per my recent experience for adding node in Hbase cluster I have below observation. (1) First you need to take down time. (2) You need to balance your cluster by running HDFS blancer it will take 4 to 5 hours, here i am assuming your all data node having same h/w configurations. (3)

hbase monitoring SNMP

Hi All We are using Cloudera Enterprise edition which is not supporting SNMP support for Hbase and licence is quite expensive. does any one know any 3rd party tool which can support SNMP for Hbase. Does Hbase it self support for SNMP? if yes how can i send traps or poll on Region server. Thanks M

Re: Does Hbase recommend any limit in term of size for RowKey, column qualifyer and value

Thanks Ted for your reply On Wed, Jan 4, 2017 at 10:47 AM, Ted Yu wrote: > There is hbase.client.keyvalue.maxsize which defaults to 10485760 > > You can find its explanation > in hbase-common/src/main/resources/hbase-default.xml > > On Tue, Jan 3, 2017 at 9:09 PM, Manj

Does Hbase support to store MultiMedia files?

Hi All, I have question regarding, does hbase support to store content like Media files (audio, video, pictures etc) as I have read one blog I need to change hfile format version 3 hfile.format.version 3 If hbase support to store media more question so other things like region split, maj

Does Hbase recommend any limit in term of size for RowKey, column qualifyer and value

Hi All, I have question regarding does Hbase impose any limit in term of maximum size can not be increase? I have seen lots of blog all are saying that their is no limit it's depend on your design. but it must be clear, what is the exact calculation for knwing max limit of-> (a) rowkey (b) column

Re: Hbase Logs

mistake. -Manjeet On Fri, Dec 16, 2016 at 9:57 PM, Ted Yu wrote: > impetus-n519 is not accessible to other people on the mailing list. > > Can you clarify what you wanted to ask about these UI's ? > > Thanks > > On Fri, Dec 16, 2016 at 12:32 AM, Manjeet Singh < > man

Hbase Logs

Hi All, I have one basic question regarding Hbase log I want to see logs – (1) Hbase Master (2) Region server. If I perform any operation say get operation on rowkey it must be logged in some where I want to see that this is the rowkey and this this op

Re: [Query :] hbase rebalancing the data after adding new nodes in cluster

> > locality. > > > > For #2, no. hdfs rebalance is orthogonal to hbase. > > > > For #3, perform major compaction at earliest convenience. > > > > On Thu, Oct 6, 2016 at 11:47 PM, Manjeet Singh < > manjeet.chand...@gmail.com > > > > > wrote

Re: [Query :] hbase rebalancing the data after adding new nodes in cluster

; > > For #3, perform major compaction at earliest convenience. > > > > On Thu, Oct 6, 2016 at 11:47 PM, Manjeet Singh < > manjeet.chand...@gmail.com > > > > > wrote: > > > > > Hi All, > > > I have question on re balance, my query is how hbase

Re: Adding new node in cluster (Load balancer)

you can > re-enable it with the "balance_switch true" command from the hbase shell. > See also the "balancer" command from the shell. > > cheers, > esteban. > > > -- > Cloudera, Inc. > > > On Tue, Dec 6, 2016 at 9:19 AM, Manjeet Singh > wro

Re: Adding new node in cluster (Load balancer)

; See also the "balancer" command from the shell. > > cheers, > esteban. > > > -- > Cloudera, Inc. > > > On Tue, Dec 6, 2016 at 9:19 AM, Manjeet Singh > wrote: > > > Hi All, > > Can anyone help me if I want add new node of same configu

Re: Adding new node in cluster (Load balancer)

e balancer is off (false) then you can > re-enable it with the "balance_switch true" command from the hbase shell. > See also the "balancer" command from the shell. > > cheers, > esteban. > > > -- > Cloudera, Inc. > > > On Tue, Dec 6, 2016 at

Adding new node in cluster (Load balancer)

Hi All, Can anyone help me if I want add new node of same configuration in cluser, then how and what step I need to perform for distribution of data via load balancer. Thanks Manjeet

Hbase regions are balanced or not

Hi All, can anyone help me on 1) How can we determine if the regions are balanced or not? 2) Which regions are assigned to which region server? How is that controlled? Thanks Manjeet -- luv all

Hbase 1.2 connection pool in java

Hi All Cna anyone help me out on How to create Hbase connection pool in java also org.apache.hadoop.hbase.client.HConnection; is deprecated so is their any new approch? Thanks Manjeet -- luv all

Hbase HA and read replication

Hi All, I have one question about replication. I am saving my data in Hbase table latest Hbase version documentation said its coming with read replication. 1)- we also have hdfs replication at the end of the day Hbase data actually land on hdfs, so if I have replicated data by using hdfs why I r

Re: How Hbase perform distribution of rowkey in region server

a_data1, it corresponds to > exactly ONE region which is in exactly ONE RS. So the client reaches > that RS and region for this RK. > > -Anoop- > > On Mon, Nov 21, 2016 at 1:59 PM, Manjeet Singh > wrote: > > Hi All > > > > My question is very simple I have cr

How Hbase perform distribution of rowkey in region server

Hi All My question is very simple I have created my table using pre-split as showing below create 'test_table', 'CF1', SPLITS=> ['a', 'b', 'c'] and i have 4 region server can anyone help me how hbase distribute rowkey into the regio server please mind I am asking resion server not regions what

Re: How can I achieve HBase row level atomicity?

some complex logic behind? > > -Anoop- > > On Mon, Nov 7, 2016 at 11:09 PM, Manjeet Singh > wrote: > > Yes Anoop you are right > > my input source is kafka pipeline we have 7 spark ETL jobs which are > > responsible for aggregation, and to get last value from Hba

Re: How can I achieve HBase row level atomicity?

for same rowkey. > > > > Have you considered using: > > > > public boolean checkAndPut(final byte [] row, > > > > final byte [] family, final byte [] qualifier, final byte [] value, > > > > final Put put) > > > > Cheers > > >

what is the Hbase cluster requirement as per Industry standard

Hi All, I have below points if anyone answer it Q1. how many hardware core required by Hbase? what is the calculation at the end of the day one need to sure how much core it required? Q2. what is the RAM distribution calculation on each RS, Master, Java Heap, Client? please consider my require

Re: How can I achieve HBase row level atomicity?

trol is involved in answering your question. > > See http://hbase.apache.org/book.html#hregion.scans > > Cheers > > On Wed, Nov 2, 2016 at 10:57 AM, Manjeet Singh > > wrote: > > > Hi All > > > > I have ETL process for inserting data into hbase for this I have

How can I achieve HBase row level atomicity?

Hi All I have ETL process for inserting data into hbase for this I have spark jobs which are responsible for reading data from kafka topics. so my question is if I have some rowkey already exist in hbase and I have 3 spark job running and they all try to update on same rowkey how hbase deal for at

Re: Hbase Row key lock

e held for seconds. On Sun, Oct 23, 2016 at 12:57 PM, Manjeet Singh wrote: > Anil all information are correct I am talking about suppose I didn't set > any version and I have very simple requirement to update if I found xyz > record and if I hv few ETL process which are responsible

Re: Hbase Row key lock

to more than 1 in column family, then you will > be able to see both the values if you query for multiple versions. > > HTH, > Anil Gupta > > On Sun, Oct 23, 2016 at 12:02 PM, Manjeet Singh < > manjeet.chand...@gmail.com> > wrote: > > > Till now what i under

Re: Hbase Row key lock

MVCC mean any configuration for handlling multiple thread at same time? On Mon, Oct 24, 2016 at 12:24 AM, Manjeet Singh wrote: > No I don't have 50 clients? I want to understand internal working of Hbase > in my usecase I have bulk update operation from spark job we have 7 > different

Re: Hbase Row key lock

k why you're trying to update the same > row in such a manner. > > On Sunday, October 23, 2016, Manjeet Singh > wrote: > > > Hi Dima, > > > > I didn't get ? point is assume I have 50 different client all having same > > rowkey all want to update on sa

Re: Hbase Row key lock

> > (1) Write to Write-Ahead-Log (WAL) > > (2) Update MemStore: write each cell to the memstore > > (3) Release Row Lock > > > > > > SO question is how to solve this if I have very frequent update on Hbase > > > > Thanks > > Manjeet > > >

Re: Hbase Row key lock

SO question is how to solve this if I have very frequent update on Hbase Thanks Manjeet On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh wrote: > Hi All > > Can anyone help me about how and in which version of Hbase support Rowkey > lock ? > I have seen article about rowkey lock b

Re: Hbase native API Utility to test hbase table stats

08:05, "Ted Yu" wrote: > Is your method of getting stats for unique records similar to what > RowCounter > does ? > > Can you describe your requirement in bit more detail (I don't fully > understand from your initial email) ? > > Thanks > > O

Hbase native API Utility to test hbase table stats

Hi All I am using native API to test hbase table stats like unique records how many column qualifier size of hbase table etc. It woking fine with 1 lack or 2 but when record get increased its hang. Is their any reanimate utility? Or any buddy get chance to create it? Thanks Manjeet

Re: Hbase cluster not getting UP one Region server get down

gotta do a bit of troubleshooting > yourself first; a long dump of your machine specs doesn't change that. > > Can you describe what happened before/after the node went down? The log > just says server isn't running, so we can't tell much from that alone. > > -Dima

Re: [Query :] hbase rebalancing the data after adding new nodes in cluster

arc Spaggiari < jean-m...@spaggiari.org> wrote: > Hi Manjeet, > > Probably because your table is not really deleted. Can you "list" the > tables to confirm? For the balacing, just run a major compaction of your > table and locallity will come back. > > JMS

Re: [Query :] hbase rebalancing the data after adding new nodes in cluster

balance is orthogonal to hbase. >> >> For #3, perform major compaction at earliest convenience. >> >> On Thu, Oct 6, 2016 at 11:47 PM, Manjeet Singh < >> manjeet.chand...@gmail.com> >> wrote: >> >> > Hi All, >> > I have question on

Re: [Query :] hbase rebalancing the data after adding new nodes in cluster

it depends on whether major compaction is disabled. If major > compaction is enabled, timing of major compaction would affect the data > locality. > > For #2, no. hdfs rebalance is orthogonal to hbase. > > For #3, perform major compaction at earliest convenience. > > On Thu, Oct 6

Re: Hbase cluster not getting UP one Region server get down

er of Row in Hbase 11015719 Thanks Manjeet On Thu, Oct 20, 2016 at 10:45 AM, Manjeet Singh wrote: > Hi All > Can any one help me to figure out the root cause I have 4 node cluster and > one data node get down , I did not understand why my Hbase Master not able > to get up >

Hbase cluster not getting UP one Region server get down

Hi All Can any one help me to figure out the root cause I have 4 node cluster and one data node get down , I did not understand why my Hbase Master not able to get up I have belo log ERROR: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.h

HBase Tuning parameters and memory sizing

Hi All, I have question regarding how can i determine RAM sizing { RAM - include all required Parameters (1)Master (2) RS (3) Java heap (4) Java client etc} I have below configuration parameter can anyone help me to configure heap size and RAM for Hbase I have CDH 5.8.0 Hbase 1.2.1 we have

[Query :] hbase rebalancing the data after adding new nodes in cluster

Hi All, I have question on re balance, my query is how hbase rebalancing the data after adding new nodes in cluster ? 1.Do I need to explicitly rebalance hbase after adding the new node in cluster

Re: Query regarding pre-split Major compaction

, Manjeet Singh wrote: > Thanks Ted for your inputs > I have write some algorithm to convert my some String to single char like > # $ ! etc and its my salt so based on these I know whats my salt > as my input data was so random and I need to know in advance what is my > rowkey (Hash lik

Re: Query regarding pre-split Major compaction

e detail of your use would be helpful in providing better answer. > > Cheers > > On Fri, Sep 9, 2016 at 6:09 PM, Manjeet Singh > wrote: > > > Thanks Ted for links links will help to determine how region split what > > should be the size etc which will really helpful > >

Re: Query regarding pre-split Major compaction

ache.org/book.html#ops.capacity.regions.total > > On Fri, Sep 9, 2016 at 5:35 PM, Manjeet Singh > wrote: > > > Yeah its in weekdays > > Yeah default is 10 gb so what is the way/forumla to knw what shuld be the > > size of RS > > On 9 Sep 2016 19:03, "Ted Yu&q

Re: Query regarding pre-split Major compaction

lt is 10GB. > > If you know roughly how the key space would be filled, presplit your table > accordingly. > > On Thu, Sep 8, 2016 at 11:24 PM, Manjeet Singh > > wrote: > > > Hi All > > > > I have some basic question can anyone help me out > > > &g

Query regarding pre-split Major compaction

Hi All I have some basic question can anyone help me out Q1. this is my understanding To perform splitting I need to create table like below create 'test_table','c1', SPLITS=>['#", '!', '$''] and I have to design row key in this way #_123456789 !_123456789 $_123456789 so my data distributed on

Re: Avro schema getting changed dynamically

: "record", "name" : "twitter_schema", "namespace" : "com.miguno.avro", "fields" : [ { "name" : "username", "type" : "string", "doc" : "Name of the user account on Twitter.com" }, { &q

Avro schema getting changed dynamically

Hi All, I have use case to put data in avro format in Hbase , I have frequent read write operations but its not a problem. Problem is what if my avro schema get changed how shuld I deal with it? This should in mind what about older data which already inserted in Hbase and now we have new schema.

Re: Hbase Heap Size problem and Native API response is slow

trying to query; if compactions haven't occurred at the time that you try to do your reads, HBase may have to go to disk repeatedly to access HFiles, even when only accessing columns within one row. On Sat, Aug 27, 2016 at 11:12 AM, Manjeet Singh wrote: > Thanks Vladrodionov for your rep

Re: Hbase Heap Size problem and Native API response is slow

, "Vladimir Rodionov" wrote: > >> Problem is its very slow > > rows are not indexed by column qualifier, and you need to scan all of them. > I suggest you consider different row-key design or > add additional index-table for your table. > > -Vlad > > On

Re: Hbase Heap Size problem and Native API response is slow

rs1 = hTable.get(get1); int i = 0; for (KeyValue kv : rs1.raw()) { list.add(new String(kv.getQualifier()) + " "); } } catch (Exception e) { //System.out.println(e.getMessage()); } return list; } On Fri, Aug 26, 2016 at 7:56 PM, Manjeet Singh wrote: > Hi All > > I am using

Re: Hbase Heap Size problem and Native API response is slow

Adding I am getting below error on truncating the table [image: Inline image 1] On Fri, Aug 26, 2016 at 7:56 PM, Manjeet Singh wrote: > Hi All > > I am using wide table approach where I have might have more 1,00, > column qualifier > > I am getting problem as below >

Hbase Heap Size problem and Native API response is slow

Hi All I am using wide table approach where I have might have more 1,00, column qualifier I am getting problem as below Heap size problem by using scan on shell , as a solution I increase java heap size by using cloudera manager to 4 GB second I have below Native API code It took very long

Re: How to get Last 1000 records from 1 millions records

, Aug 25, 2016 at 12:36 PM, Manjeet Singh wrote: > I am using some logical salt say I have mobile number in my row key so I > am using some algo and fitting this mobile number into some ASCII char > So each time I know what will be the salt so its clear to me and it will > never chan

Re: How to get Last 1000 records from 1 millions records

. > > > On Wed, Aug 24, 2016 at 12:31 PM, Ted Yu wrote: > > > The following API should help in your case: > > > > public Scan setReversed(boolean reversed) { > > > > Cheers > > > > On Wed, Aug 24, 2016 at 12:05 PM, Manjeet Singh < > > ma

Re: How to Speed up Prefix scan on column qualifier

Get also support List object On Thu, Aug 25, 2016 at 12:32 AM, Ted Yu wrote: > Get is used to retrieve single row. > > If Get serves your need, you don't need PrefixFilter. > > On Wed, Aug 24, 2016 at 11:58 AM, Manjeet Singh < > manjeet.chand...@gmail.com> &g

How to get Last 1000 records from 1 millions records

Hi all Hbase didnt provide sorting on column but rowkey store in sorted form like small value first and greater value last example 1 2 3 4 5 6 7 and so on Assume I have 1 Miilions record but i want to look last 1000 records only Is their any way to do this? I don't want to perform any calculatio

Re: How to Speed up Prefix scan on column qualifier

(byte [] startRow) { > > > > On Wed, Aug 24, 2016 at 5:08 AM, Manjeet Singh < > manjeet.chand...@gmail.com > > > > > wrote: > > > > > Hi All, > > > > > > I have below code where I have row key like 9865327845_#RandomChar > >

  1   2   >