Re: Hbase not taking inserts from Remote Machine

2015-02-16 Thread Serega Sheypak
You need to open region server ports. Client directly sends put to appropriate region server. вторник, 17 февраля 2015 г. пользователь Vineet Mishra написал: > -- Forwarded message -- > From: Vineet Mishra > > Date: Tue, Feb 17, 2015 at 12:32 PM > Subject: Hbase not taking inserts

Fwd: Hbase not taking inserts from Remote Machine

2015-02-16 Thread Vineet Mishra
-- Forwarded message -- From: Vineet Mishra Date: Tue, Feb 17, 2015 at 12:32 PM Subject: Hbase not taking inserts from Remote Machine To: cdh-u...@cloudera.org Hi All, I am trying to communicate and insert some data to my Hbase(0.98.6-cdh5.3.0) running on Hadoop 2.5 using Hbase

Re: Does hbase WAL ensures no data loss?

2015-02-16 Thread hongbin ma
​Thanks everyone, I was being careless​ On Tue, Feb 17, 2015 at 3:22 AM, Andrew Purtell wrote: > cated users, but those users have to ask for special > handling by setting a flag > . In general a > ​user​ > should never submit mutations > ​asking for them >

Re: managing HConnection

2015-02-16 Thread Serega Sheypak
Hi, I'm closing it in servlet.destroy. I didn't see any problems here for months. I'm using servlet lifecycle to deal with hconnection. вторник, 17 февраля 2015 г. пользователь Liu, Ming (HPIT-GADSC) написал: > Hi, > > Thank you Serega for the helpful reply and thanks Jneidi for asking this. > I

RE: HTable or HConnectionManager, how a client connect to HBase?

2015-02-16 Thread Liu, Ming (HPIT-GADSC)
Hi, I have to spend a lot of time to look into the source code of HTable, HConnectionManager. IMHO, it seems the document on hbase website is misleading. In the hbase online document : http://hbase.apache.org/book.html#architecture.client . It mentioned:

RE: managing HConnection

2015-02-16 Thread Liu, Ming (HPIT-GADSC)
Hi, Thank you Serega for the helpful reply and thanks Jneidi for asking this. I have similar confusion. So Serega, when does your application finally close the HConnection? Or the connection is NEVER closed as long as your application is running? Is it OK to NOT close the HConnection and the a

Re: PerformanceEvaluation: filterScan

2015-02-16 Thread Jerry He
Hi, I was on 0.98 running PerformanceEvaluation. randomRead or sequentialRead all show good read counts on the table regions. filterScan shows exactly the same as you had. Looking at the code. It could be as expected. The filter has no matching returns, so the region server is in nextInternal

Re: Does hbase WAL ensures no data loss?

2015-02-16 Thread Andrew Purtell
> What is hbase's philosophy in this? Does it allow some degree of data loss? ​HBase doesn't "allow" data loss, in the sense that HBase never chooses on its own to be less than fully durable. However, our client API does allow users to submit mutations with different durability guarantees. The def

Re: Help needed on choosing OCR software

2015-02-16 Thread anil gupta
Hi Rams, I don't think HBase mailing list is appropriate to search for an OCR. Please use appropriate mailing list. ~Anil On Mon, Feb 16, 2015 at 5:27 AM, hongbin ma wrote: > I used to came across this: https://code.google.com/p/tesseract-ocr/ > AFAIK, OCR requires training if you want to get

Re: Q regarding thrift server to expose RDD via SQL

2015-02-16 Thread Nick Dimiduk
Using TableInputFormat directly will have better scalability than HS2. Better still, use TableSnapshotInputFormat to work from a snapshot (since RDDs are immutable anyway). -n On Monday, February 16, 2015, Marco wrote: > Hi, > > I've played with the feature to expose RDD via Thrift to enable JD

Re: ColumnSuffixFilter in HBase

2015-02-16 Thread anil gupta
Hi Alok, "xyz$" regex is enough to get the desired result. It will perform better than "^.*xyz$". "^.*xyz$" is doing some unnecessary work. Thanks, Anil Gupta On Wed, Feb 11, 2015 at 10:44 AM, Alok Singh wrote: > A simple String suffix check will be definitely faster, but I doubt > you will no

Re: Region balancing query

2015-02-16 Thread Abe Weinograd
balancer said "true" and it is not disabled. Thanks again for your help. Abe On Mon, Feb 16, 2015 at 11:23 AM, Ted Yu wrote: > What was the output from 'balancer' command ? > > Was is possible that balancer was disabled ? > > Cheers > > On Mon, Feb 16, 2015 at 8:04 AM, Abe Weinograd wrote: >

Q regarding thrift server to expose RDD via SQL

2015-02-16 Thread Marco
Hi, I've played with the feature to expose RDD via Thrift to enable JDBC access. (Spark 1.2) val eventsView = sqlContext.createSchemaRDD(eventSchemaRdd) eventsView.registerTempTable("Events") HiveThriftServer2.startWithContext(sqlContext) This works all fine. Now, my understanding is yo

Re: Region balancing query

2015-02-16 Thread Ted Yu
What was the output from 'balancer' command ? Was is possible that balancer was disabled ? Cheers On Mon, Feb 16, 2015 at 8:04 AM, Abe Weinograd wrote: > Ok. I forced the balancer run and am not seeing anything after a few > minutes. Master logs isn't showing anything. Should I look at the

Re: Region balancing query

2015-02-16 Thread Abe Weinograd
Ok. I forced the balancer run and am not seeing anything after a few minutes. Master logs isn't showing anything. Should I look at the RS ones instead? On Mon, Feb 16, 2015 at 11:03 AM, Ted Yu wrote: > You should see effect in the next balancer run. > > Cheers > > On Mon, Feb 16, 2015 at 7:52

Re: Region balancing query

2015-02-16 Thread Ted Yu
You should see effect in the next balancer run. Cheers On Mon, Feb 16, 2015 at 7:52 AM, Abe Weinograd wrote: > Excellent. If i trigger the balancer, should this start to help or only for > future region creation? > > Thanks, > Abe > > On Mon, Feb 16, 2015 at 9:35 AM, Ted Yu wrote: > > > Yes. T

Re: Region balancing query

2015-02-16 Thread Abe Weinograd
Excellent. If i trigger the balancer, should this start to help or only for future region creation? Thanks, Abe On Mon, Feb 16, 2015 at 9:35 AM, Ted Yu wrote: > Yes. This setting should be modified on Master. > > Cheers > > On Mon, Feb 16, 2015 at 6:27 AM, Abe Weinograd wrote: > > > Thanks Ted

PerformanceEvaluation: filterScan

2015-02-16 Thread Jean-Marc Spaggiari
Any one tries this recently? hbase org.apache.hadoop.hbase.PerformanceEvaluation --nomapred --rows=100 filterScan 3 I let it run for 30 minutes and killed it. 0 requets against the table, 649 requests against the META. Seems to not be working anymore. Did not get a chance to look at the code yet

Re: Region balancing query

2015-02-16 Thread Ted Yu
Yes. This setting should be modified on Master. Cheers On Mon, Feb 16, 2015 at 6:27 AM, Abe Weinograd wrote: > Thanks Ted. We are putting this in the hbase-site.xml for the Master? > > Abe > > On Fri, Feb 13, 2015 at 5:03 PM, Shahab Yunus > wrote: > > > Thanks, we will try that and report bac

Re: Does hbase WAL ensures no data loss?

2015-02-16 Thread Ted Yu
Hongbin: Please go through HRegion#doMiniBatchMutation(). You will see the connection of wal.append() and the WAL sync in step 7. Cheers On Mon, Feb 16, 2015 at 6:05 AM, Dave Latham wrote: > Hi Hongbin, > > The WAL class is used internally to the region server. Typically an HBase > write oper

Re: Region balancing query

2015-02-16 Thread Abe Weinograd
Thanks Ted. We are putting this in the hbase-site.xml for the Master? Abe On Fri, Feb 13, 2015 at 5:03 PM, Shahab Yunus wrote: > Thanks, we will try that and report back. > > Regards, > Shahab > > On Fri, Feb 13, 2015 at 4:56 PM, Ted Yu wrote: > > > You can make TableSkewCostFunction more pro

Re: Does hbase WAL ensures no data loss?

2015-02-16 Thread Dave Latham
Hi Hongbin, The WAL class is used internally to the region server. Typically an HBase write operation will first call WAL.append() with the data, then later, after releasing locks, call WAL.sync() to ensure that the data for that write has been synced to be durable before returning to the client

Re: Help needed on choosing OCR software

2015-02-16 Thread hongbin ma
I used to came across this: https://code.google.com/p/tesseract-ocr/ AFAIK, OCR requires training if you want to get a high quality recognition. and it's not easy to have a model that suits all styles of hand writings On Mon, Feb 16, 2015 at 7:33 PM, N. Ramasubramanian < ramasubramanian.naraya...@

Does hbase WAL ensures no data loss?

2015-02-16 Thread hongbin ma
hi, all It seems WAL.append() in hbase, the javadoc says: * * Append a set of edits to the WAL. The WAL is not flushed/sync'd after this transaction* * * completes BUT on return this edit must have its region edit/sequence id assigned* * * else it messes up our unification of mvcc and sequenc

Re: Streaming data to htable

2015-02-16 Thread hongbin ma
Hi Andrey, thanks for your reply, I found them very inspiring. I'm curious about the scheme you described: It looks to me you've sharded your data into separate regions, are you fixing the number of these regions? If yes, each of the growing region will spawn lots of compactions while writing da

Re:Re: Re: managing HConnection

2015-02-16 Thread David chen
In your scene, per servlet used a HConnecton object, so the response for 5 rpm is so easy. If these servlets should share the only one HConnecton object, whether or not the response latency for 5 rpm would increase?

Re: Re: managing HConnection

2015-02-16 Thread Serega Sheypak
Newrelic shows 50K RPM each request to servlet == 1-3 put/get to HBase. I have mixed workload. Is it strange :) ? 2015-02-16 10:37 GMT+03:00 David chen : > 5 rpm? I am curious how the result is concluded?