Re: "Quickstart" on OmniOS/illumos not working, java eats CPU

way too > > fast due the ZooKeeper server. One way to test this theory is to start > > each process on separate JVM and see how things behave. > > > > cheers, > > esteban. > > > > > > > > -- > > Cloudera, Inc. > > > > >

Re: "Quickstart" on OmniOS/illumos not working, java eats CPU

niOS mailing list. > > Olaf > > > On 13/11/2016 17:28, Ted Yu wrote: > >> bq. Caused by: java.lang.UnsatisfiedLinkError: no saproc in >> java.library.path >> >> Was the jstack of version 1.7.0_101 ? >> I used google and found: >> http://umchee.blo

Re: "Quickstart" on OmniOS/illumos not working, java eats CPU

t; Hope this can help pinpointing the issue. > > Olaf > > > > On 13/11/2016 01:18, Ted Yu wrote: > >> Can you turn on DEBUG logging ? >> In log4j.properties : >> >> log4j.logger.org.apache.hadoop.hbase=DEBUG >> >> Since this is reprodu

Re: how to use StochasticLoadBalancer

If you look at the source code, you would see ReadRequestCostFunction: private static final String READ_REQUEST_COST_KEY = "hbase.master.balancer.stochastic.readRequestCost"; private static final float DEFAULT_READ_REQUEST_COST = 5; and WriteRequestCostFunction : private

Re: "Quickstart" on OmniOS/illumos not working, java eats CPU

Can you turn on DEBUG logging ? In log4j.properties : log4j.logger.org.apache.hadoop.hbase=DEBUG Since this is reproducible, can you capture one or two stack traces before killing the server ? Thanks On Sat, Nov 12, 2016 at 3:09 PM, Olaf Marzocchi wrote: > Hello, > I

Re: Can connect to HBase via Java API but can't create table

Can you show more of the UnknownHostException ? Could this be related to the firewall on the Docker container ? Cheers On Mon, Nov 7, 2016 at 6:49 AM, Oliver Buckley-Salmon < oliver.buckley-sal...@db.com> wrote: > Hi > > I am trying to create a table in HBase using the Java API, I can connect

Re: Hbase 1.1.1 API documentation link

@Rajeshkumar > > Best Regards, > Yu > >> On 7 November 2016 at 13:06, Ted Yu <yuzhih...@gmail.com> wrote: >> >> I actually run the (modified) command in your first email quite often - >> when building tar balls for cluster testing. >> >> In that

Re: Hbase 1.1.1 API documentation link

was just copying from the book > and forgot to remove that execution :) > > > Ted Yu wrote: > >> If I am not mistaken, the 'assembly:single' goal is not needed for >> building >> the site. >> >> Cheers >> >> On Sun, Nov 6, 2016 at 12:57 PM,

Re: Hbase 1.1.1 API documentation link

If I am not mistaken, the 'assembly:single' goal is not needed for building the site. Cheers On Sun, Nov 6, 2016 at 12:57 PM, Josh Elser wrote: > Hi, > > I don't believe all versions of Javadoc are published on the website. > Something similar to the following should

Re: How can I achieve HBase row level atomicity?

me type of question regarding rowkey lock, > but still i did't get correct anser > > Thanks > Manjeet > > On Wed, Nov 2, 2016 at 11:36 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > Were you including code in the image (which didn't come through) ? > > > > M

Re: How can I achieve HBase row level atomicity?

Were you including code in the image (which didn't come through) ? MultiVersionConcurrencyControl is involved in answering your question. See http://hbase.apache.org/book.html#hregion.scans Cheers On Wed, Nov 2, 2016 at 10:57 AM, Manjeet Singh wrote: > Hi All > >

Re: HBase rest custom authentication

>From RESTServer.java : if (conf.get(REST_AUTHENTICATION_TYPE) != null) { containerClass = RESTServletContainer.class; FilterHolder authFilter = new FilterHolder(); authFilter.setClassName(AuthFilter.class.getName());

Re: Scanner timeouts

ibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > >

Re: Scanner timeouts

ssing in the empty > config to the spark-submit but it didn’t match the containers and fixing > that has made the system much happier. > > Anyway good call, we will be more aware of this with other services now. > Thanks for ending our weeks long fight! :-) > > > On Oct 28, 20

Re: Scanner timeouts

mage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > On 28 October 2016 at

Re: Scanner timeouts

ls is very long compared to other stages. And with 400 > threads hitting HBase at the same time, this seems like something I may > need to restructure and any advice about that would be welcome. > > HBase is 1.2.3 > > > On Oct 28, 2016, at 10:36 AM, Ted Yu <yuzhih...@gmail.c

Re: Scanner timeouts

and so must be > passed using --files to spark-submit? If so I can do this. > > But do I have a deeper issue? Is it typical to need a scan like this? Have > I missed indexing some column maybe? > > > On Oct 28, 2016, at 9:59 AM, Ted Yu <yuzhih...@gmail.com> wrote: > >

Re: Scanner timeouts

ase be liable for any monetary damages arising from > such loss, damage or destruction. > > > > On 28 October 2016 at 17:52, Pat Ferrel <p...@occamsmachete.com> wrote: > > > I will check that, but if that is a server startup thing I was not aware > I > > had to s

Re: Scanner timeouts

et to 6 > > Not sure where that is coming from. Does the driver machine making queries > need to have the timeout config also? > > And why so large, am I doing something wrong? > > > On Oct 28, 2016, at 8:50 AM, Ted Yu <yuzhih...@gmail.com> wrote: > > Mich: > The

Re: Scanner timeouts

Mich: The OutOfOrderScannerNextException indicated problem with read from hbase. How did you know connection to Spark cluster was lost ? Cheers On Fri, Oct 28, 2016 at 8:47 AM, Mich Talebzadeh wrote: > Looks like it lost the connection to Spark cluster. > > What

Re: Scanner timeouts

Which release of hbase are you using ? Since the query to hbase comes from Spark, I assume there is no hbase Filter involved. So HBASE-13704 wouldn't be applicable in your case. Can you pastebin region server log(s) around the OutOfOrderScannerNextException ? Thanks On Fri, Oct 28, 2016 at

Re: Graph search

Possibly related: http://search-hadoop.com/m/HBase/YGbbKdh132n5pF01?subj=Introducing+Project+S2Graph On Thu, Oct 27, 2016 at 10:06 AM, Dima Spivak wrote: > Hey Cheyenne, > > HBase itself only provides primitives for operations like put, get, and > scan, so you'd need to

Re: hBase - the server has too many connections (maxClientConn property set to 0 does not help)

The config parameters outlined previously should be added / modified in hbase-site.xml On Tue, Oct 25, 2016 at 4:35 AM, mohan wrote: > Hi, Can i know in which file i should add that properties.. > > > > -- > View this message in context: http://apache-hbase.679495.n3. >

Re: Spikes in HBase response time

Can you take a few jstack traces when this happens (and pastebin the traces) ? Do you use Phoenix in this cluster ? Thanks On Mon, Oct 24, 2016 at 5:44 AM, Alexander Ilyin wrote: > Hi, > > I'm experiencing some spikes in HBase response time. The problem appeared > in

Re: [In Memory] Questions about Make HBase Table In Memory

Puts are stored in memstore. Get / Scan would include such data in memstore when row keys match. For #2, see http://hbase.apache.org/book.html#block.cache.design For #3, one indication is latency of the result. Latency would be longer when disk is involved. On Sun, Oct 23, 2016 at 8:01 PM,

Re: Hbase Row key lock

Manjeet: Did you have a chance to get jstack during the lock contention period ? Cheers > On Oct 23, 2016, at 12:57 PM, Manjeet Singh > wrote: > > Anil all information are correct I am talking about suppose I didn't set > any version and I have very simple

Re: Hbase fast access

f data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > On 21 October 2016 at 21:46, Ted Yu <

Re: Hbase fast access

s, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > On 21 October

Re: mapreduce get error while update HBase

Can you give us more information so that we can match the line numbers in the stack trace with actual code ? release of hbase hadoop version If you can show snippet of related code, that would be nice. Thanks On Fri, Oct 21, 2016 at 11:16 AM, 乔彦克 wrote: > Hi, all > > I

Re: Hbase fast access

nsibility for any > > loss, damage or destruction of data or any other property which may arise > > from relying on this email's technical content is explicitly disclaimed. > > The author will in no case be liable for any monetary damages arising > from > > such loss, dam

Re: Hbase fast access

Mich: Here is brief description of hbase architecture: https://hbase.apache.org/book.html#arch.overview You can also get more details from Lars George's or Nick Dimiduk's books. HBase doesn't support SQL directly. There is no cost based optimization. Cheers > On Oct 21, 2016, at 1:43 AM,

Re: Hbase native API Utility to test hbase table stats

Is your method of getting stats for unique records similar to what RowCounter does ? Can you describe your requirement in bit more detail (I don't fully understand from your initial email) ? Thanks On Thu, Oct 20, 2016 at 5:48 PM, Manjeet Singh wrote: > Hi All > >

Re: Hbase Coprocessor postPut not triggered for

> > -Original Message- > From: Ted Yu [mailto:yuzhih...@gmail.com] > Sent: Monday, October 17, 2016 9:55 AM > To: user@hbase.apache.org > Subject: Re: Hbase Coprocessor postPut not triggered for > > Your scenario should be covered by unit tests already. > Ta

Re: Tips on Writing Custom HBase Filters

master branch is compiled with Java 8 where removeIf() is supported. For master branch, there is no need to use Guava, right ? On Thu, Oct 20, 2016 at 12:08 PM, Robert Yokota wrote: > Here are some tips on writing HBase filters in case anyone is interested: > >

Re: Doing map-reduce with Hive external table on Hbase throws error

I downloaded hive 2.0.1 source tar ball. In their pom.xml : 1.1.1 Can you run against 1.1.1 or newer hbase release ? On Thu, Oct 20, 2016 at 8:58 AM, Mich Talebzadeh wrote: > Hive 2.0.1 > Hbase 0.98 > > hive> select max(price) from test.marketdatahbase; > >

Re: Parallelizing TableSnapshotScanner

Can you come up with unit test that shows what you observed ? You may have seen the javadoc for ClientSideRegionScanner : * A client scanner for a region opened for read-only on the client side. Assumes region data * is not changing. Cheers On Thu, Oct 20, 2016 at 8:26 AM, Randy Fox

Re: HBase resgionServer crashed with no gc detected

But that make JVM pause ? > > > > > -- Original -- > From: "Ted Yu";<yuzhih...@gmail.com>; > Send time: Thursday, Oct 20, 2016 10:27 AM > To: "user@hbase.apache.org"<user@hbase.apache.org>; > > Subject: Re: HBase resgionServer crashed w

Re: [Query :] hbase rebalancing the data after adding new nodes in cluster

e "drop_all '.*'\ny" | hbase shell -n >>> >>> >>> Thanks >>> Manjeet >>> >>> On Thu, Oct 20, 2016 at 4:19 PM, Manjeet Singh < >> manjeet.chand...@gmail.com >>> wrote: >>> >>>> Hi

Re: HBase resgionServer crashed with no gc detected

ve posted the more detail RS,Gc log and the ZK ,HBase config, > https://github.com/eswidy/waterspider/tree/master/rscase > Thanks > > > > > -- Original ------ > From: "Ted Yu";<yuzhih...@gmail.com>; > Date: Oct 20, 2016

Re: java.lang.OutOfMemoryError when count hbase table

Storing the value on hdfs and using reference to the hdfs location in key value is an option. > On Oct 19, 2016, at 6:49 PM, big data wrote: > > actually, there is only one huge value in the hbase cell which large > than 100M, maybe it's not a good idea to store such

Re: HBase resgionServer crashed with no gc detected

o git hub ,and the url is : > https://github.com/eswidy/waterspider/blob/master/regionServer.log > > thanks so much. > > > > > -- Original -- > From: "Ted Yu";<yuzhih...@gmail.com>; > Date: Oct 19, 2016 > To: "user@hb

Re: HBase resgionServer crashed with no gc detected

y(10/18). > Details in the attachment 'regionServer.log', and the JVM pause at > "2016-10-17 18:44:07,232" in line 82 . > Thanks so much. > > > > > > -- 原始邮件 -- > *发件人:* "Ted Yu";<yuzhih...@gmail.com>; > *发送时间:* 201

Re: Re: Re: Re: What way to improve MTTR other than DLR(distributed log replay)

ons. > what I afraid is that, there are other issues I don't know yet. So if you > or other guys know other issues related to DLR, please let me know > > > Regards > Allan Yang > > > > > > > > At 2016-10-19 00:19:06, "Ted Yu" <yuzhih...@gmail.com> wrote: &

Re: Re: Re: What way to improve MTTR other than DLR(distributed log replay)

x > > > > > > > At 2016-10-18 13:47:17, "Anoop John" <anoop.hb...@gmail.com> wrote: > >Agree with ur observation.. But DLR feature we wanted to get removed.. > >Because it is known to have issues.. Or else we need major work to > >correct a

Re: Re: What way to improve MTTR other than DLR(distributed log replay)

> > > Regards > Allan Yang > > > > > > > At 2016-10-17 21:01:16, "Ted Yu" <yuzhih...@gmail.com> wrote: > >Here was the thread discussing DLR: > > > >http://search-hadoop.com/m/YGbbOxBK2n4ES12=Re+ > DISCUSS+retiring+current+DLR+code > &g

Re: HBase restart without region reassigning

Since you are using 1.1.2, you may want to look at HBASE-14531 which was fixed in 1.1.3 FYI On Mon, Oct 17, 2016 at 9:13 AM, Alexander Ilyin wrote: > I'm restarting it through Ambari. First time I specified a delay between > regionserver restarts, second time I didn't.

Re: Hbase Coprocessor postPut not triggered for

il.com] > Sent: Monday, October 17, 2016 2:06 AM > To: user@hbase.apache.org > Subject: Re: Hbase Coprocessor postPut not triggered for > > Ya what API used for this update op u mention? > > -Anoop- > > On Mon, Oct 17, 2016 at 9:25 AM, Ted Yu <yuzhih...@gmai

Re: What way to improve MTTR other than DLR(distributed log replay)

Here was the thread discussing DLR: http://search-hadoop.com/m/YGbbOxBK2n4ES12=Re+DISCUSS+retiring+current+DLR+code > On Oct 17, 2016, at 4:15 AM, allanwin wrote: > > Hi, All > DLR can improve MTTR dramatically, but since it have many bugs like > HBASE-13567, HBASE-12743,

Re: Hbase Coprocessor postPut not triggered for

Which release of hbase are you using ? How did you determine that coprocessor's postPut method is not triggered for the update ? By additional logging ? Can you give code snippet around the update ? Thanks On Sun, Oct 16, 2016 at 8:48 PM, Begar, Veena wrote: > Hello, > >

Re: Accessing Hbase tables through Spark, this seems to work

Please take a look at http://hbase.apache.org/book.html#_language_integrated_query The above is based on hbase-spark module which is part of the upcoming hbase 2.0 release. Cheers On Sun, Oct 16, 2016 at 11:37 AM, Mich Talebzadeh wrote: > Hi, > > I have trade data

Re: Modifying the existing column family and value in a Put

Looking at prePut / postPut methods, I think the answer is no. On Thu, Oct 13, 2016 at 7:46 AM, Nkechi Achara wrote: > Hi All, > > I am writing a transformation observer co-processor, and I would like to > know if there is a way to modify the family, and the value

Re: Support for oozie credentials in Hbase backup

Looking at the stack trace, it seems to be oozie issue. Have you mailed user@oozie ? Thanks On Wed, Oct 12, 2016 at 3:32 AM, mathes waran wrote: > Hi, > > > I have setup the Oozie credentials to connect secure HBase secure cluster > using Java action,, as per

Re: HBase acl commands are too slow

I recently worked on HBASE-16773 where 20 PriorityRpcServer handlers were all occupied accessing acl table. Can you take a few jstack of the server hosting acl table when acl command was executing ? Pastebin one or two stack trace. Please also check server log where acl table is hosted. Thanks

Re: HBase rest service did not respond in a secure cluster

For the first config parameter, why is thrift.keytab there ? > On Oct 10, 2016, at 10:24 PM, kumar r wrote: > > Hi, > > > HBase Version 1.1.5 > OS - Windows > > I have enabled HBase rest service with SSL and Proxy. > > When accessing HBase rest url, > >

Re: HBase rest service did not respond in a secure cluster

For hbase.rest.keytab.file, why is the value thrift.keytab ? Is it shared key tab ? On Mon, Oct 10, 2016 at 10:24 PM, kumar r wrote: > Hi, > > > HBase Version 1.1.5 > OS - Windows > > I have enabled HBase rest service with SSL and Proxy. > > When accessing HBase rest url, >

Re: Coprocessor exception in ResionServer log

> > why it can not running in dynamic load method? > > > >> 在 16/10/11 上午10:18, big data 写道: >> No, it is the original log from hbase, no missing. >> >> I'll try it through hbase-site.xml. >> >> thanks >> >> >> >>

Re: Coprocessor exception in ResionServer log

gt; [INFO] \- junit:junit:jar:4.12:test > [INFO]\- org.hamcrest:hamcrest-core:jar:1.3:test > [INFO] > ------------ > [INFO] BUILD SUCCESS > [INFO] > > [IN

Re: Coprocessor exception in ResionServer log

Could be related to incompatible protobuf versions. What's the output of: mvn dependency:tree Please pastebin it - it should be fairly long. On Mon, Oct 10, 2016 at 6:29 PM, big data wrote: > Dear all, > > I've created an Endpoint coprocessor, and deployed it through

Re: reading Hbase table in Spark

iable for any monetary damages arising from > such loss, damage or destruction. > > > > On 10 October 2016 at 22:49, Ted Yu <yuzhih...@gmail.com> wrote: > > > In hbase master branch, there is hbase-spark module which would allow you > > to integrate with Spark seamlessly.

Re: reading Hbase table in Spark

gt; > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary

Re: reading Hbase table in Spark

For org.apache.hadoop.hbase.client.Result, there is this method: public byte[] getValue(byte [] family, byte [] qualifier) { which allows you to retrieve value for designated column. FYI On Mon, Oct 10, 2016 at 2:08 PM, Mich Talebzadeh wrote: > Hi, > > I am

Re: Scan Performance Decreases Over Time

Have you taken jstack for the slow scans ? If so, can you pastebin the stack trace ? 1.0.0 is quite old. Any chance of upgrading to 1.2 release ? Cheers > On Oct 10, 2016, at 2:04 AM, 陆巍 wrote: > > Hi All, > > I met with a problem where the scan perfoamance decreases

Re: [Query :] hbase rebalancing the data after adding new nodes in cluster

For #1, it depends on whether major compaction is disabled. If major compaction is enabled, timing of major compaction would affect the data locality. For #2, no. hdfs rebalance is orthogonal to hbase. For #3, perform major compaction at earliest convenience. On Thu, Oct 6, 2016 at 11:47 PM,

Re: workaround for HBASE-16603 - Detect unavailability of hbase:backup table to avoid extraneous logging

Backup / restore feature hasn't been merged yet. If you use a vendor's distro, please use corresponding mailing list. On Fri, Sep 30, 2016 at 12:28 PM, mliem wrote: > Hello - > > Am i able to manually create hbase:backup table to avoid the extraneous > logging? I had to do

Re: [RegionServer Dead] Identify HBase Table Cause RegionServer Dead(Version 1.0.0-cdh5.5.2)

Server.java:2080) > > at org.apache.hadoop.ipc.Client.call(Client.java:1466) > at org.apache.hadoop.ipc.Client.call(Client.java:1403) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker. > invoke(ProtobufRpcEngine.java:230) > at com.sun.proxy.$Proxy23.crea

Re: Loading HBase table into HDFS

o the HDFS. how can i > achieve it ? > > >> On Wed, Sep 21, 2016 at 9:30 PM, Ted Yu <yuzhih...@gmail.com> wrote: >> >> By specifying "hbase.mapreduce.scan.column.family", you can export >> selected >> column family. >> >> On Wed, S

Re: [RegionServer Dead] Identify HBase Table Cause RegionServer Dead(Version 1.0.0-cdh5.5.2)

t; > 2. Migrate to 1.8 Java and switch to GCG1. Search "tuning GCG1 for HBase" > on the Internet > > 3. Analyze what has triggered long GC in RS. Usually it is some long > running M/R jobs or data import. Consider other approaches which are not so > intrusive (ex. bulk load

Re: [RegionServer Dead] Identify HBase Table Cause RegionServer Dead(Version 1.0.0-cdh5.5.2)

bq. the MutateCount metrics didn't show there is a peak time before and after the problem on that table In the graph, there was a purple line with spikes - I assume this was not for the table with max MemStoreSize. The legend in the graph is hard to read. If the sizes of the Puts vary, that may

Re: Increased response time of hbase calls

Which hbase release are you using ? Can you tell us the values for handler related config such as hbase.regionserver.handler.count ? How many regions does m7 have ? Have many servers does m7 span ? Are regions of m7 spread evenly ? Thanks ​

Re: Loading HBase table into HDFS

By specifying "hbase.mapreduce.scan.column.family", you can export selected column family. On Wed, Sep 21, 2016 at 8:49 AM, sudhir patil wrote: > You can use export command, but not sure if u can export just one column > family

Re: Loading HBase table into HDFS

Can you clarify your scenario ? Normally hbase is backed by hdfs - the table is already stored on hdfs. Cheers On Wed, Sep 21, 2016 at 4:46 AM, karthi keyan wrote: > Can any one please guide me to load the HBase table in to HDFS with > specific columnfamily. > >

Re: Fast Bulk Load

Please take a look at: hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/BulkLoadSuite.scala where usage of LoadIncrementalHFiles is demonstrated. This is in master branch of hbase. On Mon, Sep 19, 2016 at 12:10 PM, Punit Naik wrote: > Hi Guys > > I am currently

Re: Table export fails with NPE

, Sep 18, 2016 at 11:34 AM, Krishna <research...@gmail.com> wrote: > I will try that. And when inserting KeyValues, how would I set CellType? > > > On Sunday, September 18, 2016, Ted Yu <yuzhih...@gmail.com> wrote: > > > If you have bandwidth, you can try the

Re: Table export fails with NPE

), timestamp, field.getBytes())); > > On Sat, Sep 17, 2016 at 1:04 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > Here is related code from CellProtos.java : > > > > public Builder > > setCellType(org.apache.hadoop.hbase.protobuf.generated. > Cel

Re: Table export fails with NPE

Here is related code from CellProtos.java : public Builder setCellType(org.apache.hadoop.hbase.protobuf.generated.CellProtos.CellType value) { if (value == null) { throw new NullPointerException(); This means CellType.valueOf() returned null for the Cell. Which release

Re: copyTable mapreduce job failing with ScannerTimeoutException

You can examine /jmx : http://search-hadoop.com/m/YGbb3E2a71UVLBK=Re+HBase+Count+Rows+in+Regions+and+Region+Servers Looks like your load consists of both write and read. Have you turned on bucket cache ? http://hbase.apache.org/book.html#offheap.blockcache On Fri, Sep 9, 2016 at 1:58 PM,

Re: Query regarding pre-split Major compaction

Singh <manjeet.chand...@gmail.com> wrote: > Yeah its in weekdays > Yeah default is 10 gb so what is the way/forumla to knw what shuld be the > size of RS > On 9 Sep 2016 19:03, "Ted Yu" <yuzhih...@gmail.com> wrote: > > > Can you clarify whether the incoming

Re: copyTable mapreduce job failing with ScannerTimeoutException

The 'Above memstore limit' warning meant that your region server(s) was under pressure of the write load. Can you share memstore related config parameters ? Did you observe hot spotting in the region server(s) ? Cheers On Fri, Sep 9, 2016 at 1:20 PM, marjana wrote: >

Re: copyTable mapreduce job failing with ScannerTimeoutException

How long was your mapreduce job ? You may need to check log for map tasks to get more information. Thanks On Fri, Sep 9, 2016 at 11:17 AM, marjana wrote: > I haven't tried that, afraid of how it will affect client connections. > Any idea why it fails at the very end? > >

Re: copyTable mapreduce job failing with ScannerTimeoutException

Have you tried increasing the value of hbase.client.scanner.timeout.period (default 6) ? On Fri, Sep 9, 2016 at 11:06 AM, marjana wrote: > Hi, > I am trying to copy a table from one cluster to another. This worked fine > for smaller tables, but trying to copy a few of

Re: [DISCUSS] Drop the support of jdk7 at a future 1.x release

Yeah, we should keep support for Java 7 in branch-1. We can use CompletableFuture for 2.0 release. On Thu, Sep 8, 2016 at 8:56 PM, Andrew Purtell wrote: > I think we should wait until 2.0 before dropping support for less than JDK > 8. That's a pretty big deal. But,

Re: Maximum limit on HBase cluster size

gment of a column > family.". > > This number 50-100 regions per table at the level of individual region > server or for the entire cluster ? > > Thanks, > Sreeram > > > > > > On Wed, Sep 7, 2016 at 4:18 PM, Ted Yu <yuzhih...@gmail.com> wro

Re: Maximum limit on HBase cluster size

With properly designed schema, you don't need to split the cluster. Please see: http://hbase.apache.org/book.html#schema > On Sep 7, 2016, at 1:59 AM, Sreeram wrote: > > Dear All, > > > > Looking forward to your views on the maximum limit of HBase cluster size. > > >

Re: [ANNOUNCE] Duo Zhang (张铎) joins the Apache HBase PMC

Congratulations, Duo. > On Sep 6, 2016, at 9:26 PM, Stack wrote: > > On behalf of the Apache HBase PMC I am pleased to announce that 张铎 > has accepted our invitation to become a PMC member on the Apache > HBase project. Duo has healthy notions on where the project should be >

Re: [from EMC Isilon] -- pls. review "Why HBase on EMC Isilon" post

Interesting. Minor correction: bq. The locations of all files and regions are kept in a special metadata table “*hbase:meta*” The locations of hfiles are not tracked in hbase:meta On Thu, Sep 1, 2016 at 1:52 AM, Chernov, Arseny wrote: > Dear colleagues at User@HBase

Re: Multirange Scan performance

Can you take a look at TestMultiRowRangeFilter to see if your usage is different ? It would be easier if you pastebin snippet of your code w.r.t. MultiRowRangeFilter. Thanks On Tue, Aug 30, 2016 at 8:29 AM, daunnc wrote: > Hi HBase users. I'm using HBase with Spark; > What I

Re: Avro schema getting changed dynamically

Probably you can poll user@avro for how the new field is handled given old data. FYI On Mon, Aug 29, 2016 at 11:28 PM, Manjeet Singh wrote: > I want ot add few more points > > I am using Java native Api for Hbase get/put > > and below is the example > > assume i

Re: HBase for Small Key Value Tables

, > > Where do we set this value DEFAULT_TABLE_SKEW_COST = 35. I see it in only > in StochasticLoadBalancer.java > We don't find this in any of the HBase Config files. Do we need to re-build > HBase from code for this? > > Thanks, > Manish > >> On Tue, Aug 30,

Re: RegionServer shutdown by some unknown reason.

Please use user@ in the future. You said: zk session timout is 40s Default value is 90s. Why did you configure it with lower value ? The "RegionServer ephemeral node deleted" message means that znode for olap3.data.lq,16020,1470799848293 expired. Can you pastebin JVM parameters (are you using

Re: HBase for Small Key Value Tables

gs and see how it goes. Will this also make sure that the > region migrates to another region server? Or do we still need to do that > manually? > > On JMX, Since the environment is production, we are yet unable to use jmx > for stats collection. But in dev we are trying it out. > >

Re: HBase for Small Key Value Tables

bq. We cannot change the maxregionsize parameter The region size can be changed on per table basis: hbase> alter 't1', MAX_FILESIZE => '134217728' See the beginning of hbase-shell/src/main/ruby/shell/commands/alter.rb for more details. FYI On Sun, Aug 28, 2016 at 10:44 PM, Manish Maheshwari

Re: HBase for Small Key Value Tables

Cycling old bits: http://search-hadoop.com/m/YGbb3E2a71UVLBK=Re+HBase+Count+Rows+in+Regions+and+Region+Servers You can use /jmx to inspect regions and find the hotspot. On Mon, Aug 29, 2016 at 7:29 AM, Manish Maheshwari wrote: > Hi Dima, > > Thanks for the suggestion. We

Re: HBase Region Size of 2.5 TB

; please let me know, > > Thanks, > Yeshwanth > > > > On Fri, Aug 26, 2016 at 5:41 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > From IncreasingToUpperBoundRegionSplitPolicy#configureForRegion(): > > > > initialSize = conf.getLong(

Re: Issues with Spark On Hbase Connector

For hortonworks product(s), consider raising question on https://community.hortonworks.com FYI On Sun, Aug 28, 2016 at 6:45 PM, spats wrote: > Regarding hbase connector by hortonworks > https://github.com/hortonworks-spark/shc, it would be great if someone can > answer

Re: HBase Region Size of 2.5 TB

>From IncreasingToUpperBoundRegionSplitPolicy#configureForRegion(): initialSize = conf.getLong("hbase.increasing.policy.initial.size", -1); ... if (initialSize <= 0) { initialSize = 2 * conf.getLong(HConstants.HREGION_MEMSTORE_FLUSH_SIZE,

Re: HBase - Count Rows in Regions and Region Servers

a Map<EncodedRegionName, SizeInBytes> > > > > Then you can use the RegionLocator to find which region resides on which > > machine. > > > > That will tell you the overall skew of your data in terms of raw bytes. > > > > Should be a pretty decent estim

Re: HBase - Count Rows in Regions and Region Servers

ity for restart. In general the > impact on JMX would be less than 2-3% on HBase performance? > > Thanks, > Manish > > > On Fri, Aug 26, 2016 at 12:11 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > Have you looked at /jmx endpoint on the servers ? > >

Re: HBase - Count Rows in Regions and Region Servers

lt_table_x_region_66bbec5f7e136b226a19b5fdf9f17cbe_metric_incrementCount" : 0, On Fri, Aug 26, 2016 at 11:59 AM, Manish Maheshwari <mylogi...@gmail.com> wrote: > Hi Ted, > > I understand the region crash/migration/splitting impact. Currently we have > hotspotting on few region s

Re: HBase - Count Rows in Regions and Region Servers

Can you elaborate on your use case ? Suppose row A is on server B, after you retrieve row A, the region for row A gets moved to server C (load balancer or server crash). Server B would no longer be relevant. Cheers On Fri, Aug 26, 2016 at 10:07 AM, Manish Maheshwari wrote:

Re: Hbase Heap Size problem and Native API response is slow

Looks like the image didn't go through. Can you pastebin the error ? Cheers On Fri, Aug 26, 2016 at 7:28 AM, Manjeet Singh wrote: > Adding > I am getting below error on truncating the table > > [image: Inline image 1] > > On Fri, Aug 26, 2016 at 7:56 PM, Manjeet

<    1   2   3   4   5   6   7   8   9   10   >