Re: Monitor off heap Bucket Cache

Hi Dejan, On Mon, Jun 1, 2015 at 5:24 AM, Dejan Menges wrote: > Hi, > > What's the best way to monitor / know how's bucket cache being used, how > much stuff is cached there, etc? > Have a look at these: https://apps.sematext.com/spm-reports/s/ivzcFizKhG https://apps.sematext.com/spm-reports/s/

Re: Hbase vs Cassandra

Hi Ajay, You won't be able to get unbiased opinion here easily. You'll need to try and see how each works for your use case. We use HBase for the SPM backend and it has worked well for us - it's stable, handles billions and billions of rows (I lost track of the actual number many moons ago) and

Re: Monitoring tools for Hbase

Hi Siva, Have a look at SPM . It captures a pile of HBase metrics , has built in alerting, anomaly detection, event and log correlation, even transaction tracking. Here's one of the interesting HBase metric

Re: HBase read/write statistics per table basis

Hi, I *think* these metric are not available on a per-table basis otherwise we'd have it in SPM for HBase , and I know we don't. If these are available somewhere, I'm all eyeballs! Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Ela

Re: Poll: HBase usage by HBase version

Mar 12, 2015 at 10:45 AM, Otis Gospodnetic < otis.gospodne...@gmail.com> wrote: > Hi, > > Let's see where we are! :) > > 1-question poll: > http://blog.sematext.com/2015/03/11/hbase-poll-version/ > > Btw: > 6 months ago the guess was that HBase version usa

Re: Poll: HBase usage by HBase version

ban. > > > > > > > -- > Cloudera, Inc. > > > On Thu, Mar 12, 2015 at 7:45 AM, Otis Gospodnetic < > otis.gospodne...@gmail.com> wrote: > > > Hi, > > > > Let's see where we are! :) > > > > 1-question poll: > > http://blo

Poll: HBase usage by HBase version

Hi, Let's see where we are! :) 1-question poll: http://blog.sematext.com/2015/03/11/hbase-poll-version/ Btw: 6 months ago the guess was that HBase version usage distribution was: 60% 0.94 30% 0.96 10% 0.98 Another person said: *Now that 0.96 has been EOL'd you should see users migrating to 0.9

Re: Region-level metrics being phased out?

27;d like us look at Otis? > St.Ack > > > > On Wed, Feb 18, 2015 at 12:37 PM, Otis Gospodnetic < > otis.gospodne...@gmail.com> wrote: > > > Hi, > > > > Are Region-level metrics here to stay? > > > > I'm asking because I remember seeing an

Region-level metrics being phased out?

Hi, Are Region-level metrics here to stay? I'm asking because I remember seeing an email or JIRA comment (from Stack?) from a fe wmonths ago about some Region-level metrics being removed ... maybe that was just for one specific Region-level metric, but in general Region-level metrics are her

Re: How can i get metrics logs for example compactionqueuesize

Hi, I think you are asking how you can get this metric from HBase and store it in a file? You can always use something like HBase JMX API or something like JMXC to get metrics, grep the output, extract the metric value, and store to the file. Otis -- Monitoring

Re: Does 'online region merge' make regions unavailable for some time?

to the proper location in the filesystem. That's why > the whole process is fast. > > Cheers > > On Wed, Jan 21, 2015 at 7:58 PM, Otis Gospodnetic < > otis.gospodne...@gmail.com> wrote: > > > Hi, > > > > On Wed, Jan 21, 2015 at 10:26 PM, Ted Yu w

Re: Does 'online region merge' make regions unavailable for some time?

will block/buffer/delay reads and writes until the regions are merged and will then let them through ? Thanks, Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.com/ On Wed, Jan 21, 2015 at 7:09 PM, Otis Gospodnet

Re: Does 'online region merge' make regions unavailable for some time?

61887/merge%20region.pdf > > > > It is clear that the two regions to be merged are taken offline in step > 1. > > > > Cheers > > > > On Tue, Jan 20, 2015 at 5:26 PM, Otis Gospodnetic < > > otis.gospodne...@gmail.com> wrote: > > > > > Hi, &

Re: Does 'online region merge' make regions unavailable for some time?

Hi, Considering this is called the *online* region merge, I would assume regions being merged never go offline during the merge and both regions being merged are available for reading and writing at all times, even during the merge though I don't get how writes would work if one region is bein

Re: 1 table, 1 dense CF => N tables, 1 dense CF ?

rocessor really be used for that? Also, would that come into play during minor or major compactions or both? Thanks, Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.com/ > > -n > > On Wed, Jan 7, 2015

Re: 1 vs. N CFs, dense vs. sparse CFs, flushing

arch Support * http://sematext.com/ On Wed, Jan 7, 2015 at 12:31 PM, Ted Yu wrote: > Please see HBASE-10201 which would come in 1.1.0 release. > > Cheers > > On Wed, Jan 7, 2015 at 9:10 AM, Otis Gospodnetic < > otis.gospodne...@gmail.com > > wrote: > > > Hi,

Re: Region Server OutOfMemory Error

Tue, Jan 6, 2015 at 11:42 PM, Shuai Lin wrote: > Cool, how can I get a graph like that? > > On Wed, Jan 7, 2015 at 4:06 AM, Otis Gospodnetic < > otis.gospodne...@gmail.com > > wrote: > > > Hi, > > > > The first thing I'd want to know is which memory p

1 table, 1 dense CF => N tables, 1 dense CF ?

Hi, It's been asked before, but I didn't find any *definite* answers and a lot of answers I found via are from a whle back. e.g. Tsuna provided pretty convincing info here: http://search-hadoop.com/m/xAiiO8ttU2/%2522%2522I+generally+recommend+to+stick+to+a+single+table%2522&subj=Re+One+table

1 vs. N CFs, dense vs. sparse CFs, flushing

Hi, I recently came across this good thread about 1 vs. N ColumnFamilies, the max recommended number of CFs, dense vs. sparse structure, etc. -- http://search-hadoop.com/m/TozMw1jqh262 This thread is from 2013. Even though people say HBase should handle more than 3 CFs, the docs still recommend t

Re: Region Server OutOfMemory Error

Hi, The first thing I'd want to know is which memory poor is getting filled. There are several in the JVM. Here's an example: https://apps.sematext.com/spm-reports/s/kZgBWLsJRd (this one is actually from an HBase cluster). If you see any of the lines at 100% that's potential trouble. If it stays

Re: how to tell there is a OOM in regionserver

ike this > > # java.lang.OutOfMemoryError: Java heap space > > # -XX:OnOutOfMemoryError="kill -9 %p" > > # Executing /bin/sh -c "kill -9 "... > > > > On Tue, Dec 2, 2014 at 11:06 AM, Otis Gospodnetic < > otis.gospodne...@gmail.com> wrote: > >

Re: how to tell there is a OOM in regionserver

Hi Ming, 1) There typically is an OOM message from the JVM itself 2) I would monitor the server instead of relying on log messages mentioning OOMs. For example, in SPM we have "hearbeat alerts" that tell us when we stop hearing from RegionServers and other types of ser

Re: Newbie Question about 37TB binary storage on HBase

Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.com/ > Am 28.11.2014 um 06:37 schrieb Otis Gospodnetic: > > Hi, > > > > There was a mention of Elasticsearch here that caught my attention. > > We use both HBase and Elasticsearch

Re: Newbie Question about 37TB binary storage on HBase

Hi, There was a mention of Elasticsearch here that caught my attention. We use both HBase and Elasticsearch at Sematext. SPM , which monitors things like Hadoop, Spark, etc. etc. including HBase and ES, can actually use either HBase or Elasticsearch as the data store. W

Re: Programmatic HBase version detection/extraction

On Wed, Nov 12, 2014 at 1:37 PM, Otis Gospodnetic > wrote: > > Hi, > > > > Is there a way to detect which version of HBase one is running? > > Is there an API for that, or a constant with this value, or maybe an > MBean > > or some other way to get to this info?

Re: Programmatic HBase version detection/extraction

m/ On Wed, Nov 12, 2014 at 4:41 PM, Ted Yu wrote: > Using hbase shell: > > hbase(main):002:0> status 'detailed' > version 0.98.4.2-hadoop2 > > Cheers > > On Wed, Nov 12, 2014 at 1:37 PM, Otis Gospodnetic < > otis.gospodne...@gmail.com> wrote: >

Programmatic HBase version detection/extraction

Hi, Is there a way to detect which version of HBase one is running? Is there an API for that, or a constant with this value, or maybe an MBean or some other way to get to this info? Thanks, Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support

Re: HBase w/o Master - TTL?

l a global region (re-)assignment is > > needed. > > > > Note, though, that the HBase client does work without a running master. > > -- Lars > > From: Otis Gospodnetic > > To: "user@hbase.apache.org" > > Sent: Friday, October 10, 2014 11:52

HBase w/o Master - TTL?

Hi, How long can HBase 0.94 RegionServers live (accept reads and writes) without a Master? Thanks, Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.com/

Re: Adding 64-bit nodes to 32-bit cluster?

Why 32 bit? Because it was a cheaper and more suitable option when we set up the cluster. Btw. we've added the 64-bit machines to the 32-bit cluster and everything survived - HBase 0.94. Here's a graph showing just disk utilization and the 2 new nodes joining the cluster and gradually taking mor

Re: Performance oddity between AWS instance sizes

Hi, The oddity in this thread is that there is no mention of metrics (sorry if I missed them being mentioned!). For example, that 1GB heap makes me think a graph showing JVM heap memory pool sizes/utilization and GC counts/times would quickly tell us/you if you are simply not giving the JVM enoug

Re: Adding 64-bit nodes to 32-bit cluster?

boxes and phase out the old 32-bit nodes after you test for a > while in a couple of the 64-bit nodes new settings. > > cheers, > esteban, > > > -- > Cloudera, Inc. > > > On Mon, Sep 15, 2014 at 1:37 PM, Otis Gospodnetic < > otis.gospodne...@gmail.com> wrote

Adding 64-bit nodes to 32-bit cluster?

Hi, We are thinking of adding some new 64-bit servers to run RegionServers to our 32-bit HBase cluster. Anything we should worry about or pay extra attention to? Thanks, Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.

HBase usage by HBase version?

Hi, Does anyone know or have any guesses about which HBase versions are being used the most? I'd love to know what percentage of HBase clusters out there are still using 0.94.x or below? Are there more users using 0.94.x and below or 0.96.x and above? Any educated guesses? Thanks, Otis -- Perfo

Re: Is this a long GC pause, or something else?

experience micromanaging our JVMs. How did you > generate those graphs? > > --Tom > > > On Tue, Jun 10, 2014 at 4:52 PM, Otis Gospodnetic < > otis.gospodne...@gmail.com> wrote: > > > No, I don't think so. We had it until this morning and didn't see this > &g

Re: Is this a long GC pause, or something else?

er > Carrier IQ, www.carrieriq.com > e-mail: vrodio...@carrieriq.com > > ________ > From: Otis Gospodnetic [otis.gospodne...@gmail.com] > Sent: Tuesday, June 10, 2014 2:43 PM > To: user@hbase.apache.org > Subject: Re: Is this a long GC pause, o

Re: Is this a long GC pause, or something else?

ar on all nodes in the cluster, just not at the same time. Otis -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Tue, Jun 10, 2014 at 5:43 PM, Otis Gospodnetic < otis.gospodne...@gmail.com> wrote: > Does it repeat? &

Re: Is this a long GC pause, or something else?

Does it repeat? We are seeing this with u60 oracle JVM too! SPM shows the whole JVM blocking for about 16 minutes every M minutes. Otis > On Jun 10, 2014, at 2:05 PM, Tom Brown wrote: > > Last night a regionserver in my cluster stopped responding in a timely > manner for about 20 minutes.

HBase (Compaction and Region) Event Listeners

Hi, I spotted https://issues.apache.org/jira/browse/HBASE-11220 which looks very handy for capturing some HBase events ... but I was wondering if there is already something in HBase that would allow one to listen for thing like compactions or any other important HBase events? Thanks, Oti

Re: Any lessons from hbase deployments on on-premise products ?

Hi, Our SPM product (monitors HBase, Hadoop and a bunch of other stuff) comes in On Premise version in addition to the Cloud one, and it uses HBase to store all the performance metrics. I can tell you that the bulk of the Installation Guide is really about HBase (and t

Htrace vs. Zipkin?

Hi, What is the difference and relationship between Htrace and Zipkin? Is Htrace an alternative to Zipkin? Zipkin already existed when Htrace was created, I believe, so is Htrace considered better in some way? Thanks, Otis -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elast

Re: 0.8.1 high cpu usage

Wrong ML? Otis -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Fri, Apr 4, 2014 at 10:53 PM, Libo Yu wrote: > Hi all, > > To test 0.8.1, I recently upgraded our kafka from 0.8-beta1 to 0.81. For > the same config, I''ve notice

Re: 0.92 => 0.96 in one go?

at 9:52 AM, Kevin O'dell >wrote: > > > Just your first born > > > > > > On Fri, Jan 31, 2014 at 2:35 PM, Otis Gospodnetic < > > otis.gospodne...@gmail.com> wrote: > > > > > You've got that much time!? I mean, sure, sure, but

Re: 0.92 => 0.96 in one go?

y it for you Otis w/ artificial data? (Usual disclaimers...) > St.Ack > > > On Fri, Jan 31, 2014 at 6:35 AM, Otis Gospodnetic < > otis.gospodne...@gmail.com> wrote: > > > Hi, > > > > It looks like going from 0.96 to 0.98 should be relatively easy, which is >

Re: 0.92 => 0.96 in one go?

* Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Fri, Jan 31, 2014 at 10:10 AM, Ted Yu wrote: > Otis: > See following: > http://hbase.apache.org/book.html#upgrade0.94 > > Cheers > > > On Fri, Jan 31, 2014 at 6:35 AM, Otis

0.92 => 0.96 in one go?

Hi, It looks like going from 0.96 to 0.98 should be relatively easy, which is great. We are on 0.92 and I'm wondering if it's possible do the 0.92 ==> 0.96 jump without making two jumps: 0.92 ==> 0.94 and then 0.94. ==> 0.96 ? Thanks, Otis -- Performance Monitoring * Log Analytics * Search Analy

Re: 0.96 Replication to Elasticsearch

Why not copy the approach (+ some code) from https://github.com/NGDATA/hbase-indexer ? Otis -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Wed, Jan 15, 2014 at 9:03 PM, Pradeep Gollakota wrote: > Hi All, > > I have a use case

Re: use hbase as distributed crawl's scheduler

t realize Nutch uses HBase underneath. Might be interesting if you > serialized data in a Phoenix-compliant manner, as you could run SQL queries > directly on top of it. > > Thanks, > James > > > On Thu, Jan 2, 2014 at 10:17 PM, Otis Gospodnetic < > otis.gospodne...@gmail

Re: use hbase as distributed crawl's scheduler

Hi, Have a look at http://nutch.apache.org . Version 2.x uses HBase under the hood. Otis -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Fri, Jan 3, 2014 at 1:12 AM, Li Li wrote: > hi all, > I want to use hbase to store

Re: Uneven write request to regions

We use https://github.com/sematext/HBaseWD and I just learned Amazon.com people are using it and are happy with it, so it may work for you, too. Otis -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Wed, Nov 20, 2013 at 1:00 AM,

ANNOUNCEMENT: HBase Meetup next Monday night in NYC

Hi, We'll be getting together Monday (Oct 28) evening in Facebook's office in Manhattan to talk HBase. We'll have food, drinks, several conference rooms with projectors, two main talks, one of which will cover HBase 0.96, plus barcamp-style sessions, followed by a visit to an actual bar. To RSVP

Re: Block cache hit ratio question

Hi, Our SPM for HBase doesn't show per table or region cache stats, and looks like that is not available yet, but it will let you slice by host and see changes over time, e.g., as you tune. See http://sematext.com/spm . We use it for monitoring our own clusters. Otis Solr & ElasticSearch Support

Re: HBase Random Read latency > 100ms

Hi Ramu, I think I saw mentions of this possibly being a GC issue though now it seems it may be a disk IO issue? 3 things: 1) http://blog.sematext.com/2013/06/24/g1-cms-java-garbage-collector/ - our G1 experience, with HBase specifically 2) If you can share some of your performance graphs (GC

Re: Spatial data posting in HBase

14 AM, Michael Segel wrote: > And Solr has what to do with storing data in HBase? > > I guess its true… if all you have is a hammer… > > The point I was raising was that geohash isn't the most efficient way to go > when you look at the problem at a global level… > > O

Re: Spatial data posting in HBase

Consider using Solr, which provides a lot of geospatial search support. Otis Solr & ElasticSearch Support http://sematext.com/ On Sep 24, 2013 8:29 AM, "cto" wrote: > Hi , > > I am very new in HBase. Could you please let me know , how to insert > spatial > data (Latitude / Longitude) in HBase us

Re: Multi-master info missing from book?

p 25, 2013 at 1:11 PM, Ted Yu wrote: > I found: > > 2.5.1.2. If a backup Master, making primary Master fail fast > > Cheers > > > On Wed, Sep 25, 2013 at 10:08 AM, Otis Gospodnetic < > otis.gospodne...@gmail.com> wrote: > >> Hi, >> >> I was

Multi-master info missing from book?

Hi, I was looking for info about running multiple HBase masters on http://hbase.apache.org/book.html and wasn't able to find any references to it. I think I spotted one mention of "active" in the context of master, but nothing else. Either I'm not seeing it there, or I'm not looking at the right

Re: quieting HBase metrics

Me, too! Otis -- HBase Performance Monitoring -- http://sematext.com/spm On Mon, Sep 23, 2013 at 7:10 AM, Arati Patro wrote: > Hi All, > > > I have the same query. Tried searching for a solution on the web but found > nothing very helpful. Is there some way I could disable metrics for every >

Re: G1 before/after GC time graph

zuryy Yu wrote: > > > Otis, > > > > I will do this test, maybe on the end of this month. because I haven't > big > > memory server for test now util the end of this month. > > > > > > > > > > On Thu, Jul 11, 2013 at 9:38 AM, Otis Gospodnet

Re: G1 before/after GC time graph

HBase Performance Monitoring -- http://sematext.com/spm On Wed, Jul 10, 2013 at 5:27 PM, Kevin O'dell wrote: > Anyway you can test this with a 32GB heap? :) > > > On Wed, Jul 10, 2013 at 5:23 PM, Otis Gospodnetic < > otis.gospodne...@gmail.com> wrote: > >>

Re: G1 before/after GC time graph

sweating more. This gives you smaller pauses, no big STW pauses, but be prepared to pay in CPU cycles. HTH Otis -- HBase Performance Monitoring -- http://sematext.com/spm On Wed, Jul 10, 2013 at 4:20 PM, Otis Gospodnetic wrote: > Not big in this case - 1800m > > Otis > -- > HB

Re: G1 before/after GC time graph

arrieriq.com > e-mail: vrodio...@carrieriq.com > > ________ > From: Otis Gospodnetic [otis.gospodne...@gmail.com] > Sent: Wednesday, July 10, 2013 12:00 PM > To: user@hbase.apache.org > Subject: G1 before/after GC time graph > > Hi, > >

Re: G1 before/after GC time graph

On Wed, Jul 10, 2013 at 12:00 PM, Otis Gospodnetic < > otis.gospodne...@gmail.com> wrote: > >> Hi, >> >> There was talk of G1 and HBase here over the past few days. I wanted >> to share a fresh and telling SPM graph showing G1 + some other params >> in ac

G1 before/after GC time graph

Hi, There was talk of G1 and HBase here over the past few days. I wanted to share a fresh and telling SPM graph showing G1 + some other params in action: Result first: https://apps.sematext.com/spm/s/wN0hKVixxX This is what we had before: -XX:+HeapDumpOnOutOfMemoryError -XX:+UseG1GC -XX:Initi

Re: GC recommendations for large Region Server heaps

eforeCompaction=0 -XX:+CMSClassUnloadingEnabled >> -XX:CMSMaxAbortablePrecleanTime=300 -XX:+CMSScavengeBeforeRemark >> >> >> >> On Tue, Jul 9, 2013 at 1:12 PM, Otis Gospodnetic < >> otis.gospodne...@gmail.com >> > wrote: >> >> > Hi, >> > >>

Re: GC recommendations for large Region Server heaps

Hi, Check http://blog.sematext.com/2013/06/24/g1-cms-java-garbage-collector/ Those graphs show RegionServer before and after switch to G1. The dashboard screenshot further below shows CMS (top row) vs. G1 (bottom row). After those tests we ended up switching to G1 across the whole cluster and h

Re: Zookeeper session timeout

Hi, Give G1 GC a try if you are on a recent JVM. We didn't have issues with ZK, but we gave G1 a try recently and look what happened: http://blog.sematext.com/2013/06/24/g1-cms-java-garbage-collector/ - maybe that will help you with ZK, too. Otis -- HBase Performance Monitoring -- http://sematex

Re: Schema design for filters

s with reasonable performance. > > Thanks for the tip! > > > On Fri, Jun 28, 2013 at 8:34 PM, Otis Gospodnetic < > otis.gospodne...@gmail.com> wrote: > >> Kristoffer, >> >> You could also consider using something other than HBase, something >> tha

Re: is hbase cluster support multi-instance?

You can have multiple and completely separate tables inside the same HBase cluster... Otis -- Solr & ElasticSearch Support -- http://sematext.com/ Performance Monitoring -- http://sematext.com/spm On Fri, Jun 28, 2013 at 12:35 AM, ch huang wrote: > hi all: > can hbase start more than one insta

Re: How many column families in one table ?

Hm, works for me - http://search-hadoop.com/m/qOx8l15Z1q42/column+families+fb&subj=Re+HBase+Column+Family+Limit+Reasoning Shorter version: http://search-hadoop.com/m/qOx8l15Z1q42 Otis -- Solr & ElasticSearch Support -- http://sematext.com/ Performance Monitoring -- http://sematext.com/spm On F

Re: Schema design for filters

Kristoffer, You could also consider using something other than HBase, something that supports "secondary indices", like anything that is Lucene based - Solr and ElasticSearch for example. We recently compared how we aggregate data in HBase (see my signature) and how we would do it if we were to u

Re: Missing metrics in any context inheriting from AbstractMetricsContext

Hi, I don't have a direct answer to your question, but if getting metrics from JMX is an option, everything should be in there. You can use a tool like https://github.com/sematext/jmxc to quickly get a JMX dump and see what's in there. Otis -- HBase / Hadoop Performance Monitoring - http://semat

Re: Extending HBase Metrics Framework

Hi, Not 100% sure if I understand what you are after, but it sounds like you are OK/want to modify HBase source code and add tracking of some metric that is currently not being tracked. Could you use Coda Hale Metric lib for that then?  Any metric you capture with it can then go into JMX or Ga

Re: HBase cluster over multiple EC2 Availability Zones?

g larger instances with placement groups, they do not work cross AZ as >> well. >> >> With this in mind we went the multi-cluster route, and kept all nodes for a >> cluster in 1 AZ. >> >> >> On Mon, May 6, 2013 at 10:37 AM, Otis Gospodnetic < >> otis.gospodne

Re: HBase cluster over multiple EC2 Availability Zones?

. Of > course, the numbers would vary based on the workload and my testing was > definitely not comprehensive enough to cover all scenarios. This does not > take into account the extra $$ you pay for going cross AZ, which as you > can > imagine would add up pretty quickly when looking at a

Re: HBase cluster over multiple EC2 Availability Zones?

t; take into account the extra $$ you pay for going cross AZ, which as you can > imagine would add up pretty quickly when looking at a large scale > deployment. > > > On Mon, May 6, 2013 at 11:00 AM, Otis Gospodnetic < > otis.gospodne...@gmail.com> wrote: > >>

Re: HBase cluster over multiple EC2 Availability Zones?

index.html HBASE Performance Monitoring - http://sematext.com/spm/index.html On Mon, May 6, 2013 at 1:58 PM, Otis Gospodnetic wrote: > Hi, > > Actually, cross-AZ traffic is actually free. It's cross-Region > traffic that's not free. I realize the cross-AZ (translate

Re: HBase cluster over multiple EC2 Availability Zones?

e link is slower than within a single AZ. > > Amandeep > > > On Mon, May 6, 2013 at 10:37 AM, Otis Gospodnetic < > otis.gospodne...@gmail.com> wrote: > >> Hi, >> >> Do people spread HBase clusters over multiple EC2 Availability Zones, >> or is th

HBase cluster over multiple EC2 Availability Zones?

Hi, Do people spread HBase clusters over multiple EC2 Availability Zones, or is that a big no-no? Cross-AZ communication is supposedly going over fast fiber optic cables. Thanks, Otis -- Search Analytics - http://sematext.com/search-analytics/index.html HBASE Performance Monitoring - http://sema

Re: talk list table

+ http://blog.sematext.com/2012/12/24/hbasewd-and-hbasehut-handy-hbase-libraries-available-in-public-maven-repo/ if you use Maven and want to use HBaseWD. Otis -- HBASE Performance Monitoring - http://sematext.com/spm/index.html On Sat, Apr 20, 2013 at 11:24 AM, Amit Sela wrote: > Hope I'm n

Re: ANN: HBase Refcard available

hich HBase version the refcard targets ? > > > On Wed, Aug 8, 2012 at 4:14 PM, Otis Gospodnetic > wrote: > >> Hi, >> >> We wrote an HBase Refcard and published it via DZone. Here is our very >> brief announcement: >> http://blog.sematext.com/2012/08/06/a

Re: ANN: HBase Refcard available

hat points to this Otis? Or just > tell me where to insert? > Thanks, > St.Ack > > > On Wed, Aug 8, 2012 at 4:14 PM, Otis Gospodnetic > wrote: > >> Hi, >> >> We wrote an HBase Refcard and published it via DZone. Here is our very >> brief announcement

Re: Question about compactions

Hi, A few pointers: http://search-hadoop.com/m/heoc617XV29/otis+compactions&subj=Re+How+to+check+if+a+major_compact+is+done+ http://search-hadoop.com/m/BDx4S1jMjF92/otis+compactions&subj=How+to+know+it+s+time+for+a+major+compaction+ http://search-hadoop.com/m/9oN1DqPx2R1/otis+compactions&subj=Comp

Re: Ganglia Vs Nagios

gement functionality. > > > > Thank you! > > > > Sincerely, > > Leonid Fedotov > > > > > > > > On Feb 28, 2013, at 8:31 AM, Otis Gospodnetic wrote: > > > >> Hi, > >> > >> Have a look at > >> http://sem

Re: Ganglia Vs Nagios

Hi, Have a look at http://sematext.com/spm/hbase-performance-monitoring/index.html (screenshot included). It does both monitoring and alerting and it's a SaaS, so you don't have to fiddle with Ganglia and Nagios, install, upgrade, etc. Otis -- Solr & ElasticSearch Support http://sematext.com/

Re: HBase without compactions?

e word, yes. > > Or rather you can't have a compactionless HBase without fixing the > deficiencies in HDFS. > > On Feb 18, 2013, at 11:01 PM, Otis Gospodnetic > wrote: > > > Hi, > > > > > > On Mon, Feb 18, 2013 at 11:50 PM, Michael Segel > >

Re: HBase without compactions?

anks, Otis -- HBASE Performance Monitoring - http://sematext.com/spm/index.html On Mon, Feb 18, 2013 at 11:54 PM, Ted Yu wrote: > Take a look at HBASE-7667 Support stripe compaction > > Cheers > > On Mon, Feb 18, 2013 at 7:30 PM, Otis Gospodnetic < > otis.gospodne..

Re: HBase without compactions?

s? Otis -- HBASE Performance Monitoring - http://sematext.com/spm/index.html > > > On Feb 18, 2013, at 9:30 PM, Otis Gospodnetic > wrote: > > > Hello, > > > > It's kind of funny, we run SPM, which includes SPM for HBase (performance > > monitoring s

Re: hbase olap cube

Hi, There is one getting started on Github. Also google for HBase lattice. We've also built something like that at Sematext, with real-time in memory agg and HBase persistence (we use it for the thing in my sig and more), but have not OSSed it. Otis -- HBASE Performance Monitoring - http://sema

Re: persistence in Hbase

Hi, Just for my own edification - isn't data loss always going to be possible due to caches present in HDDs and inability(?) for force them to flush. I believe I've read even fsync lies... Thanks, Otis -- HBASE Performance Monitoring - http://sematext.com/spm/index.html On Thu, Jan 10, 2013

Re: responsetooslow from regionserver

For what it's worth, SPM for HBase may be of help here, as it graphs this sort of info - JVM GC, various server metrics, and a bunch of HBase metrics. Otis -- Solr & ElasticSearch Support http://sematext.com/ On Fri, Jan 4, 2013 at 8:00 PM, Azuryy Yu wrote: > generally check the gc log and

Re: H-Rider / HTable UI

If this were to become a part of Apache HBase, HBase devs would have to maintain it. And I have a feeling nobody really wants to maintain this here :) Otis -- HBASE Performance Monitoring - http://sematext.com/spm/index.html Search Analytics - http://sematext.com/search-analytics/index.html

Re: 答复: how to store 100billion short text messages with hbase

Hello, If you want to use Lucenewhy not use Lucene, or one of the fancy search servers built on top of it - Solr(Cloud), ElasticSearch, or SenseiDB? You can easily shard the index by time, lookup by key, and search using full-text search with results sorted by some key value or relevance to th

Re: How to monitor total disk access in HBase?

Hi, I was just trawling HBase JIRA for this info after one happy user of SPM for HBase asked if we could show read requests separately from write requests. See https://issues.apache.org/jira/browse/HBASE-4989 https://issues.apache.org/jira/browse/HBASE-3647 There is also https://issues.apache.or

Re: How to check if a major_compact is done?

Hi, https://github.com/aravind/hbase_compact may have what you need (thanks Jeremy) Otis -- Search Analytics - http://sematext.com/search-analytics/index.html Performance Monitoring - http://sematext.com/spm/index.html On Thu, Nov 8, 2012 at 1:31 PM, yun peng wrote: > Yes, JMX exposes compact

Re: How to check if a major_compact is done?

Hi, I *think* SPM for HBase shows this. URL in signature. I'm guessing Cloudera Manager may also show this if you are using CDH. Otherwise, you can just read from JMX. Handy no-deps command-line tool for that: https://github.com/sematext/jmxc Otis -- Performance Monitoring - http://sematext.c

Re: Compaction throttling and per-region compaction automation

/github.com/aravind/hbase_compact > > On Tue, Nov 6, 2012 at 10:38 AM, Otis Gospodnetic < > otis.gospodne...@gmail.com> wrote: > > > Hi, > > > > Major compactions are. you know... :) > > So I saw there is https://issues.apache.org/jira/browse/HBASE-37

Re: RequestsPerSecond on master console

Varun, I'll wait for the answer like you, but http://search-hadoop.com/?q=RequestsPerSecond+&fc_project=HBase found me https://issues.apache.org/jira/browse/HBASE-6780, which may be relevant here. Otis -- Search Analytics - http://sematext.com/search-analytics/index.html Performance Monitoring -

Re: Lucene instead of HFiles?

looms with Lucene is also I think overkill. > Just saying. > > Lars > > On Oct 5, 2012, at 5:34 AM, Otis Gospodnetic > wrote: > >> Hi, >> >> Has anyone attempted using Lucene instead of HFiles (see >> https://twitter.com/otisg/status/254047978174701568 )?

Re: Lucene instead of HFiles?

er distributed >> storage is required. Katta does not look as powerful as Hadoop. >> >> On Fri, Oct 5, 2012 at 5:34 AM, Otis Gospodnetic >> wrote: >>> >>> Hi, >>> >>> Has anyone attempted using Lucene instead of HFiles (see >>> h

  1   2   3   >