Hi Christian,
It's a component internal to HBase, so you don't have to use it directly.
See http://hbase.apache.org/book/wal.html on how writes are handled by
HBase to ensure reliability data distribution...
Cheers,
N.
On Fri, Mar 16, 2012 at 7:39 PM, Kleegrewe, Christian
christian.kleegr...@siemens.com wrote:
Hi
Is this memstore replicated? Since we store a significant amount of data
in the memory cache we need a replicated solution. Also I can't find lots
of information besides a java api doc for the MemStore class. I will
continue searching for this, but if you have any URL with more
documentation please send it. Thanks in advance
regards
Christian
8--
Siemens AG
Corporate Technology
Corporate Research and Technologies
CT T DE IT3
Otto-Hahn-Ring 6
81739 München, Deutschland
Tel.: +49 89 636-42722
Fax: +49 89 636-41423
mailto:christian.kleegr...@siemens.com
Siemens Aktiengesellschaft: Vorsitzender des Aufsichtsrats: Gerhard
Cromme; Vorstand: Peter Löscher, Vorsitzender; Roland Busch, Brigitte
Ederer, Klaus Helmrich, Joe Kaeser, Barbara Kux, Hermann Requardt,
Siegfried Russwurm, Peter Y. Solmssen, Michael Süß; Sitz der Gesellschaft:
Berlin und München, Deutschland; Registergericht: Berlin Charlottenburg,
HRB 12300, München, HRB 6684; WEEE-Reg.-Nr. DE 23691322
-Ursprüngliche Nachricht-
Von: N Keywal [mailto:nkey...@gmail.com]
Gesendet: Freitag, 16. März 2012 18:02
An: user@hbase.apache.org
Betreff: Re: Streaming data processing and hBase
Hi,
The way you describe the in memory caching component, it looks very
similar to HBase memstore. Any reason for not relying on it?
N.
On Fri, Mar 16, 2012 at 4:21 PM, Kleegrewe, Christian
christian.kleegr...@siemens.com wrote:
Dear all,
We are currently working on an architecture for a system that should be
serve as an archive for 1000+ measuring components that frequently
(~30/s)
send messages containing measurement values (~300 bytes/message). The
archiving system should be capable of not only serving as a long term
storage but also as a kind of streaming data processing and caching
component. There are several functions that should be computed on the
incoming data before finally storing it.
We suggested an architecture that comprises of:
A message routing component that could route data to calculations and
route calculation results to other components that are interested in
these
data.
An in memory caching component that is used for storing up to 10 - 20
minutes of data before it is written to the long term archive.
An hBase database that is used for the long term storage.
MapReduce framework for doing analytics on the data stored in the hBase
database.
The complete system should be failsafe and reliable regarding component
failures and it should scale with the number of computers that are
utilized.
Are there any suggestions or feedback to this approach from the
community?
and are there any suggestions which tools or systems to use for the
message
routing component and the in memory cache.
Thanks for any help and suggestions
all the best
Christian
8---
Siemens AG
Corporate Technology
Corporate Research and Technologies
CT T DE IT3
Otto-Hahn-Ring 6
81739 Munich, Germany
Tel.: +49 89 636-42722
Fax: +49 89 636-41423
mailto:christian.kleegr...@siemens.com
Siemens Aktiengesellschaft: Chairman of the Supervisory Board: Gerhard
Cromme; Managing Board: Peter Loescher, Chairman, President and Chief
Executive Officer; Roland Busch, Brigitte Ederer, Klaus Helmrich, Joe
Kaeser, Barbara Kux, Hermann Requardt, Siegfried Russwurm, Peter Y.
Solmssen, Michael Suess; Registered offices: Berlin and Munich, Germany;
Commercial registries: Berlin Charlottenburg, HRB 12300, Munich, HRB
6684;
WEEE-Reg.-No. DE 23691322
