Hi Is this memstore replicated? Since we store a significant amount of data in the memory cache we need a replicated solution. Also I can't find lots of information besides a java api doc for the MemStore class. I will continue searching for this, but if you have any URL with more documentation please send it. Thanks in advance
regards Christian --------------------------------------------8<-------------------------------------- Siemens AG Corporate Technology Corporate Research and Technologies CT T DE IT3 Otto-Hahn-Ring 6 81739 München, Deutschland Tel.: +49 89 636-42722 Fax: +49 89 636-41423 mailto:christian.kleegr...@siemens.com Siemens Aktiengesellschaft: Vorsitzender des Aufsichtsrats: Gerhard Cromme; Vorstand: Peter Löscher, Vorsitzender; Roland Busch, Brigitte Ederer, Klaus Helmrich, Joe Kaeser, Barbara Kux, Hermann Requardt, Siegfried Russwurm, Peter Y. Solmssen, Michael Süß; Sitz der Gesellschaft: Berlin und München, Deutschland; Registergericht: Berlin Charlottenburg, HRB 12300, München, HRB 6684; WEEE-Reg.-Nr. DE 23691322 -----Ursprüngliche Nachricht----- Von: N Keywal [mailto:nkey...@gmail.com] Gesendet: Freitag, 16. März 2012 18:02 An: user@hbase.apache.org Betreff: Re: Streaming data processing and hBase Hi, The way you describe the "in memory caching component", it looks very similar to HBase memstore. Any reason for not relying on it? N. On Fri, Mar 16, 2012 at 4:21 PM, Kleegrewe, Christian < christian.kleegr...@siemens.com> wrote: > Dear all, > > We are currently working on an architecture for a system that should be > serve as an archive for 1000+ measuring components that frequently (~30/s) > send messages containing measurement values (~300 bytes/message). The > archiving system should be capable of not only serving as a long term > storage but also as a kind of streaming data processing and caching > component. There are several functions that should be computed on the > incoming data before finally storing it. > > We suggested an architecture that comprises of: > A message routing component that could route data to calculations and > route calculation results to other components that are interested in these > data. > An in memory caching component that is used for storing up to 10 - 20 > minutes of data before it is written to the long term archive. > An hBase database that is used for the long term storage. > MapReduce framework for doing analytics on the data stored in the hBase > database. > > The complete system should be failsafe and reliable regarding component > failures and it should scale with the number of computers that are utilized. > > Are there any suggestions or feedback to this approach from the community? > and are there any suggestions which tools or systems to use for the message > routing component and the in memory cache. > > Thanks for any help and suggestions > > all the best > > Christian > > > ----------------------------8<----------------------------------------------- > > Siemens AG > Corporate Technology > Corporate Research and Technologies > CT T DE IT3 > Otto-Hahn-Ring 6 > 81739 Munich, Germany > Tel.: +49 89 636-42722 > Fax: +49 89 636-41423 > mailto:christian.kleegr...@siemens.com > > Siemens Aktiengesellschaft: Chairman of the Supervisory Board: Gerhard > Cromme; Managing Board: Peter Loescher, Chairman, President and Chief > Executive Officer; Roland Busch, Brigitte Ederer, Klaus Helmrich, Joe > Kaeser, Barbara Kux, Hermann Requardt, Siegfried Russwurm, Peter Y. > Solmssen, Michael Suess; Registered offices: Berlin and Munich, Germany; > Commercial registries: Berlin Charlottenburg, HRB 12300, Munich, HRB 6684; > WEEE-Reg.-No. DE 23691322 >
