Thanks Alex and Ncleung for your inputs. Both the options looks to be valid based on the data size. I am thinking now, even RDBMS might scale well for this scenario if bolt is using only “Read” operations on database. However, will update later on the approach taken.
Regards Balakrishna From: alex kamil [mailto:alex.ka...@gmail.com] Sent: Monday, June 02, 2014 10:18 PM To: user@storm.incubator.apache.org Subject: Re: Storm with RDBMS for parallel reads of massive historical data and high volume writes you could you a distributed db with SQL layer such as Apache Hbase+Phoenix<http://phoenix.incubator.apache.org/>, I think it might complement Storm nicely On Mon, Jun 2, 2014 at 10:19 AM, Nathan Leung <ncle...@gmail.com<mailto:ncle...@gmail.com>> wrote: Something like memcached is commonly used for this scenario. Is memcached poorly suited for your goals or data access patterns? On Mon, Jun 2, 2014 at 10:06 AM, Balakrishna R <balakrishn...@spanservices.com<mailto:balakrishn...@spanservices.com>> wrote: Hi, We are evaluating ‘Apache storm’ for one of the business use cases. In this use case, the incoming transactions/stream should be processed by set of rules or logic. In this process, there is a need of considering the historical data (may be 2 weeks or a month old) also. Understand that, Storm will give better performance to process the incoming transactions in real-time. What if we have to read the historical data from RDBMS and use that data in the bolts? Will this degrade the performance of whole cluster (as RDBMS systems might cause some delay due to the high load of reads from the parallelizing different bolts to achieve the better performance). Any suggestion on solving this situation? Please share. Thanks Balakrishna DISCLAIMER: This email message and all attachments are confidential and may contain information that is Privileged, Confidential or exempt from disclosure under applicable law. If you are not the intended recipient, you are notified that any dissemination, distribution or copying of this email is strictly prohibited. If you have received this email in error, please notify us immediately by return email to mailad...@spanservices.com<mailto:mailad...@spanservices.com> and destroy the original message. Opinions, conclusions and other information in this message that do not relate to the official of SPAN, shall be understood to be nether given nor endorsed by SPAN. DISCLAIMER: This email message and all attachments are confidential and may contain information that is Privileged, Confidential or exempt from disclosure under applicable law. If you are not the intended recipient, you are notified that any dissemination, distribution or copying of this email is strictly prohibited. If you have received this email in error, please notify us immediately by return email to mailad...@spanservices.com and destroy the original message. Opinions, conclusions and other information in this message that do not relate to the official of SPAN, shall be understood to be nether given nor endorsed by SPAN.
