Hi, Thank you all, you defiantly cleared the issue (and it wasn't boring at all ☺ )
So Craig, is it the "Big data" processing that makes the "Storm" the right choice? If I have ONLY large number of events, and they do not add-up to "Big data", is it still relevant to consider Storm? (I am sure Storm can handle it…yet is it a criteria to select Storm as a platform) Aliza From: Craig Charleton [mailto:[email protected]] Sent: Wednesday, November 18, 2015 3:44 PM To: [email protected] Cc: [email protected] Subject: Re: Storm typical application Sorry I didn't make it clear in my response but market data messages are typically very small, received super-fast and (in my situation) coming from a number of sources that trigger other processes in addition to being aggregated up for large-set, real-time analysis the results of which might be run through a separate Storm Topology. Craig Charleton [email protected]<mailto:[email protected]> On Nov 18, 2015, at 8:24 AM, <[email protected]<mailto:[email protected]>> <[email protected]<mailto:[email protected]>> wrote: Hi, There was a point in the question that, I think, translates to : Will Storm be useful if my data packets are small(“Messages are typically short ones”), but they add up to the size of Big Data ? The answer is Yes. The other parts of the questions have been answered by others, I hope. Regards, Prajod From: Craig Charleton [mailto:[email protected]] Sent: 18 November 2015 18:39 To: [email protected]<mailto:[email protected]> Cc: John Fang <[email protected]<mailto:[email protected]>> Subject: Re: Storm typical application Aliza, If I may, I would like to share A few random thoughts about your question. I worked for a large enterprise software company and our customers were always struggling with how to use the massive amounts of data that were being input/created by their systems to understand their business and make decisions. Traditionally the data had to come to its final resting place before it could be analyzed for decision support. There was no way to reformat, clean, analyze, aggregate the data as it was flowing through the systems, let alone for different user populations to apply their own perspective to the "streams" without affecting the operations of others. That is where I see the value to large organizations. In fact, it was the limitations of traditional enterprise systems that became obvious once companies like Twitter, Linked-In, Google, Yahoo, Facebook needed to do things to large volumes of data in real time. They not only needed to perform these operations quickly, the load was continually growing, so solutions needed to be able to scale beyond one server on an ongoing basis. This is what Storm is for in my opinion. I am currently implementing it to perform a lot of operations on stock trade and quote information as it is received from the markets. The number of stocks that need to be handled by the system is unknown. Therefore I am able to use Storm to write the operations once and then scale the load across an unlimited number of servers. Hope this wasn't too boring. Craig Charleton [email protected]<mailto:[email protected]> On Nov 18, 2015, at 3:25 AM, Aliza Nagauker <[email protected]<mailto:[email protected]>> wrote: Hi, Thanks for your response. I will try the examples. I understand that Storm can do the functionality required in my application, yet my question is whether it is the right platform. So far we worked with Karaf-framework for our applications, and I am trying to understand what should be the motivations to move to Storm framework? Is it for cases of: large amount of real time data processing (Big Data: files, DB, WEB pages) over distributed machines? Large amount of real time events processing – usually control protocols (network protocols – like routing protocol, VOIP protocols, SNMP, REST) over distributed machines? Thanks, Aliza From: John Fang [mailto:[email protected]] Sent: Wednesday, November 18, 2015 2:21 AM To: [email protected]<mailto:[email protected]> Subject: 答复: Storm typical application Yes,storm can do it. I suggest you read some storm’ example.: https://github.com/apache/storm/tree/master/examples/storm-starter 发件人: Aliza Nagauker [mailto:[email protected]] 发送时间: 2015年11月17日 23:23 收件人: [email protected]<mailto:[email protected]> 主题: Storm typical application Hello all, I am new at Storm. I read Storm Doc and tutorial as published in storm site and have few basic questions. I am trying to learn and understand whether Storm is suitable for my application. Is Storm mainly intended for distributed real time applications that has to handle "massive input data" and apply "data analytics over this data"? Is it indented to application where the data-size is large and need analytic over the data itself (word count, search words, convert formats, write it to DB etc.)? Assuming my application is a kind of a Controller that: receive messages from multiple sources: Management Systems, Network Elements, Internal timers, Internal modules Act upon these messages: update protocol –state-machines, it may send messages to other servers/applications. Messages are typically short ones – control protocols messages (Not HTTP pages, Not Documents, Not Database info). We may need to run this application in multiple machines. In this case, is Storm is the right choice for this application? I understand that Storm is indeed very recommended for Distributed Real Time application, yet, I am not sure it is intended for network applications that are mainly control application and not Data Processing Applications (Not Big Data applications) I'll appreciate your consult on this. Thanks, Aliza The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com<http://www.wipro.com>
