Hi Eyal, I used the same approach as Michael.
As far as I understand, your shared collection should be changeable. So do not forget use concurrent collection. Solution for copies by worker are: 1) Represent one collection as group of collections (collection per worker), using storm grouping make sure that input bolts emit to the appropriate cache-bolts. 2) use external storage for state persistence. It depends from your needs. Best regards, Dmytro Dragan On Dec 4, 2014 10:54 PM, "Michael Rose" <mich...@fullcontact.com> wrote: > Yes, as long as they're in the same JVM. If you have 4 workers, that's 4 > JVMs, and 4 copies of the data. > > Storm will ensure that if you have 4 bolts of type A, and 8 of type B, > that 1 of type A and 2 of type B exist in each JVM. Since the singleton's > field is static, it'll be shared. > > Michael Rose (@Xorlev <https://twitter.com/xorlev>) > Senior Platform Engineer, FullContact <http://www.fullcontact.com/> > mich...@fullcontact.com > > On Thu, Dec 4, 2014 at 1:34 PM, Eyal Golan <egola...@gmail.com> wrote: > >> Michael, >> I understand your solution, however, if bolt of type A "wins" the >> initialization, will bolts of type B see the same instance? >> >> Sent from my Android phone >> On Dec 4, 2014 10:26 PM, "Eyal Golan" <egola...@gmail.com> wrote: >> >>> Thanks Michael and Ben ! >>> >>> Sent from my Android phone >>> On Dec 4, 2014 9:53 PM, "Flint, Ben" <bfl...@popcap.com> wrote: >>> >>>> I’ve used Hazelcast for use cases like this with great success. It >>>> makes it relatively straightforward to implement a distributed data >>>> structure. >>>> >>>> From: Eyal Golan <egola...@gmail.com> >>>> Reply-To: "user@storm.apache.org" <user@storm.apache.org> >>>> Date: Thursday, December 4, 2014 at 11:40 AM >>>> To: "user@storm.apache.org" <user@storm.apache.org> >>>> Subject: Re: Shared static data between bolts >>>> >>>> Thanks! >>>> All the answers give me good idea. >>>> Here's why I need this list be sharable. >>>> One bolt needs the list for computation. >>>> Another bolt will periodically update the list. >>>> In other words, the list is changeable. >>>> >>>> I tried passing it in conf but there are some 3rd party libs in the >>>> wrapper. >>>> >>>> Anyway, I also need to improve the topology. Separate bolt to two etc. >>>> That way, it would be easier to handle that cache. >>>> >>>> One more question, I thought about giving hazelcast a try. >>>> Create local node and then the bolts will have clients. >>>> >>>> What do you think? >>>> >>>> Sent from my Android phone >>>> On Dec 4, 2014 9:14 PM, "Kushan Maskey" < >>>> kushan.mas...@mmillerassociates.com> wrote: >>>> >>>>> Hi Eyal, >>>>> >>>>> I had the similar issue earlier. Static objects are not a good way of >>>>> passing the variables around in Storm due to clustered nature of the >>>>> Storm. >>>>> You will see null for all the static variables. >>>>> >>>>> This is how I implemented mine, for instance I wanted to pass around >>>>> the server properties such and host name and username/password. I read the >>>>> properties file and then created a map of these properties then set it >>>>> into >>>>> the Config object. Like below. >>>>> >>>>> *In topology:* >>>>> >>>>> Map<Stirng, Stirng> map = new HashMap(); >>>>> map.put("hostname", "localhost"); >>>>> map.put("username", "testuser"); >>>>> map.put("password", "testpassord"); >>>>> map.put("port", "1234"); >>>>> >>>>> Config config = new Config(); >>>>> >>>>> config.put("MY_MAP", map) >>>>> *In Bolts:* >>>>> >>>>> public void prepare(Map stormConf, TopologyContext context) { >>>>> >>>>> HashMap<Stirng, Stirng> map = stormConfig.get("MY_MAP"); >>>>> >>>>> } >>>>> This how i resolved my problem.Try that if that helps. >>>>> -- >>>>> Kushan Maskey >>>>> 817.403.7500 >>>>> M. Miller & Associates <http://mmillerassociates.com/> >>>>> kushan.mas...@mmillerassociates.com >>>>> >>>>> On Thu, Dec 4, 2014 at 1:01 PM, Eyal Golan <egola...@gmail.com> wrote: >>>>> >>>>>> Hi all, >>>>>> I am newbie with Storm and having some issues with static objects in >>>>>> the topology. >>>>>> In the main class, where I set the topology, I also set a static >>>>>> object. It's actually a wrapper for a collection of strings. >>>>>> This object needs to he shared between different bolts. >>>>>> It's big so I want it to be only one instance. >>>>>> >>>>>> The problem is that when a bolt tries to get this static instance: >>>>>> example - MyMain.Shared... It's null. Although I instantiated it in the >>>>>> main method. >>>>>> >>>>>> I guess I'm doing something wrong. >>>>>> >>>>>> Thanks for any assistance. >>>>>> >>>>>> Eyal >>>>>> >>>>>> Sent from my Android phone >>>>>> >>>>> >>>>> >
