Hazelcast is killing a cockroach with a sledgehammer in this case. Have
your singleton able to initialize its self, then in prepare() of your bolts
call out to init it. Whichever one gets there first will lock it, then
it'll initialize and unlock. Other bolts will call init, find it already
initialized, then move on.
class MySingleton {
private volatile static ImmutableSet<String> expensiveSet;
public ImmutableSet<String> getInstance() {
if (expensiveSet == null) {
synchronized(MySingleton.class) {
if (expensiveSet == null) {
expensiveSet = <init code>;
}
}
}
}
}
class MyBolt extends BaseRichBolt {
public void prepare(..) {
// first bolt instance to reach this will start init, all others will
wait until the lock is dropped
// the doublecheck will ensure only one does the init
MySingleton.getInstance();
}
public void execute(Tuple t) {
ImmutableSet<String> mySet = MySingleton.getInstance();
...
}
}
Michael Rose (@Xorlev <https://twitter.com/xorlev>)
Senior Platform Engineer, FullContact <http://www.fullcontact.com/>
mich...@fullcontact.com
On Thu, Dec 4, 2014 at 12:40 PM, Eyal Golan <egola...@gmail.com> wrote:
> Thanks!
> All the answers give me good idea.
> Here's why I need this list be sharable.
> One bolt needs the list for computation.
> Another bolt will periodically update the list.
> In other words, the list is changeable.
>
> I tried passing it in conf but there are some 3rd party libs in the
> wrapper.
>
> Anyway, I also need to improve the topology. Separate bolt to two etc.
> That way, it would be easier to handle that cache.
>
> One more question, I thought about giving hazelcast a try.
> Create local node and then the bolts will have clients.
>
> What do you think?
>
> Sent from my Android phone
> On Dec 4, 2014 9:14 PM, "Kushan Maskey" <
> kushan.mas...@mmillerassociates.com> wrote:
>
>> Hi Eyal,
>>
>> I had the similar issue earlier. Static objects are not a good way of
>> passing the variables around in Storm due to clustered nature of the Storm.
>> You will see null for all the static variables.
>>
>> This is how I implemented mine, for instance I wanted to pass around the
>> server properties such and host name and username/password. I read the
>> properties file and then created a map of these properties then set it into
>> the Config object. Like below.
>>
>> *In topology:*
>>
>> Map<Stirng, Stirng> map = new HashMap();
>> map.put("hostname", "localhost");
>> map.put("username", "testuser");
>> map.put("password", "testpassord");
>> map.put("port", "1234");
>>
>> Config config = new Config();
>>
>> config.put("MY_MAP", map)
>> *In Bolts:*
>>
>> public void prepare(Map stormConf, TopologyContext context) {
>>
>> HashMap<Stirng, Stirng> map = stormConfig.get("MY_MAP");
>>
>> }
>> This how i resolved my problem.Try that if that helps.
>> --
>> Kushan Maskey
>> 817.403.7500
>> M. Miller & Associates <http://mmillerassociates.com/>
>> kushan.mas...@mmillerassociates.com
>>
>> On Thu, Dec 4, 2014 at 1:01 PM, Eyal Golan <egola...@gmail.com> wrote:
>>
>>> Hi all,
>>> I am newbie with Storm and having some issues with static objects in the
>>> topology.
>>> In the main class, where I set the topology, I also set a static object.
>>> It's actually a wrapper for a collection of strings.
>>> This object needs to he shared between different bolts.
>>> It's big so I want it to be only one instance.
>>>
>>> The problem is that when a bolt tries to get this static instance:
>>> example - MyMain.Shared... It's null. Although I instantiated it in the
>>> main method.
>>>
>>> I guess I'm doing something wrong.
>>>
>>> Thanks for any assistance.
>>>
>>> Eyal
>>>
>>> Sent from my Android phone
>>>
>>
>>
