Re: problem in running topology in local mode

[sam mohel]

i forgot to mention that iam using jdk 1.6

On Sat, Jan 28, 2017 at 8:09 AM, sam mohel  wrote:

> thanks for replying . i searched or how to know the appropratie versions i
> should write it but couldn't find . I'm using apache-storm-0.9.6 and when i
> ran this command mvn dependency:list i got
> org.slf4j:log4j-over-slf4j:jar:1.6.6:provided
> org.slf4j:slf4j-api:jar:1.7.5:provided
>
> if you please ,what should i do ? Thanks in advance
>
> On Sat, Jan 28, 2017 at 3:04 AM, Ambud Sharma 
> wrote:
>
>> Your dependencies need to be overridden. If you are using Maven please
>> look at your dependency hierarchy and check what all is using slf log4j and
>> fix it the version of storm you are using.
>>
>>
>> On Jan 25, 2017 3:35 AM, "sam mohel"  wrote:
>>
>> is there any help , please ?
>>
>> On Tue, Jan 24, 2017 at 8:53 AM, sam mohel  wrote:
>>
>>> I got this error when i ran my topology in local mode
>>> clojure.tools.logging$eval1$fn__7 invoke
>>> SEVERE:
>>> java.lang.NoSuchMethodError: org.slf4j.spi.LocationAwareLog
>>> ger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/Stri
>>> ng;[Ljava/lang/Object;Ljava/lang/Throwable;)V
>>> at org.apache.log4j.Category.differentiatedLog(Category.java:186)
>>> at org.apache.log4j.Category.info(Category.java:229)
>>> at backtype.storm.utils.ShellProcess.getSerializer(ShellProcess
>>> .java:78)
>>> at backtype.storm.utils.ShellProcess.launch(ShellProcess.java:58)
>>> at backtype.storm.task.ShellBolt.prepare(ShellBolt.java:117)
>>> at backtype.storm.daemon.executor$fn__3439$fn__3451.invoke(exec
>>> utor.clj:699)
>>> at backtype.storm.util$async_loop$fn__460.invoke(util.clj:461)
>>> at clojure.lang.AFn.run(AFn.java:24)
>>> at java.lang.Thread.run(Thread.java:701)
>>>
>>>
>>
>>
>

Re: problem in running topology in local mode

[sam mohel]

thanks for replying . i searched or how to know the appropratie versions i
should write it but couldn't find . I'm using apache-storm-0.9.6 and when i
ran this command mvn dependency:list i got
org.slf4j:log4j-over-slf4j:jar:1.6.6:provided
org.slf4j:slf4j-api:jar:1.7.5:provided

if you please ,what should i do ? Thanks in advance

On Sat, Jan 28, 2017 at 3:04 AM, Ambud Sharma 
wrote:

> Your dependencies need to be overridden. If you are using Maven please
> look at your dependency hierarchy and check what all is using slf log4j and
> fix it the version of storm you are using.
>
>
> On Jan 25, 2017 3:35 AM, "sam mohel"  wrote:
>
> is there any help , please ?
>
> On Tue, Jan 24, 2017 at 8:53 AM, sam mohel  wrote:
>
>> I got this error when i ran my topology in local mode
>> clojure.tools.logging$eval1$fn__7 invoke
>> SEVERE:
>> java.lang.NoSuchMethodError: org.slf4j.spi.LocationAwareLog
>> ger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/Stri
>> ng;[Ljava/lang/Object;Ljava/lang/Throwable;)V
>> at org.apache.log4j.Category.differentiatedLog(Category.java:186)
>> at org.apache.log4j.Category.info(Category.java:229)
>> at backtype.storm.utils.ShellProcess.getSerializer(ShellProcess
>> .java:78)
>> at backtype.storm.utils.ShellProcess.launch(ShellProcess.java:58)
>> at backtype.storm.task.ShellBolt.prepare(ShellBolt.java:117)
>> at backtype.storm.daemon.executor$fn__3439$fn__3451.invoke(exec
>> utor.clj:699)
>> at backtype.storm.util$async_loop$fn__460.invoke(util.clj:461)
>> at clojure.lang.AFn.run(AFn.java:24)
>> at java.lang.Thread.run(Thread.java:701)
>>
>>
>
>

Re: Storm deployment in wide-area network

[Ambud Sharma]

Nifi is a better tool for WAN data movements. There is no particular
benefit to deploy Storm either face internet or moving data over the
internet.


On Jan 25, 2017 9:43 AM, "Fan Jiang"  wrote:

Hi all,

I have a general question - is there any Storm deployment in wide-area
network? If so, what are the typical applications that would benefit from
such deployment?

Thanks,
Fan

Re: problem in running topology in local mode

[Ambud Sharma]

Your dependencies need to be overridden. If you are using Maven please look
at your dependency hierarchy and check what all is using slf log4j and fix
it the version of storm you are using.

On Jan 25, 2017 3:35 AM, "sam mohel"  wrote:

is there any help , please ?

On Tue, Jan 24, 2017 at 8:53 AM, sam mohel  wrote:

> I got this error when i ran my topology in local mode
> clojure.tools.logging$eval1$fn__7 invoke
> SEVERE:
> java.lang.NoSuchMethodError: org.slf4j.spi.LocationAwareLog
> ger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/
> String;[Ljava/lang/Object;Ljava/lang/Throwable;)V
> at org.apache.log4j.Category.differentiatedLog(Category.java:186)
> at org.apache.log4j.Category.info(Category.java:229)
> at backtype.storm.utils.ShellProcess.getSerializer(ShellProcess
> .java:78)
> at backtype.storm.utils.ShellProcess.launch(ShellProcess.java:58)
> at backtype.storm.task.ShellBolt.prepare(ShellBolt.java:117)
> at backtype.storm.daemon.executor$fn__3439$fn__3451.invoke(
> executor.clj:699)
> at backtype.storm.util$async_loop$fn__460.invoke(util.clj:461)
> at clojure.lang.AFn.run(AFn.java:24)
> at java.lang.Thread.run(Thread.java:701)
>
>

Re: Good examples of StatefulWindowedBoltExecutor

[Arun Mahadevan]

There is an example here - 
https://github.com/apache/storm/blob/master/examples/storm-starter/src/jvm/storm/starter/StatefulWindowingTopology.java.
 Also checkout the “Stateful windowing” section under 
https://community.hortonworks.com/articles/14171/windowing-and-state-checkpointing-in-apache-storm.html.
 

As an alternative, you could instead have a regular windowed bolt which sends 
the aggregate results of you 2-3 min window to a downstream bolt (like 
RedisBolt) which updates the result in a permanent state.

Thanks,
Arun

On 1/27/17, 11:39 AM, "Punit Tiwan"  wrote:

>Hi,
>Doe anyone know any good example of StatefulWindowedBoltExecutor and how does 
>it actually work?
>
>I am currently using 30min window but memory usage is issue for us. To solve 
>that i am thinking of using lesser time window(like 2-3mins) plus something to 
>keep my computed states into redis. 
>I think StatefulWindowedBoltExecutor will solve my problem. Please correct me 
>if i am wrong.
>
>Regards,
>Punit Tiwan
>
>
>
>

Re: Why aren't storm topology send and receive buffers equal sized?

[Navin Ipe]

I found a part of the answer in the article itself "Note that
topology.receiver.buffer.size is in contrast to the other buffer size
related parameters described in this article actually not configuring the
size of an LMAX Disruptor queue. Rather it sets the size of a simple
ArrayList that is used to buffer incoming messages because in this specific
case the data structure does not need to be shared with other threads, i.e.
it is local to the worker’s receive thread."

But the question still remains. If Bolt A sends 1024 tuples, then won't it
overwhelm Bolt B?


On Fri, Jan 27, 2017 at 3:11 PM, Navin Ipe 
wrote:

> Hi,
>
> When going through Michael Noll's explanation
> 
> on the buffers, I noticed the topology.receive.buffer.size default is 8 and
> topology.send.buffer.size default is 1024.
>
> If I have a topology where Spout --> Bolt A ---> BoltB, then if Bolt A
> sends a tuple of size 1024, then BoltB won't be able to handle 1024 easily
> and will get overwhelmed because Bolt B's receive buffer size is just 8.
>
> Is my above understanding correct? Why are the buffers so different in
> sizes?
>
> --
> Regards,
> Navin
>



-- 
Regards,
Navin

Why aren't storm topology send and receive buffers equal sized?

[Navin Ipe]

Hi,

When going through Michael Noll's explanation

on the buffers, I noticed the topology.receive.buffer.size default is 8 and
topology.send.buffer.size default is 1024.

If I have a topology where Spout --> Bolt A ---> BoltB, then if Bolt A
sends a tuple of size 1024, then BoltB won't be able to handle 1024 easily
and will get overwhelmed because Bolt B's receive buffer size is just 8.

Is my above understanding correct? Why are the buffers so different in
sizes?

-- 
Regards,
Navin