result better why and when

2016-11-06 Thread researcher cs
I'm new for this point and need help from you .

If i used project using storm but used another algorithm in simialrty
between tweets and gave me better results that i noticed it in storm ui
such as ( complete latency column and some of bolts in execute latency
column) . What is the reason the can cause that ?

i didn't change anything in the settings of the project . just the
algorithm of similarity . how can it react on the project ?

Thanks for any help in advance


Re: CPU utilization with Trident

2016-03-09 Thread researcher cs
thanks for replying , really i'm apperciate your help , so cpu usage is
important to measure performance ?
i need to know how can i use all numbers in storm ui to evaluate topology
means like this

Topology A


​
Topology B

​i see execute latency , capcaity  and process latency
from these numbers how can i detect performance
high is better or what ?
Thanks for your time


On Wed, Mar 9, 2016 at 6:51 AM, Cody Innowhere  wrote:

> @sam,
> CPU usage is very close to your application logic, i.e., it depends on
> whether your topology is CPU-intensive or not.
> It may be also related to the throughput of your topologies. So it's really
> hard to tell if a certain cpu usage like .5 is good or bad.
>
> In your case of comparing two topologies, let's say topology A and B, I can
> give some rough hints:
> 1. If A's CPU usage is higher than B with higher throughput, I'd say
> possibly it's worth higher CPU usage to gain extra throughput;
> 2. If A's throughput and B's throughput are almost the same, I'd prefer the
> topology which has lower CPU usage.
>
> Hope this helps.
>
> On Wed, Mar 9, 2016 at 12:28 PM, sam mohel  wrote:
>
> > What is the importance of CPU utilization after submitted topology as I
> > need to compare between two topologies which  is better in performance
> >
> > And numbers which close to .5 is better or not ?
> > How can I know ?
> > Thanks
> >
>


Re: storm ui result

2016-03-06 Thread researcher cs
Can I find help ?

On Saturday, March 5, 2016, researcher cs <prog.researc...@gmail.com> wrote:

> I've a project using storm and made change in code , tried to get which is
> better in performance , i submitted old code in storm and took screenshot
> of numbers appeared in storm ui
>
>
> ​
> and now i tested my code "new code" in storm with single machine
>
> ​
>  and in real cluster with another machine
>
> ​is result with using single machine is same with using 2 machine ?
> i need someone to help me in this numbers Why these numbers changed ?
> and how can i decide which is better "old code" or "new code " ?
>
> Thanks in advance ,
>


storm ui result

2016-03-05 Thread researcher cs
I've a project using storm and made change in code , tried to get which is
better in performance , i submitted old code in storm and took screenshot
of numbers appeared in storm ui


​
and now i tested my code "new code" in storm with single machine

​
 and in real cluster with another machine

​is result with using single machine is same with using 2 machine ?
i need someone to help me in this numbers Why these numbers changed ?
and how can i decide which is better "old code" or "new code " ?

Thanks in advance ,


What i need for single macjine in distributed mode

2016-01-29 Thread researcher cs
I want to try suit topology in distributed mode with single machine so what
are the configurations I neef should I have one file for storm.yaml and
connect IP 127.0.0.1 for storm.zookeeper.server ,nimbus.host and
drpc.server or what ?

Thanks for any help


Who and where can I ask for drpc problems ?

2016-01-29 Thread researcher cs
As the title is the question said . I've problem with drpc and asked but no
answer . where can I find help ? I searched more for finding solution but
no answer for same problem I'm facing

Appreciate any suggestions


Re: DRPC server not working

2016-01-28 Thread researcher cs
Is there anybody here used dtpc before ?

On Thursday, January 28, 2016, researcher cs <prog.researc...@gmail.com>
wrote:

> storm vresion -0.9.0.1 zookeeper -3.4.6 i tried example of nathan marz but
> didn't work too , gave me same result so i felt it's my problem with
> configuration
> as i'm using one machine for distributed but someone told me that i should
> first submit topology with single machine wit ip 127.0.0.1 first to make
> sure it's working well in distrbuted then i can make real network
>
> On Thu, Jan 28, 2016 at 1:56 AM, Erik Weathers <
> eweath...@groupon.com.invalid
> <javascript:_e(%7B%7D,'cvml','eweath...@groupon.com.invalid');>> wrote:
>
>> Not sure I'll have time, but if I do then I'll try to get the DRPC starter
>> example working and then can probably tell you more explicitly how to make
>> your example work.
>>
>> Can you please tell us which version of Storm you are using?
>>
>> - Erik
>>
>> On Wed, Jan 27, 2016 at 3:25 PM, researcher cs <prog.researc...@gmail.com
>> <javascript:_e(%7B%7D,'cvml','prog.researc...@gmail.com');>>
>> wrote:
>>
>> > can you try this project ? i hope you can
>> >
>>
>
>


Re: DRPC server not working

2016-01-27 Thread researcher cs
thanks but Agian i imported this project from github that it's supposed
work well , really i'm on this problem for months without fixing it !! i
sent coder message but he didn't reply with this problem . don't know who
can i ask but thanks for your time

On Thu, Jan 28, 2016 at 1:12 AM, Erik Weathers <
eweath...@groupon.com.invalid> wrote:

> hey Sam,
>
> Again, I've never used the DRPC feature.  But it naively looks like you
> aren't using it correctly.
> Note that the working case for you is with a LocalCluster & LocalDRPC, and
> you are explicitly invoking the execute():
> String result = drpc.execute(TOPOLOGY_NAME, tweetJson);
>
> In the remote case you aren't doing anything except submitting the
> topology.  I assume you need to invoke drpc.execute *somewhere*... Here's
> what I think is the same question as you are posing:
>
>- http://stackoverflow.com/a/26440260/318428
>
> - Erik
>
> On Wed, Jan 27, 2016 at 3:04 PM, researcher cs <prog.researc...@gmail.com>
> wrote:
>
> > also got zeros after open storm ui except executed column
> >
> > ​
> > ​
> >
> > On Thu, Jan 28, 2016 at 12:43 AM, researcher cs <
> prog.researc...@gmail.com
> > > wrote:
> >
> >> i atteached what i got when i submitted topology and this happened also
> >> when i submitted storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar
> >> storm.starter.BasicDRPCTopology basic-drpc
> >>
> >> this storm.yaml
> >>  storm.zookeeper.servers:
> >>  - "localhost"
> >> #- "server2"
> >>  nimbus.host: "localhost"
> >>
> >>  storm.local.dir: "/var/storm"
> >>  supervisor.slots.ports:
> >>  - 7660
> >>  - 7659
> >>  - 7658
> >>  - 7657
> >>  supervisor.childopts: "-Djava.net.preferIPv4Stack=true"
> >>  nimbus.childopts: "-Djava.net.preferIPv4Stack=true"
> >>  worker.childopts: "-Djava.net.preferIPv4Stack=true"
> >> # topology.message.timeout.secs: 30
> >> # topology.workers: 1
> >>  topology.stats.sample.rate: 1.0
> >>  topology.acker.executors: 1
> >> # topology.executor.receive.buffer.size: 16384
> >> # topology.executor.send.buffer.size: 16384
> >> # topology.transfer.buffer.size: 32
> >> # topology.receiver.buffer.size: 8
> >> #
> >> # # These may optionally be filled in:
> >> #
> >> ## List of custom serializations
> >> # topology.kryo.register:
> >> # - org.mycompany.MyType
> >> # - org.mycompany.MyType2: org.mycompany.MyType2Serializer
> >> #
> >> ## List of custom kryo decorators
> >> # topology.kryo.decorators:
> >> # - org.mycompany.MyDecorator
> >> #
> >> ## Locations of the drpc servers
> >> # drpc.servers:
> >>   #  - "localhost"
> >> # - "server2"
> >>
> >> ## Metrics Consumers
> >> # topology.metrics.consumer.register:
> >> #   - class: "backtype.storm.metrics.LoggingMetricsConsumer"
> >> # parallelism.hint: 1
> >> #   - class: "org.mycompany.MyMetricsConsumer"
> >> # parallelism.hint: 1
> >> # argument:
> >> #   - endpoint: "metrics-collector.mycompany.org"
> >>  storm.messaging.transport: "backtype.storm.messaging.netty.Context"
> >>  storm.messaging.netty.server_worker_threads: 1
> >>  storm.messaging.netty.client_worker_threads: 1
> >>  storm.messaging.netty.buffer_size: 5242880
> >>  storm.messaging.netty.max_retries: 100
> >>  storm.messaging.netty.max_wait_ms: 1000
> >>  storm.messaging.netty.min_wait_ms: 100
> >>
> >> this etc/hosts
> >> 127.0.0.1  localhost
> >> 127.0.1.1  user-Lenovo-G50-70
> >> ##The following lines are desirable for IPv6 capable hosts##
> >> ::1 ip6-localhost ip6-loopback
> >> fe00::0 ip6-localnet
> >> ff00::0 ip6-mcastprefix
> >> ff02::1 ip6-allnodes
> >> ff02::2 ip6-allrouters
> >>
> >> i set drpc in code
> >> private static Config createTopologyConfiguration(Properties prop,
> >> boolean localMode) {
> >> Config conf = new Config();
> >> List dprcServers = new ArrayList();
> >> dprcServers.add("localhost");
> >>
> >> conf.put(Config.DRPC_SERVERS, dprcServers);
> >> conf.put(Config.DRPC_PORT, 3772);

Re: DRPC server not working

2016-01-27 Thread researcher cs
storm vresion -0.9.0.1 zookeeper -3.4.6 i tried example of nathan marz but
didn't work too , gave me same result so i felt it's my problem with
configuration
as i'm using one machine for distributed but someone told me that i should
first submit topology with single machine wit ip 127.0.0.1 first to make
sure it's working well in distrbuted then i can make real network

On Thu, Jan 28, 2016 at 1:56 AM, Erik Weathers <
eweath...@groupon.com.invalid> wrote:

> Not sure I'll have time, but if I do then I'll try to get the DRPC starter
> example working and then can probably tell you more explicitly how to make
> your example work.
>
> Can you please tell us which version of Storm you are using?
>
> - Erik
>
> On Wed, Jan 27, 2016 at 3:25 PM, researcher cs <prog.researc...@gmail.com>
> wrote:
>
> > can you try this project ? i hope you can
> >
>


Re: DRPC server not working

2016-01-27 Thread researcher cs
can you try this project ? i hope you can


Re: DRPC server not working

2016-01-27 Thread researcher cs
also got zeros after open storm ui except executed column

​
​

On Thu, Jan 28, 2016 at 12:43 AM, researcher cs <prog.researc...@gmail.com>
wrote:

> i atteached what i got when i submitted topology and this happened also
> when i submitted storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar
> storm.starter.BasicDRPCTopology basic-drpc
>
> this storm.yaml
>  storm.zookeeper.servers:
>  - "localhost"
> #- "server2"
>  nimbus.host: "localhost"
>
>  storm.local.dir: "/var/storm"
>  supervisor.slots.ports:
>  - 7660
>  - 7659
>  - 7658
>  - 7657
>  supervisor.childopts: "-Djava.net.preferIPv4Stack=true"
>  nimbus.childopts: "-Djava.net.preferIPv4Stack=true"
>  worker.childopts: "-Djava.net.preferIPv4Stack=true"
> # topology.message.timeout.secs: 30
> # topology.workers: 1
>  topology.stats.sample.rate: 1.0
>  topology.acker.executors: 1
> # topology.executor.receive.buffer.size: 16384
> # topology.executor.send.buffer.size: 16384
> # topology.transfer.buffer.size: 32
> # topology.receiver.buffer.size: 8
> #
> # # These may optionally be filled in:
> #
> ## List of custom serializations
> # topology.kryo.register:
> # - org.mycompany.MyType
> # - org.mycompany.MyType2: org.mycompany.MyType2Serializer
> #
> ## List of custom kryo decorators
> # topology.kryo.decorators:
> # - org.mycompany.MyDecorator
> #
> ## Locations of the drpc servers
> # drpc.servers:
>   #  - "localhost"
> # - "server2"
>
> ## Metrics Consumers
> # topology.metrics.consumer.register:
> #   - class: "backtype.storm.metrics.LoggingMetricsConsumer"
> # parallelism.hint: 1
> #   - class: "org.mycompany.MyMetricsConsumer"
> # parallelism.hint: 1
> # argument:
> #   - endpoint: "metrics-collector.mycompany.org"
>  storm.messaging.transport: "backtype.storm.messaging.netty.Context"
>  storm.messaging.netty.server_worker_threads: 1
>  storm.messaging.netty.client_worker_threads: 1
>  storm.messaging.netty.buffer_size: 5242880
>  storm.messaging.netty.max_retries: 100
>  storm.messaging.netty.max_wait_ms: 1000
>  storm.messaging.netty.min_wait_ms: 100
>
> this etc/hosts
> 127.0.0.1  localhost
> 127.0.1.1  user-Lenovo-G50-70
> ##The following lines are desirable for IPv6 capable hosts##
> ::1 ip6-localhost ip6-loopback
> fe00::0 ip6-localnet
> ff00::0 ip6-mcastprefix
> ff02::1 ip6-allnodes
> ff02::2 ip6-allrouters
>
> i set drpc in code
> private static Config createTopologyConfiguration(Properties prop,
> boolean localMode) {
> Config conf = new Config();
> List dprcServers = new ArrayList();
> dprcServers.add("localhost");
>
> conf.put(Config.DRPC_SERVERS, dprcServers);
> conf.put(Config.DRPC_PORT, 3772);
> if (!localMode)
> conf.put(Config.STORM_CLUSTER_MODE, new String("distributed"));
>
> }
>public static void main(String[] args) throws Exception {
>if (args == null || args.length == 0) {
> Config conf = createTopologyConfiguration(prop, true);
> LocalDRPC drpc = new LocalDRPC();
> LocalCluster cluster = new LocalCluster();
>
> cluster.submitTopology(TOPOLOGY_NAME, conf,
> buildTopology(drpc));
>
> Thread.sleep(2000); // give it some time to setup
>
> BufferedReader br = new BufferedReader(new
> FileReader(queryFile));
> String tweetJson;
> fos.write("Start: ".getBytes());
>
> fos.write(String.valueOf(System.currentTimeMillis()).getBytes());
> byte[] newLine = "\n".getBytes();
> int times = 0;
> // emit tweets into topology
> while ((tweetJson = br.readLine()) != null) {
>
> String result = drpc.execute(TOPOLOGY_NAME, tweetJson);
>
> Status s = null;
> try {
> s = DataObjectFactory.createStatus(tweetJson);
> result = s.getId() + "\t" + s.getText() + "\t" +
> result;
> } catch (TwitterException e) {
> LOG.error(e.toString());
> }
>
> fos.write(result.getBytes());
> fos.write(newLine);
>
> // times++;
> // if (times == 1000)
> // break;
> }
> fos.write(newLine);
> fos.write("Finish: ".getBytes());
>
> fos.write(Str

Re: DRPC server not working

2016-01-26 Thread researcher cs
yes i tried in local and worked well
and about /etc/hosts . i'm feeling that this file has a mistake , i made
alot of changes in this file and didn't remember what was default

In nimbus log file when it connected zookeeper i got
2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client connection,
connectString=localhost:2181 sessionTimeout=2
watcher=com.netflix.curator.ConnectionState@2fa423d2
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection to
server localhost/127.0.1.1:2181. Will not attempt to authenticate using
SASL (unknown error)
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection established
to localhost/127.0.1.1:2181, initiating session
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment complete
on server localhost/127.0.1.1:2181, sessionid = 0x152804f3a3a0002,
negotiated timeout = 2
2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update:
:connected:none
2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session: 0x152804f3a3a0002 closed
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down
2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client connection,
connectString=localhost:2181/storm sessionTimeout=2
watcher=com.netflix.curator.ConnectionState@2ee8b0bf
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection to
server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to authenticate
using SASL (unknown error)
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection established
to user-Lenovo-G50-70/127.0.0.1:2181, initiating session
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment complete
on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid = 0x152804f3a3a0003,
negotiated timeout = 2

and i set for drpc.server : localhost
storm.zookeeper.server : localhost
nimbus.host : localhost

in my /etc/hosts
127.0.0.1  user-Lenovo-G50-70  localhost
127.0.1.1  localhost

 is that right ?


On Wed, Jan 27, 2016 at 6:12 AM, Erik Weathers <
eweath...@groupon.com.invalid> wrote:

> You said: "except the statement of drpc server trying to connect"
>
> Maybe you are confused about what "b.s.d.drpc [INFO] Starting Distributed
> RPC servers..." implies?
> That is just saying that the server is being started.   It's a server, not
> a client, so it's basic operation is *not* to connect to some other thing.
> It's up and waiting for you to tell it to do stuff.
>
> Have you gotten the Local-mode version of DRPC working?
>
>-
>
> https://storm.apache.org/documentation/Distributed-RPC.html#local-mode-drpc
>
> Not sure what you're asking with regards to DNS and /etc/hosts, those seem
> unrelated to your basic issue.
>
> - Erik
>
> On Tue, Jan 26, 2016 at 6:58 PM, researcher cs <prog.researc...@gmail.com>
> wrote:
>
> > There is no error except the statement of drpc server trying to connect
> . I
> > guess I have problem with Dns . if you have any idea about this pleaes
> help
> >
> >
> > I want to submit topology with single machine
> > Only on my laptop without any other devices as a first step so
> >  What this file /etc/hosts should contains ?
> > As I set in drpc.server : localhost
> > Storm.zookeeper. server: localhost
> > Nimbus.host:localhost
> >
> > As this file contains 127.0.1.1 and 127.0.0.1 and my IP address ?
> >
> > What should I use and what should I hash it to not using it ?
> >
> >
> >
> > On Wednesday, January 27, 2016, Erik Weathers
> > <eweath...@groupon.com.invalid>
> > wrote:
> >
> > > What does the client code that is supposed to make the DRPC connection
> > > telling you?  i.e., you should see some exception or log about not
> > > establishing the connection, right?
> > >
> > > Alternatively, perhaps the connections aren't persistent and there's no
> > > actual problem?
> > >
> > > - Erik
> > >
> > > On Tue, Jan 26, 2016 at 4:55 PM, researcher cs <
> > prog.researc...@gmail.com
> > > <javascript:;>>
> > > wrote:
> > >
> > > >  thanks for replying , i read the documentation before , i imported
> > > project
> > > > supposed to work well but not working with me
> > > > i checked port by lsof -i gave me all ports i connected it for storm
> > > > java  10675   root   20u  IPv4  98126  0t0  TCP *:52022
> > (LISTEN)
> > > > java  10675   root   26u  IPv4  98131  0t0  TCP *:2181
> (LISTEN)
> > > > java  10675   root   27u  IPv4 101944  0t0  TCP
>

Re: DRPC server not working

2016-01-26 Thread researcher cs
 thanks for replying , i read the documentation before , i imported project
supposed to work well but not working with me
i checked port by lsof -i gave me all ports i connected it for storm
java  10675   root   20u  IPv4  98126  0t0  TCP *:52022 (LISTEN)
java  10675   root   26u  IPv4  98131  0t0  TCP *:2181 (LISTEN)
java  10675   root   27u  IPv4 101944  0t0  TCP
localhost:2181->user-Lenovo-G50-70:38150 (ESTABLISHED)
java  10675   root   29u  IPv4  98974  0t0  TCP
user-Lenovo-G50-70:2181->user-Lenovo-G50-70:50526 (ESTABLISHED)
java  10675   root   30u  IPv4  99105  0t0  TCP
localhost:2181->user-Lenovo-G50-70:38165 (ESTABLISHED)
java  10715   root   90u  IPv4  98953  0t0  TCP
user-Lenovo-G50-70:38150->localhost:2181 (ESTABLISHED)
java  10715   root   91u  IPv4  98245  0t0  TCP *:6627 (LISTEN)
java  10792   root   90u  IPv4  99973  0t0  TCP
user-Lenovo-G50-70:50526->user-Lenovo-G50-70:2181 (ESTABLISHED)
java  10864   root   82u  IPv4 102425  0t0  TCP *:3772 (LISTEN)
java  10864   root   84u  IPv4 102429  0t0  TCP *:3773 (LISTEN)
java  10864   root   92u  IPv4 102197  0t0  TCP
user-Lenovo-G50-70:3773->user-Lenovo-G50-70:50825 (ESTABLISHED)
java  10928   root   81u  IPv4 102070  0t0  TCP *:http-alt (LISTEN)
java  11087   root   81u  IPv4 100091  0t0  TCP
user-Lenovo-G50-70:50825->user-Lenovo-G50-70:3773 (ESTABLISHED)
java  11087   root   91u  IPv4 102196  0t0  TCP
user-Lenovo-G50-70:38165->localhost:2181 (ESTABLISHED)
java  11087   root   94u  IPv4 102561  0t0  TCP *:7660 (LISTEN)


here you can see that 3772 not established

On Wed, Jan 27, 2016 at 2:47 AM, Erik Weathers <
eweath...@groupon.com.invalid> wrote:

> hey,
>
> The DRPC server is up and listening on port 3772.   Why do you expect
> established connections?
>
> I'm not familiar with using Storm's DRPC feature, but I'm sure you need to
> write code that interacts with the DRPC server, and you've made no mention
> of doing so in your email.  I'd start here:
>
>- https://storm.apache.org/documentation/Distributed-RPC.html
>
> - Erik
>
> On Tue, Jan 26, 2016 at 4:29 PM, researcher cs <prog.researc...@gmail.com>
> wrote:
>
> > I set in the code
> > conf.put(Config.DRPC_SERVERS, dprcServers);
> > conf.put(Config.DRPC_PORT, 3772);
> > but when i submit topolgoy i found at the end of the file
> >  b.s.d.drpc [INFO] Starting Distributed RPC servers...
> >
> > i checked port 3772 by
> >
> > sudo netstat -ap | grep 3772
> >
> > i got
> >
> > tcp 00 *:3772*:* LISTEN  10864/java
> > unix  3  [ ] STREAM CONNECTED 13772
> > 587/dbus-daemon /var/run/dbus/system_bus_socket
> >
> >
> > why it's not established  ?
> >
> > can i find help ?
> >
>


Re: DRPC server not working

2016-01-26 Thread researcher cs
There is no error except the statement of drpc server trying to connect . I
guess I have problem with Dns . if you have any idea about this pleaes help


I want to submit topology with single machine
Only on my laptop without any other devices as a first step so
 What this file /etc/hosts should contains ?
As I set in drpc.server : localhost
Storm.zookeeper. server: localhost
Nimbus.host:localhost

As this file contains 127.0.1.1 and 127.0.0.1 and my IP address ?

What should I use and what should I hash it to not using it ?



On Wednesday, January 27, 2016, Erik Weathers <eweath...@groupon.com.invalid>
wrote:

> What does the client code that is supposed to make the DRPC connection
> telling you?  i.e., you should see some exception or log about not
> establishing the connection, right?
>
> Alternatively, perhaps the connections aren't persistent and there's no
> actual problem?
>
> - Erik
>
> On Tue, Jan 26, 2016 at 4:55 PM, researcher cs <prog.researc...@gmail.com
> <javascript:;>>
> wrote:
>
> >  thanks for replying , i read the documentation before , i imported
> project
> > supposed to work well but not working with me
> > i checked port by lsof -i gave me all ports i connected it for storm
> > java  10675   root   20u  IPv4  98126  0t0  TCP *:52022 (LISTEN)
> > java  10675   root   26u  IPv4  98131  0t0  TCP *:2181 (LISTEN)
> > java  10675   root   27u  IPv4 101944  0t0  TCP
> > localhost:2181->user-Lenovo-G50-70:38150 (ESTABLISHED)
> > java  10675   root   29u  IPv4  98974  0t0  TCP
> > user-Lenovo-G50-70:2181->user-Lenovo-G50-70:50526 (ESTABLISHED)
> > java  10675   root   30u  IPv4  99105  0t0  TCP
> > localhost:2181->user-Lenovo-G50-70:38165 (ESTABLISHED)
> > java  10715   root   90u  IPv4  98953  0t0  TCP
> > user-Lenovo-G50-70:38150->localhost:2181 (ESTABLISHED)
> > java  10715   root   91u  IPv4  98245  0t0  TCP *:6627 (LISTEN)
> > java  10792   root   90u  IPv4  99973  0t0  TCP
> > user-Lenovo-G50-70:50526->user-Lenovo-G50-70:2181 (ESTABLISHED)
> > java  10864   root   82u  IPv4 102425  0t0  TCP *:3772 (LISTEN)
> > java  10864   root   84u  IPv4 102429  0t0  TCP *:3773 (LISTEN)
> > java  10864   root   92u  IPv4 102197  0t0  TCP
> > user-Lenovo-G50-70:3773->user-Lenovo-G50-70:50825 (ESTABLISHED)
> > java  10928   root   81u  IPv4 102070  0t0  TCP *:http-alt
> (LISTEN)
> > java  11087   root   81u  IPv4 100091  0t0  TCP
> > user-Lenovo-G50-70:50825->user-Lenovo-G50-70:3773 (ESTABLISHED)
> > java  11087   root   91u  IPv4 102196  0t0  TCP
> > user-Lenovo-G50-70:38165->localhost:2181 (ESTABLISHED)
> > java  11087   root   94u  IPv4 102561  0t0  TCP *:7660 (LISTEN)
> >
> >
> > here you can see that 3772 not established
> >
> > On Wed, Jan 27, 2016 at 2:47 AM, Erik Weathers <
> > eweath...@groupon.com.invalid> wrote:
> >
> > > hey,
> > >
> > > The DRPC server is up and listening on port 3772.   Why do you expect
> > > established connections?
> > >
> > > I'm not familiar with using Storm's DRPC feature, but I'm sure you need
> > to
> > > write code that interacts with the DRPC server, and you've made no
> > mention
> > > of doing so in your email.  I'd start here:
> > >
> > >- https://storm.apache.org/documentation/Distributed-RPC.html
> > >
> > > - Erik
> > >
> > > On Tue, Jan 26, 2016 at 4:29 PM, researcher cs <
> > prog.researc...@gmail.com <javascript:;>>
> > > wrote:
> > >
> > > > I set in the code
> > > > conf.put(Config.DRPC_SERVERS, dprcServers);
> > > > conf.put(Config.DRPC_PORT, 3772);
> > > > but when i submit topolgoy i found at the end of the file
> > > >  b.s.d.drpc [INFO] Starting Distributed RPC servers...
> > > >
> > > > i checked port 3772 by
> > > >
> > > > sudo netstat -ap | grep 3772
> > > >
> > > > i got
> > > >
> > > > tcp 00 *:3772*:* LISTEN  10864/java
> > > > unix  3  [ ] STREAM CONNECTED 13772
> > > > 587/dbus-daemon /var/run/dbus/system_bus_socket
> > > >
> > > >
> > > > why it's not established  ?
> > > >
> > > > can i find help ?
> > > >
> > >
> >
>


Re: [jira] [Created] (STORM-1500) Couldn't extract resources in the supervisor

2016-01-25 Thread researcher cs
Sorry is it an issue . I asked in storm group without any answer and
searching also didn't find any solution !!

On Monday, January 25, 2016, Robert Joseph Evans (JIRA) 
wrote:

>
>  [
> https://issues.apache.org/jira/browse/STORM-1500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> ]
>
> Robert Joseph Evans closed STORM-1500.
> --
> Resolution: Invalid
>
> [~beginner],
>
> Please stop opening JIRA to ask for help.  I am happy to answer your
> question if you send an email to dev@storm.apache.org  or
> u...@storm.apache.org .
>
> > Couldn't extract resources in the supervisor
> > 
> >
> > Key: STORM-1500
> > URL: https://issues.apache.org/jira/browse/STORM-1500
> > Project: Apache Storm
> >  Issue Type: Bug
> >Reporter: beginner
> >
> > i need this files to be extracted
> fromvar/storm/nimbus/stormdist/topology_name/stormjar.jar  to
> /var/storm/supervisor/tmp but i got in supervisor log file
> > [INFO] Could not extract resources from
> /var/storm/supervisor/tmp/f5989f74-aae5-43c8-8b41-056074cb7e55/stormjar.jar
> > i tried to exctract it manually but didn't fix ! how can i fix it ?
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>


Re: should i open this post in JIRA ?

2016-01-22 Thread researcher cs
sorry maxClientCnxns=50 not 5

On Fri, Jan 22, 2016 at 11:15 PM, researcher cs <prog.researc...@gmail.com>
wrote:

> i submitted topology and once it submitted i got zeros in all columns
> except executed column as you see
>
> ​
>
> But when i clicked Show system stats button i got numbres as you see
>
>
> ​
> Should i open this post in Jira or here to fix it ?
>
> this topology isn't mine i got it from github and it's supposed to work
> well
>
> and i tried storm-starter project by submitting word count topology
> and got numbers before and after click on show system stats button but
> Acked in still zero as you can see
>
> before click button
>
> ​after click button
>
> ​i'm using single machine with this configuration in zookeeper (zoo.cfg)
> tickTime=2000
> initLimit=10
> syncLimit=5
> dataDir=/var/zookeeper
> clientPort=2181
> maxClientCnxns=5
>
> this storm.yaml
>
> storm.zookeeper.servers:
>  - "127.0.0.1"
>  nimbus.host: "127.0.0.1"
>  storm.local.dir: "/var/storm"
>  supervisor.slots.ports:
>  - 7660
>  - 7659
>  - 7658
>  - 7657
>  supervisor.childopts: "-Djava.net.preferIPv4Stack=true"
>  nimbus.childopts: "-Djava.net.preferIPv4Stack=true"
>  worker.childopts: "-Djava.net.preferIPv4Stack=true"
>  topology.stats.sample.rate: 1.0
>  topology.message.timeout.secs: 300
>  topology.acker.executors: 1
>  storm.messaging.transport: "backtype.storm.messaging.netty.Context"
>  storm.messaging.netty.server_worker_threads: 1
>  storm.messaging.netty.client_worker_threads: 1
>  storm.messaging.netty.buffer_size: 5242880
>  storm.messaging.netty.max_retries: 100
>  storm.messaging.netty.max_wait_ms: 1000
>  storm.messaging.netty.min_wait_ms: 100
>
> if you need details i can provide you , just tell me
>
> My topolgoy has one spout -- 10 bolts -- spout.max.pending= 5000
>
>
>
>
>


should i open this post in JIRA ?

2016-01-22 Thread researcher cs
i submitted topology and once it submitted i got zeros in all columns
except executed column as you see

​

But when i clicked Show system stats button i got numbres as you see


​
Should i open this post in Jira or here to fix it ?

this topology isn't mine i got it from github and it's supposed to work
well

and i tried storm-starter project by submitting word count topology
and got numbers before and after click on show system stats button but
Acked in still zero as you can see

before click button

​after click button

​i'm using single machine with this configuration in zookeeper (zoo.cfg)
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/var/zookeeper
clientPort=2181
maxClientCnxns=5

this storm.yaml

storm.zookeeper.servers:
 - "127.0.0.1"
 nimbus.host: "127.0.0.1"
 storm.local.dir: "/var/storm"
 supervisor.slots.ports:
 - 7660
 - 7659
 - 7658
 - 7657
 supervisor.childopts: "-Djava.net.preferIPv4Stack=true"
 nimbus.childopts: "-Djava.net.preferIPv4Stack=true"
 worker.childopts: "-Djava.net.preferIPv4Stack=true"
 topology.stats.sample.rate: 1.0
 topology.message.timeout.secs: 300
 topology.acker.executors: 1
 storm.messaging.transport: "backtype.storm.messaging.netty.Context"
 storm.messaging.netty.server_worker_threads: 1
 storm.messaging.netty.client_worker_threads: 1
 storm.messaging.netty.buffer_size: 5242880
 storm.messaging.netty.max_retries: 100
 storm.messaging.netty.max_wait_ms: 1000
 storm.messaging.netty.min_wait_ms: 100

if you need details i can provide you , just tell me

My topolgoy has one spout -- 10 bolts -- spout.max.pending= 5000


Re: Should i have numbers in storm ui ?

2016-01-22 Thread researcher cs
can someone help ?

On Fri, Jan 22, 2016 at 9:55 AM, researcher cs <prog.researc...@gmail.com>
wrote:

> Should i have numbers in all columns after submitted topology in storm ui
> without click ShowSystemstats ?
>
> Or i must click the button to get numbers ?
>


Storm.yaml and configuration in code

2016-01-22 Thread researcher cs
If I set drpc.servers in code of topology , could it make conflict with
default valuesof drpc  in storm.yaml or not  ?


Should i have numbers in storm ui ?

2016-01-21 Thread researcher cs
Should i have numbers in all columns after submitted topology in storm ui
without click ShowSystemstats ?

Or i must click the button to get numbers ?


Re: STDIO [INFO] java.io.FileNotFoundException

2016-01-18 Thread researcher cs
i mean from last question is that error or not as i know if there is an
error it'll appear with [ERROR] ! so Should i check it agian or not ?

On Tue, Jan 19, 2016 at 9:25 AM, researcher cs <prog.researc...@gmail.com>
wrote:

> i've this statement in the worker log file
> STDIO [INFO] java.io.FileNotFoundException: file.txt (No such file or
> directory)
>
> i checked project.jar and didn't find the file there !
>
> i used mvn package to make a jar file for the project
>
> how can i add this file to jar file ?!
>
> and what does  STDIO mean ?
>


STDIO [INFO] java.io.FileNotFoundException

2016-01-18 Thread researcher cs
i've this statement in the worker log file
STDIO [INFO] java.io.FileNotFoundException: file.txt (No such file or
directory)

i checked project.jar and didn't find the file there !

i used mvn package to make a jar file for the project

how can i add this file to jar file ?!

and what does  STDIO mean ?


Re: Metrics,access and custom Log files

2016-01-13 Thread researcher cs
sorry i mean after submitted topology i found those files but empty ! how
can i use it  ?

On Wed, Jan 13, 2016 at 4:45 PM, researcher cs <prog.researc...@gmail.com>
wrote:

> There are files in storm/logs  like metrics , access and custom but
> there are empty how can i use it ?
>


topology.stats.sample.rate

2016-01-13 Thread researcher cs
what is this properties used for topology.stats.sample.rate ?


Metrics,access and custom Log files

2016-01-13 Thread researcher cs
There are files in storm/logs  like metrics , access and custom but
there are empty how can i use it ?


Show system stats button

2016-01-11 Thread researcher cs
when i submitted topology i got that in storm ui as you can see zeros in
emitted and transferred columns but there are numbers in executed column
and this before click show system stats



​
​but after click show system stats i got numbers in emitted and executed
columns but transeferred still has zeros !


​
1- i need to know is that normal ?
2- should i click show system stats after sumbitted topology to monitor it
OR  should i have numbers in emitted columns without click button

3- why transeferred has zeros ?


Firefox can't establish a connection to the server at nameofmachine:8000.

2016-01-11 Thread researcher cs
when i clicked port number in storm ui i got the following message

Unable to connect
Firefox can't establish a connection to the server at nameofmachine:8000.


Transferred and Acked is zeros !

2016-01-11 Thread researcher cs
After i submitted topology i got numbers in Emitted and executed coumns but
got zeros in Acked and Transferred columns

How can i fix it ?


Re: what should be in /etc/hosts ?

2016-01-10 Thread researcher cs
Please , Can you give me example ?

On Monday, January 11, 2016, John Fang <xiaojian@alibaba-inc.com> wrote:

> It has better  configure  domain and ip ;
>
> -邮件原件-
> 发件人: researcher cs [mailto:prog.researc...@gmail.com <javascript:;>]
> 发送时间: 2016年1月11日 7:44
> 收件人: dev@storm.apache.org <javascript:;>; u...@storm.apache.org
> <javascript:;>
> 主题: what should be in /etc/hosts ?
>
> i need to submit topology in single machine first can i ?
> and what should be in /etc/hosts for doing it
>
>


Re: Error on initialization of server mk-worker

2016-01-10 Thread researcher cs
is there any help ?

On Sat, Jan 9, 2016 at 8:03 PM, researcher cs <prog.researc...@gmail.com>
wrote:

> i've error in worker.log
> [ERROR] Error on initialization of server mk-worker
> java.io.IOException: No such file or directory
> at java.io.UnixFileSystem.createFileExclusively(Native Method)
> ~[na:1.6.0_37]
> at java.io.File.createNewFile(File.java:959) ~[na:1.6.0_37]
> at backtype.storm.util$touch.invoke(util.clj:462)
> ~[storm-core-0.9.0.1.jar:na]
> at
> backtype.storm.daemon.worker$fn__5882$exec_fn__1229__auto5883.invoke(worker.clj:337)
> ~[storm-core-0.9.0.1.jar:na]
> at clojure.lang.AFn.applyToHelper(AFn.java:185) ~[clojure-1.4.0.jar:na]
> at clojure.lang.AFn.applyTo(AFn.java:151) ~[clojure-1.4.0.jar:na]
> at clojure.core$apply.invoke(core.clj:601) ~[clojure-1.4.0.jar:na]
> at
> backtype.storm.daemon.worker$fn__5882$mk_worker__5938.doInvoke(worker.clj:329)
> ~[storm-core-0.9.0.1.jar:na]
> at clojure.lang.RestFn.invoke(RestFn.java:512) ~[clojure-1.4.0.jar:na]
> at backtype.storm.daemon.worker$_main.invoke(worker.clj:439)
> ~[storm-core-0.9.0.1.jar:na]
> at clojure.lang.AFn.applyToHelper(AFn.java:172) ~[clojure-1.4.0.jar:na]
> at clojure.lang.AFn.applyTo(AFn.java:151) ~[clojure-1.4.0.jar:na]
> at backtype.storm.daemon.worker.main(Unknown Source)
> ~[storm-core-0.9.0.1.jar:na]
> 2016-01-09 20:01:51 b.s.util [INFO] Halting process: ("Error on
> initialization")
>


Re: what are aporpraite configuration for this cluster ?

2016-01-10 Thread researcher cs
is there any help ?

On Sat, Jan 9, 2016 at 3:26 PM, researcher cs <prog.researc...@gmail.com>
wrote:

> FIrst i want to make sure that i get this point well from you
> i have topology i ran it with maven in local and worked successfully
>
> 1- Is execute code with maven here means Local mode ?
>
> 2- i want to run it in cluster i read that i can run it with single
> machine so
>
>- so with single machine means distributed too ?
>
>- what are the configurations should i have in storm.yaml
>
>  like this
>
> storm.zookeeper.servers:
>- "127.0.0.1"
>nimbus.host: "127.0.0.1"
>drpc.servers:
>- "127.0.0.1"
> * * i mean i will use localhost in all ip in the config ?
>
> 2- what if i want to run it with another machine "i have one worker only "
> so what are the configurations should i write in storm.yaml for
>
>   storm.zookeeper.servers
>   nimbus host
>   drpc.servers
> in the master and the other machine
>
>
> I Hope can find help to get answers well , THANKS
>


Re: [INFO] still hasn't start

2016-01-10 Thread researcher cs
is there any help ?

On Tue, Jan 5, 2016 at 4:52 PM, researcher cs <prog.researc...@gmail.com>
wrote:

> any help i'll apperciate it
>
> On Tue, Jan 5, 2016 at 5:33 AM, researcher cs <prog.researc...@gmail.com>
> wrote:
>
>> i forget to mention that the code using drpc i attached here too
>>
>> On Tue, Jan 5, 2016 at 3:52 AM, researcher cs <prog.researc...@gmail.com>
>> wrote:
>>
>>> i submitted topology using storm -0.9.0.1 , it's supposed no error in
>>> logs file except supervisor log file has [INFO] still hasn't start
>>>
>>> i can see worker in storm ui
>>>
>>> i attached screenshots for result in storm ui
>>>
>>> Question is :
>>> 1-   Should i see numbers in Emitted and transferred columns before
>>> click Show system stats Button ?
>>> or i must click it to see the numbers  ?
>>>
>>> 2- I imported project , i ran it in local mode without any problems but
>>> after submit it to a cluster Emitted coulmn has numbers But Transeferred
>>> has zeros and this after click show system stats button !
>>>
>>> 3- How can i know why workers still hasn't start ? however no errors and
>>> i can see it in stom ui ?
>>>
>>> 4- topology has one worker , Spout has one task , 10 Bolts , should i
>>> edit in topology.stats.sample.rate ? it's 0.05 now
>>>
>>> Thanks for you help in advance
>>>
>>> this image before click show system stats button
>>>
>>>
>>>
>>> this after click show system stats button
>>>
>>> ​
>>>
>>>
>>>
>>
>>
>


Re: Binding port from code

2016-01-10 Thread researcher cs
is there any help ?

On Sun, Jan 10, 2016 at 3:28 AM, researcher cs <prog.researc...@gmail.com>
wrote:

> What if I want to write configuration of storm in the topology code like
> binding port for supervisor to start worker , can I or not ?
>
> And if yes how can I stop reading configuration from storm.yaml when I
> submit topology
>


Re: Failed to bind to: 0.0.0.0/0.0.0.0:6703

2016-01-10 Thread researcher cs
I noticed that once I submitted topology. Conflict in port happened I mean
port are free until I submit the topology I got port already in use

I tried storm-0.8.2 and got zeromq already in  use
Then I upgrade version of storm and got failed to bind port
0.0.0.0/0.0.0.:6703

I changed the port but got same error

I checked the code but couldn't find any port in the code as I thought that
one I submit the topology it may be has port config so conflicting happened

Really I cannot fix it and finish all the attempts I have .
 Can anyone or the coder or Nathan can help !

On Saturday, January 9, 2016, researcher cs <prog.researc...@gmail.com>
wrote:

> sorry ip address of zookeeper, nimbus host and drpc is the same
>
> On Sat, Jan 9, 2016 at 1:43 AM, researcher cs <prog.researc...@gmail.com
> <javascript:_e(%7B%7D,'cvml','prog.researc...@gmail.com');>> wrote:
>
>> this is my storm.yaml
>>
>>  storm.zookeeper.servers:
>>  - "192.168.1.5"
>>  nimbus.host: "192.168.1.5"
>>
>>  supervisor.slots.ports:
>>- 7700
>>
>>  storm.local.dir: "/var/storm"
>>
>>  supervisor.childopts: "-Djava.net.preferIPv4Stack=true"
>>  nimbus.childopts: "-Djava.net.preferIPv4Stack=true"
>>  worker.childopts: "-Djava.net.preferIPv4Stack=true"
>>  drpc.servers:
>>  - "192.168.x.x"
>>  storm.messaging.transport: "backtype.storm.messaging.netty.Context"
>>  storm.messaging.netty.server_worker_threads: 1
>>  storm.messaging.netty.client_worker_threads: 1
>>  storm.messaging.netty.buffer_size: 5242880
>>  storm.messaging.netty.max_retries: 100
>>  storm.messaging.netty.max_wait_ms: 1000
>>  storm.messaging.netty.min_wait_ms: 100
>>
>> i checked it before but couldn't find anything wrong
>>
>>
>> On Fri, Jan 8, 2016 at 5:56 PM, Jean-Sebastien Vachon <
>> jean-sebastien.vac...@wantedanalytics.com
>> <javascript:_e(%7B%7D,'cvml','jean-sebastien.vac...@wantedanalytics.com');>
>> > wrote:
>>
>>> You won't find 0.0.0.0 (or you should not) in your /etc/hosts file. You
>>> have a process accepting connections on all known interfaces (localhost,
>>> 127.0.0.1 or any network addresses listed in ifconfig).
>>>
>>> You need to configure your UI, supervisor and worker to use a valid Ip
>>> address and different ports so that they can talk to each other over the
>>> network. If you're running two processes on the same box, then they must
>>> use different port numbers. Look at your storm.yaml file and init scripts
>>> to make sure that everything (IP and port) is correctly configured and you
>>> should be fine.
>>>
>>>
>>> --
>>> *From:* researcher cs <prog.researc...@gmail.com
>>> <javascript:_e(%7B%7D,'cvml','prog.researc...@gmail.com');>>
>>> *Sent:* Friday, January 8, 2016 10:50 AM
>>> *To:* Jean-Sebastien Vachon; u...@storm.apache.org
>>> <javascript:_e(%7B%7D,'cvml','u...@storm.apache.org');>;
>>> dev@storm.apache.org
>>> <javascript:_e(%7B%7D,'cvml','dev@storm.apache.org');>; Derek Dagit;
>>> Michael Vogiatzis; Michael Vogiatzis; nat...@nathanmarz.com
>>> <javascript:_e(%7B%7D,'cvml','nat...@nathanmarz.com');>
>>> *Subject:* Re: Failed to bind to: 0.0.0.0/0.0.0.0:6703
>>>
>>> Thanks, it's a good point , actually i searched for 0.0.0.0 and found
>>> that it read all ipv4 and ipv6 could you guide me how to fix it . here my
>>> /etc/hosts
>>> #127.0.0.1  localhost
>>> 192.168.x.x  computername
>>> #127.0.1.1 localhost
>>> # The following lines are desirable for IPv6 capable hosts
>>> #::1 ip6-localhost ip6-loopback
>>> #fe00::0 ip6-localnet
>>> #ff00::0 ip6-mcastprefix
>>> #ff02::1 ip6-allnodes
>>> #ff02::2 ip6-allrouters
>>> and when i used ifconfig
>>> eth0  Link encap:Ethernet  HWaddr 68:f7:28:a0:1b:b0
>>>   UP BROADCAST MULTICAST  MTU:1500  Metric:1
>>>   RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>   TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>>   collisions:0 txqueuelen:1000
>>>   RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
>>>
>>> loLink encap:Local Loopback
>>>   inet addr:127.0.0.1  Mask:255.0.0.0
>>>   UP LOOPBACK RUNNING  MTU:65536  Metric:1
>>>   RX packets:1899 errors:0 dropped:0 overruns:0 frame:0
>>>   TX 

what should be in /etc/hosts ?

2016-01-10 Thread researcher cs
i need to submit topology in single machine first can i ?
and what should be in /etc/hosts for doing it


what are aporpraite configuration for this cluster ?

2016-01-09 Thread researcher cs
FIrst i want to make sure that i get this point well from you
i have topology i ran it with maven in local and worked successfully

1- Is execute code with maven here means Local mode ?

2- i want to run it in cluster i read that i can run it with single machine
so

   - so with single machine means distributed too ?

   - what are the configurations should i have in storm.yaml

 like this

storm.zookeeper.servers:
   - "127.0.0.1"
   nimbus.host: "127.0.0.1"
   drpc.servers:
   - "127.0.0.1"
* * i mean i will use localhost in all ip in the config ?

2- what if i want to run it with another machine "i have one worker only "
so what are the configurations should i write in storm.yaml for

  storm.zookeeper.servers
  nimbus host
  drpc.servers
in the master and the other machine


I Hope can find help to get answers well , THANKS


Error on initialization of server mk-worker

2016-01-09 Thread researcher cs
i've error in worker.log
[ERROR] Error on initialization of server mk-worker
java.io.IOException: No such file or directory
at java.io.UnixFileSystem.createFileExclusively(Native Method)
~[na:1.6.0_37]
at java.io.File.createNewFile(File.java:959) ~[na:1.6.0_37]
at backtype.storm.util$touch.invoke(util.clj:462)
~[storm-core-0.9.0.1.jar:na]
at
backtype.storm.daemon.worker$fn__5882$exec_fn__1229__auto5883.invoke(worker.clj:337)
~[storm-core-0.9.0.1.jar:na]
at clojure.lang.AFn.applyToHelper(AFn.java:185) ~[clojure-1.4.0.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:151) ~[clojure-1.4.0.jar:na]
at clojure.core$apply.invoke(core.clj:601) ~[clojure-1.4.0.jar:na]
at
backtype.storm.daemon.worker$fn__5882$mk_worker__5938.doInvoke(worker.clj:329)
~[storm-core-0.9.0.1.jar:na]
at clojure.lang.RestFn.invoke(RestFn.java:512) ~[clojure-1.4.0.jar:na]
at backtype.storm.daemon.worker$_main.invoke(worker.clj:439)
~[storm-core-0.9.0.1.jar:na]
at clojure.lang.AFn.applyToHelper(AFn.java:172) ~[clojure-1.4.0.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:151) ~[clojure-1.4.0.jar:na]
at backtype.storm.daemon.worker.main(Unknown Source)
~[storm-core-0.9.0.1.jar:na]
2016-01-09 20:01:51 b.s.util [INFO] Halting process: ("Error on
initialization")


Binding port from code

2016-01-09 Thread researcher cs
What if I want to write configuration of storm in the topology code like
binding port for supervisor to start worker , can I or not ?

And if yes how can I stop reading configuration from storm.yaml when I
submit topology


Re: Failed to bind to: 0.0.0.0/0.0.0.0:6703

2016-01-08 Thread researcher cs
sorry ip address of zookeeper, nimbus host and drpc is the same

On Sat, Jan 9, 2016 at 1:43 AM, researcher cs <prog.researc...@gmail.com>
wrote:

> this is my storm.yaml
>
>  storm.zookeeper.servers:
>  - "192.168.1.5"
>  nimbus.host: "192.168.1.5"
>
>  supervisor.slots.ports:
>- 7700
>
>  storm.local.dir: "/var/storm"
>
>  supervisor.childopts: "-Djava.net.preferIPv4Stack=true"
>  nimbus.childopts: "-Djava.net.preferIPv4Stack=true"
>  worker.childopts: "-Djava.net.preferIPv4Stack=true"
>  drpc.servers:
>  - "192.168.x.x"
>  storm.messaging.transport: "backtype.storm.messaging.netty.Context"
>  storm.messaging.netty.server_worker_threads: 1
>  storm.messaging.netty.client_worker_threads: 1
>  storm.messaging.netty.buffer_size: 5242880
>  storm.messaging.netty.max_retries: 100
>  storm.messaging.netty.max_wait_ms: 1000
>  storm.messaging.netty.min_wait_ms: 100
>
> i checked it before but couldn't find anything wrong
>
>
> On Fri, Jan 8, 2016 at 5:56 PM, Jean-Sebastien Vachon <
> jean-sebastien.vac...@wantedanalytics.com> wrote:
>
>> You won't find 0.0.0.0 (or you should not) in your /etc/hosts file. You
>> have a process accepting connections on all known interfaces (localhost,
>> 127.0.0.1 or any network addresses listed in ifconfig).
>>
>> You need to configure your UI, supervisor and worker to use a valid Ip
>> address and different ports so that they can talk to each other over the
>> network. If you're running two processes on the same box, then they must
>> use different port numbers. Look at your storm.yaml file and init scripts
>> to make sure that everything (IP and port) is correctly configured and you
>> should be fine.
>>
>>
>> --
>> *From:* researcher cs <prog.researc...@gmail.com>
>> *Sent:* Friday, January 8, 2016 10:50 AM
>> *To:* Jean-Sebastien Vachon; u...@storm.apache.org; dev@storm.apache.org;
>> Derek Dagit; Michael Vogiatzis; Michael Vogiatzis; nat...@nathanmarz.com
>> *Subject:* Re: Failed to bind to: 0.0.0.0/0.0.0.0:6703
>>
>> Thanks, it's a good point , actually i searched for 0.0.0.0 and found
>> that it read all ipv4 and ipv6 could you guide me how to fix it . here my
>> /etc/hosts
>> #127.0.0.1  localhost
>> 192.168.x.x  computername
>> #127.0.1.1 localhost
>> # The following lines are desirable for IPv6 capable hosts
>> #::1 ip6-localhost ip6-loopback
>> #fe00::0 ip6-localnet
>> #ff00::0 ip6-mcastprefix
>> #ff02::1 ip6-allnodes
>> #ff02::2 ip6-allrouters
>> and when i used ifconfig
>> eth0  Link encap:Ethernet  HWaddr 68:f7:28:a0:1b:b0
>>   UP BROADCAST MULTICAST  MTU:1500  Metric:1
>>   RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>   TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>   collisions:0 txqueuelen:1000
>>   RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
>>
>> loLink encap:Local Loopback
>>   inet addr:127.0.0.1  Mask:255.0.0.0
>>   UP LOOPBACK RUNNING  MTU:65536  Metric:1
>>   RX packets:1899 errors:0 dropped:0 overruns:0 frame:0
>>   TX packets:1899 errors:0 dropped:0 overruns:0 carrier:0
>>   collisions:0 txqueuelen:0
>>   RX bytes:193309 (193.3 KB)  TX bytes:193309 (193.3 KB)
>>
>> wlan0 Link encap:Ethernet  HWaddr d0:53:49:ce:c5:2d
>>   inet addr:192.168.x.x  Bcast:192.168.1.255  Mask:255.255.255.0
>>   inet6 addr: fd58:2af7:1428:7200:d253:49ff:fece:c52d/64
>> Scope:Global
>>   inet6 addr: fe80::d253:49ff:fece:c52d/64 Scope:Link
>>   inet6 addr: fd58:2af7:1428:7200:d554:6712:d36f:95fa/64
>> Scope:Global
>>   UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>   RX packets:11503 errors:0 dropped:0 overruns:0 frame:0
>>   TX packets:11831 errors:0 dropped:0 overruns:0 carrier:0
>>   collisions:0 txqueuelen:1000
>>   RX bytes:8105372 (8.1 MB)  TX bytes:2425942 (2.4 MB)
>>
>
>


Re: Failed to bind to: 0.0.0.0/0.0.0.0:6703

2016-01-08 Thread researcher cs
this is my storm.yaml

 storm.zookeeper.servers:
 - "192.168.1.5"
 nimbus.host: "192.168.1.5"

 supervisor.slots.ports:
   - 7700

 storm.local.dir: "/var/storm"

 supervisor.childopts: "-Djava.net.preferIPv4Stack=true"
 nimbus.childopts: "-Djava.net.preferIPv4Stack=true"
 worker.childopts: "-Djava.net.preferIPv4Stack=true"
 drpc.servers:
 - "192.168.x.x"
 storm.messaging.transport: "backtype.storm.messaging.netty.Context"
 storm.messaging.netty.server_worker_threads: 1
 storm.messaging.netty.client_worker_threads: 1
 storm.messaging.netty.buffer_size: 5242880
 storm.messaging.netty.max_retries: 100
 storm.messaging.netty.max_wait_ms: 1000
 storm.messaging.netty.min_wait_ms: 100

i checked it before but couldn't find anything wrong


On Fri, Jan 8, 2016 at 5:56 PM, Jean-Sebastien Vachon <
jean-sebastien.vac...@wantedanalytics.com> wrote:

> You won't find 0.0.0.0 (or you should not) in your /etc/hosts file. You
> have a process accepting connections on all known interfaces (localhost,
> 127.0.0.1 or any network addresses listed in ifconfig).
>
> You need to configure your UI, supervisor and worker to use a valid Ip
> address and different ports so that they can talk to each other over the
> network. If you're running two processes on the same box, then they must
> use different port numbers. Look at your storm.yaml file and init scripts
> to make sure that everything (IP and port) is correctly configured and you
> should be fine.
>
>
> --
> *From:* researcher cs <prog.researc...@gmail.com>
> *Sent:* Friday, January 8, 2016 10:50 AM
> *To:* Jean-Sebastien Vachon; u...@storm.apache.org; dev@storm.apache.org;
> Derek Dagit; Michael Vogiatzis; Michael Vogiatzis; nat...@nathanmarz.com
> *Subject:* Re: Failed to bind to: 0.0.0.0/0.0.0.0:6703
>
> Thanks, it's a good point , actually i searched for 0.0.0.0 and found that
> it read all ipv4 and ipv6 could you guide me how to fix it . here my
> /etc/hosts
> #127.0.0.1  localhost
> 192.168.x.x  computername
> #127.0.1.1 localhost
> # The following lines are desirable for IPv6 capable hosts
> #::1 ip6-localhost ip6-loopback
> #fe00::0 ip6-localnet
> #ff00::0 ip6-mcastprefix
> #ff02::1 ip6-allnodes
> #ff02::2 ip6-allrouters
> and when i used ifconfig
> eth0  Link encap:Ethernet  HWaddr 68:f7:28:a0:1b:b0
>   UP BROADCAST MULTICAST  MTU:1500  Metric:1
>   RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>   TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>   collisions:0 txqueuelen:1000
>   RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
>
> loLink encap:Local Loopback
>   inet addr:127.0.0.1  Mask:255.0.0.0
>   UP LOOPBACK RUNNING  MTU:65536  Metric:1
>   RX packets:1899 errors:0 dropped:0 overruns:0 frame:0
>   TX packets:1899 errors:0 dropped:0 overruns:0 carrier:0
>   collisions:0 txqueuelen:0
>   RX bytes:193309 (193.3 KB)  TX bytes:193309 (193.3 KB)
>
> wlan0 Link encap:Ethernet  HWaddr d0:53:49:ce:c5:2d
>   inet addr:192.168.x.x  Bcast:192.168.1.255  Mask:255.255.255.0
>   inet6 addr: fd58:2af7:1428:7200:d253:49ff:fece:c52d/64
> Scope:Global
>   inet6 addr: fe80::d253:49ff:fece:c52d/64 Scope:Link
>   inet6 addr: fd58:2af7:1428:7200:d554:6712:d36f:95fa/64
> Scope:Global
>   UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>   RX packets:11503 errors:0 dropped:0 overruns:0 frame:0
>   TX packets:11831 errors:0 dropped:0 overruns:0 carrier:0
>   collisions:0 txqueuelen:1000
>   RX bytes:8105372 (8.1 MB)  TX bytes:2425942 (2.4 MB)
>


Re: Failed to bind to: 0.0.0.0/0.0.0.0:6703

2016-01-08 Thread researcher cs
Thanks, it's a good point , actually i searched for 0.0.0.0 and found that
it read all ipv4 and ipv6 could you guide me how to fix it . here my
/etc/hosts
#127.0.0.1  localhost
192.168.x.x  computername
#127.0.1.1 localhost
# The following lines are desirable for IPv6 capable hosts
#::1 ip6-localhost ip6-loopback
#fe00::0 ip6-localnet
#ff00::0 ip6-mcastprefix
#ff02::1 ip6-allnodes
#ff02::2 ip6-allrouters
and when i used ifconfig
eth0  Link encap:Ethernet  HWaddr 68:f7:28:a0:1b:b0
  UP BROADCAST MULTICAST  MTU:1500  Metric:1
  RX packets:0 errors:0 dropped:0 overruns:0 frame:0
  TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
  collisions:0 txqueuelen:1000
  RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

loLink encap:Local Loopback
  inet addr:127.0.0.1  Mask:255.0.0.0
  UP LOOPBACK RUNNING  MTU:65536  Metric:1
  RX packets:1899 errors:0 dropped:0 overruns:0 frame:0
  TX packets:1899 errors:0 dropped:0 overruns:0 carrier:0
  collisions:0 txqueuelen:0
  RX bytes:193309 (193.3 KB)  TX bytes:193309 (193.3 KB)

wlan0 Link encap:Ethernet  HWaddr d0:53:49:ce:c5:2d
  inet addr:192.168.x.x  Bcast:192.168.1.255  Mask:255.255.255.0
  inet6 addr: fd58:2af7:1428:7200:d253:49ff:fece:c52d/64
Scope:Global
  inet6 addr: fe80::d253:49ff:fece:c52d/64 Scope:Link
  inet6 addr: fd58:2af7:1428:7200:d554:6712:d36f:95fa/64
Scope:Global
  UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
  RX packets:11503 errors:0 dropped:0 overruns:0 frame:0
  TX packets:11831 errors:0 dropped:0 overruns:0 carrier:0
  collisions:0 txqueuelen:1000
  RX bytes:8105372 (8.1 MB)  TX bytes:2425942 (2.4 MB)


Re: Failed to bind to: 0.0.0.0/0.0.0.0:6703

2016-01-08 Thread researcher cs
i think the problem as you said in 0.0.0.0:0.0.00/7700 so how can i make it
listen to ip adderss like 192.168.x.x:7700 ?
i disabled ipv6 but without any change

is there any possible solution ?

On Fri, Jan 8, 2016 at 5:50 PM, researcher cs <prog.researc...@gmail.com>
wrote:

> Thanks, it's a good point , actually i searched for 0.0.0.0 and found that
> it read all ipv4 and ipv6 could you guide me how to fix it . here my
> /etc/hosts
> #127.0.0.1  localhost
> 192.168.x.x  computername
> #127.0.1.1 localhost
> # The following lines are desirable for IPv6 capable hosts
> #::1 ip6-localhost ip6-loopback
> #fe00::0 ip6-localnet
> #ff00::0 ip6-mcastprefix
> #ff02::1 ip6-allnodes
> #ff02::2 ip6-allrouters
> and when i used ifconfig
> eth0  Link encap:Ethernet  HWaddr 68:f7:28:a0:1b:b0
>   UP BROADCAST MULTICAST  MTU:1500  Metric:1
>   RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>   TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>   collisions:0 txqueuelen:1000
>   RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
>
> loLink encap:Local Loopback
>   inet addr:127.0.0.1  Mask:255.0.0.0
>   UP LOOPBACK RUNNING  MTU:65536  Metric:1
>   RX packets:1899 errors:0 dropped:0 overruns:0 frame:0
>   TX packets:1899 errors:0 dropped:0 overruns:0 carrier:0
>   collisions:0 txqueuelen:0
>   RX bytes:193309 (193.3 KB)  TX bytes:193309 (193.3 KB)
>
> wlan0 Link encap:Ethernet  HWaddr d0:53:49:ce:c5:2d
>   inet addr:192.168.x.x  Bcast:192.168.1.255  Mask:255.255.255.0
>   inet6 addr: fd58:2af7:1428:7200:d253:49ff:fece:c52d/64
> Scope:Global
>   inet6 addr: fe80::d253:49ff:fece:c52d/64 Scope:Link
>   inet6 addr: fd58:2af7:1428:7200:d554:6712:d36f:95fa/64
> Scope:Global
>   UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>   RX packets:11503 errors:0 dropped:0 overruns:0 frame:0
>   TX packets:11831 errors:0 dropped:0 overruns:0 carrier:0
>   collisions:0 txqueuelen:1000
>   RX bytes:8105372 (8.1 MB)  TX bytes:2425942 (2.4 MB)
>


What is the benefit of metrices and access log files ?

2016-01-04 Thread researcher cs
i was using storm-0.8.2 but now i'm using storm -0.9.0.1 after i submitted
topology found metrices and access log files how can i use them or what
should i do with them ?


Re: Failed to bind to: 0.0.0.0/0.0.0.0:6703

2016-01-04 Thread researcher cs
i checked it already and write the result i got here in seperate message ,
what i got is the topology i submitted it ! i can't see any other process ,
can you please check the message again to see if i'm right or not ?

On Mon, Jan 4, 2016 at 5:15 PM, Derek Dagit <der...@yahoo-inc.com.invalid>
wrote:

> > org.jboss.netty.channel.ChannelException: Failed to bind to:
> 0.0.0.0/0.0.0.0:6703
>
>
> If you see this, you can use a tool like lsof to find out what was
> listening on the port.
>
> `lsof -i :6703` as root user.
>
>
> Most likely, because it was port 6703, it was another worker JVM that was
> still running.
>
>
>
> --
> Derek
>
>
>
>
> 
> From: researcher cs <prog.researc...@gmail.com>
> To: dev@storm.apache.org; u...@storm.apache.org; nat...@nathanmarz.com;
> Michael Vogiatzis <mich...@micvog.com>; Michael Vogiatzis <
> michaelvogiat...@gmail.com>
> Sent: Sunday, January 3, 2016 6:09 PM
> Subject: Failed to bind to: 0.0.0.0/0.0.0.0:6703
>
>
>
> I'm using storm -0.9.0.1 with zookeeper -3.4.6 ubuntu 14.04
> now i ran my topology in local mode with maven commands and worked well
> BUT when i submitted it i got
>
> org.jboss.netty.channel.ChannelException: Failed to bind to:
> 0.0.0.0/0.0.0.0:6703
> at
> org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
> ~[netty-3.6.3.Final.jar:na]
> at backtype.storm.messaging.netty.Server.(Server.java:56)
> ~[storm-netty-0.9.0.1.jar:na]
> at backtype.storm.messaging.netty.Context.bind(Context.java:27)
> ~[storm-netty-0.9.0.1.jar:na]
> at
> backtype.storm.messaging.loader$launch_receive_thread_BANG_$fn__2984.invoke(loader.clj:20)
> ~[storm-core-0.9.0.1.jar:na]
> at backtype.storm.util$async_loop$fn__444.invoke(util.clj:401)
> ~[storm-core-0.9.0.1.jar:na]
> at clojure.lang.AFn.run(AFn.java:24) ~[clojure-1.4.0.jar:na]
> at java.lang.Thread.run(Thread.java:701) ~[na:1.6.0_37]
> Caused by: java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method) ~[na:1.6.0_37]
> at sun.nio.ch.Net.bind(Net.java:174) ~[na:1.6.0_37]
> at
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:139)
> ~[na:1.6.0_37]
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:77)
> ~[na:1.6.0_37]
> at
> org.jboss.netty.channel.socket.nio.NioServerBoss$RegisterTask.run(NioServerBoss.java:193)
> ~[netty-3.6.3.Final.jar:na]
> at
> org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:366)
> ~[netty-3.6.3.Final.jar:na]
> at
> org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:290)
> ~[netty-3.6.3.Final.jar:na]
> at
> org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
> ~[netty-3.6.3.Final.jar:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
> ~[na:1.6.0_37]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ~[na:1.6.0_37]
> ... 1 common frames omitted
> 2016-01-04 01:52:07 b.s.util [INFO] Halting process:
>
>
>
> the topology using one worker for one supervisor i'm trying to run or
> submit it first in single machine
>
>
> so my only storm.yaml is
>
> storm.zookeeper.servers:
>
>  - "192.168.x.x"
>
>  nimbus.host: "192.168.x.x"
>  storm.local.dir: "/var/storm"
>  supervisor.childopts: "-Djava.net.preferIPv4Stack=true"
>  nimbus.childopts: "-Djava.net.preferIPv4Stack=true"
>  worker.childopts: "-Djava.net.preferIPv4Stack=true"
>  drpc.servers:
>
>  - "192.168.x.x"
>
> storm.messaging.transport: "backtype.storm.messaging.netty.Context"
>  storm.messaging.netty.server_worker_threads: 1
>  storm.messaging.netty.client_worker_threads: 1
>  storm.messaging.netty.buffer_size: 5242880
>  storm.messaging.netty.max_retries: 100
>  storm.messaging.netty.max_wait_ms: 1000
>  storm.messaging.netty.min_wait_ms: 100
>
>
> zoo.cfg is
>
> tickTime=2000
> initLimit=10
> syncLimit=5
> dataDir=/var/zookeeper
> clientPort=2181
>
>
> i tried to use localhost but got same error then tried to use my ip address
> i searched to fix it more but got this solution
> can someone illustrate it beacuse i didn't get it well
>
> [[
>
> It turned out that my logback/cluster.xml configuration had a
> SocketAppender that didn't have a listener on the remote host. Having an
> appender defined in configuration file, even if it is not set up to receive
> any log messages, will cause Storm to block and the above error message.its
> due to zookeeper you have add curator separetly not required its the part
> of zookeeper
> ]]
>
>
> or can someone help me to fix it ?
>
>
> importaant thing
>
>
> topology using drpc and i tried to submit another topoogies worked well
> but topology with drpc got this error
>
>
> the project i'm trying to submit it here
> https://github.com/mvogiatzis/first-stories-twitter
>


[INFO] still hasn't start

2016-01-04 Thread researcher cs
i submitted topology using storm -0.9.0.1 , it's supposed no error in logs
file except supervisor log file has [INFO] still hasn't start

i can see worker in storm ui

i attached screenshots for result in storm ui

Question is :
1-   Should i see numbers in Emitted and transferred columns before click
Show system stats Button ?
or i must click it to see the numbers  ?

2- I imported project , i ran it in local mode without any problems but
after submit it to a cluster Emitted coulmn has numbers But Transeferred
has zeros and this after click show system stats button !

3- How can i know why workers still hasn't start ? however no errors and i
can see it in stom ui ?

4- topology has one worker , Spout has one task , 10 Bolts , should i edit
in topology.stats.sample.rate ? it's 0.05 now

Thanks for you help in advance

this image before click show system stats button



this after click show system stats button

​


show system stats in storm ui

2016-01-03 Thread researcher cs
what is the purpose of this button ?
i clicked it found column of emitted after become zeros , it have numbers
and everythind is ok as i thought but transfered still zeros

but after hide system status emitted return to zeros again

thanks


Re: Failed to bind to: 0.0.0.0/0.0.0.0:6703

2016-01-03 Thread researcher cs
i checked the port with its process got

root  8265  8009 33 02:36 pts/800:00:16 java -server
-Djava.net.preferIPv4Stack=true
-Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
-Dlogfile.name=worker-6703.log -Dstorm.home=/usr/local/storm
-Dlogback.configurationFile=/usr/local/storm/logback/cluster.xml
-Dstorm.id=fsd-1-1451867773
-Dworker.id=e44a8daf-313d-4f7d-a67f-5b900eee3764 -Dworker.port=6703 -cp
/usr/local/storm/storm-core-0.9.0.1.jar:/usr/local/storm/storm-netty-0.9.0.1.jar:/usr/local/storm/storm-console-logging-0.9.0.1.jar:/usr/local/storm/lib/netty-3.6.3.Final.jar:/usr/local/storm/lib/commons-exec-1.1.jar:/usr/local/storm/lib/ring-devel-0.3.11.jar:/usr/local/storm/lib/carbonite-1.5.0.jar:/usr/local/storm/lib/ring-jetty-adapter-0.3.11.jar:/usr/local/storm/lib/minlog-1.2.jar:/usr/local/storm/lib/tools.nrepl-0.2.3.jar:/usr/local/storm/lib/hiccup-0.3.6.jar:/usr/local/storm/lib/commons-lang-2.5.jar:/usr/local/storm/lib/math.numeric-tower-0.0.1.jar:/usr/local/storm/lib/servlet-api-2.5-20081211.jar:/usr/local/storm/lib/commons-logging-1.1.1.jar:/usr/local/storm/lib/tools.logging-0.2.3.jar:/usr/local/storm/lib/clout-1.0.1.jar:/usr/local/storm/lib/httpcore-4.1.jar:/usr/local/storm/lib/logback-classic-1.0.6.jar:/usr/local/storm/lib/servlet-api-2.5.jar:/usr/local/storm/lib/objenesis-1.2.jar:/usr/local/storm/lib/clojure-1.4.0.jar:/usr/local/storm/lib/zookeeper-3.4.6.jar:/usr/local/storm/lib/snakeyaml-1.11.jar:/usr/local/storm/lib/json-simple-1.1.jar:/usr/local/storm/lib/clj-stacktrace-0.2.2.jar:/usr/local/storm/lib/httpclient-4.1.1.jar:/usr/local/storm/lib/jzmq-2.1.0.jar:/usr/local/storm/lib/core.incubator-0.1.0.jar:/usr/local/storm/lib/libthrift7-0.7.0-2.jar:/usr/local/storm/lib/slf4j-api-1.6.5.jar:/usr/local/storm/lib/junit-3.8.1.jar:/usr/local/storm/lib/guava-13.0.jar:/usr/local/storm/lib/logback-core-1.0.6.jar:/usr/local/storm/lib/jgrapht-0.8.3.jar:/usr/local/storm/lib/commons-io-1.4.jar:/usr/local/storm/lib/curator-framework-1.0.1.jar:/usr/local/storm/lib/ring-servlet-0.3.11.jar:/usr/local/storm/lib/tools.macro-0.1.0.jar:/usr/local/storm/lib/asm-4.0.jar:/usr/local/storm/lib/jetty-6.1.26.jar:/usr/local/storm/lib/reflectasm-1.07-shaded.jar:/usr/local/storm/lib/joda-time-2.0.jar:/usr/local/storm/lib/jline-0.9.94.jar:/usr/local/storm/lib/tools.cli-0.2.2.jar:/usr/local/storm/lib/commons-fileupload-1.2.1.jar:/usr/local/storm/lib/zookeeper-3.3.3.jar:/usr/local/storm/lib/kryo-2.17.jar:/usr/local/storm/lib/clojure-complete-0.2.3.jar:/usr/local/storm/lib/compojure-1.1.3.jar:/usr/local/storm/lib/curator-client-1.0.1.jar:/usr/local/storm/lib/jetty-util-6.1.26.jar:/usr/local/storm/lib/ring-core-1.1.5.jar:/usr/local/storm/lib/disruptor-2.10.1.jar:/usr/local/storm/lib/commons-codec-1.4.jar:/usr/local/storm/lib/mockito-all-1.9.5.jar:/usr/local/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/local/storm/lib/clj-time-0.4.1.jar:/usr/local/storm/conf:/var/storm/supervisor/stormdist/fsd-1-1451867773/stormjar.jar
backtype.storm.daemon.worker fsd-1-1451867773
34879f97-2ce0-479c-b555-944a9bd50b9d 6703
e44a8daf-313d-4f7d-a67f-5b900eee3764

user  8488  7018  0 02:37 pts/30   00:00:00 grep --color=auto 8265

may be it help ! waiting your reply , Thanks in advance


On Mon, Jan 4, 2016 at 2:10 AM, researcher cs <prog.researc...@gmail.com>
wrote:

> sorry it's not my toplogy i refered to the coder hope he can help in this
> issue
>
> On Mon, Jan 4, 2016 at 2:09 AM, researcher cs <prog.researc...@gmail.com>
> wrote:
>
>> I'm using storm -0.9.0.1 with zookeeper -3.4.6 ubuntu 14.04
>> now i ran my topology in local mode with maven commands and worked well
>> BUT when i submitted it i got
>>
>> org.jboss.netty.channel.ChannelException: Failed to bind to:
>> 0.0.0.0/0.0.0.0:6703
>> at
>> org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
>> ~[netty-3.6.3.Final.jar:na]
>> at backtype.storm.messaging.netty.Server.(Server.java:56)
>> ~[storm-netty-0.9.0.1.jar:na]
>> at backtype.storm.messaging.netty.Context.bind(Context.java:27)
>> ~[storm-netty-0.9.0.1.jar:na]
>> at
>> backtype.storm.messaging.loader$launch_receive_thread_BANG_$fn__2984.invoke(loader.clj:20)
>> ~[storm-core-0.9.0.1.jar:na]
>> at backtype.storm.util$async_loop$fn__444.invoke(util.clj:401)
>> ~[storm-core-0.9.0.1.jar:na]
>> at clojure.lang.AFn.run(AFn.java:24) ~[clojure-1.4.0.jar:na]
>> at java.lang.Thread.run(Thread.java:701) ~[na:1.6.0_37]
>> Caused by: java.net.BindException: Address already in use
>> at sun.nio.ch.Net.bind0(Native Method) ~[na:1.6.0_37]
>> at sun.nio.ch.Net.bind(Net.java:174) ~[na:1.6.0_37]
>> at
>> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:139)
>> ~[na:1.6.0_37]
>> at sun.nio.ch.ServerSocket

Failed to bind to: 0.0.0.0/0.0.0.0:6703

2016-01-03 Thread researcher cs
I'm using storm -0.9.0.1 with zookeeper -3.4.6 ubuntu 14.04
now i ran my topology in local mode with maven commands and worked well
BUT when i submitted it i got

org.jboss.netty.channel.ChannelException: Failed to bind to:
0.0.0.0/0.0.0.0:6703
at
org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
~[netty-3.6.3.Final.jar:na]
at backtype.storm.messaging.netty.Server.(Server.java:56)
~[storm-netty-0.9.0.1.jar:na]
at backtype.storm.messaging.netty.Context.bind(Context.java:27)
~[storm-netty-0.9.0.1.jar:na]
at
backtype.storm.messaging.loader$launch_receive_thread_BANG_$fn__2984.invoke(loader.clj:20)
~[storm-core-0.9.0.1.jar:na]
at backtype.storm.util$async_loop$fn__444.invoke(util.clj:401)
~[storm-core-0.9.0.1.jar:na]
at clojure.lang.AFn.run(AFn.java:24) ~[clojure-1.4.0.jar:na]
at java.lang.Thread.run(Thread.java:701) ~[na:1.6.0_37]
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method) ~[na:1.6.0_37]
at sun.nio.ch.Net.bind(Net.java:174) ~[na:1.6.0_37]
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:139)
~[na:1.6.0_37]
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:77)
~[na:1.6.0_37]
at
org.jboss.netty.channel.socket.nio.NioServerBoss$RegisterTask.run(NioServerBoss.java:193)
~[netty-3.6.3.Final.jar:na]
at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:366)
~[netty-3.6.3.Final.jar:na]
at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:290)
~[netty-3.6.3.Final.jar:na]
at
org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
~[netty-3.6.3.Final.jar:na]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
~[na:1.6.0_37]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
~[na:1.6.0_37]
... 1 common frames omitted
2016-01-04 01:52:07 b.s.util [INFO] Halting process:

the topology using one worker for one supervisor i'm trying to run or
submit it first in single machine

so my only storm.yaml is

storm.zookeeper.servers:
 - "192.168.x.x"
 nimbus.host: "192.168.x.x"
 storm.local.dir: "/var/storm"
 supervisor.childopts: "-Djava.net.preferIPv4Stack=true"
 nimbus.childopts: "-Djava.net.preferIPv4Stack=true"
 worker.childopts: "-Djava.net.preferIPv4Stack=true"
 drpc.servers:
 - "192.168.x.x"
storm.messaging.transport: "backtype.storm.messaging.netty.Context"
 storm.messaging.netty.server_worker_threads: 1
 storm.messaging.netty.client_worker_threads: 1
 storm.messaging.netty.buffer_size: 5242880
 storm.messaging.netty.max_retries: 100
 storm.messaging.netty.max_wait_ms: 1000
 storm.messaging.netty.min_wait_ms: 100

zoo.cfg is

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/var/zookeeper
clientPort=2181

i tried to use localhost but got same error then tried to use my ip address
i searched to fix it more but got this solution
can someone illustrate it beacuse i didn't get it well

[[
It turned out that my logback/cluster.xml configuration had a
SocketAppender that didn't have a listener on the remote host. Having an
appender defined in configuration file, even if it is not set up to receive
any log messages, will cause Storm to block and the above error message.its
due to zookeeper you have add curator separetly not required its the part
of zookeeper
]]

or can someone help me to fix it ?

importaant thing

topology using drpc and i tried to submit another topoogies worked well but
topology with drpc got this error


the project i'm trying to submit it here
https://github.com/mvogiatzis/first-stories-twitter


Re: zookeeper setup for storm cluster

2015-12-24 Thread researcher cs
i implemented what your wrote in the link but got in the storm ui zeros in
emitted and transfered
also have in the supervisor log file couldn't extract resources from jar
file then got still hasn't start

i got at hte end of the drpc log file  Starting Distributed RPC servers...



On Thu, Dec 24, 2015 at 9:27 PM, researcher cs <prog.researc...@gmail.com>
wrote:

> many thanks for your answers , sorry for my lot question but i need to
> understand many thanks for you
>
> Now i tested the topology bu this command
>
> 1-mvn package
> mvn compile exec:java -Dexec.classpathScope=compile
> -Dexec.mainClass=trident.Topology
>
> that mean i tested it in local rigth  ?
>
>
> 2- now i want to implement 2 point which coder wrote
> "production mode "
> you said that i should submit the topology first in single machine with
> the above configuration right ?
>
> 3- should i write myipaddress started by 192.168.x.x or 127.0.0.1 ?
> 4- ipaddress i will write it in
>
> storm.zookeeper.servers:
>  - "ipaddress"
> nimbus.host: "ipaddress"
> drpc.servers:
>- "localhost"
>
> are the same ?
> 5- what do you mean by host.name ?
> 6- in file /etc/hosts
> 127.0.0.1  localhost
> 127.0.1.1  jhk
>
> # The following lines are desirable for IPv6 capable hosts
> #::1 ip6-localhost ip6-loopback
> #fe00::0 ip6-localnet
> #ff00::0 ip6-mcastprefix
> #ff02::1 ip6-allnodes
> #ff02::2 ip6-allrouters
> should i hashing ipv6 ?
>
> On Thu, Dec 24, 2015 at 9:17 PM, Harsha <m...@harsha.io> wrote:
>
>> 1- is production. mode means distributed mode ?
>>Yes. Typically more than one machine. But I am suggesting if you want
>>to test this topology you can do so by using single node cluster as
>>well.
>> 2. Typical production clusters usually have more than 1 node.
>> 3. Usually there will be one config file that gets deployed across the
>> nodes and on each node you choose start a nimbus and other nodes will
>> have supervisors.
>> 4. The config you've is suffice and defaults will takes place for the
>> configs you don't mention.
>> 5.  drpc.servers:
>> - "localhost" you should replace this host.name
>>
>>
>>
>> -Harsha
>>
>> On Thu, Dec 24, 2015, at 11:12 AM, researcher cs wrote:
>> > thanks i checked the link it's good for me but the question that i'm
>> > searching about
>> > 1- is production mode means distributed mode ?
>> > 2- if question 1 = yes , is that mean working on more than one machine ?
>> > 3- if question 2 = yes , is there specific configuration should i wrote
>> > in
>> > storm.yaml and zoo.cfg means is that configuration right
>> > storm.zookeeper.servers:
>> >  - "ipaddress"
>> > nimbus.host: "ipaddress"
>> > storm.zookeeper.port: 2181
>> >
>> > storm.local.dir: /home/storm
>> >
>> > supervisor.slots.ports:
>> > - 6700
>> > - 6701
>> > - 6702
>> > - 6703
>> > drpc.port: 3772
>> > drpc.servers:
>> >- "localhost"
>> >
>> > and zoo.cfg is
>> > tickTime=2000
>> > dataDir=/home/storm/zookeeper
>> > clientPort=2181
>> > initLimit=5
>> > syncLimit=2
>> >
>> > sorry for lot questions i'm searching to find a good solution but i hope
>> > you can help me , Thanks
>> >
>> > On Thu, Dec 24, 2015 at 9:06 PM, Harsha <m...@harsha.io> wrote:
>> >
>> > > You can actually use single node cluster for testing this out. The
>> link
>> > > I gave you have instructions to start all the required storm services
>> > > and once thats up and running you can use storm jar to deploy the
>> > > topology.
>> > >
>> > > On Thu, Dec 24, 2015, at 10:58 AM, researcher cs wrote:
>> > > > thanks for replying , but what i mean that i found project using
>> storm
>> > > > and
>> > > > coder wrote this lines
>> > > >
>> > > > *To compile and run the project in local mode*, type the following
>> > > > commands
>> > > > while being on the project root directory
>> > > > mvn package
>> > > > mvn compile exec:java -Dexec.classpathScope=compile
>> > > > -Dexec.mainClass=trident.topology_name
>> > > >
>> > > > *To run in production cluster*, first package the code into a jar by
>> > > > running
&g

need to get what this statement mean

2015-12-24 Thread researcher cs
what is user-defined Storm Lib means and this statement too

"pointed java library path to directory
with that executable "?


Re: zookeeper setup for storm cluster

2015-12-24 Thread researcher cs
thanks for replying , but what i mean that i found project using storm and
coder wrote this lines

*To compile and run the project in local mode*, type the following commands
while being on the project root directory
mvn package
mvn compile exec:java -Dexec.classpathScope=compile
-Dexec.mainClass=trident.topology_name

*To run in production cluster*, first package the code into a jar by running
mvn package.
This will package your code into a jar at the path
target/Topology_name-{version}-jar-with-dependencies.jar.
Then you can submit your jar to the cluster using the storm client:
storm jar target/Topology_namr-1.0-SNAPSHOT-jar-with-dependencies.jar
trident.Topology_name

Please note that in the production cluster mode, you require a DRPCClient
to feed the topology with tweets and get results.


i ran it in local mode by the above commands and worked well but now i want
to run it in distrbuted mode as he wote i need to submit the topology to
storm but what i should need in configuration for zoo.cfg and storm.yaml
for working ?

should i need more than single machine or not ?

how it can be in distributed mode in single machine ?


Thanks for your help



On Thu, Dec 24, 2015 at 7:43 PM, Harsha <st...@harsha.io> wrote:

> In storm, local mode means you run by using LocalCluster class its a
> simulated cluster for testing topologies and aid development of
> topologies . Example of LocalCluster
>
> https://github.com/apache/storm/blob/master/examples/storm-starter/src/jvm/storm/starter/WordCountTopology.java#L99
>
> In production it would be a distributed cluster. One needs to setup a
> distributed cluster and config you showed seems ok .
> You can follow steps here setup up a single node cluster
> http://blog.harsha.io/setting-up-a-single-node-apache-storm-cluster/
> and same can be extended for multi-node cluster.
>
> I am not quite sure about what you mean by importing a topology that run
> in two modes in local or production. In the first link I gave you ,
> wordcount topology can run local cluster and on distributed as well.
>
> -Harsha
>
> On Thu, Dec 24, 2015, at 08:54 AM, researcher cs wrote:
> > can i find help ?
> >
> > On Thu, Dec 24, 2015 at 7:08 AM, researcher cs
> > <prog.researc...@gmail.com>
> > wrote:
> >
> > > I want to import a project that run in two modes "local and production"
> > > mode
> > >
> > > want to get what is mean by production mode is that mean in cluster or
> > > distributed mode ?
> > >
> > > and if that right . are the configurations in storm.yaml is like that
> or
> > > not ?
> > > storm.zookeeper.servers:
> > >  - "ipaddress"
> > > nimbus.host: "ipaddress"
> > > storm.zookeeper.port: 2181
> > >
> > > storm.local.dir: /home/storm
> > >
> > > supervisor.slots.ports:
> > > - 6700
> > > - 6701
> > > - 6702
> > > - 6703
> > > drpc.port: 3772
> > > drpc.servers:
> > >- "localhost"
> > >
> > > and zoo.cfg is
> > > tickTime=2000
> > > dataDir=/home/storm/zookeeper
> > > clientPort=2181
> > > initLimit=5
> > > syncLimit=2
> > >
>


Re: zookeeper setup for storm cluster

2015-12-24 Thread researcher cs
thanks i checked the link it's good for me but the question that i'm
searching about
1- is production mode means distributed mode ?
2- if question 1 = yes , is that mean working on more than one machine ?
3- if question 2 = yes , is there specific configuration should i wrote in
storm.yaml and zoo.cfg means is that configuration right
storm.zookeeper.servers:
 - "ipaddress"
nimbus.host: "ipaddress"
storm.zookeeper.port: 2181

storm.local.dir: /home/storm

supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
drpc.port: 3772
drpc.servers:
   - "localhost"

and zoo.cfg is
tickTime=2000
dataDir=/home/storm/zookeeper
clientPort=2181
initLimit=5
syncLimit=2

sorry for lot questions i'm searching to find a good solution but i hope
you can help me , Thanks

On Thu, Dec 24, 2015 at 9:06 PM, Harsha <m...@harsha.io> wrote:

> You can actually use single node cluster for testing this out. The link
> I gave you have instructions to start all the required storm services
> and once thats up and running you can use storm jar to deploy the
> topology.
>
> On Thu, Dec 24, 2015, at 10:58 AM, researcher cs wrote:
> > thanks for replying , but what i mean that i found project using storm
> > and
> > coder wrote this lines
> >
> > *To compile and run the project in local mode*, type the following
> > commands
> > while being on the project root directory
> > mvn package
> > mvn compile exec:java -Dexec.classpathScope=compile
> > -Dexec.mainClass=trident.topology_name
> >
> > *To run in production cluster*, first package the code into a jar by
> > running
> > mvn package.
> > This will package your code into a jar at the path
> > target/Topology_name-{version}-jar-with-dependencies.jar.
> > Then you can submit your jar to the cluster using the storm client:
> > storm jar target/Topology_namr-1.0-SNAPSHOT-jar-with-dependencies.jar
> > trident.Topology_name
> >
> > Please note that in the production cluster mode, you require a DRPCClient
> > to feed the topology with tweets and get results.
> >
> >
> > i ran it in local mode by the above commands and worked well but now i
> > want
> > to run it in distrbuted mode as he wote i need to submit the topology to
> > storm but what i should need in configuration for zoo.cfg and storm.yaml
> > for working ?
> >
> > should i need more than single machine or not ?
> >
> > how it can be in distributed mode in single machine ?
> >
> >
> > Thanks for your help
> >
> >
> >
> > On Thu, Dec 24, 2015 at 7:43 PM, Harsha <st...@harsha.io> wrote:
> >
> > > In storm, local mode means you run by using LocalCluster class its a
> > > simulated cluster for testing topologies and aid development of
> > > topologies . Example of LocalCluster
> > >
> > >
> https://github.com/apache/storm/blob/master/examples/storm-starter/src/jvm/storm/starter/WordCountTopology.java#L99
> > >
> > > In production it would be a distributed cluster. One needs to setup a
> > > distributed cluster and config you showed seems ok .
> > > You can follow steps here setup up a single node cluster
> > > http://blog.harsha.io/setting-up-a-single-node-apache-storm-cluster/
> > > and same can be extended for multi-node cluster.
> > >
> > > I am not quite sure about what you mean by importing a topology that
> run
> > > in two modes in local or production. In the first link I gave you ,
> > > wordcount topology can run local cluster and on distributed as well.
> > >
> > > -Harsha
> > >
> > > On Thu, Dec 24, 2015, at 08:54 AM, researcher cs wrote:
> > > > can i find help ?
> > > >
> > > > On Thu, Dec 24, 2015 at 7:08 AM, researcher cs
> > > > <prog.researc...@gmail.com>
> > > > wrote:
> > > >
> > > > > I want to import a project that run in two modes "local and
> production"
> > > > > mode
> > > > >
> > > > > want to get what is mean by production mode is that mean in
> cluster or
> > > > > distributed mode ?
> > > > >
> > > > > and if that right . are the configurations in storm.yaml is like
> that
> > > or
> > > > > not ?
> > > > > storm.zookeeper.servers:
> > > > >  - "ipaddress"
> > > > > nimbus.host: "ipaddress"
> > > > > storm.zookeeper.port: 2181
> > > > >
> > > > > storm.local.dir: /home/storm
> > > > >
> > > > > supervisor.slots.ports:
> > > > > - 6700
> > > > > - 6701
> > > > > - 6702
> > > > > - 6703
> > > > > drpc.port: 3772
> > > > > drpc.servers:
> > > > >- "localhost"
> > > > >
> > > > > and zoo.cfg is
> > > > > tickTime=2000
> > > > > dataDir=/home/storm/zookeeper
> > > > > clientPort=2181
> > > > > initLimit=5
> > > > > syncLimit=2
> > > > >
> > >
>


Re: zookeeper setup for storm cluster

2015-12-24 Thread researcher cs
many thanks for your answers , sorry for my lot question but i need to
understand many thanks for you

Now i tested the topology bu this command

1-mvn package
mvn compile exec:java -Dexec.classpathScope=compile
-Dexec.mainClass=trident.Topology

that mean i tested it in local rigth  ?


2- now i want to implement 2 point which coder wrote
"production mode "
you said that i should submit the topology first in single machine with the
above configuration right ?

3- should i write myipaddress started by 192.168.x.x or 127.0.0.1 ?
4- ipaddress i will write it in

storm.zookeeper.servers:
 - "ipaddress"
nimbus.host: "ipaddress"
drpc.servers:
   - "localhost"

are the same ?
5- what do you mean by host.name ?
6- in file /etc/hosts
127.0.0.1  localhost
127.0.1.1  jhk

# The following lines are desirable for IPv6 capable hosts
#::1 ip6-localhost ip6-loopback
#fe00::0 ip6-localnet
#ff00::0 ip6-mcastprefix
#ff02::1 ip6-allnodes
#ff02::2 ip6-allrouters
should i hashing ipv6 ?

On Thu, Dec 24, 2015 at 9:17 PM, Harsha <m...@harsha.io> wrote:

> 1- is production. mode means distributed mode ?
>Yes. Typically more than one machine. But I am suggesting if you want
>to test this topology you can do so by using single node cluster as
>well.
> 2. Typical production clusters usually have more than 1 node.
> 3. Usually there will be one config file that gets deployed across the
> nodes and on each node you choose start a nimbus and other nodes will
> have supervisors.
> 4. The config you've is suffice and defaults will takes place for the
> configs you don't mention.
> 5.  drpc.servers:
> - "localhost" you should replace this host.name
>
>
>
> -Harsha
>
> On Thu, Dec 24, 2015, at 11:12 AM, researcher cs wrote:
> > thanks i checked the link it's good for me but the question that i'm
> > searching about
> > 1- is production mode means distributed mode ?
> > 2- if question 1 = yes , is that mean working on more than one machine ?
> > 3- if question 2 = yes , is there specific configuration should i wrote
> > in
> > storm.yaml and zoo.cfg means is that configuration right
> > storm.zookeeper.servers:
> >  - "ipaddress"
> > nimbus.host: "ipaddress"
> > storm.zookeeper.port: 2181
> >
> > storm.local.dir: /home/storm
> >
> > supervisor.slots.ports:
> > - 6700
> > - 6701
> > - 6702
> > - 6703
> > drpc.port: 3772
> > drpc.servers:
> >- "localhost"
> >
> > and zoo.cfg is
> > tickTime=2000
> > dataDir=/home/storm/zookeeper
> > clientPort=2181
> > initLimit=5
> > syncLimit=2
> >
> > sorry for lot questions i'm searching to find a good solution but i hope
> > you can help me , Thanks
> >
> > On Thu, Dec 24, 2015 at 9:06 PM, Harsha <m...@harsha.io> wrote:
> >
> > > You can actually use single node cluster for testing this out. The link
> > > I gave you have instructions to start all the required storm services
> > > and once thats up and running you can use storm jar to deploy the
> > > topology.
> > >
> > > On Thu, Dec 24, 2015, at 10:58 AM, researcher cs wrote:
> > > > thanks for replying , but what i mean that i found project using
> storm
> > > > and
> > > > coder wrote this lines
> > > >
> > > > *To compile and run the project in local mode*, type the following
> > > > commands
> > > > while being on the project root directory
> > > > mvn package
> > > > mvn compile exec:java -Dexec.classpathScope=compile
> > > > -Dexec.mainClass=trident.topology_name
> > > >
> > > > *To run in production cluster*, first package the code into a jar by
> > > > running
> > > > mvn package.
> > > > This will package your code into a jar at the path
> > > > target/Topology_name-{version}-jar-with-dependencies.jar.
> > > > Then you can submit your jar to the cluster using the storm client:
> > > > storm jar target/Topology_namr-1.0-SNAPSHOT-jar-with-dependencies.jar
> > > > trident.Topology_name
> > > >
> > > > Please note that in the production cluster mode, you require a
> DRPCClient
> > > > to feed the topology with tweets and get results.
> > > >
> > > >
> > > > i ran it in local mode by the above commands and worked well but now
> i
> > > > want
> > > > to run it in distrbuted mode as he wote i need to submit the
> topology to
> > > > st

Re: zookeeper setup for storm cluster

2015-12-24 Thread researcher cs
can i find help ?

On Thu, Dec 24, 2015 at 7:08 AM, researcher cs <prog.researc...@gmail.com>
wrote:

> I want to import a project that run in two modes "local and production"
> mode
>
> want to get what is mean by production mode is that mean in cluster or
> distributed mode ?
>
> and if that right . are the configurations in storm.yaml is like that or
> not ?
> storm.zookeeper.servers:
>  - "ipaddress"
> nimbus.host: "ipaddress"
> storm.zookeeper.port: 2181
>
> storm.local.dir: /home/storm
>
> supervisor.slots.ports:
> - 6700
> - 6701
> - 6702
> - 6703
> drpc.port: 3772
> drpc.servers:
>- "localhost"
>
> and zoo.cfg is
> tickTime=2000
> dataDir=/home/storm/zookeeper
> clientPort=2181
> initLimit=5
> syncLimit=2
>


zookeeper setup for storm cluster

2015-12-23 Thread researcher cs
I want to import a project that run in two modes "local and production"
mode

want to get what is mean by production mode is that mean in cluster or
distributed mode ?

and if that right . are the configurations in storm.yaml is like that or
not ?
storm.zookeeper.servers:
 - "ipaddress"
nimbus.host: "ipaddress"
storm.zookeeper.port: 2181

storm.local.dir: /home/storm

supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
drpc.port: 3772
drpc.servers:
   - "localhost"

and zoo.cfg is
tickTime=2000
dataDir=/home/storm/zookeeper
clientPort=2181
initLimit=5
syncLimit=2