thanks but Agian i imported this project from github that it's supposed work well , really i'm on this problem for months without fixing it !! i sent coder message but he didn't reply with this problem . don't know who can i ask but thanks for your time
On Thu, Jan 28, 2016 at 1:12 AM, Erik Weathers < eweath...@groupon.com.invalid> wrote: > hey Sam, > > Again, I've never used the DRPC feature. But it naively looks like you > aren't using it correctly. > Note that the working case for you is with a LocalCluster & LocalDRPC, and > you are explicitly invoking the execute(): > String result = drpc.execute(TOPOLOGY_NAME, tweetJson); > > In the remote case you aren't doing anything except submitting the > topology. I assume you need to invoke drpc.execute *somewhere*... Here's > what I think is the same question as you are posing: > > - http://stackoverflow.com/a/26440260/318428 > > - Erik > > On Wed, Jan 27, 2016 at 3:04 PM, researcher cs <prog.researc...@gmail.com> > wrote: > > > also got zeros after open storm ui except executed column > > > > > > > > > > On Thu, Jan 28, 2016 at 12:43 AM, researcher cs < > prog.researc...@gmail.com > > > wrote: > > > >> i atteached what i got when i submitted topology and this happened also > >> when i submitted storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar > >> storm.starter.BasicDRPCTopology basic-drpc > >> > >> this storm.yaml > >> storm.zookeeper.servers: > >> - "localhost" > >> # - "server2" > >> nimbus.host: "localhost" > >> > >> storm.local.dir: "/var/storm" > >> supervisor.slots.ports: > >> - 7660 > >> - 7659 > >> - 7658 > >> - 7657 > >> supervisor.childopts: "-Djava.net.preferIPv4Stack=true" > >> nimbus.childopts: "-Djava.net.preferIPv4Stack=true" > >> worker.childopts: "-Djava.net.preferIPv4Stack=true" > >> # topology.message.timeout.secs: 30 > >> # topology.workers: 1 > >> topology.stats.sample.rate: 1.0 > >> topology.acker.executors: 1 > >> # topology.executor.receive.buffer.size: 16384 > >> # topology.executor.send.buffer.size: 16384 > >> # topology.transfer.buffer.size: 32 > >> # topology.receiver.buffer.size: 8 > >> # > >> # ##### These may optionally be filled in: > >> # > >> ## List of custom serializations > >> # topology.kryo.register: > >> # - org.mycompany.MyType > >> # - org.mycompany.MyType2: org.mycompany.MyType2Serializer > >> # > >> ## List of custom kryo decorators > >> # topology.kryo.decorators: > >> # - org.mycompany.MyDecorator > >> # > >> ## Locations of the drpc servers > >> # drpc.servers: > >> # - "localhost" > >> # - "server2" > >> > >> ## Metrics Consumers > >> # topology.metrics.consumer.register: > >> # - class: "backtype.storm.metrics.LoggingMetricsConsumer" > >> # parallelism.hint: 1 > >> # - class: "org.mycompany.MyMetricsConsumer" > >> # parallelism.hint: 1 > >> # argument: > >> # - endpoint: "metrics-collector.mycompany.org" > >> storm.messaging.transport: "backtype.storm.messaging.netty.Context" > >> storm.messaging.netty.server_worker_threads: 1 > >> storm.messaging.netty.client_worker_threads: 1 > >> storm.messaging.netty.buffer_size: 5242880 > >> storm.messaging.netty.max_retries: 100 > >> storm.messaging.netty.max_wait_ms: 1000 > >> storm.messaging.netty.min_wait_ms: 100 > >> > >> this etc/hosts > >> 127.0.0.1 localhost > >> 127.0.1.1 user-Lenovo-G50-70 > >> ##The following lines are desirable for IPv6 capable hosts## > >> ::1 ip6-localhost ip6-loopback > >> fe00::0 ip6-localnet > >> ff00::0 ip6-mcastprefix > >> ff02::1 ip6-allnodes > >> ff02::2 ip6-allrouters > >> > >> i set drpc in code > >> private static Config createTopologyConfiguration(Properties prop, > >> boolean localMode) { > >> Config conf = new Config(); > >> List<String> dprcServers = new ArrayList<String>(); > >> dprcServers.add("localhost"); > >> > >> conf.put(Config.DRPC_SERVERS, dprcServers); > >> conf.put(Config.DRPC_PORT, 3772); > >> if (!localMode) > >> conf.put(Config.STORM_CLUSTER_MODE, new > >> String("distributed")); > >> > >> } > >> public static void main(String[] args) throws Exception { > >> if (args == null || args.length == 0) { > >> Config conf = createTopologyConfiguration(prop, true); > >> LocalDRPC drpc = new LocalDRPC(); > >> LocalCluster cluster = new LocalCluster(); > >> > >> cluster.submitTopology(TOPOLOGY_NAME, conf, > >> buildTopology(drpc)); > >> > >> Thread.sleep(2000); // give it some time to setup > >> > >> BufferedReader br = new BufferedReader(new > >> FileReader(queryFile)); > >> String tweetJson; > >> fos.write("Start: ".getBytes()); > >> > >> fos.write(String.valueOf(System.currentTimeMillis()).getBytes()); > >> byte[] newLine = "\n".getBytes(); > >> int times = 0; > >> // emit tweets into topology > >> while ((tweetJson = br.readLine()) != null) { > >> > >> String result = drpc.execute(TOPOLOGY_NAME, tweetJson); > >> > >> Status s = null; > >> try { > >> s = DataObjectFactory.createStatus(tweetJson); > >> result = s.getId() + "\t" + s.getText() + "\t" + > >> result; > >> } catch (TwitterException e) { > >> LOG.error(e.toString()); > >> } > >> > >> fos.write(result.getBytes()); > >> fos.write(newLine); > >> > >> // times++; > >> // if (times == 1000) > >> // break; > >> } > >> fos.write(newLine); > >> fos.write("Finish: ".getBytes()); > >> > >> fos.write(String.valueOf(System.currentTimeMillis()).getBytes()); > >> > >> fos.flush(); > >> fos.close(); > >> br.close(); > >> drpc.shutdown(); > >> cluster.shutdown(); > >> } else { > >> // distributed mode > >> Config conf = createTopologyConfiguration(prop,true); > >> LocalDRPC drpc = null; > >> StormSubmitter.submitTopology(args[0], conf, > >> buildTopology(drpc)); > >> > >> } > >> > >> } > >> > >> On Wed, Jan 27, 2016 at 11:14 PM, Erik Weathers < > >> eweath...@groupon.com.invalid> wrote: > >> > >>> Please put more effort into describing the issue. "It doesn't work" is > >>> unfortunately not enough info for anyone to provide help. > >>> e.g., post links to some code you are trying to run, and the configs of > >>> the > >>> storm components that you are running. > >>> > >>> - Erik > >>> > >>> On Wed, Jan 27, 2016 at 4:59 AM, sam mohel <sammoh...@gmail.com> > wrote: > >>> > >>> > I wrote the actual problem in My first message drpc server not > working > >>> I > >>> > hot zeros in the columns of storm ui like emitted and transferred , > >>> result > >>> > file is empty > >>> > > >>> > On Wednesday, January 27, 2016, Erik Weathers > >>> > <eweath...@groupon.com.invalid> > >>> > wrote: > >>> > > >>> > > Your mail client is wrapping the log lines prematurely, I have a > >>> really > >>> > > really hard time reading wrapped lines, I'd look into fixing that > if > >>> I > >>> > were > >>> > > you. Here they are unwrapped: > >>> > > > >>> > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting > >>> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client > >>> connection, > >>> > > connectString=localhost:2181 sessionTimeout=20000 > >>> > > watcher=com.netflix.curator.ConnectionState@2fa423d2 > >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket > >>> connection to > >>> > > server localhost/127.0.1.1:2181. Will not attempt to authenticate > >>> using > >>> > > SASL (unknown error) > >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection > >>> established > >>> > > to localhost/127.0.1.1:2181, initiating session > >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment > >>> > complete > >>> > > on server localhost/127.0.1.1:2181, sessionid = 0x152804f3a3a0002, > >>> > > negotiated timeout = 20000 > >>> > > 2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update: > >>> > > :connected:none > >>> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session: > 0x152804f3a3a0002 > >>> > > closed > >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down > >>> > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting > >>> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client > >>> connection, > >>> > > connectString=localhost:2181/storm sessionTimeout=20000 > >>> > > watcher=com.netflix.curator.ConnectionState@2ee8b0bf > >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket > >>> connection to > >>> > > server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to > >>> > authenticate > >>> > > using SASL (unknown error) > >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection > >>> established > >>> > > to user-Lenovo-G50-70/127.0.0.1:2181, initiating session > >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment > >>> > complete > >>> > > on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid = > >>> > > 0x152804f3a3a0003, > >>> > > negotiated timeout = 20000 > >>> > > > >>> > > None of those indicate a problem, they look pretty standard to me. > >>> > > > >>> > > Please spend a bit more time zeroing in on what the actual problem > >>> is so > >>> > > that the members of the list(s) can provide help. > >>> > > > >>> > > - Erik > >>> > > > >>> > > On Tue, Jan 26, 2016 at 10:15 PM, researcher cs < > >>> > prog.researc...@gmail.com > >>> > > <javascript:;>> > >>> > > wrote: > >>> > > > >>> > > > yes i tried in local and worked well > >>> > > > and about /etc/hosts . i'm feeling that this file has a mistake > , i > >>> > made > >>> > > > alot of changes in this file and didn't remember what was default > >>> > > > > >>> > > > In nimbus log file when it connected zookeeper i got > >>> > > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] > Starting > >>> > > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client > >>> > connection, > >>> > > > connectString=localhost:2181 sessionTimeout=20000 > >>> > > > watcher=com.netflix.curator.ConnectionState@2fa423d2 > >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket > >>> connection > >>> > to > >>> > > > server localhost/127.0.1.1:2181. Will not attempt to > authenticate > >>> > using > >>> > > > SASL (unknown error) > >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection > >>> > established > >>> > > > to localhost/127.0.1.1:2181, initiating session > >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment > >>> > > complete > >>> > > > on server localhost/127.0.1.1:2181, sessionid = > 0x152804f3a3a0002, > >>> > > > negotiated timeout = 20000 > >>> > > > 2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update: > >>> > > > :connected:none > >>> > > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session: > >>> 0x152804f3a3a0002 > >>> > > > closed > >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down > >>> > > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] > Starting > >>> > > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client > >>> > connection, > >>> > > > connectString=localhost:2181/storm sessionTimeout=20000 > >>> > > > watcher=com.netflix.curator.ConnectionState@2ee8b0bf > >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket > >>> connection > >>> > to > >>> > > > server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to > >>> > > authenticate > >>> > > > using SASL (unknown error) > >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection > >>> > established > >>> > > > to user-Lenovo-G50-70/127.0.0.1:2181, initiating session > >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment > >>> > > complete > >>> > > > on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid = > >>> > > > 0x152804f3a3a0003, > >>> > > > negotiated timeout = 20000 > >>> > > > > >>> > > > and i set for drpc.server : localhost > >>> > > > storm.zookeeper.server : localhost > >>> > > > nimbus.host : localhost > >>> > > > > >>> > > > in my /etc/hosts > >>> > > > 127.0.0.1 user-Lenovo-G50-70 localhost > >>> > > > 127.0.1.1 localhost > >>> > > > > >>> > > > is that right ? > >>> > > > > >>> > > > > >>> > > > On Wed, Jan 27, 2016 at 6:12 AM, Erik Weathers < > >>> > > > eweath...@groupon.com.invalid> wrote: > >>> > > > > >>> > > > > You said: "except the statement of drpc server trying to > connect" > >>> > > > > > >>> > > > > Maybe you are confused about what "b.s.d.drpc [INFO] Starting > >>> > > Distributed > >>> > > > > RPC servers..." implies? > >>> > > > > That is just saying that the server is being started. It's a > >>> > server, > >>> > > > not > >>> > > > > a client, so it's basic operation is *not* to connect to some > >>> other > >>> > > > thing. > >>> > > > > It's up and waiting for you to tell it to do stuff. > >>> > > > > > >>> > > > > Have you gotten the Local-mode version of DRPC working? > >>> > > > > > >>> > > > > - > >>> > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > > >>> > https://storm.apache.org/documentation/Distributed-RPC.html#local-mode-drpc > >>> > > > > > >>> > > > > Not sure what you're asking with regards to DNS and /etc/hosts, > >>> those > >>> > > > seem > >>> > > > > unrelated to your basic issue. > >>> > > > > > >>> > > > > - Erik > >>> > > > > > >>> > > > > On Tue, Jan 26, 2016 at 6:58 PM, researcher cs < > >>> > > > prog.researc...@gmail.com <javascript:;>> > >>> > > > > wrote: > >>> > > > > > >>> > > > > > There is no error except the statement of drpc server trying > to > >>> > > connect > >>> > > > > . I > >>> > > > > > guess I have problem with Dns . if you have any idea about > this > >>> > > pleaes > >>> > > > > help > >>> > > > > > > >>> > > > > > > >>> > > > > > I want to submit topology with single machine > >>> > > > > > Only on my laptop without any other devices as a first step > so > >>> > > > > > What this file /etc/hosts should contains ? > >>> > > > > > As I set in drpc.server : localhost > >>> > > > > > Storm.zookeeper. server: localhost > >>> > > > > > Nimbus.host:localhost > >>> > > > > > > >>> > > > > > As this file contains 127.0.1.1 and 127.0.0.1 and my IP > >>> address ? > >>> > > > > > > >>> > > > > > What should I use and what should I hash it to not using it ? > >>> > > > > > > >>> > > > > > > >>> > > > > > > >>> > > > > > On Wednesday, January 27, 2016, Erik Weathers > >>> > > > > > <eweath...@groupon.com.invalid> > >>> > > > > > wrote: > >>> > > > > > > >>> > > > > > > What does the client code that is supposed to make the DRPC > >>> > > > connection > >>> > > > > > > telling you? i.e., you should see some exception or log > >>> about > >>> > not > >>> > > > > > > establishing the connection, right? > >>> > > > > > > > >>> > > > > > > Alternatively, perhaps the connections aren't persistent > and > >>> > > there's > >>> > > > no > >>> > > > > > > actual problem? > >>> > > > > > > > >>> > > > > > > - Erik > >>> > > > > > > > >>> > > > > > > On Tue, Jan 26, 2016 at 4:55 PM, researcher cs < > >>> > > > > > prog.researc...@gmail.com <javascript:;> > >>> > > > > > > <javascript:;>> > >>> > > > > > > wrote: > >>> > > > > > > > >>> > > > > > > > thanks for replying , i read the documentation before , > i > >>> > > imported > >>> > > > > > > project > >>> > > > > > > > supposed to work well but not working with me > >>> > > > > > > > i checked port by lsof -i gave me all ports i connected > it > >>> for > >>> > > > storm > >>> > > > > > > > java 10675 root 20u IPv4 98126 0t0 TCP > >>> *:52022 > >>> > > > > > (LISTEN) > >>> > > > > > > > java 10675 root 26u IPv4 98131 0t0 TCP > >>> *:2181 > >>> > > > > (LISTEN) > >>> > > > > > > > java 10675 root 27u IPv4 101944 0t0 TCP > >>> > > > > > > > localhost:2181->user-Lenovo-G50-70:38150 (ESTABLISHED) > >>> > > > > > > > java 10675 root 29u IPv4 98974 0t0 TCP > >>> > > > > > > > user-Lenovo-G50-70:2181->user-Lenovo-G50-70:50526 > >>> (ESTABLISHED) > >>> > > > > > > > java 10675 root 30u IPv4 99105 0t0 TCP > >>> > > > > > > > localhost:2181->user-Lenovo-G50-70:38165 (ESTABLISHED) > >>> > > > > > > > java 10715 root 90u IPv4 98953 0t0 TCP > >>> > > > > > > > user-Lenovo-G50-70:38150->localhost:2181 (ESTABLISHED) > >>> > > > > > > > java 10715 root 91u IPv4 98245 0t0 TCP > >>> *:6627 > >>> > > > > (LISTEN) > >>> > > > > > > > java 10792 root 90u IPv4 99973 0t0 TCP > >>> > > > > > > > user-Lenovo-G50-70:50526->user-Lenovo-G50-70:2181 > >>> (ESTABLISHED) > >>> > > > > > > > java 10864 root 82u IPv4 102425 0t0 TCP > >>> *:3772 > >>> > > > > (LISTEN) > >>> > > > > > > > java 10864 root 84u IPv4 102429 0t0 TCP > >>> *:3773 > >>> > > > > (LISTEN) > >>> > > > > > > > java 10864 root 92u IPv4 102197 0t0 TCP > >>> > > > > > > > user-Lenovo-G50-70:3773->user-Lenovo-G50-70:50825 > >>> (ESTABLISHED) > >>> > > > > > > > java 10928 root 81u IPv4 102070 0t0 TCP > >>> > > *:http-alt > >>> > > > > > > (LISTEN) > >>> > > > > > > > java 11087 root 81u IPv4 100091 0t0 TCP > >>> > > > > > > > user-Lenovo-G50-70:50825->user-Lenovo-G50-70:3773 > >>> (ESTABLISHED) > >>> > > > > > > > java 11087 root 91u IPv4 102196 0t0 TCP > >>> > > > > > > > user-Lenovo-G50-70:38165->localhost:2181 (ESTABLISHED) > >>> > > > > > > > java 11087 root 94u IPv4 102561 0t0 TCP > >>> *:7660 > >>> > > > > (LISTEN) > >>> > > > > > > > > >>> > > > > > > > > >>> > > > > > > > here you can see that 3772 not established > >>> > > > > > > > > >>> > > > > > > > On Wed, Jan 27, 2016 at 2:47 AM, Erik Weathers < > >>> > > > > > > > eweath...@groupon.com.invalid> wrote: > >>> > > > > > > > > >>> > > > > > > > > hey, > >>> > > > > > > > > > >>> > > > > > > > > The DRPC server is up and listening on port 3772. Why > >>> do > >>> > you > >>> > > > > expect > >>> > > > > > > > > established connections? > >>> > > > > > > > > > >>> > > > > > > > > I'm not familiar with using Storm's DRPC feature, but > I'm > >>> > sure > >>> > > > you > >>> > > > > > need > >>> > > > > > > > to > >>> > > > > > > > > write code that interacts with the DRPC server, and > >>> you've > >>> > made > >>> > > > no > >>> > > > > > > > mention > >>> > > > > > > > > of doing so in your email. I'd start here: > >>> > > > > > > > > > >>> > > > > > > > > - > >>> > > https://storm.apache.org/documentation/Distributed-RPC.html > >>> > > > > > > > > > >>> > > > > > > > > - Erik > >>> > > > > > > > > > >>> > > > > > > > > On Tue, Jan 26, 2016 at 4:29 PM, researcher cs < > >>> > > > > > > > prog.researc...@gmail.com <javascript:;> <javascript:;>> > >>> > > > > > > > > wrote: > >>> > > > > > > > > > >>> > > > > > > > > > I set in the code > >>> > > > > > > > > > conf.put(Config.DRPC_SERVERS, dprcServers); > >>> > > > > > > > > > conf.put(Config.DRPC_PORT, 3772); > >>> > > > > > > > > > but when i submit topolgoy i found at the end of the > >>> file > >>> > > > > > > > > > b.s.d.drpc [INFO] Starting Distributed RPC > servers... > >>> > > > > > > > > > > >>> > > > > > > > > > i checked port 3772 by > >>> > > > > > > > > > > >>> > > > > > > > > > sudo netstat -ap | grep 3772 > >>> > > > > > > > > > > >>> > > > > > > > > > i got > >>> > > > > > > > > > > >>> > > > > > > > > > tcp 0 0 *:3772 *:* LISTEN > >>> 10864/java > >>> > > > > > > > > > unix 3 [ ] STREAM CONNECTED > 13772 > >>> > > > > > > > > > 587/dbus-daemon /var/run/dbus/system_bus_socket > >>> > > > > > > > > > > >>> > > > > > > > > > > >>> > > > > > > > > > why it's not established ? > >>> > > > > > > > > > > >>> > > > > > > > > > can i find help ? > >>> > > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > >>> > > > > > > > >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > > >>> > >> > >> > > >
