I don't have any errors in log files except drpc.log i have this 2017-07-01T07:34:02.660+0200 b.s.d.drpc [INFO] Starting Distributed RPC servers... 2017-07-01T07:34:47.267+0200 o.a.t.s.TNonblockingServer [WARN] Got an IOException in internalRead! java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[na:1.7.0_121] at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[na:1.7.0_121] at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[na:1.7.0_121] at sun.nio.ch.IOUtil.read(IOUtil.java:197) ~[na:1.7.0_121] at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384) ~[na:1.7.0_121] at org.apache.thrift7.transport.TNonblockingSocket.read(TNonblockingSocket.java:141) ~[storm-core-0.9.6.jar:0.9.6] at org.apache.thrift7.server.TNonblockingServer$FrameBuffer.internalRead(TNonblockingServer.java:669) [storm-core-0.9.6.jar:0.9.6] at org.apache.thrift7.server.TNonblockingServer$FrameBuffer.read(TNonblockingServer.java:458) [storm-core-0.9.6.jar:0.9.6] at org.apache.thrift7.server.TNonblockingServer$SelectThread.handleRead(TNonblockingServer.java:359) [storm-core-0.9.6.jar:0.9.6] at org.apache.thrift7.server.TNonblockingServer$SelectThread.select(TNonblockingServer.java:304) [storm-core-0.9.6.jar:0.9.6] at org.apache.thrift7.server.TNonblockingServer$SelectThread.run(TNonblockingServer.java:243) [storm-core-0.9.6.jar:0.9.6] 2017-07-01T07:44:17.639+0200 b.s.d.drpc [WARN] Timeout DRPC request id: 1 start at 1498887255
and storm ui have numbers in their columns except transferred , capacity and process latency . i faced something like that before with a topology and when i increased RAM the problem fixed , i think that is the reason here also with his topology but i couldn't increase the RAM again On Sat, Jul 1, 2017 at 11:09 AM, J.R. Pauley <[email protected]> wrote: > Sam > I would guess timeout is not your problem as defaults seem generous. The > little I've learned the last couple days tell me to look at drpc.log and > topology logs for some issue. Also I learned from Bobby I can't test this > sort of thing in a LocalDRPC mode even with drpc client on same box it > won't work, has to be a regular submitted topology. Other thing I found was > drpc client version may matter. I reverted to 0.9.6 because everything was > easier to configure. In 1.0.2 the client you have to give a config param > map with lots of params that have to match the server config, and that > sounds like an easy place to have some mismatch > > On Fri, Jun 30, 2017 at 6:13 PM, sam mohel <[email protected]> wrote: > >> I searched and found this .. "The timeout on DRPC requests within the >> DRPC server. Defaults to 10 minutes. Note that requests can also timeout >> based on the socket timeout on the DRPC client, and separately based on the >> topology message timeout for the topology implementing the DRPC function." >> >> Should i change drpc.request.timeout.secs only or what ? >> Any help i will appreciate >> >> On Fri, Jun 30, 2017 at 7:16 PM, sam mohel <[email protected]> wrote: >> >>> sorry the full error i got >>> Exception in thread "main" DRPCExecutionException(msg:Request timed out) >>> at backtype.storm.generated.DistributedRPC$execute_result.read( >>> DistributedRPC.java:904) >>> at org.apache.thrift7.TServiceClient.receiveBase(TServiceClient.java:78) >>> at backtype.storm.generated.DistributedRPC$Client.recv_execute( >>> DistributedRPC.java:92) >>> at backtype.storm.generated.DistributedRPC$Client.execute(Distr >>> ibutedRPC.java:78) >>> at backtype.storm.utils.DRPCClient.execute(DRPCClient.java:71) >>> >>> >>> On Fri, Jun 30, 2017 at 3:36 PM, sam mohel <[email protected]> wrote: >>> >>>> @jim yes I ran ./storm drpc in machine A with nimbus . And got the error >>>> >>>> DRPCExecutionException(msg:Request failed) >>>> >>>> in console after sometimes from submitting the topology >>>> >>>> On Friday, June 30, 2017, Pauley, Jim <[email protected]> wrote: >>>> > Sam >>>> > Re Bobby comment that your DRPC was not even running, did you >>>> manually start the drpc from console and get it running? I know you can see >>>> if it is listening with netstat -ant | grep 3772 >>>> > ________________________________ >>>> > From: sam mohel [[email protected]] >>>> > Sent: Thursday, June 29, 2017 9:45 PM >>>> > To: [email protected] >>>> > Subject: [EXTERNAL] Re: DRPC problem >>>> > >>>> > excuse me . i tried to submit another topology and worked well with >>>> same configurations . So why this topology has a problem !! How can i >>>> figure where is the problem ? >>>> > Any help i'll really appreciate it >>>> > On Thu, Jun 29, 2017 at 8:35 PM, sam mohel <[email protected]> >>>> wrote: >>>> >> >>>> >> Thanks for replying , really i will loss my mind from this error . >>>> Can i change ip address or what should i do with Drpc server ? >>>> >> On Thu, Jun 29, 2017 at 8:29 PM, Bobby Evans <[email protected]> >>>> wrote: >>>> >>> >>>> >>> Your error message shows that your DRPC server isn't even running, >>>> so you don't need to worry about the amount of memory until you actually >>>> have the processes up and running. >>>> >>> Once you have them running then you can look at the GC metrics to >>>> see if it looks like you need to give it more heap. This is generic for >>>> java in general and has little to do with storm in particular. >>>> >>> >>>> >>> >>>> >>> - Bobby >>>> >>> >>>> >>> >>>> >>> On Thursday, June 29, 2017, 1:18:36 PM CDT, sam mohel < >>>> [email protected]> wrote: >>>> >>> >>>> >>> Thanks but how can i know if Memory is suitable for my project or >>>> not ? >>>> >>> On Thu, Jun 29, 2017 at 8:10 PM, Bobby Evans <[email protected]> >>>> wrote: >>>> >>> >>>> >>> No it has nothing to do with GC. It means that the command line >>>> confused the JVM and it thought you wanted to run a the main method in a >>>> class called _JAAS_PLACEHOLDER. >>>> >>> >>>> >>> >>>> >>> - Bobby >>>> >>> >>>> >>> >>>> >>> On Thursday, June 29, 2017, 12:45:16 PM CDT, sam mohel < >>>> [email protected]> wrote: >>>> >>> >>>> >>> Excuse me . Is that line >>>> >>> >>>> >>> Drpc.childopts"-Xmx768m _JAAS_PLACEHOLDER >>>> -Xloggc:/var/log/storm/drpc- gc.log -XX:+PrintGCDetails >>>> -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps" >>>> >>> >>>> >>> Means to check if there's a problem with garbage collector or not ? >>>> >>> And about want I got >>>> >>> >>>> >>> Could not find or load main class _JAAS_PLACEHOLDER . Is that means >>>> error I should fix and I have problem with GC ? >>>> >>> >>>> >>> >>>> >>> >>>> >>> >>>> >>> > Delete _JAAS_PLACEHOLDER. It is there as something you should >>>> replace if you want to have security for your DRPC server. >>>> >>> > >>>> >>> > >>>> >>> > - Bobby >>>> >>> > >>>> >>> > >>>> >>> > On Thursday, June 29, 2017, 10:25:24 AM CDT, sam mohel < >>>> [email protected]> wrote: >>>> >>> > >>>> >>> > Really appreciate your time Bobby . So the error I got when I >>>> added this line to drpc.childopts >>>> >>> > "-Xmx768m _JAAS_PLACEHOLDER -Xloggc:/var/log/storm/drpc- gc.log >>>> -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps" >>>> >>> > I got Error: Could not find or load main class _JAAS_PLACEHOLDER >>>> how can I fix it ? >>>> >>> > >>>> >>> > On Thursday, June 29, 2017, Bobby Evans <[email protected]> >>>> wrote: >>>> >>> >> I don't see issues with your configs on the surface. >>>> >>> >> >>>> >>> >> >>>> >>> >> - Bobby >>>> >>> >> >>>> >>> >> >>>> >>> >> On Thursday, June 29, 2017, 10:02:37 AM CDT, sam mohel < >>>> [email protected]> wrote: >>>> >>> >> >>>> >>> >> Are my configurations right ? >>>> >>> >> I got in terminal when submitted the topology after some times >>>> >>> >> drpcexecutionexception(msg: request timed out) >>>> >>> >> In log file of drpc I got >>>> >>> >> [INFO] Starting Distributed RPC servers... >>>> >>> >> [WARN] Timeout DRPC request id: 200 start at 1498657923 >>>> >>> >> In worker log file doesn't receive any error message . >>>> >>> >> But I tried to add this drpc.childopts: "-Xmx768m >>>> _JAAS_PLACEHOLDER -Xloggc:/var/log/storm/drpc- gc.log -XX:+PrintGCDetails >>>> -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps" in storm.yaml to check if >>>> there is problem in RAM and retsart submitting again . >>>> >>> >> I got in drpc terminal >>>> >>> >> Error: Could not find or load main class _JAAS_PLACEHOLDER >>>> >>> >> And in terminal when submitted >>>> >>> >> Exception in thread "main" java.lang.RuntimeException: >>>> org.apache.thrift7.transport. TTransportException: >>>> java.net.ConnectException: Connection refused (Connection refused) >>>> >>> >> at backtype.storm.utils. DRPCClient.<init>(DRPCClient. java:42) >>>> >>> >> at backtype.storm.utils. DRPCClient.<init>(DRPCClient. java:47) >>>> >>> >> at trident.FirstStoryDetection. main(FirstStoryDetection.java: >>>> 308) >>>> >>> >> Caused by: org.apache.thrift7.transport. TTransportException: >>>> java.net.ConnectException: Connection refused (Connection refused) >>>> >>> >> >>>> >>> >> >>>> >>> >> On Thursday, June 29, 2017, Bobby Evans <[email protected]> >>>> wrote: >>>> >>> >>> You are going to need to look at the logs for your topology and >>>> the logs for the drpc server to see if there is anything in there that >>>> indicates what is happening. >>>> >>> >>> >>>> >>> >>> >>>> >>> >>> - Bobby >>>> >>> >>> >>>> >>> >>> >>>> >>> >>> On Wednesday, June 28, 2017, 11:21:14 PM CDT, sam mohel < >>>> [email protected]> wrote: >>>> >>> >>> >>>> >>> >>> I submitted topology in local without any problem , but in >>>> production mode i couldn't as you can see in ui zeros values in columns >>>> except execute columns . >>>> >>> >>> i got after sometimes in terminal drpcexecutionexception(msg: >>>> request timed out) >>>> >>> >>> my configurations are >>>> >>> >>> Machine A and Machine B >>>> >>> >>> storm.yaml in Machine A is >>>> >>> >>> storm.zookeeper.servers: >>>> >>> >>> - "192.168.x.x" >>>> >>> >>> >>>> >>> >>> nimbus.host : "192.168.x.x" >>>> >>> >>> supervisor.childopts: "-Xmx4g" >>>> >>> >>> worker.childopts: "-Xmx4g" >>>> >>> >>> storm.yaml in Machine B is >>>> >>> >>> storm.zookeeper.servers: >>>> >>> >>> - "192.168.x.x" >>>> >>> >>> >>>> >>> >>> nimbus.host : "192.168.x.x" >>>> >>> >>> supervisor.childopts: "-Xmx4g" >>>> >>> >>> worker.childopts: "-Xmx4g" >>>> >>> >>> i set drpc in the code >>>> >>> >>> Config conf = new Config(); >>>> >>> >>> List<String> dprcServers = new ArrayList<String>(); >>>> >>> >>> dprcServers.add("192.168.x.x") ; >>>> >>> >>> conf.put(Config.DRPC_SERVERS, dprcServers); >>>> >>> >>> conf.put(Config.DRPC_PORT, 3772); >>>> >>> >>> // distributed mode >>>> >>> >>> Config conf = createTopologyConfiguration( prop, true); >>>> >>> >>> LocalDRPC drpc = null; >>>> >>> >>> StormSubmitter.submitTopology( args[0], conf, >>>> buildTopology(drpc)); >>>> >>> >>> client=new DRPCClient("192.168.x.x", 3772); >>>> >>> >>> i used same ip address for storm.zookeeper.servers , >>>> nimbus.host ,dprcServers and DRPCClient . Is that wrong ? >>>> >>> >>> and i ran nimbus , drpc,ui in Machine A , I ran supervisor in >>>> Machine B >>>> >>> >>> i appreciate any help , Thanks >>>> >>> >>> >>>> >>> >>> >>>> >> >>>> > >>>> > >>>> >>> >>> >> >
