Hi, nc exits after the first connection is closed. Are you re-running the nc command every time the job finishes?
The stacktrace you copied does not indicate that a TaskManager cannot connect to the JobManager. I can only see that the SocketTextStreamFunction (from the SocketWindowWordCount job?) cannot open the connection to the address that you specified. Can you try to run examples/streaming/WordCount.jar. It is a simpler job which does not rely on external dependencies. If all the above fails, can you tell us how you submit the job? Can you post the full command? Can you also post the full JobManager & TaskManager logs? Best, Gary On Mon, Aug 6, 2018 at 4:10 PM, Felipe Gutierrez < felipe.o.gutier...@gmail.com> wrote: > do you mean "nc -l 9000"? If so, I did start before. > the task manager running on the master can connect to the job manager. but > the task manager on the slave node cannot. The second time that I start the > WordCount task it recognizes only one task manager (from the master) and > runs my task. But the task manager from the slave does not process anything > and it is started. > > here is the error stack trace from the slave node: > > 2017-05-30 05:10:39,853 INFO > org.apache.flink.runtime.state.heap.HeapKeyedStateBackend > - Initializing heap keyed state backend with stream factory. > 2017-05-30 05:10:39,977 INFO org.apache.flink.runtime.taskmanager.Task > - Source: Socket Stream -> Flat Map (1/1) ( > d5e3d87395995d3977d2f472de896e23) switched from RUNNING to FAILED. > java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) > at java.net.AbstractPlainSocketImpl.doConnect( > AbstractPlainSocketImpl.java:350) > at java.net.AbstractPlainSocketImpl.connectToAddress( > AbstractPlainSocketImpl.java:206) > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java: > 188) > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > at java.net.Socket.connect(Socket.java:589) > at org.apache.flink.streaming.api.functions.source. > SocketTextStreamFunction.run(SocketTextStreamFunction.java:96) > at org.apache.flink.streaming.api.operators.StreamSource. > run(StreamSource.java:87) > at org.apache.flink.streaming.api.operators.StreamSource. > run(StreamSource.java:56) > at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run( > SourceStreamTask.java:99) > at org.apache.flink.streaming.runtime.tasks.StreamTask. > invoke(StreamTask.java:306) > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703) > at java.lang.Thread.run(Thread.java:745) > 2017-05-30 05:10:40,016 INFO org.apache.flink.runtime.taskmanager.Task > - Freeing task resources for Source: Socket Stream -> > Flat Map (1/1) (d5e3d87395995d3977d2f472de896e23). > > > *--* > *-- Felipe Gutierrez* > > *-- skype: felipe.o.gutierrez* > *--* *https://felipeogutierrez.blogspot.com > <https://felipeogutierrez.blogspot.com>* > > > On Mon, Aug 6, 2018 at 2:17 PM vino yang <yanghua1...@gmail.com> wrote: > >> Hi Felipe, >> >> From the exception information, it seems that you did not start the >> socket server, the socket source needs to connect to the socket server. >> >> Please make sure the socket server has started and is available. >> >> Thanks, vino. >> >> 2018-08-06 18:45 GMT+08:00 Felipe Gutierrez <felipe.o.gutier...@gmail.com >> >: >> >>> yes. >>> >>> when I execute the jps command on the master node I >>> see TaskManagerRunner and StandaloneSessionClusterEntrypoint (which I >>> believe it is the jobManager). On the slave nodes I see TaskManagerRunner >>> when I run jps command >>> >>> >>> *--* >>> *-- Felipe Gutierrez* >>> >>> *-- skype: felipe.o.gutierrez* >>> *--* *https://felipeogutierrez.blogspot.com >>> <https://felipeogutierrez.blogspot.com>* >>> >>> >>> On Mon, Aug 6, 2018 at 12:13 PM miki haiat <miko5...@gmail.com> wrote: >>> >>>> Did you start job manager and task manager on the same resbery pi ? >>>> >>>> On Mon, 6 Aug 2018, 12:01 Felipe Gutierrez, < >>>> felipe.o.gutier...@gmail.com> wrote: >>>> >>>>> Hello everyone, >>>>> >>>>> I am trying to run Flink on Raspberry Pis. My first test for word >>>>> count in a single node worked. I just have to decrease the Heap memory of >>>>> the jobmanager.heap.mb and taskmanager.heap.mb to 512. >>>>> My second test is to add 2 slave nodes I got the error: "Java >>>>> HotSpot(TM) Client VM warning: G1 GC is disabled in this release." at the >>>>> file log/flink-root-taskexecutor-0-*.out. >>>>> >>>>> This link (https://blog.sflow.com/2016/06/raspberry-pi-real-time- >>>>> network-analytics.html) says that in order to Raspberry Pi ARM >>>>> architecture works with JVM it is necessary to configure the JVM as: >>>>> -Xms600M >>>>> -Xmx600M >>>>> -XX:+UseParNewGC >>>>> -XX:+UseConcMarkSweepGC >>>>> -XX:+CMSIncrementalMode >>>>> >>>>> then I set this variables on the path inside the file flink-conf.yaml >>>>> env.java.opts: "-XX:+UseParNewGC -XX:+UseConcMarkSweepGC >>>>> -XX:+CMSIncrementalMode" >>>>> env.java.opts.jobmanager: "-XX:+UseParNewGC -XX:+UseConcMarkSweepGC >>>>> -XX:+CMSIncrementalMode" >>>>> env.java.opts.taskmanager: "-XX:+UseParNewGC -XX:+UseConcMarkSweepGC >>>>> -XX:+CMSIncrementalMode" >>>>> >>>>> and the error "Java HotSpot(TM) Client VM warning: G1 GC is disabled >>>>> in this release." is not showing anymore. However, the connection from the >>>>> master node to the slave node is still not possible. Does anybody know how >>>>> I must configure flink to deal with that? >>>>> >>>>> This is the error stack trace: >>>>> >>>>> 2017-05-25 12:40:26,421 INFO >>>>> org.apache.flink.runtime.executiongraph.ExecutionGraph >>>>> - Source: Socket Stream -> Flat Map (1/1) ( >>>>> b81b6492fc0860367be422d0b0bf4358) switched from DEPLOYING to RUNNING. >>>>> 2017-05-25 12:40:26,891 INFO >>>>> org.apache.flink.runtime.executiongraph.ExecutionGraph >>>>> - Source: Socket Stream -> Flat Map (1/1) ( >>>>> b81b6492fc0860367be422d0b0bf4358) switched from RUNNING to FAILED. >>>>> java.net.ConnectException: Connection refused >>>>> at java.net.PlainSocketImpl.socketConnect(Native Method) >>>>> at java.net.AbstractPlainSocketImpl.doConnect( >>>>> AbstractPlainSocketImpl.java:350) >>>>> at java.net.AbstractPlainSocketImpl.connectToAddress( >>>>> AbstractPlainSocketImpl.java:206) >>>>> at java.net.AbstractPlainSocketImpl.connect( >>>>> AbstractPlainSocketImpl.java:188) >>>>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) >>>>> at java.net.Socket.connect(Socket.java:589) >>>>> at org.apache.flink.streaming.api.functions.source. >>>>> SocketTextStreamFunction.run(SocketTextStreamFunction.java:96) >>>>> at org.apache.flink.streaming.api.operators.StreamSource. >>>>> run(StreamSource.java:87) >>>>> at org.apache.flink.streaming.api.operators.StreamSource. >>>>> run(StreamSource.java:56) >>>>> at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run( >>>>> SourceStreamTask.java:99) >>>>> at org.apache.flink.streaming.runtime.tasks.StreamTask. >>>>> invoke(StreamTask.java:306) >>>>> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703) >>>>> at java.lang.Thread.run(Thread.java:745) >>>>> 2017-05-25 12:40:26,898 INFO >>>>> org.apache.flink.runtime.executiongraph.ExecutionGraph >>>>> - Job Socket Window WordCount (71c6d7796eccf6587d9d1deda0490e09) >>>>> switched from state RUNNING to FAILING. >>>>> java.net.ConnectException: Connection refused >>>>> at java.net.PlainSocketImpl.socketConnect(Native Method) >>>>> at java.net.AbstractPlainSocketImpl.doConnect( >>>>> AbstractPlainSocketImpl.java:350) >>>>> at java.net.AbstractPlainSocketImpl.connectToAddress( >>>>> AbstractPlainSocketImpl.java:206) >>>>> at java.net.AbstractPlainSocketImpl.connect( >>>>> AbstractPlainSocketImpl.java:188) >>>>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) >>>>> at java.net.Socket.connect(Socket.java:589) >>>>> at org.apache.flink.streaming.api.functions.source. >>>>> SocketTextStreamFunction.run(SocketTextStreamFunction.java:96) >>>>> at org.apache.flink.streaming.api.operators.StreamSource. >>>>> run(StreamSource.java:87) >>>>> at org.apache.flink.streaming.api.operators.StreamSource. >>>>> run(StreamSource.java:56) >>>>> at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run( >>>>> SourceStreamTask.java:99) >>>>> at org.apache.flink.streaming.runtime.tasks.StreamTask. >>>>> invoke(StreamTask.java:306) >>>>> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703) >>>>> at java.lang.Thread.run(Thread.java:745) >>>>> 2017-05-25 12:40:26,921 INFO >>>>> org.apache.flink.runtime.executiongraph.ExecutionGraph >>>>> - Window(TumblingProcessingTimeWindows(5000), >>>>> ProcessingTimeTrigger, ReduceFunction$1, PassThroughWindowFunction) -> >>>>> Sink: Print to Std. Out (1/1) (aa1a0e7ee3a1d3ad8f99b2608bd64c5b) >>>>> switched from RUNNING to CANCELING. >>>>> 2017-05-25 12:40:26,975 INFO >>>>> org.apache.flink.runtime.executiongraph.ExecutionGraph >>>>> - Window(TumblingProcessingTimeWindows(5000), >>>>> ProcessingTimeTrigger, ReduceFunction$1, PassThroughWindowFunction) -> >>>>> Sink: Print to Std. Out (1/1) (aa1a0e7ee3a1d3ad8f99b2608bd64c5b) >>>>> switched from CANCELING to CANCELED. >>>>> >>>>> >>>>> >>>>> Thanks, Felipe >>>>> *--* >>>>> *-- Felipe Gutierrez* >>>>> >>>>> *-- skype: felipe.o.gutierrez* >>>>> *--* *https://felipeogutierrez.blogspot.com >>>>> <https://felipeogutierrez.blogspot.com>* >>>>> >>>> >>
