-06-17 11:28:20,561 INFO
org.apache.shaded.flink.configuration.GlobalConfiguration - Loading
configuration property: taskmanager.heap.size, 1024m
2020-06-17 11:28:20,561 INFO
org.apache.shaded.flink.configuration.GlobalConfiguration - Loading
configuration property: taskmanager.numberOfTaskSlots, 1
2020-06-17 11:28:20,561 INFO
org.apache.shaded.flink.configuration.GlobalConfiguration - Loading
configuration property: parallelism.default, 1
2020-06-17 11:28:20,561 INFO
org.apache.shaded.flink.configuration.GlobalConfiguration - Loading
configuration property: jobmanager.execution.failover-strategy, region
2020-06-17 11:28:20,562 INFO
org.apache.shaded.flink.configuration.GlobalConfiguration - Loading
configuration property: jobmanager.rpc.address, cluster-flink-poc-m
2020-06-17 11:28:20,562 INFO
org.apache.shaded.flink.configuration.GlobalConfiguration - Loading
configuration property: jobmanager.heap.mb, 12288
2020-06-17 11:28:20,562 INFO
org.apache.shaded.flink.configuration.GlobalConfiguration - Loading
configuration property: taskmanager.heap.mb, 12288
2020-06-17 11:28:20,562 INFO
org.apache.shaded.flink.configuration.GlobalConfiguration - Loading
configuration property: taskmanager.numberOfTaskSlots, 4
2020-06-17 11:28:20,562 INFO
org.apache.shaded.flink.configuration.GlobalConfiguration - Loading
configuration property: parallelism.default, 28
2020-06-17 11:28:20,563 INFO
org.apache.shaded.flink.configuration.GlobalConfiguration - Loading
configuration property: taskmanager.network.numberOfBuffers, 2048
2020-06-17 11:28:20,563 INFO
org.apache.shaded.flink.configuration.GlobalConfiguration - Loading
configuration property: fs.hdfs.hadoopconf, /etc/hadoop/conf
2020-06-17 11:28:20,563 INFO
org.apache.shaded.flink.runtime.minicluster.MiniCluster - Starting
Metrics Registry
2020-06-17 11:28:20,610 INFO
org.apache.shaded.flink.runtime.metrics.MetricRegistryImpl- No metrics
reporter configured, no metrics will be exposed/reported.
2020-06-17 11:28:20,610 INFO
org.apache.shaded.flink.runtime.minicluster.MiniCluster - Starting
RPC Service(s)
2020-06-17 11:28:20,976 INFO akka.event.slf4j.Slf4jLogger
- Slf4jLogger started
2020-06-17 11:28:21,070 INFO
org.apache.shaded.flink.runtime.rpc.akka.AkkaRpcServiceUtils - Trying to
start actor system at :0
2020-06-17 11:28:21,115 INFO akka.event.slf4j.Slf4jLogger
- Slf4jLogger started
2020-06-17 11:28:21,131 INFO akka.remote.Remoting
- Starting remoting
2020-06-17 11:28:21,279 INFO akka.remote.Remoting
- Remoting started; listening on addresses
:[akka.tcp://flink-metrics@<>]
2020-06-17 11:28:21,283 INFO
org.apache.shaded.flink.runtime.rpc.akka.AkkaRpcServiceUtils - Actor
system started at akka.tcp://flink-metrics@<>
Note : I have removed a few IP addresses from the log.
On Thu, Jun 18, 2020 at 12:08 AM Till Rohrmann wrote:
> Hi Sourabh,
>
> do you have access to the cluster logs? They could be helpful for
> debugging the problem. Which version of Flink are you using?
>
> Cheers,
> Till
>
> On Wed, Jun 17, 2020 at 7:39 PM Sourabh Mehta
> wrote:
>
>> No, I am not.
>>
>> On Wed, 17 Jun 2020 at 10:48 PM, Chesnay Schepler
>> wrote:
>>
>>> Are you by any chance creating a local environment via
>>> (Stream)ExecutionEnvironment#createLocalEnvironment?
>>>
>>> On 17/06/2020 17:05, Sourabh Mehta wrote:
>>>
>>> Hi Team,
>>>
>>> I'm exploring flink for one of my use case, I'm facing some issues
>>> while running a flink job in cluster mode. Below are the steps I followed
>>> to setup and run job in cluster mode :
>>> 1. Setup flink on google cloud dataproc using
>>> https://github.com/GoogleCloudDataproc/initialization-actions/tree/master/flink
>>>
>>> 2. After setting up the cluster I could see the flink session started
>>> and could see the UI for the same.
>>>
>>> 3 Submitted job from dataproc master node using below command
>>>
>>> sudo HADOOP_CONF_DIR=/etc/hadoop/conf /usr/lib/flink/bin/flink run -m
>>> yarn-cluster -yid application_1592311654771_0001 -class
>>> com.sm.flink.FlinkDriver /usr/lib/flink/lib/flink-1.0.10-sm-SNAPSHOT.jar
>>> hdfs://cluster-flink-poc-m:8020/user/flink/rocksdb/
>>>
>>> After running the job I see the job started successfully but created a
>>> mini local cluster and ran in local mode. I don't see any jobs submitted to
>>> JobManger and I also see 0 task managers on UI.
>>>
>>> Can someone please help me understand here?, do let me know what input
>>> is required to investigate the same.
>>>
>>>
>>>
>>>
>>>
