TaskManager docker image for Beam WordCount failing with ClassNotFound Exception

2020-07-07 Thread Avijit Saha
Hi, I am trying the run the Beam WordCount example on Flink runner using docker container-s for 'Jobcluster' and 'TaskManager'. When I put the Beam Wordcount custom jar in the /opt/flink/usrlib/ dir - the 'taskmanager' docker image fails at runtime with ClassNotFound Exception for the following:

Beam Word Cound on Flink/kubernetes Issue with task manager ot getting picked up

2020-07-13 Thread Avijit Saha
Hi, I have a docker image of the Beam WordCount example that reads a status file and produces a output one time with word counts etc. This runs fine as a separate job-manager and task-manager when run from docker-compose locally. Now, I am trying to deploy and run this on my Kubernetes cluster a

Re: Docker Taskmanager unable to connect to Flink JpbManager...Connection RefusedHi,

2020-07-22 Thread Avijit Saha
Hi, I have built a docker image containing both Flink 1.11 and the job jar as per instructions at: https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/docker.html The jobanager starts up fine as follows: - FLINK_PROPERTIES="jobmanager.rpc.address: 127.0.0.1" - docker run --env

Re: Docker Taskmanager unable to connect to Flink JpbManager...Connection RefusedHi,

2020-07-23 Thread Avijit Saha
.org/projects/flink/flink-docs-stable/ops/deployment/docker.html#start-a-session-cluster > > Best, > Yang > > Avijit Saha 于2020年7月23日周四 上午1:14写道: > >> Hi, >> I have built a docker image containing both Flink 1.11 and the job jar as >> per instructions at: >

Re: Between Flink 1.9 and 1.11 - any behavior change for web.upload.dir

2020-08-03 Thread Avijit Saha
Hello, Has there been any change in behavior related to the "web.upload.dir" behavior between Flink 1.9 and 1.11? I have a failure case where when build an image using "flink:1.11.0-scala_2.12" in Dockerfile, the job manager job submissions fail with the following Exception but the same flow work

Any change in behavior related to the "web.upload.dir" behavior between Flink 1.9 and 1.11

2020-08-03 Thread Avijit Saha
Hello, Has there been any change in behavior related to the "web.upload.dir" behavior between Flink 1.9 and 1.11? I have a failure case where when build an image using "flink:1.11.0-scala_2.12" in Dockerfile, the job manager job submissions fail with the following Exception but the same flow work

Re: Any change in behavior related to the "web.upload.dir" behavior between Flink 1.9 and 1.11

2020-08-03 Thread Avijit Saha
ations to the image? This exception should only > be thrown if there is already a file with the same path, and I don't think > Flink would do that. > > On 03/08/2020 21:43, Avijit Saha wrote: > > Hello, > > Has there been any change in behavior related to the "web.upl

Flink Promethues Metricsreporter Question

2020-08-05 Thread Avijit Saha
Hi, Have a general question about Flink support for Prometheus metrics. We already have a Prometheus setup in our cluster with ServiceMonitor-s monitoring ports like 8080 etc. for scraping metrics. In a setup like this, if we deploy Flink Job managers/Task managers in the cluster, is there any ne