Re: Flink 'Job Cluster' mode Ui Access
Hi, Recently i upgraded flink version to 1.8.3 For Session cluster it shows the version correctly. But for job cluster. I get this in the logs *Starting StandaloneJobClusterEntryPoint (Version: , Rev:6322618, Date:04.09.2019 @ 22:07:41 CST)* And my Classpath has these jars: *Classpath: /opt/flink/lib/flink-metrics-prometheus-1.8.3.jar:/opt/flink/lib/flink-shaded-hadoop-2-uber-2.6.5-7.0.jar:/opt/flink/lib/job.jar:/opt/flink/lib/log4j-over-slf4j-1.7.28.jar:/opt/flink/lib/logback-classic-1.2.3.jar:/opt/flink/lib/logback-core-1.2.3.jar:/opt/flink/lib/flink-dist_2.11-1.8.3.jar::/opt/hdfs:* Do you have any idea what could have caused this? Best Regards, Jatin On Fri, Dec 13, 2019 at 6:18 PM Chesnay Schepler wrote: > Thank you for the logs. > > Flink can indeed find the WebUI files in the distribution, which is a bit > odd. > Since there are no static files serve in this case, the > StaticFileServerHandler is never set up in the first place (hence why we > didn't find any log statements). > > What I also found in the logs (and, looking back, in one of your earlier > replies) was this: Version: , Rev:ceba8af, Date:11.02.2019 @ > 22:17:09 CST > > This tells us a few things. > a) You are not using 1.8.1, but 1.7.2 (based on the revision) > b) You are not using an official release, since the build-date differs > from the official releases > > I tried one of the official 1.7.2 releases, and the WebUI is shown both > when using: > (after copying the wordcount example into /lib) > ./bin/standalone-job.sh start-foreground -j > org.apache.flink.examples.java.wordcount.WordCount > ./bin/standalone-job.sh start -j > org.apache.flink.examples.java.wordcount.WordCount > > Right now I don't know what else to look for; there are some discrepancies > as to what your environment is vs what you described, and as such I can > only recommend to carefully evaluate what you have actually running and > possibly try again with an official release. > > Regards, > Chesnay > > On 13/12/2019 09:58, Jatin Banger wrote: > > Sure, here it is. > Job Manager Logs with logging level as DEBUG > > On Wed, Dec 11, 2019 at 3:14 PM Chesnay Schepler > wrote: > >> Would it be possible for you to provide us with full debug log file? >> >> On 10/12/2019 18:07, Jatin Banger wrote: >> >> Yes, I did. >> >> On Tue, Dec 10, 2019 at 3:47 PM Arvid Heise wrote: >> >>> Hi Jatin, >>> >>> just to be sure. Did you increase the log level to debug [1] before >>> checking for *StaticFileServerHandler*? >>> >>> Best, >>> >>> Arvid >>> >>> [1] >>> https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/logging.html#configuring-log4j >>> >>> On Mon, Dec 9, 2019 at 7:54 AM Jatin Banger >>> wrote: >>> >>>> Hi, >>>> >>>> I have checked the logs with this keyword *StaticFileServerHandler *in >>>> it, But there were no logs coming for "Flink Job Cluster". >>>> Then i checked for Flink Session Cluster, i was able to find the logs >>>> for the *StaticFileServerHandler *keyword. >>>> >>>> Can i raise this as bug ? >>>> >>>> Best Regards, >>>> Jatin >>>> >>>> >>>> On Thu, Dec 5, 2019 at 8:59 PM Chesnay Schepler >>>> wrote: >>>> >>>>> Ok, it's good to know that the WebUI files are there. >>>>> >>>>> Please enable DEBUG logging and try again, searching for messages from >>>>> the StaticFileServerHandler. >>>>> >>>>> This handler logs every file that is requested (which effectively >>>>> happens when the WebUI is being served); let's see what is actually being >>>>> requested. >>>>> >>>>> On 05/12/2019 05:57, Jatin Banger wrote: >>>>> >>>>> I have tried that already using >>>>> '$FLINK_HOME/bin/jobmanager.sh" start-foreground >>>>> Ui comes fine with this one. >>>>> Which means web/index.html is present. >>>>> >>>>> >>>>> On Wed, Dec 4, 2019 at 9:01 PM Chesnay Schepler >>>>> wrote: >>>>> >>>>>> hmm...this is quite odd. >>>>>> >>>>>> Let's try to narrow things down a bit. >>>>>> >>>>>> Could you try starting a local cluster (using the same distribution) >>>>>> and checking whether the UI is accessible? >>>>>> >>>>&
Re: Flink 'Job Cluster' mode Ui Access
Yes, I did.
On Tue, Dec 10, 2019 at 3:47 PM Arvid Heise wrote:
> Hi Jatin,
>
> just to be sure. Did you increase the log level to debug [1] before
> checking for *StaticFileServerHandler*?
>
> Best,
>
> Arvid
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/logging.html#configuring-log4j
>
> On Mon, Dec 9, 2019 at 7:54 AM Jatin Banger
> wrote:
>
>> Hi,
>>
>> I have checked the logs with this keyword *StaticFileServerHandler *in
>> it, But there were no logs coming for "Flink Job Cluster".
>> Then i checked for Flink Session Cluster, i was able to find the logs for
>> the *StaticFileServerHandler *keyword.
>>
>> Can i raise this as bug ?
>>
>> Best Regards,
>> Jatin
>>
>>
>> On Thu, Dec 5, 2019 at 8:59 PM Chesnay Schepler
>> wrote:
>>
>>> Ok, it's good to know that the WebUI files are there.
>>>
>>> Please enable DEBUG logging and try again, searching for messages from
>>> the StaticFileServerHandler.
>>>
>>> This handler logs every file that is requested (which effectively
>>> happens when the WebUI is being served); let's see what is actually being
>>> requested.
>>>
>>> On 05/12/2019 05:57, Jatin Banger wrote:
>>>
>>> I have tried that already using
>>> '$FLINK_HOME/bin/jobmanager.sh" start-foreground
>>> Ui comes fine with this one.
>>> Which means web/index.html is present.
>>>
>>>
>>> On Wed, Dec 4, 2019 at 9:01 PM Chesnay Schepler
>>> wrote:
>>>
>>>> hmm...this is quite odd.
>>>>
>>>> Let's try to narrow things down a bit.
>>>>
>>>> Could you try starting a local cluster (using the same distribution)
>>>> and checking whether the UI is accessible?
>>>>
>>>> Could you also check whether the flink-dist.jar in /lib contains
>>>> web/index.html?
>>>> On 04/12/2019 06:02, Jatin Banger wrote:
>>>>
>>>> Hi,
>>>>
>>>> I am using flink binary directly.
>>>>
>>>> I am using this command to deploy the script.
>>>>
>>>> "$FLINK_HOME/bin/standalone-job.sh"
>>>> start-foreground --job-classname ${ARGS_FOR_JOB}
>>>> where ARGS_FOR_JOB contain job class name and all other necessary
>>>> details needed by the job.
>>>>
>>>> Best regards,
>>>> Jatin
>>>>
>>>>
>>>> On Fri, Nov 29, 2019 at 4:18 PM Chesnay Schepler
>>>> wrote:
>>>>
>>>>> To clarify, you ran "mvn package -pl flink-dist -am" to build Fink?
>>>>>
>>>>> If so, could you run that again and provide us with the maven output?
>>>>>
>>>>> On 29/11/2019 11:23, Jatin Banger wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> @vino yang I am using flink 1.8.1
>>>>>
>>>>> I am using the following procedure for the deployment:
>>>>>
>>>>> https://github.com/apache/flink/blob/master/flink-container/docker/README.md
>>>>>
>>>>> And i tried accessing the path you mentioned:
>>>>>
>>>>> # curl :4081/#/overview
>>>>> {"errors":["Not found."]}
>>>>>
>>>>> Best Regards,
>>>>> Jatin
>>>>>
>>>>> On Thu, Nov 28, 2019 at 10:21 PM Chesnay Schepler
>>>>> wrote:
>>>>>
>>>>>> Could you try accessing :/#/overview ?
>>>>>>
>>>>>> The REST API is obviously accessible, and hence the WebUI should be
>>>>>> too.
>>>>>>
>>>>>> How did you setup the session cluster? Are you using some custom
>>>>>> Flink build or something, which potentially excluded flink-runtime-web
>>>>>> from
>>>>>> the classpath?
>>>>>>
>>>>>> On 28/11/2019 10:02, Jatin Banger wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I checked the log file there is no error.
>>>>>> And I checked the pods internal ports by using rest api.
>>>>>>
>>>>>> # curl : 4081
>>>>>> {"errors":["Not found."]}
>>>>>> 4081 is the Ui port
>>>>>>
>>>>>> # curl :4081/config
>>>>>> {"refresh-interval":3000,"timezone-name":"Coordinated Universal
>>>>>> Time","timezone-offset":0,"flink-version":"","flink-revision":"ceba8af
>>>>>> @ 11.02.2019 @ 22:17:09 CST"}
>>>>>>
>>>>>> # curl :4081/jobs
>>>>>> {"jobs":[{"id":"___job_Id_","status":"RUNNING"}]}
>>>>>>
>>>>>> Which shows the state of the job as running.
>>>>>>
>>>>>> What else can we do ?
>>>>>>
>>>>>> Best regards,
>>>>>> Jatin
>>>>>>
>>>>>> On Thu, Nov 28, 2019 at 1:28 PM vino yang
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Jatin,
>>>>>>>
>>>>>>> Flink web UI does not depend on any deployment mode.
>>>>>>>
>>>>>>> You should check if there are error logs in the log file and the job
>>>>>>> status is running state.
>>>>>>>
>>>>>>> Best,
>>>>>>> Vino
>>>>>>>
>>>>>>> Jatin Banger 于2019年11月28日周四 下午3:43写道：
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> It seems there is Web Ui for Flink Session cluster, But for Flink
>>>>>>>> Job Cluster it is Showing
>>>>>>>>
>>>>>>>> {"errors":["Not found."]}
>>>>>>>>
>>>>>>>> Is it the expected behavior for Flink Job Cluster Mode ?
>>>>>>>>
>>>>>>>> Best Regards,
>>>>>>>> Jatin
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
Re: Flink 'Job Cluster' mode Ui Access
I have tried that already using
'$FLINK_HOME/bin/jobmanager.sh" start-foreground
Ui comes fine with this one.
Which means web/index.html is present.
On Wed, Dec 4, 2019 at 9:01 PM Chesnay Schepler wrote:
> hmm...this is quite odd.
>
> Let's try to narrow things down a bit.
>
> Could you try starting a local cluster (using the same distribution) and
> checking whether the UI is accessible?
>
> Could you also check whether the flink-dist.jar in /lib contains
> web/index.html?
> On 04/12/2019 06:02, Jatin Banger wrote:
>
> Hi,
>
> I am using flink binary directly.
>
> I am using this command to deploy the script.
>
> "$FLINK_HOME/bin/standalone-job.sh"
> start-foreground --job-classname ${ARGS_FOR_JOB}
> where ARGS_FOR_JOB contain job class name and all other necessary details
> needed by the job.
>
> Best regards,
> Jatin
>
>
> On Fri, Nov 29, 2019 at 4:18 PM Chesnay Schepler
> wrote:
>
>> To clarify, you ran "mvn package -pl flink-dist -am" to build Fink?
>>
>> If so, could you run that again and provide us with the maven output?
>>
>> On 29/11/2019 11:23, Jatin Banger wrote:
>>
>> Hi,
>>
>> @vino yang I am using flink 1.8.1
>>
>> I am using the following procedure for the deployment:
>>
>> https://github.com/apache/flink/blob/master/flink-container/docker/README.md
>>
>> And i tried accessing the path you mentioned:
>>
>> # curl :4081/#/overview
>> {"errors":["Not found."]}
>>
>> Best Regards,
>> Jatin
>>
>> On Thu, Nov 28, 2019 at 10:21 PM Chesnay Schepler
>> wrote:
>>
>>> Could you try accessing :/#/overview ?
>>>
>>> The REST API is obviously accessible, and hence the WebUI should be too.
>>>
>>> How did you setup the session cluster? Are you using some custom Flink
>>> build or something, which potentially excluded flink-runtime-web from the
>>> classpath?
>>>
>>> On 28/11/2019 10:02, Jatin Banger wrote:
>>>
>>> Hi,
>>>
>>> I checked the log file there is no error.
>>> And I checked the pods internal ports by using rest api.
>>>
>>> # curl : 4081
>>> {"errors":["Not found."]}
>>> 4081 is the Ui port
>>>
>>> # curl :4081/config
>>> {"refresh-interval":3000,"timezone-name":"Coordinated Universal
>>> Time","timezone-offset":0,"flink-version":"","flink-revision":"ceba8af
>>> @ 11.02.2019 @ 22:17:09 CST"}
>>>
>>> # curl :4081/jobs
>>> {"jobs":[{"id":"___job_Id_","status":"RUNNING"}]}
>>>
>>> Which shows the state of the job as running.
>>>
>>> What else can we do ?
>>>
>>> Best regards,
>>> Jatin
>>>
>>> On Thu, Nov 28, 2019 at 1:28 PM vino yang wrote:
>>>
>>>> Hi Jatin,
>>>>
>>>> Flink web UI does not depend on any deployment mode.
>>>>
>>>> You should check if there are error logs in the log file and the job
>>>> status is running state.
>>>>
>>>> Best,
>>>> Vino
>>>>
>>>> Jatin Banger 于2019年11月28日周四 下午3:43写道：
>>>>
>>>>> Hi,
>>>>>
>>>>> It seems there is Web Ui for Flink Session cluster, But for Flink Job
>>>>> Cluster it is Showing
>>>>>
>>>>> {"errors":["Not found."]}
>>>>>
>>>>> Is it the expected behavior for Flink Job Cluster Mode ?
>>>>>
>>>>> Best Regards,
>>>>> Jatin
>>>>>
>>>>
>>>
>>
>
Re: Flink 'Job Cluster' mode Ui Access
Hi,
I am using flink binary directly.
I am using this command to deploy the script.
"$FLINK_HOME/bin/standalone-job.sh"
start-foreground --job-classname ${ARGS_FOR_JOB}
where ARGS_FOR_JOB contain job class name and all other necessary details
needed by the job.
Best regards,
Jatin
On Fri, Nov 29, 2019 at 4:18 PM Chesnay Schepler wrote:
> To clarify, you ran "mvn package -pl flink-dist -am" to build Fink?
>
> If so, could you run that again and provide us with the maven output?
>
> On 29/11/2019 11:23, Jatin Banger wrote:
>
> Hi,
>
> @vino yang I am using flink 1.8.1
>
> I am using the following procedure for the deployment:
>
> https://github.com/apache/flink/blob/master/flink-container/docker/README.md
>
> And i tried accessing the path you mentioned:
>
> # curl :4081/#/overview
> {"errors":["Not found."]}
>
> Best Regards,
> Jatin
>
> On Thu, Nov 28, 2019 at 10:21 PM Chesnay Schepler
> wrote:
>
>> Could you try accessing :/#/overview ?
>>
>> The REST API is obviously accessible, and hence the WebUI should be too.
>>
>> How did you setup the session cluster? Are you using some custom Flink
>> build or something, which potentially excluded flink-runtime-web from the
>> classpath?
>>
>> On 28/11/2019 10:02, Jatin Banger wrote:
>>
>> Hi,
>>
>> I checked the log file there is no error.
>> And I checked the pods internal ports by using rest api.
>>
>> # curl : 4081
>> {"errors":["Not found."]}
>> 4081 is the Ui port
>>
>> # curl :4081/config
>> {"refresh-interval":3000,"timezone-name":"Coordinated Universal
>> Time","timezone-offset":0,"flink-version":"","flink-revision":"ceba8af
>> @ 11.02.2019 @ 22:17:09 CST"}
>>
>> # curl :4081/jobs
>> {"jobs":[{"id":"___job_Id_","status":"RUNNING"}]}
>>
>> Which shows the state of the job as running.
>>
>> What else can we do ?
>>
>> Best regards,
>> Jatin
>>
>> On Thu, Nov 28, 2019 at 1:28 PM vino yang wrote:
>>
>>> Hi Jatin,
>>>
>>> Flink web UI does not depend on any deployment mode.
>>>
>>> You should check if there are error logs in the log file and the job
>>> status is running state.
>>>
>>> Best,
>>> Vino
>>>
>>> Jatin Banger 于2019年11月28日周四 下午3:43写道：
>>>
>>>> Hi,
>>>>
>>>> It seems there is Web Ui for Flink Session cluster, But for Flink Job
>>>> Cluster it is Showing
>>>>
>>>> {"errors":["Not found."]}
>>>>
>>>> Is it the expected behavior for Flink Job Cluster Mode ?
>>>>
>>>> Best Regards,
>>>> Jatin
>>>>
>>>
>>
>
Re: Flink 'Job Cluster' mode Ui Access
Hi,
@vino yang I am using flink 1.8.1
I am using the following procedure for the deployment:
https://github.com/apache/flink/blob/master/flink-container/docker/README.md
And i tried accessing the path you mentioned:
# curl :4081/#/overview
{"errors":["Not found."]}
Best Regards,
Jatin
On Thu, Nov 28, 2019 at 10:21 PM Chesnay Schepler
wrote:
> Could you try accessing :/#/overview ?
>
> The REST API is obviously accessible, and hence the WebUI should be too.
>
> How did you setup the session cluster? Are you using some custom Flink
> build or something, which potentially excluded flink-runtime-web from the
> classpath?
>
> On 28/11/2019 10:02, Jatin Banger wrote:
>
> Hi,
>
> I checked the log file there is no error.
> And I checked the pods internal ports by using rest api.
>
> # curl : 4081
> {"errors":["Not found."]}
> 4081 is the Ui port
>
> # curl :4081/config
> {"refresh-interval":3000,"timezone-name":"Coordinated Universal
> Time","timezone-offset":0,"flink-version":"","flink-revision":"ceba8af
> @ 11.02.2019 @ 22:17:09 CST"}
>
> # curl :4081/jobs
> {"jobs":[{"id":"___job_Id_","status":"RUNNING"}]}
>
> Which shows the state of the job as running.
>
> What else can we do ?
>
> Best regards,
> Jatin
>
> On Thu, Nov 28, 2019 at 1:28 PM vino yang wrote:
>
>> Hi Jatin,
>>
>> Flink web UI does not depend on any deployment mode.
>>
>> You should check if there are error logs in the log file and the job
>> status is running state.
>>
>> Best,
>> Vino
>>
>> Jatin Banger 于2019年11月28日周四 下午3:43写道：
>>
>>> Hi,
>>>
>>> It seems there is Web Ui for Flink Session cluster, But for Flink Job
>>> Cluster it is Showing
>>>
>>> {"errors":["Not found."]}
>>>
>>> Is it the expected behavior for Flink Job Cluster Mode ?
>>>
>>> Best Regards,
>>> Jatin
>>>
>>
>
Re: Flink 'Job Cluster' mode Ui Access
Hi,
I checked the log file there is no error.
And I checked the pods internal ports by using rest api.
# curl : 4081
{"errors":["Not found."]}
4081 is the Ui port
# curl :4081/config
{"refresh-interval":3000,"timezone-name":"Coordinated Universal
Time","timezone-offset":0,"flink-version":"","flink-revision":"ceba8af
@ 11.02.2019 @ 22:17:09 CST"}
# curl :4081/jobs
{"jobs":[{"id":"___job_Id_","status":"RUNNING"}]}
Which shows the state of the job as running.
What else can we do ?
Best regards,
Jatin
On Thu, Nov 28, 2019 at 1:28 PM vino yang wrote:
> Hi Jatin,
>
> Flink web UI does not depend on any deployment mode.
>
> You should check if there are error logs in the log file and the job
> status is running state.
>
> Best,
> Vino
>
> Jatin Banger 于2019年11月28日周四 下午3:43写道：
>
>> Hi,
>>
>> It seems there is Web Ui for Flink Session cluster, But for Flink Job
>> Cluster it is Showing
>>
>> {"errors":["Not found."]}
>>
>> Is it the expected behavior for Flink Job Cluster Mode ?
>>
>> Best Regards,
>> Jatin
>>
>
Flink 'Job Cluster' mode Ui Access
Hi,
It seems there is Web Ui for Flink Session cluster, But for Flink Job
Cluster it is Showing
{"errors":["Not found."]}
Is it the expected behavior for Flink Job Cluster Mode ?
Best Regards,
Jatin
