I've also verified that the problem persist also using a modified version
of the WordCount class.
If you add the code pasted at the end of this email at the end of its main
method you can verify that the listener is called if you run the program
from the IDE, but it's not called if you submit the job using the CLI
client using the command
- bin/flink run
/home/okkam/git/flink/flink-examples/flink-examples-batch/target/WordCount.jar
Maybe this is an expected result but I didn't find any documentation of
this behaviour (neither in the Javadoc or in the flink web site, where I
can't find any documentation about JobListener at all).
[Code to add to main()]
// emit result
if (params.has("output")) {
counts.writeAsCsv(params.get("output"), "\n", " ");
// execute program
env.registerJobListener(new JobListener() {
@Override
public void onJobSubmitted(JobClient arg0, Throwable arg1) {
System.out.println("**************** SUBMITTED");
}
@Override
public void onJobExecuted(JobExecutionResult arg0, Throwable arg1) {
System.out.println("**************** EXECUTED");
}
});
env.execute("WordCount Example");
} else {
System.out.println("Printing result to stdout. Use --output to
specify output path.");
counts.print();
}
On Fri, Nov 13, 2020 at 4:25 PM Flavio Pompermaier <pomperma...@okkam.it>
wrote:
> see inline
>
> Il ven 13 nov 2020, 14:31 Matthias Pohl <matth...@ververica.com> ha
> scritto:
>
>> Hi Flavio,
>> thanks for sharing this with the Flink community. Could you answer the
>> following questions, please:
>> - What's the code of your Job's main method?
>>
>
> it's actually very simple...the main class creates a batch execution env
> using ExecutionEnvironment.getExecutionEnvironment(), I register a job
> listener to the env and I do some stuff before calling env.execute().
> The listener is executed correctly but if I use the RestClusterClient to
> sibmit the jobGraph exyracted from that main contained in a jar, the
> program is executed as usual but the job listener is not called.
>
> - What cluster backend and application do you use to execute the job?
>>
>
> I use a standalone session cluster for the moment
>
> - Is there anything suspicious you can find in the logs that might be
>> related?
>>
>
> no unfortunately..
>
>
>> Best,
>> Matthias
>>
>> On Thu, Nov 12, 2020 at 7:48 PM Flavio Pompermaier <pomperma...@okkam.it>
>> wrote:
>>
>>> Actually what I'm experiencing is that the JobListener is executed
>>> successfully if I run my main class from the IDE, while the job listener is
>>> not fired at all if I submit the JobGraph of the application to a cluster
>>> using the RestClusterClient..
>>> Am I doing something wrong?
>>>
>>> My main class ends with the env.execute() and i do
>>> env.registerJobListener() when I create the Exceution env
>>> via ExecutionEnvironment.getExecutionEnvironment().
>>>
>>> Thanks in advance for any help,
>>> Flavio
>>>
>>> On Thu, Nov 12, 2020 at 2:13 PM Flavio Pompermaier <pomperma...@okkam.it>
>>> wrote:
>>>
>>>> Hello everybody,
>>>> I'm trying to use the JobListener to track when a job finishes (with
>>>> Flink 1.11.0).
>>>> It works great but I have the problem that logs inside
>>>> the onJobExecuted are not logged anywhere..is it normal?
>>>>
>>>> Best,
>>>> Flavio
>>>>
>>>
