System properties when submitting flink job to YARN Session

[Jins George]

Hello,

I want to set the path of a properties file as System property in my 
application(something like -Dkey=value).
Is there a way to set it while submitting a flink job to running YARN 
Session? I am using //bin/flink run/ to submit the job to a already 
running YARN session.


Thanks,
Jins George

Re: Flink Jobs disappers

[Chesnay Schepler]

If a TaskManager ran out of memory there should be something in the 
JobManager logs about a unreachable TaskManager.
That said, there should also be something in the JobManager logs about 
the job disappearing...


Could you set the logging level to DEBUG, run the job again, and provide 
us (or me directly) with the logs?


Regards,
Chesnay

On 08.07.2017 08:44, G.S.Vijay Raajaa wrote:

HI Chesnay,


I am currently using Flink - 1.3 using docker containers. I am not 
using it in HA mode. I have 3 task managers and one job manager. This 
happens randomly and not every time. Does it mean the task manager ran 
out of memory etc? I am using slots more than the available core , I 
hope compute is shared in round robin. Any pointers to tuning and HA 
setup will be greatly appreciated.


Regards,
Vijay Raajaa GS

On Sat, Jul 8, 2017 at 12:04 PM, Chesnay Schepler > wrote:


Hello,

could you tell us a bit more about your setup? Which Flink version
you're using, whether HA is enabled, does this happen every time
etc. .
Regards,
Chesnay


On 06.07.2017 21:43, G.S.Vijay Raajaa wrote:

HI,

I am using Flink Task manager and Job Manager as docker
containers. Strangely, I find the jobs to disappear from the
web portal after some time. The jobs don't move to the failed
state either. Any pointers will be really helpful. Not able to
get a clue from the logs.

Kindly let me know if I need specific tuning and ways to
persists the uploaded jars.

Regards,
Vijay Raajaa G S

Re: Flink Jobs disappers

[G.S.Vijay Raajaa]

HI Chesnay,


I am currently using Flink - 1.3 using docker containers. I am not using it
in HA mode. I have 3 task managers and one job manager. This happens
randomly and not every time. Does it mean the task manager ran out of
memory etc? I am using slots more than the available core , I hope compute
is shared in round robin. Any pointers to tuning and HA setup will be
greatly appreciated.

Regards,
Vijay Raajaa GS

On Sat, Jul 8, 2017 at 12:04 PM, Chesnay Schepler 
wrote:

> Hello,
>
> could you tell us a bit more about your setup? Which Flink version you're
> using, whether HA is enabled, does this happen every time etc. .
> Regards,
> Chesnay
>
>
> On 06.07.2017 21:43, G.S.Vijay Raajaa wrote:
>
>> HI,
>>
>> I am using Flink Task manager and Job Manager as docker containers.
>> Strangely, I find the jobs to disappear from the web portal after some
>> time. The jobs don't move to the failed state either. Any pointers will be
>> really helpful. Not able to get a clue from the logs.
>>
>> Kindly let me know if I need specific tuning and ways to persists the
>> uploaded jars.
>>
>> Regards,
>> Vijay Raajaa G S
>>
>
>
>

Re: Flink Jobs disappers

[Chesnay Schepler]

Hello,

could you tell us a bit more about your setup? Which Flink version 
you're using, whether HA is enabled, does this happen every time etc. .

Regards,
Chesnay

On 06.07.2017 21:43, G.S.Vijay Raajaa wrote:

HI,

I am using Flink Task manager and Job Manager as docker containers. 
Strangely, I find the jobs to disappear from the web portal after some 
time. The jobs don't move to the failed state either. Any pointers 
will be really helpful. Not able to get a clue from the logs.


Kindly let me know if I need specific tuning and ways to persists the 
uploaded jars.


Regards,
Vijay Raajaa G S