TM failure when deploying a large number of sources

Hey guys, I am writing a job which involves creating many different sources to read data from (in this case 80 sources wiht the parallelism of 8 each, running locally on my mac). I cannot create less unfortunately. The problem is that the job fails while deploying the tasks with the following exc

Re: TM failure when deploying a large number of sources

Any further information from teh log? If you create so many tasks (8 x 80) on one machine, the JVM often has not enough memory reserved for the stack space to create enough threads (1-2 threads per task)... On Wed, Oct 7, 2015 at 2:13 PM, Gyula Fóra wrote: > Hey guys, > > I am writing a job whi

Re: TM failure when deploying a large number of sources

Thanks! Yes, it was indeed a memory issue: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:714) at org.apache.flink.runtime.taskmanager.Task.startTaskThread(Task.java:415) at org.apache.flink.runtime.tas

Re: TM failure when deploying a large number of sources

I think the error message could have been better, though... This actually warrants a JIRA issue... On Wed, Oct 7, 2015 at 2:44 PM, Gyula Fóra wrote: > Thanks! > > Yes, it was indeed a memory issue: > java.lang.OutOfMemoryError: unable to create new native thread > at java.lang.Thread.start0(Nat

Re: TM failure when deploying a large number of sources

Alright, I am creating one. Stephan Ewen ezt írta (időpont: 2015. okt. 7., Sze, 15:44): > I think the error message could have been better, though... > > This actually warrants a JIRA issue... > > On Wed, Oct 7, 2015 at 2:44 PM, Gyula Fóra wrote: > > > Thanks! > > > > Yes, it was indeed a memor