No that should suffice. Can you check whether there are any task manager logs for the second TM on that machine (taskmanager-X-j-011.log where X is the TM number)? If yes, the task manager process does start up and there is another problem. If not, the task managers seems not to start even.
– Ufuk On Thu, Jul 7, 2016 at 7:34 AM, Saliya Ekanayake <esal...@gmail.com> wrote: > I tried to run more than one task manager per node by duplicating the slave > IPs. At startup it says for example, > > [INFO] 1 instance(s) of taskmanager are already running on j-011. > Starting taskmanager daemon on host j-011. > > but I only see 1 task manager process running. > > Is there anything else I need to do? > > On Sun, Jul 3, 2016 at 11:28 AM, Ufuk Celebi <u...@apache.org> wrote: >> >> Yes, exactly. >> >> On Sat, Jul 2, 2016 at 6:28 PM, Saliya Ekanayake <esal...@gmail.com> >> wrote: >> > Thank you, yes, it can be done externally, if not supported within >> > Flink. >> > >> > So the way to spawn multiple task managers would be to list the same >> > slave >> > machines N times as necessary in the slaves file? >> > >> > On Sat, Jul 2, 2016 at 11:22 AM, Ufuk Celebi <u...@apache.org> wrote: >> >> >> >> No, not inside of Flink. That sounds like something like the OS or >> >> resource manager should handle. >> >> >> >> On Sat, Jul 2, 2016 at 5:12 PM, Saliya Ekanayake <esal...@gmail.com> >> >> wrote: >> >> > That's great, so is there support to pin task managers to sockets as >> >> > well? >> >> > >> >> > On Sat, Jul 2, 2016 at 11:08 AM, Ufuk Celebi <u...@apache.org> wrote: >> >> >> >> >> >> Regarding 2) if you don't manually configure something else, that >> >> >> should happen always. >> >> >> >> >> >> Yes, you can run more than one task manager per node depending on >> >> >> the >> >> >> process isolation you want. Within a task manager, there are >> >> >> multiple >> >> >> threads for each slot. For example, if you have 2 task managers with >> >> >> 2 >> >> >> slots each and submit a job with parallelism 4, each task manager >> >> >> will >> >> >> execute 2 sub tasks in separate Threads. >> >> >> >> >> >> >> >> >> On Sat, Jul 2, 2016 at 3:26 AM, Saliya Ekanayake <esal...@gmail.com> >> >> >> wrote: >> >> >> > Hi Ufuk, >> >> >> > >> >> >> > Looking at the document you sent it seems only 1 task manager per >> >> >> > node >> >> >> > exist >> >> >> > and within that you have multiple slots. Is it possible to run >> >> >> > more >> >> >> > than >> >> >> > 1 >> >> >> > task manager per node? Also, within a task manager is the >> >> >> > parallelism >> >> >> > done >> >> >> > through threads or processes? >> >> >> > >> >> >> > Thank you, >> >> >> > Saliya >> >> >> > >> >> >> > On Thu, Jun 30, 2016 at 5:27 PM, Saliya Ekanayake >> >> >> > <esal...@gmail.com> >> >> >> > wrote: >> >> >> >> >> >> >> >> Thank you, I'll check these. >> >> >> >> >> >> >> >> In 2.) you said they are likely to exchange through memory. Is >> >> >> >> there >> >> >> >> a >> >> >> >> case why they wouldn't? >> >> >> >> >> >> >> >> On Thu, Jun 30, 2016 at 5:03 AM, Ufuk Celebi <u...@apache.org> >> >> >> >> wrote: >> >> >> >>> >> >> >> >>> On Thu, Jun 30, 2016 at 1:44 AM, Saliya Ekanayake >> >> >> >>> <esal...@gmail.com> >> >> >> >>> wrote: >> >> >> >>> > 1. What parameters are available to control parallelism within >> >> >> >>> > a >> >> >> >>> > node? >> >> >> >>> >> >> >> >>> Task Manager processing slots: >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> https://ci.apache.org/projects/flink/flink-docs-release-1.0/setup/config.html#configuring-taskmanager-processing-slots >> >> >> >>> >> >> >> >>> > 2. Does Flink support shared memory-based messaging within a >> >> >> >>> > node >> >> >> >>> > (without >> >> >> >>> > doing TCP calls)? >> >> >> >>> >> >> >> >>> Yes, local exchanges happen via memory and not TCP, for example >> >> >> >>> if >> >> >> >>> you >> >> >> >>> have a map-reduce, map subtask 1 and reduce subtask 1 are likely >> >> >> >>> to >> >> >> >>> exchange data locally. >> >> >> >>> >> >> >> >>> > 3. Is there support for Infiniband interconnect? >> >> >> >>> >> >> >> >>> No, not that I'm aware of. >> >> >> >>> >> >> >> >>> – Ufuk >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> >> Saliya Ekanayake >> >> >> >> Ph.D. Candidate | Research Assistant >> >> >> >> School of Informatics and Computing | Digital Science Center >> >> >> >> Indiana University, Bloomington >> >> >> >> >> >> >> > >> >> >> > >> >> >> > >> >> >> > -- >> >> >> > Saliya Ekanayake >> >> >> > Ph.D. Candidate | Research Assistant >> >> >> > School of Informatics and Computing | Digital Science Center >> >> >> > Indiana University, Bloomington >> >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > -- >> >> > Saliya Ekanayake >> >> > Ph.D. Candidate | Research Assistant >> >> > School of Informatics and Computing | Digital Science Center >> >> > Indiana University, Bloomington >> >> > >> > >> > >> > >> > >> > -- >> > Saliya Ekanayake >> > Ph.D. Candidate | Research Assistant >> > School of Informatics and Computing | Digital Science Center >> > Indiana University, Bloomington >> > > > > > > -- > Saliya Ekanayake > Ph.D. Candidate | Research Assistant > School of Informatics and Computing | Digital Science Center > Indiana University, Bloomington >
