Karol Brejna created GEARPUMP-149: ------------------------------------- Summary: Binding tasks to workers Key: GEARPUMP-149 URL: https://issues.apache.org/jira/browse/GEARPUMP-149 Project: Apache Gearpump Issue Type: Improvement Reporter: Karol Brejna Priority: Minor
Let's assume that we have a cluster with some "specialized" nodes (for example a worker on IoT device that is responsible for controlling some actuators, dedicated workers to achieve data locality, some hardware specific - mem/cpu - tasks, etc.). There should be a way to "hint" Gearpump to prefer some nodes (workers) over others for certain tasks (executors). We already have "localities" feature. Developer can add a configuration (when submitting the application) to bind a task with a worker: {code} gearpump { streaming { localities { app1: { //This is the application name to be submitted workerId: [ //The worker id where to launch Task0_0 and Task0_1 TaskId(0,0), TaskId(0,1) ] } } } } {code} The problem is: users have no idea what each Task's task ID is (especially if there are lots of nodes in a DAG). We could introduce more flexible "binding" mechanism, for example labeling tasks and workers or using some other "awareness" algorithm. -- This message was sent by Atlassian JIRA (v6.3.4#6332)