Ngone51 commented on pull request #32136: URL: https://github.com/apache/spark/pull/32136#issuecomment-857475125
> Isn't it the mapping still executor id <-> statestore? Executor id could be changed due to executor lost. More robust mapping, e.g. for our use-case, might be PVC id <-> statestore. The mapping between executor id and statestore is necessarily needed. And it can be achieved by the existing framework - `ExecutorData.resourcesInfo`, so it won't be a problem. But the "mapping" between tasks and the specific resources (statestore in this case) is a new requirement that we have to add. As mentioned above, we probably even don't need the stage level scheduling API ability if we follow the `StateStoreTaskLocation` solution. And, then, the `isSupport` checking also won't be a problem. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
