y1chi commented on pull request #11916: URL: https://github.com/apache/beam/pull/11916#issuecomment-647623292
> > @mxm Seems like flink runner is sending cache token per state, instead of one user state token per bundle. This will cause python sdk to fail > > https://github.com/apache/beam/blob/d5c3391119061581b13d8b647b5c1f423a39d4b2/sdks/python/apache_beam/runners/worker/sdk_worker.py#L893 > > > > is this a known issue and being worked on? > > The Flink Runner only sends one cache token per worker and application attempt. Please see > > https://github.com/apache/beam/blob/f3d5fc4b9fdbfdf943bb4d57d16a4dcb022923dc/runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/ExecutableStageDoFnOperator.java#L400 it looks like the fn-execution core module is trying to add a cache token for each state key? https://github.com/apache/beam/blob/2fd785dedb979a248e63c6385f978fd18fd2fbc4/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/state/StateRequestHandlers.java#L272 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
