[
https://issues.apache.org/jira/browse/BEAM-7126?focusedWorklogId=249544&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-249544
]
ASF GitHub Bot logged work on BEAM-7126:
----------------------------------------
Author: ASF GitHub Bot
Created on: 28/May/19 17:58
Start Date: 28/May/19 17:58
Worklog Time Spent: 10m
Work Description: mxm commented on pull request #8697: [BEAM-7126] Fix
StateRequestHandler type variables to allow for proper key parameterization
URL: https://github.com/apache/beam/pull/8697#discussion_r288230283
##########
File path:
runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/ExecutableStageDoFnOperator.java
##########
@@ -339,7 +339,8 @@ private void prepareStateBackend(K key) {
// Key for state request is shipped already encoded as ByteString,
// this is mostly a wrapping with ByteBuffer. We still follow the
// usual key encoding procedure.
- final ByteBuffer encodedKey = FlinkKeyUtils.encodeKey(key, keyCoder);
Review comment:
I don't think the overhead is measurable. It's essentially a Noop.
> Perhaps we could also assign the key coder in the operator instead of
passing it through the translation?
Is this part of the model, that we always expect ByteString for the key
coder? If not, I'd leave it flexible because this is strictly a job-compile
time micro optimization.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 249544)
Time Spent: 3h 40m (was: 3.5h)
> Double encoding of state keys in portable Flink runner
> ------------------------------------------------------
>
> Key: BEAM-7126
> URL: https://issues.apache.org/jira/browse/BEAM-7126
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Affects Versions: 2.11.0
> Reporter: Thomas Weise
> Assignee: Maximilian Michels
> Priority: Major
> Labels: portability-flink
> Fix For: 2.13.0
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> State keys currently need to be encoded as NESTED. My attempt to use the
> ByteString directly in BEAM-7112 caused checkpointing to fail. We should look
> into eliminating the redundant key encoding and adjusting
> StateRequestHandlers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
