stankiewicz commented on code in PR #38230:
URL: https://github.com/apache/beam/pull/38230#discussion_r3126082820
##########
runners/core-java/src/main/java/org/apache/beam/runners/core/ReduceFnRunner.java:
##########
@@ -590,6 +595,17 @@ private void processElement(Map<W, W> windowToMergeResult,
WindowedValue<InputT>
value.getTimestamp(),
StateStyle.DIRECT,
value.causedByDrain());
+
+ ValueState<CombinedMetadata> metadataState =
directContext.state().access(METADATA_TAG);
+ CombinedMetadata currentMetadata = metadataState.read();
+ if (currentMetadata == null) {
+ currentMetadata = CombinedMetadata.createDefault();
+ }
+ CombinedMetadata inputMetadata =
CombinedMetadata.create(value.causedByDrain());
+ CombinedMetadata newMetadata =
+ CombinedMetadataCombiner.of().addInput(currentMetadata,
inputMetadata);
+ metadataState.write(newMetadata);
Review Comment:
this is interesting observation, @kennknowles I'm curious about your
opinion.
for every windowed value, we accumulate values in bagstate.
for metadata I've decided to implement value state, which requires read,
mutate, write per element.
maybe I should rewrite it into CombiningState , would it improve
performance?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
