[
https://issues.apache.org/jira/browse/BEAM-11104?focusedWorklogId=757042&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-757042
]
ASF GitHub Bot logged work on BEAM-11104:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 14/Apr/22 15:56
Start Date: 14/Apr/22 15:56
Worklog Time Spent: 10m
Work Description: lostluck commented on code in PR #17334:
URL: https://github.com/apache/beam/pull/17334#discussion_r850585916
##########
sdks/go/pkg/beam/core/runtime/exec/sdf.go:
##########
@@ -343,7 +348,12 @@ func (n *ProcessSizedElementsAndRestrictions)
ProcessElement(_ context.Context,
defer func() {
<-n.SU
}()
- return n.PDo.processSingleWindow(mainIn)
+ continuation, processResult := n.PDo.processSingleWindow(mainIn)
+ if continuation != nil {
+ n.source.pc = continuation
+ n.source.selfSu = n
Review Comment:
FYI I do like leveraging the existing "ownership" mechanism using the
channel as the changed approach seems to be taking.
As long as we clearly consider who is writing, and reading when it's fine.
We want to avoid heavy mutexes on a per element basis if we can manage it (that
can get very expensive very quickly).
Issue Time Tracking
-------------------
Worklog Id: (was: 757042)
Time Spent: 10h 40m (was: 10.5h)
> [Go SDK] DoFn Self Checkpointing
> --------------------------------
>
> Key: BEAM-11104
> URL: https://issues.apache.org/jira/browse/BEAM-11104
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-go
> Reporter: Robert Burke
> Assignee: Jack McCluskey
> Priority: P3
> Time Spent: 10h 40m
> Remaining Estimate: 0h
>
> Allow SplittableDoFns to self checkpoint.
> Design doc:
> https://docs.google.com/document/d/1_JbzjY9JR07ZK5v7PcZevUfzHPsqwzfV7W6AouNpMPk/edit?usp=sharing
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
