[
https://issues.apache.org/jira/browse/BEAM-11104?focusedWorklogId=755310&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-755310
]
ASF GitHub Bot logged work on BEAM-11104:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 11/Apr/22 16:46
Start Date: 11/Apr/22 16:46
Worklog Time Spent: 10m
Work Description: jrmccluskey commented on code in PR #17334:
URL: https://github.com/apache/beam/pull/17334#discussion_r847536784
##########
sdks/go/pkg/beam/core/runtime/exec/sdf.go:
##########
@@ -343,7 +348,12 @@ func (n *ProcessSizedElementsAndRestrictions)
ProcessElement(_ context.Context,
defer func() {
<-n.SU
}()
- return n.PDo.processSingleWindow(mainIn)
+ continuation, processResult := n.PDo.processSingleWindow(mainIn)
+ if continuation != nil {
+ n.source.pc = continuation
+ n.source.selfSu = n
Review Comment:
I think that works up at the DataSource level if we wanted to (that's how we
init the SplittableUnit.) That's cleaner with how I was thinking about doing
the actual split, too (trying to keep all of the logic within DataSource and
just returning the split to the harness if we do so.) I'll mock up the code for
that after I go through the rest of the comments
Issue Time Tracking
-------------------
Worklog Id: (was: 755310)
Time Spent: 6h 10m (was: 6h)
> [Go SDK] DoFn Self Checkpointing
> --------------------------------
>
> Key: BEAM-11104
> URL: https://issues.apache.org/jira/browse/BEAM-11104
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-go
> Reporter: Robert Burke
> Assignee: Jack McCluskey
> Priority: P3
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> Allow SplittableDoFns to self checkpoint.
> Design doc:
> https://docs.google.com/document/d/1_JbzjY9JR07ZK5v7PcZevUfzHPsqwzfV7W6AouNpMPk/edit?usp=sharing
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
