lostluck commented on code in PR #35413:
URL: https://github.com/apache/beam/pull/35413#discussion_r2190641873
##########
sdks/go/pkg/beam/core/metrics/sampler.go:
##########
@@ -21,28 +21,35 @@ import (
"time"
"unsafe"
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/internal/errors"
"github.com/apache/beam/sdks/v2/go/pkg/beam/log"
)
+// RestartLullTimeout is the minimum timeout for restarting the sdk harness if
job option element_processing_timeout is set.
+var (
+ restartLullMinTimeoutDefault time.Duration = 10 * time.Minute
+)
+
// StateSampler tracks the state of a bundle.
type StateSampler struct {
store *Store // used to store states into state
registry
millisSinceLastTransition time.Duration
transitionsAtLastSample int64
nextLogTime time.Duration
logInterval time.Duration
+ restartLullTimeout time.Duration
}
// NewSampler creates a new state sampler.
-func NewSampler(store *Store) StateSampler {
- return StateSampler{store: store, nextLogTime: 5 * time.Minute,
logInterval: 5 * time.Minute}
+func NewSampler(store *Store, elementProcessingTimeout string) StateSampler {
Review Comment:
I recommend adding to the comment saying what the expected format the string
is going to be, along with the behavior" "elementProcessingTimeout must be
duration formatted string or `-1`, with a minimum interval of foo. Otherwise
...." etc.
Alternatively:
Lift the flag parsing out of the NewSampler logic, and into the caller. This
makes the sampler behavior easier to test, and avoids the awkward comment. The
minimum threshold and such move to the sampler part of the harness package
instead. Further, you could pass in a context from there to avoid the
context.TODO.
##########
sdks/go/pkg/beam/core/metrics/sampler.go:
##########
@@ -117,3 +128,17 @@ func getState(s bundleProcState) string {
return ""
}
}
+
+func getRestartLullTimeout(elementProcessingTimeout string) time.Duration {
Review Comment:
eg. This just moves to harness/sampler.go essentially, and can accept a
context.Context.
Also, it could be tested independently from the main Sampler logic, since
the main sampler just needs to validate that the time thresholds are being
reached, instead of also accounting for the default.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@beam.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
