[ https://issues.apache.org/jira/browse/BEAM-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Burke reassigned BEAM-5977: ---------------------------------- Assignee: Robert Burke > Benchmark buffer use in the Go SDK Harness > ------------------------------------------ > > Key: BEAM-5977 > URL: https://issues.apache.org/jira/browse/BEAM-5977 > Project: Beam > Issue Type: Improvement > Components: sdk-go > Reporter: Robert Burke > Assignee: Robert Burke > Priority: Major > Fix For: Not applicable > > > There's opportunity to reduce CPU & RAM usage with better buffer re-use in > [datamgr.go|https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/runtime/harness/datamgr.go] > WRT both the small element batch buffer and with large elements. > There should be some benchmarking around both large elements, and smaller > elements of varying sizes, (and ideally mixing), and using that we can > measure subsequent improvements. > As a performance improvement, offhand, maintaining a pair of a couple of > `chunkSize` buffers could be handy and avoid associated GRPC costs, to handle > the smaller elements, as well as flushing large elements immediately and > without copying. -- This message was sent by Atlassian JIRA (v7.6.3#76005)