[
https://issues.apache.org/jira/browse/BEAM-7431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16848895#comment-16848895
]
Lukasz Gajowy edited comment on BEAM-7431 at 5/27/19 12:34 PM:
---------------------------------------------------------------
Doubts that we have after an offline discussion with [~mwalenia]
- measuring bytes this way has a huge overhead
- how to measure bytes (correctly) without such overhead from objects of
generic T type?
- should we rely on ByteMonitor or maybe use a pipeline option such as
"avgRecordSize" and multiply it times records count (we already have a
"countMonitor")? Will this approach work for IO tests?
was (Author: łukaszg):
Doubts that we have after an offline discussion with [~mwalenia]
- measuring bytes this way has a huge overhead
- how to measure bytes (correctly) without such overhead from objects of
generic T type?
-should we rely on ByteMonitor or maybe use a pipeline option such as
"avgRecordSize" and multiply it times records count (we already have a
"countMonitor")? Will this approach work for IO tests?
> ObjectSizeCalculator causing regressions in load tests
> ------------------------------------------------------
>
> Key: BEAM-7431
> URL: https://issues.apache.org/jira/browse/BEAM-7431
> Project: Beam
> Issue Type: Bug
> Components: testing
> Reporter: Lukasz Gajowy
> Assignee: Lukasz Gajowy
> Priority: Critical
>
> It looks (based on my investigation) that `ObjectSizeCalculator` is causing a
> regression in load tests of core operations. the runtime increased from ~400s
> to ~2000s. Calculating object size this way seems to be an expensive
> operation - we probably could change it to something simpler (or not?).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
