[jira] [Updated] (BEAM-4750) Beam performance degraded significantly since 2.4
[ https://issues.apache.org/jira/browse/BEAM-4750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Beam JIRA Bot updated BEAM-4750: Labels: (was: stale-assigned) > Beam performance degraded significantly since 2.4 > - > > Key: BEAM-4750 > URL: https://issues.apache.org/jira/browse/BEAM-4750 > Project: Beam > Issue Type: Bug > Components: runner-core >Affects Versions: 2.4.0, 2.5.0, 2.6.0, 2.7.0, 2.8.0, 2.9.0 >Reporter: Vojtech Janota >Priority: P1 > > Starting from Beam 2.4 onwards the *InMemoryStateInternals* class in the > *beam-runners-core-java* module does an expensive Coder encode/decode > operation when copying object state. This has significant impact on > performance and pipelines that previously took low minutes do not finish > within hours in our case. Based on the discussion on the dev mailing list, > the main motivation for this change was to enforce Coder sanity, something > that should arguably remain within the realm of the DirectRunner and should > not leak into the core layer. > Links to commits that introduced the new behaviour: > * [https://github.com/apache/beam/commit/32a427c] > * [https://github.com/apache/beam/commit/8151d82] > > Additional details and surrounding discussion can be found here: > * > https://lists.apache.org/thread.html/1e329318a4dafe27b9ff304d9460d05d8966e5ceebaf4ebfb948e2b8@%3Cdev.beam.apache.org%3E > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (BEAM-4750) Beam performance degraded significantly since 2.4
[ https://issues.apache.org/jira/browse/BEAM-4750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-4750: -- Labels: stale-assigned (was: ) > Beam performance degraded significantly since 2.4 > - > > Key: BEAM-4750 > URL: https://issues.apache.org/jira/browse/BEAM-4750 > Project: Beam > Issue Type: Bug > Components: runner-core >Affects Versions: 2.4.0, 2.5.0, 2.6.0, 2.7.0, 2.8.0, 2.9.0 >Reporter: Vojtech Janota >Assignee: Kenneth Knowles >Priority: P1 > Labels: stale-assigned > > Starting from Beam 2.4 onwards the *InMemoryStateInternals* class in the > *beam-runners-core-java* module does an expensive Coder encode/decode > operation when copying object state. This has significant impact on > performance and pipelines that previously took low minutes do not finish > within hours in our case. Based on the discussion on the dev mailing list, > the main motivation for this change was to enforce Coder sanity, something > that should arguably remain within the realm of the DirectRunner and should > not leak into the core layer. > Links to commits that introduced the new behaviour: > * [https://github.com/apache/beam/commit/32a427c] > * [https://github.com/apache/beam/commit/8151d82] > > Additional details and surrounding discussion can be found here: > * > https://lists.apache.org/thread.html/1e329318a4dafe27b9ff304d9460d05d8966e5ceebaf4ebfb948e2b8@%3Cdev.beam.apache.org%3E > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (BEAM-4750) Beam performance degraded significantly since 2.4
[ https://issues.apache.org/jira/browse/BEAM-4750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-4750: -- Affects Version/s: 2.6.0 2.7.0 2.8.0 2.9.0 > Beam performance degraded significantly since 2.4 > - > > Key: BEAM-4750 > URL: https://issues.apache.org/jira/browse/BEAM-4750 > Project: Beam > Issue Type: Bug > Components: runner-core >Affects Versions: 2.4.0, 2.5.0, 2.6.0, 2.7.0, 2.8.0, 2.9.0 >Reporter: Vojtech Janota >Assignee: Jean-Baptiste Onofré >Priority: Critical > Fix For: 2.10.0 > > > Starting from Beam 2.4 onwards the *InMemoryStateInternals* class in the > *beam-runners-core-java* module does an expensive Coder encode/decode > operation when copying object state. This has significant impact on > performance and pipelines that previously took low minutes do not finish > within hours in our case. Based on the discussion on the dev mailing list, > the main motivation for this change was to enforce Coder sanity, something > that should arguably remain within the realm of the DirectRunner and should > not leak into the core layer. > Links to commits that introduced the new behaviour: > * [https://github.com/apache/beam/commit/32a427c] > * [https://github.com/apache/beam/commit/8151d82] > > Additional details and surrounding discussion can be found here: > * > https://lists.apache.org/thread.html/1e329318a4dafe27b9ff304d9460d05d8966e5ceebaf4ebfb948e2b8@%3Cdev.beam.apache.org%3E > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-4750) Beam performance degraded significantly since 2.4
[ https://issues.apache.org/jira/browse/BEAM-4750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-4750: -- Fix Version/s: (was: 2.10.0) > Beam performance degraded significantly since 2.4 > - > > Key: BEAM-4750 > URL: https://issues.apache.org/jira/browse/BEAM-4750 > Project: Beam > Issue Type: Bug > Components: runner-core >Affects Versions: 2.4.0, 2.5.0, 2.6.0, 2.7.0, 2.8.0, 2.9.0 >Reporter: Vojtech Janota >Assignee: Jean-Baptiste Onofré >Priority: Critical > > Starting from Beam 2.4 onwards the *InMemoryStateInternals* class in the > *beam-runners-core-java* module does an expensive Coder encode/decode > operation when copying object state. This has significant impact on > performance and pipelines that previously took low minutes do not finish > within hours in our case. Based on the discussion on the dev mailing list, > the main motivation for this change was to enforce Coder sanity, something > that should arguably remain within the realm of the DirectRunner and should > not leak into the core layer. > Links to commits that introduced the new behaviour: > * [https://github.com/apache/beam/commit/32a427c] > * [https://github.com/apache/beam/commit/8151d82] > > Additional details and surrounding discussion can be found here: > * > https://lists.apache.org/thread.html/1e329318a4dafe27b9ff304d9460d05d8966e5ceebaf4ebfb948e2b8@%3Cdev.beam.apache.org%3E > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-4750) Beam performance degraded significantly since 2.4
[ https://issues.apache.org/jira/browse/BEAM-4750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-4750: -- Priority: Critical (was: Major) > Beam performance degraded significantly since 2.4 > - > > Key: BEAM-4750 > URL: https://issues.apache.org/jira/browse/BEAM-4750 > Project: Beam > Issue Type: Bug > Components: runner-core >Affects Versions: 2.4.0, 2.5.0 >Reporter: Vojtech Janota >Assignee: Jean-Baptiste Onofré >Priority: Critical > Fix For: 2.10.0 > > > Starting from Beam 2.4 onwards the *InMemoryStateInternals* class in the > *beam-runners-core-java* module does an expensive Coder encode/decode > operation when copying object state. This has significant impact on > performance and pipelines that previously took low minutes do not finish > within hours in our case. Based on the discussion on the dev mailing list, > the main motivation for this change was to enforce Coder sanity, something > that should arguably remain within the realm of the DirectRunner and should > not leak into the core layer. > Links to commits that introduced the new behaviour: > * [https://github.com/apache/beam/commit/32a427c] > * [https://github.com/apache/beam/commit/8151d82] > > Additional details and surrounding discussion can be found here: > * > https://lists.apache.org/thread.html/1e329318a4dafe27b9ff304d9460d05d8966e5ceebaf4ebfb948e2b8@%3Cdev.beam.apache.org%3E > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-4750) Beam performance degraded significantly since 2.4
[ https://issues.apache.org/jira/browse/BEAM-4750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-4750: - Fix Version/s: (was: 2.9.0) 2.10.0 > Beam performance degraded significantly since 2.4 > - > > Key: BEAM-4750 > URL: https://issues.apache.org/jira/browse/BEAM-4750 > Project: Beam > Issue Type: Bug > Components: runner-core >Affects Versions: 2.4.0, 2.5.0 >Reporter: Vojtech Janota >Assignee: Jean-Baptiste Onofré >Priority: Major > Fix For: 2.10.0 > > > Starting from Beam 2.4 onwards the *InMemoryStateInternals* class in the > *beam-runners-core-java* module does an expensive Coder encode/decode > operation when copying object state. This has significant impact on > performance and pipelines that previously took low minutes do not finish > within hours in our case. Based on the discussion on the dev mailing list, > the main motivation for this change was to enforce Coder sanity, something > that should arguably remain within the realm of the DirectRunner and should > not leak into the core layer. > Links to commits that introduced the new behaviour: > * [https://github.com/apache/beam/commit/32a427c] > * [https://github.com/apache/beam/commit/8151d82] > > Additional details and surrounding discussion can be found here: > * > https://lists.apache.org/thread.html/1e329318a4dafe27b9ff304d9460d05d8966e5ceebaf4ebfb948e2b8@%3Cdev.beam.apache.org%3E > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
