[ https://issues.apache.org/jira/browse/BEAM-4552?focusedWorklogId=232713&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-232713 ]
ASF GitHub Bot logged work on BEAM-4552: ---------------------------------------- Author: ASF GitHub Bot Created on: 25/Apr/19 10:27 Start Date: 25/Apr/19 10:27 Worklog Time Spent: 10m Work Description: aromanenko-dev commented on issue #8387: [DO NOT MERGE][BEAM-4552] Use Spark AccumulatorsV2 API URL: https://github.com/apache/beam/pull/8387#issuecomment-486617009 I performed 3 times in a row every Nexmark run against master and my feature branch with `AccumulatorV2` and I see that for some tests we have always better results with new accumulator API. The results of testing: ``` Master branch Performance: Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline) 0000 1.8 56980.1 100000 0001 0.6 158982.5 92000 0002 0.4 237529.7 351 0003 3.7 26824.0 580 0004 1.5 6738.5 40 0005 4.4 22888.5 12 0006 1.4 7102.3 103 0007 4.1 24230.7 1 0008 4.1 24539.9 6000 0009 1.4 7256.9 298 0010 2.6 37792.9 1 0011 2.4 40933.3 1919 0012 2.4 41841.0 1919 0013 0.8 120772.9 92000 0014 6.4 31060.7 184000 ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 232713) Time Spent: 1h 10m (was: 1h) > Spark runner should use AccumulatorsV2 > -------------------------------------- > > Key: BEAM-4552 > URL: https://issues.apache.org/jira/browse/BEAM-4552 > Project: Beam > Issue Type: Sub-task > Components: runner-spark > Reporter: Ismaël Mejía > Assignee: Alexey Romanenko > Priority: Minor > Time Spent: 1h 10m > Remaining Estimate: 0h > > Spark runner currently uses the old (and deprecated) API for Accumulators for > Beam metrics. We should upgrade this now that we are fully based on Spark 2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)