[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15667833#comment-15667833 ] ASF GitHub Bot commented on BEAM-891: - Github user asfgit closed the pull request at:

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15663349#comment-15663349 ] Amit Sela commented on BEAM-891: Because most tests are batch.. > Flake in Spark metrics library? >

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15663281#comment-15663281 ] Stas Levin commented on BEAM-891: - Sounds good. Btw, why can't we just set

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15661544#comment-15661544 ] Amit Sela commented on BEAM-891: It can be overriden by specifying

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15661227#comment-15661227 ] Stas Levin commented on BEAM-891: - Cool, turning it off in relevant tests sounds reasonable. we'll need to

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15661197#comment-15661197 ] Amit Sela commented on BEAM-891: I see your point. How about setting in all tests (that don't test metrics)

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15661096#comment-15661096 ] Stas Levin commented on BEAM-891: - [~amitsela], as far as the name goes, we can try to make it clearer, say

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15659356#comment-15659356 ] Amit Sela commented on BEAM-891: [~staslev] I don't think {{enableSparkSinks}} property in

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15654002#comment-15654002 ] Amit Sela commented on BEAM-891: This seems to be the race:

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653971#comment-15653971 ] Amit Sela commented on BEAM-891: My best guess for what causes the race condition is the fact that Spark

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653827#comment-15653827 ] ASF GitHub Bot commented on BEAM-891: - GitHub user staslev opened a pull request:

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15650807#comment-15650807 ] Stas Levin commented on BEAM-891: - Perhaps it would be safe to go the other way around, i.e., "only add this

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15650573#comment-15650573 ] Amit Sela commented on BEAM-891: Do we "remove source" in case of context reuse ? if so, why not bound it to

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15648828#comment-15648828 ] Amit Sela commented on BEAM-891: OK, happened again here:

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15644539#comment-15644539 ] Amit Sela commented on BEAM-891: Just to update: the mentioned failure occurred in build #43 and now we've

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15633378#comment-15633378 ] Amit Sela commented on BEAM-891: We can consider synchronizing

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15633343#comment-15633343 ] Amit Sela commented on BEAM-891: We started looking at this, and it looks like a weird race although driver

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15633366#comment-15633366 ] Amit Sela commented on BEAM-891: Well, it "can't" run in parallel because surefire is configured with {{1}}

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15633346#comment-15633346 ] Stas Levin commented on BEAM-891: - Sure. I looked at it a bit this morning, my first impression would be a