We tested that patch from aarondav's branch, and are no longer seeing that
deadlock. Seems to have solved the problem, at least for us.
On Mon, Jul 14, 2014 at 7:22 PM, Patrick Wendell pwend...@gmail.com wrote:
Andrew and Gary,
Would you guys be able to test
Hi all, just wanted to give a heads up that we're seeing a reproducible
deadlock with spark 1.0.1 with 2.3.0-mr1-cdh5.0.2
If jira is a better place for this, apologies in advance - figured talking
about it on the mailing list was friendlier than randomly (re)opening jira
tickets.
I know Gary had
Hey Cody,
This Jstack seems truncated, would you mind giving the entire stack
trace? For the second thread, for instance, we can't see where the
lock is being acquired.
- Patrick
On Mon, Jul 14, 2014 at 1:42 PM, Cody Koeninger
cody.koenin...@mediacrossing.com wrote:
Hi all, just wanted to give
The full jstack would still be useful, but our current working theory is
that this is due to the fact that Configuration#loadDefaults goes through
every Configuration object that was ever created (via
Configuration.REGISTRY) and locks it, thus introducing a dependency from
new Configuration to
Hi Aaron, I'm not sure if synchronizing on an arbitrary lock object would
help. I suspect we will start seeing the ConcurrentModificationException
again. The right fix has gone into Hadoop through 10456. Unfortunately, I
don't have any bright ideas on how to synchronize this at the Spark level
Hey Nishkam,
Aaron's fix should prevent two concurrent accesses to getJobConf (and
the Hadoop code therein). But if there is code elsewhere that tries to
mutate the configuration, then I could see how we might still have the
ConcurrentModificationException.
I looked at your patch for
We use the Hadoop configuration inside of our code executing on Spark as we
need to list out files in the path. Maybe that is why it is exposed for us.
On Mon, Jul 14, 2014 at 6:57 PM, Patrick Wendell pwend...@gmail.com wrote:
Hey Nishkam,
Aaron's fix should prevent two concurrent accesses
We'll try to run a build tomorrow AM.
On Mon, Jul 14, 2014 at 7:22 PM, Patrick Wendell pwend...@gmail.com wrote:
Andrew and Gary,
Would you guys be able to test
https://github.com/apache/spark/pull/1409/files and see if it solves
your problem?
- Patrick
On Mon, Jul 14, 2014 at 4:18 PM,
The patch won't solve the problem where two people try to add a
configuration option at the same time, but I think there is currently an
issue where two people can try to initialize the Configuration at the same
time and still run into a ConcurrentModificationException. This at least
reduces
hi，Cody
i met this issue days before and i post a PR for this(
https://github.com/apache/spark/pull/1385)
it's very strange that if i synchronize conf it will deadlock but it is ok when
synchronize initLocalJobConfFuncOpt
Here's the entire jstack output.
On Mon, Jul 14, 2014 at 4:44 PM,
I'm not sure either of those PRs will fix the concurrent adds to
Configuration issue I observed. I've got a stack trace and writeup I'll
share in an hour or two (traveling today).
On Jul 14, 2014 9:50 PM, scwf wangf...@huawei.com wrote:
hi，Cody
i met this issue days before and i post a PR for
Andrew is your issue also a regression from 1.0.0 to 1.0.1? The
immediate priority is addressing regressions between these two
releases.
On Mon, Jul 14, 2014 at 9:05 PM, Andrew Ash and...@andrewash.com wrote:
I'm not sure either of those PRs will fix the concurrent adds to
Configuration issue I
I don't believe mine is a regression. But it is related to thread safety on
Hadoop Configuration objects. Should I start a new thread?
On Jul 15, 2014 12:55 AM, Patrick Wendell pwend...@gmail.com wrote:
Andrew is your issue also a regression from 1.0.0 to 1.0.1? The
immediate priority is
Hey Andrew,
Yeah, that would be preferable. Definitely worth investigating both,
but the regression is more pressing at the moment.
- Patrick
On Mon, Jul 14, 2014 at 10:02 PM, Andrew Ash and...@andrewash.com wrote:
I don't believe mine is a regression. But it is related to thread safety on
14 matches
Mail list logo
