Awesome, thanks! On Sat, Sep 11, 2021 at 6:34 AM Sean Owen <sro...@gmail.com> wrote:
> Looks like this was improved in > https://issues.apache.org/jira/browse/SPARK-35701 for 3.2.0 > > On Fri, Sep 10, 2021 at 10:21 PM Kohki Nishio <tarop...@gmail.com> wrote: > >> Hello, >> I'm running spark in local mode and seeing multiple threads showing like >> below, anybody knows why it's not using a concurrent hash map ? >> >> --- >> "request-handler-dispatcher-19785" #107532 prio=5 os_prio=0 >> tid=0x00007fbd78036000 nid=0x4ebf runnable [0x00007fc6e83af000] >> java.lang.Thread.State: RUNNABLE >> at java.util.Collections$SynchronizedMap.get(Collections.java:2586) >> - locked <0x00007fc901c7d9f8> (a java.util.Collections$SynchronizedMap) >> at org.apache.spark.sql.internal.SQLConf.getConf(SQLConf.scala:3750) >> at >> org.apache.spark.sql.internal.SQLConf.constraintPropagationEnabled(SQLConf.scala:3346) >> >> "Executor task launch worker for task 23.0 in stage 279783.0 (TID >> 449746)" #107929 daemon prio=5 os_prio=0 tid=0x00007fbe2c005000 nid=0x55f6 >> waiting for monitor entry [0x00007fc6e4037000] >> java.lang.Thread.State: BLOCKED (on object monitor) >> at java.util.Collections$SynchronizedMap.get(Collections.java:2586) >> - waiting to lock <0x00007fc901c7d9f8> (a >> java.util.Collections$SynchronizedMap) >> at org.apache.spark.sql.internal.SQLConf.getConf(SQLConf.scala:3750) >> at >> org.apache.spark.sql.internal.SQLConf.methodSplitThreshold(SQLConf.scala:3330) >> >> "Executor task launch worker for task 16.0 in stage 279883.0 (TID >> 449728)" #107690 daemon prio=5 os_prio=0 tid=0x00007fbc60056800 nid=0x53c6 >> waiting for monitor entry [0x00007fc65a3a8000] >> java.lang.Thread.State: BLOCKED (on object monitor) >> at java.util.Collections$SynchronizedMap.get(Collections.java:2586) >> - waiting to lock <0x00007fc901c7d9f8> (a >> java.util.Collections$SynchronizedMap) >> at org.apache.spark.sql.internal.SQLConf.getConf(SQLConf.scala:3750) >> at >> org.apache.spark.sql.internal.SQLConf.planChangeLogLevel(SQLConf.scala:3160) >> at >> org.apache.spark.sql.catalyst.rules.PlanChangeLogger.<init>(RuleExecutor.scala:49) >> >> --- >> >> >> -- >> Kohki Nishio >> > -- Kohki Nishio
