Looks like this was improved in https://issues.apache.org/jira/browse/SPARK-35701 for 3.2.0
On Fri, Sep 10, 2021 at 10:21 PM Kohki Nishio <tarop...@gmail.com> wrote: > Hello, > I'm running spark in local mode and seeing multiple threads showing like > below, anybody knows why it's not using a concurrent hash map ? > > --- > "request-handler-dispatcher-19785" #107532 prio=5 os_prio=0 > tid=0x00007fbd78036000 nid=0x4ebf runnable [0x00007fc6e83af000] > java.lang.Thread.State: RUNNABLE > at java.util.Collections$SynchronizedMap.get(Collections.java:2586) > - locked <0x00007fc901c7d9f8> (a java.util.Collections$SynchronizedMap) > at org.apache.spark.sql.internal.SQLConf.getConf(SQLConf.scala:3750) > at > org.apache.spark.sql.internal.SQLConf.constraintPropagationEnabled(SQLConf.scala:3346) > > "Executor task launch worker for task 23.0 in stage 279783.0 (TID 449746)" > #107929 daemon prio=5 os_prio=0 tid=0x00007fbe2c005000 nid=0x55f6 waiting > for monitor entry [0x00007fc6e4037000] > java.lang.Thread.State: BLOCKED (on object monitor) > at java.util.Collections$SynchronizedMap.get(Collections.java:2586) > - waiting to lock <0x00007fc901c7d9f8> (a > java.util.Collections$SynchronizedMap) > at org.apache.spark.sql.internal.SQLConf.getConf(SQLConf.scala:3750) > at > org.apache.spark.sql.internal.SQLConf.methodSplitThreshold(SQLConf.scala:3330) > > "Executor task launch worker for task 16.0 in stage 279883.0 (TID 449728)" > #107690 daemon prio=5 os_prio=0 tid=0x00007fbc60056800 nid=0x53c6 waiting > for monitor entry [0x00007fc65a3a8000] > java.lang.Thread.State: BLOCKED (on object monitor) > at java.util.Collections$SynchronizedMap.get(Collections.java:2586) > - waiting to lock <0x00007fc901c7d9f8> (a > java.util.Collections$SynchronizedMap) > at org.apache.spark.sql.internal.SQLConf.getConf(SQLConf.scala:3750) > at > org.apache.spark.sql.internal.SQLConf.planChangeLogLevel(SQLConf.scala:3160) > at > org.apache.spark.sql.catalyst.rules.PlanChangeLogger.<init>(RuleExecutor.scala:49) > > --- > > > -- > Kohki Nishio >
