[
https://issues.apache.org/jira/browse/KUDU-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15291710#comment-15291710
]
Andy Grove commented on KUDU-1453:
----------------------------------
This is partially resolved by this change which ensures that each JVM instance
only creates a single KuduClient, instead of creating one client per task per
partition.
After this fix we are able to run a TPC-H benchmark successfully.
http://gerrit.cloudera.org:8080/#/c/3124
> Spark executors leak kudu clients and netty threads
> ---------------------------------------------------
>
> Key: KUDU-1453
> URL: https://issues.apache.org/jira/browse/KUDU-1453
> Project: Kudu
> Issue Type: Bug
> Components: spark
> Affects Versions: 0.8.0
> Reporter: Todd Lipcon
> Priority: Blocker
>
> On a test cluster, every time I run a spark SQL query against a table, each
> of my spark worker tasks end up with another ~500 netty worker threads
> created. It seems like each Spark partition/task is creating its own
> KuduClient which then creates a bunch of worker threads and never cleans them
> up.
> I'm calling this a blocker since after ~20 queries or so, the machines woudl
> run out of threads and crash.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
