[
https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16758034#comment-16758034
]
Uwe Schindler commented on SOLR-9515:
-------------------------------------
I agree with Mark Miller. BTW, the commonPool and the ForkJoin pools are made
for cpu-intensive calculation, so in general they won't need any special
permissions. What Hadoop is doing here is very untypical, because it misuses
the fork-join-pool for I/O stuff which generally just waits for I/O devices to
finish, so it's not a cpu-intensive job. This especially applies for the common
pool (everybody in Java says: Whenever you do something with common-pool like
stream.parallel() never ever do I/O inside.
The change in Java 9 now unfortunately changes this to all fork join pools.
I think we should file a bug to change hadoop to use their own thread factory
or use another pool.
bq. Definitely didn't work. From the ForkJoinPool javadocs
The sentence in the docs was only meant for the "commonPool", so when you
change that to 0 it does not create any threads at all, so stream().parallel()
is not doing anything. Hadoop is using its own ForkJoin pool that breaks.
I am fine with the same approach for tests that we did for the Jetty server
(it's a test only class here): Fork it and fix it until Hadoop fixed their bug.
We'd really open a bug report.
> Update to Hadoop 3
> ------------------
>
> Key: SOLR-9515
> URL: https://issues.apache.org/jira/browse/SOLR-9515
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Mark Miller
> Assignee: Kevin Risden
> Priority: Blocker
> Fix For: 8.0, master (9.0)
>
> Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch,
> SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared.
> I'll start up a dev branch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
