[ https://issues.apache.org/jira/browse/SPARK-25286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16597921#comment-16597921 ]
Apache Spark commented on SPARK-25286: -------------------------------------- User 'MaxGekk' has created a pull request for this issue: https://github.com/apache/spark/pull/22292 > Remove dangerous parmap > ----------------------- > > Key: SPARK-25286 > URL: https://issues.apache.org/jira/browse/SPARK-25286 > Project: Spark > Issue Type: Improvement > Components: Spark Core > Affects Versions: 2.4.0 > Reporter: Maxim Gekk > Priority: Major > > One of parmap methods accepts an execution context created outside of parmap. > If the parmap method is called recursively on a thread pool limited by size, > it could lead to deadlocks. See the JIRA tickets: SPARK-25240 and SPARK-25283 > . To eliminate the problems in the future, need to remove parmap() with the > signature: > {code:scala} > def parmap[I, O, Col[X] <: TraversableLike[X, Col[X]]] > (in: Col[I]) > (f: I => O) > (implicit > cbf: CanBuildFrom[Col[I], Future[O], Col[Future[O]]], // For in.map > cbf2: CanBuildFrom[Col[Future[O]], O, Col[O]], // for Future.sequence > ec: ExecutionContext > ): Col[O] > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org