[ https://issues.apache.org/jira/browse/PIO-182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663026#comment-16663026 ]
ASF GitHub Bot commented on PIO-182: ------------------------------------ takezoe commented on a change in pull request #482: [PIO-182] Add async methods to LEventStore URL: https://github.com/apache/predictionio/pull/482#discussion_r227999001 ########## File path: data/src/main/scala/org/apache/predictionio/data/store/LEventStore.scala ########## @@ -20,15 +20,27 @@ package org.apache.predictionio.data.store import org.apache.predictionio.data.storage.Storage import org.apache.predictionio.data.storage.Event - import org.joda.time.DateTime -import scala.concurrent.Await -import scala.concurrent.ExecutionContext.Implicits.global +import scala.concurrent.{Await, ExecutionContext, Future} import scala.concurrent.duration.Duration /** This object provides a set of operation to access Event Store - * without going through Spark's parallelization + * without going through Spark's parallelization. + * + * Note that blocking methods of this object uses + * `scala.concurrent.ExecutionContext.Implicits.global` internally. + * Since this is a thread pool which has a number of threads equal to available + * processors, parallelism is limited up to the number of processors. + * + * If this limitation become bottleneck of resource usage, you can increase the + * number of threads by declaring following VM options before calling "pio deploy": + * + * <pre> + * export JAVA_OPTS="$JAVA_OPTS \ + * -Dscala.concurrent.context.numThreads=1000 \ + * -Dscala.concurrent.context.maxThreads=1000" + * </pre> Review comment: @Wei-1 Thanks for your feedback! I added line comment on importing as below: ```scala // Import here to ensure ExecutionContext.Implicits.global is used only this method import scala.concurrent.ExecutionContext.Implicits.global ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add asynchronous (non-blocking) methods to LEventStore > ------------------------------------------------------ > > Key: PIO-182 > URL: https://issues.apache.org/jira/browse/PIO-182 > Project: PredictionIO > Issue Type: Improvement > Components: Core > Affects Versions: 0.13.0 > Reporter: Naoki Takezoe > Assignee: Naoki Takezoe > Priority: Major > > The current implementation of {{LEventStore}} has only synchronous (blocking) > methods. Since they use {{ExecutionContext.Implicit.global}}, its parallelism > is limited up to the number of processors. This means engine server's > parallelism is also limited if we use these methods in prediction logic. > To solve this problem, {{Future}} version of these methods should be added to > {{LEventStore}} and also current blocking methods should be modified to take > {{ExecutionContext}} (as an implicit parameter). > See also: > https://lists.apache.org/thread.html/f14e4f8f29410e4585b3d8e9f646b88293a605f4716d3c4d60771854@%3Cuser.predictionio.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005)