[GitHub] [openwhisk] ningyougang commented on a diff in pull request #5313: Exclude warmed containers in disabled invokers.

[GitBox]

ningyougang commented on code in PR #5313:
URL: https://github.com/apache/openwhisk/pull/5313#discussion_r949988091


##########
core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/container/ContainerManager.scala:
##########
@@ -157,43 +139,64 @@ class ContainerManager(jobManagerFactory: ActorRefFactory 
=> ActorRef,
     case _ =>
   }
 
-  private def createContainer(msgs: List[ContainerCreationMessage],
-                              memory: ByteSize,
-                              invocationNamespace: String): Unit = {
+  private def createContainer(msgs: List[ContainerCreationMessage], memory: 
ByteSize, invocationNamespace: String)(
+    implicit logging: Logging): Unit = {
     logging.info(this, s"received ${msgs.size} creation message 
[${msgs.head.invocationNamespace}:${msgs.head.action}]")
-    val coldCreations = filterWarmedCreations(msgs)
-    if (coldCreations.nonEmpty)
-      ContainerManager
-        .getAvailableInvokers(etcdClient, memory, invocationNamespace)
-        .flatMap { invokers =>
-          if (invokers.isEmpty) {
-            coldCreations.foreach { msg =>
-              ContainerManager.sendState(
-                FailedCreationJob(
-                  msg.creationId,
-                  msg.invocationNamespace,
-                  msg.action,
-                  msg.revision,
-                  NoAvailableInvokersError,
-                  s"No available invokers."))
-            }
-            Future.failed(NoCapacityException("No available invokers."))
-          } else {
-            coldCreations.foreach { msg =>
-              creationJobManager ! RegisterCreationJob(msg)
-            }
+    ContainerManager
+      .getAvailableInvokers(etcdClient, memory, invocationNamespace)
+      .foreach { invokers =>
+        if (invokers.isEmpty) {
+          msgs.foreach(ContainerManager.sendState(_, NoAvailableInvokersError, 
s"No available invokers."))
+        } else {
+          val (coldCreations, warmedCreations) =
+            ContainerManager.filterWarmedCreations(warmedContainers, 
inProgressWarmedContainers, invokers, msgs)
+
+          // handle warmed creation
+          val chosenInvokers: immutable.Seq[Option[(Int, 
ContainerCreationMessage)]] = warmedCreations.map {
+            warmedCreation =>
+              // update the in-progress map for warmed containers
+              // even if it is done in the filterWarmedCreations method, it is 
still necessary to apply the change to the original map
+              
warmedCreation._3.foreach(inProgressWarmedContainers.update(warmedCreation._1.creationId.asString,
 _))
+
+              // send creation message to the target invoker
+              warmedCreation._2 map { chosenInvoker =>
+                val msg = warmedCreation._1
+                creationJobManager ! RegisterCreationJob(msg)
+                sendCreationContainerToInvoker(messagingProducer, 
chosenInvoker, msg)
+                (chosenInvoker, msg)
+              }
+          }
 
-            Future {
-              ContainerManager
-                .schedule(invokers, coldCreations, memory)
-                .map { pair =>
-                  sendCreationContainerToInvoker(messagingProducer, 
pair.invokerId.toInt, pair.msg)
-                }
+          val updatedInvokers = chosenInvokers.foldLeft(invokers) { (invokers, 
chosenInvoker) =>
+            chosenInvoker match {
+              case Some((chosenInvoker, msg)) =>
+                updateInvokerMemory(chosenInvoker, 
msg.whiskActionMetaData.limits.memory.megabytes, invokers)

Review Comment:
   hm.for `coldCreations`, i understand why need to 
`updateInvokerMemory(scheduledPair.invokerId, 
msg.whiskActionMetaData.limits.memory.megabytes, invokers)`
   
   But for `warmCreations`, i don't understand why need to that, 
   in my understanding， seems has no need to update memory for `warmCreations`
   
   But for `coldCreations`, i still have some concerns, e.g.
   If has a lot of `coldCreations`,  so update memory operation may executes 
frequently during a very short time, but invoker side, there still exist a lot 
of
   free memory.
   I am not sure whether this case would happen.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@openwhisk.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org