Yingyi Bu has posted comments on this change. Change subject: Prepare AsterixDB for Pre-Distributed Jobs ......................................................................
Patch Set 4: (4 comments) I have major comments here: 1. To my understanding, there are three things that we can avoid for a stored job: -- serialize acg objects at CC side; -- ship the acg bytes from CC to NC; -- deseralize acg bytes at NC. It seems the first two items are missing in this change? 2. Could we have a separate interface method, e.g., called prepare, to pre-distribute a stored job, instead of using job flag to specialize it? The input jobId in the startJob method seems sufficient to tell it's a stored job or not. 3. Full lifecycle support is needed, e.g., destroying a pre-distributed job. Maybe we can have another interface method call destroy. https://asterix-gerrit.ics.uci.edu/#/c/1366/4/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/IHyracksClientConnection.java File hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/IHyracksClientConnection.java: Line 94: * @throws Exception Pls add annotations for the jobId parameter and the return value. Particularly, explain the relationship between the input job id and the returned job id. Line 197: EnumSet<JobFlag> jobFlags) Is the code format right? https://asterix-gerrit.ics.uci.edu/#/c/1366/4/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java File hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java: Line 195: acg = (ActivityClusterGraph) DeploymentUtils.deserialize(acgBytes, deploymentId, appCtx); The acg bytes shouldn't be sent from CC (i.e., it should be a null) if it's not the first time for a stored job execution. To my understanding, the acgBytes are still sent every time? Line 197: acgMap.put(jobId, acg); Need a way to remove a stored job. -- To view, visit https://asterix-gerrit.ics.uci.edu/1366 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Steven Jacobs <sjaco...@ucr.edu> Gerrit-Reviewer: Jenkins <jenk...@fulliautomatix.ics.uci.edu> Gerrit-Reviewer: Steven Jacobs <sjaco...@ucr.edu> Gerrit-Reviewer: Till Westmann <ti...@apache.org> Gerrit-Reviewer: Yingyi Bu <buyin...@gmail.com> Gerrit-Reviewer: abdullah alamoudi <bamou...@gmail.com> Gerrit-HasComments: Yes
