Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[Steven Jacobs (Code Review)]

Steven Jacobs has submitted this change and it was merged.

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..


Prepare AsterixDB for Pre-Distributed Jobs

Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1366
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: abdullah alamoudi 
---
M 
asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveSourceOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/HyracksClientInterfaceFunctions.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/HyracksClientInterfaceRemoteProxy.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/HyracksConnection.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/IHyracksClientConnection.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/IHyracksClientInterface.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/job/JobFlag.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClientInterfaceIPCI.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/NodeControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java
10 files changed, 90 insertions(+), 22 deletions(-)

Approvals:
  abdullah alamoudi: Looks good to me, approved
  Jenkins: Verified; Verified

Objections:
  Jenkins: Violations found



diff --git 
a/asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveSourceOperatorNodePushable.java
 
b/asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveSourceOperatorNodePushable.java
index 7f25896..51ed40d 100644
--- 
a/asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveSourceOperatorNodePushable.java
+++ 
b/asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveSourceOperatorNodePushable.java
@@ -23,9 +23,10 @@
 import org.apache.hyracks.api.comm.IFrameWriter;
 import org.apache.hyracks.api.context.IHyracksTaskContext;
 import org.apache.hyracks.api.exceptions.HyracksDataException;
-import org.apache.hyracks.dataflow.std.base.AbstractOperatorNodePushable;
+import 
org.apache.hyracks.dataflow.std.base.AbstractUnaryOutputSourceOperatorNodePushable;
 
-public abstract class ActiveSourceOperatorNodePushable extends 
AbstractOperatorNodePushable implements IActiveRuntime {
+public abstract class ActiveSourceOperatorNodePushable extends 
AbstractUnaryOutputSourceOperatorNodePushable
+implements IActiveRuntime {
 
 protected final IHyracksTaskContext ctx;
 protected final ActiveManager activeManager;
@@ -108,10 +109,6 @@
 }
 }
 
-@Override
-public final int getInputArity() {
-return 0;
-}
 
 @Override
 public final IFrameWriter getInputFrameWriter(int index) {
diff --git 
a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/HyracksClientInterfaceFunctions.java
 
b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/HyracksClientInterfaceFunctions.java
index ef0af26..780e205 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/HyracksClientInterfaceFunctions.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/HyracksClientInterfaceFunctions.java
@@ -107,17 +107,25 @@
 private final byte[] acggfBytes;
 private final EnumSet jobFlags;
 private final DeploymentId deploymentId;
+private final JobId jobId;
 
-public StartJobFunction(byte[] acggfBytes, EnumSet jobFlags) {
-this.acggfBytes = acggfBytes;
-this.jobFlags = jobFlags;
-this.deploymentId = null;
-}
-
-public StartJobFunction(DeploymentId deploymentId, byte[] acggfBytes, 
EnumSet jobFlags) {
+public StartJobFunction(DeploymentId deploymentId, byte[] acggfBytes, 
EnumSet jobFlags, JobId jobId) {
 this.acggfBytes = acggfBytes;
 this.jobFlags = jobFlags;
 this.deploymentId = deploymentId;
+this.jobId = jobId;
+}
+
+public StartJobFunction(byte[] acggfBytes, EnumSet jobFlags, 
JobId jobId) {
+this(null, acggfBytes, jobFlags, jobId);
+}
+
+public StartJobFunction(byte[] acggfBytes, EnumSet jobFlags) {
+this(null, acggfBytes, jobFlags, null);
+}
+
+public StartJobFunction(DeploymentId deploymentId, byte[] acggfBytes, 

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..


Patch Set 5: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..


Patch Set 5: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1256/ 
: SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..


Patch Set 5:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1256/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..


Patch Set 5:

WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack

PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..


Patch Set 5:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3460/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[Steven Jacobs (Code Review)]

Hello abdullah alamoudi, Till Westmann, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1366

to look at the new patch set (#5).

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..

Prepare AsterixDB for Pre-Distributed Jobs

Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
---
M 
asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveSourceOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/HyracksClientInterfaceFunctions.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/HyracksClientInterfaceRemoteProxy.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/HyracksConnection.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/IHyracksClientConnection.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/IHyracksClientInterface.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/job/JobFlag.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClientInterfaceIPCI.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/NodeControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java
10 files changed, 90 insertions(+), 22 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/66/1366/5
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[Steven Jacobs (Code Review)]

Steven Jacobs has posted comments on this change.

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..


Patch Set 4:

(4 comments)

Addressed small comments

https://asterix-gerrit.ics.uci.edu/#/c/1366/4/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/IHyracksClientConnection.java
File 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/IHyracksClientConnection.java:

Line 94:  * @throws Exception
> Pls add annotations for the jobId parameter and the return value.  
Done


Line 197: EnumSet jobFlags)
> Is the code format right?
Done


https://asterix-gerrit.ics.uci.edu/#/c/1366/4/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java
File 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java:

Line 195: acg = (ActivityClusterGraph) 
DeploymentUtils.deserialize(acgBytes, deploymentId, appCtx);
> The acg bytes shouldn't be sent from CC (i.e., it should be a null) if it's
Yes, for now they sent even when not needed. This is one of the things that 
needs to be addressed in future. Added TODO in ClientInterfaceIPCI


Line 197: acgMap.put(jobId, acg);
> Need a way to remove a stored job.
Added a TODO


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[Yingyi Bu (Code Review)]

Yingyi Bu has posted comments on this change.

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..


Patch Set 4:

(4 comments)

I have major comments here:

1. To my understanding, there are three things that we can avoid for a stored 
job:
-- serialize acg objects at CC side;
-- ship the acg bytes from CC to NC;
-- deseralize acg bytes at NC.

It seems the first two items are missing in this change?

2. Could we have a separate interface method, e.g., called prepare, to 
pre-distribute a stored job,  instead of using job flag to specialize it?  The 
input jobId in the startJob method seems sufficient to tell it's a stored job 
or not.

3. Full lifecycle support is needed, e.g., destroying a pre-distributed job. 
Maybe we can have another interface method call destroy.

https://asterix-gerrit.ics.uci.edu/#/c/1366/4/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/IHyracksClientConnection.java
File 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/IHyracksClientConnection.java:

Line 94:  * @throws Exception
Pls add annotations for the jobId parameter and the return value.  

Particularly, explain the relationship between the input job id and the 
returned job id.


Line 197: EnumSet jobFlags)
Is the code format right?


https://asterix-gerrit.ics.uci.edu/#/c/1366/4/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java
File 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java:

Line 195: acg = (ActivityClusterGraph) 
DeploymentUtils.deserialize(acgBytes, deploymentId, appCtx);
The acg bytes shouldn't be sent from CC (i.e., it should be a null) if it's not 
the first time for a stored job execution.

To my understanding, the acgBytes are still sent every time?


Line 197: acgMap.put(jobId, acg);
Need a way to remove a stored job.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..


Patch Set 4: Code-Review+1

I am happy with this change as long as we capture the issues we discussed and 
marked to be resolved later.
I would give a +2 but I am giving others a chance to take a look.

To others:
This is a fairly compact change and reviewing it will not take you time :)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..


Patch Set 4: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1230/ 
: SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..


Patch Set 4:

WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack

PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..


Patch Set 4:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3430/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..


Patch Set 3:

(3 comments)

Looks good overall. just minor comments.

https://asterix-gerrit.ics.uci.edu/#/c/1366/3/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/HyracksClientInterfaceFunctions.java
File 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/HyracksClientInterfaceFunctions.java:

Line 112: private StartJobFunction(DeploymentId deploymentId, byte[] 
acggfBytes, EnumSet jobFlags, JobId jobId) {
why is this private? this is the only Constructor that can be used to set all 4 
members.
with the current change, seems like it is not possible to have both deployment 
id and job id.
any particular reason? can we add a comment about that. to me, that strongly 
indicates a need for refactoring?

Not necessarily in this change since I think this was already bad! maybe create 
an issue to refactor?


https://asterix-gerrit.ics.uci.edu/#/c/1366/3/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/NodeControllerService.java
File 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/NodeControllerService.java:

Line 363: public Map getACGMap() {
should we rename this to getActivityClusterGraphMap()?. just a suggestion. 
seems easier than trying to know what ACG is


https://asterix-gerrit.ics.uci.edu/#/c/1366/3/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java
File 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java:

Line 197: acgMap.put(jobId, acg);
this map is only growing. when/how do we remove activity cluster graphs from 
the map?

I think we need a way to do that. maybe a message from cc to nc using message 
broker to uncache job/ remove stored job, or something like that?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..


Patch Set 3: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1217/ 
: SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[Till Westmann (Code Review)]

Till Westmann has posted comments on this change.

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..


Patch Set 3: Code-Review+1

Let Abdullah or Yingyi have a look as well.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..


Patch Set 3:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1217/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..


Patch Set 3:

WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack

PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..


Patch Set 3:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3417/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[Steven Jacobs (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1366

to look at the new patch set (#3).

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..

Prepare AsterixDB for Pre-Distributed Jobs

Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
---
M 
asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveSourceOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/HyracksClientInterfaceFunctions.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/HyracksClientInterfaceRemoteProxy.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/HyracksConnection.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/IHyracksClientConnection.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/IHyracksClientInterface.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/job/JobFlag.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClientInterfaceIPCI.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/NodeControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java
10 files changed, 84 insertions(+), 21 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/66/1366/3
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..


Patch Set 2:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3416/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: Prepare AsterixDB for Pre-Distributed Jobs

[Steven Jacobs (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1366

to look at the new patch set (#2).

Change subject: Prepare AsterixDB for Pre-Distributed Jobs
..

Prepare AsterixDB for Pre-Distributed Jobs

Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
---
M 
asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveSourceOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/HyracksClientInterfaceFunctions.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/HyracksClientInterfaceRemoteProxy.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/HyracksConnection.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/IHyracksClientConnection.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/IHyracksClientInterface.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/job/JobFlag.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClientInterfaceIPCI.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/NodeControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java
10 files changed, 101 insertions(+), 39 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/66/1366/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann