Change in asterixdb[master]: ASTERIXDB-1451: Fix type propagation to enforced indexes

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: ASTERIXDB-1451: Fix type propagation to enforced indexes
..


Patch Set 2:

(14 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1146/2//COMMIT_MSG
Commit Message:

Line 7: ASTERIXDB-1451: Fix type propagation to enforced indexes
> This sounds everything is fixed:-)
Done


Line 16: - Replace all record casts with open field casts.
> Again, two options similar to the above message :-)
Done


https://asterix-gerrit.ics.uci.edu/#/c/1146/2/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java:

Line 123: if (usedNewRecordVars.size() == 1) {
> It seems that check newRecordExpr.getExpressionTag() is more accurate.
Done


Line 154: expr.getValue().getUsedVariables(metaVars);
> use expr.getExpressionTag()
Done


Line 156: if (metaVars.size() > 1) {
> What matters is the size of newMetaExprs?
Done


Line 158: "Number of meta fields can't be more than 1. 
Number of meta fields found = " + metaVars.size());
> The error message seems misleading?
Done


Line 245: Map fieldVarsForNewRecord = 
new HashMap<>();
> It's not intuitive what "Old" means here.  "New" is used in previous variab
Done


Line 257: || primaryIndexModificationOp.getOperation() == 
Kind.DELETE) {
> "prepareVarsAndExpressions" -> "injectFieldAccessesForIndexes"?
Done


Line 258: prepareVarsAndExpressions(context, dataset, indexes, 
fieldVarsForNewRecord, recType,
> Can we have a separate method to populate fieldVarsForNewRecord of put that
Done


Line 271: currentTop = prepareVarsAndExpressions(context, 
dataset, indexes, fieldVarsForOldRecord, recType,
> It's not very clear to me why we need to call prepareVarsAndExpressions(...
for upsert, we have a record before the operation and a record after the 
operation. we need both in the upsert case to properly fix values in the 
secondary indexes. Actually, for the delete case, we only need the record 
before the delete but that is not the way it is now which could lead to 
inconsistency between primary and secondary indexes. hence, the comment about 
the issue.


Line 519: List indexes, Map 
fieldVarsForNewRecord, ARecordType recType,
> Eliminate the side effect on fieldVarsForNewRecord?
Done


Line 597: private static int getFieldPosition(String[] recordFields, 
List fields) {
> This method is not necessary.  You can call:
Done


Line 692: return indicator | fieldName.hashCode();
> using the standard way?
Done


https://asterix-gerrit.ics.uci.edu/#/c/1146/2/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
File 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java:

Line 3137:  * the type of an indexed record
> I'm not sure if we still need this mechanism as all casting should be field
That is correct but for now, I am keeping this since the index create mechanism 
still uses this


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1146
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6a80105798ea1c86a6a0eb69a79b9573b54931b7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1146

to look at the new patch set (#3).

Change subject: ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert
..

ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert

This change includes the following:
- Introduce cast function in case of delete operation
  after the primary index to ensure types are passed
  correctly to enforced indexes.
- Introduce cast function in case of upsert operation
  before old secondary keys extraction to ensure types
  are passed correctly to enforced indexes.
- Replace all record casts with open field casts for
  insert/delete/upsert operations.
- Sonar-Qube fixes.

Change-Id: I6a80105798ea1c86a6a0eb69a79b9573b54931b7
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/ReplaceSinkOpWithCommitOpRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/typecast/StaticTypeCastUtil.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/disjunction-to-join-delete-1.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/disjunction-to-join-delete-2.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/disjunction-to-join-delete-3.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/insert-and-scan-dataset-with-index.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/scan-delete-rtree-secondary-index.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/scan-insert-secondary-index.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-ngram-index-search-in-delete.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-rtree-index-search-in-delete.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-secondary-btree-index-search-in-delete.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-word-index-search-in-delete.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.5.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.5.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entities/Index.java
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/base/TypeCastUtils.java
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/CastTypeComputer.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/IndexInsertDeleteUpsertOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/InsertDeleteUpsertOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/UsedVariableVisitor.java
M 
hyracks-fullstack/alge

Change in asterixdb[master]: Small Cleanup Towards Fixing LifeCycle Issues

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1148

to look at the new patch set (#5).

Change subject: Small Cleanup Towards Fixing LifeCycle Issues
..

Small Cleanup Towards Fixing LifeCycle Issues

Before this change, dataset lifecycle manager was providing a set
of functionalities that are loosly related to management of datasets
and indexes. However, it was not clear what the possible states of
a dataset or an index and what is the responsibility of each object.

This change takes the first step towards fixing this area. Indexes
of a dataset are now grouped together under a single lifecycle class

A resource aka dataset must be created outside the lifecycle manager
and registered with it before it can be assigned resources (memory)
and before it can be used by any operation. This is still not the
case.

Change-Id: I84005a33837725f41ae63297a3711215dccce1d8
---
M 
asterixdb/asterix-active/src/main/java/org/apache/asterix/active/message/ActiveManagerMessage.java
M 
asterixdb/asterix-active/src/main/java/org/apache/asterix/active/message/ActivePartitionMessage.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/AsterixAppRuntimeContextProviderForRecovery.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/AsterixNCAppRuntimeContext.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/messaging/CCMessageBroker.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/messaging/NCMessageBroker.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IAsterixAppRuntimeContext.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IDatasetLifecycleManager.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/BaseOperationTracker.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/CorrelatedPrefixMergePolicy.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/CorrelatedPrefixMergePolicyFactory.java
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetInfo.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetResource.java
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetVirtualBufferCaches.java
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/IndexInfo.java
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/Info.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
D 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/messaging/AbstractApplicationMessage.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/messaging/api/IApplicationMessage.java
D 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/messaging/api/IApplicationMessageCallback.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/messaging/api/ICCMessageBroker.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/messaging/api/INCMessageBroker.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/IAsterixAppRuntimeContextProvider.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataNode.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java
M 
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/management/ReplicationChannel.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/AbstractFailbackPlanMessage.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/CompleteFailbackRequestMessage.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/CompleteFailbackResponseMessage.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/PreparePartitionsFailbackRequestMessage.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/PreparePartitionsFailbackResponseMessage.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/ReplicaEventMessage.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/ReportMaxResourceIdMessage.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/ReportMaxResourceIdRequestMessage.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/ResourceIdRequestMessage.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/ResourceIdRequestResponseMessage.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/ru

Change in asterixdb[master]: Small Cleanup Towards Fixing LifeCycle Issues

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Small Cleanup Towards Fixing LifeCycle Issues
..


Patch Set 4:

(3 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1148/4/asterixdb/asterix-app/src/main/java/org/apache/asterix/messaging/NCMessageBroker.java
File 
asterixdb/asterix-app/src/main/java/org/apache/asterix/messaging/NCMessageBroker.java:

Line 61: 
ncs.sendApplicationMessageToCC(JavaSerializationUtils.serialize(message), null);
> Why is the callback mechanism removed? Did you discuss with Murtadha?
I did discuss it with Murtadha,
instead of the callback, the return message should be the callback itself.

The functionality is still there and this is much cleaner.


https://asterix-gerrit.ics.uci.edu/#/c/1148/4/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycle.java
File 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycle.java:

Line 39: public class DatasetLifecycle implements Comparable {
> The name is confusing. An object lifecycle is the time between creation and
I actually have renamed this already on my local branch to DatasetResource


https://asterix-gerrit.ics.uci.edu/#/c/1148/4/hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/IndexDataflowHelper.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/IndexDataflowHelper.java:

Line 77: //how is this right?? 
> Please file an Issue. Rarely do we remember comments in the code.
Done


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1148
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I84005a33837725f41ae63297a3711215dccce1d8
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1146

to look at the new patch set (#4).

Change subject: ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert
..

ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert

This change includes the following:
- Introduce cast function in case of delete operation
  after the primary index to ensure types are passed
  correctly to enforced indexes.
- Introduce cast function in case of upsert operation
  before old secondary keys extraction to ensure types
  are passed correctly to enforced indexes.
- Replace all record casts with open field casts for
  insert/delete/upsert operations.
- Sonar-Qube fixes.

Change-Id: I6a80105798ea1c86a6a0eb69a79b9573b54931b7
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/ReplaceSinkOpWithCommitOpRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/typecast/StaticTypeCastUtil.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/disjunction-to-join-delete-1.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/disjunction-to-join-delete-2.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/disjunction-to-join-delete-3.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/insert-and-scan-dataset-with-index.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/scan-delete-rtree-secondary-index.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/scan-insert-secondary-index.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-ngram-index-search-in-delete.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-rtree-index-search-in-delete.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-secondary-btree-index-search-in-delete.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-word-index-search-in-delete.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.5.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.5.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entities/Index.java
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/base/TypeCastUtils.java
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/CastTypeComputer.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/IndexInsertDeleteUpsertOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/InsertDeleteUpsertOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/UsedVariableVisitor.java
M 
hyracks-fullstack/alge

Change in asterixdb[master]: Small Cleanup Towards Fixing LifeCycle Issues

[abdullah alamoudi (Code Review)]

Hello Till Westmann, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1148

to look at the new patch set (#6).

Change subject: Small Cleanup Towards Fixing LifeCycle Issues
..

Small Cleanup Towards Fixing LifeCycle Issues

Before this change, dataset lifecycle manager was providing a set
of functionalities that are loosly related to management of datasets
and indexes. However, it was not clear what the possible states of
a dataset or an index and what is the responsibility of each object.

This change takes the first step towards fixing this area. Indexes
of a dataset are now grouped together under a single lifecycle class

A resource aka dataset must be created outside the lifecycle manager
and registered with it before it can be assigned resources (memory)
and before it can be used by any operation. This is still not the
case.

Change-Id: I84005a33837725f41ae63297a3711215dccce1d8
---
M 
asterixdb/asterix-active/src/main/java/org/apache/asterix/active/message/ActiveManagerMessage.java
M 
asterixdb/asterix-active/src/main/java/org/apache/asterix/active/message/ActivePartitionMessage.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/AsterixAppRuntimeContextProviderForRecovery.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/AsterixNCAppRuntimeContext.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/messaging/CCMessageBroker.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/messaging/NCMessageBroker.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IAsterixAppRuntimeContext.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IDatasetLifecycleManager.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/BaseOperationTracker.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/CorrelatedPrefixMergePolicy.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/CorrelatedPrefixMergePolicyFactory.java
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetInfo.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetResource.java
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetVirtualBufferCaches.java
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/IndexInfo.java
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/Info.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
D 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/messaging/AbstractApplicationMessage.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/messaging/api/IApplicationMessage.java
D 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/messaging/api/IApplicationMessageCallback.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/messaging/api/ICCMessageBroker.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/messaging/api/INCMessageBroker.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/IAsterixAppRuntimeContextProvider.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataNode.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java
M 
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/management/ReplicationChannel.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/AbstractFailbackPlanMessage.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/CompleteFailbackRequestMessage.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/CompleteFailbackResponseMessage.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/PreparePartitionsFailbackRequestMessage.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/PreparePartitionsFailbackResponseMessage.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/ReplicaEventMessage.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/ReportMaxResourceIdMessage.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/ReportMaxResourceIdRequestMessage.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/ResourceIdRequestMessage.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/ResourceIdRequestResponseMessage.java
M 
asterixdb/asterix-runtime/src/main/java/org/ap

Change in asterixdb[master]: Small Cleanup Towards Fixing LifeCycle Issues

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Small Cleanup Towards Fixing LifeCycle Issues
..


Patch Set 6: Code-Review+2

Forwarding

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1148
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I84005a33837725f41ae63297a3711215dccce1d8
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1146

to look at the new patch set (#5).

Change subject: ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert
..

ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert

This change includes the following:
- Introduce cast function in case of delete operation
  after the primary index to ensure types are passed
  correctly to enforced indexes.
- Introduce cast function in case of upsert operation
  before old secondary keys extraction to ensure types
  are passed correctly to enforced indexes.
- Replace all record casts with open field casts for
  insert/delete/upsert operations.
- Sonar-Qube fixes.

Change-Id: I6a80105798ea1c86a6a0eb69a79b9573b54931b7
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/ReplaceSinkOpWithCommitOpRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/typecast/StaticTypeCastUtil.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/disjunction-to-join-delete-1.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/disjunction-to-join-delete-2.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/disjunction-to-join-delete-3.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/insert-and-scan-dataset-with-index.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/scan-delete-rtree-secondary-index.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/scan-insert-secondary-index.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-ngram-index-search-in-delete.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-rtree-index-search-in-delete.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-secondary-btree-index-search-in-delete.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-word-index-search-in-delete.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.5.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.5.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entities/Index.java
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/base/TypeCastUtils.java
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/CastTypeComputer.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/IndexInsertDeleteUpsertOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/InsertDeleteUpsertOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/UsedVariableVisitor.java
M 
hyracks-fullstack/alge

Change in asterixdb[master]: ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert

[abdullah alamoudi (Code Review)]

Hello Yingyi Bu, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1146

to look at the new patch set (#6).

Change subject: ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert
..

ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert

This change includes the following:
- Introduce cast function in case of delete operation
  after the primary index to ensure types are passed
  correctly to enforced indexes.
- Introduce cast function in case of upsert operation
  before old secondary keys extraction to ensure types
  are passed correctly to enforced indexes.
- Replace all record casts with open field casts for
  insert/delete/upsert operations.
- Sonar-Qube fixes.

Change-Id: I6a80105798ea1c86a6a0eb69a79b9573b54931b7
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/ReplaceSinkOpWithCommitOpRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/typecast/StaticTypeCastUtil.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/disjunction-to-join-delete-1.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/disjunction-to-join-delete-2.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/disjunction-to-join-delete-3.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/insert-and-scan-dataset-with-index.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/scan-delete-rtree-secondary-index.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/scan-insert-secondary-index.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-ngram-index-search-in-delete.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-rtree-index-search-in-delete.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-secondary-btree-index-search-in-delete.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-word-index-search-in-delete.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.5.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.5.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entities/Index.java
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/base/TypeCastUtils.java
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/CastTypeComputer.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/IndexInsertDeleteUpsertOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/InsertDeleteUpsertOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/UsedVariableVisitor.java
M 
hyracks-ful

Change in asterixdb[master]: ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert

[abdullah alamoudi (Code Review)]

Hello Yingyi Bu, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1146

to look at the new patch set (#7).

Change subject: ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert
..

ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert

This change includes the following:
- Introduce cast function in case of delete operation
  after the primary index to ensure types are passed
  correctly to enforced indexes.
- Introduce cast function in case of upsert operation
  before old secondary keys extraction to ensure types
  are passed correctly to enforced indexes.
- Replace all record casts with open field casts for
  insert/delete/upsert operations.
- Sonar-Qube fixes.

Change-Id: I6a80105798ea1c86a6a0eb69a79b9573b54931b7
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/ReplaceSinkOpWithCommitOpRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/typecast/StaticTypeCastUtil.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/disjunction-to-join-delete-1.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/disjunction-to-join-delete-2.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/disjunction-to-join-delete-3.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/insert-and-scan-dataset-with-index.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/scan-delete-rtree-secondary-index.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/scan-insert-secondary-index.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-ngram-index-search-in-delete.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-rtree-index-search-in-delete.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-secondary-btree-index-search-in-delete.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-word-index-search-in-delete.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.5.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.5.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entities/Index.java
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/base/TypeCastUtils.java
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/CastTypeComputer.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/IndexInsertDeleteUpsertOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/InsertDeleteUpsertOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/UsedVariableVisitor.java
M 
hyracks-ful

Change in asterixdb[master]: ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert
..


Patch Set 7: Code-Review+2

Forwarding

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1146
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6a80105798ea1c86a6a0eb69a79b9573b54931b7
Gerrit-PatchSet: 7
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert

[abdullah alamoudi (Code Review)]

abdullah alamoudi has submitted this change and it was merged.

Change subject: ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert
..


ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert

This change includes the following:
- Introduce cast function in case of delete operation
  after the primary index to ensure types are passed
  correctly to enforced indexes.
- Introduce cast function in case of upsert operation
  before old secondary keys extraction to ensure types
  are passed correctly to enforced indexes.
- Replace all record casts with open field casts for
  insert/delete/upsert operations.
- Sonar-Qube fixes.

Change-Id: I6a80105798ea1c86a6a0eb69a79b9573b54931b7
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1146
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: abdullah alamoudi 
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/ReplaceSinkOpWithCommitOpRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/typecast/StaticTypeCastUtil.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/disjunction-to-join-delete-1.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/disjunction-to-join-delete-2.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/disjunction-to-join-delete-3.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/insert-and-scan-dataset-with-index.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/scan-delete-rtree-secondary-index.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/scan-insert-secondary-index.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-ngram-index-search-in-delete.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-rtree-index-search-in-delete.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-secondary-btree-index-search-in-delete.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/skip-index/skip-word-index-search-in-delete.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/open-index-enforced/type-checking/enforced-type-delete/enforced-type-delete.5.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/open-index-enforced/type-checking/enforced-type-upsert/enforced-type-upsert.5.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entities/Index.java
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/base/TypeCastUtils.java
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/CastTypeComputer.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/IndexInsertDeleteUpsertOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/InsertDeleteUpsertOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/

Change in asterixdb[master]: Add Default Dataverse and AnyObject Datatype

[abdullah alamoudi (Code Review)]

abdullah alamoudi has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1174

Change subject: Add Default Dataverse and AnyObject Datatype
..

Add Default Dataverse and AnyObject Datatype

Change-Id: Ie02c74c8b005dce73888afd4dfe19ede2c60a8e8
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplicationEntryPoint.java
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta01/meta01.1.adm
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_dataverse/metadata_dataverse.1.adm
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java
A 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBuiltinEntities.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/MetadataConstants.java
7 files changed, 196 insertions(+), 107 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/74/1174/1

diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplicationEntryPoint.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplicationEntryPoint.java
index c11b875..a8637cc 100644
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplicationEntryPoint.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplicationEntryPoint.java
@@ -37,16 +37,15 @@
 import org.apache.asterix.common.config.MessagingProperties;
 import org.apache.asterix.common.replication.IRemoteRecoveryManager;
 import org.apache.asterix.common.transactions.IRecoveryManager;
-import 
org.apache.asterix.transaction.management.resource.PersistentLocalResourceRepository;
 import org.apache.asterix.common.transactions.IRecoveryManager.SystemState;
 import org.apache.asterix.common.utils.StoragePathUtil;
 import org.apache.asterix.event.schema.cluster.Cluster;
 import org.apache.asterix.event.schema.cluster.Node;
 import org.apache.asterix.messaging.MessagingChannelInterfaceFactory;
 import org.apache.asterix.messaging.NCMessageBroker;
-import org.apache.asterix.metadata.bootstrap.MetadataBootstrap;
 import org.apache.asterix.runtime.message.ReportMaxResourceIdMessage;
 import org.apache.asterix.runtime.util.AsterixClusterProperties;
+import 
org.apache.asterix.transaction.management.resource.PersistentLocalResourceRepository;
 import 
org.apache.asterix.transaction.management.service.recovery.RecoveryManager;
 import org.apache.commons.io.FileUtils;
 import org.apache.hyracks.api.application.INCApplicationContext;
@@ -190,11 +189,6 @@
 if (LOGGER.isLoggable(Level.INFO)) {
 LOGGER.info("Stopping Asterix node controller: " + nodeId);
 }
-
-if (isMetadataNode) {
-MetadataBootstrap.stopUniverse();
-}
-
 //Clean any temporary files
 performLocalCleanUp();
 
diff --git 
a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta01/meta01.1.adm
 
b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta01/meta01.1.adm
index e341727..313cefe 100644
--- 
a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta01/meta01.1.adm
+++ 
b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta01/meta01.1.adm
@@ -1,2 +1,3 @@
+{ "DataverseName": "Default", "DataFormat": 
"org.apache.asterix.runtime.formats.NonTaggedDataFormat", "Timestamp": "Wed Sep 
14 23:26:12 EEST 2016", "PendingOp": 0 }
 { "DataverseName": "Metadata", "DataFormat": 
"org.apache.asterix.runtime.formats.NonTaggedDataFormat", "Timestamp": "Tue Jun 
21 15:54:20 PDT 2016", "PendingOp": 0 }
 { "DataverseName": "testdv", "DataFormat": 
"org.apache.asterix.runtime.formats.NonTaggedDataFormat", "Timestamp": "Tue Jun 
21 15:54:27 PDT 2016", "PendingOp": 0 }
diff --git 
a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm
 
b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm
index 1920bab..91639a8 100644
--- 
a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm
+++ 
b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm
@@ -1,3 +1,4 @@
+{ "DataverseName": "Metadata", "DatatypeName": "AnyObject", "Derived": { 
"Tag": "RECORD", "IsAnonymous": false, "Record": { "IsOpen": true, "Fields": [  
] } }, "Timestamp": "Wed Sep 14 23:26:12 EEST 2016" }
 { "DataverseName": "Metadata", "DatatypeName": "CompactionPolicyRecordType", 
"Derived": { "Tag": "RECORD", "IsAnonymous": false, "Record": { "IsOpen": true, 
"Fields": [ { "Fie

Change in asterixdb[master]: Add Default Dataverse and AnyObject Datatype

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Add Default Dataverse and AnyObject Datatype
..


Patch Set 1:

@Till,
Not sure. I will give it a try and add a test case.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1174
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ie02c74c8b005dce73888afd4dfe19ede2c60a8e8
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Add Default Dataverse and AnyObject Datatype

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1174

to look at the new patch set (#2).

Change subject: Add Default Dataverse and AnyObject Datatype
..

Add Default Dataverse and AnyObject Datatype

Change-Id: Ie02c74c8b005dce73888afd4dfe19ede2c60a8e8
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/TypeTranslator.java
M asterixdb/asterix-algebra/src/main/javacc/AQLPlus.jj
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplicationEntryPoint.java
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta01/meta01.1.adm
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_dataverse/metadata_dataverse.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/types/any-object/any-object.3.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M 
asterixdb/asterix-installer/src/test/resources/integrationts/library/results/library-metadata/dataverseDataset/dataverseDataset.1.adm
M asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
M 
asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/expression/TypeReferenceExpression.java
M 
asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/visitor/FormatPrintVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java
A 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBuiltinEntities.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/MetadataConstants.java
M 
asterixdb/asterix-yarn/src/test/resources/library/results/library-metadata/dataverseDataset/dataverseDataset.1.adm
24 files changed, 460 insertions(+), 138 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/74/1174/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1174
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie02c74c8b005dce73888afd4dfe19ede2c60a8e8
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 

Change in asterixdb[master]: Add Default Dataverse and AnyObject Datatype

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1174

to look at the new patch set (#3).

Change subject: Add Default Dataverse and AnyObject Datatype
..

Add Default Dataverse and AnyObject Datatype

Change-Id: Ie02c74c8b005dce73888afd4dfe19ede2c60a8e8
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/TypeTranslator.java
M asterixdb/asterix-algebra/src/main/javacc/AQLPlus.jj
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplicationEntryPoint.java
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta01/meta01.1.adm
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_dataverse/metadata_dataverse.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/types/any-object/any-object.3.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M 
asterixdb/asterix-installer/src/test/resources/integrationts/library/results/library-metadata/dataverseDataset/dataverseDataset.1.adm
M asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
M 
asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/expression/TypeReferenceExpression.java
M 
asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/visitor/FormatPrintVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java
A 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBuiltinEntities.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/MetadataConstants.java
M 
asterixdb/asterix-yarn/src/test/resources/library/results/library-metadata/dataverseDataset/dataverseDataset.1.adm
24 files changed, 473 insertions(+), 153 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/74/1174/3
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1174
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie02c74c8b005dce73888afd4dfe19ede2c60a8e8
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 

Change in asterixdb[master]: Add Default Dataverse and AnyObject Datatype

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1174

to look at the new patch set (#5).

Change subject: Add Default Dataverse and AnyObject Datatype
..

Add Default Dataverse and AnyObject Datatype

This change adds "Default" dataverse and
Metadata.AnyObject type which is the completely
open record type. In addition, this change cleansup between
subsequent test cases lowering the chance that a failed
test causes the following tests to fail. In addition, it
prints out the list of tests which left some leftover.

Change-Id: Ie02c74c8b005dce73888afd4dfe19ede2c60a8e8
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/TypeTranslator.java
M asterixdb/asterix-algebra/src/main/javacc/AQLPlus.jj
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplicationEntryPoint.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/metadata/MetadataTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/optimizer/OptimizerTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/ExecutionTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/SqlppExecutionTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/OptimizerParserTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/RuntimeParserTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/SmokeParserTest.java
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta01/meta01.1.adm
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_dataverse/metadata_dataverse.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/cross-dataverse/drop-dataverse/drop-dataverse.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/cross-dataverse/drop-type-used-elsewhere/drop-type-used-elsewhere.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/cross-dataverse/query-dataset-with-foreign-type/query-dataset-with-foreign-type.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/load-with-index-open_02/load-with-index-open_02.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/query-issue382/query-issue382.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/escapes-err-1/escapes-err-1.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/escapes01/escapes01.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/escapes02/escapes02.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load_non-empty_index/load_non-empty_index.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/type_promotion_0/type_promotion_0.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/misc/flushtest/flushtest.5.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-closed/query-issue208/query-issue208.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-closed/query-issue236/query-issue236.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/upsert/issue1587-foreignDataType/issue1587-foreignDataType.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/flushtest/flushtest.5.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/types/any-object/any-object.3.adm
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comparison/binary_null/binary_null.2.ast
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comparison/binary_null/binary_null.3.ast
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comparison/int16_null/int16_null.2.ast
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comparison/int16_null/int16_null.3.ast
M 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/dml/ins

Change in asterixdb[master]: Add Default Dataverse and AnyObject Datatype

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1174

to look at the new patch set (#6).

Change subject: Add Default Dataverse and AnyObject Datatype
..

Add Default Dataverse and AnyObject Datatype

This change adds "Default" dataverse and
Metadata.AnyObject type which is the completely
open record type. In addition, this change cleansup between
subsequent test cases lowering the chance that a failed
test causes the following tests to fail. In addition, it
prints out the list of tests which left some leftover.

Change-Id: Ie02c74c8b005dce73888afd4dfe19ede2c60a8e8
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/TypeTranslator.java
M asterixdb/asterix-algebra/src/main/javacc/AQLPlus.jj
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplicationEntryPoint.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/metadata/MetadataTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/optimizer/OptimizerTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/ExecutionTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/SqlppExecutionTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/OptimizerParserTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/RuntimeParserTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/SmokeParserTest.java
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta01/meta01.1.adm
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_dataverse/metadata_dataverse.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/cross-dataverse/drop-dataverse/drop-dataverse.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/cross-dataverse/drop-type-used-elsewhere/drop-type-used-elsewhere.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/cross-dataverse/query-dataset-with-foreign-type/query-dataset-with-foreign-type.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/load-with-index-open_02/load-with-index-open_02.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/query-issue382/query-issue382.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/escapes-err-1/escapes-err-1.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/escapes01/escapes01.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/escapes02/escapes02.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load_non-empty_index/load_non-empty_index.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/type_promotion_0/type_promotion_0.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/misc/flushtest/flushtest.5.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-closed/query-issue208/query-issue208.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-closed/query-issue236/query-issue236.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/upsert/issue1587-foreignDataType/issue1587-foreignDataType.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/flushtest/flushtest.5.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/types/any-object/any-object.3.adm
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comparison/binary_null/binary_null.2.ast
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comparison/binary_null/binary_null.3.ast
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comparison/int16_null/int16_null.2.ast
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comparison/int16_null/int16_null.3.ast
M 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/dml/ins

Change in asterixdb[master]: Easier to read parser errors for SQL++

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Easier to read parser errors for SQL++
..


Patch Set 3: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1182
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I13fd54fe2b1237b937a1706cf83fb47ce536b546
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Till Westmann 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Add upsert option for feed

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Add upsert option for feed
..


Patch Set 13: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1068
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ic5133e7c6941fea4110cc9983f99502f364dc810
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Separate static cluster properties from runtime state

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Separate static cluster properties from runtime state
..


Patch Set 2: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1183
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3da72f639476f081ba8a0ef21ec176d32ff8bdec
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Add Default Dataverse and AnyObject Datatype

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Add Default Dataverse and AnyObject Datatype
..


Patch Set 6:

Good catch Till. I am going to fix this.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1174
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ie02c74c8b005dce73888afd4dfe19ede2c60a8e8
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Add Default Dataverse and AnyObject Datatype

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1174

to look at the new patch set (#7).

Change subject: Add Default Dataverse and AnyObject Datatype
..

Add Default Dataverse and AnyObject Datatype

This change adds "Default" dataverse and
Metadata.AnyObject type which is the completely
open record type. In addition, this change cleansup between
subsequent test cases lowering the chance that a failed
test causes the following tests to fail. In addition, it
prints out the list of tests which left some leftover.

Change-Id: Ie02c74c8b005dce73888afd4dfe19ede2c60a8e8
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/TypeTranslator.java
M asterixdb/asterix-algebra/src/main/javacc/AQLPlus.jj
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplicationEntryPoint.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/metadata/MetadataTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/optimizer/OptimizerTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/ExecutionTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/SqlppExecutionTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/OptimizerParserTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/ParserTestExecutor.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/RuntimeParserTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/SmokeParserTest.java
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta01/meta01.1.adm
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_dataverse/metadata_dataverse.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/cross-dataverse/drop-dataverse/drop-dataverse.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/cross-dataverse/drop-type-used-elsewhere/drop-type-used-elsewhere.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/cross-dataverse/query-dataset-with-foreign-type/query-dataset-with-foreign-type.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/load-with-index-open_02/load-with-index-open_02.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/query-issue382/query-issue382.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/escapes-err-1/escapes-err-1.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/escapes01/escapes01.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/escapes02/escapes02.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load_non-empty_index/load_non-empty_index.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/type_promotion_0/type_promotion_0.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/misc/flushtest/flushtest.5.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-closed/query-issue208/query-issue208.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-closed/query-issue236/query-issue236.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/upsert/issue1587-foreignDataType/issue1587-foreignDataType.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/flushtest/flushtest.5.ddl.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/range-hints/order-by-exception_02/order-by-exception_02.4.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/types/any-object/any-object.3.adm
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comparison/binary_null/binary_null.2.ast
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comparison/binary_null/binary_null.3.ast
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comp

Change in asterixdb[master]: Add Default Dataverse and AnyObject Datatype

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Add Default Dataverse and AnyObject Datatype
..


Patch Set 6:

(5 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1174/6/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
File 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java:

Line 1186: MetadataBuiltinEntities.DEFAULT_DATAVERSE_NAME + 
" dataverse can't be dropped");
> That's right, but we need to have a way to clean it up. Please create an is
Done


https://asterix-gerrit.ics.uci.edu/#/c/1174/6/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/SmokeParserTest.java
File 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/SmokeParserTest.java:

Line 51: private static final String PATH_ACTUAL = "target" + 
File.separator + "parserts" + SEPARATOR;
> Can we cleanup the .gitignore now?
Done


https://asterix-gerrit.ics.uci.edu/#/c/1174/6/asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.2.update.aql
File 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.2.update.aql:

Line 18:  */
> Do we need this file?
I don't think we do.


https://asterix-gerrit.ics.uci.edu/#/c/1174/6/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java
File 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java:

Line 326: throw new MetadataException("Unable to instantiate 
builtin Adapter", e);
> Do we have a test for this?
We don't. I added a JIRA for this


Line 338: throw new MetadataException("Unable to instantiate 
builtin Merge Policy Factory", e);
> Do we have a test for this?
We don't. I added a JIRA for this


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1174
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ie02c74c8b005dce73888afd4dfe19ede2c60a8e8
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Add Default Dataverse and AnyObject Datatype

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1174

to look at the new patch set (#8).

Change subject: Add Default Dataverse and AnyObject Datatype
..

Add Default Dataverse and AnyObject Datatype

This change adds "Default" dataverse and
Metadata.AnyObject type which is the completely
open record type. In addition, this change cleansup between
subsequent test cases lowering the chance that a failed
test causes the following tests to fail. In addition, it
prints out the list of tests which left some leftover.

Change-Id: Ie02c74c8b005dce73888afd4dfe19ede2c60a8e8
---
M .gitignore
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/TypeTranslator.java
M asterixdb/asterix-algebra/src/main/javacc/AQLPlus.jj
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplicationEntryPoint.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/metadata/MetadataTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/optimizer/OptimizerTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/ExecutionTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/SqlppExecutionTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/OptimizerParserTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/ParserTestExecutor.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/RuntimeParserTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/SmokeParserTest.java
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta01/meta01.1.adm
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_dataverse/metadata_dataverse.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/cross-dataverse/drop-dataverse/drop-dataverse.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/cross-dataverse/drop-type-used-elsewhere/drop-type-used-elsewhere.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/cross-dataverse/query-dataset-with-foreign-type/query-dataset-with-foreign-type.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/load-with-index-open_02/load-with-index-open_02.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/query-issue382/query-issue382.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/escapes-err-1/escapes-err-1.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/escapes01/escapes01.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/escapes02/escapes02.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load_non-empty_index/load_non-empty_index.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/type_promotion_0/type_promotion_0.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/misc/flushtest/flushtest.5.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-closed/query-issue208/query-issue208.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-closed/query-issue236/query-issue236.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/upsert/issue1587-foreignDataType/issue1587-foreignDataType.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/flushtest/flushtest.5.ddl.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/range-hints/order-by-exception_02/order-by-exception_02.4.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.2.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/types/any-object/any-object.2.adm
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comparison/binary_null/binary_null.2.ast
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comparison/binary_null/binary_null.3.ast
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comparison/int16_null/int16_null.2.ast
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comparison/int16_null/int16_null.3.ast
R 
asterixdb/asterix-app/src/test/resources/runtimet

Change in asterixdb[master]: Enhanced Insert AQL

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Enhanced Insert AQL
..


Patch Set 10:

(6 comments)

Comments added.

Beside the comments, I am worried about the implications of the use of the 
extension operator. I don't fully understand it but would be nice to get an 
explanation on how it changes things.

https://asterix-gerrit.ics.uci.edu/#/c/1150/10/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/extension/IExtensionStatement.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/extension/IExtensionStatement.java:

Line 42:  * Called when the {@code IQueryTranslator} encounters an 
extension statement.
Fix the javadoc?


Line 52: void handle(IStatementExecutor statementExecutor, 
AqlMetadataProvider metadataProvider,
Should we change this to only pass IStatementExecutor and create getters for 
the other passed parameters?


https://asterix-gerrit.ics.uci.edu/#/c/1150/10/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/CommitOperator.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/CommitOperator.java:

Line 63: 
setVar is not used by core. add a comment that it is used by extensions? 
Thoughts @Till?


https://asterix-gerrit.ics.uci.edu/#/c/1150/10/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/UpsertCommitRuntime.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/UpsertCommitRuntime.java:

Line 35: resourcePartition, true);
mm, wouldn't this cause a problem if someone attempts Upsert with returning 
new values?


Add a test case?


https://asterix-gerrit.ics.uci.edu/#/c/1150/10/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java:

Line 96: if (op0.getOperatorTag() != 
LogicalOperatorTag.EXTENSION_OPERATOR
I think that all rewrite rules should be extension-agnostic. If an extension 
needs a rewrite rule behavior to change, it should override it.


https://asterix-gerrit.ics.uci.edu/#/c/1150/10/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java:

Line 423: ILogicalOperator leafOperator = new ExtensionOperator(new 
CommitOperator(true));
I just noticed this ExtensionOperator which seems to have been here for a long 
time. why would we want to use it with a commit operator instead of a sink?
I think this could have some hidden effects!


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1150
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I65789d2a861d15232dd29156a6987d0635ec6c94
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Carey 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Add Default Dataverse and AnyObject Datatype

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Add Default Dataverse and AnyObject Datatype
..


Patch Set 8:

Till,
The deleted test files were never compared before because they are for failure 
test cases. what used to happen is that the failure test case will throw an 
exception which would get swallowed and the test would move to the next test 
case.

Even after the tests' fixes, we still don't compare them but at least we don't 
have them. not sure if there is a way to get the output in case of failures!!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1174
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ie02c74c8b005dce73888afd4dfe19ede2c60a8e8
Gerrit-PatchSet: 8
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Add Default Dataverse and AnyObject Datatype

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Add Default Dataverse and AnyObject Datatype
..


Patch Set 8:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1174/8/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/ParserTestExecutor.java
File 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/ParserTestExecutor.java:

Line 104: if (e instanceof ComparisonException) {
> This instanceof check is a relatively bad hack that I used to document a pr
Agree. created an issue for this but also willing to do it in the current 
change if you think that's better.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1174
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ie02c74c8b005dce73888afd4dfe19ede2c60a8e8
Gerrit-PatchSet: 8
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Add Default Dataverse and AnyObject Datatype

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1174

to look at the new patch set (#9).

Change subject: Add Default Dataverse and AnyObject Datatype
..

Add Default Dataverse and AnyObject Datatype

This change adds "Default" dataverse and
Metadata.AnyObject type which is the completely
open record type. In addition, this change cleansup between
subsequent test cases lowering the chance that a failed
test causes the following tests to fail. In addition, it
prints out the list of tests which left some leftover.

Change-Id: Ie02c74c8b005dce73888afd4dfe19ede2c60a8e8
---
M .gitignore
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/TypeTranslator.java
M asterixdb/asterix-algebra/src/main/javacc/AQLPlus.jj
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplicationEntryPoint.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/metadata/MetadataTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/optimizer/OptimizerTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/ExecutionTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/SqlppExecutionTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/OptimizerParserTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/ParserTestExecutor.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/RuntimeParserTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/SmokeParserTest.java
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta01/meta01.1.adm
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_dataverse/metadata_dataverse.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/cross-dataverse/drop-dataverse/drop-dataverse.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/cross-dataverse/drop-type-used-elsewhere/drop-type-used-elsewhere.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/cross-dataverse/query-dataset-with-foreign-type/query-dataset-with-foreign-type.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/load-with-index-open_02/load-with-index-open_02.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/query-issue382/query-issue382.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/escapes-err-1/escapes-err-1.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/escapes01/escapes01.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/escapes02/escapes02.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load_non-empty_index/load_non-empty_index.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/type_promotion_0/type_promotion_0.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/misc/flushtest/flushtest.5.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-closed/query-issue208/query-issue208.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-closed/query-issue236/query-issue236.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/upsert/issue1587-foreignDataType/issue1587-foreignDataType.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/flushtest/flushtest.5.ddl.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/range-hints/order-by-exception_02/order-by-exception_02.4.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.2.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/types/any-object/any-object.2.adm
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comparison/binary_null/binary_null.2.ast
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comparison/binary_null/binary_null.3.ast
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comparison/int16_null/int16_null.2.ast
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_

Change in asterixdb[master]: Add Default Dataverse and AnyObject Datatype

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Add Default Dataverse and AnyObject Datatype
..


Patch Set 9:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1174/9/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/ParserTestExecutor.java
File 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/ParserTestExecutor.java:

Line 104: if (e instanceof ComparisonException) {
> Did you file an issue for this?
Yes I did


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1174
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ie02c74c8b005dce73888afd4dfe19ede2c60a8e8
Gerrit-PatchSet: 9
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Add Default Dataverse and AnyObject Datatype

[abdullah alamoudi (Code Review)]

abdullah alamoudi has submitted this change and it was merged.

Change subject: Add Default Dataverse and AnyObject Datatype
..


Add Default Dataverse and AnyObject Datatype

This change adds "Default" dataverse and
Metadata.AnyObject type which is the completely
open record type. In addition, this change cleansup between
subsequent test cases lowering the chance that a failed
test causes the following tests to fail. In addition, it
prints out the list of tests which left some leftover.

Change-Id: Ie02c74c8b005dce73888afd4dfe19ede2c60a8e8
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1174
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Till Westmann 
---
M .gitignore
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/TypeTranslator.java
M asterixdb/asterix-algebra/src/main/javacc/AQLPlus.jj
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplicationEntryPoint.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/metadata/MetadataTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/optimizer/OptimizerTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/ExecutionTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/SqlppExecutionTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/OptimizerParserTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/ParserTestExecutor.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/RuntimeParserTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/SmokeParserTest.java
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta01/meta01.1.adm
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm
M 
asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_dataverse/metadata_dataverse.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/cross-dataverse/drop-dataverse/drop-dataverse.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/cross-dataverse/drop-type-used-elsewhere/drop-type-used-elsewhere.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/cross-dataverse/query-dataset-with-foreign-type/query-dataset-with-foreign-type.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/load-with-index-open_02/load-with-index-open_02.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/query-issue382/query-issue382.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/escapes-err-1/escapes-err-1.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/escapes01/escapes01.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/escapes02/escapes02.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load_non-empty_index/load_non-empty_index.3.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/type_promotion_0/type_promotion_0.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/misc/flushtest/flushtest.5.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-closed/query-issue208/query-issue208.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/open-closed/query-issue236/query-issue236.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/types/any-object/any-object.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/upsert/issue1587-foreignDataType/issue1587-foreignDataType.4.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/flushtest/flushtest.5.ddl.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/range-hints/order-by-exception_02/order-by-exception_02.4.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/any-object/any-object.2.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/types/any-object/any-object.2.adm
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comparison/binary_null/binary_null.2.ast
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comparison/binary_null/binary_null.3.ast
D 
asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/comparison/int16_null/int16_null.2.ast
D 

Change in asterixdb[master]: Fix Failure Handling on Tasks Start

[abdullah alamoudi (Code Review)]

abdullah alamoudi has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1197

Change subject: Fix Failure Handling on Tasks Start
..

Fix Failure Handling on Tasks Start

Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
---
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/util/JavaSerializationUtils.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/JobScheduler.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/base/INodeController.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/ipc/CCNCFunctions.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/ipc/NodeControllerRemoteProxy.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/NodeControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/AbortTasksWork.java
A 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartJobWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/AbstractMultiNCIntegrationTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/JobFailureTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/ExceptionOnCreatePushRuntimeOperatorDescriptor.java
13 files changed, 523 insertions(+), 199 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/97/1197/1

diff --git 
a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/util/JavaSerializationUtils.java
 
b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/util/JavaSerializationUtils.java
index a92e700..d7936e0 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/util/JavaSerializationUtils.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/util/JavaSerializationUtils.java
@@ -81,8 +81,8 @@
 private static class ClassLoaderObjectInputStream extends 
ObjectInputStream {
 private ClassLoader classLoader;
 
-protected ClassLoaderObjectInputStream(InputStream in, ClassLoader 
classLoader)
-throws IOException, SecurityException {
+protected ClassLoaderObjectInputStream(InputStream in, ClassLoader 
classLoader) throws IOException,
+SecurityException {
 super(in);
 this.classLoader = classLoader;
 }
diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/JobScheduler.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/JobScheduler.java
index b377b1a..1541190 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/JobScheduler.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/JobScheduler.java
@@ -31,9 +31,6 @@
 import java.util.logging.Level;
 import java.util.logging.Logger;
 
-import org.json.JSONException;
-import org.json.JSONObject;
-
 import org.apache.hyracks.api.comm.NetworkAddress;
 import org.apache.hyracks.api.constraints.Constraint;
 import 
org.apache.hyracks.api.constraints.expressions.LValueConstraintExpression;
@@ -66,6 +63,8 @@
 import org.apache.hyracks.control.cc.work.JobCleanupWork;
 import org.apache.hyracks.control.common.job.PartitionState;
 import org.apache.hyracks.control.common.job.TaskAttemptDescriptor;
+import org.json.JSONException;
+import org.json.JSONObject;
 
 public class JobScheduler {
 private static final Logger LOGGER = 
Logger.getLogger(JobScheduler.class.getName());
@@ -160,7 +159,7 @@
 }
 
 private void startRunnableActivityClusters() throws HyracksException {
-Set taskClusterRoots = new HashSet();
+Set taskClusterRoots = new HashSet<>();
 findRunnableTaskClusterRoots(taskClusterRoots, 
jobRun.getActivityClusterGraph().getActivityClusterMap()
 .values());
 if (LOGGER.isLoggable(Level.FINE)) {
@@ -312,1

Change in asterixdb[master]: Fix Failure Handling on Tasks Start

[abdullah alamoudi (Code Review)]

abdullah alamoudi has uploaded a new patch set (#2).

Change subject: Fix Failure Handling on Tasks Start
..

Fix Failure Handling on Tasks Start

Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/JobScheduler.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/base/INodeController.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/ipc/CCNCFunctions.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/ipc/NodeControllerRemoteProxy.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/NodeControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/AbortTasksWork.java
A 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartJobWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/AbstractMultiNCIntegrationTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/JobFailureTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/ExceptionOnCreatePushRuntimeOperatorDescriptor.java
12 files changed, 521 insertions(+), 197 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/97/1197/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1197
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 

Change in asterixdb[master]: Add Record To Map Methods

[abdullah alamoudi (Code Review)]

abdullah alamoudi has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1198

Change subject: Add Record To Map Methods
..

Add Record To Map Methods

Change-Id: I525b99fa56251579e3ba42d79dd578c625fcab67
---
M 
asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/LangRecordParseUtil.java
1 file changed, 51 insertions(+), 7 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/98/1198/1

diff --git 
a/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/LangRecordParseUtil.java
 
b/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/LangRecordParseUtil.java
index c46b0be..00d8683 100644
--- 
a/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/LangRecordParseUtil.java
+++ 
b/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/LangRecordParseUtil.java
@@ -22,6 +22,7 @@
 import java.util.Collections;
 import java.util.HashSet;
 import java.util.List;
+import java.util.Map;
 
 import org.apache.asterix.builders.OrderedListBuilder;
 import org.apache.asterix.builders.RecordBuilder;
@@ -38,9 +39,14 @@
 import org.apache.asterix.om.base.AInt64;
 import org.apache.asterix.om.base.AMutableString;
 import org.apache.asterix.om.base.ANull;
+import org.apache.asterix.om.base.AOrderedList;
+import org.apache.asterix.om.base.ARecord;
 import org.apache.asterix.om.base.AString;
+import org.apache.asterix.om.base.IACursor;
+import org.apache.asterix.om.base.IAObject;
 import org.apache.asterix.om.types.ARecordType;
 import org.apache.asterix.om.types.BuiltinType;
+import org.apache.hyracks.algebricks.common.exceptions.AlgebricksException;
 import org.apache.hyracks.algebricks.common.utils.Pair;
 import org.apache.hyracks.api.dataflow.value.ISerializerDeserializer;
 import org.apache.hyracks.api.exceptions.HyracksDataException;
@@ -78,16 +84,14 @@
 break;
 default:
 throw new HyracksDataException(ErrorCode.ASTERIX, 
ErrorCode.ERROR_PARSE_ERROR,
-NOT_ALLOWED_EXPRESSIONS_ERROR_MESSAGE,
-new Serializable[] { 
Expression.Kind.LITERAL_EXPRESSION.toString(),
-
Expression.Kind.RECORD_CONSTRUCTOR_EXPRESSION.toString(),
+NOT_ALLOWED_EXPRESSIONS_ERROR_MESSAGE, new 
Serializable[] { Expression.Kind.LITERAL_EXPRESSION
+.toString(), 
Expression.Kind.RECORD_CONSTRUCTOR_EXPRESSION.toString(),
 
Expression.Kind.LIST_CONSTRUCTOR_EXPRESSION.toString() });
 }
 }
 
 public static void parseRecord(RecordConstructor recordValue, 
ArrayBackedValueStorage serialized, boolean tagged,
-List> defaults)
-throws HyracksDataException {
+List> defaults) throws HyracksDataException {
 AMutableString fieldNameString = new AMutableString(null);
 ArrayBackedValueStorage fieldName = new ArrayBackedValueStorage();
 ArrayBackedValueStorage fieldValue = new ArrayBackedValueStorage();
@@ -102,8 +106,8 @@
 // get key
 
fieldNameString.setValue(exprToStringLiteral(fb.getLeftExpr()).getStringValue());
 if (!fieldNames.add(fieldNameString.getStringValue())) {
-throw new HyracksDataException(
-"Field " + fieldNameString.getStringValue() + " was 
specified multiple times");
+throw new HyracksDataException("Field " + 
fieldNameString.getStringValue()
++ " was specified multiple times");
 }
 stringSerde.serialize(fieldNameString, fieldName.getDataOutput());
 // get value
@@ -187,4 +191,44 @@
 "Unknown Literal Type %1$s", value.getLiteralType());
 }
 }
+
+public static void recordToMap(Map map, ARecord record)
+throws AlgebricksException {
+String[] keys = record.getType().getFieldNames();
+for (int i = 0; i < keys.length; i++) {
+String key = keys[i];
+String value = aObjToString(record.getValueByPos(i));
+map.put(key, value);
+}
+}
+
+public static String aObjToString(IAObject aObj) throws 
AlgebricksException {
+switch (aObj.getType().getTypeTag()) {
+case DOUBLE:
+return Double.toString(((ADouble) aObj).getDoubleValue());
+case INT64:
+return Long.toString(((AInt64) aObj).getLongValue());
+case ORDEREDLIST:
+return aOrderedListToString((AOrderedList) aObj);
+case STRING:
+return ((AString) aObj).getStringValue();
+default:
+throw new AlgebricksException("value of type " + 
aObj.getType() + " is not supported yet")

Change in asterixdb[master]: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1458 Fix Failure Han...

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1197

to look at the new patch set (#3).

Change subject: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1458 Fix Failure 
Handling
..

ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1458 Fix Failure Handling

Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/JobScheduler.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/PartitionConstraintSolver.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/TaskFailureWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/base/INodeController.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/ipc/CCNCFunctions.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/ipc/NodeControllerRemoteProxy.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/work/AbstractWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/NodeControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/AbortTasksWork.java
A 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartJobWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/AbstractMultiNCIntegrationTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/JobFailureTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/ExceptionOnCreatePushRuntimeOperatorDescriptor.java
M 
hyracks-fullstack/hyracks/hyracks-ipc/src/main/java/org/apache/hyracks/ipc/api/IPayloadSerializerDeserializer.java
M 
hyracks-fullstack/hyracks/hyracks-ipc/src/main/java/org/apache/hyracks/ipc/impl/JavaSerializationBasedPayloadSerializerDeserializer.java
17 files changed, 689 insertions(+), 355 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/97/1197/3
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1197
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 

Change in asterixdb[master]: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Failure Han...

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1197

to look at the new patch set (#4).

Change subject: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Failure 
Handling
..

ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Failure Handling

Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/JobScheduler.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/PartitionConstraintSolver.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/TaskFailureWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/base/INodeController.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/ipc/CCNCFunctions.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/ipc/NodeControllerRemoteProxy.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/work/AbstractWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/NodeControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/AbortTasksWork.java
A 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartJobWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/AbstractMultiNCIntegrationTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/JobFailureTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/ExceptionOnCreatePushRuntimeOperatorDescriptor.java
M 
hyracks-fullstack/hyracks/hyracks-ipc/src/main/java/org/apache/hyracks/ipc/api/IPayloadSerializerDeserializer.java
M 
hyracks-fullstack/hyracks/hyracks-ipc/src/main/java/org/apache/hyracks/ipc/impl/JavaSerializationBasedPayloadSerializerDeserializer.java
17 files changed, 689 insertions(+), 355 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/97/1197/4
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1197
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 

Change in asterixdb[master]: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failur...

[abdullah alamoudi (Code Review)]

Hello Till Westmann, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1197

to look at the new patch set (#5).

Change subject: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failure 
Handling
..

ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failure Handling

Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/JobScheduler.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/TaskFailureWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/AbortTasksWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/AbstractMultiNCIntegrationTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/JobFailureTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/ExceptionOnCreatePushRuntimeOperatorDescriptor.java
8 files changed, 329 insertions(+), 83 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/97/1197/5
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1197
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 

Change in asterixdb[master]: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failur...

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failure 
Handling
..


Patch Set 4:

submitted a new patch and I think I have addressed all your concerns Till.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1197
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failur...

[abdullah alamoudi (Code Review)]

Hello Till Westmann, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1197

to look at the new patch set (#6).

Change subject: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failure 
Handling
..

ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failure Handling

Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/JobScheduler.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/TaskFailureWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/AbortTasksWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/AbstractMultiNCIntegrationTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/JobFailureTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/ExceptionOnCreatePushRuntimeOperatorDescriptor.java
8 files changed, 306 insertions(+), 60 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/97/1197/6
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1197
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 

Change in asterixdb[master]: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failur...

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failure 
Handling
..


Patch Set 6:

(7 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1197/6/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
File 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java:

Line 350: taskInitializationThreads.interrupt();
> why do we need to interrupt the work queue thread?  Maybe I misunderstand t
It is because sometimes the task initialization blocks and we need to notify it 
somehow to return. otherwise, the nc hangs.


https://asterix-gerrit.ics.uci.edu/#/c/1197/6/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/AbortTasksWork.java
File 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/AbortTasksWork.java:

Line 53: }
> +1
-1


Line 53: }
> Why do this in the constructor? Isn't that risky?
because if the task initialization is blocking the abort will never get a 
chance to run. this way, we ensure the task initialization aborts since the 
constructor is called from another thread.


https://asterix-gerrit.ics.uci.edu/#/c/1197/6/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java
File 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java:

Line 110: 
joblet.setTaskInitializationThreads(Thread.currentThread());
> is Thread.currentThread() the single work queue thread?
mmm, seems like it is


Line 184: LOGGER.log(Level.SEVERE, "Failure acquiring a joblet. 
Task has most likely been aborted", e);
> Is an aborted job always something SEVERE, or can it be an expected outcome
I believe it is always severe. should not happen unless something is seriously 
wrong.


Line 205: throw new NullPointerException("Joblet was not found. 
This job was most likely aborted!!");
> what else could this be other than aborted?   anything other than a bug?
It is either an abort or a bug!


Line 263: {
> why newline?
stupid formatter


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1197
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failur...

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failure 
Handling
..


Patch Set 6:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1197/6/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
File 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java:

Line 350: taskInitializationThreads.interrupt();
> Interrupting the work queue thread seems dangerous. Could this interrupt an
Mmmm, let's have a discussion about this with at least Mike, Mike, and Till.
Also, we should make it clear to everyone what calls can block and what calls 
can't.

In this change, I added a test case which blocks on the startTask.run() and 
when another task of the same job fails, the blocking one gets interrupted and 
the work queue resume. The important thing is that the thread.interrupt() 
doesn't get called.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1197
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failur...

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failure 
Handling
..


Patch Set 6:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1197/6/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
File 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java:

Line 350: taskInitializationThreads.interrupt();
> IMO, anything (e.g., xxxWork) put into the work queue shouldn't be blocked,
Agree. let's just make that clear to everyone.

Will fix this.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1197
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: move SessionConfig to translator

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: move SessionConfig to translator
..


Patch Set 1: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1224
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I06d85aa361bb5d50f885311aad2545514d8a94b7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Till Westmann 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failur...

[abdullah alamoudi (Code Review)]

Hello Michael Blow, Till Westmann, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1197

to look at the new patch set (#7).

Change subject: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failure 
Handling
..

ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failure Handling

Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/JobScheduler.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/TaskFailureWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/AbortTasksWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/AbstractMultiNCIntegrationTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/JobFailureTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/ExceptionOnCreatePushRuntimeOperatorDescriptor.java
8 files changed, 279 insertions(+), 65 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/97/1197/7
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1197
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
Gerrit-PatchSet: 7
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 

Change in asterixdb[master]: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failur...

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failure 
Handling
..


Patch Set 6:

(5 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1197/6/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/JobScheduler.java
File 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/JobScheduler.java:

Line 633:  *- Activity Cluster that owns this Task
> fix comment as well
Done


https://asterix-gerrit.ics.uci.edu/#/c/1197/6/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
File 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java:

Line 105: private final AtomicBoolean aborted;
> why Atomic?  volatile boolean would work, right?
Done


Line 106: private volatile Thread taskInitializationThreads;
> Why is the "threads" instead of "thread"?
Done


https://asterix-gerrit.ics.uci.edu/#/c/1197/6/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/AbortTasksWork.java
File 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/AbortTasksWork.java:

Line 53: }
> So I think that we just agreed, on the discussion on Joblet.java, that the 
I am leaving this here. what it does is just mark the job as aborted so if some 
tasks didn't start yet, they will not start.


https://asterix-gerrit.ics.uci.edu/#/c/1197/6/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/AbstractMultiNCIntegrationTest.java
File 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/AbstractMultiNCIntegrationTest.java:

Line 73: outputFiles = new ArrayList();;
> Just remove the 2nd semicolon?
Done


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1197
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failur...

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failure 
Handling
..


Patch Set 7:

(2 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1197/7/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
File 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java:

Line 343: return aborted;
> why reading is not synchronized but writing is synchronized?
the abort call will be done from the ipci.
so one of them synching should be enough since the variable is volatile. since 
abort is expected to be called waaay less than isAborted(), I went with that 
one.


https://asterix-gerrit.ics.uci.edu/#/c/1197/7/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/AbortTasksWork.java
File 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/AbortTasksWork.java:

Line 52: ji.abort();
> Sorry for annoying... 
no problem :)

This is just setting a flag so if some tasks didn't start, they will check by 
calling isAborted() and then will simply not start since the job has been 
aborted.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1197
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
Gerrit-PatchSet: 7
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failur...

[abdullah alamoudi (Code Review)]

Hello Michael Blow, Till Westmann, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1197

to look at the new patch set (#8).

Change subject: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failure 
Handling
..

ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failure Handling

Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/JobScheduler.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/TaskFailureWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/AbortTasksWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/AbstractMultiNCIntegrationTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/JobFailureTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/ExceptionOnCreatePushRuntimeOperatorDescriptor.java
8 files changed, 259 insertions(+), 65 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/97/1197/8
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1197
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
Gerrit-PatchSet: 8
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 

Change in asterixdb[master]: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failur...

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failure 
Handling
..


Patch Set 8:

(2 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1197/8/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java
File 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java:

Line 103: Task task = null;
> revert joblet and task declarations to where they used to be?
can't as we need to access them in the catch block.


Line 179: LOGGER.log(Level.WARNING, "Failure acquiring a 
joblet. Task has most likely been aborted", e);
> This if block is no longer needed?
theoretically, it is not needed because of the ordering of the work queues. but 
I would like to keep it just because I feel we didn't cover all the possible 
scenarios


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1197
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
Gerrit-PatchSet: 8
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failur...

[abdullah alamoudi (Code Review)]

Hello Michael Blow, Till Westmann, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1197

to look at the new patch set (#9).

Change subject: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failure 
Handling
..

ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failure Handling

Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/JobScheduler.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/TaskFailureWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/AbortTasksWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/AbstractMultiNCIntegrationTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/JobFailureTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/ExceptionOnCreatePushRuntimeOperatorDescriptor.java
8 files changed, 254 insertions(+), 66 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/97/1197/9
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1197
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
Gerrit-PatchSet: 9
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 

Change in asterixdb[master]: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failur...

[abdullah alamoudi (Code Review)]

abdullah alamoudi has submitted this change and it was merged.

Change subject: ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failure 
Handling
..


ASTERIXDB-1642,ASTERIXDB-1657,ASTERIXDB-1658 Fix Task Failure Handling

Change-Id: I2ec2c798b704ca426d5937f22e6d2bd394a9095a
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1197
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Yingyi Bu 
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/JobScheduler.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/TaskFailureWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/AbortTasksWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/AbstractMultiNCIntegrationTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/JobFailureTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/ExceptionOnCreatePushRuntimeOperatorDescriptor.java
8 files changed, 254 insertions(+), 66 deletions(-)

Approvals:
  Yingyi Bu: Looks good to me, approved
  Jenkins: Verified; No violations found; Verified



diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/JobScheduler.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/JobScheduler.java
index b377b1a..ab026eb 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/JobScheduler.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/JobScheduler.java
@@ -626,15 +626,10 @@
 
 /**
  * Indicates that a single task attempt has encountered a failure.
- *
- * @param ta
- *- Failed Task Attempt
- * @param ac
- *- Activity Cluster that owns this Task
- * @param details
- *- Cause of the failure
+ * @param ta Failed Task Attempt
+ * @param exceptions exeptions thrown during the failure
  */
-public void notifyTaskFailure(TaskAttempt ta, ActivityCluster ac, 
List exceptions) {
+public void notifyTaskFailure(TaskAttempt ta, List exceptions) {
 try {
 LOGGER.fine("Received failure notification for TaskAttempt " + 
ta.getTaskAttemptId());
 TaskAttemptId taId = ta.getTaskAttemptId();
diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/TaskFailureWork.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/TaskFailureWork.java
index 6db3700..8bca4e7 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/TaskFailureWork.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/TaskFailureWork.java
@@ -21,7 +21,6 @@
 import java.util.List;
 
 import org.apache.hyracks.api.dataflow.TaskAttemptId;
-import org.apache.hyracks.api.job.ActivityCluster;
 import org.apache.hyracks.api.job.JobId;
 import org.apache.hyracks.control.cc.ClusterControllerService;
 import org.apache.hyracks.control.cc.job.JobRun;
@@ -40,8 +39,7 @@
 protected void performEvent(TaskAttempt ta) {
 JobRun run = ccs.getActiveRunMap().get(jobId);
 ccs.getDatasetDirectoryService().reportJobFailure(jobId, exceptions);
-ActivityCluster ac = 
ta.getTask().getTaskCluster().getActivityCluster();
-run.getScheduler().notifyTaskFailure(ta, ac, exceptions);
+run.getScheduler().notifyTaskFailure(ta, exceptions);
 }
 
 @Override
diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
index 0d9ff5d..c91ebd4 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
@

Change in asterixdb[master]: Fix potential hanging in op.initialize().

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Fix potential hanging in op.initialize().
..


Patch Set 3:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1243/3/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/ExceptionOnCreatePushRuntimeOperatorDescriptor.java
File 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/ExceptionOnCreatePushRuntimeOperatorDescriptor.java:

Line 134: // should be executed.
let's fix the comment and then:
instead of using atomic integers, lets' use mutable static  int[]s where the 
size = the number of partitions. we can set the int[]s before creating the jobs 
and then starts the job, each task will then work on its own 
createPushRuntime[partition], openCloseCounter[partition], and 
initializeCounter[partition].

then in succeed(), we can check that the values in all of the counters == 0. 
that is more precise and would tell us if exactly what we expect is happening.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1243
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I701b271bc6dc78e67274fa845dec013756843a70
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Yingyi Bu 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Fix potential hanging in op.initialize().

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Fix potential hanging in op.initialize().
..


Patch Set 3: Code-Review-1

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1243
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I701b271bc6dc78e67274fa845dec013756843a70
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Yingyi Bu 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Enhanced Insert AQL

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Enhanced Insert AQL
..


Patch Set 15: Code-Review+1

I am good with this change. Till? Yingyi?

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1150
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I65789d2a861d15232dd29156a6987d0635ec6c94
Gerrit-PatchSet: 15
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Michael Carey 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Log Actual vs Expected in SuperActivity ReWriting Test

[abdullah alamoudi (Code Review)]

abdullah alamoudi has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1255

Change subject: Log Actual vs Expected in SuperActivity ReWriting Test
..

Log Actual vs Expected in SuperActivity ReWriting Test

Change-Id: Ie8aa0082ca916775c64baf5981b0384dd4892390
---
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/rewriting/SuperActivityRewritingTest.java
1 file changed, 3 insertions(+), 2 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/55/1255/1

diff --git 
a/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/rewriting/SuperActivityRewritingTest.java
 
b/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/rewriting/SuperActivityRewritingTest.java
index bba18b3..32a82b6 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/rewriting/SuperActivityRewritingTest.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/rewriting/SuperActivityRewritingTest.java
@@ -122,7 +122,7 @@
 @Override
 public IOperatorNodePushable createPushRuntime(IHyracksTaskContext ctx,
 IRecordDescriptorProvider recordDescProvider, int partition, 
int nPartitions)
-throws HyracksDataException {
+throws HyracksDataException {
 return new IOperatorNodePushable() {
 private Set threads = new HashSet();
 
@@ -134,7 +134,8 @@
 @Override
 public void deinitialize() throws HyracksDataException {
 if (threads.size() != inputArity) {
-throw new HyracksDataException("The number of worker 
threads is not as expected");
+throw new HyracksDataException("The number of worker 
threads (" + threads.size()
++ ") is not as expected(" + inputArity + ")");
 }
 }
 

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1255
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie8aa0082ca916775c64baf5981b0384dd4892390
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 

Change in asterixdb[master]: Log Actual vs Expected in SuperActivity ReWriting Test

[abdullah alamoudi (Code Review)]

abdullah alamoudi has abandoned this change.

Change subject: Log Actual vs Expected in SuperActivity ReWriting Test
..


Abandoned

test is broken

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1255
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: Ie8aa0082ca916775c64baf5981b0384dd4892390
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Yingyi Bu 

Change in asterixdb[master]: Enhanced Insert AQL

[abdullah alamoudi (Code Review)]

Hello Michael Carey, Michael Blow, Steven Jacobs, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1150

to look at the new patch set (#17).

Change subject: Enhanced Insert AQL
..

Enhanced Insert AQL

The optional "as Variable" provides a variable binding for the inserted records
The optional "returning Query" allows users to run simple
queries/functions on the records returned by the insert, and can refer
to the variable bound in "as Variable"

Allow commits to be non-sink operators (contnue job pipeline after commit)

Additionally, this change makes small modifications to
the extension code to prepare for the BAD extension

Change-Id: I65789d2a861d15232dd29156a6987d0635ec6c94
---
M asterixdb/asterix-active/pom.xml
M 
asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEntityEventsListener.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/extension/IExtensionStatement.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/CommitOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/CommitPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/CommitRuntime.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/CommitRuntimeFactory.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/UpsertCommitRuntime.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceAutogenerateIDRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceRapidFrameFlushProjectAssignRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceStaticTypeCastForInsertRule.java
R 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SetupCommitExtensionOpRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SweepIllegalNonfunctionalFunctions.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/UnnestToDataScanRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineAllNtsInSubplanVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineLeftNtsInSubplanJoinFlatteningVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/SubplanSpecialFlatteningCheckVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/CompiledStatements.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/servlet/FeedServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/optimizer/OptimizerTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/ExecutionTestUtil.java
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/insert-return-custom-result.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/insert-return-custom-result.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/insert-return-records/insert-return-records.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/insert-return-records/insert-return-records.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/insert-returning-fieldname/insert-returning-fieldname.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/insert-returning-fieldname/insert-returning-fieldname.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/upsert-return-custom-result/upsert-return-custom-result.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/upsert-return-custom-result/upsert-return-custom-result.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/dml/insert-return-records/insert-return-records.1.adm
A 
asterixdb/asterix-app/

Change in asterixdb[master]: ASTERIXDB-1693: Fix Default dataverse types referencing

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: ASTERIXDB-1693: Fix Default dataverse types referencing
..


Patch Set 1: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1284
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Iae7ed8a2de345531345e4df2f9e06aa0d7680ebc
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Move Twitter Firehose Datasource to Test Source Folder

[abdullah alamoudi (Code Review)]

abdullah alamoudi has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1290

Change subject: Move Twitter Firehose Datasource to Test Source Folder
..

Move Twitter Firehose Datasource to Test Source Folder

Change-Id: Iefe2130707012b8ce60f5dfac96635a1a515a076
---
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_07/feeds_07.1.ddl.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.1.ddl.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_09/feeds_09.1.ddl.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feeds_07/feeds_07.1.ddl.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feeds_08/feeds_08.1.ddl.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feeds_09/feeds_09.1.ddl.sqlpp
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/DatasourceFactoryProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
R 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/generator/DataGenerator.java
R 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/generator/TweetGenerator.java
R 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/input/stream/TwitterFirehoseInputStream.java
R 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/input/stream/TwitterFirehoseStreamFactory.java
M 
asterixdb/asterix-tools/src/main/java/org/apache/asterix/tools/external/data/DataGeneratorForSpatialIndexEvaluation.java
13 files changed, 25 insertions(+), 27 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/90/1290/1

diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_07/feeds_07.1.ddl.aql
 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_07/feeds_07.1.ddl.aql
index 70ea8d6..f3f8f7d 100644
--- 
a/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_07/feeds_07.1.ddl.aql
+++ 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_07/feeds_07.1.ddl.aql
@@ -54,12 +54,12 @@
 primary key id;
 
 create feed  SyntheticTweetFeed
-using twitter_firehose(
+using stream(
 ("duration"="5"),
 ("tps"="50"),
 ("type-name"="TweetMessageType"),
 ("format"="adm"),
-("reader-stream"="twitter_firehose"),
+("stream-source"="org.apache.asterix.external.input.stream.TwitterFirehoseStreamFactory"),
 ("tput-duration"="5"),
 ("dataverse-dataset"="feeds:SyntheticTweets"),
 ("mode"="controlled"));
\ No newline at end of file
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.1.ddl.aql
 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.1.ddl.aql
index 658487b..c339563 100644
--- 
a/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.1.ddl.aql
+++ 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.1.ddl.aql
@@ -58,8 +58,9 @@
 create index ngram_index on SyntheticTweets(message_text) type ngram(3);
 
 create feed  SyntheticTweetFeed
-using twitter_firehose
+using stream
 (("duration"="5"),
+("stream-source"="org.apache.asterix.external.input.stream.TwitterFirehoseStreamFactory"),
 ("tps"="50"),
 ("type-name"="TweetMessageType"),
 ("tput-duration"="5"),
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_09/feeds_09.1.ddl.aql
 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_09/feeds_09.1.ddl.aql
index 6714850..59385c4 100644
--- 
a/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_09/feeds_09.1.ddl.aql
+++ 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_09/feeds_09.1.ddl.aql
@@ -56,12 +56,12 @@
 create index message_text on SyntheticTweets(message_text) type btree;
 
 create feed  SyntheticTweetFeed
-using twitter_firehose
+using stream
 (("duration"="5"),
 ("tps"="50"),
 ("tput-duration"="5"),
 ("type-name"="TweetMessageType"),
 ("dataverse-dataset"="feeds:SyntheticTweets"),
 ("format"="adm"),
-("reader-stream"="twitter_firehose"),
+("stream-source"="org.apache.asterix.external.input.stream.TwitterFirehoseStreamFactory"),
 ("mode"="controlled"));
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feeds_07/feeds_07.1.ddl.sqlpp
 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feeds_07/feeds_07.1.ddl.sqlpp
index e3f3ae5..1f24192 100644
--- 
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feeds_07/feeds_07.1.ddl.sqlpp
+++ 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feeds_07/feeds_07.1.ddl.sqlpp
@@ -45,12 +45,12 @@
 
 create  dataset

Change in asterixdb[master]: Move Twitter Firehose Datasource to Test Source Folder

[abdullah alamoudi (Code Review)]

abdullah alamoudi has uploaded a new patch set (#2).

Change subject: Move Twitter Firehose Datasource to Test Source Folder
..

Move Twitter Firehose Datasource to Test Source Folder

Change-Id: Iefe2130707012b8ce60f5dfac96635a1a515a076
---
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_07/feeds_07.1.ddl.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.1.ddl.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_09/feeds_09.1.ddl.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feeds_07/feeds_07.1.ddl.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feeds_08/feeds_08.1.ddl.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feeds_09/feeds_09.1.ddl.sqlpp
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/DatasourceFactoryProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
R 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/generator/DataGenerator.java
R 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/generator/TweetGenerator.java
R 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/input/stream/TwitterFirehoseInputStream.java
R 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/input/stream/TwitterFirehoseStreamFactory.java
M 
asterixdb/asterix-tools/src/main/java/org/apache/asterix/tools/external/data/DataGeneratorForSpatialIndexEvaluation.java
13 files changed, 24 insertions(+), 26 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/90/1290/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1290
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iefe2130707012b8ce60f5dfac96635a1a515a076
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 

Change in asterixdb[master]: Move Twitter Firehose Datasource to Test Source Folder

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Move Twitter Firehose Datasource to Test Source Folder
..


Patch Set 1:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1290/1/asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/generator/TweetGenerator.java
File 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/generator/TweetGenerator.java:

Line 38: public static final String KEY_TPS = "tps"; 
> WS
Done


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1290
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Iefe2130707012b8ce60f5dfac96635a1a515a076
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Move Twitter Firehose Datasource to Test Source Folder

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1290

to look at the new patch set (#3).

Change subject: Move Twitter Firehose Datasource to Test Source Folder
..

Move Twitter Firehose Datasource to Test Source Folder

Change-Id: Iefe2130707012b8ce60f5dfac96635a1a515a076
---
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_07/feeds_07.1.ddl.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.1.ddl.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_09/feeds_09.1.ddl.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feeds_07/feeds_07.1.ddl.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feeds_08/feeds_08.1.ddl.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feeds_09/feeds_09.1.ddl.sqlpp
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/DatasourceFactoryProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
R 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/generator/DataGenerator.java
R 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/generator/TweetGenerator.java
R 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/input/stream/TwitterFirehoseInputStream.java
R 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/input/stream/TwitterFirehoseStreamFactory.java
M asterixdb/asterix-tools/pom.xml
M 
asterixdb/asterix-tools/src/main/java/org/apache/asterix/tools/external/data/DataGeneratorForSpatialIndexEvaluation.java
14 files changed, 30 insertions(+), 26 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/90/1290/3
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1290
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iefe2130707012b8ce60f5dfac96635a1a515a076
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 

Change in asterixdb[master]: Move Twitter Firehose Datasource to Test Source Folder

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1290

to look at the new patch set (#6).

Change subject: Move Twitter Firehose Datasource to Test Source Folder
..

Move Twitter Firehose Datasource to Test Source Folder

Change-Id: Iefe2130707012b8ce60f5dfac96635a1a515a076
---
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_07/feeds_07.1.ddl.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.1.ddl.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_09/feeds_09.1.ddl.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feeds_07/feeds_07.1.ddl.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feeds_08/feeds_08.1.ddl.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feeds_09/feeds_09.1.ddl.sqlpp
M asterixdb/asterix-external-data/pom.xml
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/DatasourceFactoryProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
R 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/generator/DataGenerator.java
R 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/generator/TweetGenerator.java
R 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/input/stream/TwitterFirehoseInputStream.java
R 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/input/stream/TwitterFirehoseStreamFactory.java
M asterixdb/asterix-tools/pom.xml
R 
asterixdb/asterix-tools/src/test/java/org/apache/asterix/tools/datagen/AdgClientDriver.java
R 
asterixdb/asterix-tools/src/test/java/org/apache/asterix/tools/datagen/AdmDataGen.java
R 
asterixdb/asterix-tools/src/test/java/org/apache/asterix/tools/datagen/CustOrdDataGen.java
R 
asterixdb/asterix-tools/src/test/java/org/apache/asterix/tools/datagen/EventDataGen.java
R 
asterixdb/asterix-tools/src/test/java/org/apache/asterix/tools/external/data/DataGeneratorForSpatialIndexEvaluation.java
R 
asterixdb/asterix-tools/src/test/java/org/apache/asterix/tools/external/data/GULongIDGenerator.java
R 
asterixdb/asterix-tools/src/test/java/org/apache/asterix/tools/external/data/TweetGeneratorForSpatialIndexEvaluation.java
R 
asterixdb/asterix-tools/src/test/java/org/apache/asterix/tools/tbltoadm/TblToAdm.java
R 
asterixdb/asterix-tools/src/test/java/org/apache/asterix/tools/translator/ADGenDmlTranslator.java
23 files changed, 33 insertions(+), 37 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/90/1290/6
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1290
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iefe2130707012b8ce60f5dfac96635a1a515a076
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 

Change in asterixdb[master]: Move Twitter Firehose Datasource to Test Source Folder

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1290

to look at the new patch set (#7).

Change subject: Move Twitter Firehose Datasource to Test Source Folder
..

Move Twitter Firehose Datasource to Test Source Folder

Change-Id: Iefe2130707012b8ce60f5dfac96635a1a515a076
---
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_07/feeds_07.1.ddl.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.1.ddl.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_09/feeds_09.1.ddl.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feeds_07/feeds_07.1.ddl.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feeds_08/feeds_08.1.ddl.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feeds_09/feeds_09.1.ddl.sqlpp
M asterixdb/asterix-experiments/pom.xml
M asterixdb/asterix-external-data/pom.xml
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/DatasourceFactoryProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
R 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/generator/DataGenerator.java
R 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/generator/TweetGenerator.java
R 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/input/stream/TwitterFirehoseInputStream.java
R 
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/input/stream/TwitterFirehoseStreamFactory.java
M asterixdb/asterix-tools/pom.xml
R 
asterixdb/asterix-tools/src/test/java/org/apache/asterix/tools/datagen/AdgClientDriver.java
R 
asterixdb/asterix-tools/src/test/java/org/apache/asterix/tools/datagen/AdmDataGen.java
R 
asterixdb/asterix-tools/src/test/java/org/apache/asterix/tools/datagen/CustOrdDataGen.java
R 
asterixdb/asterix-tools/src/test/java/org/apache/asterix/tools/datagen/EventDataGen.java
R 
asterixdb/asterix-tools/src/test/java/org/apache/asterix/tools/external/data/DataGeneratorForSpatialIndexEvaluation.java
R 
asterixdb/asterix-tools/src/test/java/org/apache/asterix/tools/external/data/GULongIDGenerator.java
R 
asterixdb/asterix-tools/src/test/java/org/apache/asterix/tools/external/data/TweetGeneratorForSpatialIndexEvaluation.java
R 
asterixdb/asterix-tools/src/test/java/org/apache/asterix/tools/tbltoadm/TblToAdm.java
R 
asterixdb/asterix-tools/src/test/java/org/apache/asterix/tools/translator/ADGenDmlTranslator.java
24 files changed, 42 insertions(+), 37 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/90/1290/7
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1290
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iefe2130707012b8ce60f5dfac96635a1a515a076
Gerrit-PatchSet: 7
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 

Change in asterixdb[master]: More Dependency Cleanup

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: More Dependency Cleanup
..


Patch Set 2: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1294
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I9cd6f1c9829c1a71f8b10994823a5f1fbe09d797
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Enhanced Insert AQL

[abdullah alamoudi (Code Review)]

Hello Michael Carey, Michael Blow, Steven Jacobs, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1150

to look at the new patch set (#21).

Change subject: Enhanced Insert AQL
..

Enhanced Insert AQL

The optional "as Variable" provides a variable binding for the inserted records
The optional "returning Query" allows users to run simple
queries/functions on the records returned by the insert, and can refer
to the variable bound in "as Variable"

Allow commits to be non-sink operators (contnue job pipeline after commit)

Additionally, this change makes small modifications to
the extension code to prepare for the BAD extension

Also made the OptimizerTests able to work for Extensions

Change-Id: I65789d2a861d15232dd29156a6987d0635ec6c94
---
M asterixdb/asterix-active/pom.xml
M 
asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEntityEventsListener.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/extension/IExtensionStatement.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/CommitOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/CommitPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/CommitRuntime.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/CommitRuntimeFactory.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/UpsertCommitRuntime.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceAutogenerateIDRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceRapidFrameFlushProjectAssignRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceStaticTypeCastForInsertRule.java
R 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SetupCommitExtensionOpRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SweepIllegalNonfunctionalFunctions.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/UnnestToDataScanRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineAllNtsInSubplanVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineLeftNtsInSubplanJoinFlatteningVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/SubplanSpecialFlatteningCheckVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/CompiledStatements.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/servlet/FeedServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/optimizer/OptimizerTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/ExecutionTestUtil.java
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/insert-return-custom-result.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/insert-return-custom-result.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/insert-return-records/insert-return-records.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/insert-return-records/insert-return-records.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/insert-returning-fieldname/insert-returning-fieldname.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/insert-returning-fieldname/insert-returning-fieldname.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/insert-with-bad-return/insert-with-bad-return.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/insert-with-bad-return/insert-with-bad-return.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/upsert-return-custom-result/upsert-re

Change in asterixdb[master]: Enhanced Insert AQL

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Enhanced Insert AQL
..


Patch Set 22: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1150
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I65789d2a861d15232dd29156a6987d0635ec6c94
Gerrit-PatchSet: 22
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Michael Carey 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1695 Fix Functions in Default Dataverse

[abdullah alamoudi (Code Review)]

abdullah alamoudi has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1297

Change subject: ASTERIXDB-1695 Fix Functions in Default Dataverse
..

ASTERIXDB-1695 Fix Functions in Default Dataverse

Change-Id: I7f08c0acaccdcfaeb4fc9cfb1529214b543d0e86
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/default/default.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/default/default.2.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/default/default.3.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/types/default/default.2.adm
M 
asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/FunctionUtil.java
6 files changed, 73 insertions(+), 4 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/97/1297/1

diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
index 8a25888..d5b1415 100644
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
@@ -1773,6 +1773,7 @@
 throws Exception {
 CreateFunctionStatement cfs = (CreateFunctionStatement) stmt;
 String dataverse = 
getActiveDataverseName(cfs.getSignature().getNamespace());
+cfs.getSignature().setNamespace(dataverse);
 String functionName = cfs.getaAterixFunction().getName();
 
 MetadataTransactionContext mdTxnCtx = 
MetadataManager.INSTANCE.beginTransaction();
@@ -1801,6 +1802,7 @@
 protected void handleFunctionDropStatement(AqlMetadataProvider 
metadataProvider, Statement stmt) throws Exception {
 FunctionDropStatement stmtDropFunction = (FunctionDropStatement) stmt;
 FunctionSignature signature = stmtDropFunction.getFunctionSignature();
+
signature.setNamespace(getActiveDataverseName(signature.getNamespace()));
 MetadataTransactionContext mdTxnCtx = 
MetadataManager.INSTANCE.beginTransaction();
 metadataProvider.setMetadataTxnContext(mdTxnCtx);
 
MetadataLockManager.INSTANCE.functionStatementBegin(signature.getNamespace(),
@@ -1965,7 +1967,7 @@
 @Override
 public JobSpecification rewriteCompileQuery(AqlMetadataProvider 
metadataProvider, Query query,
 ICompiledDmlStatement stmt)
-throws AsterixException, RemoteException, 
AlgebricksException, JSONException, ACIDException {
+throws AsterixException, RemoteException, AlgebricksException, 
JSONException, ACIDException {
 
 // Query Rewriting (happens under the same ongoing metadata 
transaction)
 Pair reWrittenQuery = 
apiFramework.reWriteQuery(declaredFunctions, metadataProvider, query,
@@ -2266,7 +2268,7 @@
  */
 protected Triple> 
getFeedConnectionRequest(String dataverse,
 Feed feed, String dataset, FeedPolicyEntity feedPolicy, 
MetadataTransactionContext mdTxnCtx)
-throws AsterixException {
+throws AsterixException {
 IFeedJoint sourceFeedJoint;
 FeedConnectionRequest request;
 List functionsToApply = new ArrayList<>();
@@ -2557,7 +2559,7 @@
 
 protected JobSpecification handleQuery(AqlMetadataProvider 
metadataProvider, Query query,
 IHyracksClientConnection hcc, IHyracksDataset hdc, ResultDelivery 
resultDelivery, Stats stats)
-throws Exception {
+throws Exception {
 MetadataTransactionContext mdTxnCtx = 
MetadataManager.INSTANCE.beginTransaction();
 boolean bActiveTxn = true;
 metadataProvider.setMetadataTxnContext(mdTxnCtx);
@@ -2591,7 +2593,7 @@
 
 private void handleQueryResult(AqlMetadataProvider metadataProvider, 
IHyracksClientConnection hcc,
 IHyracksDataset hdc, JobSpecification compiled, ResultDelivery 
resultDelivery, Stats stats)
-throws Exception {
+throws Exception {
 if (GlobalConfig.ASTERIX_LOGGER.isLoggable(Level.FINE)) {
 GlobalConfig.ASTERIX_LOGGER.fine(compiled.toJSON().toString(1));
 }
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/default/default.1.ddl.sqlpp
 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/default/default.1.ddl.sqlpp
new file mode 100644
index 000..63c275b
--- /dev/null
+++ 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/default/default.1.ddl.sqlpp
@@ -0,0 +1,23 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more con

Change in asterixdb[master]: ASTERIXDB-1695 Fix Functions in Default Dataverse

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1297

to look at the new patch set (#2).

Change subject: ASTERIXDB-1695 Fix Functions in Default Dataverse
..

ASTERIXDB-1695 Fix Functions in Default Dataverse

Change-Id: I7f08c0acaccdcfaeb4fc9cfb1529214b543d0e86
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/default/default.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/default/default.2.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/default/default.3.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/types/default/default.2.adm
M 
asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/FunctionUtil.java
6 files changed, 77 insertions(+), 10 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/97/1297/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1297
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7f08c0acaccdcfaeb4fc9cfb1529214b543d0e86
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Yingyi Bu 

Change in asterixdb[master]: dead code elimination in asterix-events

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: dead code elimination in asterix-events
..


Patch Set 3: Code-Review+1

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1300
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I8873652ca6b3add003e7454d09c175b2218ed685
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Till Westmann 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1695 Fix Functions in Default Dataverse

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1297

to look at the new patch set (#3).

Change subject: ASTERIXDB-1695 Fix Functions in Default Dataverse
..

ASTERIXDB-1695 Fix Functions in Default Dataverse

Change-Id: I7f08c0acaccdcfaeb4fc9cfb1529214b543d0e86
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/default/default.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/default/default.2.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/default/default.3.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/types/default/default.2.adm
M 
asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/FunctionUtil.java
6 files changed, 78 insertions(+), 10 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/97/1297/3
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1297
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7f08c0acaccdcfaeb4fc9cfb1529214b543d0e86
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Yingyi Bu 

Change in asterixdb[master]: ASTERIXDB-1695 Fix Functions in Default Dataverse

[abdullah alamoudi (Code Review)]

abdullah alamoudi has submitted this change and it was merged.

Change subject: ASTERIXDB-1695 Fix Functions in Default Dataverse
..


ASTERIXDB-1695 Fix Functions in Default Dataverse

Change-Id: I7f08c0acaccdcfaeb4fc9cfb1529214b543d0e86
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1297
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Yingyi Bu 
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/default/default.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/default/default.2.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/default/default.3.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/types/default/default.2.adm
M 
asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/FunctionUtil.java
6 files changed, 78 insertions(+), 10 deletions(-)

Approvals:
  Yingyi Bu: Looks good to me, approved
  Jenkins: Verified; No violations found; Verified



diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
index 8a25888..d5b1415 100644
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
@@ -1773,6 +1773,7 @@
 throws Exception {
 CreateFunctionStatement cfs = (CreateFunctionStatement) stmt;
 String dataverse = 
getActiveDataverseName(cfs.getSignature().getNamespace());
+cfs.getSignature().setNamespace(dataverse);
 String functionName = cfs.getaAterixFunction().getName();
 
 MetadataTransactionContext mdTxnCtx = 
MetadataManager.INSTANCE.beginTransaction();
@@ -1801,6 +1802,7 @@
 protected void handleFunctionDropStatement(AqlMetadataProvider 
metadataProvider, Statement stmt) throws Exception {
 FunctionDropStatement stmtDropFunction = (FunctionDropStatement) stmt;
 FunctionSignature signature = stmtDropFunction.getFunctionSignature();
+
signature.setNamespace(getActiveDataverseName(signature.getNamespace()));
 MetadataTransactionContext mdTxnCtx = 
MetadataManager.INSTANCE.beginTransaction();
 metadataProvider.setMetadataTxnContext(mdTxnCtx);
 
MetadataLockManager.INSTANCE.functionStatementBegin(signature.getNamespace(),
@@ -1965,7 +1967,7 @@
 @Override
 public JobSpecification rewriteCompileQuery(AqlMetadataProvider 
metadataProvider, Query query,
 ICompiledDmlStatement stmt)
-throws AsterixException, RemoteException, 
AlgebricksException, JSONException, ACIDException {
+throws AsterixException, RemoteException, AlgebricksException, 
JSONException, ACIDException {
 
 // Query Rewriting (happens under the same ongoing metadata 
transaction)
 Pair reWrittenQuery = 
apiFramework.reWriteQuery(declaredFunctions, metadataProvider, query,
@@ -2266,7 +2268,7 @@
  */
 protected Triple> 
getFeedConnectionRequest(String dataverse,
 Feed feed, String dataset, FeedPolicyEntity feedPolicy, 
MetadataTransactionContext mdTxnCtx)
-throws AsterixException {
+throws AsterixException {
 IFeedJoint sourceFeedJoint;
 FeedConnectionRequest request;
 List functionsToApply = new ArrayList<>();
@@ -2557,7 +2559,7 @@
 
 protected JobSpecification handleQuery(AqlMetadataProvider 
metadataProvider, Query query,
 IHyracksClientConnection hcc, IHyracksDataset hdc, ResultDelivery 
resultDelivery, Stats stats)
-throws Exception {
+throws Exception {
 MetadataTransactionContext mdTxnCtx = 
MetadataManager.INSTANCE.beginTransaction();
 boolean bActiveTxn = true;
 metadataProvider.setMetadataTxnContext(mdTxnCtx);
@@ -2591,7 +2593,7 @@
 
 private void handleQueryResult(AqlMetadataProvider metadataProvider, 
IHyracksClientConnection hcc,
 IHyracksDataset hdc, JobSpecification compiled, ResultDelivery 
resultDelivery, Stats stats)
-throws Exception {
+throws Exception {
 if (GlobalConfig.ASTERIX_LOGGER.isLoggable(Level.FINE)) {
 GlobalConfig.ASTERIX_LOGGER.fine(compiled.toJSON().toString(1));
 }
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/default/default.1.ddl.sqlpp
 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/types/default/default.1.ddl.sqlpp
new file mode 100644
index 000..63c275b
--- /dev/null
+++ 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/ty

Change in asterixdb[master]: dead code elimination in asterix-events

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: dead code elimination in asterix-events
..


Patch Set 3: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1300
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I8873652ca6b3add003e7454d09c175b2218ed685
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Till Westmann 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: dead code elimination in asterix-events

[abdullah alamoudi (Code Review)]

abdullah alamoudi has submitted this change and it was merged.

Change subject: dead code elimination in asterix-events
..


dead code elimination in asterix-events

Change-Id: I8873652ca6b3add003e7454d09c175b2218ed685
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1300
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: abdullah alamoudi 
---
D 
asterixdb/asterix-events/src/main/java/org/apache/asterix/event/api/ClusterInfo.java
D 
asterixdb/asterix-events/src/main/java/org/apache/asterix/event/api/NodeInfo.java
M 
asterixdb/asterix-events/src/main/java/org/apache/asterix/event/driver/EventDriver.java
M 
asterixdb/asterix-events/src/main/java/org/apache/asterix/event/error/OutputHandler.java
M 
asterixdb/asterix-events/src/main/java/org/apache/asterix/event/error/VerificationUtil.java
M 
asterixdb/asterix-events/src/main/java/org/apache/asterix/event/management/AsterixEventServiceClient.java
D 
asterixdb/asterix-events/src/main/java/org/apache/asterix/event/management/DefaultOutputHandler.java
D 
asterixdb/asterix-events/src/main/java/org/apache/asterix/event/management/ErrorHandler.java
M 
asterixdb/asterix-events/src/main/java/org/apache/asterix/event/management/EventExecutor.java
M 
asterixdb/asterix-events/src/main/java/org/apache/asterix/event/management/EventTask.java
M 
asterixdb/asterix-events/src/main/java/org/apache/asterix/event/management/EventUtil.java
M 
asterixdb/asterix-events/src/main/java/org/apache/asterix/event/model/AsterixInstance.java
M 
asterixdb/asterix-events/src/main/java/org/apache/asterix/event/model/BackupInfo.java
M 
asterixdb/asterix-events/src/main/java/org/apache/asterix/event/model/ProcessInfo.java
M 
asterixdb/asterix-events/src/main/java/org/apache/asterix/event/service/AsterixEventService.java
M 
asterixdb/asterix-events/src/main/java/org/apache/asterix/event/service/AsterixEventServiceUtil.java
M 
asterixdb/asterix-events/src/main/java/org/apache/asterix/event/service/ServiceProvider.java
M 
asterixdb/asterix-events/src/main/java/org/apache/asterix/event/service/ZooKeeperService.java
M 
asterixdb/asterix-events/src/main/java/org/apache/asterix/event/util/PatternCreator.java
M 
asterixdb/asterix-installer/src/main/java/org/apache/asterix/installer/command/LogCommand.java
20 files changed, 54 insertions(+), 323 deletions(-)

Approvals:
  abdullah alamoudi: Looks good to me, approved
  Jenkins: Verified; Verified

Objections:
  Jenkins: Violations found



diff --git 
a/asterixdb/asterix-events/src/main/java/org/apache/asterix/event/api/ClusterInfo.java
 
b/asterixdb/asterix-events/src/main/java/org/apache/asterix/event/api/ClusterInfo.java
deleted file mode 100644
index d996e0e..000
--- 
a/asterixdb/asterix-events/src/main/java/org/apache/asterix/event/api/ClusterInfo.java
+++ /dev/null
@@ -1,30 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements.  See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership.  The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License.  You may obtain a copy of the License at
- *
- *   http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied.  See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-package org.apache.asterix.event.api;
-
-import java.util.List;
-
-public class ClusterInfo {
-
-List nodes;
-
-public ClusterInfo(List nodes) {
-this.nodes = nodes;
-}
-}
diff --git 
a/asterixdb/asterix-events/src/main/java/org/apache/asterix/event/api/NodeInfo.java
 
b/asterixdb/asterix-events/src/main/java/org/apache/asterix/event/api/NodeInfo.java
deleted file mode 100644
index d2dab4c..000
--- 
a/asterixdb/asterix-events/src/main/java/org/apache/asterix/event/api/NodeInfo.java
+++ /dev/null
@@ -1,38 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements.  See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership.  The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License.  You may obtain a copy of the License at
- *
- *   http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied.  See the License for the
- * specific langua

Change in asterixdb[master]: Fix Upsert Pipeline

[abdullah alamoudi (Code Review)]

abdullah alamoudi has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1308

Change subject: Fix Upsert Pipeline
..

Fix Upsert Pipeline

Change-Id: I5c19d448f9664ecaeac600668a6dbdcf40673c56
---
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AqlMetadataProvider.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/AsterixLSMPrimaryUpsertOperatorNodePushable.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/metadata/IMetadataProvider.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/InsertDeleteUpsertOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/SchemaVariableVisitor.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/InsertDeleteUpsertPOperator.java
M 
hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/comm/io/FrameTupleAccessor.java
7 files changed, 96 insertions(+), 59 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/08/1308/1

diff --git 
a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AqlMetadataProvider.java
 
b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AqlMetadataProvider.java
index e05aa25..619ef94 100644
--- 
a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AqlMetadataProvider.java
+++ 
b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AqlMetadataProvider.java
@@ -26,10 +26,10 @@
 import java.util.Map;
 
 import org.apache.asterix.common.config.AsterixStorageProperties;
-import org.apache.asterix.common.config.GlobalConfig;
 import org.apache.asterix.common.config.DatasetConfig.DatasetType;
 import org.apache.asterix.common.config.DatasetConfig.ExternalFilePendingOp;
 import org.apache.asterix.common.config.DatasetConfig.IndexType;
+import org.apache.asterix.common.config.GlobalConfig;
 import org.apache.asterix.common.context.AsterixVirtualBufferCacheProvider;
 import org.apache.asterix.common.context.ITransactionSubsystemProvider;
 import org.apache.asterix.common.context.TransactionSubsystemProvider;
@@ -42,8 +42,8 @@
 import 
org.apache.asterix.common.ioopcallbacks.LSMInvertedIndexIOOperationCallbackFactory;
 import 
org.apache.asterix.common.ioopcallbacks.LSMRTreeIOOperationCallbackFactory;
 import org.apache.asterix.common.library.ILibraryManager;
-import org.apache.asterix.common.transactions.JobId;
 import org.apache.asterix.common.transactions.IRecoveryManager.ResourceType;
+import org.apache.asterix.common.transactions.JobId;
 import org.apache.asterix.common.utils.StoragePathUtil;
 import 
org.apache.asterix.dataflow.data.nontagged.valueproviders.AqlPrimitiveValueProviderFactory;
 import org.apache.asterix.external.adapter.factory.LookupAdapterFactory;
@@ -1097,7 +1097,7 @@
 
 @Override
 public Pair 
getUpsertRuntime(
-IDataSource dataSource, IOperatorSchema 
propagatedSchema, IVariableTypeEnvironment typeEnv,
+IDataSource dataSource, IOperatorSchema inputSchema, 
IVariableTypeEnvironment typeEnv,
 List primaryKeys, LogicalVariable payload, 
List filterKeys,
 List additionalNonFilterFields, RecordDescriptor 
recordDesc, JobGenContext context,
 JobSpecification spec) throws AlgebricksException {
@@ -1119,22 +1119,22 @@
 int i = 0;
 // set the keys' permutations
 for (LogicalVariable varKey : primaryKeys) {
-int idx = propagatedSchema.findVariable(varKey);
+int idx = inputSchema.findVariable(varKey);
 fieldPermutation[i] = idx;
 bloomFilterKeyFields[i] = i;
 i++;
 }
 // set the record permutation
-fieldPermutation[i++] = propagatedSchema.findVariable(payload);
+fieldPermutation[i++] = inputSchema.findVariable(payload);
 // set the filters' permutations.
 if (numFilterFields > 0) {
-int idx = propagatedSchema.findVariable(filterKeys.get(0));
+int idx = inputSchema.findVariable(filterKeys.get(0));
 fieldPermutation[i++] = idx;
 }
 
 if (additionalNonFilterFields != null) {
 for (LogicalVariable var : additionalNonFilterFields) {
-int idx = propagatedSchema.findVariable(var);
+int idx = inputSchema.findVariable(var);
 fieldPermutation[i++] = idx;
 }
 }
@@ -1195,22 +1195,23 @@
 + (dataset.hasMetaPart() ? 2 : 1) + numFilterFields];
 ISerializerDeserializer[] outp

Change in asterixdb[master]: Fix Upsert Pipeline

[abdullah alamoudi (Code Review)]

abdullah alamoudi has uploaded a new patch set (#2).

Change subject: Fix Upsert Pipeline
..

Fix Upsert Pipeline

Change-Id: I5c19d448f9664ecaeac600668a6dbdcf40673c56
---
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AqlMetadataProvider.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/AsterixLSMPrimaryUpsertOperatorNodePushable.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/metadata/IMetadataProvider.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/InsertDeleteUpsertOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/SchemaVariableVisitor.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/InsertDeleteUpsertPOperator.java
M 
hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/comm/io/FrameTupleAccessor.java
7 files changed, 95 insertions(+), 59 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/08/1308/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1308
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5c19d448f9664ecaeac600668a6dbdcf40673c56
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 

Change in asterixdb[master]: Fix Upsert Pipeline

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Fix Upsert Pipeline
..


Patch Set 2:

(3 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1308/2/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AqlMetadataProvider.java
File 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AqlMetadataProvider.java:

Line 1232: outputSerDes[j+f] = recordDesc.getFields()[j];
> Pls fix it?
Done


https://asterix-gerrit.ics.uci.edu/#/c/1308/2/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/AsterixLSMPrimaryUpsertOperatorNodePushable.java
File 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/AsterixLSMPrimaryUpsertOperatorNodePushable.java:

Line 212: 
dos.write(prevTuple.getFieldData(numOfPrimaryKeys + (hasMeta ? 2 : 1)),
> Extract that the assignment of 2:1 as a variable out of the while loop as i
Done


https://asterix-gerrit.ics.uci.edu/#/c/1308/2/hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/comm/io/FrameTupleAccessor.java
File 
hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/comm/io/FrameTupleAccessor.java:

Line 138: ByteBufferInputStream bbis = new ByteBufferInputStream();
> Why do we need this public pretty print methods?
We are using them when we debug. They are quite useful. ask Till ;)
I will add a comment for them


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1308
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I5c19d448f9664ecaeac600668a6dbdcf40673c56
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Fix Upsert Pipeline

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1308

to look at the new patch set (#3).

Change subject: Fix Upsert Pipeline
..

Fix Upsert Pipeline

Change-Id: I5c19d448f9664ecaeac600668a6dbdcf40673c56
---
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AqlMetadataProvider.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/AsterixLSMPrimaryUpsertOperatorNodePushable.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/metadata/IMetadataProvider.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/InsertDeleteUpsertOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/SchemaVariableVisitor.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/InsertDeleteUpsertPOperator.java
M 
hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/comm/io/FrameTupleAccessor.java
7 files changed, 270 insertions(+), 170 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/08/1308/3
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1308
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5c19d448f9664ecaeac600668a6dbdcf40673c56
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 

Change in asterixdb[master]: Fix Upsert Pipeline

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Fix Upsert Pipeline
..


Patch Set 3:

To address the broad questions:
1. This doesn't need to be applied to secondary upsert operator because it 
doesn't produce any fields from the indexes, it only applies whatever operation 
it should.
2. This doesn't affect other operations since other operations (insert/delete) 
don't produce anything but apply operations using incoming tuples.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1308
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I5c19d448f9664ecaeac600668a6dbdcf40673c56
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Fix Upsert Pipeline

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Fix Upsert Pipeline
..


Patch Set 3:

(4 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1308/3/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/AsterixLSMPrimaryUpsertOperatorNodePushable.java
File 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/AsterixLSMPrimaryUpsertOperatorNodePushable.java:

Line 97: || (filterFieldIndex >= 0 && (fieldPermutation.length 
> numOfPrimaryKeys + 2)));
> Why is meta always before filter? Shouldn't meta be set to something like -
because if hasMeta is false, then this field will not be used at all.


Line 202: }
> What was happening previously with meta/filter?
we used to write those to the output frame after we do the insert, by that 
time, it is too late because their values have changed


Line 218: }
> Is the crux of the fix here that the modCallBack.setOp(Operation.DELETE) go
not that the modCallback was moved here but that the delete (219-224) was 
postponed after we have written the value to the output frame


https://asterix-gerrit.ics.uci.edu/#/c/1308/3/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/InsertDeleteUpsertOperator.java
File 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/InsertDeleteUpsertOperator.java:

Line 96: }
> Why does the order matter here? Is it determining the order of the fields t
it determines the order of fields that will come out of the 
insert-delete-upsert pipeline. Upsert is the only operation which produces new 
variables and I switched the order to avoid copying of the delete tuple to a 
temporary memory perform the delete/upsert, write the new record value and then 
write the deleted value.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1308
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I5c19d448f9664ecaeac600668a6dbdcf40673c56
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Fix Upsert Pipeline

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Fix Upsert Pipeline
..


Patch Set 3:

Done updating the format

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1308
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I5c19d448f9664ecaeac600668a6dbdcf40673c56
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Fix Upsert Pipeline

[abdullah alamoudi (Code Review)]

Hello Michael Blow, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1308

to look at the new patch set (#4).

Change subject: Fix Upsert Pipeline
..

Fix Upsert Pipeline

Change-Id: I5c19d448f9664ecaeac600668a6dbdcf40673c56
---
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AqlMetadataProvider.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/AsterixLSMPrimaryUpsertOperatorNodePushable.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/metadata/IMetadataProvider.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/InsertDeleteUpsertOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/SchemaVariableVisitor.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/InsertDeleteUpsertPOperator.java
M 
hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/comm/io/FrameTupleAccessor.java
7 files changed, 208 insertions(+), 125 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/08/1308/4
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1308
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5c19d448f9664ecaeac600668a6dbdcf40673c56
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 

Change in asterixdb[master]: Fix Upsert Pipeline

[abdullah alamoudi (Code Review)]

Hello Michael Blow, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1308

to look at the new patch set (#5).

Change subject: Fix Upsert Pipeline
..

Fix Upsert Pipeline

Change-Id: I5c19d448f9664ecaeac600668a6dbdcf40673c56
---
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FeedExceptionHandler.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AqlMetadataProvider.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/AsterixLSMPrimaryUpsertOperatorNodePushable.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/metadata/IMetadataProvider.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/InsertDeleteUpsertOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/SchemaVariableVisitor.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/InsertDeleteUpsertPOperator.java
M 
hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/comm/io/FrameTupleAccessor.java
M 
hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/comm/io/ResultFrameTupleAccessor.java
A 
hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/util/FrameDebugUtils.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/unit/RunMergingFrameReaderTest.java
11 files changed, 370 insertions(+), 222 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/08/1308/5
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1308
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5c19d448f9664ecaeac600668a6dbdcf40673c56
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 

Change in asterixdb[master]: ASTERIXDB-1711: remove some Aql-prefixes

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: ASTERIXDB-1711: remove some Aql-prefixes
..


Patch Set 2:

(8 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1314/2/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/OptimizableOperatorSubTree.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/OptimizableOperatorSubTree.java:

Line 230: if (datasource instanceof DataSource) {
this has nothing to do with the change but
can the datasource be an instance of something other than DataSource?

If yes, give an example and whether we should return false in that case?
if no, then why do we even have this check?


https://asterix-gerrit.ics.uci.edu/#/c/1314/2/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java:

Line 1235: FunctionIdentifier fid = (lc.getType() == 
ListConstructor.Type.ORDERED_LIST_CONSTRUCTOR)
why did we remove the import for ListConstructor.Type and imported 
DataSource.Type instead?

I think we should have consistency. we either always use the qualified enum 
name, or have a different name that is more descriptive for enums.


https://asterix-gerrit.ics.uci.edu/#/c/1314/2/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AqlIndex.java
File 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AqlIndex.java:

Line 46: DataSourceId asid = new DataSourceId(dataverse, dataset);
since we changed the class name, let's also change the variable name?


https://asterix-gerrit.ics.uci.edu/#/c/1314/2/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AqlMetadataProvider.java
File 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AqlMetadataProvider.java:

Line 170: public class AqlMetadataProvider implements 
IMetadataProvider {
create an issue to rename this?


Line 345: DataSource ads = findDataSource(dataSourceId);
what is ads? rename the variable?


https://asterix-gerrit.ics.uci.edu/#/c/1314/2/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/DatasetDataSource.java
File 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/DatasetDataSource.java:

Line 113: DataSourceId asid = getId();
rename var?


https://asterix-gerrit.ics.uci.edu/#/c/1314/2/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/LoadableDataSource.java
File 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/LoadableDataSource.java:

Line 59: super(new DataSourceId("loadable_dv", "loadable_ds"), 
itemType, metaItemType, Type.LOADABLE, null);
these two strings are supposed to be dataverse and dataset names. do we have 
dataverse or dataset with those names? no

this then is broken and needs to be fixed.

Not necessarily in this change but at least, let's create an issue for it.


https://asterix-gerrit.ics.uci.edu/#/c/1314/2/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/MetadataManagerUtil.java
File 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/MetadataManagerUtil.java:

Line 156: DataSourceId aqlId = id;
why assign to another variable?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1314
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0b64ffa7c50cd62fc3303fdb44eb769f56c978a
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Till Westmann 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Remove Server Context

[abdullah alamoudi (Code Review)]

abdullah alamoudi has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1320

Change subject: Remove Server Context
..

Remove Server Context

Change-Id: I4261cf44410b498e84ffcac5e217281a306a7f03
---
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/service/IControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/application/CCApplicationContext.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/web/ApplicationInstallationHandler.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/CliDeployBinaryWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/CliUnDeployBinaryWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/application/ApplicationContext.java
D 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/context/ServerContext.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/deployment/DeploymentUtils.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/NodeControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/application/NCApplicationContext.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/DeployBinaryWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/UnDeployBinaryWork.java
13 files changed, 62 insertions(+), 92 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/20/1320/1

diff --git 
a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/service/IControllerService.java
 
b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/service/IControllerService.java
index 2e742ac..e2b13a1 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/service/IControllerService.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/service/IControllerService.java
@@ -18,8 +18,26 @@
  */
 package org.apache.hyracks.api.service;
 
-public interface IControllerService {
-public void start() throws Exception;
+import java.io.File;
 
-public void stop() throws Exception;
+/**
+ * The base interface for the cluster controller service and the node 
controller service
+ */
+public interface IControllerService {
+/**
+ * Start the service
+ * @throws Exception
+ */
+void start() throws Exception;
+
+/**
+ * Stop the service
+ * @throws Exception
+ */
+void stop() throws Exception;
+
+/**
+ * @return the baseDir for the service process (used for binary deployment)
+ */
+File getBaseDir();
 }
diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
index ac38523..80eb159 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
@@ -68,11 +68,11 @@
 import org.apache.hyracks.control.cc.work.GetJobInfoWork;
 import org.apache.hyracks.control.cc.work.GetJobStatusWork;
 import org.apache.hyracks.control.cc.work.GetNodeControllersInfoWork;
-import org.apache.hyracks.control.cc.work.GetThreadDumpWork;
-import org.apache.hyracks.control.cc.work.GetThreadDumpWork.ThreadDumpRun;
 import org.apache.hyracks.control.cc.work.GetNodeDetailsJSONWork;
 import org.apache.hyracks.control.cc.work.GetResultPartitionLocationsWork;
 import org.apache.hyracks.control.cc.work.GetResultStatusWork;
+import org.apache.hyracks.control.cc.work.GetThreadDumpWork;
+import org.apache.hyracks.control.cc.work.GetThreadDumpWork.ThreadDumpRun;
 import org.apache.hyracks.control.cc.work.JobStartWork;
 import org.apache.hyracks.control.cc.work.JobletCleanupNotificationWork;
 import org.apache.hyracks.control.cc.work.NodeHeartbeatWork;
@@ -94,7 +94,6 @@
 import org.apache.hyracks.control.cc.work.TriggerNCWork;
 imp

Change in asterixdb[master]: Remove HyracksRootContext

[abdullah alamoudi (Code Review)]

abdullah alamoudi has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1321

Change subject: Remove HyracksRootContext
..

Remove HyracksRootContext

Change-Id: I24f9d7b1fc5bdcf9d09abdac107001ff6b70bf02
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/AsterixNCAppRuntimeContext.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/application/INCApplicationContext.java
D 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/context/IHyracksRootContext.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/NodeControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/application/NCApplicationContext.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/dataset/DatasetPartitionManager.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/partitions/PartitionManager.java
D 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/runtime/RootHyracksContext.java
M 
hyracks-fullstack/hyracks/hyracks-examples/btree-example/btreehelper/src/main/java/org/apache/hyracks/examples/btree/helper/RuntimeContext.java
M 
hyracks-fullstack/hyracks/hyracks-test-support/src/main/java/org/apache/hyracks/test/support/TestJobletContext.java
M 
hyracks-fullstack/hyracks/hyracks-test-support/src/main/java/org/apache/hyracks/test/support/TestNCApplicationContext.java
D 
hyracks-fullstack/hyracks/hyracks-test-support/src/main/java/org/apache/hyracks/test/support/TestRootContext.java
M 
hyracks-fullstack/hyracks/hyracks-test-support/src/main/java/org/apache/hyracks/test/support/TestUtils.java
16 files changed, 65 insertions(+), 187 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/21/1321/1

diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/AsterixNCAppRuntimeContext.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/AsterixNCAppRuntimeContext.java
index c2c214c..3447648 100644
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/AsterixNCAppRuntimeContext.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/AsterixNCAppRuntimeContext.java
@@ -171,7 +171,7 @@
 fileMapManager = new AsterixFileMapManager();
 ICacheMemoryAllocator allocator = new HeapBufferAllocator();
 IPageCleanerPolicy pcp = new DelayPageCleanerPolicy(60);
-ioManager = ncApplicationContext.getRootContext().getIOManager();
+ioManager = ncApplicationContext.getIoManager();
 IPageReplacementStrategy prs = new 
ClockPageReplacementStrategy(allocator,
 storageProperties.getBufferCachePageSize(), 
storageProperties.getBufferCacheNumPages());
 
diff --git 
a/asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java
 
b/asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java
index 88ca736..df8d22e 100644
--- 
a/asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java
+++ 
b/asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java
@@ -370,7 +370,7 @@
 ctx = Mockito.spy(ctx);
 Mockito.when(ctx.getJobletContext()).thenReturn(jobletCtx);
 Mockito.when(ctx.getIOManager())
-
.thenReturn(ExecutionTestUtil.integrationUtil.ncs[0].getRootContext().getIOManager());
+
.thenReturn(ExecutionTestUtil.integrationUtil.ncs[0].getIoManager());
 return ctx;
 }
 
diff --git 
a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java
 
b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java
index cf014d5..0990998 100644
--- 
a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java
+++ 
b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java
@@ -32,9 +32,9 @@
 import org.apache.asterix.common.cluster.ClusterPartition;
 import org.apache.asterix.common.config.AsterixMetadataProperties;
 import org.apache.asterix.common.config.ClusterProperties;
+import org.apache.asterix.common.config.DatasetConfig.DatasetType;
 import org.apache.asterix.common.config.Gl

Change in asterixdb[master]: Remove Server Context

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Remove Server Context
..


Patch Set 1:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1320/1/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/NodeControllerService.java
File 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/NodeControllerService.java:

Line 186: baseDir = new File(new 
File(NodeControllerService.class.getName()), id);
> what is this used for?  seems like this should be configurable, and that th
it is used for binary deployment. if we want to make it configurable. let's do 
that in a different change,

let's keep changes small always.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1320
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4261cf44410b498e84ffcac5e217281a306a7f03
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Remove ICCContext

[abdullah alamoudi (Code Review)]

abdullah alamoudi has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1322

Change subject: Remove ICCContext
..

Remove ICCContext

Change-Id: I6f6a769741f14e91bcd4b970b4a022c0a453d380
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplicationEntryPoint.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/HDFSUtils.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/util/RuntimeUtils.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/application/ICCApplicationContext.java
D 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/context/ICCContext.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/application/CCApplicationContext.java
D 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/GetIpAddressNodeNameMapWork.java
8 files changed, 47 insertions(+), 184 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/22/1322/1

diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplicationEntryPoint.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplicationEntryPoint.java
index b097244..14b0554 100644
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplicationEntryPoint.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplicationEntryPoint.java
@@ -166,8 +166,10 @@
 }
 
 protected IHyracksClientConnection getNewHyracksClientConnection() throws 
Exception {
-String strIP = 
appCtx.getCCContext().getClusterControllerInfo().getClientNetAddress();
-int port = 
appCtx.getCCContext().getClusterControllerInfo().getClientNetPort();
+String strIP = ((ClusterControllerService) 
appCtx.getControllerService()).getClusterControllerInfo()
+.getClientNetAddress();
+int port = ((ClusterControllerService) 
appCtx.getControllerService()).getClusterControllerInfo()
+.getClientNetPort();
 return new HyracksConnection(strIP, port);
 }
 
diff --git 
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/HDFSUtils.java
 
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/HDFSUtils.java
index 2ce0383..f3c6ea3 100644
--- 
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/HDFSUtils.java
+++ 
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/HDFSUtils.java
@@ -42,18 +42,19 @@
 import org.apache.hadoop.mapred.SequenceFileInputFormat;
 import org.apache.hadoop.mapred.TextInputFormat;
 import 
org.apache.hyracks.algebricks.common.constraints.AlgebricksAbsolutePartitionConstraint;
-import org.apache.hyracks.api.context.ICCContext;
 import org.apache.hyracks.api.exceptions.HyracksException;
+import org.apache.hyracks.control.cc.ClusterControllerService;
 import org.apache.hyracks.hdfs.scheduler.Scheduler;
 
 public class HDFSUtils {
 
 public static Scheduler initializeHDFSScheduler() {
-ICCContext ccContext = 
AsterixAppContextInfo.INSTANCE.getCCApplicationContext().getCCContext();
+ClusterControllerService ccs = (ClusterControllerService) 
AsterixAppContextInfo.INSTANCE
+.getCCApplicationContext().getControllerService();
 Scheduler scheduler = null;
 try {
-scheduler = new 
Scheduler(ccContext.getClusterControllerInfo().getClientNetAddress(),
-ccContext.getClusterControllerInfo().getClientNetPort());
+scheduler = new 
Scheduler(ccs.getClusterControllerInfo().getClientNetAddress(),
+ccs.getClusterControllerInfo().getClientNetPort());
 } catch (HyracksException e) {
 throw new IllegalStateException("Cannot obtain hdfs scheduler");
 }
@@ -61,11 +62,12 @@
 }
 
 public static IndexingScheduler initializeIndexingHDFSScheduler() {
-ICCContext ccContext = 
AsterixAppContextInfo.INSTANCE.getCCApplicationContext().getCCContext();
+ClusterControllerService ccs = (ClusterControllerService) 
AsterixAppContextInfo.INSTANCE
+.getCCApplicationContext().getControllerService();
 IndexingScheduler scheduler = null;
 try {
-scheduler = new 
IndexingScheduler(ccContext.getClusterControllerInfo().getClientNetAddress(),
-ccContext.getClusterControllerInfo().getClientNetPort());
+scheduler = new 
IndexingScheduler(ccs.getClusterControllerInfo().getClientNetAddres

Change in asterixdb[master]: Remove ICCContext

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1322

to look at the new patch set (#2).

Change subject: Remove ICCContext
..

Remove ICCContext

Change-Id: I6f6a769741f14e91bcd4b970b4a022c0a453d380
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplicationEntryPoint.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/HDFSUtils.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/util/RuntimeUtils.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/application/ICCApplicationContext.java
D 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/context/ICCContext.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/application/CCApplicationContext.java
D 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/GetIpAddressNodeNameMapWork.java
8 files changed, 35 insertions(+), 155 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/22/1322/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1322
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6f6a769741f14e91bcd4b970b4a022c0a453d380
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 

Change in asterixdb[master]: Remove ICCContext

[abdullah alamoudi (Code Review)]

Hello Michael Blow, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1322

to look at the new patch set (#3).

Change subject: Remove ICCContext
..

Remove ICCContext

Change-Id: I6f6a769741f14e91bcd4b970b4a022c0a453d380
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplicationEntryPoint.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/HDFSUtils.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/util/RuntimeUtils.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/application/ICCApplicationContext.java
R 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/service/IClusterControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/application/CCApplicationContext.java
D 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/GetIpAddressNodeNameMapWork.java
8 files changed, 60 insertions(+), 136 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/22/1322/3
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1322
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6f6a769741f14e91bcd4b970b4a022c0a453d380
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 

Change in asterixdb[master]: Remove ICCContext

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Remove ICCContext
..


Patch Set 2:

The motivation is to get rid of yet another confusing context.
The ICCContext is just a wrapper for cluster topology (which I think should go 
away since it is unused right now but Till and Yingyi think otherwise). In 
fact, the cluster topology is just a wrapper around NetworkSwitch. why don't we 
use network switch directly? I have no idea.

In addition, this change gets rid of a confusing way of getting the ip-nodes 
mapping. with the context, it creates a work and put it in the work queue, once 
it is time to execute, it will simply fill the map with the mapping at the 
cluster controller service.

In the new patch, I got rid of the casting by introducing an interface for 
ClusterControllerService which to me is much better than having the confusing 
ICCContext.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1322
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6f6a769741f14e91bcd4b970b4a022c0a453d380
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Remove ICCContext

[abdullah alamoudi (Code Review)]

Hello Michael Blow, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1322

to look at the new patch set (#4).

Change subject: Remove ICCContext
..

Remove ICCContext

Change-Id: I6f6a769741f14e91bcd4b970b4a022c0a453d380
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplicationEntryPoint.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/HDFSUtils.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/util/RuntimeUtils.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/application/ICCApplicationContext.java
R 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/service/IClusterControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/application/CCApplicationContext.java
D 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/GetIpAddressNodeNameMapWork.java
8 files changed, 61 insertions(+), 138 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/22/1322/4
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1322
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6f6a769741f14e91bcd4b970b4a022c0a453d380
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 

Change in asterixdb[master]: Unify runtime type exceptions by using error code and messag...

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Unify runtime type exceptions by using error code and message 
template.
..


Patch Set 8:

(1 comment)

Can we list the new exception classes created in the change commit message? can 
we add some explanation to the list as a guide to the review and a reference 
for subsequent commits (maybe it should go somewhere in the code)?

https://asterix-gerrit.ics.uci.edu/#/c/1313/8/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/MetaFunctionToMetaVariableRule.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/MetaFunctionToMetaVariableRule.java:

undo this


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1313
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ie4fff8f5e64ffb027910a4899c0246b37ed5bce7
Gerrit-PatchSet: 8
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Yingyi Bu 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Remove ICCContext

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Remove ICCContext
..


Patch Set 4:

(3 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1322/4/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/util/RuntimeUtils.java
File 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/util/RuntimeUtils.java:

Line 58: map.putAll(((ClusterControllerService) 
AsterixAppContextInfo.INSTANCE.getCCApplicationContext()
> Still cast here.
Done


https://asterix-gerrit.ics.uci.edu/#/c/1322/4/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/service/IClusterControllerService.java
File 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/service/IClusterControllerService.java:

Line 39: Map> getIpAddressNodeNameMap();
> Why do we need this new interface?
This interface is to avoid the casting. 

1) These methods are already in ClusterControllerService. it is not getting 
bigger than it is. In fact, it is getting smaller.
What makes it big, is the ICCContext inner implementation in there.

2) This change in no way enable the user to do anything they couldn't do 
before. Again, it is only here to get rid of the cast and IMO, we need two 
different interfaces for ClusterControllerService and NodeControllerService 
since they are both fundamentally different and we need to avoid castings.


https://asterix-gerrit.ics.uci.edu/#/c/1322/4/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
File 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java:

Line 357: return ipAddressNodeNameMap;
> How do you handle dynamic cluster membership now?
whenever register node is called, this map is updated. so it is not always 
empty.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1322
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6f6a769741f14e91bcd4b970b4a022c0a453d380
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Remove ICCContext

[abdullah alamoudi (Code Review)]

Hello Michael Blow, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1322

to look at the new patch set (#5).

Change subject: Remove ICCContext
..

Remove ICCContext

Change-Id: I6f6a769741f14e91bcd4b970b4a022c0a453d380
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplicationEntryPoint.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/HDFSUtils.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/util/RuntimeUtils.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/application/ICCApplicationContext.java
R 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/service/IClusterControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/application/CCApplicationContext.java
D 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/GetIpAddressNodeNameMapWork.java
8 files changed, 61 insertions(+), 139 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/22/1322/5
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1322
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6f6a769741f14e91bcd4b970b4a022c0a453d380
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 

Change in asterixdb[master]: Unify runtime type exceptions by using error code and messag...

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Unify runtime type exceptions by using error code and message 
template.
..


Patch Set 8:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1313/8/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/MetaFunctionToMetaVariableRule.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/MetaFunctionToMetaVariableRule.java:

> I changed "once" to "one", which fixes the error message.
I see.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1313
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ie4fff8f5e64ffb027910a4899c0246b37ed5bce7
Gerrit-PatchSet: 8
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Yingyi Bu 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Remove Server Context

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Remove Server Context
..


Patch Set 1:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1320/1/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/service/IControllerService.java
File 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/service/IControllerService.java:

Line 42: File getBaseDir();
> Since there was no ServerContext here, it seems that we also shouldn't need
there was no server context but all the sub interfaces had this. so when there 
is a method which returns an IControllerService, one still needs to cast before 
calling getBaseDir().


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1320
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4261cf44410b498e84ffcac5e217281a306a7f03
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Remove Server Context

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Remove Server Context
..


Patch Set 1:

too many contexts = confusion. not only for existing developers but also for 
new ones.
and those are wrappers with unused methods.

right now, on CC, we have:
1. ServerContext (wrapper for baseDir)
2. ICCContext (wrapper for unused cluster topology?)
3. ICCApplicationContext (the only used one)

We can't add contexts for potential future use. Once we have the need for 
additional contexts, we can add them.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1320
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4261cf44410b498e84ffcac5e217281a306a7f03
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Remove Server Context

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Remove Server Context
..


Patch Set 1:

For the SonarQube argument, show me a place where having the
ICCContext or the ServerContext will get rid of that.

We have that issue in so many other places

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1320
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4261cf44410b498e84ffcac5e217281a306a7f03
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Remove Server Context

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Remove Server Context
..


Patch Set 1:

Also, forgot the IHyracksRootContext which was a wrapper for IOManager :S

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1320
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4261cf44410b498e84ffcac5e217281a306a7f03
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Remove Server Context

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Remove Server Context
..


Patch Set 1:

NoddControllerService and ClusterControllerService are semi kitchen sinks right 
now and I am stopping that from happening.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1320
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4261cf44410b498e84ffcac5e217281a306a7f03
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No