Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 10:

Thanks Mike :)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Michael Blow (Code Review)]

Michael Blow has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 9: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 9
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Michael Blow (Code Review)]

Michael Blow has submitted this change and it was merged.

Change subject: Cast Record Before Insert for Open Indexes
..


Cast Record Before Insert for Open Indexes

This change modifies the insert plan to perform cast for open
indexes before inserting to primary index. This avoids modification
of indexes and then doing complex work to try and undo the insert.
In addition, feeds can simply log and remove tuples which have
open fields of incompatible type.

Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Reviewed-on: https://asterix-gerrit.ics.uci.edu/997
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Michael Blow 
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java
A asterixdb/asterix-app/data/external-parser/dropbox2/jobads3.txt
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/servlet/QueryServiceServlet.java
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.1.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.2.lib.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.3.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.4.update.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.5.query.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.6.lib.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.7.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.1.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.2.lib.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.3.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.4.update.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.5.query.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.6.lib.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.7.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInsertDeleteOperatorNodePushable.java
R 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/exceptions/ErrorCode.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FeedExceptionHandler.java
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
M 
hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java
M 
hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/std/AssignRuntimeFactory.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/ErrorCode.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/HyracksDataException.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/utils/ExceptionUtils.java
30 

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 9:

I agree. I added them and even I am confused :)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 9
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 9:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/153/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 9
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 9:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/1987/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 9
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 9:

WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack

PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 9
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Till Westmann (Code Review)]

Hello Yingyi Bu, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/997

to look at the new patch set (#9).

Change subject: Cast Record Before Insert for Open Indexes
..

Cast Record Before Insert for Open Indexes

This change modifies the insert plan to perform cast for open
indexes before inserting to primary index. This avoids modification
of indexes and then doing complex work to try and undo the insert.
In addition, feeds can simply log and remove tuples which have
open fields of incompatible type.

Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java
A asterixdb/asterix-app/data/external-parser/dropbox2/jobads3.txt
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/servlet/QueryServiceServlet.java
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.1.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.2.lib.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.3.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.4.update.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.5.query.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.6.lib.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.7.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.1.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.2.lib.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.3.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.4.update.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.5.query.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.6.lib.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.7.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInsertDeleteOperatorNodePushable.java
R 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/exceptions/ErrorCode.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FeedExceptionHandler.java
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
M 
hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java
M 
hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/std/AssignRuntimeFactory.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/ErrorCode.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/HyracksDataException.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/utils/ExceptionUtils.java
30 files changed, 8,088 insertions(+), 448 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Till Westmann (Code Review)]

Till Westmann has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 8: Code-Review+2

I think that I got confused by the big number of added constructors. 

I think that we should reduce that number again in the future as we make better 
and more consistent use of error codes and components.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 8
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Yingyi Bu (Code Review)]

Yingyi Bu has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 8: Code-Review+1

LGTM.
Not sure if that's still a problem:
"Seems that $1s would be replaced by 'e', not 'tupleIndex'. Is that intended?". 
 Leave that to Till.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 8
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 8: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/140/ : 
SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 8
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 8:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/1973/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 8
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 8:

I believe that I have addressed all comments. 
Once this gets merged, I can rebase my other almost ready change (For the 
extensible grammar)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 8
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 8:

WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack

PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 8
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 7:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/1971/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 7
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 7:

WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack

PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 7
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 7:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/1970/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 7
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 6:

(4 comments)

https://asterix-gerrit.ics.uci.edu/#/c/997/6/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java:

Line 251: LogicalVariable castedRecVar = context.newVar();
> move line 250-252 into the next if block?
Done


https://asterix-gerrit.ics.uci.edu/#/c/997/6/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInsertDeleteOperatorNodePushable.java
File 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInsertDeleteOperatorNodePushable.java:

Line 145: }
> This indicates that we haven't gotten this part of exception handling right
Indeed.


https://asterix-gerrit.ics.uci.edu/#/c/997/6/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
File 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java:

Line 238: requiredTypeTag.toString());
> Don't need the toString() here ..
Done


https://asterix-gerrit.ics.uci.edu/#/c/997/6/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/std/AssignRuntimeFactory.java
File 
hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/std/AssignRuntimeFactory.java:

Line 181: "Error evaluating tuple %1$s in 
AssignRuntime", e, tupleIndex);
> Seems that $1s would be replaced by 'e', not 'tupleIndex'. 
That is weird. for me, $1 get replaced by tupleIndex. Anyway, I made it more 
clear.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Yingyi Bu (Code Review)]

Yingyi Bu has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 6: Code-Review+1

(1 comment)

Just one minor comment.

https://asterix-gerrit.ics.uci.edu/#/c/997/6/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java:

Line 251: LogicalVariable castedRecVar = context.newVar();
move line 250-252 into the next if block?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Till Westmann (Code Review)]

Till Westmann has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 6: Code-Review+1

(3 comments)

Still needs Yingyi's rewriting expertise ...

https://asterix-gerrit.ics.uci.edu/#/c/997/6/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInsertDeleteOperatorNodePushable.java
File 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInsertDeleteOperatorNodePushable.java:

Line 145: }
This indicates that we haven't gotten this part of exception handling right 
yet. Needs to be fixed, but not in this change.


https://asterix-gerrit.ics.uci.edu/#/c/997/6/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
File 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java:

Line 238: requiredTypeTag.toString());
Don't need the toString() here ..


https://asterix-gerrit.ics.uci.edu/#/c/997/6/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/std/AssignRuntimeFactory.java
File 
hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/std/AssignRuntimeFactory.java:

Line 181: "Error evaluating tuple %1$s in 
AssignRuntime", e, tupleIndex);
Seems that $1s would be replaced by 'e', not 'tupleIndex'. 
Is that intended?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 6: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/128/ : 
SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 6:

WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack

PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 5:

WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack

PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 5:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/1954/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/997

to look at the new patch set (#5).

Change subject: Cast Record Before Insert for Open Indexes
..

Cast Record Before Insert for Open Indexes

This change modifies the insert plan to perform cast for open
indexes before inserting to primary index. This avoids modification
of indexes and then doing complex work to try and undo the insert.
In addition, feeds can simply log and remove tuples which have
open fields of incompatible type.

Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java
A asterixdb/asterix-app/data/external-parser/dropbox2/jobads3.txt
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.1.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.2.lib.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.3.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.4.update.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.5.query.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.6.lib.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.7.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.1.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.2.lib.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.3.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.4.update.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.5.query.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.6.lib.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.7.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInsertDeleteOperatorNodePushable.java
R 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/exceptions/ErrorCode.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FeedExceptionHandler.java
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
M 
hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java
M 
hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/std/AssignRuntimeFactory.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/ErrorCode.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/HyracksDataException.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/utils/ExceptionUtils.java
29 files changed, 8,077 insertions(+), 443 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/97/997/5
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit 

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 4:

WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack

PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 4:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/1952/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/997

to look at the new patch set (#4).

Change subject: Cast Record Before Insert for Open Indexes
..

Cast Record Before Insert for Open Indexes

This change modifies the insert plan to perform cast for open
indexes before inserting to primary index. This avoids modification
of indexes and then doing complex work to try and undo the insert.
In addition, feeds can simply log and remove tuples which have
open fields of incompatible type.

Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/ExternalDataLookupPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceRandomPartitioningFeedComputationRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java
A 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/ResolveVariableRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SetClosedRecordConstructorsRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/AqlPlusExpressionToPlanTranslator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/SqlppExpressionToPlanTranslator.java
M asterixdb/asterix-algebra/src/main/javacc/AQLPlus.jj
A asterixdb/asterix-app/data/external-parser/dropbox2/jobads3.txt
M asterixdb/asterix-app/pom.xml
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/AsterixAppRuntimeContext.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/SessionConfig.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/servlet/APIServlet.java
D 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/servlet/JSONUtil.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/servlet/QueryResultAPIServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/servlet/QueryServiceServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalIndexingOperations.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryClassLoader.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryUtils.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/aql/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplicationEntryPoint.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplicationEntryPoint.java
M asterixdb/asterix-app/src/main/java/org/apache/asterix/result/ResultUtils.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/external/TestLibrarian.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/ExecutionTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/ExecutionTestUtil.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/RepeatedTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/ParserTestExecutor.java
M 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/query-issue697.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/subquery/exists.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/subquery/in.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/subquery/in_correlated.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/subquery/not_exists.sqlpp
M 
asterixdb/asterix-app/src/test/resources/optimizerts/queries_sqlpp/disjunction-to-join-delete-1.sqlpp
M 
asterixdb/asterix-app/src/test/resources/optimizerts/queries_sqlpp/disjunction-to-join-delete-2.sqlpp
M 
asterixdb/asterix-app/src/test/resources/optimizerts/queries_sqlpp/disjunction-to-join-delete-3.sqlpp
M 
asterixdb/asterix-app/src/test/resources/optimizerts/queries_sqlpp/scan-delete-all.sqlpp
M 
asterixdb/asterix-app/src/test/resources/optimizerts/queries_sqlpp/scan-delete-rtree-secondary-index.sqlpp
M 
asterixdb/asterix-app/src/test/resources/optimizerts/queries_sqlpp/scan-delete.sqlpp
M 
asterixdb/asterix-app/src/test/resources/optimizerts/queries_sqlpp/skip-index/dont-skip-primary-index-search-in-delete.sqlpp
M 
asterixdb/asterix-app/src/test/resources/optimizerts/queries_sqlpp/skip-index/skip-ngram-index-search-in-delete.sqlpp
M 
asterixdb/asterix-app/src/test/resources/optimizerts/queries_sqlpp/skip-index/skip-rtree-index-search-in-delete.sqlpp
M 

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Till Westmann (Code Review)]

Till Westmann has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 3:

(7 comments)

Mostly comments on exception handling/error codes.

https://asterix-gerrit.ics.uci.edu/#/c/997/3/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInsertDeleteOperatorNodePushable.java
File 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInsertDeleteOperatorNodePushable.java:

Line 134: throw new HyracksDataException(
Probably we should not wrap this exception as the ErrorCode.TUPLE is not 
appropriate here? Having a different operation here sounds more like a system 
error than an error in the data (which I believe is the case for TUPLE). Is 
that right?


https://asterix-gerrit.ics.uci.edu/#/c/997/3/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
File 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java:

Line 251: // check unmatched fields in the input type
Here we should use the message formatting mechanism HyracksDataException and 
pass the types a error objects:

throw new HyracksDataException(
HyracksDataException.NONE,
ErrorCode.CAST,
"Field type %1$s cannot be promoted to type %2$s",
inputTypeTag,
requiredTypeTag);


https://asterix-gerrit.ics.uci.edu/#/c/997/3/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/std/AssignRuntimeFactory.java
File 
hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/std/AssignRuntimeFactory.java:

Line 101: private int t = 0;
What is 't'? Could we have a more explicit name?


Line 173: try {
Can we just remove this try-catch and ...


Line 183: } catch (HyracksDataException hde) {
... 

catch (HyracksDataException|AlgebricksException e) {

instead?


https://asterix-gerrit.ics.uci.edu/#/c/997/3/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/ErrorCode.java
File 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/ErrorCode.java:

Line 30: 
A few comments on the error codes:

1) CAST is only used in AsterixDB. So we should create a new error code 
namespace for AsterixDB and a class for ErrorCodes in AsterixDB.

2) I think that the error codes should be more descriptive so that readers of 
the code can relate to what's happening. E.g.
ERROR_PROCESSING_TUPLE instead of TUPLE or
CANNOT_PROMOTE_VALUE or CANNOT_CAST_VALUE instead of CAST.

3) I would also remove unused codes.

Does this make sense?


https://asterix-gerrit.ics.uci.edu/#/c/997/3/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/utils/ExceptionUtils.java
File 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/utils/ExceptionUtils.java:

Line 60: newExceptions.add(new HyracksDataException(e, nodeId));
I guess this could use a code like "FAILURE_ON_NODE" to explain the nodeId?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 3: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/101/ : 
SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 3:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/101/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 3:

WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack

PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 3:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/1918/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/997

to look at the new patch set (#3).

Change subject: Cast Record Before Insert for Open Indexes
..

Cast Record Before Insert for Open Indexes

This change modifies the insert plan to perform cast for open
indexes before inserting to primary index. This avoids modification
of indexes and then doing complex work to try and undo the insert.
In addition, feeds can simply log and remove tuples which have
open fields of incompatible type.

Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java
A asterixdb/asterix-app/data/external-parser/dropbox2/jobads3.txt
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.1.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.2.lib.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.3.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.4.update.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.5.query.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.6.lib.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.7.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.1.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.2.lib.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.3.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.4.update.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.5.query.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.6.lib.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.7.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInsertDeleteOperatorNodePushable.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FeedExceptionHandler.java
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
M 
hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java
M 
hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/std/AssignRuntimeFactory.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/ErrorCode.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/HyracksDataException.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/utils/ExceptionUtils.java
28 files changed, 8,058 insertions(+), 419 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/97/997/3
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: 

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 2:

WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack

PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 2:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/1917/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[abdullah alamoudi (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/997

to look at the new patch set (#2).

Change subject: Cast Record Before Insert for Open Indexes
..

Cast Record Before Insert for Open Indexes

This change modifies the insert plan to perform cast for open
indexes before inserting to primary index. This avoids modification
of indexes and then doing complex work to try and undo the insert.
In addition, feeds can simply log and remove tuples which have
open fields of incompatible type.

Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java
A asterixdb/asterix-app/data/external-parser/dropbox2/jobads3.txt
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.1.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.2.lib.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.3.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.4.update.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.5.query.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.6.lib.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.7.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.1.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.2.lib.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.3.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.4.update.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.5.query.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.6.lib.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.7.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInsertDeleteOperatorNodePushable.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FeedExceptionHandler.java
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
M 
hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java
M 
hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/std/AssignRuntimeFactory.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/ErrorCode.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/HyracksDataException.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/utils/ExceptionUtils.java
28 files changed, 8,052 insertions(+), 413 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/97/997/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: 

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 1:

(5 comments)

I think that I have addressed all the comments.
Let me know if I missed anything.

https://asterix-gerrit.ics.uci.edu/#/c/997/1/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FeedExceptionHandler.java
File 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FeedExceptionHandler.java:

Line 52: public ByteBuffer handle(HyracksDataException th, ByteBuffer 
frame) {
> Can we use an integer error code instead of a new Exception type to handle 
Done


https://asterix-gerrit.ics.uci.edu/#/c/997/1/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
File 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java:

Line 236: "Open field type " + inputTypeTag + " 
can't be promoted to type " + requiredTypeTag);
> "Open field"-> "Field",  the input field could be in the closed part.
Done


https://asterix-gerrit.ics.uci.edu/#/c/997/1/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java
File 
hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java:

Line 241: if (hasInsert && InlineVariablesRule.opIsPostInsert(op)) {
> remove "InlineVariablesRule."
Shouldn't static methods be called using the class name?


Line 299: return false;
> This doesn't seem right. In that case, we missed the case the operator is a
This was used to check whether one of the parents of this operator is an insert 
operator. In such case, we will never have a join or any operator with two 
parents.

variable names are misleading a bit. anyway, I replaced this logic with the 
better way to stop inlining the variables by calling context add to not inline 
vars.


https://asterix-gerrit.ics.uci.edu/#/c/997/1/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/std/AssignRuntimeFactory.java
File 
hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/std/AssignRuntimeFactory.java:

Line 184: throw new FrameDataException(t, hde);
> Why any exceptions during assign x:= expr becomes a FormatDataException?   
Done


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Yingyi Bu (Code Review)]

Yingyi Bu has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 1:

(5 comments)

1. I do not understand why the change must be in InlineVariableRule?  Could we 
have a separate rule to perform this?  Now InlineVariableRule becomes a bit 
harder to understand/read.

2. I prefer to use error code instead of new Exception types to deal with 
mismatched input records.

Detailed comments are inlined.

https://asterix-gerrit.ics.uci.edu/#/c/997/1/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FeedExceptionHandler.java
File 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FeedExceptionHandler.java:

Line 52: public ByteBuffer handle(HyracksDataException th, ByteBuffer 
frame) {
Can we use an integer error code instead of a new Exception type to handle this?


https://asterix-gerrit.ics.uci.edu/#/c/997/1/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
File 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java:

Line 236: "Open field type " + inputTypeTag + " 
can't be promoted to type " + requiredTypeTag);
"Open field"-> "Field",  the input field could be in the closed part.

I prefer to use HyracksDataException here with an error code, instead of a 
using a new Exception type.


https://asterix-gerrit.ics.uci.edu/#/c/997/1/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java
File 
hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java:

Line 241: if (hasInsert && InlineVariablesRule.opIsPostInsert(op)) {
remove "InlineVariablesRule."


Line 299: return false;
This doesn't seem right. In that case, we missed the case the operator is a 
join.


https://asterix-gerrit.ics.uci.edu/#/c/997/1/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/std/AssignRuntimeFactory.java
File 
hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/std/AssignRuntimeFactory.java:

Line 184: throw new FrameDataException(t, hde);
Why any exceptions during assign x:= expr becomes a FormatDataException?   
Mismatched records could only be one case of many possible cases.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 1:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/997/1/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/FrameDataException.java
File 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/FrameDataException.java:

Line 21: public class FrameDataException extends HyracksDataException {
> What is the point of moving this Exception to Hyracks? It seems that single
Till, the assign runtime needs to throw this type of exception in order for the 
feed pipeline to handle it. Hence, I had to move it somewhere where it is 
accessible by both the assign operator, the insert operator and the feed 
exception handler.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 1: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/82/ : 
SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 1:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/82/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Cast Record Before Insert for Open Indexes
..


Patch Set 1:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/1879/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: No

Change in asterixdb[master]: Cast Record Before Insert for Open Indexes

[abdullah alamoudi (Code Review)]

abdullah alamoudi has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/997

Change subject: Cast Record Before Insert for Open Indexes
..

Cast Record Before Insert for Open Indexes

This change modifies the insert plan to perform cast for open
indexes before inserting to primary index. This avoids modification
of indexes and then doing complex work to try and undo the insert.
In addition, feeds can simply log and remove tuples which have
open fields of incompatible type.

Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java
A asterixdb/asterix-app/data/external-parser/dropbox2/jobads3.txt
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.1.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.2.lib.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.3.ddl.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.4.update.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.5.query.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.6.lib.aql
C 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.7.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInsertDeleteOperatorNodePushable.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FeedExceptionHandler.java
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
C 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/RecordCastException.java
M 
hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java
M 
hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/std/AssignRuntimeFactory.java
R 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/FrameDataException.java
19 files changed, 7,816 insertions(+), 199 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/97/997/1


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/997
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi