abdullah alamoudi has posted comments on this change. Change subject: Support Change Feeds and Ingestion of Records with MetaData (WIP-rebased) ......................................................................
Patch Set 2: (17 comments) https://asterix-gerrit.ics.uci.edu/#/c/620/2/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/metadata/IMetadataProvider.java File algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/metadata/IMetadataProvider.java: Line 70: LogicalVariable payLoadVar, List<LogicalVariable> additionalNonKeyFields, RecordDescriptor recordDesc, > remove meta var from interface Done https://asterix-gerrit.ics.uci.edu/#/c/620/2/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/InsertDeleteUpsertOperator.java File algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/InsertDeleteUpsertOperator.java: Line 60: > make two constructors and remove setMeta Done https://asterix-gerrit.ics.uci.edu/#/c/620/2/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/UnnestOperator.java File algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/UnnestOperator.java: Line 35: > Remove from this interface and add when performing unnest to datascan Done https://asterix-gerrit.ics.uci.edu/#/c/620/2/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/BulkloadPOperator.java File algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/BulkloadPOperator.java: Line 105: propagatedSchema, typeEnv, primaryKeys, payload, additionalFilteringKeys, inputDesc, context, spec, > remove when changing the interface Removing this from here would force us to do some hackish solution like passing the meta variables inside dataSource. This is a mandatory change. If you look at the InsertDeleteUpsertOperator, you will see that it only has: 1 variable for payload. list of variables for PK. list of variables for filters. what about other fields which don't fall within any of those? hence, the addition of nonFiltering expressions. https://asterix-gerrit.ics.uci.edu/#/c/620/2/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/InsertDeleteUpsertPOperator.java File algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/InsertDeleteUpsertPOperator.java: Line 110: runtimeAndConstraints = mp.getInsertRuntime(dataSource, propagatedSchema, typeEnv, keys, payload, This is a mandatory change. If you look at the InsertDeleteUpsertOperator, you will see that it only has: 1 variable for payload. list of variables for PK. list of variables for filters. what about other fields which don't fall within any of those? hence, the addition of nonFiltering expressions. https://asterix-gerrit.ics.uci.edu/#/c/620/2/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitor.java File algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitor.java: Line 291: } > remove from here Done https://asterix-gerrit.ics.uci.edu/#/c/620/2/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/SetAlgebricksPhysicalOperatorsRule.java File algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/SetAlgebricksPhysicalOperatorsRule.java: Line 314: metaVariable = ((VariableReferenceExpression) metaExpr.getValue()).getVariableReference(); > revert this and move to asterix set physical operator rule Changed this to something that makes more sense. again, this change had to be here. https://asterix-gerrit.ics.uci.edu/#/c/620/2/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/comm/io/ArrayTupleBuilder.java File hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/comm/io/ArrayTupleBuilder.java: Line 36: */ > deprecate it Done Line 170: > change to ivaluereference and add a todo to get rid of this completely. Done https://asterix-gerrit.ics.uci.edu/#/c/620/2/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/file/FieldCursorForDelimitedDataParser.java File hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/file/FieldCursorForDelimitedDataParser.java: Line 332: > revert Done https://asterix-gerrit.ics.uci.edu/#/c/620/2/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/IndexInsertUpdateDeleteOperatorNodePushable.java File hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/IndexInsertUpdateDeleteOperatorNodePushable.java: Line 66: this.op = op; > remove comment Done https://asterix-gerrit.ics.uci.edu/#/c/620/2/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/TreeIndexCreateOperatorDescriptor.java File hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/TreeIndexCreateOperatorDescriptor.java: Line 45: super(spec, 0, 0, null, storageManager, lifecycleManagerProvider, fileSplitProvider, typeTraits, > revert Done https://asterix-gerrit.ics.uci.edu/#/c/620/2/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/util/LSMBTreeUtils.java File hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/util/LSMBTreeUtils.java: Line 170 > reverrt Done https://asterix-gerrit.ics.uci.edu/#/c/620/2/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/api/LSMOperationType.java File hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/api/LSMOperationType.java: Line 23: LOCKING_SEARCH, > remove Done https://asterix-gerrit.ics.uci.edu/#/c/620/2/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/AbstractDiskLSMComponent.java File hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/AbstractDiskLSMComponent.java: Line 47: case LOCKING_SEARCH: > remove locking search. Done https://asterix-gerrit.ics.uci.edu/#/c/620/2/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/AbstractMemoryLSMComponent.java File hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/AbstractMemoryLSMComponent.java: Line 74: case MODIFICATION: removed. https://asterix-gerrit.ics.uci.edu/#/c/620/2/pom.xml File pom.xml: Line 175: <exclude>**/*.prefs</exclude> > make it more specific. it is fine if there are multiple Done -- To view, visit https://asterix-gerrit.ics.uci.edu/620 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I3749349e2b9f1b03c8b310eb99d3f44d08be77df Gerrit-PatchSet: 2 Gerrit-Project: hyracks Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi <[email protected]> Gerrit-Reviewer: Ildar Absalyamov <[email protected]> Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Steven Jacobs <[email protected]> Gerrit-Reviewer: Till Westmann <[email protected]> Gerrit-Reviewer: abdullah alamoudi <[email protected]> Gerrit-HasComments: Yes
