Jackie-Jiang commented on a change in pull request #7860:
URL: https://github.com/apache/pinot/pull/7860#discussion_r762179049
##########
File path:
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/mutable/MutableSegmentImpl.java
##########
@@ -472,12 +472,14 @@ public boolean index(GenericRow row, @Nullable
RowMetadata rowMetadata)
throws IOException {
boolean canTakeMore;
if (isUpsertEnabled()) {
- row = handleUpsert(row, _numDocsIndexed);
-
- updateDictionary(row);
- addNewRow(row);
- // Update number of documents indexed at last to make the latest row
queryable
+ PartitionUpsertMetadataManager.RecordInfo recordInfo =
getRecordInfo(row, _numDocsIndexed);
+ GenericRow updatedRow =
_partitionUpsertMetadataManager.updateRecord(row, recordInfo);
+ updateDictionary(updatedRow);
+ addNewRow(updatedRow);
+ // Update number of documents indexed before handling the upsert
metadata so that the record becomes queryable
+ // once validated
canTakeMore = _numDocsIndexed++ < _capacity;
+ _partitionUpsertMetadataManager.addRecord(this, recordInfo);
Review comment:
The inconsistency across segments is hard to solve without locking, but
this PR can solve the inconsistency within the segment.
More importantly, with the current code the new record is always missing
before it is indexed, instead of immediately queryable when the upsert metadata
is updated, so I would count this as a bug which need to be fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
