HarshitGupta11 commented on a change in pull request #2771:
URL: https://github.com/apache/hive/pull/2771#discussion_r783023540
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java
##########
@@ -1230,12 +1232,16 @@ else if(statementId != parsedDelta.statementId) {
* that for any dir, either all files are acid or all are not.
*/
public static ParsedDelta parsedDelta(Path deltaDir, FileSystem fs) throws
IOException {
- return parsedDelta(deltaDir, fs, null);
+ return parsedDelta(deltaDir, fs, null, false, -1);
}
- private static ParsedDelta parsedDelta(Path deltaDir, FileSystem fs,
HdfsDirSnapshot dirSnapshot)
+ private static ParsedDelta parsedDelta(Path deltaDir, FileSystem fs,
HdfsDirSnapshot dirSnapshot,
+ boolean canTrim, long highWaterMark)
throws IOException {
ParsedDeltaLight deltaLight = ParsedDeltaLight.parse(deltaDir);
+ if(canTrim && !(deltaLight.minWriteId >= highWaterMark)){
Review comment:
Yeah, it will not read the deltas that are not part of the current
transaction or were written before the current transaction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
