difin commented on code in PR #3559:
URL: https://github.com/apache/hive/pull/3559#discussion_r971190432
##########
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSplit.java:
##########
@@ -104,28 +104,43 @@ public OrcSplit(Path path, Object fileId, long offset,
long length, String[] hos
this.isOriginal = isOriginal;
this.hasBase = hasBase;
this.rootDir = rootDir;
- this.deltas.addAll(filterDeltasByBucketId(deltas,
AcidUtils.parseBucketId(path)));
+ int bucketId = AcidUtils.parseBucketId(path);
+ AcidUtils.ParsedDeltaLight parentDelta =
AcidUtils.ParsedDeltaLight.parse(getPath().getParent());
Review Comment:
Hi @deniskuzZ,
Many tests failed with the change of using
AcidUtils.ParsedDeltaLight.parse() instead of
AcidUtils.parseBaseOrDeltaBucketFilename(). As I understand the split is not
always a delta folder, it can be some older format not supported by
ParsedDeltaLight. I saw that ParsedDeltaLight.parse() is used in some cases
internally in AcidUtils.parseBaseOrDeltaBucketFilename(), but not always. Can
you please advise if I should revert to using
AcidUtils.parseBaseOrDeltaBucketFilename() that worked in all cases or there is
some better way?
https://github.com/apache/hive/blob/f6bd0eb80767adfa9ce9f47a6d02a4940903effb/ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java#L538-L552
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
