vamsikarnika commented on code in PR #13803:
URL: https://github.com/apache/hudi/pull/13803#discussion_r2324275372
##########
hudi-common/src/main/java/org/apache/hudi/common/fs/FSUtils.java:
##########
@@ -446,9 +447,9 @@ public static boolean isLogFile(StoragePath logPath) {
}
public static boolean isLogFile(String fileName) {
- if (fileName.contains(LOG_FILE_EXTENSION)) {
+ if (fileName.startsWith(LOG_FILE_START_WITH_CHARACTER) &&
fileName.contains(LOG_FILE_EXTENSION)) {
Review Comment:
From the tests I see that we're creating files with `.archive` extension
which should be skipped by isLogFile check
```
@Test
public void testArchiveLogFileName() {
String partitionPath = "2022/11/04/";
String fileName = "commits";
String logFile = FSUtils.makeLogFileName(fileName, ".archive", "", 2,
"1-0-1");
StoragePath path = new StoragePath(new StoragePath(partitionPath),
logFile);
```
https://github.com/apache/hudi/blob/48dfe281a3141ce828f7d83153070799c4a96d1c/hudi-hadoop-common/src/test/java/org/apache/hudi/common/fs/TestFSUtils.java#L285
##########
hudi-common/src/main/java/org/apache/hudi/common/fs/FSUtils.java:
##########
@@ -446,9 +447,9 @@ public static boolean isLogFile(StoragePath logPath) {
}
public static boolean isLogFile(String fileName) {
- if (fileName.contains(LOG_FILE_EXTENSION)) {
+ if (fileName.startsWith(LOG_FILE_START_WITH_CHARACTER) &&
fileName.contains(LOG_FILE_EXTENSION)) {
Review Comment:
From the tests I see that we're creating files with `.archive` extension
which should be skipped by isLogFile check
```java
@Test
public void testArchiveLogFileName() {
String partitionPath = "2022/11/04/";
String fileName = "commits";
String logFile = FSUtils.makeLogFileName(fileName, ".archive", "", 2,
"1-0-1");
StoragePath path = new StoragePath(new StoragePath(partitionPath),
logFile);
```
https://github.com/apache/hudi/blob/48dfe281a3141ce828f7d83153070799c4a96d1c/hudi-hadoop-common/src/test/java/org/apache/hudi/common/fs/TestFSUtils.java#L285
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
