lovemylover042 commented on issue #8100:
URL: https://github.com/apache/hudi/issues/8100#issuecomment-1757266624
@danny0405 I found delta commit became so slowly because it use secondary
filesystem view when got a bad response from remote timeline server. I think
the bad response was caused by compaction running at the same time and timeline
server was behind the client. Can i force sync local view if timeline server
was behind the client ?
-----------------------------------------------------------------
org.apache.hudi.timeline.service.RequestHandler line 501:
// TODO: set refreshCheck to be true when timeline server became behind
several times or some seconds
if (refreshCheck) {
long beginFinalCheck = System.currentTimeMillis();
if (isLocalViewBehind(context)) {
String errMsg =
"Last known instant from client was "
+
context.queryParam(RemoteHoodieTableFileSystemView.LAST_INSTANT_TS,
HoodieTimeline.INVALID_INSTANT_TS)
+ " but server has the following timeline "
+
viewManager.getFileSystemView(context.queryParam(RemoteHoodieTableFileSystemView.BASEPATH_PARAM))
.getTimeline().getInstants().collect(Collectors.toList());
throw new BadRequestResponse(errMsg);
}
long endFinalCheck = System.currentTimeMillis();
finalCheckTimeTaken = endFinalCheck - beginFinalCheck;
}
-----------------------------------------------------------------
Environment Description:
Hudi version : 0.10.1
Spark version : 3.0.1
Hadoop version : 3.1.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
