[
https://issues.apache.org/jira/browse/SENTRY-957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hao Hao updated SENTRY-957:
---------------------------
Description:
Right now if there is an error condition in evaluating paths in
MetastoreCacheInitializer in one of the tasks, we throw a Runtime Exception (in
Sentry-888). Changed the behavior to be:
* Retry the failed tasks x times. Each retry has a y millis wait duration. x
and y are based upon the user config
(sentry.hdfs.sync.metastore.cache.retry.max.num and
sentry.hdfs.sync.metastore.cache.retry.wait.duration.millis).
* After retry failure, throw exception or sync incomplete paths update based on
user config (sentry.hdfs.sync.metastore.cache.fail.on.partial.update). The
default values is fail on partial update, which will throw runtime exception.
{noformat}
for (Future<CallResult> result : results) {
CallResult callResult = result.get();
// Fail the HMS startup if tasks are not all successful and
// fail on partial updates flag is set in the config.
if (callResult.getSuccessStatus() == false && failOnRetry) {
throw new RuntimeException(callResult.getFailure());
}
}
{noformat}
was:
Right now if there is an error condition in evaluating paths in
MetastoreCacheInitializer in one of the tasks, we throw a Runtime Exception (in
Sentry-888). Changed the behavior to be:
* Retry the failed tasks x times. Each retry has a y millis wait duration. x
and y are based upon the user config
(SENTRY_HDFS_SYNC_METASTORE_CACHE_RETRY_MAX_NUM and
SENTRY_HDFS_SYNC_METASTORE_CACHE_RETRY_WAIT_DURAION_IN_MILLIS).
* After retry failure, throw exception or sync incomplete paths update based on
user config (SENTRY_HDFS_SYNC_METASTORE_CACHE_FAIL_ON_PARTIAL_UPDATE). The
default values is fail on partial update, which will throw runtime exception.
{noformat}
for (Future<CallResult> result : results) {
CallResult callResult = result.get();
// Fail the HMS startup if tasks are not all successful and
// fail on partial updates flag is set in the config.
if (callResult.getSuccessStatus() == false && failOnRetry) {
throw new RuntimeException(callResult.getFailure());
}
}
{noformat}
> Exceptions in MetastoreCacheInitializer should probably not prevent HMS from
> starting up
> ----------------------------------------------------------------------------------------
>
> Key: SENTRY-957
> URL: https://issues.apache.org/jira/browse/SENTRY-957
> Project: Sentry
> Issue Type: Bug
> Components: Hdfs Plugin
> Affects Versions: 1.7.0
> Reporter: Sravya Tirukkovalur
> Assignee: Hao Hao
> Fix For: 1.7.0
>
> Attachments: SENTRY-957.001.patch, SENTRY-957.002.patch,
> SENTRY-957.003.patch
>
>
> Right now if there is an error condition in evaluating paths in
> MetastoreCacheInitializer in one of the tasks, we throw a Runtime Exception
> (in Sentry-888). Changed the behavior to be:
> * Retry the failed tasks x times. Each retry has a y millis wait duration. x
> and y are based upon the user config
> (sentry.hdfs.sync.metastore.cache.retry.max.num and
> sentry.hdfs.sync.metastore.cache.retry.wait.duration.millis).
> * After retry failure, throw exception or sync incomplete paths update based
> on user config (sentry.hdfs.sync.metastore.cache.fail.on.partial.update). The
> default values is fail on partial update, which will throw runtime exception.
> {noformat}
> for (Future<CallResult> result : results) {
> CallResult callResult = result.get();
> // Fail the HMS startup if tasks are not all successful and
> // fail on partial updates flag is set in the config.
> if (callResult.getSuccessStatus() == false && failOnRetry) {
> throw new RuntimeException(callResult.getFailure());
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
