[
https://issues.apache.org/jira/browse/DRILL-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17609429#comment-17609429
]
ASF GitHub Bot commented on DRILL-8314:
---------------------------------------
jnturton commented on code in PR #2655:
URL: https://github.com/apache/drill/pull/2655#discussion_r979835243
##########
exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java:
##########
@@ -1094,6 +1095,35 @@ public static String bootDefaultFor(String name) {
new OptionDescription("Enables recursive files listing when querying the
`INFORMATION_SCHEMA.FILES` table or executing the SHOW FILES command. " +
"Default is false. (Drill 1.15+)"));
+ public static final String STORAGE_PLUGIN_ACCESS_ATTEMPTS =
"storage.plugin_access_attempts";
+ public static final PositiveLongValidator
STORAGE_PLUGIN_ACCESS_ATTEMPTS_VALIDATOR = new PositiveLongValidator(
+ STORAGE_PLUGIN_ACCESS_ATTEMPTS,
+ 10,
+ new OptionDescription(
+ "The maximum number of attempts that will be made to request metadata " +
+ "needed for query planning from a storage plugin."
+ )
+ );
+ public static final String STORAGE_PLUGIN_ATTEMPT_DELAY =
"storage.plugin_access_attempt_delay";
+ public static final NonNegativeLongValidator
STORAGE_PLUGIN_ATTEMPT_DELAY_VALIDATOR = new NonNegativeLongValidator(
+ STORAGE_PLUGIN_ATTEMPT_DELAY,
+ 5*60*1000,
Review Comment:
Done. I asked IDEA to reformat the code and it indented the subsequent lines
in the string additions by one level. Let me know if you meant a bigger
reformatting.
> Add support for automatically retrying and disabling broken storage plugins
> ---------------------------------------------------------------------------
>
> Key: DRILL-8314
> URL: https://issues.apache.org/jira/browse/DRILL-8314
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - Other
> Affects Versions: 1.20.2
> Reporter: James Turton
> Assignee: James Turton
> Priority: Minor
> Fix For: 2.0.0
>
>
> Enabled storage plugins may malfunction for different reasons, e.g. they have
> been misconfigured or their remote data source has gone offline. Depending on
> the plugin's implementation, this could cause it to fail to return optimizer
> rules or register schemas. In some cases this can have a wider impact, e.g.
> unconditioned queries against the info schema will fail if a single plugin is
> failing in registerSchemas.
> Rather than us swallowing such errors and silently returning a subset of
> results, this Jira proposes new features that allow firstly for the retrying
> of attempts to get rules or schemas from a plugin and secondly for a plugin
> to be automatically disabled after the configured maximum number of attempts
> have failed. A broken plugin will still cause a query failure but the user
> will be informed that the plugin has been disabled as a result. Knowing what
> has just happened, the user can choose to reissue the query knowing that the
> broken plugin is now disabled or they might choose to investigate the problem
> affecting the broken plugin instead.
> These new features are optional and can be disabled using new SYSTEM options.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
