[
https://issues.apache.org/jira/browse/DRILL-7975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386274#comment-17386274
]
ASF GitHub Bot commented on DRILL-7975:
---------------------------------------
vvysotskyi commented on a change in pull request #2278:
URL: https://github.com/apache/drill/pull/2278#discussion_r675621464
##########
File path:
logical/src/main/java/org/apache/drill/common/logical/StoragePluginConfig.java
##########
@@ -29,6 +31,13 @@
// DO NOT include enabled status in equality and hash
// comparisons; doing so will break the plugin registry.
private Boolean enabled;
+ protected final Integer reconnectRetries;
Review comment:
But some plugins might not use this property, or it may be invalid for
them, for example, there is no reason to use it for `sys` or `cp` plugins, but
they all will have it...
##########
File path:
contrib/storage-splunk/src/test/java/org/apache/drill/exec/store/splunk/SplunkTestSuite.java
##########
@@ -71,7 +79,9 @@ public static void initSplunk() throws Exception {
String hostname = splunk.getHost();
Integer port = splunk.getFirstMappedPort();
StoragePluginRegistry pluginRegistry =
cluster.drillbit().getContext().getStorage();
- SPLUNK_STORAGE_PLUGIN_CONFIG = new SplunkPluginConfig(SPLUNK_LOGIN,
SPLUNK_PASS, hostname, port, "1", "now", null);
+ JsonNode storagePluginJson = mapper.readTree(new
File(Resources.getResource("bootstrap-storage-plugins.json").toURI()));
+ SPLUNK_STORAGE_PLUGIN_CONFIG =
mapper.treeToValue(storagePluginJson.get("storage").get("splunk"),
SplunkPluginConfig.class);
+ setPort(SPLUNK_STORAGE_PLUGIN_CONFIG, port);
Review comment:
1. But the way you are deserializing slightly differs from how it is
deserialized in runtime. For example, we don't hack port at runtime, use
configured object mapper, etc. Also, you are setting the port number through
the reflection, but what if it wasn't deserialized correctly, so it wouldn't be
checked.
If we want to test the config deserializer for it, let's do it in a separate
test, but this change only makes things more complex.
2. I don't see how Jackson will set the correct `reconnectRetries` value to
the `SplunkPluginConfig` instance. It will call the `SplunkPluginConfig`
constructor, marked with `@JsonCreator` annotation, this constructor will call
parent one that has the hardcoded number of the `reconnectRetries` value, so no
actual value should be passed...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
> Connection to Splunk Drill Storage Plugin fails intermittently
> --------------------------------------------------------------
>
> Key: DRILL-7975
> URL: https://issues.apache.org/jira/browse/DRILL-7975
> Project: Apache Drill
> Issue Type: Bug
> Components: splunk
> Affects Versions: 1.19.0
> Reporter: Vitalii Diravka
> Assignee: Vitalii Diravka
> Priority: Major
> Fix For: 1.20.0
>
>
> This query:
> {code:java}
> SELECT * FROM splunk.data LIMIT 1000{code}
> fails sometimes with the following error:
> {code:java}
> VALIDATION ERROR: HTTP 401 -- call not properly authenticated
> (com.splunk.HttpException) HTTP 401 -- call not properly authenticated
> com.splunk.HttpException.create():8{code}
> {code:java}
> {
> "id": {
> "part1": 2268975635403754500,
> "part2": -3379215825344213000
> },
> "type": 1,
> "start": 1619196589746,
> "end": 1619196589763,
> "query": "SELECT * FROM splunk.ddr LIMIT 1000",
> "foreman": {
> "address": "eadd7105aa7a",
> "userPort": 31010,
> "controlPort": 31011,
> "dataPort": 31012,
> "version": "1.19.0-SNAPSHOT",
> "state": 0
> },
> "state": 4,
> "totalFragments": 0,
> "finishedFragments": 0,
> "user": "anonymous",
> "error": "VALIDATION ERROR: HTTP 401 -- call not properly
> authenticated\n\n",
> "verboseError": "VALIDATION ERROR: HTTP 401 -- call not properly
> authenticated\n\n\n\n (com.splunk.HttpException) HTTP 401 -- call not
> properly authenticated\n com.splunk.HttpException.create():84\n
> com.splunk.HttpService.send():500\n com.splunk.Service.send():1295\n
> com.splunk.HttpService.get():169\n
> com.splunk.ResourceCollection.list():288\n
> com.splunk.ResourceCollection.refresh():331\n
> com.splunk.ResourceCollection.refresh():29\n
> com.splunk.Resource.validate():186\n
> com.splunk.ResourceCollection.validate():356\n
> com.splunk.ResourceCollection.keySet():279\n
> org.apache.drill.exec.store.splunk.SplunkSchemaFactory$SplunkSchema.registerIndexes():114\n
>
> org.apache.drill.exec.store.splunk.SplunkSchemaFactory$SplunkSchema.<init>():72\n
>
> org.apache.drill.exec.store.splunk.SplunkSchemaFactory.registerSchemas():60\n
>
> org.apache.drill.exec.store.splunk.SplunkStoragePlugin.registerSchemas():60\n
> org.apache.calcite.jdbc.DynamicRootSchema.loadSchemaFactory():87\n
> org.apache.calcite.jdbc.DynamicRootSchema.getImplicitSubSchema():72\n
> org.apache.calcite.jdbc.CalciteSchema.getSubSchema():265\n
> org.apache.calcite.sql.validate.SqlValidatorUtil.getSchema():1050\n
> org.apache.drill.exec.planner.sql.conversion.DrillCalciteCatalogReader.isValidSchema():171\n
>
> org.apache.drill.exec.planner.sql.conversion.DrillValidator.validateFrom():55\n
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3383\n
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60\n
> org.apache.calcite.sql.validate.AbstractNamespace.validate():84\n
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():1009\n
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():969\n
> org.apache.calcite.sql.SqlSelect.validate():216\n
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression():944\n
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate():651\n
> org.apache.drill.exec.planner.sql.conversion.SqlConverter.validate():189\n
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode():641\n
>
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert():195\n
>
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():169\n
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():283\n
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan():163\n
> org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan():128\n
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():93\n
> org.apache.drill.exec.work.foreman.Foreman.runSQL():592\n
> org.apache.drill.exec.work.foreman.Foreman.run():273\n
> java.util.concurrent.ThreadPoolExecutor.runWorker():1149\n
> java.util.concurrent.ThreadPoolExecutor$Worker.run():624\n
> java.lang.Thread.run():748\n",
> "errorId": "ff2561d4-b71f-445d-98ea-ccc93b8ede8e",
> "optionsJson": "[ {\n \"kind\" : \"LONG\",\n \"accessibleScopes\" :
> \"ALL\",\n \"name\" : \"exec.query.max_rows\",\n \"num_val\" : 0,\n
> \"scope\" : \"QUERY\"\n} ]",
> "planEnd": 1619196589760,
> "queueWaitEnd": 0,
> "totalCost": 0,
> "queueName": "-",
> "queryId": "1f7d0551-a041-cbe0-d11a-9d1c56facfcf"
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
