[
https://issues.apache.org/jira/browse/DRILL-7030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16832618#comment-16832618
]
ASF GitHub Bot commented on DRILL-7030:
---------------------------------------
arina-ielchiieva commented on pull request #1780: DRILL-7030: Make format
plugins fully pluggable
URL: https://github.com/apache/drill/pull/1780#discussion_r280834695
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePluginRegistryImpl.java
##########
@@ -334,6 +331,48 @@ private StoragePlugins
loadBootstrapPlugins(LogicalPlanPersistence lpPersistence
}
}
+ private void loadStoragePlugins(URL url, StoragePlugins bootstrapPlugins,
Map<String, URL> pluginURLMap, LogicalPlanPersistence lpPersistence) throws
IOException {
+ StoragePlugins plugins = getPluginsFromResource(url, lpPersistence);
+ plugins.forEach(plugin -> {
+ StoragePluginConfig oldPluginConfig =
bootstrapPlugins.putIfAbsent(plugin.getKey(), plugin.getValue());
+ if (oldPluginConfig != null) {
+ logger.warn("Duplicate plugin instance '{}' defined in [{}, {}],
ignoring the later one.",
+ plugin.getKey(), pluginURLMap.get(plugin.getKey()), url);
+ } else {
+ pluginURLMap.put(plugin.getKey(), url);
+ }
+ });
+ }
+
+ private void loadFormatPlugins(URL url, StoragePlugins bootstrapPlugins,
Map<String, URL> pluginURLMap, LogicalPlanPersistence lpPersistence) throws
IOException {
+ StoragePlugins plugins = getPluginsFromResource(url, lpPersistence);
+ plugins.forEach(formatPlugin -> {
+ String targetStoragePluginName = formatPlugin.getKey();
+ StoragePluginConfig storagePlugin =
bootstrapPlugins.getConfig(targetStoragePluginName);
+ StoragePluginConfig formatPluginValue = formatPlugin.getValue();
+ if (storagePlugin == null) {
+ logger.warn("No storage plugins with the given name are registered:
'{}'", targetStoragePluginName);
Review comment:
```suggestion
logger.warn("No storage plugins with the given name are registered:
[{}]", targetStoragePluginName);
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Make format plugins fully pluggable
> -----------------------------------
>
> Key: DRILL-7030
> URL: https://issues.apache.org/jira/browse/DRILL-7030
> Project: Apache Drill
> Issue Type: Improvement
> Affects Versions: 1.15.0
> Reporter: Arina Ielchiieva
> Assignee: Anton Gozhiy
> Priority: Major
> Labels: doc-impacting
> Fix For: 1.17.0
>
>
> Discussion on the mailing list -
> [https://lists.apache.org/thread.html/0c7de9c23ee9a8e18f8548ae0a323284cf1311b9570bd77ba544f63d@%3Cdev.drill.apache.org%3E]
> {noformat}
> Before we were adding new formats / plugins into the exec module. Eventually
> we came up to the point that exec package size is growing and adding plugin
> and format contributions is better to separate out in the different module.
> Now we have contrib module where we add such contributions. Plugins are
> pluggable, there are added automatically by means of having drill-module.conf
> file which points to the scanning packages.
> Format plugins are using the same approach, the only problem is that they are
> not added into bootstrap-storage-plugins.json. So when adding new format
> plugin, in order for it to automatically appear in Drill Web UI, developer
> has to update bootstrap file which is in the exec module.
> My suggestion we implement some functionality that would merge format config
> with the bootstrap one. For example, each plugin would have to have
> bootstrap-format.json file with the information to which plugin format should
> be added (structure the same as in bootstrap-storage-plugins.json):
> Example:
> {
> "storage":{
> dfs: {
> formats: {
> "psv" : {
> type: "msgpack",
> extensions: [ "mp" ]
> }
> }
> }
> }
> }
> Then during Drill start up such bootstrap-format.json files will be merged
> with bootstrap-storage-plugins.json.{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
