[
https://issues.apache.org/jira/browse/DRILL-4047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15096786#comment-15096786
]
Julien Le Dem commented on DRILL-4047:
--------------------------------------
Here is how it works:
Context:
You can query files in a hadoop file system in the following way:
{noformat}
select columns from dfs.`mytable/file.csv`;
{noformat}
The format plugin is selected based on file extension or header according to
the configuration.
This feature enables providing the format plugin and its config in the SQL
query instead as follows:
{noformat}
select columns from table(dfs.`mytable/file.csv`(type => 'text',
fieldDelimiter => ','))
{noformat}
The table(...) syntax tells the SQL parser that we're passing parameters to the
table.
The type parameter is required and selects the Format plugin as defined by its
Config class (as would be done when deserializing from json)
The other parameters are defined by the fields of the same class. (excluding
"extensions" that does not apply in that case)
For example in the case of Text:
https://github.com/apache/drill/blob/1.4.0/exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/TextFormatPlugin.java#L135
{noformat}
@JsonTypeName("text") @JsonInclude(Include.NON_DEFAULT)
public static class TextFormatConfig implements FormatPluginConfig {
public List<String> extensions = ImmutableList.of();
public String lineDelimiter = "\n";
public char fieldDelimiter = '\n';
public char quote = '"';
public char escape = '"';
public char comment = '#';
public boolean skipFirstLine = false;
public boolean extractHeader = false;
{noformat}
> Select with options
> -------------------
>
> Key: DRILL-4047
> URL: https://issues.apache.org/jira/browse/DRILL-4047
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Relational Operators
> Reporter: Julien Le Dem
> Assignee: Julien Le Dem
> Fix For: 1.4.0
>
>
> Add a mechanism to pass parameters down to the StoragePlugin when writing a
> Select statement.
> Some discussion here:
> http://mail-archives.apache.org/mod_mbox/drill-dev/201510.mbox/%3CCAO%2Bvc4AcGK3%2B3QYvQV1-xPPdpG3Tc%2BfG%3D0xDGEUPrhd6ktHv5Q%40mail.gmail.com%3E
> http://mail-archives.apache.org/mod_mbox/drill-dev/201511.mbox/%3ccao+vc4clzylvjevisfjqtcyxb-zsmfy4bqrm-jhbidwzgqf...@mail.gmail.com%3E
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
