[
https://issues.apache.org/jira/browse/DRILL-6575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535784#comment-16535784
]
ASF GitHub Bot commented on DRILL-6575:
---------------------------------------
arina-ielchiieva commented on a change in pull request #1365: DRILL-6575: Add
store.hive.conf.properties option to allow set Hive properties at session level
URL: https://github.com/apache/drill/pull/1365#discussion_r200814173
##########
File path:
contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveUtilities.java
##########
@@ -471,20 +498,28 @@ public static void
restoreColumns(HiveTableWithColumnCache table, HivePartition
* Wrapper around {@link MetaStoreUtils#getSchema(StorageDescriptor,
StorageDescriptor, Map, String, String, List)}
* which also sets columns from table cache to table and returns properties
returned by
* {@link MetaStoreUtils#getSchema(StorageDescriptor, StorageDescriptor,
Map, String, String, List)}.
+ *
+ * @param table Hive table with cached columns
+ * @return Hive table metadata
*/
public static Properties getTableMetadata(HiveTableWithColumnCache table) {
restoreColumns(table, null);
return MetaStoreUtils.getSchema(table.getSd(), table.getSd(),
table.getParameters(),
table.getDbName(), table.getTableName(), table.getPartitionKeys());
}
+ /**
+ * Generates unsupported types exception message with list of supported types
+ * and throws user exception.
+ *
+ * @param unsupportedType unsupported type
+ */
public static void throwUnsupportedHiveDataTypeError(String unsupportedType)
{
StringBuilder errMsg = new StringBuilder();
- errMsg.append(String.format("Unsupported Hive data type %s. ",
unsupportedType));
+ errMsg.append("Unsupported Hive data type
").append(unsupportedType).append(". ");
Review comment:
Done.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Add store.hive.conf.properties option to allow set Hive properties at session
> level
> -----------------------------------------------------------------------------------
>
> Key: DRILL-6575
> URL: https://issues.apache.org/jira/browse/DRILL-6575
> Project: Apache Drill
> Issue Type: Improvement
> Affects Versions: 1.13.0
> Reporter: Arina Ielchiieva
> Assignee: Arina Ielchiieva
> Priority: Major
> Labels: doc-impacting
> Fix For: 1.15.0
>
>
> *Use case*
> Hive external table ddl:
> {noformat}
> create external table my(key int, val string)
> row format delimited
> fields terminated by ','
> stored as textfile
> location '/data/my_tbl';
> {noformat}
> Path {{/data/my_tb}} contains sub directory and file in it:
> {{/data/my_tbl/sub_dir/data.txt}} with the following data:
> {noformat}
> 1, value_1
> 2, value_2
> {noformat}
> When querying such table from Hive, user gets the following exception:
> {noformat}
> Failed with exception java.io.IOException:java.io.IOException: Not a file:
> file:///data/my_tbl/sub_dir
> {noformat}
> To be able to query this table user needs to set two properties to true:
> {{hive.mapred.supports.subdirectories}} and {{mapred.input.dir.recursive}}.
> They can be set at system level in hive-site.xml or at session in Hive
> console:
> {noformat}
> set hive.mapred.supports.subdirectories=true;
> set mapred.input.dir.recursive=true;
> {noformat}
> Currently to be able to query such table from Drill, user can specify this
> properties in Hive plugin only:
> {noformat}
> {
> "type": "hive",
> "configProps": {
> "hive.metastore.uris": "thrift://localhost:9083",
> "hive.metastore.sasl.enabled": "false",
> "hbase.zookeeper.quorum": "localhost",
> "hbase.zookeeper.property.clientPort": "5181",
> "hive.mapred.supports.subdirectories": "true",
> "mapred.input.dir.recursive": "true"
> }
> "enabled": true
> }
> {noformat}
> *Jira scope*
> This Jira aims to add new session option to Drill
> {{store.hive.conf.properties}} which will allow user to specify hive
> properties at session level.
> User should write properties in string delimiter by new line symbol.
> Properties values should NOT be set in double-quotes or any other quotes
> otherwise they would be parsed incorrectly. Key and value should be separated
> by {{=}}. Each `alter session set` will override previously set properties at
> session level. If during query Drill couldn't unparse property string,
> warning will be logged. Properties will be parsed by loading into
> {{java.util.Properties}}. Default value is empty string ("").
> Example:
> {noformat}
> alter session set `store.hive.conf.properties` =
> 'hive.mapred.supports.subdirectories=true\nmapred.input.dir.recursive=true';
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
