[
https://issues.apache.org/jira/browse/DRILL-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16807815#comment-16807815
]
ASF GitHub Bot commented on DRILL-7115:
---------------------------------------
ihuzenko commented on pull request #1706: DRILL-7115: Improve Hive schema show
tables performance
URL: https://github.com/apache/drill/pull/1706#discussion_r271339849
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/WorkspaceSchemaFactory.java
##########
@@ -920,46 +920,11 @@ public void dropTable(String table) {
}
@Override
- public List<Pair<String, TableType>> getTableNamesAndTypes(boolean
bulkLoad, int bulkSize) {
- final List<Pair<String, TableType>> tableNamesAndTypes =
Lists.newArrayList();
-
- // Look for raw tables first
- if (!tables.isEmpty()) {
- for (Map.Entry<TableInstance, DrillTable> tableEntry :
tables.entrySet()) {
- tableNamesAndTypes
- .add(Pair.of(tableEntry.getKey().sig.name,
tableEntry.getValue().getJdbcTableType()));
- }
- }
- // Then look for files that start with this name and end in .drill.
- List<DotDrillFile> files = Collections.emptyList();
- try {
- files = DotDrillUtil.getDotDrills(getFS(), new
Path(config.getLocation()), DotDrillType.VIEW);
- } catch (AccessControlException e) {
- if (!schemaConfig.getIgnoreAuthErrors()) {
- logger.debug(e.getMessage());
- throw UserException.permissionError(e)
- .message("Not authorized to list or query tables in schema
[%s]", getFullSchemaName())
- .build(logger);
- }
- } catch (IOException e) {
- logger.warn("Failure while trying to list view tables in workspace
[{}]", getFullSchemaName(), e);
- } catch (UnsupportedOperationException e) {
- // the file system (e.g. the classpath filesystem) may not support
listing
- // of files. But see getViews(), it ignores the exception and continues
- logger.debug("Failure while trying to list view tables in workspace
[{}]", getFullSchemaName(), e);
- }
-
- try {
- for (DotDrillFile f : files) {
- if (f.getType() == DotDrillType.VIEW) {
- tableNamesAndTypes.add(Pair.of(f.getBaseName(), TableType.VIEW));
- }
- }
- } catch (UnsupportedOperationException e) {
- logger.debug("The filesystem for this workspace does not support this
operation.", e);
Review comment:
This deleted code mostly duplicated body of existing ```getViews()```
method. This logging statement also present in the method.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Improve Hive schema show tables performance
> -------------------------------------------
>
> Key: DRILL-7115
> URL: https://issues.apache.org/jira/browse/DRILL-7115
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - Hive, Storage - Information Schema
> Affects Versions: 1.15.0
> Reporter: Igor Guzenko
> Assignee: Igor Guzenko
> Priority: Major
> Fix For: 1.16.0
>
>
> In Sqlline(Drill), "show tables" on a Hive schema is taking nearly 15mins to
> 20mins. The schema has nearly ~8000 tables.
> Whereas the same in beeline(Hive) is throwing the result in a split second(~
> 0.2 secs).
> I tested the same in my test cluster by creating 6000 tables(empty!) in Hive
> and then doing "show tables" in Drill. It took more than 2 mins(~140 secs).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
