[jira] [Created] (DRILL-4792) Include session options used for a query as part of the profile
Arina Ielchiieva created DRILL-4792: --- Summary: Include session options used for a query as part of the profile Key: DRILL-4792 URL: https://issues.apache.org/jira/browse/DRILL-4792 Project: Apache Drill Issue Type: Improvement Affects Versions: 1.7.0 Reporter: Arina Ielchiieva Assignee: Arina Ielchiieva Priority: Minor Fix For: 1.8.0 Include session options used for a query as part of the profile. This will be very useful for debugging/diagnostics. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4792) Include session options used for a query as part of the profile
[ https://issues.apache.org/jira/browse/DRILL-4792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-4792: Attachment: session_options_json.JPG session_options_collapsed.JPG session_options_block.JPG no_session_options.JPG > Include session options used for a query as part of the profile > --- > > Key: DRILL-4792 > URL: https://issues.apache.org/jira/browse/DRILL-4792 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.7.0 >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva >Priority: Minor > Fix For: 1.8.0 > > Attachments: no_session_options.JPG, session_options_block.JPG, > session_options_collapsed.JPG, session_options_json.JPG > > > Include session options used for a query as part of the profile. > This will be very useful for debugging/diagnostics. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4792) Include session options used for a query as part of the profile
[ https://issues.apache.org/jira/browse/DRILL-4792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15385976#comment-15385976 ] Arina Ielchiieva commented on DRILL-4792: - Session options effective at the time query was run will be displayed in query profile. Appropriate block will appear after Query Profile information (screenshot - session_options_block.JPG). User may collapse Session Options block (screenshot - session_options_collapsed.JPG). Also appropriate information can be seen in json profile (screenshot - session_options_json.JPG). If there were no session options set at the time query was run, Session Options block won't appear (screenshot - no_session_options.JPG). > Include session options used for a query as part of the profile > --- > > Key: DRILL-4792 > URL: https://issues.apache.org/jira/browse/DRILL-4792 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.7.0 >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva >Priority: Minor > Fix For: 1.8.0 > > > Include session options used for a query as part of the profile. > This will be very useful for debugging/diagnostics. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4682) Allow full schema identifier in SELECT clause
[ https://issues.apache.org/jira/browse/DRILL-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386021#comment-15386021 ] ASF GitHub Bot commented on DRILL-4682: --- GitHub user vdiravka opened a pull request: https://github.com/apache/drill/pull/549 DRILL-4682: Allow full schema identifier in SELECT clause - changed SqlBracketlessSyntax logic in the DrillCompoundIdentifier; - added checking of using full schema name in column names; - added unit test testColumnNamesWithSchemaName; You can merge this pull request into a Git repository by running: $ git pull https://github.com/vdiravka/drill DRILL-4682 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/drill/pull/549.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #549 commit 0e5123c80e0b7d2dca36503dab3917bbae827702 Author: Vitalii Diravka Date: 2016-06-13T11:44:33Z DRILL-4682: Allow full schema identifier in SELECT clause - changed SqlBracketlessSyntax logic in the DrillCompoundIdentifier; - added checking of using full schema name in column names; - added unit test testColumnNamesWithSchemaName; > Allow full schema identifier in SELECT clause > - > > Key: DRILL-4682 > URL: https://issues.apache.org/jira/browse/DRILL-4682 > Project: Apache Drill > Issue Type: Improvement > Components: SQL Parser >Reporter: Andries Engelbrecht > > Currently Drill requires aliases to identify columns in the SELECT clause > when working with multiple tables/workspaces. > Many BI/Analytical and other tools by default will use the full schema > identifier in the select clause when generating SQL statements for execution > for generic JDBC or ODBC sources. Not supporting this feature causes issues > and a slower adoption of utilizing Drill as an execution engine within the > larger Analytical SQL community. > Propose to support > SELECT ... FROM > .. > Also see DRILL-3510 for double quote support as per ANSI_QUOTES > SELECT ""."".""."" FROM > "".""."" > Which is very common generic SQL being generated by most tools when dealing > with a generic SQL data source. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (DRILL-4785) Limit 0 queries regressed in Drill 1.7.0
[ https://issues.apache.org/jira/browse/DRILL-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dechang Gu closed DRILL-4785. - verified with limit 0 queries. LGTM. > Limit 0 queries regressed in Drill 1.7.0 > - > > Key: DRILL-4785 > URL: https://issues.apache.org/jira/browse/DRILL-4785 > Project: Apache Drill > Issue Type: Bug > Components: Functions - Drill >Affects Versions: 1.7.0 > Environment: Redhat EL6 >Reporter: Dechang Gu >Assignee: Venki Korukanti > Fix For: 1.8.0 > > > We noticed a bunch of limit 0 queries regressed quite a bit: +2500ms, while > the same queries took ~400ms in Apache Drill 1.6.0. 5-6X regression. Further > investigation indicates that most likely the root cause of the regression is > in the commit: > vkorukanti committed with vkorukanti DRILL-4446: Support mandatory work > assignment to endpoint requirement… > commit id: 10afc708600ea9f4cb0e7c2cd981b5b1001fea0d > With drill build on this commit, query takes 3095ms > and in the drillbit.log: > 2016-07-15 17:27:55,048 ucs-node2.perf.lab > [28768074-4ed6-a70a-2e6a-add3201ab801:foreman] INFO > o.a.drill.exec.work.foreman.Foreman - Query text for query id > 28768074-4ed6-a70a-2e6a-add3201ab801: SELECT * FROM (SELECT > CAST(EXTRACT(MONTH FROM CAST(`rfm_sales`.`business_date` AS DATE)) AS > INTEGER) AS `mn_business_date_ok`,AVG((CASE WHEN ((CAST(EXTRACT(YEAR FROM > CAST(`rfm_sales`.`business_date` AS DATE)) AS INTEGER) = 2014) AND > (CAST((EXTRACT(MONTH FROM CAST(`rfm_sales`.`business_date` AS DATE)) - 1) / 3 > + 1 AS INTEGER) <= 4)) THEN `rfm_sales`.`pos_netsales` ELSE NULL END)) AS > `avg_Calculation_CIDBACJBCCCBHDGB_ok`,SUM((CASE WHEN ((CAST(EXTRACT(YEAR FROM > CAST(`rfm_sales`.`business_date` AS DATE)) AS INTEGER) = 2014) AND > (CAST((EXTRACT(MONTH FROM CAST(`rfm_sales`.`business_date` AS DATE)) - 1) / 3 > + 1 AS INTEGER) <= 4)) THEN `rfm_sales`.`pos_netsales` ELSE NULL END)) AS > `sum_Calculation_CIDBACJBCCCBHDGB_ok`,SUM((CASE WHEN ((CAST(EXTRACT(YEAR FROM > CAST(`rfm_sales`.`business_date` AS DATE)) AS INTEGER) = 2014) AND > (CAST((EXTRACT(MONTH FROM CAST(`rfm_sales`.`business_date` AS DATE)) - 1) / 3 > + 1 AS INTEGER) <= 4)) THEN 1 ELSE NULL END)) AS > `sum_Calculation_CJEBBAEBBFADBDFJ_ok`,SUM((CASE WHEN ((CAST(EXTRACT(YEAR FROM > CAST(`rfm_sales`.`business_date` AS DATE)) AS INTEGER) = 2014) AND > (CAST((EXTRACT(MONTH FROM CAST(`rfm_sales`.`business_date` AS DATE)) - 1) / 3 > + 1 AS INTEGER) <= 4)) THEN (`rfm_sales`.`pos_comps` + > `rfm_sales`.`pos_promos`) ELSE NULL END)) AS `sum_Net_Sales__YTD___copy__ok` > FROM `dfs.xxx`.`views/rfm_sales` `rfm_sales` GROUP BY CAST(EXTRACT(MONTH FROM > CAST(`rfm_sales`.`business_date` AS DATE)) AS INTEGER)) T LIMIT 0 > 2016-07-15 17:27:55,664 ucs-node2.perf.lab > [28768074-4ed6-a70a-2e6a-add3201ab801:foreman] INFO > o.a.d.exec.store.parquet.Metadata - Took 208 ms to read metadata from cache > file > 2016-07-15 17:27:56,783 ucs-node2.perf.lab > [28768074-4ed6-a70a-2e6a-add3201ab801:foreman] INFO > o.a.d.exec.store.parquet.Metadata - Took 129 ms to read metadata from cache > file > 2016-07-15 17:27:57,960 ucs-node2.perf.lab > [28768074-4ed6-a70a-2e6a-add3201ab801:frag:0:0] INFO > o.a.d.e.w.fragment.FragmentExecutor - > 28768074-4ed6-a70a-2e6a-add3201ab801:0:0: State change requested > AWAITING_ALLOCATION --> RUNNING > 2016-07-15 17:27:57,961 ucs-node2.perf.lab > [28768074-4ed6-a70a-2e6a-add3201ab801:frag:0:0] INFO > o.a.d.e.w.f.FragmentStatusReporter - > 28768074-4ed6-a70a-2e6a-add3201ab801:0:0: State to report: RUNNING > 2016-07-15 17:27:57,989 ucs-node2.perf.lab > [28768074-4ed6-a70a-2e6a-add3201ab801:frag:0:0] INFO > o.a.d.e.w.fragment.FragmentExecutor - > 28768074-4ed6-a70a-2e6a-add3201ab801:0:0: State change requested RUNNING --> > FINISHED > 2016-07-15 17:27:57,989 ucs-node2.perf.lab > [28768074-4ed6-a70a-2e6a-add3201ab801:frag:0:0] INFO > o.a.d.e.w.f.FragmentStatusReporter - > 28768074-4ed6-a70a-2e6a-add3201ab801:0:0: State to report: FINISHED > while running the same query on the parent commit (commit id > 9f4fff800d128878094ae70b454201f79976135d), it only takes 492ms. > and in the drillbit.log: > 2016-07-15 17:19:27,309 ucs-node7.perf.lab > [2876826f-ee19-9466-0c0c-869f47c409f8:foreman] INFO > o.a.drill.exec.work.foreman.Foreman - Query text for query id > 2876826f-ee19-9466-0c0c-869f47c409f8: SELECT * FROM (SELECT > CAST(EXTRACT(MONTH FROM CAST(`rfm_sales`.`business_date` AS DATE)) AS > INTEGER) AS `mn_business_date_ok`,AVG((CASE WHEN ((CAST(EXTRACT(YEAR FROM > CAST(`rfm_sales`.`business_date` AS DATE)) AS INTEGER) = 2014) AND > (CAST((EXTRACT(MONTH FROM CAST(`rfm_sales`.`business_date` AS DATE)) - 1) / 3 > + 1 AS INTEGER) <= 4)) THEN `rfm_sales`.`pos_netsales` ELSE NULL END)) AS > `avg_Calculation_CIDBACJB
[jira] [Created] (DRILL-4793) integration oozie with drill--using oozie to support the activity for executing the drill queries
Kathleen Li created DRILL-4793: -- Summary: integration oozie with drill--using oozie to support the activity for executing the drill queries Key: DRILL-4793 URL: https://issues.apache.org/jira/browse/DRILL-4793 Project: Apache Drill Issue Type: Bug Components: Functions - Drill Affects Versions: 1.6.0 Reporter: Kathleen Li Priority: Minor Using Oozie for chaining the ETL jobs and want to know if we can make Oozie support an activity for executing the drill queries like hive queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4793) integration oozie with drill--using oozie to support the activity for executing the drill queries
[ https://issues.apache.org/jira/browse/DRILL-4793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kathleen Li updated DRILL-4793: --- Issue Type: New Feature (was: Bug) > integration oozie with drill--using oozie to support the activity for > executing the drill queries > - > > Key: DRILL-4793 > URL: https://issues.apache.org/jira/browse/DRILL-4793 > Project: Apache Drill > Issue Type: New Feature > Components: Functions - Drill >Affects Versions: 1.6.0 >Reporter: Kathleen Li >Priority: Minor > > Using Oozie for chaining the ETL jobs and want to know if we can make Oozie > support an activity for executing the drill queries like hive queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4783) Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty
[ https://issues.apache.org/jira/browse/DRILL-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386238#comment-15386238 ] ASF GitHub Bot commented on DRILL-4783: --- Github user jaltekruse commented on the issue: https://github.com/apache/drill/pull/546 The original behavior was not by design, it was a bug. I would just detect the case where a scalar is being flattened and throw an appropriate user facing message like "Flatten does not support inputs of non-list values". The tests you have included here don't check their results. I would also suggest adding a test for the new negative case, checking for the appropriate message. If you want to more precisely control the input to your test, I would recommend taking a look at the operator unit test system I checked in and demonstrated at the hangout last month. This will allow you to write more targeted tests for this case. Feel free to ask questions if the usage is not clear from the examples in this class. https://github.com/apache/drill/blob/master/exec/java-exec/src/test/java/org/apache/drill/exec/physical/unit/BasicPhysicalOpUnitTest.java > Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty > --- > > Key: DRILL-4783 > URL: https://issues.apache.org/jira/browse/DRILL-4783 > Project: Apache Drill > Issue Type: Bug >Reporter: Chunhui Shi >Assignee: Chunhui Shi >Priority: Critical > > Flatten failed to work on top of convert_from when the resultset is empty. > For a HBase table like this: > 0: jdbc:drill:zk=localhost:5181> select convert_from(t.address.cities,'json') > from hbase.`/tmp/flattentest` t; > +--+ > | EXPR$0 > | > +--+ > | {"list":[{"city":"SunnyVale"},{"city":"Palo Alto"},{"city":"Mountain > View"}]}| > | {"list":[{"city":"Seattle"},{"city":"Bellevue"},{"city":"Renton"}]} > | > | {"list":[{"city":"Minneapolis"},{"city":"Falcon Heights"},{"city":"San > Paul"}]} | > +--+ > Flatten works when row_key is in (1,2,3) > 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select > convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t > where row_key=1) t1; > +---+ > | EXPR$0 | > +---+ > | {"city":"SunnyVale"} | > | {"city":"Palo Alto"} | > | {"city":"Mountain View"} | > +---+ > But Flatten throws exception if the resultset is empty > 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select > convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t > where row_key=4) t1; > Error: SYSTEM ERROR: ClassCastException: Cannot cast > org.apache.drill.exec.vector.NullableIntVector to > org.apache.drill.exec.vector.complex.RepeatedValueVector > Fragment 0:0 > [Error Id: 07fd0cab-d1e6-4259-bfec-ad80f02d93a2 on atsqa4-127.qa.lab:31010] > (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4673) Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on command return
[ https://issues.apache.org/jira/browse/DRILL-4673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386413#comment-15386413 ] ASF GitHub Bot commented on DRILL-4673: --- Github user vdiravka commented on a diff in the pull request: https://github.com/apache/drill/pull/541#discussion_r71583345 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DropTableIfExistsHandler.java --- @@ -28,47 +30,57 @@ import org.apache.drill.exec.physical.PhysicalPlan; import org.apache.drill.exec.planner.sql.DirectPlan; import org.apache.drill.exec.planner.sql.SchemaUtilites; -import org.apache.drill.exec.planner.sql.parser.SqlDropTable; +import org.apache.drill.exec.planner.sql.parser.SqlDropTableIfExists; import org.apache.drill.exec.store.AbstractSchema; // SqlHandler for dropping a table. -public class DropTableHandler extends DefaultSqlHandler { +public class DropTableIfExistsHandler extends DefaultSqlHandler { - private static org.slf4j.Logger logger = org.slf4j.LoggerFactory.getLogger(DropTableHandler.class); + private static org.slf4j.Logger logger = org.slf4j.LoggerFactory.getLogger(DropTableIfExistsHandler.class); - public DropTableHandler(SqlHandlerConfig config) { + public DropTableIfExistsHandler(SqlHandlerConfig config) { super(config); } /** - * Function resolves the schema and invokes the drop method. Raises an exception if the schema is - * immutable. - * @param sqlNode - Table name identifier - * @return - Single row indicating drop succeeded, raise exception otherwise + * Function resolves the schema and invokes the drop method + * (while IF EXISTS statement is used function invokes the drop method only if table exists). + * Raises an exception if the schema is immutable. + * @param sqlNode - SqlDropTableIfExists (SQL parse tree of drop table [if exists] query) + * @return - Single row indicating drop succeeded or table is not found while IF EXISTS statement is used, + * raise exception otherwise * @throws ValidationException * @throws RelConversionException * @throws IOException */ @Override public PhysicalPlan getPlan(SqlNode sqlNode) throws ValidationException, RelConversionException, IOException { -SqlDropTable dropTableNode = ((SqlDropTable) sqlNode); -SqlIdentifier tableIdentifier = dropTableNode.getTableIdentifier(); +SqlDropTableIfExists dropTableIfExistsNode = ((SqlDropTableIfExists) sqlNode); +SqlIdentifier tableIdentifier = dropTableIfExistsNode.getTableIdentifier(); SchemaPlus defaultSchema = config.getConverter().getDefaultSchema(); AbstractSchema drillSchema = null; if (tableIdentifier != null) { - drillSchema = SchemaUtilites.resolveToMutableDrillSchema(defaultSchema, dropTableNode.getSchema()); + drillSchema = SchemaUtilites.resolveToMutableDrillSchema(defaultSchema, dropTableIfExistsNode.getSchema()); } -String tableName = ((SqlDropTable) sqlNode).getName(); +String tableName = dropTableIfExistsNode.getName(); if (drillSchema == null) { throw UserException.validationError() .message("Invalid table_name [%s]", tableName) .build(logger); } +if (dropTableIfExistsNode.checkTableExistence()) { + final Table tableToDrop = SqlHandlerUtil.getTableFromSchema(drillSchema, tableName); + if (tableToDrop == null || tableToDrop.getJdbcTableType() != Schema.TableType.TABLE) { --- End diff -- I checked every `schema#getTable` and found that every method can throw exception only if something go wrong (permissionError, dataReadError, IOException etc). `getTableFromSchema `can not throw any exception, because in this method `catch (Exception e)` is used. > Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on > command return > - > > Key: DRILL-4673 > URL: https://issues.apache.org/jira/browse/DRILL-4673 > Project: Apache Drill > Issue Type: New Feature > Components: Functions - Drill >Reporter: Vitalii Diravka >Assignee: Vitalii Diravka >Priority: Minor > Labels: drill > > Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on > command "DROP TABLE" return if table doesn't exist. > The same for "DROP VIEW IF EXISTS" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4673) Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on command return
[ https://issues.apache.org/jira/browse/DRILL-4673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386433#comment-15386433 ] ASF GitHub Bot commented on DRILL-4673: --- Github user vdiravka commented on a diff in the pull request: https://github.com/apache/drill/pull/541#discussion_r71585003 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DropTableIfExistsHandler.java --- @@ -28,47 +30,57 @@ import org.apache.drill.exec.physical.PhysicalPlan; import org.apache.drill.exec.planner.sql.DirectPlan; import org.apache.drill.exec.planner.sql.SchemaUtilites; -import org.apache.drill.exec.planner.sql.parser.SqlDropTable; +import org.apache.drill.exec.planner.sql.parser.SqlDropTableIfExists; import org.apache.drill.exec.store.AbstractSchema; // SqlHandler for dropping a table. -public class DropTableHandler extends DefaultSqlHandler { +public class DropTableIfExistsHandler extends DefaultSqlHandler { - private static org.slf4j.Logger logger = org.slf4j.LoggerFactory.getLogger(DropTableHandler.class); + private static org.slf4j.Logger logger = org.slf4j.LoggerFactory.getLogger(DropTableIfExistsHandler.class); - public DropTableHandler(SqlHandlerConfig config) { + public DropTableIfExistsHandler(SqlHandlerConfig config) { super(config); } /** - * Function resolves the schema and invokes the drop method. Raises an exception if the schema is - * immutable. - * @param sqlNode - Table name identifier - * @return - Single row indicating drop succeeded, raise exception otherwise + * Function resolves the schema and invokes the drop method + * (while IF EXISTS statement is used function invokes the drop method only if table exists). + * Raises an exception if the schema is immutable. + * @param sqlNode - SqlDropTableIfExists (SQL parse tree of drop table [if exists] query) + * @return - Single row indicating drop succeeded or table is not found while IF EXISTS statement is used, + * raise exception otherwise * @throws ValidationException * @throws RelConversionException * @throws IOException */ @Override public PhysicalPlan getPlan(SqlNode sqlNode) throws ValidationException, RelConversionException, IOException { -SqlDropTable dropTableNode = ((SqlDropTable) sqlNode); -SqlIdentifier tableIdentifier = dropTableNode.getTableIdentifier(); +SqlDropTableIfExists dropTableIfExistsNode = ((SqlDropTableIfExists) sqlNode); +SqlIdentifier tableIdentifier = dropTableIfExistsNode.getTableIdentifier(); SchemaPlus defaultSchema = config.getConverter().getDefaultSchema(); AbstractSchema drillSchema = null; if (tableIdentifier != null) { - drillSchema = SchemaUtilites.resolveToMutableDrillSchema(defaultSchema, dropTableNode.getSchema()); + drillSchema = SchemaUtilites.resolveToMutableDrillSchema(defaultSchema, dropTableIfExistsNode.getSchema()); } -String tableName = ((SqlDropTable) sqlNode).getName(); +String tableName = dropTableIfExistsNode.getName(); if (drillSchema == null) { throw UserException.validationError() .message("Invalid table_name [%s]", tableName) .build(logger); } +if (dropTableIfExistsNode.checkTableExistence()) { + final Table tableToDrop = SqlHandlerUtil.getTableFromSchema(drillSchema, tableName); + if (tableToDrop == null || tableToDrop.getJdbcTableType() != Schema.TableType.TABLE) { --- End diff -- I checked such query `drop table if exists hbase.TestTableNullStr` and found that the message is proper and same like for `DROP TABLE` statement: `PARSE ERROR: Unable to create or drop tables/views. Schema [hbase] is immutable.`. > Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on > command return > - > > Key: DRILL-4673 > URL: https://issues.apache.org/jira/browse/DRILL-4673 > Project: Apache Drill > Issue Type: New Feature > Components: Functions - Drill >Reporter: Vitalii Diravka >Assignee: Vitalii Diravka >Priority: Minor > Labels: drill > > Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on > command "DROP TABLE" return if table doesn't exist. > The same for "DROP VIEW IF EXISTS" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4673) Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on command return
[ https://issues.apache.org/jira/browse/DRILL-4673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386437#comment-15386437 ] ASF GitHub Bot commented on DRILL-4673: --- Github user vdiravka commented on a diff in the pull request: https://github.com/apache/drill/pull/541#discussion_r71585214 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/ViewHandler.java --- @@ -106,36 +106,43 @@ public PhysicalPlan getPlan(SqlNode sqlNode) throws ValidationException, RelConv } } - /** Handler for Drop View DDL command. */ - public static class DropView extends ViewHandler { -public DropView(SqlHandlerConfig config) { + /** Handler for Drop View [If Exists] DDL command. */ + public static class DropViewIfExists extends ViewHandler { +public DropViewIfExists(SqlHandlerConfig config) { super(config); } @Override public PhysicalPlan getPlan(SqlNode sqlNode) throws ValidationException, RelConversionException, IOException, ForemanSetupException { - SqlDropView dropView = unwrap(sqlNode, SqlDropView.class); - final String viewToDrop = dropView.getName(); + SqlDropViewIfExists dropView = unwrap(sqlNode, SqlDropViewIfExists.class); + final String viewName = dropView.getName(); final AbstractSchema drillSchema = SchemaUtilites.resolveToMutableDrillSchema(context.getNewDefaultSchema(), dropView.getSchemaPath()); final String schemaPath = drillSchema.getFullSchemaName(); - final Table existingTable = SqlHandlerUtil.getTableFromSchema(drillSchema, viewToDrop); - if (existingTable != null && existingTable.getJdbcTableType() != Schema.TableType.VIEW) { -throw UserException.validationError() -.message("[%s] is not a VIEW in schema [%s]", viewToDrop, schemaPath) -.build(logger); - } else if (existingTable == null) { -throw UserException.validationError() -.message("Unknown view [%s] in schema [%s].", viewToDrop, schemaPath) -.build(logger); + final Table viewToDrop = SqlHandlerUtil.getTableFromSchema(drillSchema, viewName); + if (dropView.checkViewExistence()) { +if (viewToDrop == null || viewToDrop.getJdbcTableType() != Schema.TableType.VIEW){ --- End diff -- Answered above. > Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on > command return > - > > Key: DRILL-4673 > URL: https://issues.apache.org/jira/browse/DRILL-4673 > Project: Apache Drill > Issue Type: New Feature > Components: Functions - Drill >Reporter: Vitalii Diravka >Assignee: Vitalii Diravka >Priority: Minor > Labels: drill > > Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on > command "DROP TABLE" return if table doesn't exist. > The same for "DROP VIEW IF EXISTS" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4673) Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on command return
[ https://issues.apache.org/jira/browse/DRILL-4673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386454#comment-15386454 ] ASF GitHub Bot commented on DRILL-4673: --- Github user vdiravka commented on the issue: https://github.com/apache/drill/pull/541 Agree with class/method names. I was confused with `SqlCreateOrReplaceView()`. Thanks Julian. @sudheeshkatkam Could you review please? > Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on > command return > - > > Key: DRILL-4673 > URL: https://issues.apache.org/jira/browse/DRILL-4673 > Project: Apache Drill > Issue Type: New Feature > Components: Functions - Drill >Reporter: Vitalii Diravka >Assignee: Vitalii Diravka >Priority: Minor > Labels: drill > > Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on > command "DROP TABLE" return if table doesn't exist. > The same for "DROP VIEW IF EXISTS" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4783) Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty
[ https://issues.apache.org/jira/browse/DRILL-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386786#comment-15386786 ] ASF GitHub Bot commented on DRILL-4783: --- Github user chunhui-shi commented on the issue: https://github.com/apache/drill/pull/546 Thanks @jaltekruse for the review. I updated exception text and added test case following your suggestion. > Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty > --- > > Key: DRILL-4783 > URL: https://issues.apache.org/jira/browse/DRILL-4783 > Project: Apache Drill > Issue Type: Bug >Reporter: Chunhui Shi >Assignee: Chunhui Shi >Priority: Critical > > Flatten failed to work on top of convert_from when the resultset is empty. > For a HBase table like this: > 0: jdbc:drill:zk=localhost:5181> select convert_from(t.address.cities,'json') > from hbase.`/tmp/flattentest` t; > +--+ > | EXPR$0 > | > +--+ > | {"list":[{"city":"SunnyVale"},{"city":"Palo Alto"},{"city":"Mountain > View"}]}| > | {"list":[{"city":"Seattle"},{"city":"Bellevue"},{"city":"Renton"}]} > | > | {"list":[{"city":"Minneapolis"},{"city":"Falcon Heights"},{"city":"San > Paul"}]} | > +--+ > Flatten works when row_key is in (1,2,3) > 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select > convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t > where row_key=1) t1; > +---+ > | EXPR$0 | > +---+ > | {"city":"SunnyVale"} | > | {"city":"Palo Alto"} | > | {"city":"Mountain View"} | > +---+ > But Flatten throws exception if the resultset is empty > 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select > convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t > where row_key=4) t1; > Error: SYSTEM ERROR: ClassCastException: Cannot cast > org.apache.drill.exec.vector.NullableIntVector to > org.apache.drill.exec.vector.complex.RepeatedValueVector > Fragment 0:0 > [Error Id: 07fd0cab-d1e6-4259-bfec-ad80f02d93a2 on atsqa4-127.qa.lab:31010] > (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4783) Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty
[ https://issues.apache.org/jira/browse/DRILL-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386875#comment-15386875 ] ASF GitHub Bot commented on DRILL-4783: --- Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/546#discussion_r71629104 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/nested/TestFastComplexSchema.java --- @@ -80,4 +80,33 @@ public void test4() throws Exception { " AND r.r_regionkey = 4)) t \n" + "ORDER BY t.f.name"); } + + @Test //DRILL-4783 when resultset is empty, don't throw exception. + public void test5() throws Exception { +test("alter session set `planner.enable_hashjoin` = false"); --- End diff -- Any reason why you need set these two options? > Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty > --- > > Key: DRILL-4783 > URL: https://issues.apache.org/jira/browse/DRILL-4783 > Project: Apache Drill > Issue Type: Bug >Reporter: Chunhui Shi >Assignee: Chunhui Shi >Priority: Critical > > Flatten failed to work on top of convert_from when the resultset is empty. > For a HBase table like this: > 0: jdbc:drill:zk=localhost:5181> select convert_from(t.address.cities,'json') > from hbase.`/tmp/flattentest` t; > +--+ > | EXPR$0 > | > +--+ > | {"list":[{"city":"SunnyVale"},{"city":"Palo Alto"},{"city":"Mountain > View"}]}| > | {"list":[{"city":"Seattle"},{"city":"Bellevue"},{"city":"Renton"}]} > | > | {"list":[{"city":"Minneapolis"},{"city":"Falcon Heights"},{"city":"San > Paul"}]} | > +--+ > Flatten works when row_key is in (1,2,3) > 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select > convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t > where row_key=1) t1; > +---+ > | EXPR$0 | > +---+ > | {"city":"SunnyVale"} | > | {"city":"Palo Alto"} | > | {"city":"Mountain View"} | > +---+ > But Flatten throws exception if the resultset is empty > 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select > convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t > where row_key=4) t1; > Error: SYSTEM ERROR: ClassCastException: Cannot cast > org.apache.drill.exec.vector.NullableIntVector to > org.apache.drill.exec.vector.complex.RepeatedValueVector > Fragment 0:0 > [Error Id: 07fd0cab-d1e6-4259-bfec-ad80f02d93a2 on atsqa4-127.qa.lab:31010] > (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4783) Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty
[ https://issues.apache.org/jira/browse/DRILL-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386886#comment-15386886 ] ASF GitHub Bot commented on DRILL-4783: --- Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/546#discussion_r71630181 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java --- @@ -129,11 +129,21 @@ private void setFlattenVector() { try { final TypedFieldId typedFieldId = incoming.getValueVectorId(popConfig.getColumn()); final MaterializedField field = incoming.getSchema().getColumn(typedFieldId.getFieldIds()[0]); - final RepeatedValueVector vector = RepeatedValueVector.class.cast(incoming.getValueAccessorById( - field.getValueClass(), typedFieldId.getFieldIds()).getValueVector()); + RepeatedValueVector vector = null; + try { +vector = RepeatedValueVector.class.cast(incoming.getValueAccessorById( +field.getValueClass(), typedFieldId.getFieldIds()).getValueVector()); + } catch(ClassCastException ex) { --- End diff -- In stead of catch ClassCastException, you may check if the input value vector is an instance of RepeatedValueVector. Using try{} catch{} seems a bit over-kill for this case. > Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty > --- > > Key: DRILL-4783 > URL: https://issues.apache.org/jira/browse/DRILL-4783 > Project: Apache Drill > Issue Type: Bug >Reporter: Chunhui Shi >Assignee: Chunhui Shi >Priority: Critical > > Flatten failed to work on top of convert_from when the resultset is empty. > For a HBase table like this: > 0: jdbc:drill:zk=localhost:5181> select convert_from(t.address.cities,'json') > from hbase.`/tmp/flattentest` t; > +--+ > | EXPR$0 > | > +--+ > | {"list":[{"city":"SunnyVale"},{"city":"Palo Alto"},{"city":"Mountain > View"}]}| > | {"list":[{"city":"Seattle"},{"city":"Bellevue"},{"city":"Renton"}]} > | > | {"list":[{"city":"Minneapolis"},{"city":"Falcon Heights"},{"city":"San > Paul"}]} | > +--+ > Flatten works when row_key is in (1,2,3) > 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select > convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t > where row_key=1) t1; > +---+ > | EXPR$0 | > +---+ > | {"city":"SunnyVale"} | > | {"city":"Palo Alto"} | > | {"city":"Mountain View"} | > +---+ > But Flatten throws exception if the resultset is empty > 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select > convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t > where row_key=4) t1; > Error: SYSTEM ERROR: ClassCastException: Cannot cast > org.apache.drill.exec.vector.NullableIntVector to > org.apache.drill.exec.vector.complex.RepeatedValueVector > Fragment 0:0 > [Error Id: 07fd0cab-d1e6-4259-bfec-ad80f02d93a2 on atsqa4-127.qa.lab:31010] > (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (DRILL-2792) Killing the drillbit which is the foreman results in direct memory being held on
[ https://issues.apache.org/jira/browse/DRILL-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Padma Penumarthy reassigned DRILL-2792: --- Assignee: Padma Penumarthy > Killing the drillbit which is the foreman results in direct memory being held > on > > > Key: DRILL-2792 > URL: https://issues.apache.org/jira/browse/DRILL-2792 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow >Affects Versions: 0.8.0 >Reporter: Ramana Inukonda Nagaraj >Assignee: Padma Penumarthy > Fix For: Future > > > Killed one of the drillbits which is the foreman for the query- > Profiles page reports that query has cancelled. > Due to bug Drill-2778 sqlline hangs. However after killing sqlline the > current direct memory used does not go down to pre query levels. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4783) Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty
[ https://issues.apache.org/jira/browse/DRILL-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386889#comment-15386889 ] ASF GitHub Bot commented on DRILL-4783: --- Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/546#discussion_r71630375 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java --- @@ -129,11 +129,21 @@ private void setFlattenVector() { try { final TypedFieldId typedFieldId = incoming.getValueVectorId(popConfig.getColumn()); final MaterializedField field = incoming.getSchema().getColumn(typedFieldId.getFieldIds()[0]); - final RepeatedValueVector vector = RepeatedValueVector.class.cast(incoming.getValueAccessorById( - field.getValueClass(), typedFieldId.getFieldIds()).getValueVector()); + RepeatedValueVector vector = null; + try { +vector = RepeatedValueVector.class.cast(incoming.getValueAccessorById( +field.getValueClass(), typedFieldId.getFieldIds()).getValueVector()); + } catch(ClassCastException ex) { +if(incoming.getRecordCount() != 0) { + throw UserException.unsupportedError(ex).message("Flatten does not support inputs of non-list values.").build(logger); +} +//when incoming recordCount is 0, don't throw exception since the type being seen here is not solid +logger.error("setFlattenVector cast failed and recordcount is 0, create empty vector anyway."); +vector = new RepeatedMapVector(field, oContext.getAllocator(), null); + } flattener.setFlattenField(vector); } catch (Exception ex) { - throw UserException.unsupportedError(ex).message("Trying to flatten a non-repeated field.").build(logger); + throw UserException.unsupportedError(ex).message("Flatten does not support inputs of non-list values.").build(logger); --- End diff -- UserException may be thrown twice? Line 138 & Line 146. > Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty > --- > > Key: DRILL-4783 > URL: https://issues.apache.org/jira/browse/DRILL-4783 > Project: Apache Drill > Issue Type: Bug >Reporter: Chunhui Shi >Assignee: Chunhui Shi >Priority: Critical > > Flatten failed to work on top of convert_from when the resultset is empty. > For a HBase table like this: > 0: jdbc:drill:zk=localhost:5181> select convert_from(t.address.cities,'json') > from hbase.`/tmp/flattentest` t; > +--+ > | EXPR$0 > | > +--+ > | {"list":[{"city":"SunnyVale"},{"city":"Palo Alto"},{"city":"Mountain > View"}]}| > | {"list":[{"city":"Seattle"},{"city":"Bellevue"},{"city":"Renton"}]} > | > | {"list":[{"city":"Minneapolis"},{"city":"Falcon Heights"},{"city":"San > Paul"}]} | > +--+ > Flatten works when row_key is in (1,2,3) > 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select > convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t > where row_key=1) t1; > +---+ > | EXPR$0 | > +---+ > | {"city":"SunnyVale"} | > | {"city":"Palo Alto"} | > | {"city":"Mountain View"} | > +---+ > But Flatten throws exception if the resultset is empty > 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select > convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t > where row_key=4) t1; > Error: SYSTEM ERROR: ClassCastException: Cannot cast > org.apache.drill.exec.vector.NullableIntVector to > org.apache.drill.exec.vector.complex.RepeatedValueVector > Fragment 0:0 > [Error Id: 07fd0cab-d1e6-4259-bfec-ad80f02d93a2 on atsqa4-127.qa.lab:31010] > (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4783) Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty
[ https://issues.apache.org/jira/browse/DRILL-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386952#comment-15386952 ] ASF GitHub Bot commented on DRILL-4783: --- Github user chunhui-shi commented on a diff in the pull request: https://github.com/apache/drill/pull/546#discussion_r71634838 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java --- @@ -129,11 +129,21 @@ private void setFlattenVector() { try { final TypedFieldId typedFieldId = incoming.getValueVectorId(popConfig.getColumn()); final MaterializedField field = incoming.getSchema().getColumn(typedFieldId.getFieldIds()[0]); - final RepeatedValueVector vector = RepeatedValueVector.class.cast(incoming.getValueAccessorById( - field.getValueClass(), typedFieldId.getFieldIds()).getValueVector()); + RepeatedValueVector vector = null; + try { +vector = RepeatedValueVector.class.cast(incoming.getValueAccessorById( +field.getValueClass(), typedFieldId.getFieldIds()).getValueVector()); + } catch(ClassCastException ex) { +if(incoming.getRecordCount() != 0) { + throw UserException.unsupportedError(ex).message("Flatten does not support inputs of non-list values.").build(logger); +} +//when incoming recordCount is 0, don't throw exception since the type being seen here is not solid +logger.error("setFlattenVector cast failed and recordcount is 0, create empty vector anyway."); +vector = new RepeatedMapVector(field, oContext.getAllocator(), null); + } flattener.setFlattenField(vector); } catch (Exception ex) { - throw UserException.unsupportedError(ex).message("Trying to flatten a non-repeated field.").build(logger); + throw UserException.unsupportedError(ex).message("Flatten does not support inputs of non-list values.").build(logger); --- End diff -- The two catch blocks are parallel. The UserException in line 138 wont be caught and thrown again in line 146. > Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty > --- > > Key: DRILL-4783 > URL: https://issues.apache.org/jira/browse/DRILL-4783 > Project: Apache Drill > Issue Type: Bug >Reporter: Chunhui Shi >Assignee: Chunhui Shi >Priority: Critical > > Flatten failed to work on top of convert_from when the resultset is empty. > For a HBase table like this: > 0: jdbc:drill:zk=localhost:5181> select convert_from(t.address.cities,'json') > from hbase.`/tmp/flattentest` t; > +--+ > | EXPR$0 > | > +--+ > | {"list":[{"city":"SunnyVale"},{"city":"Palo Alto"},{"city":"Mountain > View"}]}| > | {"list":[{"city":"Seattle"},{"city":"Bellevue"},{"city":"Renton"}]} > | > | {"list":[{"city":"Minneapolis"},{"city":"Falcon Heights"},{"city":"San > Paul"}]} | > +--+ > Flatten works when row_key is in (1,2,3) > 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select > convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t > where row_key=1) t1; > +---+ > | EXPR$0 | > +---+ > | {"city":"SunnyVale"} | > | {"city":"Palo Alto"} | > | {"city":"Mountain View"} | > +---+ > But Flatten throws exception if the resultset is empty > 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select > convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t > where row_key=4) t1; > Error: SYSTEM ERROR: ClassCastException: Cannot cast > org.apache.drill.exec.vector.NullableIntVector to > org.apache.drill.exec.vector.complex.RepeatedValueVector > Fragment 0:0 > [Error Id: 07fd0cab-d1e6-4259-bfec-ad80f02d93a2 on atsqa4-127.qa.lab:31010] > (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4794) Regression: Wrong result for query with disjunctive partition filters
Aman Sinha created DRILL-4794: - Summary: Regression: Wrong result for query with disjunctive partition filters Key: DRILL-4794 URL: https://issues.apache.org/jira/browse/DRILL-4794 Project: Apache Drill Issue Type: Bug Components: Query Planning & Optimization Affects Versions: 1.7.0 Reporter: Aman Sinha Assignee: Aman Sinha For a query that contains certain types of disjunctive filter conditions such as 'dir0=x OR dir1=y' we get wrong result when metadata caching is used. This is a regression due to DRILL-4530. Note that the filter involves OR of 2 different directory levels. For the normal case of OR condition at the same level the problem does not occur. Correct result (without metadata cache) {noformat} 0: jdbc:drill:zk=local> select count(*) from dfs.`orders` where dir0=1994 or dir1='Q3' ; +-+ | EXPR$0 | +-+ | 60 | +-+ {noformat} Wrong result (with metadata cache): {noformat} 0: jdbc:drill:zk=local> select count(*) from dfs.`orders` where dir0=1994 or dir1='Q3' ; +-+ | EXPR$0 | +-+ | 50 | +-+ {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4794) Regression: Wrong result for query with disjunctive partition filters
[ https://issues.apache.org/jira/browse/DRILL-4794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387235#comment-15387235 ] ASF GitHub Bot commented on DRILL-4794: --- GitHub user amansinha100 opened a pull request: https://github.com/apache/drill/pull/550 DRILL-4794: Fix a premature exit of the outer loop during pruning. You can merge this pull request into a Git repository by running: $ git pull https://github.com/amansinha100/incubator-drill DRILL-4794 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/drill/pull/550.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #550 commit c13eee3c6c295ddc09c83476c039500ebd3dee72 Author: Aman Sinha Date: 2016-07-21T05:36:02Z DRILL-4794: Fix a premature exit of the outer loop during pruning. > Regression: Wrong result for query with disjunctive partition filters > - > > Key: DRILL-4794 > URL: https://issues.apache.org/jira/browse/DRILL-4794 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning & Optimization >Affects Versions: 1.7.0 >Reporter: Aman Sinha >Assignee: Aman Sinha > > For a query that contains certain types of disjunctive filter conditions such > as 'dir0=x OR dir1=y' we get wrong result when metadata caching is used. > This is a regression due to DRILL-4530. > Note that the filter involves OR of 2 different directory levels. For the > normal case of OR condition at the same level the problem does not occur. > Correct result (without metadata cache) > {noformat} > 0: jdbc:drill:zk=local> select count(*) from dfs.`orders` where dir0=1994 or > dir1='Q3' ; > +-+ > | EXPR$0 | > +-+ > | 60 | > +-+ > {noformat} > Wrong result (with metadata cache): > {noformat} > 0: jdbc:drill:zk=local> select count(*) from dfs.`orders` where dir0=1994 or > dir1='Q3' ; > +-+ > | EXPR$0 | > +-+ > | 50 | > +-+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4794) Regression: Wrong result for query with disjunctive partition filters
[ https://issues.apache.org/jira/browse/DRILL-4794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387245#comment-15387245 ] ASF GitHub Bot commented on DRILL-4794: --- Github user amansinha100 commented on the issue: https://github.com/apache/drill/pull/550 @jinfengni could you pls review this small patch ? The bug was caused by the fact that there was an extraneous 'break' statement that exited from the outer loop (my intent was to only exit from an inner loop when isSinglePartition=false ... but in this case there was no inner loop). > Regression: Wrong result for query with disjunctive partition filters > - > > Key: DRILL-4794 > URL: https://issues.apache.org/jira/browse/DRILL-4794 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning & Optimization >Affects Versions: 1.7.0 >Reporter: Aman Sinha >Assignee: Aman Sinha > > For a query that contains certain types of disjunctive filter conditions such > as 'dir0=x OR dir1=y' we get wrong result when metadata caching is used. > This is a regression due to DRILL-4530. > Note that the filter involves OR of 2 different directory levels. For the > normal case of OR condition at the same level the problem does not occur. > Correct result (without metadata cache) > {noformat} > 0: jdbc:drill:zk=local> select count(*) from dfs.`orders` where dir0=1994 or > dir1='Q3' ; > +-+ > | EXPR$0 | > +-+ > | 60 | > +-+ > {noformat} > Wrong result (with metadata cache): > {noformat} > 0: jdbc:drill:zk=local> select count(*) from dfs.`orders` where dir0=1994 or > dir1='Q3' ; > +-+ > | EXPR$0 | > +-+ > | 50 | > +-+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)