[jira] [Created] (DRILL-4809) Drill to provide ability to support parameterized conditions
Yuliya Feldman created DRILL-4809: - Summary: Drill to provide ability to support parameterized conditions Key: DRILL-4809 URL: https://issues.apache.org/jira/browse/DRILL-4809 Project: Apache Drill Issue Type: Improvement Components: Execution - Flow, Query Planning & Optimization, SQL Parser Reporter: Yuliya Feldman Currently Drill does not provide ability to specify variables in the WHERE clause which means that user has to create a new query to handle any new condition. For example if someone wants to execute following query: select id, name from foo where dir0=$1 and dir1=$2 (s)he unable to do it and thus if dir0 and dir1 get created on the fly (by day, month or what not) new query needs to be created to handle data in new directories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4597) Calcite type validation assertions when planner.enable_type_inference is enabled for system tables
Yuliya Feldman created DRILL-4597: - Summary: Calcite type validation assertions when planner.enable_type_inference is enabled for system tables Key: DRILL-4597 URL: https://issues.apache.org/jira/browse/DRILL-4597 Project: Apache Drill Issue Type: Bug Components: Query Planning & Optimization Reporter: Yuliya Feldman With calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11 and type inference enabled following query fails: select concat(hostname, ':', user_port) from sys.drillbits where `current`=true; with below exception at org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:62) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11] at org.apache.drill.exec.planner.sql.TypeInferenceUtils$DrillConcatSqlReturnTypeInference.inferReturnType(TypeInferenceUtils.java:420) ~[classes/:na] at org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:468) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11] at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:435) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11] at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:287) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11] at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:222) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11] at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:4288) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11] at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:4275) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11] at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:130) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11] at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1495) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11] at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1478) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11] at org.apache.calcite.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:440) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11] at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:3447) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11] at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:2995) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11] at org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11] at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:86) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11] at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:877) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11] at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:863) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11] at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:210) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11] at org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:837) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11] at org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:551) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11] at org.apache.drill.exec.planner.sql.SqlConverter.validate(SqlConverter.java:155) ~[classes/:na] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode(DefaultSqlHandler.java:596) ~[classes/:na] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert(DefaultSqlHandler.java:192) ~[classes/:na] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:164) ~[classes/:na] at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:94) ~[classes/:na] at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:970) [classes/:na] at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:254) [classes/:na] So far we could not repro it on non-system tables, but any type inference on system table leads to the exception -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4412) Have an array of DrillBitEndPoints (at least) for leaf fragments instead of single one
Yuliya Feldman created DRILL-4412: - Summary: Have an array of DrillBitEndPoints (at least) for leaf fragments instead of single one Key: DRILL-4412 URL: https://issues.apache.org/jira/browse/DRILL-4412 Project: Apache Drill Issue Type: Improvement Components: Query Planning & Optimization Reporter: Yuliya Feldman Assignee: Yuliya Feldman To follow up on the ability to submit simple physical plan directly to a DrillBit for execution [JIRA-4132|https://issues.apache.org/jira/browse/DRILL-4132] it would be beneficial to have an array of DrillBitEndPoint in PlanFragment. Leaf fragments that scan the data can have an array of DrillBitEndPoint based on data locality, as data may be replicated and in case it is necessary to restart Scan fragment it can be restarted on DrillBits that have replica of the data, versus always retrying the same DrillBit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-2209) Save on CPU cycles by adding Project with column that has hash calculated
Yuliya Feldman created DRILL-2209: - Summary: Save on CPU cycles by adding Project with column that has hash calculated Key: DRILL-2209 URL: https://issues.apache.org/jira/browse/DRILL-2209 Project: Apache Drill Issue Type: Improvement Components: Execution - Flow, Query Planning Optimization Reporter: Yuliya Feldman Assignee: Yuliya Feldman Related to DRILL-133. Wrapping HashToRandomExhcnage and/or LocalExchange with Project operator with additional column that represents hash function of column(s) we are hashing on. This is to save CPU cycles and not recalculate hash every time -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-2210) Allow multithreaded copy and/or flush in ParittionSender
Yuliya Feldman created DRILL-2210: - Summary: Allow multithreaded copy and/or flush in ParittionSender Key: DRILL-2210 URL: https://issues.apache.org/jira/browse/DRILL-2210 Project: Apache Drill Issue Type: Improvement Components: Execution - Flow Reporter: Yuliya Feldman Assignee: Yuliya Feldman Related to DRILL-133. As in LocalExchange we merge data from multiple receivers into LocalExchange to fan it out later to multiple Senders, amount of data that needs to be sent out increases. Add ability to copy/flush data in multiple threads -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-1926) Fix back pressure logic
Yuliya Feldman created DRILL-1926: - Summary: Fix back pressure logic Key: DRILL-1926 URL: https://issues.apache.org/jira/browse/DRILL-1926 Project: Apache Drill Issue Type: Bug Components: Execution - Flow Reporter: Yuliya Feldman Assignee: Yuliya Feldman While enqueueing coming requests in UnlimitedRawBatchBuffer replies to the sender(s) will queue up only if size of the queue is equal to soft limit and not when it is = softlimit - which means it will work only once, while requests will be piling up. Also improving logic of sending responses back to the senders by not just sending them in one shot that can create flood of requests again, but in batches based on difference between softlimit and queue size -- This message was sent by Atlassian JIRA (v6.3.4#6332)
