[jira] [Created] (FLINK-12420) Add support of cache/invalidateCache for TableAPI
Ruidong Li created FLINK-12420: -- Summary: Add support of cache/invalidateCache for TableAPI Key: FLINK-12420 URL: https://issues.apache.org/jira/browse/FLINK-12420 Project: Flink Issue Type: Sub-task Reporter: Ruidong Li Assignee: Ruidong Li Add cache/invalidateCache api and its implementations for Table -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-12419) Add support for consuming BLOCKING_PERSISTENT ResultPartition
Ruidong Li created FLINK-12419: -- Summary: Add support for consuming BLOCKING_PERSISTENT ResultPartition Key: FLINK-12419 URL: https://issues.apache.org/jira/browse/FLINK-12419 Project: Flink Issue Type: Sub-task Components: Runtime / Coordination Reporter: Ruidong Li Assignee: Ruidong Li Add support for deploying Jobs which can consume BLOCKING_PERSISTENT ResultPartition generated by previous Jobs -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-12406) Report BLOCKING_PERSISTENT result partition meta back to client
Ruidong Li created FLINK-12406:
--
Summary: Report BLOCKING_PERSISTENT result partition meta back to
client
Key: FLINK-12406
URL: https://issues.apache.org/jira/browse/FLINK-12406
Project: Flink
Issue Type: Sub-task
Components: Runtime / Coordination
Reporter: Ruidong Li
Assignee: Ruidong Li
After each job finishes, the new {{BLOCKING_PERSISTENT}} result partitions are
generated, and locations of these result partitions should be report back to
client via {{JobExecutionResult}}, they will be later used for Table
{{cache()}} and {{invalidateCache()}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (FLINK-12405) Introduce BLOCKING_PERSISTENT result partition type
Ruidong Li created FLINK-12405:
--
Summary: Introduce BLOCKING_PERSISTENT result partition type
Key: FLINK-12405
URL: https://issues.apache.org/jira/browse/FLINK-12405
Project: Flink
Issue Type: Sub-task
Components: API / DataSet
Reporter: Ruidong Li
Assignee: Ruidong Li
The new {{ResultPartitionType}} : {{BLOCKING_PERSISTENT}} is similar to
{{BLOCKING}} except it might be consumed for several times and will be released
after TM shutdown or {{ResultPartition}} removal request.
This is the basis for Interactive Programming.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (FLINK-12000) Introduce TableServiceManager to TableEnvironment
Ruidong Li created FLINK-12000: -- Summary: Introduce TableServiceManager to TableEnvironment Key: FLINK-12000 URL: https://issues.apache.org/jira/browse/FLINK-12000 Project: Flink Issue Type: Sub-task Components: Table SQL / API Reporter: Ruidong Li Assignee: Ruidong Li -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-11999) Introduce cache and invalidateCache API to Table
Ruidong Li created FLINK-11999: -- Summary: Introduce cache and invalidateCache API to Table Key: FLINK-11999 URL: https://issues.apache.org/jira/browse/FLINK-11999 Project: Flink Issue Type: Sub-task Components: Table SQL / API Reporter: Ruidong Li Assignee: Ruidong Li -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-11998) Flink Interactive Programming (Umbrella JIRA)
Ruidong Li created FLINK-11998: -- Summary: Flink Interactive Programming (Umbrella JIRA) Key: FLINK-11998 URL: https://issues.apache.org/jira/browse/FLINK-11998 Project: Flink Issue Type: Improvement Components: Table SQL / API, Table SQL / Planner, Table SQL / Runtime Reporter: Ruidong Li Assignee: Ruidong Li This is the Umbrella JIRA for [FLIP-36|https://cwiki.apache.org/confluence/display/FLINK/FLIP-36%3A+Support+Interactive+Programming+in+Flink]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-10670) Fix Correlate codegen error
Ruidong Li created FLINK-10670: -- Summary: Fix Correlate codegen error Key: FLINK-10670 URL: https://issues.apache.org/jira/browse/FLINK-10670 Project: Flink Issue Type: Bug Components: Table API SQL Reporter: Ruidong Li Assignee: Ruidong Li TableFunctionCollector should handle reuseInitCode and reuseMemberCode -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-10451) TableFunctionCollector should handle the life cycle of ScalarFunction
Ruidong Li created FLINK-10451:
--
Summary: TableFunctionCollector should handle the life cycle of
ScalarFunction
Key: FLINK-10451
URL: https://issues.apache.org/jira/browse/FLINK-10451
Project: Flink
Issue Type: Bug
Components: Table API SQL
Reporter: Ruidong Li
Assignee: Ruidong Li
Considering the following query:
table.join(udtf('a)).where(udf('b))
the filter will be pushed into DataSetCorrelate/DataStreamCorrelate without
triggering open() and close()
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (FLINK-9997) Improve ExpressionReduce
Ruidong Li created FLINK-9997: - Summary: Improve ExpressionReduce Key: FLINK-9997 URL: https://issues.apache.org/jira/browse/FLINK-9997 Project: Flink Issue Type: Improvement Components: Table API SQL Reporter: Ruidong Li Assignee: Ruidong Li RepressionReduce do not reduce some expressions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-9958) Fix potential NPE for delta iteration of DataSet
Ruidong Li created FLINK-9958: - Summary: Fix potential NPE for delta iteration of DataSet Key: FLINK-9958 URL: https://issues.apache.org/jira/browse/FLINK-9958 Project: Flink Issue Type: Bug Components: DataSet API Reporter: Ruidong Li Assignee: Ruidong Li -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-9433) SystemProcessingTimeService does not work properly
Ruidong Li created FLINK-9433: - Summary: SystemProcessingTimeService does not work properly Key: FLINK-9433 URL: https://issues.apache.org/jira/browse/FLINK-9433 Project: Flink Issue Type: Improvement Components: Streaming Reporter: Ruidong Li Assignee: Ruidong Li if (WindowOperator --> AsyncWaitOperator) chained together, when the queue of AsyncWaitOperator is full and timeTrigger of WindowOperator is triggered to call collect(), it will wait until the queue of AsyncWaitOperator is not full, at the moment, the timeTrigger of AsyncWaitOperator will not be triggered because the SystemProcessingTimeService has only one capacity. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-9346) Update documents for supporting intersect/except in streaming
Ruidong Li created FLINK-9346: - Summary: Update documents for supporting intersect/except in streaming Key: FLINK-9346 URL: https://issues.apache.org/jira/browse/FLINK-9346 Project: Flink Issue Type: Improvement Components: Table API SQL Reporter: Ruidong Li Assignee: Ruidong Li Update documents for supporting intersect/except in streaming -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-9345) Support EXCEPT and EXCEPT ALL for streaming
Ruidong Li created FLINK-9345: - Summary: Support EXCEPT and EXCEPT ALL for streaming Key: FLINK-9345 URL: https://issues.apache.org/jira/browse/FLINK-9345 Project: Flink Issue Type: Improvement Components: Table API SQL Reporter: Ruidong Li Assignee: Ruidong Li support except, except all for SQL and minus, minus all for TableAPI -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-9344) Support INTERSECT and INTERSECT ALL for streaming
Ruidong Li created FLINK-9344: - Summary: Support INTERSECT and INTERSECT ALL for streaming Key: FLINK-9344 URL: https://issues.apache.org/jira/browse/FLINK-9344 Project: Flink Issue Type: Improvement Components: Table API SQL Reporter: Ruidong Li Assignee: Ruidong Li support intersect and intersect all for both SQL and TableAPI -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-9332) Fix Codegen error of CallGenerator
Ruidong Li created FLINK-9332: - Summary: Fix Codegen error of CallGenerator Key: FLINK-9332 URL: https://issues.apache.org/jira/browse/FLINK-9332 Project: Flink Issue Type: Bug Components: Table API SQL Reporter: Ruidong Li Assignee: Ruidong Li function call may return null, but nullTerm did not handle it correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-8868) Support Table Function as Table
Ruidong Li created FLINK-8868:
-
Summary: Support Table Function as Table
Key: FLINK-8868
URL: https://issues.apache.org/jira/browse/FLINK-8868
Project: Flink
Issue Type: Improvement
Components: Table API SQL
Reporter: Ruidong Li
Assignee: Ruidong Li
support SQL like: SELECT * FROM TABLE(tf("a"))
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (FLINK-8821) Fix BigDecimal divide in AvgAggFunction
Ruidong Li created FLINK-8821: - Summary: Fix BigDecimal divide in AvgAggFunction Key: FLINK-8821 URL: https://issues.apache.org/jira/browse/FLINK-8821 Project: Flink Issue Type: Bug Components: Table API SQL Reporter: Ruidong Li Assignee: Ruidong Li The DecimalAvgAggFunction lacks precision protection -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-8617) Fix code generation bug while accessing Map type
Ruidong Li created FLINK-8617:
-
Summary: Fix code generation bug while accessing Map type
Key: FLINK-8617
URL: https://issues.apache.org/jira/browse/FLINK-8617
Project: Flink
Issue Type: Bug
Components: Table API SQL
Reporter: Ruidong Li
Assignee: Ruidong Li
There's a code generation bug in {code}ScalarOperatos.generateMapGet{code}.
And there's two more bugs found in {code}ScalarOperators.generateIsNull{code}
and {code}ScalarOperators.generateIsNotNull{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (FLINK-8555) Fix TableFunction varargs length exceeds 254 for SQL
Ruidong Li created FLINK-8555: - Summary: Fix TableFunction varargs length exceeds 254 for SQL Key: FLINK-8555 URL: https://issues.apache.org/jira/browse/FLINK-8555 Project: Flink Issue Type: Improvement Components: Table API SQL Reporter: Ruidong Li Assignee: Ruidong Li With Varargs, TableAPI can handle table function call with parameters exceeds 254 correctly. This issue is intend to support long parameters for SQL -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-8312) Fix ScalarFunction varargs length exceeds 254 for SQL
Ruidong Li created FLINK-8312: - Summary: Fix ScalarFunction varargs length exceeds 254 for SQL Key: FLINK-8312 URL: https://issues.apache.org/jira/browse/FLINK-8312 Project: Flink Issue Type: Bug Components: Table API & SQL Reporter: Ruidong Li Assignee: Ruidong Li With Varargs, TableAPI can handle scalar function call with parameters exceeds 254 correctly. This issue is intend to support long parameters for SQL -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-8301) Support Unicode in codegen for TableAPI && SQL
Ruidong Li created FLINK-8301: - Summary: Support Unicode in codegen for TableAPI && SQL Key: FLINK-8301 URL: https://issues.apache.org/jira/browse/FLINK-8301 Project: Flink Issue Type: Improvement Components: Table API & SQL Reporter: Ruidong Li Assignee: Ruidong Li The current code generation do not support Unicode, "\u0001" will be generated to "\\u0001", function call like concat(str, "\u0001") will lead to wrong result. This issue intend to handle char/varchar literal correctly, some examples followed as below. literal codegen '\u0001abc' "\u0001abc" '\u0022\' "\"\\" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-8095) Introduce ProjectSetOpTransposeRule to Flink
Ruidong Li created FLINK-8095: - Summary: Introduce ProjectSetOpTransposeRule to Flink Key: FLINK-8095 URL: https://issues.apache.org/jira/browse/FLINK-8095 Project: Flink Issue Type: Improvement Reporter: Ruidong Li Assignee: Ruidong Li ProjectSetOpTransposeRule is similar to FilterSetOpTransposeRule, adding ProjectSetOpTransposeRule is necessary. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-7986) Introduce FilterSetOpTransposeRule to Flink
Ruidong Li created FLINK-7986: - Summary: Introduce FilterSetOpTransposeRule to Flink Key: FLINK-7986 URL: https://issues.apache.org/jira/browse/FLINK-7986 Project: Flink Issue Type: Improvement Reporter: Ruidong Li Assignee: Ruidong Li Priority: Trivial A.unionAll(B).where.groupBy.select => A.where.unionAll(B.where).groupBy.select this rule will reduce networkIO -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-7776) remove duplicate records in group aggregation
Ruidong Li created FLINK-7776:
-
Summary: remove duplicate records in group aggregation
Key: FLINK-7776
URL: https://issues.apache.org/jira/browse/FLINK-7776
Project: Flink
Issue Type: Improvement
Components: Table API & SQL
Reporter: Ruidong Li
Assignee: Ruidong Li
the current group aggregation will compare the last {{Row}} and current {{Row}}
when {{generateRetraction}} is {{true}} and {{firstRow}} is {{false}} in
{{GroupAggProcessFunction}},
this logic should be applied to all cases when {{firstRow}} is false
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (FLINK-7759) Fix Bug that fieldName/functionName with prefix equals "true" or "false" can't be parsed by ExpressionParser.
Ruidong Li created FLINK-7759:
-
Summary: Fix Bug that fieldName/functionName with prefix equals
"true" or "false" can't be parsed by ExpressionParser.
Key: FLINK-7759
URL: https://issues.apache.org/jira/browse/FLINK-7759
Project: Flink
Issue Type: Bug
Components: Table API & SQL
Reporter: Ruidong Li
Assignee: Ruidong Li
just call {{ExpressionParser.parseExpression}} with a prefix equals "true" or
"false"
{{ExpressionParser.parseExpression("true_target")}} or
{{ExpressionParser.parseExpression("falsex")}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (FLINK-7596) Fix bug during Set Operation (Union, Minus ... ) with Any(GenericRelDataType)
Ruidong Li created FLINK-7596:
-
Summary: Fix bug during Set Operation (Union, Minus ... ) with
Any(GenericRelDataType)
Key: FLINK-7596
URL: https://issues.apache.org/jira/browse/FLINK-7596
Project: Flink
Issue Type: Bug
Components: Table API & SQL
Reporter: Ruidong Li
Assignee: Ruidong Li
If two inputs with Any(GenericRelDataType), when they comes to Set
Operation(Union, minus...), it will cause a `TableException` with info is "Type
is not supported: ANY"
Here is the test case:
`
@Test
def testUnion(): Unit = {
val list = List((1, new NODE), (2, new NODE))
val list2 = List((3, new NODE), (4, new NODE))
val env = StreamExecutionEnvironment.getExecutionEnvironment
val tEnv = TableEnvironment.getTableEnvironment(env)
val s1 = tEnv.fromDataStream(env.fromCollection(list))
val s2 = tEnv.fromDataStream(env.fromCollection(list2))
val result = s1.unionAll(s2).toAppendStream[Row]
result.addSink(new StreamITCase.StringSink[Row])
env.execute()
}
class NODE {
val x = new util.HashMap[String, String]()
}
`
this bug happens because flink did't handle createSqlType(ANY) and Calcite
does't know the differences between `ANY` and `ANY(GenericRelDataType)`, so the
`createSqlType(ANY)` of Calcite will return a BasicSqlType instead
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (FLINK-7385) Fix ArrayIndexOutOfBoundsException when object-reuse is enabled
Ruidong Li created FLINK-7385: - Summary: Fix ArrayIndexOutOfBoundsException when object-reuse is enabled Key: FLINK-7385 URL: https://issues.apache.org/jira/browse/FLINK-7385 Project: Flink Issue Type: Bug Reporter: Ruidong Li Assignee: Ruidong Li In OperatorChain.java, there is a potential ArrayIndexOutOfBoundsException when object-reuse is enabled -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-7168) Add support for Table API & SQL steam-stream inner/left join
Ruidong Li created FLINK-7168: - Summary: Add support for Table API & SQL steam-stream inner/left join Key: FLINK-7168 URL: https://issues.apache.org/jira/browse/FLINK-7168 Project: Flink Issue Type: Improvement Components: Table API & SQL Reporter: Ruidong Li Assignee: Ruidong Li -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-7163) Make LogicalTableFunctionCall immutable
Ruidong Li created FLINK-7163: - Summary: Make LogicalTableFunctionCall immutable Key: FLINK-7163 URL: https://issues.apache.org/jira/browse/FLINK-7163 Project: Flink Issue Type: Improvement Components: Table API & SQL Reporter: Ruidong Li Assignee: Ruidong Li All sub-classes of LogicalNode are immutable except for `LogicalTableFunctionCall`, it's better to make LogicalTableFunctionCall immutable -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-7014) Expose isDeterministic interface to ScalarFunction and TableFunction
Ruidong Li created FLINK-7014: - Summary: Expose isDeterministic interface to ScalarFunction and TableFunction Key: FLINK-7014 URL: https://issues.apache.org/jira/browse/FLINK-7014 Project: Flink Issue Type: Improvement Components: Table API & SQL Reporter: Ruidong Li Currently, the `isDeterministic` method of implementations of `SqlFuntion` are always returning true, which cause inappropriate optimization in Calcite, such as taking user's stateful UDF as a pure functional procedure. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-6992) add support for IN, NOT IN in tableAPI
Ruidong Li created FLINK-6992:
-
Summary: add support for IN, NOT IN in tableAPI
Key: FLINK-6992
URL: https://issues.apache.org/jira/browse/FLINK-6992
Project: Flink
Issue Type: Improvement
Components: Table API & SQL
Reporter: Ruidong Li
Assignee: Ruidong Li
Priority: Minor
to support table.where('a in ("x", "y", "z"))
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (FLINK-6852) Fix misuse of GCD
Ruidong Li created FLINK-6852: - Summary: Fix misuse of GCD Key: FLINK-6852 URL: https://issues.apache.org/jira/browse/FLINK-6852 Project: Flink Issue Type: Improvement Reporter: Ruidong Li Assignee: Ruidong Li Priority: Minor There's no need to compare and swap values for gcd input -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (FLINK-6593) Fix Bug in ProctimeAttribute or RowtimeAttribute with join UDTF
Ruidong Li created FLINK-6593:
-
Summary: Fix Bug in ProctimeAttribute or RowtimeAttribute with
join UDTF
Key: FLINK-6593
URL: https://issues.apache.org/jira/browse/FLINK-6593
Project: Flink
Issue Type: Bug
Reporter: Ruidong Li
Assignee: Ruidong Li
when a Table has ProctimeAttribute or RowtimeAttribute and join with A UDTF
function, the `resultNames` parameter of `generateResultExpression` method in
class `CodeGenerator` did not filter the time indicators, causing a
CodeGenException("Arity of result field names does not match number of
expressions.")
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (FLINK-6457) Clean up ScalarFunction interface
Ruidong Li created FLINK-6457: - Summary: Clean up ScalarFunction interface Key: FLINK-6457 URL: https://issues.apache.org/jira/browse/FLINK-6457 Project: Flink Issue Type: Improvement Reporter: Ruidong Li Assignee: Ruidong Li clean up ScalarFunction interface -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (FLINK-6441) Improve the UDTF
Ruidong Li created FLINK-6441: - Summary: Improve the UDTF Key: FLINK-6441 URL: https://issues.apache.org/jira/browse/FLINK-6441 Project: Flink Issue Type: Improvement Reporter: Ruidong Li Assignee: Ruidong Li -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (FLINK-6334) Refactoring UDTF interface
Ruidong Li created FLINK-6334:
-
Summary: Refactoring UDTF interface
Key: FLINK-6334
URL: https://issues.apache.org/jira/browse/FLINK-6334
Project: Flink
Issue Type: Improvement
Components: Table API & SQL
Reporter: Ruidong Li
UDTF's apply method returns a Table Type, so Join(UDTF('a, 'b, ...) as 'c)
shall be viewed as join(Table)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
