[jira] [Commented] (FLINK-31025) Release Testing: Verify FLINK-30650 Introduce EXPLAIN PLAN_ADVICE to provide SQL advice
[
https://issues.apache.org/jira/browse/FLINK-31025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688845#comment-17688845
]
Jane Chan commented on FLINK-31025:
---
[~Weijie Guo] Thanks for testing thoroughly and finding the typo! I'll take a
look right now.
> Release Testing: Verify FLINK-30650 Introduce EXPLAIN PLAN_ADVICE to provide
> SQL advice
> ---
>
> Key: FLINK-31025
> URL: https://issues.apache.org/jira/browse/FLINK-31025
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / API
>Affects Versions: 1.17.0
>Reporter: Jane Chan
>Assignee: Weijie Guo
>Priority: Blocker
> Fix For: 1.17.0
>
> Attachments: image-2023-02-13-14-48-53-758.png,
> image-2023-02-13-14-52-41-855.png, test1.png
>
>
> This ticket aims for verifying FLINK-30650: Introduce EXPLAIN PLAN_ADVICE to
> provide SQL advice.
> More details about this feature and how to use it can be found in this
> [documentation|https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/explain/].
> The verification is divided into two parts:
> Part I: Verify "EXPLAIN PLAN_ADVICE" can work with different queries under
> the streaming mode, such as a single select/insert/statement set w/ or w/o
> sub-plan reuse (configured by "table.optimizer.reuse-sub-plan-enabled").
> This indicates once specifying the ExplainDetail as "PLAN_ADVICE"
> {code:sql}
> EXPLAIN PLAN_ADVICE [SELECT ... FROM ...| INSERT INTO ...| EXECUTE STATEMENT
> SET BEGIN INSERT INTO ... END]
> {code}
> You should find the output should be like the following format (note that the
> title is changed to "Optimized Physical Plan with Advice")
> {code:sql}
> == Abstract Syntax Tree ==
> ...
>
> == Optimized Physical Plan With Advice ==
> ...
>
>
> == Optimized Execution Plan ==
> ...
> {code}
> The available advice is attached at the end of "== Optimized Physical Plan
> With Advice ==", and in front of "== Optimized Execution Plan =="
> If switching to batch mode, you should find the "EXPLAIN PLAN_ADVICE" should
> throw UnsupportedOperationException as expected.
>
> Part II: Verify the advice content
> Write a group aggregate query, and enable/disable the local-global two-phase
> configuration, and test the output.
> You should find once the following configurations are enabled, you will get
> the "no available advice..." output.
> {code:java}
> table.exec.mini-batch.enabled
> table.exec.mini-batch.allow-latency
> table.exec.mini-batch.size
> table.optimizer.agg-phase-strategy {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-31025) Release Testing: Verify FLINK-30650 Introduce EXPLAIN PLAN_ADVICE to provide SQL advice
[
https://issues.apache.org/jira/browse/FLINK-31025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17687766#comment-17687766
]
Weijie Guo commented on FLINK-31025:
BTW. I also found a type in `explain.md`, I have created
`https://github.com/apache/flink/pull/21913` to fix this, could you help
reviewing this?
> Release Testing: Verify FLINK-30650 Introduce EXPLAIN PLAN_ADVICE to provide
> SQL advice
> ---
>
> Key: FLINK-31025
> URL: https://issues.apache.org/jira/browse/FLINK-31025
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / API
>Affects Versions: 1.17.0
>Reporter: Jane Chan
>Assignee: Weijie Guo
>Priority: Blocker
> Fix For: 1.17.0
>
> Attachments: image-2023-02-13-14-48-53-758.png,
> image-2023-02-13-14-52-41-855.png, test1.png
>
>
> This ticket aims for verifying FLINK-30650: Introduce EXPLAIN PLAN_ADVICE to
> provide SQL advice.
> More details about this feature and how to use it can be found in this
> [documentation|https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/explain/].
> The verification is divided into two parts:
> Part I: Verify "EXPLAIN PLAN_ADVICE" can work with different queries under
> the streaming mode, such as a single select/insert/statement set w/ or w/o
> sub-plan reuse (configured by "table.optimizer.reuse-sub-plan-enabled").
> This indicates once specifying the ExplainDetail as "PLAN_ADVICE"
> {code:sql}
> EXPLAIN PLAN_ADVICE [SELECT ... FROM ...| INSERT INTO ...| EXECUTE STATEMENT
> SET BEGIN INSERT INTO ... END]
> {code}
> You should find the output should be like the following format (note that the
> title is changed to "Optimized Physical Plan with Advice")
> {code:sql}
> == Abstract Syntax Tree ==
> ...
>
> == Optimized Physical Plan With Advice ==
> ...
>
>
> == Optimized Execution Plan ==
> ...
> {code}
> The available advice is attached at the end of "== Optimized Physical Plan
> With Advice ==", and in front of "== Optimized Execution Plan =="
> If switching to batch mode, you should find the "EXPLAIN PLAN_ADVICE" should
> throw UnsupportedOperationException as expected.
>
> Part II: Verify the advice content
> Write a group aggregate query, and enable/disable the local-global two-phase
> configuration, and test the output.
> You should find once the following configurations are enabled, you will get
> the "no available advice..." output.
> {code:java}
> table.exec.mini-batch.enabled
> table.exec.mini-batch.allow-latency
> table.exec.mini-batch.size
> table.optimizer.agg-phase-strategy {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-31025) Release Testing: Verify FLINK-30650 Introduce EXPLAIN PLAN_ADVICE to provide SQL advice
[
https://issues.apache.org/jira/browse/FLINK-31025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17687761#comment-17687761
]
Weijie Guo commented on FLINK-31025:
I verified this feature according to the description, and everything seems to
meet expectations.
Verification steps:
I just create a simple table like
```
CREATE TABLE MyTable(
f1 INT,
f2 STRING
) WITH (
'connector' = 'filesystem',
'path' = '/data.csv',
'format' = 'csv'
);
```
1. Execute `EXPLAIN PLAN_ADVICE SELECT max(f1) FROM MyTable GROUP BY f2;`, the
results are shown in the figure below:
!test1.png|width=563,height=326!
We can see the advice before `Optimized Execution Plan`.
2. Set runtime-mode to batch, re-execute the explain cmd, the results are shown
in the figure below:
!image-2023-02-13-14-48-53-758.png|width=573,height=99!
3. Set parameters as follow:
```
SET 'table.exec.mini-batch.enabled' = 'true';
SET 'table.exec.mini-batch.allow-latency' = '5s';
SET 'table.exec.mini-batch.size' = '200';
SET 'table.optimizer.agg-phase-strategy' = 'AUTO';
```
re-execute the explain cmd, the result are shown in the figure blow:
!image-2023-02-13-14-52-41-855.png|width=475,height=244!
> Release Testing: Verify FLINK-30650 Introduce EXPLAIN PLAN_ADVICE to provide
> SQL advice
> ---
>
> Key: FLINK-31025
> URL: https://issues.apache.org/jira/browse/FLINK-31025
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / API
>Affects Versions: 1.17.0
>Reporter: Jane Chan
>Assignee: Weijie Guo
>Priority: Blocker
> Fix For: 1.17.0
>
> Attachments: image-2023-02-13-14-48-53-758.png,
> image-2023-02-13-14-52-41-855.png, test1.png
>
>
> This ticket aims for verifying FLINK-30650: Introduce EXPLAIN PLAN_ADVICE to
> provide SQL advice.
> More details about this feature and how to use it can be found in this
> [documentation|https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/explain/].
> The verification is divided into two parts:
> Part I: Verify "EXPLAIN PLAN_ADVICE" can work with different queries under
> the streaming mode, such as a single select/insert/statement set w/ or w/o
> sub-plan reuse (configured by "table.optimizer.reuse-sub-plan-enabled").
> This indicates once specifying the ExplainDetail as "PLAN_ADVICE"
> {code:sql}
> EXPLAIN PLAN_ADVICE [SELECT ... FROM ...| INSERT INTO ...| EXECUTE STATEMENT
> SET BEGIN INSERT INTO ... END]
> {code}
> You should find the output should be like the following format (note that the
> title is changed to "Optimized Physical Plan with Advice")
> {code:sql}
> == Abstract Syntax Tree ==
> ...
>
> == Optimized Physical Plan With Advice ==
> ...
>
>
> == Optimized Execution Plan ==
> ...
> {code}
> The available advice is attached at the end of "== Optimized Physical Plan
> With Advice ==", and in front of "== Optimized Execution Plan =="
> If switching to batch mode, you should find the "EXPLAIN PLAN_ADVICE" should
> throw UnsupportedOperationException as expected.
>
> Part II: Verify the advice content
> Write a group aggregate query, and enable/disable the local-global two-phase
> configuration, and test the output.
> You should find once the following configurations are enabled, you will get
> the "no available advice..." output.
> {code:java}
> table.exec.mini-batch.enabled
> table.exec.mini-batch.allow-latency
> table.exec.mini-batch.size
> table.optimizer.agg-phase-strategy {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-31025) Release Testing: Verify FLINK-30650 Introduce EXPLAIN PLAN_ADVICE to provide SQL advice
[
https://issues.apache.org/jira/browse/FLINK-31025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17687755#comment-17687755
]
Weijie Guo commented on FLINK-31025:
Thanks [~qingyue] for creating this, I'd like to do this testing work.
> Release Testing: Verify FLINK-30650 Introduce EXPLAIN PLAN_ADVICE to provide
> SQL advice
> ---
>
> Key: FLINK-31025
> URL: https://issues.apache.org/jira/browse/FLINK-31025
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / API
>Affects Versions: 1.17.0
>Reporter: Jane Chan
>Priority: Blocker
> Fix For: 1.17.0
>
>
> This ticket aims for verifying FLINK-30650: Introduce EXPLAIN PLAN_ADVICE to
> provide SQL advice.
> More details about this feature and how to use it can be found in this
> [documentation|https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/explain/].
> The verification is divided into two parts:
> Part I: Verify "EXPLAIN PLAN_ADVICE" can work with different queries under
> the streaming mode, such as a single select/insert/statement set w/ or w/o
> sub-plan reuse (configured by "table.optimizer.reuse-sub-plan-enabled").
> This indicates once specifying the ExplainDetail as "PLAN_ADVICE"
> {code:sql}
> EXPLAIN PLAN_ADVICE [SELECT ... FROM ...| INSERT INTO ...| EXECUTE STATEMENT
> SET BEGIN INSERT INTO ... END]
> {code}
> You should find the output should be like the following format (note that the
> title is changed to "Optimized Physical Plan with Advice")
> {code:sql}
> == Abstract Syntax Tree ==
> ...
>
> == Optimized Physical Plan With Advice ==
> ...
>
>
> == Optimized Execution Plan ==
> ...
> {code}
> The available advice is attached at the end of "== Optimized Physical Plan
> With Advice ==", and in front of "== Optimized Execution Plan =="
> If switching to batch mode, you should find the "EXPLAIN PLAN_ADVICE" should
> throw UnsupportedOperationException as expected.
>
> Part II: Verify the advice content
> Write a group aggregate query, and enable/disable the local-global two-phase
> configuration, and test the output.
> You should find once the following configurations are enabled, you will get
> the "no available advice..." output.
> {code:java}
> table.exec.mini-batch.enabled
> table.exec.mini-batch.allow-latency
> table.exec.mini-batch.size
> table.optimizer.agg-phase-strategy {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
