我看你的作业里面是window agg,目前 window agg 还不支持自动拆分。1.13 的基于 window tvf 的 window agg支持这个参数了。可以期待下。
Best, Jark On Wed, 24 Mar 2021 at 19:29, Robin Zhang <vincent2015q...@outlook.com> wrote: > Hi,guomuhua > 开启本地聚合,是不需要自己打散进行二次聚合的哈,建议看看官方的文档介绍。 > > Best, > Robin > > > guomuhua wrote > > 在SQL中,如果开启了 local-global 参数:set > > table.optimizer.agg-phase-strategy=TWO_PHASE; > > 或者开启了Partial-Final 参数:set > table.optimizer.distinct-agg.split.enabled=true; > > set > > table.optimizer.distinct-agg.split.bucket-num=1024; > > 还需要对应的将SQL改写为两段式吗? > > 例如: > > 原SQL: > > SELECT day, COUNT(DISTINCT buy_id) as cnt FROM T GROUP BY day, > > > > 对所需DISTINCT字段buy_id模1024自动打散后,SQL: > > SELECT day, SUM(cnt) total > > FROM ( > > SELECT day, MOD(buy_id, 1024), COUNT(DISTINCT buy_id) as cnt > > FROM T GROUP BY day, MOD(buy_id, 1024)) > > GROUP BY day > > > > 还是flink会帮我自动改写SQL,我不用关心? > > > > 另外,如果只设置开启上述参数,没有改写SQL,感觉没有优化,在flink web ui界面上也没有看到两阶段算子 > > < > http://apache-flink.147419.n8.nabble.com/file/t1346/%E7%AE%97%E5%AD%90.png> > > > > > > > > > > > > > -- > > Sent from: http://apache-flink.147419.n8.nabble.com/ > > > > > > -- > Sent from: http://apache-flink.147419.n8.nabble.com/ >