Hi Shengkai, Thank you for introducing an interesting concept of plan advice. After reviewing the FLIP, I thought that plan advice cannot fully replace the original intention of the OperationValidator. Similar to the pluggable ResourceValidator[1] in flink k8s operator, the OperationValidator aims to block certain operations and throw exceptions, acting as a safeguard rather than an interactive advisor.
For example, it can target CREATE TABLE queries that set configurations which can lead to issues due to human errors, such as group.id and scan.startup.timestamp-millis in kafka tables. Additionally, it can block INSERT operations if the operator wants to ensure the sql gateway remains read-only. I might have misunderstood the extensibility of plan advice, so please feel free to correct me. Any feedback is appreciated. Thanks. Best, Dongwoo [1] https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/operations/plugins/#custom-flink-resource-validators 2024년 7월 19일 (금) 오후 3:50, Shengkai Fang <fskm...@gmail.com>님이 작성: > Hi Dongwoo, > > Thanks for your proposal! > > I think the idea about OperationValidator is similar to the plan advice[1], > which can give advice if the statement has problems. I wonder if it would > be more beneficial to develop the concept within the existing > infrastructure, rather than introducing a new concept on the gateway side. > Additionally, I find it challenging for OperationValidator to validate the > query behavior as a plan advisor, as it lacks the necessary tools for > obtaining the upsert key or statistics. > > Best, > Shengkai > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-280%3A+Introduce+EXPLAIN+PLAN_ADVICE+to+provide+SQL+advice > > Dongwoo Kim <dongwoo7....@gmail.com> 于2024年7月18日周四 21:14写道： > > > Hi Martijn, > > > > Thanks for the guidance on making a FLIP. > > > > Before proceeding, I’d like to gather some initial thoughts from the > > community to see if we agree on the usefulness of this feature. > > Could anyone share their opinions on the custom query operation validator > > for Flink SQL Gateway? > > Thanks in advance! > > > > Best, > > Dongwoo > > > > 2024년 7월 18일 (목) 오후 9:52, Martijn Visser <martijnvis...@apache.org>님이 > 작성: > > > > > Hi Dongwoo, > > > > > > I think this should first have been a FLIP [1], given that it proposes > to > > > make something pluggable with custom interfaces. > > > > > > Best regards, > > > > > > Martijn > > > > > > [1] > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals > > > > > > On Thu, Jul 18, 2024 at 2:50 PM Dongwoo Kim <dongwoo7....@gmail.com> > > > wrote: > > > > > > > Hi devs, > > > > > > > > A month ago, I created a jira issue and pr for a custom query > operation > > > > validator in Flink SQL Gateway to block unsafe queries. > > > > Since there has been no feedback yet, I would appreciate it if > someone > > > > could review it or provide some feedback. > > > > I'm curious if this feature can be added and if others have felt the > > same > > > > need. > > > > > > > > Thanks in advance! > > > > > > > > jira: https://issues.apache.org/jira/browse/FLINK-35560 > > > > pr: https://github.com/apache/flink/pull/25092 > > > > > > > > Best, > > > > Dongwoo > > > > > > > > > >
