The FLIP looks good to me. +1 to start a vote. I just have a question: what will happen if the "executeTruncation" returns false without any exceptions?
Best, Jark On Thu, 13 Apr 2023 at 19:59, Jing Ge <j...@ververica.com.invalid> wrote: > Thanks Yuxia for the clarification and FLIP update. The FLIP looks good! > > Best regards, > Jing > > On Mon, Apr 10, 2023 at 3:51 AM yuxia <luoyu...@alumni.sjtu.edu.cn> wrote: > > > 1: > > Actaully, considering the Flink's implementation, Flink just provides > > Truncate Table syntax to help user simlify data management as said in > this > > FLIP and push the implementation of Truncate Table to external connector. > > Normally, the effect of TRUENCATE TABLE is same as Drop Table + Create > > Table. But the real difference/benefit depends on the implementation of > the > > external connector. > > For example, for DROP Table statement, some external connectors may also > > drop the view related or other things. > > But for Truncate Table, the connectors may just delete all data without > > other operations. > > > > > > 2: > > At very begining, I'm thinking about in which case user may want to > > truncate a temporary table. > > I thought users can always create a table in catalog(if the table doesn't > > exist in a catalog) and truncate the table. So I tend not to expose it to > > user. > > But after I think it over again, I think it may be reasonable to support > > truncate a temporary table for the case that user just want to delete all > > datas from a table in an external storage without storing the metadata of > > the table in a catalog so that the other user/session can't see the > metada. > > I think we can relax to the constraint to support truncate temporary > > table. Now, I update it to the FLIP. > > > > > > 3: > > Thanks for your input, I agree that we can dicuss it in a different FLIP. > > > > > > > > Best regards, > > Yuxia > > > > ----- 原始邮件 ----- > > 发件人: "Jing Ge" <j...@ververica.com.INVALID> > > 收件人: "dev" <dev@flink.apache.org> > > 发送时间: 星期六, 2023年 4 月 08日 上午 3:05:11 > > 主题: Re: [DISCUSS] FLIP-302: Support TRUNCATE TABLE statement > > > > Hi yuxia, > > > > Thanks for raising this topic. It is indeed a useful feature. +1 for > > having it in Flink. I have some small questions and it would be great if > > related information could be described in the FLIP. > > > > 1. Speaking of data warehouse use cases, what is the benefit of using > > TRUNCATE table over DROP table + CREATE table IF NOT EXISTS with the > > consideration of concrete Flink implementations? What would be the > > suggestion for users to use TRUNCATE instead of DROP + CREATE... and > > vise versa? > > > > 2. Since some engines support it, would you like to describe your > > thought about why TRUNCATE table does not support temporary table? > > > > 3. The partition support is an important feature, afaic. It might > > deserve a different FLIP and consider e.g.: TRUNCATE TABLE > > tt_dw_usr_exp_xxx PARTITION(dt='20230303') and ALTER TABLE > > tt_dw_usr_exp_xxx DROP IF EXISTS PARTITION(dt='20230303'). > > > > Looking forward to your thoughts. Thanks! > > > > Best regards, > > > > Jing > > > > On 4/7/23 05:04, Jingsong Li wrote: > > > +1 for voting. > > > > > > Best, > > > Jingsong > > > > > > On Thu, Apr 6, 2023 at 4:52 PM yuxia <luoyu...@alumni.sjtu.edu.cn> > > wrote: > > >> Hi everyone. > > >> > > >> If there are no other questions or concerns for the FLIP[1], I'd like > > to start the vote next Monday (4.10). > > >> > > >> [1] > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-302%3A+Support+TRUNCATE+TABLE+statement > > >> > > >> Best regards, > > >> Yuxia > > >> > > >> ----- 原始邮件 ----- > > >> 发件人: "yuxia" <luoyu...@alumni.sjtu.edu.cn> > > >> 收件人: "dev" <dev@flink.apache.org> > > >> 发送时间: 星期五, 2023年 3 月 24日 上午 11:27:42 > > >> 主题: Re: [DISCUSS] FLIP-302: Support TRUNCATE TABLE statement > > >> > > >> Thanks all for your feedback. > > >> > > >> @Shammon FY > > >> My gut feeling is that the end user shouldn't care about whether it'll > > delete direcotry or move to Trash directory with the TRUNCATE TABLE > > statement. They only need to know it will delete all rows from a table. > > >> To me, I think delete directory or move to trash is more likely to be > a > > behavior of external storage level instead of SQL statement level. In > Hive, > > if user configure Trash, it will then move files to trash for DROP > statment. > > >> Also, hardly did I see such usage with TRUNCATE TABLE statement in > > other engines. What's more, to support it, we have to extend the TRUNCATE > > TABLE synax which won't then compliant with SQL standard. I really don't > > want to do that and I believe it'll make user confused if we do so. > > >> > > >> @Hang > > >> `TRUNCATE TABLE` is meant to delete all rows of a base table. So, it > > makes no sense that table source implements it. > > >> If user use TRUNCATE TABLE statement to truncate a table, the planner > > will only try to > > >> find the DynamicTableSink for the corresponding table. > > >> > > >> @Ran Tao > > >> 1: Thanks for you reminder. I said it won't support view in the FLIP, > > but forget to said temporary table is also not supported. Now, I add this > > part to this FLIP. > > >> > > >> 2: Yes, I also considered to incldue it in this FLIP before. But as > far > > as I see, I haven't seen much usage of truncate table with partition. > It's > > not as useful as truncate table. So, I tend to keep this FLIP simple in > > here without supporting truncate table with partition. > > >> Also, seems for `truncate table with partition`, differnet engines may > > have differernt syntax; > > >> Hive[1]/Spark[2] use the following syntax: > > >> TRUNCATE TABLE table_name [PARTITION partition_spec] > > >> > > >> SqlServer[3] use the follwoing syntax: > > >> TRUNCATE TABLE { database_name.schema_name.table_name | > > schema_name.table_name | table_name } [ WITH ( PARTITIONS ( { > > <partition_number_expression> | <range> } > > >> So, I'm tend to be cautious about it. > > >> > > >> But I'm open to this. If there's any feedback or strong requirement, I > > don't mind to add it in this FLIP. > > >> If we do need it in some day, I can propose it in a new FLIP. It won't > > break the current design. > > >> > > >> As for concrete syntax in the FLIP, I think the current one is the > > concrete syntax, we don't allow TABLE keyword to be optional. > > >> > > >> 3: Thanks for your reminder, I have updadted the FLIP for this. > > >> > > >> > > >> [1] > > > https://cwiki.apache.org/confluence/display/hive/languagemanual+ddl#LanguageManualDDL-TruncateTable > > >> [2] > > > https://spark.apache.org/docs/3.0.0-preview/sql-ref-syntax-ddl-truncate-table.html > > >> [3] > > > https://learn.microsoft.com/en-us/sql/t-sql/statements/truncate-table-transact-sql?view=sql-server-ver16 > > >> > > >> > > >> > > >> Best regards, > > >> Yuxia > > >> > > >> ----- 原始邮件 ----- > > >> 发件人: "Ran Tao" <chucheng...@gmail.com> > > >> 收件人: "dev" <dev@flink.apache.org> > > >> 发送时间: 星期四, 2023年 3 月 23日 下午 6:28:17 > > >> 主题: Re: [DISCUSS] FLIP-302: Support TRUNCATE TABLE statement > > >> > > >> Hi, yuxia. > > >> > > >> Thanks for starting the discussion. > > >> I think it's a nice improvement to support TRUNCATE TABLE statement > > because > > >> many other mature engines supports it. > > >> > > >> I have some questions. > > >> 1. because table has different types, whether we will support view or > > >> temporary tables? > > >> > > >> 2. some other engines such as spark and hive support TRUNCATE TABLE > with > > >> partition. whether we will support？ > > >> btw, i think you need give the TRUNCATE TABLE concrete syntax in the > > FLIP > > >> because some engines has different syntaxes. > > >> for example, hive allow TRUNCATE TABLE be TRUNCATE [TABLE] which means > > >> TABLE keyword can be optional. > > >> > > >> 3. The Proposed Changes try to use SqlToOperationConverter and run in > > >> TableEnvironmentImpl#executeInternal. > > >> I think it's out of date, the community is refactoring the conversion > > logic > > >> from SqlNode to operation[1] and executions in > TableEnvironmentImpl[2]. > > >> I suggest you can use new way to support it. > > >> > > >> [1] https://issues.apache.org/jira/browse/FLINK-31464 > > >> [2] https://issues.apache.org/jira/browse/FLINK-31368 > > >> > > >> Best Regards, > > >> Ran Tao > > >> https://github.com/chucheng92 > > >> > > >> > > >> yuxia <luoyu...@alumni.sjtu.edu.cn> 于2023年3月22日周三 21:13写道： > > >> > > >>> Hi, devs. > > >>> > > >>> I'd like to start a discussion about FLIP-302: Support TRUNCATE TABLE > > >>> statement [1]. > > >>> > > >>> The TRUNCATE TABLE statement is a SQL command that allows users to > > quickly > > >>> and efficiently delete all rows from a table without dropping the > table > > >>> itself. This statement is commonly used in data warehouse, where > large > > data > > >>> sets are frequently loaded and unloaded from tables. > > >>> So, this FLIP is meant to support TRUNCATE TABLE statement. M ore > > exactly, > > >>> this FLIP will bring Flink the TRUNCATE TABLE syntax and an interface > > with > > >>> which the coresponding connectors can implement their own logic for > > >>> truncating table. > > >>> > > >>> Looking forwards to your feedback. > > >>> > > >>> [1]: [ > > >>> > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-302%3A+Support+TRUNCATE+TABLE+statement > > >>> | > > >>> > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-302%3A+Support+TRUNCATE+TABLE+statement > > >>> ] > > >>> > > >>> > > >>> Best regards, > > >>> Yuxia > > >>> > > >
