Thanks Yuxia for the clarification and FLIP update. The FLIP looks good! Best regards, Jing
On Mon, Apr 10, 2023 at 3:51 AM yuxia <luoyu...@alumni.sjtu.edu.cn> wrote: > 1: > Actaully, considering the Flink's implementation, Flink just provides > Truncate Table syntax to help user simlify data management as said in this > FLIP and push the implementation of Truncate Table to external connector. > Normally, the effect of TRUENCATE TABLE is same as Drop Table + Create > Table. But the real difference/benefit depends on the implementation of the > external connector. > For example, for DROP Table statement, some external connectors may also > drop the view related or other things. > But for Truncate Table, the connectors may just delete all data without > other operations. > > > 2: > At very begining, I'm thinking about in which case user may want to > truncate a temporary table. > I thought users can always create a table in catalog(if the table doesn't > exist in a catalog) and truncate the table. So I tend not to expose it to > user. > But after I think it over again, I think it may be reasonable to support > truncate a temporary table for the case that user just want to delete all > datas from a table in an external storage without storing the metadata of > the table in a catalog so that the other user/session can't see the metada. > I think we can relax to the constraint to support truncate temporary > table. Now, I update it to the FLIP. > > > 3: > Thanks for your input, I agree that we can dicuss it in a different FLIP. > > > > Best regards, > Yuxia > > ----- 原始邮件 ----- > 发件人: "Jing Ge" <j...@ververica.com.INVALID> > 收件人: "dev" <dev@flink.apache.org> > 发送时间: 星期六, 2023年 4 月 08日 上午 3:05:11 > 主题: Re: [DISCUSS] FLIP-302: Support TRUNCATE TABLE statement > > Hi yuxia, > > Thanks for raising this topic. It is indeed a useful feature. +1 for > having it in Flink. I have some small questions and it would be great if > related information could be described in the FLIP. > > 1. Speaking of data warehouse use cases, what is the benefit of using > TRUNCATE table over DROP table + CREATE table IF NOT EXISTS with the > consideration of concrete Flink implementations? What would be the > suggestion for users to use TRUNCATE instead of DROP + CREATE... and > vise versa? > > 2. Since some engines support it, would you like to describe your > thought about why TRUNCATE table does not support temporary table? > > 3. The partition support is an important feature, afaic. It might > deserve a different FLIP and consider e.g.: TRUNCATE TABLE > tt_dw_usr_exp_xxx PARTITION(dt='20230303') and ALTER TABLE > tt_dw_usr_exp_xxx DROP IF EXISTS PARTITION(dt='20230303'). > > Looking forward to your thoughts. Thanks! > > Best regards, > > Jing > > On 4/7/23 05:04, Jingsong Li wrote: > > +1 for voting. > > > > Best, > > Jingsong > > > > On Thu, Apr 6, 2023 at 4:52 PM yuxia <luoyu...@alumni.sjtu.edu.cn> > wrote: > >> Hi everyone. > >> > >> If there are no other questions or concerns for the FLIP[1], I'd like > to start the vote next Monday (4.10). > >> > >> [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-302%3A+Support+TRUNCATE+TABLE+statement > >> > >> Best regards, > >> Yuxia > >> > >> ----- 原始邮件 ----- > >> 发件人: "yuxia" <luoyu...@alumni.sjtu.edu.cn> > >> 收件人: "dev" <dev@flink.apache.org> > >> 发送时间: 星期五, 2023年 3 月 24日 上午 11:27:42 > >> 主题: Re: [DISCUSS] FLIP-302: Support TRUNCATE TABLE statement > >> > >> Thanks all for your feedback. > >> > >> @Shammon FY > >> My gut feeling is that the end user shouldn't care about whether it'll > delete direcotry or move to Trash directory with the TRUNCATE TABLE > statement. They only need to know it will delete all rows from a table. > >> To me, I think delete directory or move to trash is more likely to be a > behavior of external storage level instead of SQL statement level. In Hive, > if user configure Trash, it will then move files to trash for DROP statment. > >> Also, hardly did I see such usage with TRUNCATE TABLE statement in > other engines. What's more, to support it, we have to extend the TRUNCATE > TABLE synax which won't then compliant with SQL standard. I really don't > want to do that and I believe it'll make user confused if we do so. > >> > >> @Hang > >> `TRUNCATE TABLE` is meant to delete all rows of a base table. So, it > makes no sense that table source implements it. > >> If user use TRUNCATE TABLE statement to truncate a table, the planner > will only try to > >> find the DynamicTableSink for the corresponding table. > >> > >> @Ran Tao > >> 1: Thanks for you reminder. I said it won't support view in the FLIP, > but forget to said temporary table is also not supported. Now, I add this > part to this FLIP. > >> > >> 2: Yes, I also considered to incldue it in this FLIP before. But as far > as I see, I haven't seen much usage of truncate table with partition. It's > not as useful as truncate table. So, I tend to keep this FLIP simple in > here without supporting truncate table with partition. > >> Also, seems for `truncate table with partition`, differnet engines may > have differernt syntax; > >> Hive[1]/Spark[2] use the following syntax: > >> TRUNCATE TABLE table_name [PARTITION partition_spec] > >> > >> SqlServer[3] use the follwoing syntax: > >> TRUNCATE TABLE { database_name.schema_name.table_name | > schema_name.table_name | table_name } [ WITH ( PARTITIONS ( { > <partition_number_expression> | <range> } > >> So, I'm tend to be cautious about it. > >> > >> But I'm open to this. If there's any feedback or strong requirement, I > don't mind to add it in this FLIP. > >> If we do need it in some day, I can propose it in a new FLIP. It won't > break the current design. > >> > >> As for concrete syntax in the FLIP, I think the current one is the > concrete syntax, we don't allow TABLE keyword to be optional. > >> > >> 3: Thanks for your reminder, I have updadted the FLIP for this. > >> > >> > >> [1] > https://cwiki.apache.org/confluence/display/hive/languagemanual+ddl#LanguageManualDDL-TruncateTable > >> [2] > https://spark.apache.org/docs/3.0.0-preview/sql-ref-syntax-ddl-truncate-table.html > >> [3] > https://learn.microsoft.com/en-us/sql/t-sql/statements/truncate-table-transact-sql?view=sql-server-ver16 > >> > >> > >> > >> Best regards, > >> Yuxia > >> > >> ----- 原始邮件 ----- > >> 发件人: "Ran Tao" <chucheng...@gmail.com> > >> 收件人: "dev" <dev@flink.apache.org> > >> 发送时间: 星期四, 2023年 3 月 23日 下午 6:28:17 > >> 主题: Re: [DISCUSS] FLIP-302: Support TRUNCATE TABLE statement > >> > >> Hi, yuxia. > >> > >> Thanks for starting the discussion. > >> I think it's a nice improvement to support TRUNCATE TABLE statement > because > >> many other mature engines supports it. > >> > >> I have some questions. > >> 1. because table has different types, whether we will support view or > >> temporary tables? > >> > >> 2. some other engines such as spark and hive support TRUNCATE TABLE with > >> partition. whether we will support？ > >> btw, i think you need give the TRUNCATE TABLE concrete syntax in the > FLIP > >> because some engines has different syntaxes. > >> for example, hive allow TRUNCATE TABLE be TRUNCATE [TABLE] which means > >> TABLE keyword can be optional. > >> > >> 3. The Proposed Changes try to use SqlToOperationConverter and run in > >> TableEnvironmentImpl#executeInternal. > >> I think it's out of date, the community is refactoring the conversion > logic > >> from SqlNode to operation[1] and executions in TableEnvironmentImpl[2]. > >> I suggest you can use new way to support it. > >> > >> [1] https://issues.apache.org/jira/browse/FLINK-31464 > >> [2] https://issues.apache.org/jira/browse/FLINK-31368 > >> > >> Best Regards, > >> Ran Tao > >> https://github.com/chucheng92 > >> > >> > >> yuxia <luoyu...@alumni.sjtu.edu.cn> 于2023年3月22日周三 21:13写道： > >> > >>> Hi, devs. > >>> > >>> I'd like to start a discussion about FLIP-302: Support TRUNCATE TABLE > >>> statement [1]. > >>> > >>> The TRUNCATE TABLE statement is a SQL command that allows users to > quickly > >>> and efficiently delete all rows from a table without dropping the table > >>> itself. This statement is commonly used in data warehouse, where large > data > >>> sets are frequently loaded and unloaded from tables. > >>> So, this FLIP is meant to support TRUNCATE TABLE statement. M ore > exactly, > >>> this FLIP will bring Flink the TRUNCATE TABLE syntax and an interface > with > >>> which the coresponding connectors can implement their own logic for > >>> truncating table. > >>> > >>> Looking forwards to your feedback. > >>> > >>> [1]: [ > >>> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-302%3A+Support+TRUNCATE+TABLE+statement > >>> | > >>> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-302%3A+Support+TRUNCATE+TABLE+statement > >>> ] > >>> > >>> > >>> Best regards, > >>> Yuxia > >>> >
