Re: [DISCUSS] Make delete marker configurable?
Hi Raymond, I am trying to understand the use case . Can you please provide more context on what problem this addresses ? Thanks, Sudha On Fri, Jun 26, 2020 at 9:02 PM Shiyan Xu wrote: > Hi all, > > A small suggestion: as delta streamer relies on `_hoodie_is_deleted` to do > hard delete, can we make it configurable? as in users can specify any > boolean field for delete marker and `_hoodie_is_deleted` remains as > default. > > Regards, > Raymond >
Re: [DISCUSS] Make delete marker configurable?
+1, I just left it as a todo for future patch when I worked on it. On Sat, Jun 27, 2020 at 8:32 PM Bhavani Sudha wrote: > Hi Raymond, > > I am trying to understand the use case . Can you please provide more > context on what problem this addresses ? > > > Thanks, > Sudha > > On Fri, Jun 26, 2020 at 9:02 PM Shiyan Xu > wrote: > > > Hi all, > > > > A small suggestion: as delta streamer relies on `_hoodie_is_deleted` to > do > > hard delete, can we make it configurable? as in users can specify any > > boolean field for delete marker and `_hoodie_is_deleted` remains as > > default. > > > > Regards, > > Raymond > > > -- Regards, -Sivabalan
Re: [DISCUSS] Make delete marker configurable?
The suggestion looks good to me as well. On Sun, Jun 28, 2020 at 8:17 AM Sivabalan wrote: > +1, I just left it as a todo for future patch when I worked on it. > > On Sat, Jun 27, 2020 at 8:32 PM Bhavani Sudha > wrote: > > > Hi Raymond, > > > > I am trying to understand the use case . Can you please provide more > > context on what problem this addresses ? > > > > > > Thanks, > > Sudha > > > > On Fri, Jun 26, 2020 at 9:02 PM Shiyan Xu > > wrote: > > > > > Hi all, > > > > > > A small suggestion: as delta streamer relies on `_hoodie_is_deleted` to > > do > > > hard delete, can we make it configurable? as in users can specify any > > > boolean field for delete marker and `_hoodie_is_deleted` remains as > > > default. > > > > > > Regards, > > > Raymond > > > > > > -- > Regards, > -Sivabalan >
Re: [DISCUSS] Make delete marker configurable?
Thanks for the +1. Filed https://issues.apache.org/jira/browse/HUDI-1058 On Sat, Jun 27, 2020 at 11:34 PM Pratyaksh Sharma wrote: > The suggestion looks good to me as well. > > On Sun, Jun 28, 2020 at 8:17 AM Sivabalan wrote: > > > +1, I just left it as a todo for future patch when I worked on it. > > > > On Sat, Jun 27, 2020 at 8:32 PM Bhavani Sudha > > wrote: > > > > > Hi Raymond, > > > > > > I am trying to understand the use case . Can you please provide more > > > context on what problem this addresses ? > > > > > > > > > Thanks, > > > Sudha > > > > > > On Fri, Jun 26, 2020 at 9:02 PM Shiyan Xu > > > > wrote: > > > > > > > Hi all, > > > > > > > > A small suggestion: as delta streamer relies on `_hoodie_is_deleted` > to > > > do > > > > hard delete, can we make it configurable? as in users can specify any > > > > boolean field for delete marker and `_hoodie_is_deleted` remains as > > > > default. > > > > > > > > Regards, > > > > Raymond > > > > > > > > > -- > > Regards, > > -Sivabalan > > >
Re: [DISCUSS] Make delete marker configurable?
Hi Sudha, the delete marker being configurable can give more flexibility to users when process delete events; they can check any bool field they may have on their own schema. On Sat, Jun 27, 2020 at 5:32 PM Bhavani Sudha wrote: > Hi Raymond, > > I am trying to understand the use case . Can you please provide more > context on what problem this addresses ? > > > Thanks, > Sudha > > On Fri, Jun 26, 2020 at 9:02 PM Shiyan Xu > wrote: > > > Hi all, > > > > A small suggestion: as delta streamer relies on `_hoodie_is_deleted` to > do > > hard delete, can we make it configurable? as in users can specify any > > boolean field for delete marker and `_hoodie_is_deleted` remains as > > default. > > > > Regards, > > Raymond > > >
Re: [DISCUSS] Make delete marker configurable?
+1 as well. (sorry , for jumping in late) On Sun, Jun 28, 2020 at 11:36 AM Shiyan Xu wrote: > Thanks for the +1. Filed https://issues.apache.org/jira/browse/HUDI-1058 > > On Sat, Jun 27, 2020 at 11:34 PM Pratyaksh Sharma > wrote: > > > The suggestion looks good to me as well. > > > > On Sun, Jun 28, 2020 at 8:17 AM Sivabalan wrote: > > > > > +1, I just left it as a todo for future patch when I worked on it. > > > > > > On Sat, Jun 27, 2020 at 8:32 PM Bhavani Sudha > > > > wrote: > > > > > > > Hi Raymond, > > > > > > > > I am trying to understand the use case . Can you please provide more > > > > context on what problem this addresses ? > > > > > > > > > > > > Thanks, > > > > Sudha > > > > > > > > On Fri, Jun 26, 2020 at 9:02 PM Shiyan Xu < > xu.shiyan.raym...@gmail.com > > > > > > > wrote: > > > > > > > > > Hi all, > > > > > > > > > > A small suggestion: as delta streamer relies on > `_hoodie_is_deleted` > > to > > > > do > > > > > hard delete, can we make it configurable? as in users can specify > any > > > > > boolean field for delete marker and `_hoodie_is_deleted` remains as > > > > > default. > > > > > > > > > > Regards, > > > > > Raymond > > > > > > > > > > > > -- > > > Regards, > > > -Sivabalan > > > > > >
Re: [DISCUSS] Make delete marker configurable?
+1 Sent from Yahoo Mail for iPhone On Monday, June 29, 2020, 5:34 PM, Vinoth Chandar wrote: +1 as well. (sorry , for jumping in late) On Sun, Jun 28, 2020 at 11:36 AM Shiyan Xu wrote: > Thanks for the +1. Filed https://issues.apache.org/jira/browse/HUDI-1058 > > On Sat, Jun 27, 2020 at 11:34 PM Pratyaksh Sharma > wrote: > > > The suggestion looks good to me as well. > > > > On Sun, Jun 28, 2020 at 8:17 AM Sivabalan wrote: > > > > > +1, I just left it as a todo for future patch when I worked on it. > > > > > > On Sat, Jun 27, 2020 at 8:32 PM Bhavani Sudha > > > > wrote: > > > > > > > Hi Raymond, > > > > > > > > I am trying to understand the use case . Can you please provide more > > > > context on what problem this addresses ? > > > > > > > > > > > > Thanks, > > > > Sudha > > > > > > > > On Fri, Jun 26, 2020 at 9:02 PM Shiyan Xu < > xu.shiyan.raym...@gmail.com > > > > > > > wrote: > > > > > > > > > Hi all, > > > > > > > > > > A small suggestion: as delta streamer relies on > `_hoodie_is_deleted` > > to > > > > do > > > > > hard delete, can we make it configurable? as in users can specify > any > > > > > boolean field for delete marker and `_hoodie_is_deleted` remains as > > > > > default. > > > > > > > > > > Regards, > > > > > Raymond > > > > > > > > > > > > -- > > > Regards, > > > -Sivabalan > > > > > >
Re: [DISCUSS] Make delete marker configurable?
+1 as well. Thanks Raymond for explaining. On Sun, Jun 28, 2020 at 11:35 AM Shiyan Xu wrote: > Hi Sudha, the delete marker being configurable can give more flexibility to > users when process delete events; they can check any bool field they may > have on their own schema. > > On Sat, Jun 27, 2020 at 5:32 PM Bhavani Sudha > wrote: > > > Hi Raymond, > > > > I am trying to understand the use case . Can you please provide more > > context on what problem this addresses ? > > > > > > Thanks, > > Sudha > > > > On Fri, Jun 26, 2020 at 9:02 PM Shiyan Xu > > wrote: > > > > > Hi all, > > > > > > A small suggestion: as delta streamer relies on `_hoodie_is_deleted` to > > do > > > hard delete, can we make it configurable? as in users can specify any > > > boolean field for delete marker and `_hoodie_is_deleted` remains as > > > default. > > > > > > Regards, > > > Raymond > > > > > >
