Re: Support updating/deleting data for stream table

2018-06-04 Thread xm_zzc
Hi: ok, I will create a parent jira to trace this issue. -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: Support updating/deleting data for stream table

2018-06-04 Thread Raghunandan S
Hi, Those are 2 steps in the same solution. Not different solutions. We can create jira considering all and implement only the part. The parent jira would get closed when all the child jira are implemented Regards Raghu On Sun, 3 Jun 2018, 1:07 pm Liang Chen, wrote: > Hi > > +1 for first consid

Re: Support updating/deleting data for stream table

2018-06-03 Thread Liang Chen
Hi +1 for first considering solution1 Regards Liang xm_zzc wrote > Hi Raghu: > Yep, you are right, so I said solution 1 is not very precise when there > are still some data you want to update/delete being stored in stream > segments, solution 2 can handle this scenario you mentioned. > But,

Re: Support updating/deleting data for stream table

2018-06-02 Thread xm_zzc
Hi Raghu: Yep, you are right, so I said solution 1 is not very precise when there are still some data you want to update/delete being stored in stream segments, solution 2 can handle this scenario you mentioned. But, in my opinion, the scenario of deleting historical data is more common than t

Re: Support updating/deleting data for stream table

2018-05-30 Thread Raghunandan S
Hi, But this leads to inconsistent data returned to the user. For example user wanted to replace all 2g with 4g,but in stream segments it will still be 2g and would be returned in user query. I think we need to handle this scenario also Regards Raghu On Wed, 30 May 2018, 9:22 pm Liang Chen, wrot

Re: Support updating/deleting data for stream table

2018-05-30 Thread Liang Chen
Hi Thank you started this discussion thread. Agree with solution1, use the easy way to delete data for stream table. Regards Liang xm_zzc wrote > Hi dev: > Sometimes we need to delete some historical data from stream table to > make > the table size not too large, but currently the stream tabl

Re: Support updating/deleting data for stream table

2018-05-29 Thread ZhuWilliam
Since time lapses, the table will become bigger and bigger, we do need one way to clean data out of date. Make StreamTable support partition maybe a good choice. The first step we can make this simple, today's data will be mixed with normal segment and stream segment. when rolling, we will make non

Support updating/deleting data for stream table

2018-05-29 Thread xm_zzc
Hi dev: Sometimes we need to delete some historical data from stream table to make the table size not too large, but currently the stream table can't support updating/deleting data, so we need to stop the app and use 'alter table COMPACT 'close_streaming' command to close stream table, and then d