I think maintaining a tablestatus backlog file is a good idea. This will also help us in quick filtering of valid segments as the number of segments increase during queries execution which involve reading of table status file.
Show segment DDL can read both the files to display the output. Regards Manish Gupta On Thu, 15 Mar 2018 at 10:19 AM, xm_zzc <441586...@qq.com> wrote: > Hi Jacky, Raghunandan S: > Thanks for your reply. > Currently I am working on PR2045, this pr will automatically delete the > segment lock files when execute method > 'SegmentStatusManager.deleteLoadsAndUpdateMetadata', and it will scan > 'tablestatus' file to decide which segment lock file need to be deleted. > Ravindra Pesala considers the performance of reading tablestatus file as > the size of it is getting larger. So I want to know whether it can reduce > the size of tablestatus file. > According to Raghunandan S's suggestion, I think we can *append* the > invisible segment list to the file called 'tablestatus.history' when > execute > command 'CLEAN FILES FOR TABLE' every time, separate visible and invisible > segments into two files. If later it needs to support listing all > segments(include visible and invisible) list when execute 'SHOW SEGMENTS > FOR > TABLE', it just need to read from two files. Is it OK to do so? > > > > -- > Sent from: > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/ >
