Re: [DISCUSSION] Update the function of show segments

[David CaiQiang]

I agree with Jacky. 

I think enhanced segment metadata will help us to understand the table.  

I suggest the following properties for segment metadata:
1. total data file size
2. total index file size
3. data file count
4. index file count
5. last modified time (last update time)

Through these information,  we can answer the following questions.
1. Is there small file issue? Whether table require compaction or not, which
type should be used?
2. Whether index files is too many or not?  we will can estimate the total
size of index in memory whether it is big or small for driver memory
configuration.
3. Whether some segment has too many files?  Maybe it is useful to locate
some performance issue.



-
Best Regards
David Cai
--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: [DISCUSSION] Update the function of show segments

[Jacky Li]

I think it is good to have this feature, it may help user to decide whether 
manual compaction is needed.

Instead of outputting number of carbon index file per segment, I suggest output 
data size of the segment is more helpful.

Regards,
Jacky

> 在 2017年9月20日，上午11:46，Raghunandan S  写道：
> 
> What is the use case? When user would be interested in knowing number of
> files?
> On Sat, 16 Sep 2017 at 3:12 PM, xuchuanyin  wrote:
> 
>> Nice.
>> 
>> what about the update status of the segment? Maybe someone are interested
>> in the last modify time of a segment.
>> 
>> 
>> 
>> 
>> 
>> On 09/16/2017 17:14, Erlu Chen wrote:
>> Hi, dev
>> 
>> Currently, I am thinking about the function of show segments. We can see
>> segments of carbon table by executing this command, but it can only return
>> segmentId, status, load start time and load end time, and all this
>> information is from tablestatus, which I think it may be not enough for
>> users to know better about the situation of each segment, so now I want to
>> add two parameters, one is the number of carbon data file under segment
>> folder, another is the number of carbon index file under segment folder.
>> 
>> Any suggestion about my idea ?
>> 
>> Welcome to communicate.
>> 
>> Regards.
>> Chenerlu.
>> 
>> 
>> 
>> --
>> Sent from:
>> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>>