Forceful compaction might be required when user really wants control and wanted to force trigger compaction on required segments. How ever need to identify cases and its behavior , so that it can co exist with current compaction model, with out disturbing or imposing any limitations. I feel forceful major compaction specifying segments will be better option than forceful minor.
Regards, Venkata Ramana Gollamudi On Apr 19, 2017 14:50, "Kunal Kapoor" <kunal.kap...@knoldus.in> wrote: > Hi Liang, > Currently the user cannot choose which segments should be merged or not. > We should give a DDL which supports both forceful minor and major > compaction separately. > > Another point would be if the user can specify the segments which he needs > merged. Obviously the segments that can be given should meet some criteria. > for example:-- no major compacted or 2nd Level compacted segments can be > provided during forced minor compaction. > > > -- > Regards > > *Kunal Kapoor* > Software Consultant > > *Knoldus Software LLP <http://www.knoldus.com/home.knol>* > Twitter <http://twitter/kunal642> | LinkedIn > <https://in.linkedin.com/in/kunal642> | FB > <https://www.facebook.com/kunalkapoor642> > Mobile: +91-9711013430 > > On Wed, Apr 19, 2017 at 2:33 PM, Liang Chen <chenliang6...@gmail.com> > wrote: > > > Hi Kunal > > > > Thank you for taking the good topic for discussion. > > First , let us think about : why users want to do forceful minor > > compaction, > > which cases? > > Current "MAJOR compaction" whether can cover "forceful MINOR compaction" > > scenarios ? > > > > As we know, compaction is mainly for optimizing index effectiveness by > > merging for multiple times data loading(segments), so current system > > provides two options for users to choose :"soft compaction(minor) and > > strong > > compaction(major)"。 > > > > So i could not find the typical cases to add a new strong > > compaction(forceful minor)。 > > > > Regards > > Liang > > > > > > Kunal Kapoor wrote > > > Hi all, > > > I was looking into compaction and had a query regarding the same. > > > If you have auto compaction turned on and the threshold level is 4,3 > > > Now try loading data 7 times which will make 7 segments. Now as the > auto > > > compaction was on therefore 4 segments will be merged to 0.1. > > > The segments visible will be 0.1, 4, 5, 6. > > > > > > When i try to run the compaction command nothing happens because the > > > threshold level is not reached for compaction to happen. > > > What if i want to merge the 3 segments i.e (4, 5, 6) and make a level-1 > > > compacted segment. > > > > > > The proposed solution would be to add a new option to the compaction > > > command which contains what level of compaction the user would like to > do > > > > > > Example:- alter table carbon_table compact 'minor' level '1'. > > > This would forcefully combine the segments(4, 5, 6) in to a level 1 > > > compacted segment called 4.1 giving me 2 level-1 compacted segment (0.1 > > > and > > > 4.1) > > > Similar operation can be done with level-1 compacted segments. > > > > > > > > > > > > -- > > View this message in context: http://apache-carbondata- > > mailing-list-archive.1130556.n5.nabble.com/DISCUSSION- > > Forceful-minor-Compaction-tp10924p11319.html > > Sent from the Apache CarbonData Mailing List archive mailing list archive > > at Nabble.com. > > >
