> On Oct. 21, 2016, 9:06 a.m., Amareshwari Sriramadasu wrote: > > lens-cube/src/test/java/org/apache/lens/cube/parse/DefaultCompletenessChecker.java, > > line 25 > > <https://reviews.apache.org/r/52514/diff/3/?file=1529514#file1529514line25> > > > > Instead of calling it DefaultCompletenessChecker,please call Mock or > > something else.
Will rename it as MockCompletenessChecker > On Oct. 21, 2016, 9:06 a.m., Amareshwari Sriramadasu wrote: > > lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java, > > line 939 > > <https://reviews.apache.org/r/52514/diff/3/?file=1529517#file1529517line939> > > > > Please add appropriate asserts for the test. Will add the asserts for the hql query and the expected query. > On Oct. 21, 2016, 9:06 a.m., Amareshwari Sriramadasu wrote: > > lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java, > > line 950 > > <https://reviews.apache.org/r/52514/diff/3/?file=1529517#file1529517line950> > > > > Please add appropriate asserts for the test. Will add the asserts for the hql query and the expected query. > On Oct. 21, 2016, 9:06 a.m., Amareshwari Sriramadasu wrote: > > lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java, > > line 969 > > <https://reviews.apache.org/r/52514/diff/3/?file=1529517#file1529517line969> > > > > Not clear whats the expected and actual value here. Here we are expecting the prune cause to be the Incomplete Data. Hence to assert this, we do a substring of INCOMPLETE_PARTITION.errorFormat > On Oct. 21, 2016, 9:06 a.m., Amareshwari Sriramadasu wrote: > > lens-cube/src/main/resources/olap-query-conf.xml, line 111 > > <https://reviews.apache.org/r/52514/diff/3/?file=1529512#file1529512line111> > > > > Should we make default to 100% ? Agreed. We can give default value as 100%. > On Oct. 21, 2016, 9:06 a.m., Amareshwari Sriramadasu wrote: > > lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java, > > line 526 > > <https://reviews.apache.org/r/52514/diff/3/?file=1529511#file1529511line526> > > > > This might skip expressions containing measures. We might have to > > iterate over queriedPhrases and look for measures in them. Will add the measures after iterating over the queriedPhrases > On Oct. 21, 2016, 9:06 a.m., Amareshwari Sriramadasu wrote: > > lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java, > > line 118 > > <https://reviews.apache.org/r/52514/diff/3/?file=1529511#file1529511line118> > > > > Should we make default to 100% ? Also move the default value also as a > > constant to CubeQueryConfUtil . Agreed. Will move this to CubeQueryConfUtil. > On Oct. 21, 2016, 9:06 a.m., Amareshwari Sriramadasu wrote: > > lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java, > > line 49 > > <https://reviews.apache.org/r/52514/diff/3/?file=1529508#file1529508line49> > > > > Minor : > > > > Can we avoid the string concat with " + " ? Agreed. - Narayan ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52514/#review153514 ----------------------------------------------------------- On Oct. 10, 2016, 7:45 a.m., Narayan Periwal wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52514/ > ----------------------------------------------------------- > > (Updated Oct. 10, 2016, 7:45 a.m.) > > > Review request for lens. > > > Bugs: LENS-1333 > https://issues.apache.org/jira/browse/LENS-1333 > > > Repository: lens > > > Description > ------- > > Though lens has partition registration being done whenever data is available, > there is no guarantee the partition registered is complete. There can be > different ways to know if the data is complete for partition. One option > could be to have a partition property saying whether it is complete or not. > Other could be to do a http call to another hosted service and more. > > Proposal here is to add an interface for DataCompletenessChecker and do the > check while resolving partitions. > > Here are some of the capabilities we would like to add in Lens : > # Lens will check partition existence first, if it exists, then check the > completeness percentage. If the completeness percentage is less than a > configured threshold (default should be 98, 99 or even 100), Lens will fail > the query. > # Lens's accept query on partial data will accept on incomplete data as well. > # Lens will also option to override the completeness percentage threshold > value at query level > # Lens will still have look ahead capability of daily being incomplete, then > it will union with hourly. > # If same measure is there in two different facts , Lens will we pick the one > with higher availability. > # In case of completeness percentage threshold missed, Lens will respond back > with available percentage. > > > Diffs > ----- > > lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java > fb958c3 > > lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java > 4585ef7 > lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java > 01265a5 > > lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateTablePruneCause.java > 78fb21d > lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java > 49ed5ef > lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultChecker.java > PRE-CREATION > > lens-cube/src/main/java/org/apache/lens/cube/parse/MaxCoveringFactResolver.java > 13f1aa4 > > lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java > db26718 > lens-cube/src/main/resources/olap-query-conf.xml 0c888ca > lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java > f7f8af2 > > lens-cube/src/test/java/org/apache/lens/cube/parse/DefaultCompletenessChecker.java > PRE-CREATION > > lens-cube/src/test/java/org/apache/lens/cube/parse/FieldsCannotBeQueriedTogetherTest.java > 7afa32e > lens-cube/src/test/java/org/apache/lens/cube/parse/TestBaseCubeQueries.java > 6fb027a > lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java > b90d4d3 > lens-server/src/main/resources/lenssession-default.xml b6bfa2e > src/site/apt/admin/session-config.apt c4b3c04 > src/site/apt/user/olap-query-conf.apt ce35ec4 > > Diff: https://reviews.apache.org/r/52514/diff/ > > > Testing > ------- > > > Thanks, > > Narayan Periwal > >
