> On May 11, 2015, 11:26 a.m., Rajat Khandelwal wrote: > > lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java, > > line 336 > > <https://reviews.apache.org/r/34033/diff/1/?file=955206#file955206line336> > > > > So if multiple storages have the same partition, they are unioned? > > Sushil Mohanty wrote: > Yes. Need to make the partitions across storage tables mutually exclusive > to avoid duplicate counting. > > Amareshwari Sriramadasu wrote: > We already pick minimal storage tables covering all partitions. That > should remove duplicate partitions. Can you check? > > Sushil Mohanty wrote: > yes.
Seems this issue still exists with current changes put up. For example : Consider following query with common partitions. Range : X partitions : X1 + C (common) partitions are in S1 and X2 + C partitions are in S2. Right now, where clause is written with common partitions included. Minimal storage tables covered would give : S1 covers X1+C and S2 covers X2. If we write the query with this. where clause will correctly include partitions in storage, without duplicates. Consider the following multi range query with multiple storages : Range1 ( X ): X1 +C -> S1; X2 + C -> S2; Range2 ( Y ): Y1 +C2 -> S1; Y2+C2 -> S2; Minimal storage cover will give : S1 -> X1+Y1+C+C1; S2 -> X2+Y2; Where clause for range should look at intersection of minimal cover and range parts. Then it will give range1's where clause : S1 -> X1+c; S2- > X2; range2's where clause : S1 -> Y1+c2; S2- > Y2; - Amareshwari ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34033/#review83213 ----------------------------------------------------------- On May 12, 2015, 8:33 a.m., Sushil Mohanty wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/34033/ > ----------------------------------------------------------- > > (Updated May 12, 2015, 8:33 a.m.) > > > Review request for lens and Amareshwari Sriramadasu. > > > Bugs: LENS-204 > https://issues.apache.org/jira/browse/LENS-204 > > > Repository: lens > > > Description > ------- > > Support for union across two or more tables in cube query rewriting. > > > Diffs > ----- > > lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java > 47b9fc1 > lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java > b7c3ee9 > lens-cube/src/main/java/org/apache/lens/cube/parse/DimOnlyHQLContext.java > 8a23ded > > lens-cube/src/main/java/org/apache/lens/cube/parse/SingleFactHQLContext.java > beaa551 > > lens-cube/src/main/java/org/apache/lens/cube/parse/SingleFactMultiStorageHQLContext.java > PRE-CREATION > > lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java > e5a6b32 > lens-cube/src/main/java/org/apache/lens/cube/parse/UnionHQLContext.java > PRE-CREATION > lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java > 3203354 > lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java > c915353 > > Diff: https://reviews.apache.org/r/34033/diff/ > > > Testing > ------- > > Updated unit tests wrt union support. mvn clean test is running fine. > > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Summary: > [INFO] > [INFO] Lens Checkstyle Rules ............................. SUCCESS [8.026s] > [INFO] Lens .............................................. SUCCESS [0.299s] > [INFO] Lens API .......................................... SUCCESS [17.493s] > [INFO] Lens API for server and extensions ................ SUCCESS [15.690s] > [INFO] Lens Cube ......................................... SUCCESS [4:13.909s] > [INFO] Lens DB storage ................................... SUCCESS [16.124s] > [INFO] Lens Query Library ................................ SUCCESS [10.275s] > [INFO] Lens Hive Driver .................................. SUCCESS [3:55.868s] > [INFO] Lens Driver for JDBC .............................. SUCCESS [27.174s] > [INFO] Lens Server ....................................... SUCCESS > [10:18.860s] > [INFO] Lens client ....................................... SUCCESS [35.250s] > [INFO] Lens CLI .......................................... SUCCESS [5:21.299s] > [INFO] Lens Examples ..................................... SUCCESS [4.412s] > [INFO] Lens Distribution ................................. SUCCESS [2.964s] > [INFO] Lens ML Lib ....................................... SUCCESS [2:22.502s] > [INFO] Lens ML Ext Distribution .......................... SUCCESS [6.153s] > [INFO] Lens Regression ................................... SUCCESS [0.909s] > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 28:38.764s > [INFO] Finished at: Mon May 11 13:59:03 GMT+05:30 2015 > [INFO] Final Memory: 85M/1078M > [INFO] > ------------------------------------------------------------------------ > > > Thanks, > > Sushil Mohanty > >
