Hello Pranav, The usecase you have put out is very similar to data on DB tables, where the data is not partitioned, but its holding some time dimensions values. Another related ticket here https://issues.apache.org/jira/browse/LENS-198.
Here is how you would go ahead: Even if the data is not partitioned, you can register partitions for time values that it is holding, since the tables are external tables. for example: Say I have a directory /data/warehouse/sales - which is holding data for years 2012, 2013, 2014. Say now you registered the partitions (year=2012, location = "/data/warehouse/sales"), (year=2013, location = "/data/warehouse/sales"), (year=2014, location = "/data/warehouse/sales"). When querying, you can try setting the property "lens.cube.query.replace.timedim" to false. This will not replace the time dimension queried with the partition column registered. See tests at https://github.com/apache/incubator-lens/blob/master/lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java#L1398 and https://github.com/apache/incubator-lens/blob/master/lens-cube/src/test/java/org/apache/lens/cube/parse/TestTimeRangeWriterWithQuery.java#L129 for more understanding. Thanks Amareshwari On Tue, Apr 21, 2015 at 12:41 AM, Rajat Khandelwal <[email protected]> wrote: > Hi Pranav > > This issue doesn't cover your requirement. I'll update it in a short while. > Let's follow up on dev. > > That said, I think the best approach here would be to run a custom > partitioner on your data and partition it by at least year (or a finer > granularity if it suits you), and then try queries. I'm assuming there is > on time column on which you can partition. This approach would allow lens > to restrict the data processed for a single query. And since the time > column is already there, you can add extra conditions in the where clause > for that column. > > You can choose to not run any partitioner, register a dummy time partition, > pass that dummy value in the query's time_range_in clause and keep extra > where conditions in the query. > > Regarding this, maybe we can park a ticket for lens to add the extra where > clauses in case the finer granularity partitions don't exist. e.g. > partitions are only on YEARLY level and time range queried is June to > August, in that case lens should choose the partition of the particular > year and add additional where conditions to comply with the restriction of > June to August. Right now it'll fail saying no update periods for given > range. > > Thoughts? > > > On Tue, Apr 21, 2015 at 10:11 AM Pranav Kumar Agarwal <[email protected]> > wrote: > > > Hi Rajat, > > > > I hope you are doing well. > > > > Can you please update the Jira LENS-451 with more details. > > > > I was wondering if there is a fact that has multiple years of data, > > however its not partitioned, then to be able to query it we might have to > > add artificial partitions. Is this issue to support no partitions and > still > > make it queryable? > > > > If you can please provide more details in the Jira for this bug and if it > > doesn't matches to what I'm looking for then I'll followup in dev mailing > > list. > > > > Regards, > > -Pranav. > > > > > > -------- Forwarded Message -------- Subject: [jira] [Updated] (LENS-451) > > Make choosing of timeline implementation configurable via xml Date: Mon, > > 20 Apr 2015 16:57:59 +0000 (UTC) From: Amareshwari Sriramadasu (JIRA) > > <[email protected]> <[email protected]> Reply-To: > > [email protected] To: [email protected] > > > > > > [ > https://issues.apache.org/jira/browse/LENS-451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel > ] > > > > Amareshwari Sriramadasu updated LENS-451: > > ----------------------------------------- > > Issue Type: Improvement (was: Sub-task) > > Parent: (was: LENS-280) > > > > > Make choosing of timeline implementation configurable via xml > > > ------------------------------------------------------------- > > > > > > Key: LENS-451 > > > URL: https://issues.apache.org/jira/browse/LENS-451 > > > Project: Apache Lens > > > Issue Type: Improvement > > > Components: cube > > > Reporter: Rajat Khandelwal > > > Assignee: Rajat Khandelwal > > > Fix For: 2.2 > > > > > > > > > > > > > > > > -- > > This message was sent by Atlassian JIRA > > (v6.3.4#6332) > > > > > > >
