Hi Santosh Sorry , I could not respond earlier.
Lens supports OLAP but does to restrict the user to just OLAP. So you can fire queries even without creating a cube. In this case server will check each driver(incase there are many, say one for MySql, One for Hive) on ability to execute the issued query and then pick the correct driver. But in a production system having a cube will be a better option as it not only provides users with easier querying but also provides lots of features that are supported via cube (like calculated expressions, start time for columns, display names, data availability, correct fact resolution,etc) You can download the binary here <http://lens.apache.org/releases/download.html> (lens version 2.5) You can follow the steps for installation and running examples here : http://lens.apache.org/lenshome/install-and-run.html There is a demo video here <https://cwiki.apache.org/confluence/display/LENS/2015/07/13/20+Minute+video+demo+of+Apache+Lens+through+examples> Note: Lens has Hadoop and Hive as dependencies. If you have an existing Hadoop deployment (2.6 and above), you can use it directly. Just set HADOOP_HOME . For Hive we depend on a custom version <https://github.com/InMobi/hive/releases/tag/hive-release-0.13.4-inm> of Hive which has some extra optimizations compared to apache Hive. Lens version 2.6 will be out around August mid, where we have committed all hive optimizations to apache hive 2.x.x and Lens 2.6 will directly depend on Apache Hive 2.x.x in that case. Thanks, Puneet Gupta On Mon, Jul 11, 2016 at 11:52 AM, Santoshakhilesh < santosh.akhil...@huawei.com> wrote: > HI Puneet , > > Thanks for answering the queries. > > Lens being OLAP , does it mean I actually don’t need to build cubes , the > data can reside on sources you mentioned and Lens can be used as ROLAP > engine for query ? > > Can you point out to a link where I can download and test sample data > sources and use cases. > > Regards, > > Santosh Akhilesh > > > > > > *From:* Puneet Gupta [mailto:puneet.gu...@inmobi.com] > *Sent:* 08 July 2016 15:25 > *To:* Santoshakhilesh > *Cc:* user@lens.apache.org > *Subject:* Re: Few Questions about Lens > > > > Hi Santosh > > > > I would try to answer your questions . > > > > To begin with, lens has two large scale deployments (which crunch PBs > level data) , one at InMobi and another at Flipkart that I know of. Apart > from this there are some smaller deployments too. > > > > > > The main usp of lens is > > > > 1. Seamless access to data that sits in multiple data sources (hive, > spark, mysql, vertica, druid, elastic search, etc - more can be plugged in) > in a lamda architected analytics system. Lens can make the right choice of > data for any user query based on the data availability in underlying data > storages and their response time. > > > > 2. Provide an olap interface to query the data. This hides all the > complexity from end user who can write simple queries over cubes and > underneath lens will translate the queries by joining the right facts with > dimensions and resolving the fields to complex expressions. The olap > interface is very flexible and allows the data schema to change with time. > The schema allows for horizontal or vertical split/distribution of fact > tables (vertical split is in works and will be out soon) > > So i would say Lens is an ROLAP system but can have data stored in > multiple data sources compared to a traditional ROLAP that supports simple > storage. > > > > 3.Provide quota management for users and thus regulate the usage of lens > server to offer fair share to each user. > > > > 4. Provides multiple ways to communicate with lens sever via REST API, > Java client, python client and a lens-cli. A slack bot is also active, but > it not yet open sourced. > > > > > > For your use case involving Hbase, see if you can try and use the lens > JDBC driver with phoenix <https://phoenix.apache.org/> . Your data > ingestion and querying capacity will depend on HBase and phoenix in this > case. You can also chose to implement a lens HBase driver if you need > better control. > > > > > Thanks, > > Puneet Gupta > > > > On Fri, Jul 8, 2016 at 8:33 AM, Santoshakhilesh < > santosh.akhil...@huawei.com> wrote: > > Hi Lens Experts , > > > > I have come through this project while looking for an OLAP solution > on Hadoop Stack for my product. I have few queries , please someone answer > them. > > > > To give background of my requirement. > > > > 1. My system is collecting performance KPIs which we collect and > calculate and want to store in HBase. > > 2. Frequency and size of my data is every 5 minutes around 5 > million rows and each row about 30 – 40 KPIs (this is current load and can > increase in future) > > 3. I need to support following type of queries > > > > a) Querying raw KPI data based on filters and joins and order by > clauses – Expected latency < 2 seconds > > b) OLAP queries with groupby and aggregators like sum , max , avg > etc. – Expected latency < 2 seconds > > c) TOP N , Bottom N > > d) For large scans paging > > 4. Large scans based on filters for Machine Learning and analytics > Job > > > > Now my questions are ; (I have read the Lens documents but I still have > some confusions) > > > > 1. Lens is ROLAP solution ? – Which is the primary source of meta > data and real data > > 2. Lens is MOLAP solution ? – How do I build cubes ? whats the > latency of building cubes (any benchmarking available) > > 3. Which of my requirements can be met by Lens ? > > 4. Are there any commercial deployment with Lens ? > > > > Thanks in advance. > > > > Regards, > > Santosh Akhilesh > > > > > > > > > > > > _____________________________________________________________ > > The information contained in this communication is intended solely for the > use of the individual or entity to whom it is addressed and others > authorized to receive it. It may contain confidential or legally privileged > information. If you are not the intended recipient you are hereby notified > that any disclosure, copying, distribution or taking any action in reliance > on the contents of this information is strictly prohibited and may be > unlawful. If you have received this communication in error, please notify > us immediately by responding to this email and then delete it from your > system. The firm is neither liable for the proper and complete transmission > of the information contained in this communication nor for any delay in its > receipt. > -- _____________________________________________________________ The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.
