These are great points Bala. I personally liked 1, 3 and 6. Just curious - Does 6 also involve query caching ? I am also looking forward to explore Drill driver for Lens and HBase/Cassandra storage.
Looking forward to having loads of fun. Best Regards, On Thu, Jul 9, 2015 at 11:35 AM, Balasugavaneswaran Nathan < [email protected]> wrote: > Apologies for the repost. Resending to user@ > > On Thu, Jul 9, 2015 at 11:26 AM, Balasugavaneswaran Nathan < > [email protected]> wrote: > > > Hi, > > > > Here's a list that Sharad and I discussed about that we can add to the > > list that Amareshwari has sent: > > > > 1) New Lens UI - Ankeet has created a new Lens UI that we are using in > > flipkart. We can spend some time to review and checkin the latest code > > changes. > > > > 2) Elastic search driver - Amruth has added support for Elastic search > > driver. The plan is to review and checkin the code. > > > > 3) Lens Server Scale out - Consider moving session persistence to a > > distributed store (e.g: couchbase). Finished queries can be moved to > mysql > > or a diff datastore. Right now they are persisted in hsql. There are some > > open questions around maintaining stickiness for JDBC queries. Needs more > > discussion and finalization on approach. Me and Pranav will come up with > an > > approach that we can discuss and close on. If time permits, we can > submit a > > hack. > > > > 4) Parameterise queries - Being able to save queries and parameterize > > queries. We will need to support saving a query with and without a > > parameter. This will need API support (POST, PUT , GET, DELETE for > > creating, modifying, deleting and retrieving saved queries respectively). > > We also need to persist the saved queries in a datastore. This may > involve > > create supporting schema for persisting saved queries. > > > > 5) Schedule queries -We can use quartz scheduler - > > http://quartz-scheduler.org/ for scheduling queries. Needs API support > > and supporting schema changes for saving schedules (similar to the above) > > > > 6) MOLAP support (similar to Apache Kylin) - Materialize data in a > > datastore by pre-computing data along different dimensions of interest > and > > store resultant values in the cube - Proposal and Discussion by Sharad > > > > 7) Query Analytics - Create aggregates automatically based on consumption > > patterns. Need a method to schedule a pipeline. Similar to Amareshwari's > > proposal on "Improve query statistics and analytics on query stats : Come > > up with hot datasets in your analytics" > > > > Let me know if this sounds good. > > > > Bala > > > > > > On Wed, Jul 8, 2015 at 8:55 PM, Yash Sharma <[email protected]> wrote: > > > >> Hi Team, > >> With not lot of days left for the Hackathon we should spread the word > now. > >> Please use the meetup groups (if you own any) and social media so we > have > >> a > >> good participation for the event. > >> We can also consider tagging few JIRA's with bugbash tag. Would be > easier > >> for beginners to filter by bugbash. > >> > >> Let there be tweets :) > >> Looking forward to meet you all !! > >> > >> Best Regards, > >> Yash > >> > >> On Mon, Jun 29, 2015 at 11:59 AM, amareshwarisr . < > [email protected]> > >> wrote: > >> > >> > Looks like 11th July is dominate. We can fix to that. Inmobi has > aggreed > >> > to host the event and sponsor lunch/snacks as well. Thanks to Inmobi. > >> > > >> > Will send a calendar invite shortly. please RSVP if you are joining. > >> > > >> > Thanks > >> > Amareshwari > >> > > >> > On Fri, Jun 26, 2015 at 2:03 PM, Deepak Kumar Barr (Tech_BLR) < > >> > [email protected]> wrote: > >> > > >> >> 4th July works for me. > >> >> > >> >> Regards, > >> >> Deepak Kumar Barr > >> >> > >> >> On Thu, Jun 25, 2015 at 10:07 AM, amareshwarisr . < > >> [email protected]> > >> >> wrote: > >> >> > >> >>> Suggesting two dates 4th July or 11th July - saturdays in July. > >> >>> > >> >>> Please reply with your availability. > >> >>> > >> >>> Since we are thinking we will do bugbash/hackathon, should we plan a > >> day > >> >>> long event - with open doors for all day? Will find about logistics > at > >> >>> Inmobi, update here and send a calendar invite. > >> >>> > >> >>> Thanks > >> >>> Amareshwari > >> >>> > >> >>> On Mon, Jun 22, 2015 at 5:30 PM, Yash Sharma <[email protected]> > >> wrote: > >> >>> > >> >>>> We can assume an approximate count by the meetup group rsvp's. > >> >>>> Generally 1/3rd of the registered attendees. > >> >>>> > >> >>>> Also these are very interesting topics mentioned by Amareshwari. I > >> >>>> would fly down to Bangalore if we can have it organized on a > weekend. > >> >>>> > >> >>>> Best Regards > >> >>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> On Mon, Jun 22, 2015 at 1:26 PM, Sharad Agarwal <[email protected] > > > >> >>>> wrote: > >> >>>> > >> >>>>> +1 > >> >>>>> > >> >>>>> do we have an estimate of how many folks would be interested in > >> bugbash > >> >>>>> event ? there will be couple of folks from flipkart. > >> >>>>> > >> >>>>> On Sun, Jun 21, 2015 at 10:57 PM, amareshwarisr . < > >> >>>>> [email protected]> > >> >>>>> wrote: > >> >>>>> > >> >>>>> > I'm interested in Hackathon personally - as there is huge set of > >> >>>>> > improvements and features that can be looked at. > >> >>>>> > > >> >>>>> > Some that come to my mind : > >> >>>>> > - Support for new storages > >> >>>>> > - Support for new drivers > >> >>>>> > - Improve query statistics and analytics on query stats > >> >>>>> > - Query Cost estimation : estimate execution time > >> >>>>> > - Make cube metastore independent of Hive Metastore - allow it > >> sit > >> >>>>> on its > >> >>>>> > own. > >> >>>>> > - Move to Apache Hive > >> >>>>> > > >> >>>>> > and can be more. > >> >>>>> > > >> >>>>> > On Mon, Jun 22, 2015 at 10:32 AM, Srinivasan Hariharan < > >> >>>>> > [email protected]> wrote: > >> >>>>> > > >> >>>>> >> can we have bugbash? > >> >>>>> >> > >> >>>>> >> > Date: Mon, 22 Jun 2015 10:30:52 +0530 > >> >>>>> >> > Subject: Re: Next lens meetup > >> >>>>> >> > From: [email protected] > >> >>>>> >> > To: [email protected]; > >> [email protected] > >> >>>>> >> > >> >>>>> >> > > >> >>>>> >> > That sounds great Yash! > >> >>>>> >> > > >> >>>>> >> > What do others say : In for Hackathon or bugbash ? > >> >>>>> >> > > >> >>>>> >> > On Sat, Jun 20, 2015 at 4:04 PM, Yash Sharma < > >> [email protected]> > >> >>>>> wrote: > >> >>>>> >> > > >> >>>>> >> > > Just to add on.. > >> >>>>> >> > > > >> >>>>> >> > > Meetups are very traditional.. > >> >>>>> >> > > > >> >>>>> >> > > We can have lot more involvement if we can club a Bugbash > or > >> >>>>> Hackathon > >> >>>>> >> > > along with meetup. > >> >>>>> >> > > > >> >>>>> >> > > A Bugbash would focusses on existing features targeted on > >> >>>>> critical bug > >> >>>>> >> > > fixes whereas Hackathon can focus on new features like new > >> >>>>> execution > >> >>>>> >> > > engines/plan optimization/web interface rewamp etc. We can > >> >>>>> chose the > >> >>>>> >> one > >> >>>>> >> > > which is more beneficial for us. > >> >>>>> >> > > > >> >>>>> >> > > This will get a good awareness of Lens in user and dev > >> >>>>> community and > >> >>>>> >> would > >> >>>>> >> > > also motivate a lot of beginners to break the ice and start > >> open > >> >>>>> >> source > >> >>>>> >> > > contributions. > >> >>>>> >> > > > >> >>>>> >> > > Just thinking aloud. Thoughts? > >> >>>>> >> > > > >> >>>>> >> > > - Via mobile. Excuse brevity. > >> >>>>> >> > > On 20 Jun 2015 12:30 pm, "Yash Sharma" <[email protected]> > >> >>>>> wrote: > >> >>>>> >> > > > >> >>>>> >> > > > +1. Sounds great. > >> >>>>> >> > > > > >> >>>>> >> > > > Once the date is decided I can promote the event on my > >> small > >> >>>>> meetup > >> >>>>> >> > > group. > >> >>>>> >> > > > > >> >>>>> >> > > > http://www.meetup.com/India-Open-Source-Group/ > >> >>>>> >> > > > > >> >>>>> >> > > > > >> >>>>> >> > > > On Sat, Jun 20, 2015 at 12:03 PM, Jean-Baptiste Onofré < > >> >>>>> >> [email protected]> > >> >>>>> >> > > > wrote: > >> >>>>> >> > > > > >> >>>>> >> > > >> I will try, depending of my customer engagements. > >> >>>>> >> > > >> > >> >>>>> >> > > >> Regards > >> >>>>> >> > > >> JB > >> >>>>> >> > > >> > >> >>>>> >> > > >> > >> >>>>> >> > > >> On 06/20/2015 08:15 AM, Srikanth Sundarrajan wrote: > >> >>>>> >> > > >> > >> >>>>> >> > > >>> +1. Early Jul sounds good. > >> >>>>> >> > > >>> > >> >>>>> >> > > >>> Regards > >> >>>>> >> > > >>> Srikanth Sundarrajan > >> >>>>> >> > > >>> > >> >>>>> >> > > >>> Date: Sat, 20 Jun 2015 11:12:19 +0530 > >> >>>>> >> > > >>>> Subject: Next lens meetup > >> >>>>> >> > > >>>> From: [email protected] > >> >>>>> >> > > >>>> To: [email protected]; > >> >>>>> >> [email protected] > >> >>>>> >> > > >>>> > >> >>>>> >> > > >>>> Hello all, > >> >>>>> >> > > >>>> > >> >>>>> >> > > >>>> Its long time we had a meetup. Shall we have one end > of > >> >>>>> June or > >> >>>>> >> Early > >> >>>>> >> > > >>>> July? > >> >>>>> >> > > >>>> > >> >>>>> >> > > >>>> If you are looking forward to hear anything in the > >> meetup, > >> >>>>> please > >> >>>>> >> > > reply > >> >>>>> >> > > >>>> here. > >> >>>>> >> > > >>>> > >> >>>>> >> > > >>>> Thanks > >> >>>>> >> > > >>>> Amareshwari > >> >>>>> >> > > >>>> > >> >>>>> >> > > >>> > >> >>>>> >> > > >>> > >> >>>>> >> > > >>> > >> >>>>> >> > > >> -- > >> >>>>> >> > > >> Jean-Baptiste Onofré > >> >>>>> >> > > >> [email protected] > >> >>>>> >> > > >> http://blog.nanthrax.net > >> >>>>> >> > > >> Talend - http://www.talend.com > >> >>>>> >> > > >> > >> >>>>> >> > > > > >> >>>>> >> > > > > >> >>>>> >> > > > >> >>>>> >> > >> >>>>> > > >> >>>>> > > >> >>>>> > >> >>>> > >> >>>> > >> >>> > >> >> > >> >> > >> >> > >> > ------------------------------------------------------------------------------------------------------------------------------------------ > >> >> > >> >> This email and any files transmitted with it are confidential and > >> >> intended solely for the use of the individual or entity to whom they > >> are > >> >> addressed. If you have received this email in error please notify the > >> >> system manager. This message contains confidential information and is > >> >> intended only for the individual named. If you are not the named > >> addressee > >> >> you should not disseminate, distribute or copy this e-mail. Please > >> notify > >> >> the sender immediately by e-mail if you have received this e-mail by > >> >> mistake and delete this e-mail from your system. If you are not the > >> >> intended recipient you are notified that disclosing, copying, > >> distributing > >> >> or taking any action in reliance on the contents of this information > is > >> >> strictly prohibited. Although Flipkart has taken reasonable > >> precautions to > >> >> ensure no viruses are present in this email, the company cannot > accept > >> >> responsibility for any loss or damage arising from the use of this > >> email or > >> >> attachments > >> >> > >> > > >> > > >> > > > > > > -- > > > > ------------------------------------------------------------------------------------------------------------------------------------------ > > This email and any files transmitted with it are confidential and intended > solely for the use of the individual or entity to whom they are addressed. > If you have received this email in error please notify the system manager. > This message contains confidential information and is intended only for the > individual named. If you are not the named addressee you should not > disseminate, distribute or copy this e-mail. Please notify the sender > immediately by e-mail if you have received this e-mail by mistake and > delete this e-mail from your system. If you are not the intended recipient > you are notified that disclosing, copying, distributing or taking any > action in reliance on the contents of this information is strictly > prohibited. Although Flipkart has taken reasonable precautions to ensure no > viruses are present in this email, the company cannot accept responsibility > for any loss or damage arising from the use of this email or attachments >
