Deploying Apache Storm on Google Cloud

2015-09-04 Thread Ankur Garg
Hi ,

I wish to deploy storm cluster on Google Cloud .

Browsing on the internet , I could find this  =>
http://datadventures.ghost.io/2013/12/29/deploying-storm-on-gce/


Now , the above was written in 2013 . I read somewhere that Storm no longer
needs Zmq instead it is using netty .

Can someone here confirm , if the above is a good source (I will anyways
try it) .

In case any of u have some other document where this is explained , pls
share it .

PS-> I am new to apache Storm and  Google Cloud .

Any input is appreciated .

Thanks
Ankur


Re: Apache drill with apache spark

2015-07-06 Thread Ankur Garg
I would be interested to contribute too in this area :)




On Tue, Jul 7, 2015 at 4:35 AM, Hanifi Gunes  wrote:

> +dev
>
> Sounds cool. I have some plans to check the existing code as early as time
> permits. The existing code is forked off from ~0.5 and relies on Spark
> 1.1.0. We should discuss possible contribution areas once I check the code
> in then.
>
> -Hanifi
>
> On Mon, Jul 6, 2015 at 3:49 PM, Hafiz Mujadid 
> wrote:
>
> > yes If I could help you guys in this regard then I will contribute to
> this
> > project. It will be pleasant to work on this project.
> >
> >
> >
> > On Tue, Jul 7, 2015 at 2:46 AM, Parth Chandra  wrote:
> >
> > > Hi Hafiz,
> > >
> > >   We did do some work on integrating with Spark a while ago but never
> > > published the work.
> > >
> > >   We are looking at a way to make the work available, but I don't have
> a
> > > timeline for you.
> > >
> > >   Is this something you might be interested in contributing to?
> > >
> > > Parth
> > >
> > > On Fri, Jul 3, 2015 at 2:13 AM, Hafiz Mujadid <
> hafizmujadi...@gmail.com>
> > > wrote:
> > >
> > > > Hi All!
> > > >
> > > > I am new to apache drill and I want to know whether apache drill has
> > some
> > > > integration with apache spark or not. I read that work on integration
> > > with
> > > > spark is in progress, What is the expected time to make it available
> > for
> > > > users?
> > > >
> > > >
> > > > Thanks
> > > >
> > >
> >
> >
> >
> > --
> > Regards: HAFIZ MUJADID
> >
>


Re: Questions for Setting up Dev Environment

2015-01-15 Thread Ankur Garg
Thanks Jason and Yash . Appreciate you taking time to help me understand
Drill .

On a side note , I guess we should update the page though so that its no
longer confusing to new user .

I can do that if that sounds ok to folks :)

Regards
Ankur

On Thu, Jan 15, 2015 at 10:27 AM, Yash Sharma  wrote:

> +1 for Jason. He has mentioned everything you need for playing with and
> debugging Drill.
>
> Just to add - Join the Drill hangout to discuss areas where you are stuck
> or you would like to start. We are planning one this Tuesday 10AM PDT. Keep
> an eye on the dev list.
>
> On Thu, Jan 15, 2015 at 6:42 AM, Jason Altekruse  >
> wrote:
>
> > Hello Ankur!
> >
> > I'm sorry for your trouble getting started with Drill. I did not realize
> > how out of date that wiki page was. The reference interpreter was a proof
> > of concept execution engine that was developed early on in Drill to
> > validate some design decisions before beginning the actual project. While
> > we did maintain it for a little while alongside the full implementation
> of
> > the execution engine, it has be deprecated for some time now. If you want
> > to try to run some test queries on Drill I would recommend starting
> > sqlline, which is a bundled sql shell that leverages Drills JDBC driver.
> > The guide below show how to do it with a pre-built binary.
> >
> >
> >
> https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+in+10+Minutes
> >
> > You can also run sqlline after a build of Drill. To build, run "mvn clean
> > install -DskipTests" from the root of the project. The finished build
> will
> > be in: distribution/target/
> >
> > You are correct that java-exec is where the development has been
> happening
> > by and large. There are few pluggable components defined in the other
> > modules, as well as a few system interface and logical constructs (such
> as
> > our expression parser/AST) in the common submodule.
> >
> > For debugging during development we mostly write unit tests that submit
> sql
> > queries to test different components of the engine. The testing
> > infrastructure is available from any class that inherits from
> > BestTestQuery. Here is one example:
> >
> >
> >
> https://github.com/apache/drill/blob/35a350fd381d839b3a51587092779fd822a19734/exec/java-exec/src/test/java/org/apache/drill/TestExampleQueries.java
> >
> > We recently added a unified system for creating tests that run queries
> and
> > verify the full result set against a baseline. This has previously be
> > re-created in a number of the test classes in various forms, so we now
> have
> > one unified framework to give a single interface to these tests. This
> class
> > contains some example of the usage of the new framework.
> >
> >
> >
> https://github.com/apache/drill/blob/df56954771950cd850ecae10404007d11d9241cb/exec/java-exec/src/test/java/org/apache/drill/TestFrameworkTest.java
> >
> > -Jason
> >
> >
> >
> > On Wed, Jan 14, 2015 at 10:53 AM, Ankur Garg 
> wrote:
> >
> > > Hi All ,
> > >
> > > I joined the Apache Drill community recently and want to contribute to
> > > Drill as a developer .
> > >
> > > As a first step, I am setting up  my dev environment . I see that
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/DRILL/Setting+Up+Your+Development+Environment
> > >
> > > says that the page is Outdated and needs to be updated .
> > >
> > > Specifically , I couldn't get this( *I cant locate ReferenceInterpreter
> > > project in the workspace . Although I could see a donuts.json file
> > > under exec/java-exec . Is the ReferenceInterpreter renamed to
> > > exec/java-exec ?? *)  :
> > >
> > > Additionally , is there somewhere i can find the latest update info for
> > > setting up my dev environment . I will then update this page with those
> > so
> > > that new contributors do not face any hassle :)
> > > *Executing the ReferenceInterpreter from ref project:*
> > >
> > > *Copy "ref\src\ test\resources\donuts.json" to
> > > "ref\src\ main\resources\donuts.json"*
> > >
> > > *First time:*
> > >
> > > *Right click on "ReferenceInterpreter", select 'Run As -> Run
> > > Configurations'.*
> > >
> > > *Select Java Application and New.*
> > >
> > > *In Main tab:*
> > >
> > > *Project: ref*
> > >
> > > *Main class: org.apache.drill.exec.ref.ReferenceInterpreter*
> > >
> > > *In Arguments tab:*
> > >
> > > *Program arguments: "src/test/resources/simple_plan.json" (with
> quotes).*
> > >
> > > *Click 'Run'.*
> > >
> > > *Next time onwards:*
> > >
> > > *Right click on "ReferenceInterpreter", select 'Run As -> Java
> > > Application'.*
> > >
> > >
> > > Looking for help .
> > >
> > >
> > > Regards
> > >
> > > Ankur
> > >
> >
>


Questions for Setting up Dev Environment

2015-01-14 Thread Ankur Garg
Hi All ,

I joined the Apache Drill community recently and want to contribute to
Drill as a developer .

As a first step, I am setting up  my dev environment . I see that
https://cwiki.apache.org/confluence/display/DRILL/Setting+Up+Your+Development+Environment

says that the page is Outdated and needs to be updated .

Specifically , I couldn't get this( *I cant locate ReferenceInterpreter
project in the workspace . Although I could see a donuts.json file
under exec/java-exec . Is the ReferenceInterpreter renamed to
exec/java-exec ?? *)  :

Additionally , is there somewhere i can find the latest update info for
setting up my dev environment . I will then update this page with those  so
that new contributors do not face any hassle :)
*Executing the ReferenceInterpreter from ref project:*

*Copy "ref\src\ test\resources\donuts.json" to
"ref\src\ main\resources\donuts.json"*

*First time:*

*Right click on "ReferenceInterpreter", select 'Run As -> Run
Configurations'.*

*Select Java Application and New.*

*In Main tab:*

*Project: ref*

*Main class: org.apache.drill.exec.ref.ReferenceInterpreter*

*In Arguments tab:*

*Program arguments: "src/test/resources/simple_plan.json" (with quotes).*

*Click 'Run'.*

*Next time onwards:*

*Right click on "ReferenceInterpreter", select 'Run As -> Java
Application'.*


Looking for help .


Regards

Ankur