Benifit of Drill over Spark SQL

Hi, I am wondering the benefit of Drill over Spark SQL since both of them share a lot in sql optimization 1. DAG and Memory Computation and pipeline within stage 2. Support multiple data sources, like JSON, Parquet, CSV,TSV, HBase, Hive etc 3. code gen 4. Columnar storage .. One major difference

Understanding the concept of Drill fragment

Hi, Drillers, I am pretty new to Drill and I am trying to understand the work flow of drill query execution. When I am reading on the fragment section in http://drill.apache.org/docs/drill-query-execution/, I have some questions: 1. It looks to me that major fragment is like a spark stage in

Re: Code too large

Looking at the trace it looks like Drill is correctly using the JDK compiler which is good. However, it looks like we're not breaking apart the expressions into submethods. @Aditya, since you worked on DRILL-1155, do you have any sense to why we're not subdividing the code in this situation? If I

Re: How to modify connection timeout delay ?

This can be set in drill-override.conf, but this should not be an issue even for long running queries as we should be sending a heartbeat signal back throughout a query's execution, even if the query has not yet produced any real data. Can you share the query you are running and any errors you can

Re: Code too large

Logs and queries attached. On Thu, Mar 24, 2016 at 2:22 PM, Edmon Begoli wrote: > We are working on it. > > On Thu, Mar 24, 2016 at 2:07 PM, Ted Dunning > wrote: > >> Providing the stack traces and logs is a key first step in fixing this. >> >> >> >>

Re: Code too large

We are working on it. On Thu, Mar 24, 2016 at 2:07 PM, Ted Dunning wrote: > Providing the stack traces and logs is a key first step in fixing this. > > > > On Thu, Mar 24, 2016 at 11:02 AM, Nirav Shah > wrote: > > > I had the same issue but till

Re: Code too large

Providing the stack traces and logs is a key first step in fixing this. On Thu, Mar 24, 2016 at 11:02 AM, Nirav Shah wrote: > I had the same issue but till date even in version 1.6 it's there. > I checked about JDK also. > > it would be great if we can solve this

exception handling

Hi, We are processing user behavior data. some time same column consists different data type. some time JSON is mal-formed. Is there any way to manage this type of issues? Regards, Nirav

Re: Code too large

I had the same issue but till date even in version 1.6 it's there. I checked about JDK also. it would be great if we can solve this issue. Regards, Nirav On Thu, Mar 24, 2016 at 11:06 PM, Edmon Begoli wrote: > Here is the query: > > CREATE TABLE > cms.joe.`ALL_IP_HS_FINAL`

Re: Code too large

Here is the query: CREATE TABLE cms.joe.`ALL_IP_HS_FINAL` AS SELECT CASE WHEN columns[0] =3D '' THEN NULL ELSE CAST(columns[0] as DOUBLE= ) END as `DSYSRTKY`, CASE WHEN columns[1] =3D '' THEN NULL ELSE CAST(columns[1] as DOUBLE) END as `CLAIMNO`, NULLIF(columns[2],'') as `PROVIDER`, CASE WHEN

RE: [E] Re: JDBC storage plugin fails

It is working now. Thanks for the suggestions. It was a matter of getting the user credentials working for my database. I only wish the error messages were more descriptive, and specific to the failure reason. Glad you opened that ticket, Chris. Thanks, Scott Wilburn -Original

How to modify connection timeout delay ?

Hi, I'm trying to use Drill with the sqlline command line on linux but I'm not able to maintain a connection to my drillbits alive long enough to get the results back. The connection is reset after 30 seconds even while a query is running! I did find the "timeout" variable and set it to -1 but

Re: Code too large

It would also good to get the full stack trace. Do you have jdk or only Joe on these machines? On Mar 24, 2016 5:27 AM, "Edmon Begoli" wrote: > Does anyone know what might be causing this exception: > > *Error: SYSTEM ERROR: CompileException: File >

Re: Reading Avro Arrays

FYI: flattening of embedded structures is not supported in Parquet either. Regards, -Stefan On Wed, Mar 23, 2016 at 8:51 PM, Johannes Schulte < johannes.schu...@gmail.com> wrote: > Hi Stefan, > > thanks for your response and the link to your udf repository, it's a good > reference. I tried

Re: Code too large

This exception states that the code generated for the project is too big for that Java compiler. Can you share the query that caused this failure ? On Thu, Mar 24, 2016 at 1:27 PM, Edmon Begoli wrote: > Does anyone know what might be causing this exception: > > *Error: SYSTEM

Code too large

Does anyone know what might be causing this exception: *Error: SYSTEM ERROR: CompileException: File 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen10.java]', Line 7275, Column 17: ProjectorGen10.java:7275: error: code too large* *public void doEval(int inIndex, int outIndex)*

Regarding excel adn ms access file problem

Hello. I am trying to query from excel file(.xsl file), but i am unable to query from excel. Is Apache drill Supports excel file (not csv file) and Ms Access Files. -- Thanks & Regards. Sanjiv Kumar.

Re: Professional services for Drill

We have folks who can do this. I forwarded your request. You should get a direct followup. On Wed, Mar 23, 2016 at 3:55 PM, Stefán Baxter wrote: > Hi, > > I there anyone here that provides professional service for Drill? > > We are trying to optimize our system in