Support for Alluxio

2016-09-14 Thread Edmon Begoli
Is it possible to use Alluxio as a filesystem backend in Drill's storage formats, and if so how? Thanks.

Probabilistic data structures in Drill

2016-04-30 Thread Edmon Begoli
Is Drill using any of the probabilistic data structures [1], and if so - which ones and how? Thank you, Edmon 1. Probabilistic Data Structures - https://en.m.wikipedia.org/wiki/Category:Probabilistic_data_structures

Re: Looking for a JS web editor for Drill SQL

2016-04-23 Thread Edmon Begoli
Interesting request. Maybe incorporate something like this: http://mistic100.github.io/jQuery-QueryBuilder/ or adopt and adapt Airpal: http://nerds.airbnb.com/airpal/ On Thu, Apr 21, 2016 at 10:15 PM, jingguo yao wrote: > The SQL editor in Drill web console is rather primitive. It lacks > fe

Re: Regarding excel adn ms access file problem

2016-04-12 Thread Edmon Begoli
No, but there is JIRA open for it. I started working on the design but my other obligations limited me. Basically, we could use POI for it. On Thursday, March 24, 2016, SanjiV SwaraJ wrote: > Hello. > I am trying to query from excel file(.xsl file), but i am unable to > query from excel

Re: IllegalStateException: Memory was leaked by query - Drill error on wide table, but OK on a narrow but longer table.

2016-03-26 Thread Edmon Begoli
the original issue (problem spilling to disk). > > I will open a JIRA to fix the error message. Waiting on you to confirm it's > indeed the same issue. > > Thanks > > On Mon, Mar 21, 2016 at 1:57 PM, Edmon Begoli > wrote: > > > We are converting some raw CMS data fr

Re: Code too large

2016-03-24 Thread Edmon Begoli
Logs and queries attached. On Thu, Mar 24, 2016 at 2:22 PM, Edmon Begoli wrote: > We are working on it. > > On Thu, Mar 24, 2016 at 2:07 PM, Ted Dunning > wrote: > >> Providing the stack traces and logs is a key first step in fixing this. >> >> >> >

Re: Code too large

2016-03-24 Thread Edmon Begoli
rsion 1.6 it's there. > > I checked about JDK also. > > > > it would be great if we can solve this issue. > > > > Regards, > > Nirav > > > > On Thu, Mar 24, 2016 at 11:06 PM, Edmon Begoli > wrote: > > > > > Here is the query: >

Re: Code too large

2016-03-24 Thread Edmon Begoli
d') END as `PRCDR_DT15`, NULLIF(columns[220],'') as `ICD_PRCDR_CD16`, NULLIF(columns[221],'') as `ICD_PRCDR_VRSN_CD16`, CASE WHEN columns[222] =3D '' THEN NULL ELSE TO_DATE(columns[222], 'MMdd') END as `PRCDR_DT16`, NULLIF(columns[223],'')

Code too large

2016-03-24 Thread Edmon Begoli
Does anyone know what might be causing this exception: *Error: SYSTEM ERROR: CompileException: File 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen10.java]', Line 7275, Column 17: ProjectorGen10.java:7275: error: code too large* *public void doEval(int inIndex, int outIndex)*

IllegalStateException: Memory was leaked by query - Drill error on wide table, but OK on a narrow but longer table.

2016-03-21 Thread Edmon Begoli
We are converting some raw CMS data from csv to parquet using Drill, and using partitioning as we go. Query 1 runs OK on a narrower file: size: 13G Mar 16 18:20 out_revenuej_lds_100_201412.csv (Month 12 file) 13G Mar 16 16:37 out_claimsj_lds_100_2014_q1.csv (Quarter 1 file) q1 has 198 columns; m

Ambari and Drill

2016-03-13 Thread Edmon Begoli
Has anyone tried to hook up Ambari monitoring and Drill? If so, what were the experiences?

MapR ODBC driver and RHEL 7

2016-03-12 Thread Edmon Begoli
Has anyone tried and had luck in running queries with MapR ODBC driver on RHEL 7. It is failing for us. RHEL 7 is not officially supported, but I am hoping that getting it to run would not be too difficult. Setup: drill 1.5.0 various connection strings such as: Driver=3D/opt/mapr/drillodbc/lib

The praises for Drill

2016-02-25 Thread Edmon Begoli
Hello fellow Driilers, I have been inactive on the development side of the project, as we got busy being heavy/power users of the Drill in the last few months. I just want to share some great experiences with the latest versions of Drill. Just tonight, as we were scrambling to meet the deadline,

Re: Bug in Drill 1.3 CSV - please confirm

2015-11-25 Thread Edmon Begoli
Uwe, I filed a bug for this already: https://issues.apache.org/jira/browse/DRILL-3726 and possibly a duplicate of: https://issues.apache.org/jira/browse/DRILL-3149 Thank you, Edmon On Wed, Nov 25, 2015 at 3:09 AM, Geercken, Uwe wrote: > Abdel, > > I sent you the file to the email address. But

Benefits of parquet partitioning for non-restrictive, aggregate queries?

2015-11-21 Thread Edmon Begoli
Hey guys, Are there any benefits of generic partitioning for non-restrictive count(*) queries with Drill and Parquet files partitioned on some base criteria (by state, month, etc.) Let's say I am running: select count(*) from dfs.tmp.`claims_parquet`; where I have plain and partitioned claims_p

Re: Drill and AWS Quicksight?

2015-10-12 Thread Edmon Begoli
I did not try it yet, but if JDBC is supported, as it appears, I think it should work. We'll see about the performance. Edmon On Mon, Oct 12, 2015 at 8:06 AM, Jeroen van Dijk wrote: > Hi all, > > Maybe too early, but did someone already try out Drill together with AWS > Quicksight (https://aws.

Re: CSV with windows carriage return causes issues

2015-09-30 Thread Edmon Begoli
I have this already filled in under: https://issues.apache.org/jira/browse/DRILL-3726 On Wed, Sep 30, 2015 at 11:47 AM, rahul challapalli < challapallira...@gmail.com> wrote: > Looks like a bug to me. Can you raise a jira for this if you haven't done > it already > On Sep 30, 2015 8:04 AM, wrote

Zookeeper (semi-)automated cluster setup

2015-09-26 Thread Edmon Begoli
Hey folks, I am exploring a setup for large cluster deployment for Drill. Is anyone aware of any ZooKeeper utility for simplified and (semi-)automated setup, and, ideally, configured for Drill? Thank you, Edmon

Re: Executing Drill queries automatically (and reactively)

2015-09-22 Thread Edmon Begoli
t from my iPhone > > > On Sep 22, 2015, at 2:18 PM, Edmon Begoli wrote: > > > > Yes, that is my current thinking and most likely course of action. > > > >> On Tue, Sep 22, 2015 at 3:13 PM, Paul Ilechko > wrote: > >> > >> Perhaps you could implem

Re: Executing Drill queries automatically (and reactively)

2015-09-22 Thread Edmon Begoli
C. > > On Tue, Sep 22, 2015 at 2:18 PM, Edmon Begoli wrote: > > > Has anyone tried, and is there a possible a facility to trigger Drill > > queries to recalculate some aggregates statistics if the underlying data > > has changed. > > > > My scenario is pre

Executing Drill queries automatically (and reactively)

2015-09-22 Thread Edmon Begoli
Has anyone tried, and is there a possible a facility to trigger Drill queries to recalculate some aggregates statistics if the underlying data has changed. My scenario is pretty simple - I use Drill to query directories of files to mine some statistics out. Files are arriving at some reasonable f

Headless Drill CLI

2015-09-02 Thread Edmon Begoli
I have a query I would to benchmark in a batch. Let's call it benchmark.sql Is there a way to execute it in some way from the shell where I would not need to go in the drill shell. Something like: $drill-cli benchmark.sql > results.txt So I can parse and post-process the execution times.

Re: Error : \apache-drill-1.1.0\conf was unexpected at this time.

2015-08-29 Thread Edmon Begoli
To make it easier, given that Oracle pushes Java 8 automatically on desktops, have a separate Java 7 install and have a startup script for Drill that sets JAVA_HOME to Java 7. On Saturday, August 29, 2015, Kristine Hahn wrote: > Paolo--Screenshots do not show up in user list email. Drill doesn't

Default workspace and plug in

2015-08-28 Thread Edmon Begoli
Is it possible to set the default workspace and format, so when I am running queries in drill I can just say: select columns[1], ..n from ``; or if there is a sub-dir: select columns[1], ..n from `\\`; without having to specify workspace?

Re: Character Encoding Issue

2015-08-26 Thread Edmon Begoli
Hi Yiyi, This is a known issue with Drill 1.1, and it has been recorded. I expect that it will be fixed in the next few releases. https://issues.apache.org/jira/browse/DRILL-3712 Thank you, Edmon On Wed, Aug 26, 2015 at 4:20 AM, Carrot Hu -CIC wrote: > Dear Team, > > I have encountered the fo

Re: UTF conversion issue with gz files

2015-08-25 Thread Edmon Begoli
Aug 24, 2015 at 9:05 AM, Edmon Begoli wrote: > > > We are unable to process files that OSX identifies as character sete > > UTF16LE. After unzipping and converting to UTF8, we ere able to process > > one fine. There are CONVERT_TO and CONVERT_FROM commands that appear to

UTF conversion issue with gz files

2015-08-24 Thread Edmon Begoli
We are unable to process files that OSX identifies as character sete UTF16LE. After unzipping and converting to UTF8, we ere able to process one fine. There are CONVERT_TO and CONVERT_FROM commands that appear to address the issue, but we were unable to make them work on a gzipped or unzipped ver

Re: Show Files Command

2015-08-23 Thread Edmon Begoli
> > > > .tbl command and then query the PSV results using the COLUMNS[n] > > > > syntax, which can return the accessTime from COLUMNS[7]. > > > > Kristine Hahn > > > > Sr. Technical Writer > > > > 415-497-8107 @krishahn skype:krishahn > > > > >

Re: Show Files Command

2015-08-23 Thread Edmon Begoli
Related to this - I saw someone on stackoverflow asking if modified_time returned from show files could be treated relationally. Can the show files result be queried or converted into query result from the sql command line, and if so how? On Sunday, August 23, 2015, Ted Dunning wrote: > The cle

Re: Will drill work with Java SE 8 versions?

2015-08-20 Thread Edmon Begoli
Right. Did that. On Thursday, August 20, 2015, Abdel Hakim Deneche wrote: > you'll also need to skip the tests when building: > > mvn clean install -DskipTests > > On Thu, Aug 20, 2015 at 3:58 PM, Edmon Begoli > wrote: > > > I am running it, but for a mvn cle

Re: Will drill work with Java SE 8 versions?

2015-08-20 Thread Edmon Begoli
I am running it, but for a mvn clean install you'll need to modify the pom.xml file to be: (allows >= 1.7) [1.7,) or use range [1.7,1.9) On Thu, Aug 20, 2015 at 6:16 PM, Jim Scott wrote: > From a user perspective I have been running Drill on Java 8 for a year now > and have never had

Re: (Drill 1.1) mvn clean install build fails on tests - looks like a hostname resolution error

2015-08-20 Thread Edmon Begoli
A. Can you > try to use the work around in that JIRA? Please let us know if it works. > > > On Aug 20, 2015, at 11:29 AM, Edmon Begoli wrote: > > > > When I try to build Drill from source, I get a failure on test and what > it > > appears to be a hostname

(Drill 1.1) mvn clean install build fails on tests - looks like a hostname resolution error

2015-08-20 Thread Edmon Begoli
When I try to build Drill from source, I get a failure on test and what it appears to be a hostname resolution error (Excerpt below). I am running this on my home Wifi. Is this somehow related to DRILL-1856? Any suggestions how to fix this? (let me know if you need a full, debug-enabled log) Err

Re: Error when querying .gz file that contains single text file (actually psv, but with txt extension) - Drill 1.1.0

2015-08-20 Thread Edmon Begoli
> >> . > >> > >> Kristine Hahn > >> Sr. Technical Writer > >> 415-497-8107 @krishahn skype:krishahn > >> > >> > >> On Thu, Aug 20, 2015 at 7:52 AM, Edmon Begoli > wrote: > >> > >>> I have large number

Re: Error when querying .gz file that contains single text file (actually psv, but with txt extension) - Drill 1.1.0

2015-08-20 Thread Edmon Begoli
> > http://drill.apache.org/docs/querying-plain-text-files/#example-of-querying-a-tsv-file > > . > > > > Kristine Hahn > > Sr. Technical Writer > > 415-497-8107 @krishahn skype:krishahn > > > > > > On Thu, Aug 20, 2015 at 7:52 AM, Edmon Begoli w

Error when querying .gz file that contains single text file (actually psv, but with txt extension) - Drill 1.1.0

2015-08-20 Thread Edmon Begoli
I have large number of .txt files that are individually tarballed (i.e. compressed from .txt to .txt.tar.gz). (I received them like this.) Each .txt file is actually a psv file, but, for some reason, it is saved as txt. I have created a custom 'txt' storage configuration and I can query uncompres