Re: Review Request 37910: DRILL-3622: When user authentication is enabled, enforce admin privileges to update SYSTEM options

2015-09-08 Thread Jason Altekruse
> On Sept. 8, 2015, 3:57 p.m., Jason Altekruse wrote: > > exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java, line > > 265 > > <https://reviews.apache.org/r/37910/diff/1/?file=1058412#file1058412line265> > > > > These sho

Re: Review Request 37910: DRILL-3622: When user authentication is enabled, enforce admin privileges to update SYSTEM options

2015-09-08 Thread Jason Altekruse
/exec/ExecConstants.java (line 265) <https://reviews.apache.org/r/37910/#comment153777> These should both be updated to be subclasses of AdminOptionValidator, so that we can veryify that the string passed is the correct format when it is set. - Jason Altekruse On Aug. 28, 2015,

Re: Identifying the source of problematic records

2015-09-03 Thread Jason Altekruse
> a combination of generated code related to expressions and code not related > to expressions. > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio > > On Thu, Sep 3, 2015 at 11:31 AM, Jason Altekruse > > wrote: > > > @Jacques, > > > > On your point

Re: Identifying the source of problematic records

2015-09-03 Thread Jason Altekruse
@Jacques, On your point a) about expressing failures and the compilation model, I had thought about previously using the interpreter to figure out which expression against the current row failed, once we have caught an exception out of some part of the complete code-generated expression evaluation

[jira] [Resolved] (DRILL-3669) fix missing direct dependency

2015-09-03 Thread Jason Altekruse (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse resolved DRILL-3669. Resolution: Fixed Fixed in 4b8e85ad6fb40554e6752144f09bdfb474d62d9b > fix missing dir

Re: Identifying the source of problematic records

2015-09-02 Thread Jason Altekruse
While I initially wanted to say I agree with Jinfeng, I think the user experience should be better than having to essentially run the query twice. It doesn't seem crazy idea for Drill to carry around a source table for a batch of records (maybe even an offset/line number in the scan if it is known)

Re: Source code for forked parquet library.

2015-08-31 Thread Jason Altekruse
https://github.com/mapr/incubator-parquet-mr/tree/1.6.0rc3-drill-r0.3 I am working with Julien Le Dem on getting us off of the fork, but for now the source code is accessible here. Let me know if you need any help looking through the parquet code. Is there a particular JIRA you are trying to addre

Re: question about aggregation, possible regression

2015-08-28 Thread Jason Altekruse
This is actually an error message that was recently fixed. It happens when using sum on varchar [1]. You can see in the output of your intermediate query with the json blobs in it that the numeric values are actually surrounded by quotes, so they will need to be cast to int or double to have sum ev

Re: zeroVectors() interface for value vectors

2015-08-28 Thread Jason Altekruse
Steven makes a good point, this is not something that can currently be more strictly enforced, because of the way we use the primitive vectors inside the composite ones like the variable length vectors. We should try to come up with all of the "constraints" that are violated (or not explicitly con

Re: No of files created by CTAS auto partition feature

2015-08-26 Thread Jason Altekruse
I feel like there is a little misunderstanding here. Rahul, did you try setting the option that Steven suggested? `store.partition.hash_distribute` This will cause a re-distribution of the data so that the rows that belong in a particular partition will all be written by a single writer. They wil

Re: [DISCUSS] Most storage plug-ins don't belong in the Drill repo

2015-08-25 Thread Jason Altekruse
the hive module should manage those and > how to rerun surefire tests with each active profile. For major changes, I > think you need to have separate modules. (For example HBase 94 versus 98+) > > > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio > > On Tue, Aug

Re: [DISCUSS] Most storage plug-ins don't belong in the Drill repo

2015-08-25 Thread Jason Altekruse
I think Jacques has a good point about the code belonging together, but we should then talk about how to solve this problem. Users are going to have different versions of these datasources that they have deployed, and I don't think we should consider it an afterthought as to how they get Drill to w

Re: Resetting an option

2015-08-10 Thread Jason Altekruse
I don't know if I missed something, but the Postgres docs seem to indicate that there is no equivalent to the concept of a SYSTEM option that exists in Drill, which can be set with a query. Options can be set at server startup, either in a configuration file or with a command line parameter [2]. On

Re: anyone seen these errors on master ?

2015-08-05 Thread Jason Altekruse
I also saw this failure running the tests on my linux vm, the only changed setting is the -PlargeTests flag, memory and fork count are defaults. All of the tests pass on my mac with default settings, no flags. On Wed, Aug 5, 2015 at 1:03 PM, Hanifi Gunes wrote: > Did you tighten your memory sett

[jira] [Resolved] (DRILL-3484) Error using functions with no parameters when `drill.exec.functions.cast_empty_string_to_null` is set to true

2015-08-04 Thread Jason Altekruse (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse resolved DRILL-3484. Resolution: Fixed ae2a45c8f803affcbbb021105c1a30cba4bd4213 > Error using functions with

[jira] [Resolved] (DRILL-3341) Move OperatorWrapper list and FragmentWrapper list creation to ProfileWrapper ctor

2015-08-04 Thread Jason Altekruse (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse resolved DRILL-3341. Resolution: Fixed Assignee: Sudheesh Katkam (was: Jason Altekruse) > M

[jira] [Resolved] (DRILL-3122) Changing a session option to default value results in status as changed

2015-08-04 Thread Jason Altekruse (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse resolved DRILL-3122. Resolution: Fixed Assignee: Sudheesh Katkam (was: Jason Altekruse) > Changin

Re: Lucene Format Plugin

2015-08-03 Thread Jason Altekruse
Hey Rahul, This is really cool! Thanks for all of the time you put into writing this, I think we have a lot of available opportunities to reach new communities with efforts like this. I noticed last week another contributor opened a JIRA for a solr plugin, there might be a good opportunity for th

Re: Review Request 36210: DRILL-3341: Move OperatorWrapper list and FragmentWrapper list creation to ProfileWrapper ctor

2015-07-20 Thread Jason Altekruse
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36210/#review92328 --- Ship it! Ship It! - Jason Altekruse On July 13, 2015, 11:08 p.m

Re: [DISCUSS] Allowing the option to use github pull requests in place of reviewboard

2015-07-20 Thread Jason Altekruse
Hello All, As you might have noticed, we have started opening up pull requests for new patches and going through the review cycle over on github. I wanted to bring up one small issue that Sudheesh noticed with my review of one of his patches. He had posted his change-set in two parts (the primary

[jira] [Created] (DRILL-3521) [umbrella] Review switch statements throughout codebase to add default cases where there are none

2015-07-20 Thread Jason Altekruse (JIRA)
Jason Altekruse created DRILL-3521: -- Summary: [umbrella] Review switch statements throughout codebase to add default cases where there are none Key: DRILL-3521 URL: https://issues.apache.org/jira/browse/DRILL

Re: MaterializedField

2015-07-17 Thread Jason Altekruse
It is meant to represent that a data type has been materialized for a given column. As we can have column with the ANY type at planning time, discovery of type is the distinction made when we create one of the MaterializedField s. Unfortunately there is no explicit concept for a non-MaterializedFi

Re: Recursive CTE Support in Drill

2015-07-16 Thread Jason Altekruse
@Alexander If you want to test the speed of the ODBC driver you can do that without a new storage plugin. If you get the entire dataset into memory, it will be returned from Drill a quickly as we can possibly send it to the client. One way to do this is to insert a sort; we cannot send along any o

[jira] [Resolved] (DRILL-3483) Clarify CommonConstants' constants.

2015-07-16 Thread Jason Altekruse (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse resolved DRILL-3483. Resolution: Fixed Fixed in 9b351c945b5f10d27cf07b9b5c1a435a029614b7 > Clar

Re: Build failed in Jenkins: drill-scm #466

2015-07-16 Thread Jason Altekruse
160e5457 0 --=[pre_exec_merge], > commit a0d3c6977820516983142c96d7f9374681529968 0 --=[drill-1.0.0-m1], > commit 98b208e262dd9b54988f5eeffacea7d0e83c958c 0 > --=[drill-root-0.4.0-incubating], commit > c7cb36c8b45613bab299dfe6cafb36d4d2e00add 0 --=[0.9.0]} ] > [info] HEA

DRILL-1904 package docs I just merged violates the checkstyle, will be pushing a fix

2015-07-16 Thread Jason Altekruse
Hello All, I just pushed a commit with my package docs that I wrote a while ago. I made some last edits to remove spelling and grammar issues found in the final reviews and I stupidly did not run the build before I pushed it. I messed up the edits and left two spaces at the end of lines. This wil

Re: Looking for a final review of package documentation Drill-1904

2015-07-16 Thread Jason Altekruse
developers should be aware of and I will work on filling out a complete draft. https://issues.apache.org/jira/browse/DRILL-2077 On Thu, Jul 16, 2015 at 11:58 AM, Jason Altekruse wrote: > Bump > > On Mon, Jul 6, 2015 at 2:45 PM, Jason Altekruse > wrote: > >> Hello Drill dev

Re: Looking for a final review of package documentation Drill-1904

2015-07-16 Thread Jason Altekruse
Bump On Mon, Jul 6, 2015 at 2:45 PM, Jason Altekruse wrote: > Hello Drill devs, > > A few months ago I submitted this patch for package level documentation in > Drill. I received some comments and fixed a few reported issues with the > patch, but I did not follow through to t

Re: netty 3.6.6 jar part of Drill dependencies

2015-07-16 Thread Jason Altekruse
Just to share the strategy for debugging future instances of problems like this. Maven has a command that will show you the exploded dependency tree of a project [0]. Just pipe this into a file and look for the version number of any jars you found that look like they may be conflicting transitive d

[jira] [Created] (DRILL-3484) Error using functions with no parameters when `drill.exec.functions.cast_empty_string_to_null` is set to true

2015-07-09 Thread Jason Altekruse (JIRA)
Jason Altekruse created DRILL-3484: -- Summary: Error using functions with no parameters when `drill.exec.functions.cast_empty_string_to_null` is set to true Key: DRILL-3484 URL: https://issues.apache.org/jira

Re: Review Request 36210: DRILL-3341: Move OperatorWrapper list and FragmentWrapper list creation to ProfileWrapper ctor

2015-07-09 Thread Jason Altekruse
va-exec/src/main/java/org/apache/drill/exec/server/rest/profile/FragmentWrapper.java (line 103) <https://reviews.apache.org/r/36210/#comment144207> Can we just change the name? - Jason Altekruse On July 6, 2015, 5:15 p.m., Sudh

Re: [DISCUSS] Allowing the option to use github pull requests in place of reviewboard

2015-07-08 Thread Jason Altekruse
pagate to gh > and close the PR request there. > > No fancy plugins needed for that. Mentioning the jira gets a link to the > PR and the content of the PR gets mirrored to the mailing list as it > happens as well. > > Sent from my iPhone > > > On Jul 7, 2015

Re: Timing for 1.2 release, ideas around things to include

2015-07-08 Thread Jason Altekruse
I would like to see some attention given to the Cassandra storage plugin work that Yash submitted a few months ago. I am willing to help out with the review on this, but might not have a lot of time to contribute to new code that still needs to be written before it is merged. If anyone has interest

Review Request 36292: DRILL-2838: flatten after join failing

2015-07-07 Thread Jason Altekruse
-CREATION Diff: https://reviews.apache.org/r/36292/diff/ Testing --- Testing in progress, relevent related unit tests are passing Thanks, Jason Altekruse

[jira] [Created] (DRILL-3471) Flatten currently sharing code poorly with project operator

2015-07-07 Thread Jason Altekruse (JIRA)
Jason Altekruse created DRILL-3471: -- Summary: Flatten currently sharing code poorly with project operator Key: DRILL-3471 URL: https://issues.apache.org/jira/browse/DRILL-3471 Project: Apache Drill

[jira] [Resolved] (DRILL-1094) Using events to parse JSON

2015-07-07 Thread Jason Altekruse (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse resolved DRILL-1094. Resolution: Fixed Fix Version/s: (was: Future) 1.0.0 The

Re: [DISCUSS] Allowing the option to use github pull requests in place of reviewboard

2015-07-07 Thread Jason Altekruse
7, 2015 at 1:35 PM, Jacques Nadeau > > wrote: > > >>> > > >>> I don't think we need anything formal. Do you want to propose a few > > >> simple > > >>> guidelines? > > >>> > > >>> E.g. Should we li

Re: [DISCUSS] Allowing the option to use github pull requests in place of reviewboard

2015-07-07 Thread Jason Altekruse
Proposed process below in the linked google doc. Please make suggestions in comments. Much of the content is a description of the current process, which the dev team is already familiar with. I'm hoping that this will be detailed enough that we can just include it in the docs as the general guidel

Re: [DISCUSS] Allowing the option to use github pull requests in place of reviewboard

2015-07-07 Thread Jason Altekruse
Seems to be enough consensus that this is beneficial. I took a look at the bylaws and it doesn't say anything specific there about an official review process. Is there a need to start a separate vote thread before making a change like this? I would be in favor of allowing both for a little bit, if

[jira] [Resolved] (DRILL-2745) Query returns IOB Exception when JSON data with empty arrays is input to flatten function

2015-07-06 Thread Jason Altekruse (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-2745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse resolved DRILL-2745. Resolution: Not A Problem > Query returns IOB Exception when JSON data with empty arrays

Looking for a final review of package documentation Drill-1904

2015-07-06 Thread Jason Altekruse
Hello Drill devs, A few months ago I submitted this patch for package level documentation in Drill. I received some comments and fixed a few reported issues with the patch, but I did not follow through to try to get a +1 from another committer. With everything that was going on stabilizing Drill

Re: [VOTE] Release Apache Drill 1.1.0 (rc0)

2015-07-02 Thread Jason Altekruse
+1 binding - downloaded and built the source tarball, all tests passed (on MAC osx) - started sqlline, issued a few queries - tried a basic update of storage plugin from the web UI and looked over a few query profiles On Thu, Jul 2, 2015 at 5:42 PM, Mehant Baid wrote: > +1 (binding) > > * Down

[jira] [Resolved] (DRILL-2105) Query fails when using flatten on JSON data where some documents have an empty array

2015-07-02 Thread Jason Altekruse (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse resolved DRILL-2105. Resolution: Fixed Fix Version/s: (was: 1.2.0) 0.8.0 It looks

[jira] [Resolved] (DRILL-1770) Flatten on top a subquery which applies flatten over kvgen results in a ClassCastException

2015-07-02 Thread Jason Altekruse (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse resolved DRILL-1770. Resolution: Fixed Fix Version/s: (was: 1.2.0) 0.8.0 This was

[jira] [Resolved] (DRILL-1754) Flatten nested within another flatten fails

2015-07-02 Thread Jason Altekruse (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse resolved DRILL-1754. Resolution: Duplicate Fix Version/s: (was: 1.2.0) > Flatten nested wit

[jira] [Resolved] (DRILL-1753) Flatten fails on a repeated map, where the maps being flattened contain repeated lists

2015-07-02 Thread Jason Altekruse (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-1753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse resolved DRILL-1753. Resolution: Fixed Fix Version/s: (was: 1.2.0) 1.1.0 > Flat

Apache Drill Hangout Information

2015-06-30 Thread Jason Altekruse
Hello Drillers, I have created a Google spreadsheet to track leaders of the weekly hangout (Tuesday at 10am Pacific time) to make sure we always have someone able to attend the meeting and facilitate discussion. Commitment is pretty low, for anyone who has attended the hangout it should be easy to

Re: SchemaFactory.logger

2015-06-30 Thread Jason Altekruse
My regex won't capture every case, but it found a decent number of these: interface .*\n.*logger Targets String 'interface .*\n.*logger' Found usages (29 usages found) Unclassified usage (29 usages found) drill-common (4 usages found) org.apache.drill.common.expressi

Re: SchemaFactory.logger

2015-06-30 Thread Jason Altekruse
I assume it was added during creation of the file using a template that inserted the logger by default. We should remove it and any other places where it occurs to prevent confusion. On Tue, Jun 30, 2015 at 2:42 PM, Daniel Barclay wrote: > Why does _interface_ org.apache.drill.exec.store.SchemaF

Re: [DISCUSSION] should BufferAllocator.buffer() throw an OutOfMemoryException ?

2015-06-30 Thread Jason Altekruse
+1 to Hanifi's comments, I think it makes much more sense to have a number of sites where the operators are explicitly catching a checked OOM exception and either decide to handle it or produce a message like "Hash Aggregate does not support our of memory conditions". This would be particularly use

Hangout starting in 30 minutes

2015-06-30 Thread Jason Altekruse
Come join the Apache Drill hangout to find out what is new in the upcoming 1.1 release. Anyone with an interest in Drill is welcome to attend. https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc

[jira] [Resolved] (DRILL-3370) FLATTEN error with a where clause

2015-06-29 Thread Jason Altekruse (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse resolved DRILL-3370. Resolution: Fixed Fixed in a915085e8a8b4255ff659086d047cc5dd874a5bf > FLATTEN error wit

[jira] [Resolved] (DRILL-1673) Flatten function can not work well with nested arrays.

2015-06-29 Thread Jason Altekruse (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse resolved DRILL-1673. Resolution: Fixed Fixed in 233faf2674c9ea2a9e416d01091bbf3658406f69 > Flatten function

Re: Review Request 35942: DRILL-1673 (reopened) : issue with flatten when used with nested lists

2015-06-26 Thread Jason Altekruse
e the ternary statment before running tests and merging. - Jason --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35942/#review89567 -----

Review Request 35942: DRILL-1673 (reopened) : issue with flatten when used with nested lists

2015-06-26 Thread Jason Altekruse
: https://reviews.apache.org/r/35942/diff/ Testing --- Unit tests run, regression in progress Thanks, Jason Altekruse

Re: Review Request 35881: DRILL-3354: TestBuilder can check if the number of result batches equals some expected value

2015-06-26 Thread Jason Altekruse
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35881/#review89528 --- Ship it! Ship It! - Jason Altekruse On June 25, 2015, 8:02 p.m

[jira] [Resolved] (DRILL-2167) Order by on a repeated index from the output of a flatten on large no of records results in incorrect results

2015-06-26 Thread Jason Altekruse (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse resolved DRILL-2167. Resolution: Duplicate Fix Version/s: (was: 1.1.0) > Order by on a repeated in

Re: Review Request 35881: DRILL-3354: TestBuilder can check if the number of result batches equals some expected value

2015-06-25 Thread Jason Altekruse
big issue though. - Jason Altekruse On June 25, 2015, 4:53 p.m., abdelhakim deneche wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https

[jira] [Resolved] (DRILL-1616) Add support for count() on maps and arrays

2015-06-23 Thread Jason Altekruse (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse resolved DRILL-1616. Resolution: Duplicate > Add support for count() on maps and arr

[jira] [Resolved] (DRILL-1616) Add support for count() on maps and arrays

2015-06-23 Thread Jason Altekruse (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse resolved DRILL-1616. Resolution: Fixed > Add support for count() on maps and arr

Re: Review Request 35484: DRILL-2851: set an upper-bound on # of bytes to re-allocate to prevent overflows

2015-06-23 Thread Jason Altekruse
t, the zeroVector() method calls a method on the data buffer directly. We need to add a return false after the clear in the finally block. exec/java-exec/src/main/codegen/templates/NullableValueVectors.java (line 216) <https://reviews.apache.org/r/35484/#comment141678> Same as above - Jas

Re: Moving matured storage plugins out of contrib

2015-06-23 Thread Jason Altekruse
In light of Aditya's explanation I think this is a good way to clarify the relationship between Drill, Hive and Hbase. These are core plugins that we are planning to support completely, just as we do with the HDFS API in the FileSystemPlugin, which is currently a part of the exec module. +1 On Tu

Re: Drill should validate column names within window functions

2015-06-23 Thread Jason Altekruse
You can mistype a column that you are sorting on or joining on and get the same problem. Vicky has filed a bug for a subquery that looked to be returning wrong results, but one of the columns she was trying to refer to was not available within that scope, and we happily filled it in with all null v

Re: Hangout starting a little late this morning, should start around 10:10

2015-06-23 Thread Jason Altekruse
That is pacific time, the meeting will start in about 10 minutes. On Tue, Jun 23, 2015 at 9:59 AM, Jason Altekruse wrote: > Join us at our weekly hangout to discuss what has been happening in the > Drill community! > > https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc >

Hangout starting a little late this morning, should start around 10:10

2015-06-23 Thread Jason Altekruse
Join us at our weekly hangout to discuss what has been happening in the Drill community! https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc

[DISCUSS] Allowing the option to use github pull requests in place of reviewboard

2015-06-22 Thread Jason Altekruse
Hello Drill developers, I am writing this message today to propose allowing the use of github pull requests to perform reviews in place of the apache reviewboard instance. Reviewboard has caused a number of headaches in the past few months, and I think its time to evaluate the benefits of the apa

Re: Review Request 35484: DRILL-2851: set an upper-bound on # of bytes to re-allocate to prevent overflows

2015-06-19 Thread Jason Altekruse
> On June 19, 2015, 5:06 p.m., Jason Altekruse wrote: > > exec/java-exec/src/main/codegen/templates/FixedValueVectors.java, line 92 > > <https://reviews.apache.org/r/35484/diff/1/?file=985297#file985297line92> > > > > The reallocation loops will run inf

Re: Review Request 35484: DRILL-2851: set an upper-bound on # of bytes to re-allocate to prevent overflows

2015-06-19 Thread Jason Altekruse
> On June 19, 2015, 5:06 p.m., Jason Altekruse wrote: > > exec/java-exec/src/main/codegen/templates/FixedValueVectors.java, line 92 > > <https://reviews.apache.org/r/35484/diff/1/?file=985297#file985297line92> > > > > The reallocation loops will run inf

[jira] [Created] (DRILL-3323) Flatten planning rule creates unneeded copy of the list being flattened, causes executuion/allocation issues with large lists

2015-06-19 Thread Jason Altekruse (JIRA)
Jason Altekruse created DRILL-3323: -- Summary: Flatten planning rule creates unneeded copy of the list being flattened, causes executuion/allocation issues with large lists Key: DRILL-3323 URL: https

Re: Review Request 35484: DRILL-2851: set an upper-bound on # of bytes to re-allocate to prevent overflows

2015-06-19 Thread Jason Altekruse
at the allocator tracks buffer lengths with ints. - Jason Altekruse On June 17, 2015, 6:54 p.m., Hanifi Gunes wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.

Re: [DISCUSS] Making the drill codebase easier to unit test

2015-06-18 Thread Jason Altekruse
edge cases, > and > > the worst part is that any changes to how queries are planned or for > > example the size of the batches can make some tests useless. > > > > On Tue, Jun 16, 2015 at 12:38 PM, Jason Altekruse < > > altekruseja...@gmail.com> > > wrote:

[jira] [Resolved] (DRILL-3263) Read smallint and tinyint data from hive as integer until these types are well supported throughout Drill

2015-06-18 Thread Jason Altekruse (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse resolved DRILL-3263. Resolution: Fixed Fixed in 437706f750b0ec50b60582ea2c47e7017e2718e3 > Read smallint

Re: Review Request 35475: DRILL-3263: read tinyint and smallint columns from Hive as integer

2015-06-17 Thread Jason Altekruse
n. > Jason, can you work with Bridget and Kristine to add this. Will do - Jason --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35475/#review88243 -----

[DISCUSS] Making the drill codebase easier to unit test

2015-06-16 Thread Jason Altekruse
Hello Drill devs, I would like to propose a proactive effort to make the Drill codebase easier to unit test. Many JIRAs have been created for bugs that should have been prevented by better unit testing, and we are still fixing these kinds of bugs today as they crop up. I have a few ideas, and I pl

Review Request 35475: DRILL-3263: read tinyint and smallint columns from Hive as integer

2015-06-15 Thread Jason Altekruse
/hive/TestHiveStorage.java 27ba9fe Diff: https://reviews.apache.org/r/35475/diff/ Testing --- Unit tests passing, cluster tests are pending Thanks, Jason Altekruse

Re: eclipse:eclipse failing

2015-06-09 Thread Jason Altekruse
Please upload the image or file to a public host and share a link. Thanks, Jason On Tue, Jun 9, 2015 at 11:52 AM, Abdel Hakim Deneche wrote: > I am not able to see the error. Did you attach an image to your email ? if > so, I don't think the mailing list allows attachments to emails. > > On Mon

[jira] [Created] (DRILL-3263) Read smallint and tinyint data from hive as integer until these types are well supported throughout Drill

2015-06-08 Thread Jason Altekruse (JIRA)
Jason Altekruse created DRILL-3263: -- Summary: Read smallint and tinyint data from hive as integer until these types are well supported throughout Drill Key: DRILL-3263 URL: https://issues.apache.org/jira/browse

Re: [Discuss] Hive - Smallint and Tinyint

2015-06-08 Thread Jason Altekruse
se are opened, it >> would be easier to give feedback on the relative merit of that work versus >> the alternative solution you suggested. >> >> On Mon, Jun 8, 2015 at 11:12 AM, Jason Altekruse < >> altekruseja...@gmail.com> >> wrote: >> >&g

Re: [Discuss] Hive - Smallint and Tinyint

2015-06-08 Thread Jason Altekruse
trange behaviors. Are you planning on changing them on the > schema side as well? That way Calcite wouldn't insert weird expression > patterns that would cause other problems if you change the execution side. > > On Mon, Jun 8, 2015 at 12:41 PM, Jason Altekruse > > wrote: >

Re: [Discuss] Hive - Smallint and Tinyint

2015-06-08 Thread Jason Altekruse
ened, it > would be easier to give feedback on the relative merit of that work versus > the alternative solution you suggested. > > On Mon, Jun 8, 2015 at 11:12 AM, Jason Altekruse > > wrote: > > > Hello Drillers, > > > > I have been working on DRILL-3209, w

[Discuss] Hive - Smallint and Tinyint

2015-06-08 Thread Jason Altekruse
Hello Drillers, I have been working on DRILL-3209, which aims to speed up reading from hive tables by re-planning them as native Drill reads in the case where the tables are backed by files that have available native readers. This will begin with parquet and delimited text files. To provide the s

Reminder: Weekly hangout tomorrow(Tuesday) 10am Pacific

2015-06-01 Thread Jason Altekruse
/_/event/ci4rdiju8bv04a64efj5fedd0lc - Jason Altekruse

Re: Review Request 34541: DRILL-3147: tpcds-sf1-parquet query 73 causes memory leak

2015-06-01 Thread Jason Altekruse
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34541/#review85986 --- Ship it! Ship It! - Jason Altekruse On May 27, 2015, 2:19 p.m

Re: question about correlated arrays and flatten

2015-05-29 Thread Jason Altekruse
I understand what you want to do, unfortunately we don't have support for this right now. A UDF is the best I can suggest at this point. Just to explore the idea a little further for the sake of creating a complete feature request, I assume you would just want nulls filled in for the cases where t

[jira] [Created] (DRILL-3209) [Umbrella] Plan reads of Hive tables as native Drill reads when a native reader for the underlying table format exists

2015-05-28 Thread Jason Altekruse (JIRA)
Jason Altekruse created DRILL-3209: -- Summary: [Umbrella] Plan reads of Hive tables as native Drill reads when a native reader for the underlying table format exists Key: DRILL-3209 URL: https://issues.apache.org

Re: Please don't use assert in unit tests

2015-05-28 Thread Jason Altekruse
Hello Drillers, I merged a patch from Chris yesterday that cleaned up and fixed some issues in a good number of the unit tests. The primary problem he solved was improper usage of Java asserts to test for the detecting success conditions in the tests. There was no single contributor or component

Github mirror out of date

2015-05-28 Thread Jason Altekruse
://issues.apache.org/jira/browse/INFRA-9733 This is a good time to remind everyone to be tracking the apache repo, not the github mirror. This is the address you can add as a remote to track it. git://git.apache.org/drill.git - Jason Altekruse

Re: Review Request 34598: DRILL-2923: Ensure all unit tests pass without assertions enabled

2015-05-27 Thread Jason Altekruse
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34598/#review85453 --- Ship it! Ship It! - Jason Altekruse On May 26, 2015, 5:39 p.m

Hangout happening now

2015-05-26 Thread Jason Altekruse
Come join the Drill community as we discuss what has been happening lately and what is in the pipeline. All are welcome, if you know about Drill, want to know more or just want to listen in. https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc

Re: Review Request 34498: DRILL-3143: MaterializedField#clone should deep copy itself without disregarding its children

2015-05-22 Thread Jason Altekruse
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34498/#review85014 --- Ship it! Ship It! - Jason Altekruse On May 22, 2015, 10:21 p.m

Re: Review Request 34498: DRILL-3143: MaterializedField#clone should deep copy itself without disregarding its children

2015-05-22 Thread Jason Altekruse
/record/TestMaterializedField.java <https://reviews.apache.org/r/34498/#comment136477> Minor nitpick, grammar - Jason Altekruse On May 22, 2015, 9:15 p.m., Hanifi Gunes wrote: > > --- > This is an automatically generated e

[jira] [Created] (DRILL-3158) Add result verification to tests that currently run a query and just check to make sure no exception occurs

2015-05-20 Thread Jason Altekruse (JIRA)
Jason Altekruse created DRILL-3158: -- Summary: Add result verification to tests that currently run a query and just check to make sure no exception occurs Key: DRILL-3158 URL: https://issues.apache.org/jira

Re: [VOTE] Release Apache Drill 1.0.0 (rc1)

2015-05-17 Thread Jason Altekruse
+1 binding Downloaded the source and binary tarballs, built the source and all unit tests passed. Ran a few queries in embedded mode and checked a few pages on the web UI. Great work everyone! Even Cyanide and Happiness got into the drill spirit to celebrate the release! http://explosm.net/comic

[jira] [Created] (DRILL-3116) Headers do not resize in enhanced sqlline that correctly resizes columns to nicely format data

2015-05-15 Thread Jason Altekruse (JIRA)
Jason Altekruse created DRILL-3116: -- Summary: Headers do not resize in enhanced sqlline that correctly resizes columns to nicely format data Key: DRILL-3116 URL: https://issues.apache.org/jira/browse/DRILL-3116

[jira] [Created] (DRILL-3092) Memory leak when an allocation fails near the creation of a RecordBatchData object

2015-05-14 Thread Jason Altekruse (JIRA)
Jason Altekruse created DRILL-3092: -- Summary: Memory leak when an allocation fails near the creation of a RecordBatchData object Key: DRILL-3092 URL: https://issues.apache.org/jira/browse/DRILL-3092

[jira] [Resolved] (DRILL-2976) Set default of extended JSON support for output to false until issues are resolved

2015-05-12 Thread Jason Altekruse (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-2976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse resolved DRILL-2976. Resolution: Fixed Fixed in 4c8ccedf387a8e3539f8eaabdcbc7945655d11e9 and

Re: Review Request 34064: DRILL-1980

2015-05-11 Thread Jason Altekruse
iting data with it, external confirmation of accuracy would be nice to have. We should probably open a JIRA for testing it once it is added to parquet-mr or the c++ reader based on impala. - Jason Altekruse On May 12, 2015, 4:37 a.m., Mehant

Inconsistent Class-loading issue with recent build

2015-05-11 Thread Jason Altekruse
Drillers, Sudheesh was trying to run some queries against the parquet files we package with the build and was seeing this error message [1] in embedded mode. I tried doing a build on my machine and ran the same query and did not see the problem, we also exchanged builds and he saw the issue on his

Re: Review Request 34063: DRILL-2776: add extensive tests for empty population coverage

2015-05-11 Thread Jason Altekruse
> On May 11, 2015, 11:52 p.m., Jason Altekruse wrote: > > exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/TestEmptyPopulation.java, > > line 69 > > <https://reviews.apache.org/r/34063/diff/3/?file=955845#file955845line69> > > > > A

Re: Review Request 34063: DRILL-2776: add extensive tests for empty population coverage

2015-05-11 Thread Jason Altekruse
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34063/#review83328 --- Ship it! Ship It! - Jason Altekruse On May 12, 2015, 12:18 a.m

<    1   2   3   4   5   6   >