Re: Hive and HDFS on Kerborized Cluster

Jaques, With the help of our Freelancer, Robert Reichel, we’ve been able to get Drill running on a Kerborized CDH cluster. In the next day or so, we’ll post implementation instructions. William Witt CTO HadoopRevealed Inc. > On Dec 21, 2015, at 12:09 PM, Jacques Nadeau

Out Of Memory Error (Possible Regression)

I am seeing an OOM error while executing a simple CTAS query. I raised DRILL-4324 for this. The query mentioned in the JIRA used to complete successfully without any issue prior to 1.5. Any idea what could have caused the regression? - Rahul

[jira] [Created] (DRILL-4234) Drill running out of memory for a simple CTAS query

Rahul Challapalli created DRILL-4234: Summary: Drill running out of memory for a simple CTAS query Key: DRILL-4234 URL: https://issues.apache.org/jira/browse/DRILL-4234 Project: Apache Drill

Re: Out Of Memory Error (Possible Regression)

I installed the latest master and ran this query. So planner.memory.max_query_memory_per_node should have been the default value. I switched back to 1.4.0 branch and this query completed successfully. On Wed, Dec 30, 2015 at 10:37 AM, Abdel Hakim Deneche wrote: > Rahul,

Re: Out Of Memory Error (Possible Regression)

Since the accounting changed (more accurate), the termination condition for the sort operator will be different than before. In fact, this likely will be sooner since our accounting is much larger than previously (since we correctly consider the entire allocation rather than simply the used

Re: Naming the new ValueVector Initiative

Great to hear on the name approval! If it is OK I will consolidate to 1 repo and nest the C++ code under cpp/ and add some format Markdown files there in the next week or so. One repo is perfectly fine (as compared with Parquet) as long as the language implementations can evolve and be released

Re: Out Of Memory Error (Possible Regression)

I'll let Steven answer your question directly. FYI, we are running a regression suite that was forked from the MapR repo a month or so ago because we had to fix a bunch of things to make it work with Apache Hadoop. (There was a thread about this back then and we haven't yet figured out how to

Re: Out Of Memory Error (Possible Regression)

The following tests are failing: /Functional/data-shapes/wide-columns/5000/1000rows/parquet/q163_DRILL-2046.q > > /Functional/data-shapes/wide-columns/5000/1000rows/parquet/q177_DRILL-2046.q > /Functional/data-shapes/wide-columns/5000/1000rows/parquet/q174.q >

Re: Out Of Memory Error (Possible Regression)

are you running the tests on 32 core machines ? a different number of cores affects how much memory is available for the sort On Wed, Dec 30, 2015 at 1:02 PM, Abdel Hakim Deneche wrote: > The following tests are failing: > > >>

Re: Naming the new ValueVector Initiative

Good news guys, Apache trademarks just approved the name Apache Arrow [1]. I will update the proposal and resubmit to the board. thanks, Jacques [1] https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-92 -- Jacques Nadeau CTO and Co-Founder, Dremio On Thu, Dec 17, 2015 at 11:21 AM, Julian

[GitHub] drill pull request: DRILL-3739: (part 2) Fix issues in reading Hiv...

Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/312 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: Out Of Memory Error (Possible Regression)

Created DRILL-4236 to keep track of this improvement. On Wed, Dec 30, 2015 at 11:01 AM, Jacques Nadeau wrote: > Since the accounting changed (more accurate), the termination condition for > the sort operator will be

[jira] [Created] (DRILL-4236) ExternalSort should use the new allocator functionality to better manage it's memory usage

Deneche A. Hakim created DRILL-4236: --- Summary: ExternalSort should use the new allocator functionality to better manage it's memory usage Key: DRILL-4236 URL: https://issues.apache.org/jira/browse/DRILL-4236

Re: Out Of Memory Error (Possible Regression)

I didn't see any tests running out of memory. Which tests are you seeing with this? On Wed, Dec 30, 2015 at 1:37 PM, Abdel Hakim Deneche wrote: > Steven, > > were you able to successfully run the regression tests on the transfer > patch ? I just tried and saw several

Re: Out Of Memory Error (Possible Regression)

Rahul, How much memory was assigned to the sort operator ( planner.memory.max_query_memory_per_node) ? On Wed, Dec 30, 2015 at 9:54 AM, rahul challapalli < challapallira...@gmail.com> wrote: > I am seeing an OOM error while executing a simple CTAS query. I raised > DRILL-4324 for this. The

[jira] [Created] (DRILL-4235) Hit IllegalStateException when exec.queue.enable=ture

Dechang Gu created DRILL-4235: - Summary: Hit IllegalStateException when exec.queue.enable=ture Key: DRILL-4235 URL: https://issues.apache.org/jira/browse/DRILL-4235 Project: Apache Drill Issue

[jira] [Resolved] (DRILL-3739) NPE on select from Hive for HBase table

[ https://issues.apache.org/jira/browse/DRILL-3739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mehant Baid resolved DRILL-3739. Resolution: Fixed Fix Version/s: (was: 1.4.0) 1.5.0 Fixed in

Re: Naming the new ValueVector Initiative

Sounds good. We'll need to start the Apache repo with just Apache code given how we've done the proposal. After we get established, we can break any independent progress you've made into a set of patches to add to the existing repo through the normal Apache commit process. -- Jacques Nadeau

Re: Out Of Memory Error (Possible Regression)

I am guessing the configuration is different since some of these failing suites were added a while ago ( > 1 month) On Wed, Dec 30, 2015 at 12:42 PM, Jacques Nadeau wrote: > I'll let Steven answer your question directly. > > FYI, we are running a regression suite that was

Re: Making Drill support graceful downgrades

One additional thought on ignoring unrecognized fields, this would give the behavior of ignoring values with mistyped field names. I think that right now I will submit a JIRA to just catch the Jackson error and direct users to check or clear out their configuration in zookeeper or /tmp/drill

[GitHub] drill pull request: DRILL-3739: (part 2) Fix issues in reading Hiv...

Github user mehant commented on the pull request: https://github.com/apache/drill/pull/312#issuecomment-168060723 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

Re: Out Of Memory Error (Possible Regression)

That must be it. On Wed, Dec 30, 2015 at 1:22 PM, Steven Phillips wrote: > No, we are running on 4 4-core machines. > > On Wed, Dec 30, 2015 at 2:10 PM, Abdel Hakim Deneche < > adene...@maprtech.com> > wrote: > > > are you running the tests on 32 core machines ? a different

[jira] [Created] (DRILL-4237) Skew in hash distribution

Aman Sinha created DRILL-4237: - Summary: Skew in hash distribution Key: DRILL-4237 URL: https://issues.apache.org/jira/browse/DRILL-4237 Project: Apache Drill Issue Type: Bug