date:20180522

[jira] [Created] (DRILL-6441) IllegalStateException: Allocator[ROOT] closed with outstanding child allocators.

2018-05-22 Thread Khurram Faraaz (JIRA)

Khurram Faraaz created DRILL-6441:
-

 Summary: IllegalStateException: Allocator[ROOT] closed with 
outstanding child allocators.
 Key: DRILL-6441
 URL: https://issues.apache.org/jira/browse/DRILL-6441
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Affects Versions: 1.14.0
Reporter: Khurram Faraaz


This is seen when the drillbit is shutdown using $DRILL_HOME/bin/drillbit.sh 
stop, it happens when BaseAllocator tries to close.
Apache Drill 1.14.0 commit c6c5d27d91468a29656bee2acba55d3321978aab

{noformat}
 
2018-05-11 15:19:44,510 [2509e8fe-f8fb-0212-5bb6-f49d7c611ad0:frag:0:0] INFO 
o.a.d.e.w.f.FragmentStatusReporter - 2509e8fe-f8fb-0212-5bb6-f49d7c611ad0:0:0: 
State to report: FINISHED
Wed May 16 10:17:13 PDT 2018 Terminating drillbit pid 32076
2018-05-16 10:17:13,793 [Drillbit-ShutdownHook#0] INFO 
o.apache.drill.exec.server.Drillbit - Received shutdown request.
2018-05-16 10:17:14,876 [Drillbit-ShutdownHook#0] INFO 
o.a.drill.exec.compile.CodeCompiler - Stats: code gen count: 20, cache miss 
count: 6, hit rate: 70%
2018-05-16 10:17:14,890 [Drillbit-ShutdownHook#0] ERROR 
o.a.d.exec.server.BootStrapContext - Error while closing
java.lang.IllegalStateException: Allocator[ROOT] closed with outstanding child 
allocators.
Allocator(ROOT) 0/0/9577600/34359738368 (res/actual/peak/limit)
 child allocators: 8
 Allocator(WebServer:AnonUserSession) 0/0/0/9223372036854775807 
(res/actual/peak/limit)
 child allocators: 0
 ledgers: 0
 reservations: 0
 Allocator(WebServer:AnonUserSession) 0/0/0/9223372036854775807 
(res/actual/peak/limit)
 child allocators: 0
 ledgers: 0
 reservations: 0
 Allocator(WebServer:AnonUserSession) 0/0/0/9223372036854775807 
(res/actual/peak/limit)
 child allocators: 0
 ledgers: 0
 reservations: 0
 Allocator(WebServer:AnonUserSession) 0/0/0/9223372036854775807 
(res/actual/peak/limit)
 child allocators: 0
 ledgers: 0
 reservations: 0
 Allocator(WebServer:AnonUserSession) 0/0/0/9223372036854775807 
(res/actual/peak/limit)
 child allocators: 0
 ledgers: 0
 reservations: 0
 Allocator(WebServer:AnonUserSession) 0/0/0/9223372036854775807 
(res/actual/peak/limit)
 child allocators: 0
 ledgers: 0
 reservations: 0
 Allocator(WebServer:AnonUserSession) 0/0/0/9223372036854775807 
(res/actual/peak/limit)
 child allocators: 0
 ledgers: 0
 reservations: 0
 Allocator(WebServer:AnonUserSession) 0/0/0/9223372036854775807 
(res/actual/peak/limit)
 child allocators: 0
 ledgers: 0
 reservations: 0
 ledgers: 0
 reservations: 0

at org.apache.drill.exec.memory.BaseAllocator.close(BaseAllocator.java:496) 
~[drill-memory-base-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
 at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:81) 
[drill-common-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
 at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:69) 
[drill-common-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
 at 
org.apache.drill.exec.server.BootStrapContext.close(BootStrapContext.java:259) 
~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
 at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:81) 
[drill-common-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
 at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:69) 
[drill-common-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
 at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:263) 
[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
 at org.apache.drill.exec.server.Drillbit$ShutdownThread.run(Drillbit.java:363) 
[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
2018-05-16 10:17:14,890 [Drillbit-ShutdownHook#0] INFO 
o.apache.drill.exec.server.Drillbit - Shutdown completed (1095 ms).


{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (DRILL-6440) Fix ignored unit tests in unnest

2018-05-22 Thread Parth Chandra (JIRA)

Parth Chandra created DRILL-6440:


 Summary: Fix ignored unit tests in unnest
 Key: DRILL-6440
 URL: https://issues.apache.org/jira/browse/DRILL-6440
 Project: Apache Drill
  Issue Type: Improvement
Reporter: Parth Chandra
Assignee: Parth Chandra






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: help for native drive for .NET

2018-05-22 Thread Robert Hou

I don't think there is a Drill driver for .NET.  For Windows, we have ODBC and 
JDBC.


Can you provide more information on your performance issue?


Thanks.


--Robert


From: ariolov...@gmail.com 
Sent: Monday, May 21, 2018 3:06 PM
To: dev@drill.apache.org
Subject: help for native drive for .NET

Hi,

So, today I use ODBC driver, but is too slow..



Do you know a native driver for .NET ?



Thanks!
Ario

[jira] [Created] (DRILL-6439) Sometimes Travis Times Out

2018-05-22 Thread Timothy Farkas (JIRA)

Timothy Farkas created DRILL-6439:
-

 Summary: Sometimes Travis Times Out
 Key: DRILL-6439
 URL: https://issues.apache.org/jira/browse/DRILL-6439
 Project: Apache Drill
  Issue Type: Bug
Reporter: Timothy Farkas
Assignee: Timothy Farkas


Ocassionally Travis builds run a few minutes longer than usual and timeout.

{code}
[32;1mchanges detected, packing new archive[0m
.
.


The job exceeded the maximum time limit for jobs, and has been terminated.
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (DRILL-6438) TestLocalExchange Generates a lot of unnecessary messages that pollute test logs.

2018-05-22 Thread Timothy Farkas (JIRA)

Timothy Farkas created DRILL-6438:
-

 Summary: TestLocalExchange Generates a lot of unnecessary messages 
that pollute test logs.
 Key: DRILL-6438
 URL: https://issues.apache.org/jira/browse/DRILL-6438
 Project: Apache Drill
  Issue Type: Improvement
Reporter: Timothy Farkas
Assignee: Timothy Farkas


See example

{code}
Running 
org.apache.drill.exec.physical.impl.TestLocalExchange#testGroupByMultiFields
Plan: {
  "head" : {
"version" : 1,
"generator" : {
  "type" : "ExplainHandler",
  "info" : ""
},
"type" : "APACHE_DRILL_PHYSICAL",
"options" : [ {
  "kind" : "LONG",
  "accessibleScopes" : "ALL",
  "name" : "planner.width.max_per_node",
  "num_val" : 2,
  "scope" : "SESSION"
}, {
  "kind" : "BOOLEAN",
  "accessibleScopes" : "ALL",
  "name" : "planner.enable_mux_exchange",
  "bool_val" : true,
  "scope" : "SESSION"
}, {
  "kind" : "BOOLEAN",
  "accessibleScopes" : "ALL",
  "name" : "planner.enable_demux_exchange",
  "bool_val" : false,
  "scope" : "SESSION"
}, {
  "kind" : "LONG",
  "accessibleScopes" : "ALL",
  "name" : "planner.slice_target",
  "num_val" : 1,
  "scope" : "SESSION"
} ],
"queue" : 0,
"hasResourcePlan" : false,
"resultMode" : "EXEC"
  },
  "graph" : [ {
"pop" : "fs-scan",
"@id" : 196611,
"userName" : "travis",
"files" : [ 
"file:/home/travis/build/apache/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.TestLocalExchange/root/empTable/6.json",
 
"file:/home/travis/build/apache/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.TestLocalExchange/root/empTable/9.json",
 
"file:/home/travis/build/apache/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.TestLocalExchange/root/empTable/3.json",
 
"file:/home/travis/build/apache/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.TestLocalExchange/root/empTable/1.json",
 
"file:/home/travis/build/apache/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.TestLocalExchange/root/empTable/2.json",
 
"file:/home/travis/build/apache/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.TestLocalExchange/root/empTable/7.json",
 
"file:/home/travis/build/apache/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.TestLocalExchange/root/empTable/0.json",
 
"file:/home/travis/build/apache/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.TestLocalExchange/root/empTable/5.json",
 
"file:/home/travis/build/apache/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.TestLocalExchange/root/empTable/4.json",
 
"file:/home/travis/build/apache/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.TestLocalExchange/root/empTable/8.json"
 ],
"storage" : {
  "type" : "file",
  "enabled" : true,
  "connection" : "file:///",
  "config" : null,
  "workspaces" : {
"root" : {
  "location" : 
"/home/travis/build/apache/drill/exec/java-exec/./target/org.apache.drill.exec.physical.impl.TestLocalExchange/root",
  "writable" : true,
  "defaultInputFormat" : null,
  "allowAccessOutsideWorkspace" : false
},
"tmp" : {
  "location" : 
"/home/travis/build/apache/drill/exec/java-exec/./target/org.apache.drill.exec.physical.impl.TestLocalExchange/dfsTestTmp/1527026062606-0",
  "writable" : true,
  "defaultInputFormat" : null,
  "allowAccessOutsideWorkspace" : false
},
"default" : {
  "location" : 
"/home/travis/build/apache/drill/exec/java-exec/./target/org.apache.drill.exec.physical.impl.TestLocalExchange/root",
  "writable" : true,
  "defaultInputFormat" : null,
  "allowAccessOutsideWorkspace" : false
}
  },
  "formats" : {
"psv" : {
  "type" : "text",
  "extensions" : [ "tbl" ],
  "delimiter" : "|"
},
"csv" : {
  "type" : "text",
  "extensions" : [ "csv" ],
  "delimiter" : ","
},
"tsv" : {
  "type" : "text",
  "extensions" : [ "tsv" ],
  "delimiter" : "\t"
},
"httpd" : {
  "type" : "httpd",
  "logFormat" : "%h %t \"%r\" %>s %b \"%{Referer}i\""
},
"parquet" : {
  "type" : "parquet"
},
"json" : {
  "type" : "json",
  "extensions" : [ "json" ]
},
"pcap" : {
  "type" : "pcap"
},
"avro" : {
  "type" : "avro"
},
"sequencefile" : {
  "type" : "sequencefile",
  "extensions" : [ "seq" ]
},
"csvh" : {
  "type" : "text",
  "extensions" : [ "csvh" ],
  "extractHeader" : true,
  "delimiter" : ","

Re: Delete spurious branches from Apache

2018-05-22 Thread Boaz Ben-Zvi

  Done – all three were deleted, using github (thanks @Vlad Rozov).

There is also an active branch called “gh-pages” (and a non-active one called 
“gh-pages-master”) that seems to be used for the Drill documentation.
Maybe this work should be moved to its own Repo ?  It does not make sense to be 
a part of the Drill sources (e.g., note this branch is “3086 commits behind”).
@Bridget Bevens – what do you think ?

Thanks,

   Boaz

On 5/22/18, 1:02 PM, "Parth Chandra"  wrote:

Yes, please go ahead and remove these branches.

On Mon, May 21, 2018 at 8:06 PM, Vlad Rozov  wrote:

> There is an option  to delete the branch on github directly (no need to
> use "git push").
>
> Thank you,
>
> Vlad
>
>
> On 5/21/18 18:35, Boaz Ben-Zvi wrote:
>
>> I mistakenly pushed a branch (“MERGE-180521-01”) into the Apache
>> repo, and plan to delete it soon
>> (i.e. do  “git push 
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_drill.git=DwIFaQ=cskdkSMqhcnjZxdQVpwTXg=EqulKDxxEDCX6zbp1AZAa1-iAPQGgCioAqgDp7DE2BU=Jgwab6ezufiqEbsi7h1wNNGZPnUbR28Caq2JWPsSlp0=JiRFn2vcSJHxPA09nlPxBKSi9xk0thr-JKiaYNyMEws=
 --delete
>> MERGE-180521-01” ).
>>
>>Just in case someone notices 
>>
>>On this occasion: Looks like there are other similar such branches:
>> “DRILL-3478” and “DRILL-4235” ; any objection to deleting those as well ?
>>
>>  Thanks,
>>
>>  Boaz
>>
>>
>>
>

[jira] [Created] (DRILL-6437) Travis Fails Because Logs Are Flooded.

2018-05-22 Thread Timothy Farkas (JIRA)

Timothy Farkas created DRILL-6437:
-

 Summary: Travis Fails Because Logs Are Flooded.
 Key: DRILL-6437
 URL: https://issues.apache.org/jira/browse/DRILL-6437
 Project: Apache Drill
  Issue Type: Improvement
Reporter: Timothy Farkas
Assignee: Timothy Farkas


The Travis logs are flooded when downloading mysql.

{code}
Downloading from central: 
http://repo.maven.apache.org/maven2/com/jcabi/mysql-dist/5.6.14/mysql-dist-5.6.14-linux-amd64.zip
Progress (1): 0/325 MB
Progress (1): 0/325 MB
Progress (1): 0/325 MB
Progress (1): 0/325 MB
Progress (1): 0/325 MB
Progress (1): 0/325 MB
Progress (1): 0/325 MB
Progress (1): 0/325 MB
Progress (1): 0/325 MB
Progress (1): 0/325 MB
Progress (1): 0/325 MB
Progress (1): 0/325 MB
Progress (1): 0/325 MB
Progress (1): 0/325 MB
Progress (1): 0/325 MB
Progress (1): 0/325 MB
Progress (1): 0/325 MB
Progress (1): 0/325 MB
Progress (1): 0/325 MB
Progress (1): 0/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
Progress (1): 0.1/325 MB
{code}

And the Travis build fails with

{code}
The log length has exceeded the limit of 4 MB (this usually means that the test 
suite is raising the same exception over and over).

The job has been terminated
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (DRILL-6436) Store context and name in AbstractStoragePlugin instead of replicating fields in each StoragePlugin

2018-05-22 Thread Timothy Farkas (JIRA)

Timothy Farkas created DRILL-6436:
-

 Summary: Store context and name in AbstractStoragePlugin instead 
of replicating fields in each StoragePlugin
 Key: DRILL-6436
 URL: https://issues.apache.org/jira/browse/DRILL-6436
 Project: Apache Drill
  Issue Type: Improvement
Reporter: Timothy Farkas
Assignee: Timothy Farkas






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Delete spurious branches from Apache

2018-05-22 Thread Parth Chandra

Yes, please go ahead and remove these branches.

On Mon, May 21, 2018 at 8:06 PM, Vlad Rozov  wrote:

> There is an option  to delete the branch on github directly (no need to
> use "git push").
>
> Thank you,
>
> Vlad
>
>
> On 5/21/18 18:35, Boaz Ben-Zvi wrote:
>
>> I mistakenly pushed a branch (“MERGE-180521-01”) into the Apache
>> repo, and plan to delete it soon
>> (i.e. do  “git push https://github.com/apache/drill.git --delete
>> MERGE-180521-01” ).
>>
>>Just in case someone notices 
>>
>>On this occasion: Looks like there are other similar such branches:
>> “DRILL-3478” and “DRILL-4235” ; any objection to deleting those as well ?
>>
>>  Thanks,
>>
>>  Boaz
>>
>>
>>
>

[jira] [Resolved] (DRILL-5847) Flat Parquet Reader Performance Analysis

2018-05-22 Thread salim achouche (JIRA)


 [ 
https://issues.apache.org/jira/browse/DRILL-5847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

salim achouche resolved DRILL-5847.
---
   Resolution: Fixed
Fix Version/s: 1.14.0

> Flat Parquet Reader Performance Analysis
> 
>
> Key: DRILL-5847
> URL: https://issues.apache.org/jira/browse/DRILL-5847
> Project: Apache Drill
>  Issue Type: Sub-task
>  Components: Storage - Parquet
>Affects Versions: 1.11.0
>Reporter: salim achouche
>Assignee: salim achouche
>Priority: Major
>  Labels: performance
> Fix For: 1.14.0
>
> Attachments: Drill Framework Enhancements.pdf, Flat Parquet Scanner 
> Enhancements Presentation.pdf
>
>
> This task is to analyze the Flat Parquet Reader logic looking for performance 
> improvements opportunities.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (DRILL-5848) Implement Parquet Columnar Processing & Use Bulk APIs for processing

2018-05-22 Thread salim achouche (JIRA)


 [ 
https://issues.apache.org/jira/browse/DRILL-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

salim achouche resolved DRILL-5848.
---
   Resolution: Fixed
Fix Version/s: 1.14.0

> Implement Parquet Columnar Processing & Use Bulk APIs for processing
> 
>
> Key: DRILL-5848
> URL: https://issues.apache.org/jira/browse/DRILL-5848
> Project: Apache Drill
>  Issue Type: Sub-task
>  Components: Storage - Parquet
>Affects Versions: 1.11.0
>Reporter: salim achouche
>Assignee: salim achouche
>Priority: Major
> Fix For: 1.14.0
>
>
> * Change Flat Parquet Reader processing from row based to columnar
> * Use Bulk APIs during the parsing and data loading phase



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (DRILL-6435) MappingSet is stateful, so it can't be shared between threads

2018-05-22 Thread Vlad Rozov (JIRA)

Vlad Rozov created DRILL-6435:
-

 Summary: MappingSet is stateful, so it can't be shared between 
threads
 Key: DRILL-6435
 URL: https://issues.apache.org/jira/browse/DRILL-6435
 Project: Apache Drill
  Issue Type: Bug
Reporter: Vlad Rozov
Assignee: Vlad Rozov


There are several instances where static {{MappingSet}} instances are used (for 
example {{NestedLoopJoinBatch}} and {{BaseSortWrapper}}). This causes instance 
reuse across threads when queries are executed concurrently. As {{MappingSet}} 
is a stateful class with visitor design pattern, such reuse causes invalid 
state.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (DRILL-6441) IllegalStateException: Allocator[ROOT] closed with outstanding child allocators.

[jira] [Created] (DRILL-6440) Fix ignored unit tests in unnest

Re: help for native drive for .NET

[jira] [Created] (DRILL-6439) Sometimes Travis Times Out

[jira] [Created] (DRILL-6438) TestLocalExchange Generates a lot of unnecessary messages that pollute test logs.

Re: Delete spurious branches from Apache

[jira] [Created] (DRILL-6437) Travis Fails Because Logs Are Flooded.

[jira] [Created] (DRILL-6436) Store context and name in AbstractStoragePlugin instead of replicating fields in each StoragePlugin

Re: Delete spurious branches from Apache

[jira] [Resolved] (DRILL-5847) Flat Parquet Reader Performance Analysis

[jira] [Resolved] (DRILL-5848) Implement Parquet Columnar Processing & Use Bulk APIs for processing

[jira] [Created] (DRILL-6435) MappingSet is stateful, so it can't be shared between threads

12 matches

Site Navigation

Mail list logo

Footer information