[jira] [Resolved] (DRILL-2976) Set default of extended JSON support for output to false until issues are resolved

[Jason Altekruse (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-2976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Altekruse resolved DRILL-2976.

Resolution: Fixed

Fixed in 4c8ccedf387a8e3539f8eaabdcbc7945655d11e9 and 
4199e6bb37804bdf1925f050aaa60fd263b00f98

> Set default of extended JSON support for output to false until issues are 
> resolved
> --
>
> Key: DRILL-2976
> URL: https://issues.apache.org/jira/browse/DRILL-2976
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - JSON
>Reporter: Jacques Nadeau
>Assignee: Jason Altekruse
> Fix For: 1.0.0
>
> Attachments: 2976-part2.patch, DRILL-2976.1.patch.txt
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-3052) canceling a fragment executor before it starts running will cause the Foreman to wait indefinitely for a terminal message from that fragment

[Deneche A. Hakim (JIRA)]

Deneche A. Hakim created DRILL-3052:
---

 Summary: canceling a fragment executor before it starts running 
will cause the Foreman to wait indefinitely for a terminal message from that 
fragment
 Key: DRILL-3052
 URL: https://issues.apache.org/jira/browse/DRILL-3052
 Project: Apache Drill
  Issue Type: Bug
Reporter: Deneche A. Hakim
Assignee: Deneche A. Hakim
 Fix For: 1.0.0


When a Foreman receives a cancellation from the client it will cancel all it's 
fragments and go into a CANCELLATION_REQUEST state. Each fragment will cancel 
it's work, close it's resources and report a terminal state to the Foreman 
(either CANCELLED or FAILED). The Foreman will wait for all fragments to report 
before sending a terminal message to the client.
If a fragment is cancelled before it started running (similar to DRILL-2878) it 
will close it's resources but it will never send a terminal state to the 
foreman which will cause the foreman to wait indefinitely.

TestDrillbitResilience.cancelBeforeAnyResultsArrive() exposes this problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 34072: DRILL-2998: Implement heartbeat in C++ client

[Parth Chandra]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34072/
---

(Updated May 13, 2015, 1:38 a.m.)


Review request for drill, Hanifi Gunes and Norris Lee.


Changes
---

Added TCP_NODELAY to flush hearbeat messages immediately
Started heartbeat listener thread as soon as the connection is made
After sending a heartbeat, submit an async_read iff there is not already 
another async read in progress (critical!)
Signal the wait for results condition variable if there are no more query 
results or heartbeat responses (PONG responses) pending. A waiting client app 
(like querySubmitter) can then exit if it want to.


Repository: drill-git


Description
---

DRILL-2998: Implement heartbeat in C++ client.

As a result of adding a heartbeat mechanism, the client is always listening for 
a response from the hearbeat (i.e always has a request pending). The wait for 
results call therefore now returns when the number of query results pending is 
zero. Patch also includes a fix for a double delete caused by copying a pointer 
to the error object in the broadcastError call.
The patch does not address all cases of detecting when a connection to the 
server is gone (this will be addressed in a subsequent fix).


Diffs (updated)
-

  contrib/native/client/example/querySubmitter.cpp 
85e89e0c2429563c411f1e6a8c32f8c8fccf5f03 
  contrib/native/client/src/clientlib/drillClient.cpp 
7162f63d16d4107f555c2ed3f66f7fc79e22ee0c 
  contrib/native/client/src/clientlib/drillClientImpl.hpp 
04d59c763cc9b160ebdba83579863c4856acd701 
  contrib/native/client/src/clientlib/drillClientImpl.cpp 
eca0e75167620194c848ec38c457c496514653c6 
  contrib/native/client/src/include/drill/common.hpp 
2fa09545c6283faa6ff8aa0669ee4cde488088dc 
  contrib/native/client/src/include/drill/drillClient.hpp 
d7bf33c076f37cd244ee8cd604868d9710e450f4 
  contrib/native/client/src/protobuf/BitControl.pb.h 
865d3772843d2316e7a5f34fad42621e6360f14d 
  contrib/native/client/src/protobuf/BitControl.pb.cc 
827f708150df80908e9437a23ec4f915cb5efd7a 
  contrib/native/client/src/protobuf/GeneralRPC.pb.h 
49f4bf7f7e80939066037183c96d83f140bd7a9e 
  contrib/native/client/src/protobuf/GeneralRPC.pb.cc 
0ebb3a97c20dc1eabf3ebfc62b649be2fad12443 
  contrib/native/client/src/protobuf/User.pb.h 
c7deac38a025507bf4da16bb20d0ebce4d84571a 
  contrib/native/client/src/protobuf/User.pb.cc 
59f215795374e6f22316fe6fa5db588dfe770e9b 
  contrib/native/client/src/protobuf/UserBitShared.pb.h 
e2f5fd0faeddc5968f0cdafd2200f46246210d31 
  contrib/native/client/src/protobuf/UserBitShared.pb.cc 
b07ecda41b5159458c1d3f7d215afc6835ccf809 

Diff: https://reviews.apache.org/r/34072/diff/


Testing
---

Tested with broken connections on Mac. Linix and Win64


Thanks,

Parth Chandra

Re: Things are looking pretty good, let's do a 1.0 vote soon

[Ted Dunning]

This is huge!  (Premature) congratulations to the team.

Real congrats on account when it gets out the door.

Pom poms all around.



On Tue, May 12, 2015 at 9:56 PM, Jacques Nadeau  wrote:

> It looks like we've gotten a number of stability and performance
> improvements into the codebase since 0.9.  I'd like to suggest we start a
> 1.0 vote soon.  I'm happy to be release manager again.  If everybody thinks
> this sounds good, I'll try to cut a release candidate in the next day or
> two.
>
> thanks,
> Jacques
>

[jira] [Created] (DRILL-3051) Integer overflow in query time runnable

[Steven Phillips (JIRA)]

Steven Phillips created DRILL-3051:
--

 Summary: Integer overflow in query time runnable
 Key: DRILL-3051
 URL: https://issues.apache.org/jira/browse/DRILL-3051
 Project: Apache Drill
  Issue Type: Bug
Reporter: Steven Phillips


This can cause the timeout to become negative. Causes query to fail.

Only see this when querying a large number of files (e.g. ~150K)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-3049) Increase sort spooling threshold

[Steven Phillips (JIRA)]

Steven Phillips created DRILL-3049:
--

 Summary: Increase sort spooling threshold
 Key: DRILL-3049
 URL: https://issues.apache.org/jira/browse/DRILL-3049
 Project: Apache Drill
  Issue Type: Bug
Reporter: Steven Phillips






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-3050) Increase query context max memory

[Steven Phillips (JIRA)]

Steven Phillips created DRILL-3050:
--

 Summary: Increase query context max memory
 Key: DRILL-3050
 URL: https://issues.apache.org/jira/browse/DRILL-3050
 Project: Apache Drill
  Issue Type: Bug
Reporter: Steven Phillips






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-3048) Disable assertions by default

[Steven Phillips (JIRA)]

Steven Phillips created DRILL-3048:
--

 Summary: Disable assertions by default
 Key: DRILL-3048
 URL: https://issues.apache.org/jira/browse/DRILL-3048
 Project: Apache Drill
  Issue Type: Bug
Reporter: Steven Phillips






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-3047) Command failed while establishing connection

[Rahul Challapalli (JIRA)]

Rahul Challapalli created DRILL-3047:


 Summary: Command failed while establishing connection
 Key: DRILL-3047
 URL: https://issues.apache.org/jira/browse/DRILL-3047
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow, Execution - RPC
Reporter: Rahul Challapalli
Assignee: Chris Westin


git.commit.id.abbrev=d10769f

I believe the sequence of steps do not matter, but below is what I did
{code}
1. Started the drillbits 
2. Start sqlline

0: jdbc:drill:schema=dfs_eea> use dfs.drillTestDir;
+++
| ok |  summary   |
+++
| true   | Default schema changed to [dfs.drillTestDir] |
+++
1 row selected (1.626 seconds)
0: jdbc:drill:schema=dfs_eea> use dfs.flatten_operators_10rows;
+++
| ok |  summary   |
+++
| true   | Default schema changed to [dfs.flatten_operators_10rows] |
+++
1 row selected (0.254 seconds)
0: jdbc:drill:schema=dfs_eea> select * from sys.memory;
Error: SYSTEM ERROR: org.apache.drill.exec.rpc.RpcException: Command failed 
while establishing connection.  Failure type CONNECTION.


[Error Id: 093dcbb7-f352-438f-937e-cb6017427434 on qa-node191.qa.lab:31010] 
(state=,code=0)
{code}

Now I restarted sqlline and things seem to be working fine. 





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 34037: Patch for DRILL-2936

[Steven Phillips]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34037/
---

(Updated May 13, 2015, 12:34 a.m.)


Review request for drill.


Bugs: DRILL-2936
https://issues.apache.org/jira/browse/DRILL-2936


Repository: drill-git


Description
---

DRILL-2936: Use SpoolingRawBatchBuffer for HashToMergeExchange In order to 
avoid deadlocks

Refactored common code in UnlimitedRawBatchBuffer and SpoolingRawBatchBuffer
 into BaseRawBatchBuffer

Removed reflection-based construction of RawBatchBuffer. Now use choose 
implementation
 based on plan

Updated SpoolingRawBatchBuffer to use a separate thread for spooling


Diffs (updated)
-

  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractReceiver.java
 f01d025e8d7a9e3c4fe907a1c134c951b258dc9b 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/Receiver.java 
04d6d7eea28f6ada994384a6edbace2e1bf3d863 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/BroadcastExchange.java
 a37f638bc0fe71f61eee28e05777f636d21a0822 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/HashToMergeExchange.java
 f45ace92357dd4ad3f787a17e32f6ad755a0794c 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/HashToRandomExchange.java
 52d79c22603343f8dfe66bc08817e7486f4198c5 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/MergingReceiverPOP.java
 9416814375a39419c25a3465af9dac59c179f0f3 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/OrderedPartitionExchange.java
 c8dbc225d148deee791c96ff8f602d59f8104daf 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/SingleMergeExchange.java
 c812325807bdec70f1af7c1373ca1ff40e7ac29f 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/UnionExchange.java
 b7b7835282f529e68256ea014b4da4ea9d269b87 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/UnorderedDeMuxExchange.java
 0bc6678eddc9faa4d6b09ec378adb7712cde3960 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/UnorderedMuxExchange.java
 3028ee3ed2bcb778fe41e393af7ee1f0198d1cec 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/UnorderedReceiver.java
 e741dd422543a63ba9d7dcea54f822f03870be48 
  
exec/java-exec/src/main/java/org/apache/drill/exec/record/RawFragmentBatch.java 
edd79ac818c490a344c929c7e40a7502ef93300e 
  
exec/java-exec/src/main/java/org/apache/drill/exec/store/LocalSyncableFileSystem.java
 b88cc28d2f89dcf0004809ecedcef06e5ab187d0 
  
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/AbstractDataCollector.java
 ed163146d95ac0d5a12bf246765da373668fec84 
  
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/BaseRawBatchBuffer.java
 PRE-CREATION 
  
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/RawBatchBuffer.java
 8646a72e4789a038b331509e8cdef6af20d77851 
  
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/SpoolingRawBatchBuffer.java
 07a3505f105ad01c392d834a8df2e2075c676d53 
  
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/UnlimitedRawBatchBuffer.java
 47501c7629d7b50741cf9413977f5d8cfcc6 
  exec/java-exec/src/main/resources/drill-module.conf 
d98b97a7cfd9c0667d16832b35a44368562346f6 
  
exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/TestSpoolingBuffer.java
 dcea9bbecf1b7f068de61bfe897586412cfabcc0 
  
exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/TestUnlimitedBatchBuffer.java
 b8336e9779d45251081128ff1450adc4a2f38576 

Diff: https://reviews.apache.org/r/34037/diff/


Testing
---


Thanks,

Steven Phillips

Re: Review Request 34120: DRILL-2755: (part2) Use and handle InterruptedException during query processing

[Venki Korukanti]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34120/
---

(Updated May 12, 2015, 5:06 p.m.)


Review request for drill and Jacques Nadeau.


Changes
---

Addressed review comment.

Added timeout for Future.get() when waiting for connection.


Repository: drill-git


Description
---

Initial patch [1] reports interrupts as failures in logs. This patch addresses 
the issues to report interrupt to RpcOutcomeListener. Handle the interrupt in 
RpcOutComeListener depending upon the implementation. Specific details of the 
handling are here [2]

[1] 
https://github.com/apache/drill/commit/3a294abcc51148e0e79096af5e6d3c45b7c19a9d
[2] Goto sheet "RpcOutcomeListener Impl" in 
https://docs.google.com/spreadsheets/d/1phC5f7E6jn1WN-wXVdUllbIQ2k7K2ZftZzDdhDNLgtg/edit?usp=sharing


Diffs (updated)
-

  exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 1a10aa2 
  
exec/java-exec/src/main/java/org/apache/drill/exec/ops/AccountingDataTunnel.java
 2bcfdbc 
  exec/java-exec/src/main/java/org/apache/drill/exec/ops/Consumer.java 9b8ba38 
  exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java 
cf4e9bb 
  exec/java-exec/src/main/java/org/apache/drill/exec/ops/StatusHandler.java 
79fc0b0 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/SingleSenderCreator.java
 fe6239e 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java
 611052b 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionerDecorator.java
 c355070 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java
 e40fe54 
  
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/BaseRpcOutcomeListener.java
 9b071ad 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/BasicClient.java 
d551173 
  
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/DrillRpcFutureImpl.java 
19d9c30 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/FutureBitCommand.java 
6c7bf3e 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/ListeningCommand.java 
e32ca8a 
  
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/ReconnectingConnection.java
 f0787a5 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/RemoteConnection.java 
2ee9263 
  
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/RpcOutcomeListener.java 
7d7c860 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataTunnel.java 
ed31bed 
  
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/QueryResultHandler.java
 143d104 
  
exec/java-exec/src/main/java/org/apache/drill/exec/testing/NoOpControlsInjector.java
 bb13d1f 
  exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 
bf62ccb 
  
exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/QueryManager.java
 84a38a6 
  
exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentExecutor.java
 d96e6d6 
  exec/java-exec/src/main/resources/drill-module.conf d98b97a 
  exec/java-exec/src/test/java/org/apache/drill/exec/ZookeeperHelper.java 
7fcf4cb 
  exec/java-exec/src/test/java/org/apache/drill/exec/server/TestBitRpc.java 
3749716 
  
exec/java-exec/src/test/java/org/apache/drill/exec/server/TestDrillbitResilience.java
 d72d498 

Diff: https://reviews.apache.org/r/34120/diff/


Testing
---

Added unittest. Had to disable the unittest because of we leak memory when 
cancelling the query which causes unittests to fail.

Test suites pass


Thanks,

Venki Korukanti

[jira] [Created] (DRILL-3046) Memory Leak after cancelling a query

[Rahul Challapalli (JIRA)]

Rahul Challapalli created DRILL-3046:


 Summary: Memory Leak after cancelling a query
 Key: DRILL-3046
 URL: https://issues.apache.org/jira/browse/DRILL-3046
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Affects Versions: 1.0.0
Reporter: Rahul Challapalli
Assignee: Chris Westin
Priority: Critical


git.commit.id.abbrev=d10769f

There seems to be a memory leak after cancelling the query immediately after it 
is started
{code}
0: jdbc:drill:schema=dfs_eea> select * from sys.memory;
+++--+++++
|  hostname  | user_port  | heap_current |  heap_max  | direct_current | 
jvm_direct_current | direct_max |
+++--+++++
| qa-node191.qa.lab | 31010  | 171717928| 4151836672 | 11799087   | 
100676662  | 34359738368 |
| qa-node190.qa.lab | 31010  | 286131528| 4151836672 | 7750365| 
50345014   | 34359738368 |
+++--+++++
2 rows selected (1.022 seconds)
0: jdbc:drill:schema=dfs_eea> select * from `memleak.json`;
Error: SQL statement execution canceled; resultSet closed. (state=,code=0)
0: jdbc:drill:schema=dfs_eea> select * from sys.memory;
+++--+++++
|  hostname  | user_port  | heap_current |  heap_max  | direct_current | 
jvm_direct_current | direct_max |
+++--+++++
| qa-node190.qa.lab | 31010  | 293009856| 4151836672 | 7750365| 
83899446   | 34359738368 |
| qa-node191.qa.lab | 31010  | 633074864| 4151836672 | 4307525428 | 
4479530038 | 34359738368 |
+++--+++++
2 rows selected (0.775 seconds)
{code}






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-3045) Drill is leaking memory during partition pruning if directory tree has lots of files

[Victoria Markman (JIRA)]

Victoria Markman created DRILL-3045:
---

 Summary: Drill is leaking memory during partition pruning if 
directory tree has lots of files
 Key: DRILL-3045
 URL: https://issues.apache.org/jira/browse/DRILL-3045
 Project: Apache Drill
  Issue Type: Bug
  Components: Query Planning & Optimization
Affects Versions: 1.0.0
Reporter: Victoria Markman
Assignee: Jacques Nadeau


See MD-234



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-3044) Very deep record batch fetching stack for single table query (TestTpchLimit0.tpch01)

[Chris Westin (JIRA)]

Chris Westin created DRILL-3044:
---

 Summary: Very deep record batch fetching stack for single table 
query (TestTpchLimit0.tpch01)
 Key: DRILL-3044
 URL: https://issues.apache.org/jira/browse/DRILL-3044
 Project: Apache Drill
  Issue Type: Bug
  Components: Query Planning & Optimization
Affects Versions: 0.9.0
Reporter: Chris Westin
Assignee: Jinfeng Ni


I ran TestTpchLimit0 in a constrained memory environment while hunting for a 
memory leak.

Here are the startup parameters (from Eclipse's test launch configuration):
-Xms512m
-Xmx3g
-Ddrill.exec.http.enabled=false
-Ddrill.exec.sys.store.provider.local.write=false
-Dorg.apache.drill.exec.server.Drillbit.system_options="org.apache.drill.exec.compile.ClassTransformer.scalar_replacement=on"
-XX:MaxPermSize=256M -XX:MaxDirectMemorySize=3072M
-XX:+CMSClassUnloadingEnabled -ea
-Ddrill.exec.memory.top.max=67108864

Except for the last value, these were taken from the root pom.xml; the last 
value constrains the amount of direct memory used to 64M. (We're looking for 
leaks that happen when queries fail to allocate memory and have to be cancelled 
and aren't cleaned up properly).

I find that there is indeed a leak for tpch01 when the fragment is cleaned up. 
tpch01 looks like this:

select
  l_returnflag,
  l_linestatus,
  sum(l_quantity) as sum_qty,
  sum(l_extendedprice) as sum_base_price,
  sum(l_extendedprice * (1 - l_discount)) as sum_disc_price,
  sum(l_extendedprice * (1 - l_discount) * (1 + l_tax)) as sum_charge,
  avg(l_quantity) as avg_qty,
  avg(l_extendedprice) as avg_price,
  avg(l_discount) as avg_disc,
  count(*) as count_order
from
  cp.`tpch/lineitem.parquet`
where
  l_shipdate <= date '1998-12-01' - interval '120' day (3)
group by
  l_returnflag,
  l_linestatus

order by
  l_returnflag,
  l_linestatus;

Basically a single table query with a group and sort.
But in the trace file, this is the stack at the time of the creation of the 
leaked allocator:
org.apache.drill.exec.ops.FragmentContext.getNewChildAllocator:302
org.apache.drill.exec.ops.OperatorContextImpl.:43
org.apache.drill.exec.ops.FragmentContext.newOperatorContext:366
org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch:70
org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch:1
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch:140
org.apache.drill.exec.physical.impl.ImplCreator.getChildren:163
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch:121
org.apache.drill.exec.physical.impl.ImplCreator.getChildren:163
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch:121
org.apache.drill.exec.physical.impl.ImplCreator.getChildren:163
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch:121
org.apache.drill.exec.physical.impl.ImplCreator.getChildren:163
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch:121
org.apache.drill.exec.physical.impl.ImplCreator.getChildren:163
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch:121
org.apache.drill.exec.physical.impl.ImplCreator.getChildren:163
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch:121
org.apache.drill.exec.physical.impl.ImplCreator.getChildren:163
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch:121
org.apache.drill.exec.physical.impl.ImplCreator.getChildren:163
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch:121
org.apache.drill.exec.physical.impl.ImplCreator.getChildren:163
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch:121
org.apache.drill.exec.physical.impl.ImplCreator.getChildren:163
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch:121
org.apache.drill.exec.physical.impl.ImplCreator.getChildren:163
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch:121
org.apache.drill.exec.physical.impl.ImplCreator.getChildren:163
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch:121
org.apache.drill.exec.physical.impl.ImplCreator.getChildren:163
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch:121
org.apache.drill.exec.physical.impl.ImplCreator.getChildren:163
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch:121
org.apache.drill.exec.physical.impl.ImplCreator.getChildren:163
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch:121
org.apache.drill.exec.physical.impl.ImplCreator.getChildren:163
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch:121
org.apache.drill.exec.physical.impl.ImplCreator.getChildren:163
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch:121
org.apache.drill.exec.physical.impl.ImplCreator.getChildren:163
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch:121

[jira] [Created] (DRILL-3043) Select over corrupt parquet file must return a better message

[Khurram Faraaz (JIRA)]

Khurram Faraaz created DRILL-3043:
-

 Summary: Select over corrupt parquet file must return a better 
message
 Key: DRILL-3043
 URL: https://issues.apache.org/jira/browse/DRILL-3043
 Project: Apache Drill
  Issue Type: Bug
  Components: Storage - Parquet
Affects Versions: 1.0.0
 Environment: 21cc578b6b8c8f3ca1ebffd3dbb92e35d68bc726 | MD-272: MapRDB 
plugin needs another update on 2015-05-08 
Reporter: Khurram Faraaz
Assignee: Steven Phillips
Priority: Minor


Select over a corrupt parquet file does not return proper error message to 
user. Looks like the correct message is getting lost.

The below CTAS was used to create the original parquet file.

{code}
0: jdbc:drill:schema=dfs.tmp> create table tblJsnToPrq_01 as select key from 
`oneMlnDblData.json`;
++---+
|  Fragment  | Number of records written |
++---+
| 0_0| 99|
++---+
1 row selected (2.381 seconds)
0: jdbc:drill:schema=dfs.tmp> select count(key) from  tblJsnToPrq_01;
++
|   EXPR$0   |
++
| 99 |
++
1 row selected (0.232 seconds)

{code}

{code}
0: jdbc:drill:schema=dfs.tmp> select key from `corrupt_Parquet.parquet`;
Error: SYSTEM ERROR: java.io.IOException: FAILED_TO_UNCOMPRESS(5)

Fragment 0:0

[Error Id: 48072728-574f-450f-8370-32dbbb534da6 on centos-01.qa.lab:31010] 
(state=,code=0)
{code}

The way the corrupt parquet file was created from the original parquet file. 
The first and last 100 bytes were written to the new (corrupt) parquet file

{code}
[root@centos-01 mal_formed_parquet]# head -c 100 0_0_0.parquet >> 
corrupt_Parquet.parquet
[root@centos-01 mal_formed_parquet]# tail -c 100 0_0_0.parquet >> 
corrupt_Parquet.parquet
{code}

Stack trace from drillbit.log

{code}
2015-05-12 23:05:45,285 [2aad78b6-09c6-0e17-accf-ab279fab39dc:frag:0:0] ERROR 
o.a.d.c.e.DrillRuntimeException - SYSTEM ERROR: java.io.IOException: 
FAILED_TO_UNCOMPRESS(5)

Fragment 0:0

[Error Id: 48072728-574f-450f-8370-32dbbb534da6 on centos-01.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 
java.io.IOException: FAILED_TO_UNCOMPRESS(5)

Fragment 0:0

[Error Id: 48072728-574f-450f-8370-32dbbb534da6 on centos-01.qa.lab:31010]
at 
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:495)
 ~[drill-common-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
at 
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:286)
 [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
at 
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:254)
 [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
at 
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) 
[drill-common-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
[na:1.7.0_45]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Error in 
parquet record reader.
Message:
Hadoop path: /tmp/corrupt_Parquet.parquet
Total records read: 0
Mock records read: 0
Records to read: 32768
Row group index: 0
Records in row group: 99
Parquet Metadata: ParquetMetaData{FileMetaData{schema: message root {
  optional double key;
}
, metadata: {}}, blocks: [BlockMetaData{99, 8000439 [ColumnMetaData{SNAPPY 
[key] DOUBLE  [PLAIN, BIT_PACKED, RLE], 4}]}]}
at 
org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.handleAndRaise(ParquetRecordReader.java:339)
 ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
at 
org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:441)
 ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
at 
org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:175) 
~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
at 
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
 ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
at 
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:83) 
~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
at 
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:80)
 ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
at 
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:73

Re: Review Request 34120: DRILL-2755: (part2) Use and handle InterruptedException during query processing

[Jacques Nadeau]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34120/#review83504
---



exec/java-exec/src/main/java/org/apache/drill/exec/rpc/ReconnectingConnection.java


shouldn't this have timeout?


- Jacques Nadeau


On May 12, 2015, 9:02 p.m., Venki Korukanti wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34120/
> ---
> 
> (Updated May 12, 2015, 9:02 p.m.)
> 
> 
> Review request for drill and Jacques Nadeau.
> 
> 
> Repository: drill-git
> 
> 
> Description
> ---
> 
> Initial patch [1] reports interrupts as failures in logs. This patch 
> addresses the issues to report interrupt to RpcOutcomeListener. Handle the 
> interrupt in RpcOutComeListener depending upon the implementation. Specific 
> details of the handling are here [2]
> 
> [1] 
> https://github.com/apache/drill/commit/3a294abcc51148e0e79096af5e6d3c45b7c19a9d
> [2] Goto sheet "RpcOutcomeListener Impl" in 
> https://docs.google.com/spreadsheets/d/1phC5f7E6jn1WN-wXVdUllbIQ2k7K2ZftZzDdhDNLgtg/edit?usp=sharing
> 
> 
> Diffs
> -
> 
>   exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 
> 1a10aa2 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/ops/AccountingDataTunnel.java
>  2bcfdbc 
>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/Consumer.java 
> 9b8ba38 
>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java 
> cf4e9bb 
>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/StatusHandler.java 
> 79fc0b0 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/SingleSenderCreator.java
>  fe6239e 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java
>  b28b7b0 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionerDecorator.java
>  c355070 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java
>  e40fe54 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/BaseRpcOutcomeListener.java
>  9b071ad 
>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/BasicClient.java 
> d551173 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/DrillRpcFutureImpl.java
>  19d9c30 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/FutureBitCommand.java 
> 6c7bf3e 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/ListeningCommand.java 
> e32ca8a 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/ReconnectingConnection.java
>  f0787a5 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/RemoteConnection.java 
> 2ee9263 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/RpcOutcomeListener.java
>  7d7c860 
>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataTunnel.java 
> ed31bed 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/QueryResultHandler.java
>  143d104 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/testing/NoOpControlsInjector.java
>  bb13d1f 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 
> bf62ccb 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/QueryManager.java
>  84a38a6 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentExecutor.java
>  d96e6d6 
>   exec/java-exec/src/main/resources/drill-module.conf d98b97a 
>   exec/java-exec/src/test/java/org/apache/drill/exec/ZookeeperHelper.java 
> 7fcf4cb 
>   exec/java-exec/src/test/java/org/apache/drill/exec/server/TestBitRpc.java 
> 3749716 
>   
> exec/java-exec/src/test/java/org/apache/drill/exec/server/TestDrillbitResilience.java
>  d72d498 
> 
> Diff: https://reviews.apache.org/r/34120/diff/
> 
> 
> Testing
> ---
> 
> Added unittest. Had to disable the unittest because of we leak memory when 
> cancelling the query which causes unittests to fail.
> 
> Test suites pass
> 
> 
> Thanks,
> 
> Venki Korukanti
> 
>

[jira] [Created] (DRILL-3042) where fails on fields converted to JSON

[Will Ochandarena (JIRA)]

Will Ochandarena created DRILL-3042:
---

 Summary: where fails on fields converted to JSON
 Key: DRILL-3042
 URL: https://issues.apache.org/jira/browse/DRILL-3042
 Project: Apache Drill
  Issue Type: Bug
  Components: Query Planning & Optimization
Affects Versions: 0.9.0
Reporter: Will Ochandarena
Assignee: Jinfeng Ni


See attached sample log from Google App Engine.  When I create a view of that 
data that extracts an inner field and converts it to JSON -
--
create or replace view alltypes as select t3.line.status as status from (select 
convert_from(line,'JSON') as line from (select t1.tp.logMessage as line 
from(select flatten(t.protoPayload.line) as tp from 
dfs.`/Users/wochandarena/Downloads/WHEREfail.json` t) as t1 where 
t1.tp.logMessage LIKE '{%') as t2) as t3;
--

Then query the view with a where clause, the error below is seen.
--
0: jdbc:drill:zk=local> select * from alltypes;
++
|   status   |
++
| Status |
++
1 row selected (0.345 seconds)
0: jdbc:drill:zk=local> select * from alltypes where status=Status;
Query failed: SYSTEM ERROR: Unexpected exception during fragment 
initialization: null


[a1790b40-0055-4793-984f-b37ce72cb92f on 172.30.1.6:31010]
Error: exception while executing query: Failure while executing query. 
(state=,code=0)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-3041) Impersonation-user can create view against file that user doesn't have read access

[Krystal (JIRA)]

Krystal created DRILL-3041:
--

 Summary: Impersonation-user can create view against file that user 
doesn't have read access 
 Key: DRILL-3041
 URL: https://issues.apache.org/jira/browse/DRILL-3041
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - RPC
Affects Versions: 1.0.0
Reporter: Krystal
Assignee: Jacques Nadeau


git.commit.id.abbrev=d10769f

I have a file that has the following permission:
-rwx--   3 qa2 users  63078 2015-01-30 21:19 
/drill/testdata/csv/voter.csv

The directory right above the file has the following permission:
drwxr-xr-x   - qa2  users  3 2015-05-12 14:22 /drill/testdata/csv

Logged into sqlline as a different user and attempted to create a view:
0: jdbc:drill:schema=dfs.root> CREATE VIEW `dfs.qa1`.`test_v4` AS SELECT 
columns[0] as column_0, columns[1] as column_1, columns[2] as column_2, 
columns[3] as column_3, columns[4] as column_4, columns[5] as column_5, 
columns[6] as column_6 FROM `dfs`.`default`.`drill/testdata/csv/voter.csv` 
LIMIT 100;

The view got created successfully. However if I tried to read from the view, I 
can't because of the lack of permission to the voter.csv table:
0: jdbc:drill:schema=dfs.root> select * from `dfs.qa1`.`test_v4`;
Error: SYSTEM ERROR: org.apache.hadoop.security.AccessControlException: Open 
failed for file: /drill/testdata/csv/voter.csv, error: Permission denied (13)

Currently drill only check if the folder contains correct permission and not at 
the file level when creating views.  It seems odd that a user is allowed to 
create the view then not being able to access it afterwards.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Things are looking pretty good, let's do a 1.0 vote soon

[Alexander Zarei]

Great news!

On Tue, May 12, 2015 at 1:56 PM, Jacques Nadeau  wrote:

> It looks like we've gotten a number of stability and performance
> improvements into the codebase since 0.9.  I'd like to suggest we start a
> 1.0 vote soon.  I'm happy to be release manager again.  If everybody thinks
> this sounds good, I'll try to cut a release candidate in the next day or
> two.
>
> thanks,
> Jacques
>

Re: Things are looking pretty good, let's do a 1.0 vote soon

[Tomer Shiran]

Sounds good

On Tue, May 12, 2015 at 1:56 PM, Jacques Nadeau  wrote:

> It looks like we've gotten a number of stability and performance
> improvements into the codebase since 0.9.  I'd like to suggest we start a
> 1.0 vote soon.  I'm happy to be release manager again.  If everybody thinks
> this sounds good, I'll try to cut a release candidate in the next day or
> two.
>
> thanks,
> Jacques
>

[jira] [Created] (DRILL-3040) Accountor drill.exec.memory.enable_frag_limit not defaulted normally

[Daniel Barclay (Drill) (JIRA)]

Daniel Barclay (Drill) created DRILL-3040:
-

 Summary: Accountor drill.exec.memory.enable_frag_limit not 
defaulted normally
 Key: DRILL-3040
 URL: https://issues.apache.org/jira/browse/DRILL-3040
 Project: Apache Drill
  Issue Type: Bug
Reporter: Daniel Barclay (Drill)


Defaulting of drill.exec.memory.enable_frag_limit is implemented "manually" in 
Accountor (by catching ConfigException) rather than by using the normal 
configuration file hierarchy (being defined in the base file).





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 34120: DRILL-2755: (part2) Use and handle InterruptedException during query processing

[Venki Korukanti]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34120/
---

(Updated May 12, 2015, 2:02 p.m.)


Review request for drill and Jacques Nadeau.


Changes
---

Enabling the unittest. After applying Chirs's patch for memory leaks 
(DRILL-3033), I don't see any more leaks when query is cancelled.


Repository: drill-git


Description
---

Initial patch [1] reports interrupts as failures in logs. This patch addresses 
the issues to report interrupt to RpcOutcomeListener. Handle the interrupt in 
RpcOutComeListener depending upon the implementation. Specific details of the 
handling are here [2]

[1] 
https://github.com/apache/drill/commit/3a294abcc51148e0e79096af5e6d3c45b7c19a9d
[2] Goto sheet "RpcOutcomeListener Impl" in 
https://docs.google.com/spreadsheets/d/1phC5f7E6jn1WN-wXVdUllbIQ2k7K2ZftZzDdhDNLgtg/edit?usp=sharing


Diffs (updated)
-

  exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 1a10aa2 
  
exec/java-exec/src/main/java/org/apache/drill/exec/ops/AccountingDataTunnel.java
 2bcfdbc 
  exec/java-exec/src/main/java/org/apache/drill/exec/ops/Consumer.java 9b8ba38 
  exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java 
cf4e9bb 
  exec/java-exec/src/main/java/org/apache/drill/exec/ops/StatusHandler.java 
79fc0b0 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/SingleSenderCreator.java
 fe6239e 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java
 b28b7b0 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionerDecorator.java
 c355070 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java
 e40fe54 
  
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/BaseRpcOutcomeListener.java
 9b071ad 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/BasicClient.java 
d551173 
  
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/DrillRpcFutureImpl.java 
19d9c30 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/FutureBitCommand.java 
6c7bf3e 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/ListeningCommand.java 
e32ca8a 
  
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/ReconnectingConnection.java
 f0787a5 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/RemoteConnection.java 
2ee9263 
  
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/RpcOutcomeListener.java 
7d7c860 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataTunnel.java 
ed31bed 
  
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/QueryResultHandler.java
 143d104 
  
exec/java-exec/src/main/java/org/apache/drill/exec/testing/NoOpControlsInjector.java
 bb13d1f 
  exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 
bf62ccb 
  
exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/QueryManager.java
 84a38a6 
  
exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentExecutor.java
 d96e6d6 
  exec/java-exec/src/main/resources/drill-module.conf d98b97a 
  exec/java-exec/src/test/java/org/apache/drill/exec/ZookeeperHelper.java 
7fcf4cb 
  exec/java-exec/src/test/java/org/apache/drill/exec/server/TestBitRpc.java 
3749716 
  
exec/java-exec/src/test/java/org/apache/drill/exec/server/TestDrillbitResilience.java
 d72d498 

Diff: https://reviews.apache.org/r/34120/diff/


Testing
---

Added unittest. Had to disable the unittest because of we leak memory when 
cancelling the query which causes unittests to fail.

Test suites pass


Thanks,

Venki Korukanti

[jira] [Created] (DRILL-3039) Trying to connect to non-existent Zookeeper seems to try indefinitely

[Daniel Barclay (Drill) (JIRA)]

Daniel Barclay (Drill) created DRILL-3039:
-

 Summary: Trying to connect to non-existent Zookeeper seems to try 
indefinitely
 Key: DRILL-3039
 URL: https://issues.apache.org/jira/browse/DRILL-3039
 Project: Apache Drill
  Issue Type: Bug
Reporter: Daniel Barclay (Drill)


Trying to make a JDBC connection using a JDBC URL of "jdbc:drill:", which 
causes defaulting to a "zk" configuration parameter value of "localhost:2181", 
when there is no Zookeeper running at that address doesn't fail quickly, but 
seems to cause trying indefinitely.  

(There's no response back to the JDBC level, and turning up logging verbosity 
shows continued activity (every second or so) in org.apache.zookeeper code).





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Things are looking pretty good, let's do a 1.0 vote soon

[Jacques Nadeau]

It looks like we've gotten a number of stability and performance
improvements into the codebase since 0.9.  I'd like to suggest we start a
1.0 vote soon.  I'm happy to be release manager again.  If everybody thinks
this sounds good, I'll try to cut a release candidate in the next day or
two.

thanks,
Jacques

Re: Review Request 34072: DRILL-2998: Implement heartbeat in C++ client

[Norris Lee]

> On May 12, 2015, 7:04 p.m., Norris Lee wrote:
> > Ran some quick tests against it. If nothing happens within the heartbeat 
> > timeout (ie. no queries ran within 15 seconds), ERR_QRY_COMMERR in 
> > handleRead gets triggered and no further queries can be run.
> 
> Parth Chandra wrote:
> The only time a ERR_QRY_COMMERR occurs is if the call to 
> m_socket.read_some has an error which is an error trying to read from the 
> server. What is the error message?

I tried it with Drill Explorer and it just hung. The drillclient logs had this 
(note the timestamps):

2015-May-12 13:04:52 : TRACE : 5660 : DrillClientImpl::processQueryResult: 0 
requests pending.
2015-May-12 13:04:52 : TRACE : 419c : Clearing state for Query Id - 
[3075293457945872364:8516570227032534528] 
2015-May-12 13:05:07 : TRACE : 5660 : DrillClientImpl:: Heartbeat timer expired.
2015-May-12 13:05:07 : TRACE : 5660 : DrillClientImpl::Heartbeat Timer -  
Expires at: 2015-May-12 20:05:07.556013 and time now is: 2015-May-12 
20:05:07.562013
2015-May-12 13:05:07 : TRACE : 5660 : Heartbeat sent.
2015-May-12 13:05:07 : TRACE : 5660 : Read blocked waiting for memory.
2015-May-12 13:05:07 : TRACE : 5660 : Started new query wait timer with 30 
seconds.
2015-May-12 13:05:07 : TRACE : 5660 : Reset Heartbeat timer.
2015-May-12 13:05:07 : TRACE : 5660 : Started new heartbeat timer with 15 
seconds.
2015-May-12 13:05:07 : DEBUG : 5660 : DrillClientImpl::getNextResult: 
async_read from the server
2015-May-12 13:05:07 : TRACE : 5660 : Started new heartbeat timer with 15 
seconds.
2015-May-12 13:05:07 : TRACE : 5660 : DrillClientImpl:: Heartbeat timer expired.
2015-May-12 13:05:07 : TRACE : 5660 : DrillClientImpl::handleRead: Handle Read 
from buffer 20606550
2015-May-12 13:05:07 : TRACE : 5660 : DrillClientImpl::handleRead: Cancel 
deadline timer.
2015-May-12 13:05:07 : TRACE : 5660 : DrillClientImpl::handleRead: 
ERR_QRY_COMMERR. Boost Communication Error: End of file
2015-May-12 13:05:07 : TRACE : 5660 : Socket shutdown
2015-May-12 13:05:07 : TRACE : 5660 : DrillClientImpl:: Heartbeat timer expired.
2015-May-12 13:05:08 : DEBUG : 419c : Sent query request. Coordination id = 1178
2015-May-12 13:05:08 : DEBUG : 419c : Sending query request. Number of pending 
requests = 1
2015-May-12 13:05:08 : TRACE : 419c : Read blocked waiting for memory.
2015-May-12 13:05:08 : TRACE : 419c : Started new query wait timer with 30 
seconds.
2015-May-12 13:05:08 : TRACE : 419c : Reset Heartbeat timer.
2015-May-12 13:05:08 : TRACE : 419c : Started new heartbeat timer with 15 
seconds.
2015-May-12 13:05:08 : DEBUG : 419c : DrillClientImpl::getNextResult: 
async_read from the server


- Norris


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34072/#review83453
---


On May 12, 2015, 4:54 a.m., Parth Chandra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34072/
> ---
> 
> (Updated May 12, 2015, 4:54 a.m.)
> 
> 
> Review request for drill, Hanifi Gunes and Norris Lee.
> 
> 
> Repository: drill-git
> 
> 
> Description
> ---
> 
> DRILL-2998: Implement heartbeat in C++ client.
> 
> As a result of adding a heartbeat mechanism, the client is always listening 
> for a response from the hearbeat (i.e always has a request pending). The wait 
> for results call therefore now returns when the number of query results 
> pending is zero. Patch also includes a fix for a double delete caused by 
> copying a pointer to the error object in the broadcastError call.
> The patch does not address all cases of detecting when a connection to the 
> server is gone (this will be addressed in a subsequent fix).
> 
> 
> Diffs
> -
> 
>   contrib/native/client/example/querySubmitter.cpp 
> 85e89e0c2429563c411f1e6a8c32f8c8fccf5f03 
>   contrib/native/client/src/clientlib/drillClient.cpp 
> 7162f63d16d4107f555c2ed3f66f7fc79e22ee0c 
>   contrib/native/client/src/clientlib/drillClientImpl.hpp 
> 04d59c763cc9b160ebdba83579863c4856acd701 
>   contrib/native/client/src/clientlib/drillClientImpl.cpp 
> eca0e75167620194c848ec38c457c496514653c6 
>   contrib/native/client/src/include/drill/common.hpp 
> 2fa09545c6283faa6ff8aa0669ee4cde488088dc 
>   contrib/native/client/src/include/drill/drillClient.hpp 
> d7bf33c076f37cd244ee8cd604868d9710e450f4 
>   contrib/native/client/src/protobuf/BitControl.pb.h 
> 865d3772843d2316e7a5f34fad42621e6360f14d 
>   contrib/native/client/src/protobuf/BitControl.pb.cc 
> 827f708150df80908e9437a23ec4f915cb5efd7a 
>   contrib/native/client/src/protobuf/GeneralRPC.pb.h 
> 49f4bf7f7e80939066037183c96d83f140bd7a9e 
>   contrib/native/client/src/protobuf/GeneralRPC.pb.cc 
> 0ebb3a97c20dc1eabf3ebfc62b649be2fad12443 
>   contrib/native/client/sr

Review Request 34120: DRILL-2755: (part2) Use and handle InterruptedException during query processing

[Venki Korukanti]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34120/
---

Review request for drill and Jacques Nadeau.


Repository: drill-git


Description
---

Initial patch [1] reports interrupts as failures in logs. This patch addresses 
the issues to report interrupt to RpcOutcomeListener. Handle the interrupt in 
RpcOutComeListener depending upon the implementation. Specific details of the 
handling are here [2]

[1] 
https://github.com/apache/drill/commit/3a294abcc51148e0e79096af5e6d3c45b7c19a9d
[2] Goto sheet "RpcOutcomeListener Impl" in 
https://docs.google.com/spreadsheets/d/1phC5f7E6jn1WN-wXVdUllbIQ2k7K2ZftZzDdhDNLgtg/edit?usp=sharing


Diffs
-

  exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 1a10aa2 
  
exec/java-exec/src/main/java/org/apache/drill/exec/ops/AccountingDataTunnel.java
 2bcfdbc 
  exec/java-exec/src/main/java/org/apache/drill/exec/ops/Consumer.java 9b8ba38 
  exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java 
cf4e9bb 
  exec/java-exec/src/main/java/org/apache/drill/exec/ops/StatusHandler.java 
79fc0b0 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/SingleSenderCreator.java
 fe6239e 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java
 b28b7b0 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java
 e40fe54 
  
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/BaseRpcOutcomeListener.java
 9b071ad 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/BasicClient.java 
d551173 
  
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/DrillRpcFutureImpl.java 
19d9c30 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/FutureBitCommand.java 
6c7bf3e 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/ListeningCommand.java 
e32ca8a 
  
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/ReconnectingConnection.java
 f0787a5 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/RemoteConnection.java 
2ee9263 
  
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/RpcOutcomeListener.java 
7d7c860 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataTunnel.java 
ed31bed 
  
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/QueryResultHandler.java
 143d104 
  
exec/java-exec/src/main/java/org/apache/drill/exec/testing/NoOpControlsInjector.java
 bb13d1f 
  exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 
bf62ccb 
  
exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/QueryManager.java
 84a38a6 
  
exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentExecutor.java
 d96e6d6 
  exec/java-exec/src/main/resources/drill-module.conf d98b97a 
  exec/java-exec/src/test/java/org/apache/drill/exec/ZookeeperHelper.java 
7fcf4cb 
  exec/java-exec/src/test/java/org/apache/drill/exec/server/TestBitRpc.java 
3749716 
  
exec/java-exec/src/test/java/org/apache/drill/exec/server/TestDrillbitResilience.java
 d72d498 

Diff: https://reviews.apache.org/r/34120/diff/


Testing
---

Added unittest. Had to disable the unittest because of we leak memory when 
cancelling the query which causes unittests to fail.

Test suites pass


Thanks,

Venki Korukanti

[jira] [Resolved] (DRILL-2865) Drillbit runs out of memory on multiple consecutive CTAS

[Jacques Nadeau (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-2865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacques Nadeau resolved DRILL-2865.
---
Resolution: Pending Closed

Resolved by a combination of malloc changes and Parquet codec changes

> Drillbit runs out of memory on multiple consecutive CTAS
> 
>
> Key: DRILL-2865
> URL: https://issues.apache.org/jira/browse/DRILL-2865
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Victoria Markman
>Assignee: Steven Phillips
> Fix For: 1.0.0
>
> Attachments: drill-2865-no-compression.log, drillbit.log, scritpts.tar
>
>
> Hardware configuration:
> - single node
> - 64GB RAM
> Drill configuration
> DRILL_MAX_DIRECT_MEMORY="8G"
> DRILL_MAX_HEAP="4G"
> `planner.enable_multiphase_agg` = false;
> `store.parquet.block-size` = 134217728;
> `planner.enable_mux_exchange` = false;
> `exec.min_hash_table_size` = 67108864;
> `planner.enable_hashagg` = true; 
> `planner.width.max_per_node` = 23;
> Aggregation query on TPCDS scale factor 1: 
> {code}
> select 
> ss_sold_date_sk , 
> ss_sold_time_sk , 
> ss_item_sk , 
> ss_customer_sk , 
> ss_cdemo_sk, 
> count(*) from store_sales
> group by 
> ss_sold_date_sk , 
> ss_sold_time_sk , 
> ss_item_sk , 
> ss_customer_sk , 
> ss_cdemo_sk
> ;
> {code}
> 1. Executing CTAS with this query and store.format = 'parquet' fails on 
> iteration #9 with this configuration consistently
> 2. Ran query by itself: 47 iterations successfully
> 3. Ran CTAS with this query and store.format = 'csv': - 30 iterations did not 
> reproduce the problem
> Attached:
>   - drillbit.log
>   - scripts.tar (contains script that reproduces OOM)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (DRILL-2074) Queries fail with OutOfMemory Exception when Hash Join & Agg are turned off

[Jacques Nadeau (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-2074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacques Nadeau resolved DRILL-2074.
---
Resolution: Pending Closed

> Queries fail with OutOfMemory Exception when Hash Join & Agg are turned off
> ---
>
> Key: DRILL-2074
> URL: https://issues.apache.org/jira/browse/DRILL-2074
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Relational Operators
>Reporter: Abhishek Girish
>Assignee: Deneche A. Hakim
> Fix For: 1.0.0
>
> Attachments: 05_par1000.q, 05_par1000_d6e54ab.logical.plan, 
> 05_par1000_d6e54ab.verbose.plan, drill-env.sh
>
>
> Query attached. 
> Hash Join and Hash Agg were turned off. And the following property was added 
> to drill-override.conf:
> sort: {
> purge.threshold : 100,
> external: {
>   batch.size : 4000,
>   spill: {
> batch.size : 4000,
> group.size : 100,
> threshold : 200,
> directories : [ "/drill_spill" ],
> fs : "maprfs:///"
>   }
> }
>   }
> Query failed with the below error message:
> Query failed: RemoteRpcException: Failure while running fragment., Unable to 
> allocate sv2 buffer after repeated attempts [ 
> faf3044a-e14a-427b-b66d-7bcd7522ead5 on drone-42:31010 ]
> [ faf3044a-e14a-427b-b66d-7bcd7522ead5 on drone-42:31010 ]
> Log Snippets:
> 2015-01-26 20:07:33,239 atsqa8c42.qa.lab 
> [2b396307-2c1e-3486-90bc-fbaf09fbeb3e:frag:15:51] ERROR 
> o.a.d.e.w.f.AbstractStatusReporter - Error 
> faf3044a-e14a-427b-b66d-7bcd7522ead5: Failure while running fragment.
> java.lang.RuntimeException: 
> org.apache.drill.exec.memory.OutOfMemoryException: Unable to allocate sv2 
> buffer after repeated attempts
> at 
> org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.innerNext(ExternalSortBatch.java:309)
>  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at 
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
>  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at 
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:99)
>  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at 
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:89)
>  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at 
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
>  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at 
> org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:96)
>  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at 
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
>  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at 
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:99)
>  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at 
> org.apache.drill.exec.physical.impl.join.JoinStatus.nextRight(JoinStatus.java:80)
>  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at 
> org.apache.drill.exec.physical.impl.join.JoinStatus.ensureInitial(JoinStatus.java:95)
>  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at 
> org.apache.drill.exec.physical.impl.join.MergeJoinBatch.innerNext(MergeJoinBatch.java:147)
>  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at 
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
>  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at 
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:99)
>  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at 
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:89)
>  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at 
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
>  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at 
> org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:96)
>  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at 
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
>  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at 
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:67) 
> ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at 
> org.a

Re: Review Request 34072: DRILL-2998: Implement heartbeat in C++ client

[Parth Chandra]

> On May 12, 2015, 7:04 p.m., Norris Lee wrote:
> > Ran some quick tests against it. If nothing happens within the heartbeat 
> > timeout (ie. no queries ran within 15 seconds), ERR_QRY_COMMERR in 
> > handleRead gets triggered and no further queries can be run.

The only time a ERR_QRY_COMMERR occurs is if the call to m_socket.read_some has 
an error which is an error trying to read from the server. What is the error 
message?


- Parth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34072/#review83453
---


On May 12, 2015, 4:54 a.m., Parth Chandra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34072/
> ---
> 
> (Updated May 12, 2015, 4:54 a.m.)
> 
> 
> Review request for drill, Hanifi Gunes and Norris Lee.
> 
> 
> Repository: drill-git
> 
> 
> Description
> ---
> 
> DRILL-2998: Implement heartbeat in C++ client.
> 
> As a result of adding a heartbeat mechanism, the client is always listening 
> for a response from the hearbeat (i.e always has a request pending). The wait 
> for results call therefore now returns when the number of query results 
> pending is zero. Patch also includes a fix for a double delete caused by 
> copying a pointer to the error object in the broadcastError call.
> The patch does not address all cases of detecting when a connection to the 
> server is gone (this will be addressed in a subsequent fix).
> 
> 
> Diffs
> -
> 
>   contrib/native/client/example/querySubmitter.cpp 
> 85e89e0c2429563c411f1e6a8c32f8c8fccf5f03 
>   contrib/native/client/src/clientlib/drillClient.cpp 
> 7162f63d16d4107f555c2ed3f66f7fc79e22ee0c 
>   contrib/native/client/src/clientlib/drillClientImpl.hpp 
> 04d59c763cc9b160ebdba83579863c4856acd701 
>   contrib/native/client/src/clientlib/drillClientImpl.cpp 
> eca0e75167620194c848ec38c457c496514653c6 
>   contrib/native/client/src/include/drill/common.hpp 
> 2fa09545c6283faa6ff8aa0669ee4cde488088dc 
>   contrib/native/client/src/include/drill/drillClient.hpp 
> d7bf33c076f37cd244ee8cd604868d9710e450f4 
>   contrib/native/client/src/protobuf/BitControl.pb.h 
> 865d3772843d2316e7a5f34fad42621e6360f14d 
>   contrib/native/client/src/protobuf/BitControl.pb.cc 
> 827f708150df80908e9437a23ec4f915cb5efd7a 
>   contrib/native/client/src/protobuf/GeneralRPC.pb.h 
> 49f4bf7f7e80939066037183c96d83f140bd7a9e 
>   contrib/native/client/src/protobuf/GeneralRPC.pb.cc 
> 0ebb3a97c20dc1eabf3ebfc62b649be2fad12443 
>   contrib/native/client/src/protobuf/User.pb.h 
> c7deac38a025507bf4da16bb20d0ebce4d84571a 
>   contrib/native/client/src/protobuf/User.pb.cc 
> 59f215795374e6f22316fe6fa5db588dfe770e9b 
>   contrib/native/client/src/protobuf/UserBitShared.pb.h 
> e2f5fd0faeddc5968f0cdafd2200f46246210d31 
>   contrib/native/client/src/protobuf/UserBitShared.pb.cc 
> b07ecda41b5159458c1d3f7d215afc6835ccf809 
> 
> Diff: https://reviews.apache.org/r/34072/diff/
> 
> 
> Testing
> ---
> 
> Tested with broken connections on Mac. Linix and Win64
> 
> 
> Thanks,
> 
> Parth Chandra
> 
>

Re: Review Request 34072: DRILL-2998: Implement heartbeat in C++ client

[Norris Lee]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34072/#review83453
---


Ran some quick tests against it. If nothing happens within the heartbeat 
timeout (ie. no queries ran within 15 seconds), ERR_QRY_COMMERR in handleRead 
gets triggered and no further queries can be run.

- Norris Lee


On May 12, 2015, 4:54 a.m., Parth Chandra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34072/
> ---
> 
> (Updated May 12, 2015, 4:54 a.m.)
> 
> 
> Review request for drill, Hanifi Gunes and Norris Lee.
> 
> 
> Repository: drill-git
> 
> 
> Description
> ---
> 
> DRILL-2998: Implement heartbeat in C++ client.
> 
> As a result of adding a heartbeat mechanism, the client is always listening 
> for a response from the hearbeat (i.e always has a request pending). The wait 
> for results call therefore now returns when the number of query results 
> pending is zero. Patch also includes a fix for a double delete caused by 
> copying a pointer to the error object in the broadcastError call.
> The patch does not address all cases of detecting when a connection to the 
> server is gone (this will be addressed in a subsequent fix).
> 
> 
> Diffs
> -
> 
>   contrib/native/client/example/querySubmitter.cpp 
> 85e89e0c2429563c411f1e6a8c32f8c8fccf5f03 
>   contrib/native/client/src/clientlib/drillClient.cpp 
> 7162f63d16d4107f555c2ed3f66f7fc79e22ee0c 
>   contrib/native/client/src/clientlib/drillClientImpl.hpp 
> 04d59c763cc9b160ebdba83579863c4856acd701 
>   contrib/native/client/src/clientlib/drillClientImpl.cpp 
> eca0e75167620194c848ec38c457c496514653c6 
>   contrib/native/client/src/include/drill/common.hpp 
> 2fa09545c6283faa6ff8aa0669ee4cde488088dc 
>   contrib/native/client/src/include/drill/drillClient.hpp 
> d7bf33c076f37cd244ee8cd604868d9710e450f4 
>   contrib/native/client/src/protobuf/BitControl.pb.h 
> 865d3772843d2316e7a5f34fad42621e6360f14d 
>   contrib/native/client/src/protobuf/BitControl.pb.cc 
> 827f708150df80908e9437a23ec4f915cb5efd7a 
>   contrib/native/client/src/protobuf/GeneralRPC.pb.h 
> 49f4bf7f7e80939066037183c96d83f140bd7a9e 
>   contrib/native/client/src/protobuf/GeneralRPC.pb.cc 
> 0ebb3a97c20dc1eabf3ebfc62b649be2fad12443 
>   contrib/native/client/src/protobuf/User.pb.h 
> c7deac38a025507bf4da16bb20d0ebce4d84571a 
>   contrib/native/client/src/protobuf/User.pb.cc 
> 59f215795374e6f22316fe6fa5db588dfe770e9b 
>   contrib/native/client/src/protobuf/UserBitShared.pb.h 
> e2f5fd0faeddc5968f0cdafd2200f46246210d31 
>   contrib/native/client/src/protobuf/UserBitShared.pb.cc 
> b07ecda41b5159458c1d3f7d215afc6835ccf809 
> 
> Diff: https://reviews.apache.org/r/34072/diff/
> 
> 
> Testing
> ---
> 
> Tested with broken connections on Mac. Linix and Win64
> 
> 
> Thanks,
> 
> Parth Chandra
> 
>

[jira] [Created] (DRILL-3038) Error: CONNECTION ERROR: Connection null <--> null (user client) closed unexpectedly.

[Khurram Faraaz (JIRA)]

Khurram Faraaz created DRILL-3038:
-

 Summary: Error: CONNECTION ERROR: Connection null <--> null (user 
client) closed unexpectedly.
 Key: DRILL-3038
 URL: https://issues.apache.org/jira/browse/DRILL-3038
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Affects Versions: 1.0.0
Reporter: Khurram Faraaz
Assignee: Chris Westin
Priority: Minor


When I stop a Drillbit using drillbit.sh stop on which a SQL query is hung, I 
see these messages on sqlline prompt

{code}
0: jdbc:drill:schema=dfs.tmp> select count(*) from `twoKeyJsn.json`;
Error: CONNECTION ERROR: Connection null <--> null (user client) closed 
unexpectedly.


[Error Id: 53fdea70-6bf9-4dd7-914f-8ad1390bcfc9 ] (state=,code=0)
0: jdbc:drill:schema=dfs.tmp> +++
. . . . . . . . . . . . . . > 1,803,958 rows selected (37.233 seconds)
. . . . . . . . . . . . . . > 0: jdbc:drill:schema=dfs.tmp> select count(*) 
from `twoKeyJsn.json`;
Error: CONNECTION ERROR: Connection null <--> null (user client) closed 
unexpectedly.


[Error Id: 05f13b36-545b-4428-a547-d70d537643cf ] (state=,code=0)
0: jdbc:drill:schema=dfs.tmp> 
0: jdbc:drill:schema=dfs.tmp> +++
. . . . . . . . . . . . . . > 1,803,958 rows selected (37.233 seconds)
. . . . . . . . . . . . . . > 0: jdbc:drill:schema=dfs.tmp> select count(*) 
from `twoKeyJsn.json`;
Error: CONNECTION ERROR: Connection null <--> null (user client) closed 
unexpectedly.


[Error Id: 9d546618-7be7-4fa9-8655-37e2725805b0 ] (state=,code=0)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 34075: DRILL-3033: Add memory leak fixes found so far in DRILL-1942 to 1.0.0

[Chris Westin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34075/
---

(Updated May 12, 2015, 11:57 a.m.)


Review request for drill and Jacques Nadeau.


Changes
---

Addressed review request.


Bugs: DRILL-3033
https://issues.apache.org/jira/browse/DRILL-3033


Repository: drill-git


Description
---

Fixes some missing buffer retains() and missing vector clears().


Diffs (updated)
-

  exec/java-exec/src/main/codegen/templates/FixedValueVectors.java 7d85810 
  exec/java-exec/src/main/codegen/templates/NullableValueVectors.java 9d03efb 
  exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java 659d99b 
  
exec/java-exec/src/main/java/org/apache/drill/exec/cache/VectorAccessibleSerializable.java
 8e2ce96 
  
exec/java-exec/src/main/java/org/apache/drill/exec/memory/BufferAllocator.java 
c233ac5 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/PriorityQueueTemplate.java
 369c0ec 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/TopNBatch.java
 c3e70f5 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatch.java
 0430f1b 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatchBuilder.java
 1187bd6 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java
 5d990f0 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/orderedpartitioner/OrderedPartitionRecordBatch.java
 ca6d83c 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/sort/SortBatch.java
 8748aaf 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/sort/SortRecordBatchBuilder.java
 e559ece 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/ExternalSortBatch.java
 3159811 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/MSortTemplate.java
 9b97e1c 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/PriorityQueueCopierTemplate.java
 f7786b7 
  exec/java-exec/src/main/java/org/apache/drill/exec/record/WritableBatch.java 
308a8bc 
  exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java 
ae5fad5 
  
exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/AbstractMapVector.java
 78846dc 
  
exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/MapVector.java
 b615b66 
  
exec/java-exec/src/test/java/org/apache/drill/exec/record/vector/TestValueVector.java
 2b1dff0 

Diff: https://reviews.apache.org/r/34075/diff/


Testing
---

mvn install
precommit suite


Thanks,

Chris Westin

Re: Review Request 34071: DRILL-3034: Apply UserException to port-binding; handle in embedded-Drill case.

[abdelhakim deneche]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34071/#review83446
---


+1 (non binding)

- abdelhakim deneche


On May 12, 2015, 5:49 p.m., Daniel Barclay wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34071/
> ---
> 
> (Updated May 12, 2015, 5:49 p.m.)
> 
> 
> Review request for drill, abdelhakim deneche, Mehant Baid, and Parth Chandra.
> 
> 
> Bugs: DRILL-3034
> https://issues.apache.org/jira/browse/DRILL-3034
> 
> 
> Repository: drill-git
> 
> 
> Description
> ---
> 
> Applied UserException to can't-bind-to-port error.  [BasicServer]
> Added specific handling of UserException (above case or other) in 
> SQLException wrapping.  [DrillConnectionImpl]
> 
> 
> Diffs
> -
> 
>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/BasicServer.java 
> a148436 
>   exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillConnectionImpl.java 
> 30279e6 
> 
> Diff: https://reviews.apache.org/r/34071/diff/
> 
> 
> Testing
> ---
> 
> Manually tested in SQLLine and debugger.
> 
> Ran regular tests; no new failures.
> 
> 
> Thanks,
> 
> Daniel Barclay
> 
>

Re: Review Request 34071: DRILL-3034: Apply UserException to port-binding; handle in embedded-Drill case.

[Daniel Barclay]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34071/
---

(Updated May 12, 2015, 5:49 p.m.)


Review request for drill, abdelhakim deneche, Mehant Baid, and Parth Chandra.


Changes
---

Removed redundant logging call.


Bugs: DRILL-3034
https://issues.apache.org/jira/browse/DRILL-3034


Repository: drill-git


Description
---

Applied UserException to can't-bind-to-port error.  [BasicServer]
Added specific handling of UserException (above case or other) in SQLException 
wrapping.  [DrillConnectionImpl]


Diffs (updated)
-

  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/BasicServer.java 
a148436 
  exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillConnectionImpl.java 
30279e6 

Diff: https://reviews.apache.org/r/34071/diff/


Testing
---

Manually tested in SQLLine and debugger.

Ran regular tests; no new failures.


Thanks,

Daniel Barclay

Re: Review Request 34071: DRILL-3034: Apply UserException to port-binding; handle in embedded-Drill case.

[Daniel Barclay]

> On May 12, 2015, 3:26 p.m., abdelhakim deneche wrote:
> > exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillConnectionImpl.java, 
> > line 101
> > 
> >
> > this will alter the user exception's message. Sometimes this will 
> > clutter the original error message making it less useful to the user

I wanted to keep the "Failure in starting embedded Drillbit" text in both cases 
(whether the caught exception is a UserExcption or some other exception) for 
giving context for the lower-level error (to report what failed to be 
accomplished (starting Drill) because of the binding or other failure) and for 
consistency for usability.


> On May 12, 2015, 3:26 p.m., abdelhakim deneche wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/rpc/BasicServer.java, 
> > line 204
> > 
> >
> > build() already logs user exceptions at INFO level. Are you sure you 
> > want to log the same exception twice ?

No, I guess we don't need that.

Removed logging call.


- Daniel


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34071/#review83408
---


On May 12, 2015, 5:59 a.m., Daniel Barclay wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34071/
> ---
> 
> (Updated May 12, 2015, 5:59 a.m.)
> 
> 
> Review request for drill, abdelhakim deneche, Mehant Baid, and Parth Chandra.
> 
> 
> Bugs: DRILL-3034
> https://issues.apache.org/jira/browse/DRILL-3034
> 
> 
> Repository: drill-git
> 
> 
> Description
> ---
> 
> Applied UserException to can't-bind-to-port error.  [BasicServer]
> Added specific handling of UserException (above case or other) in 
> SQLException wrapping.  [DrillConnectionImpl]
> 
> 
> Diffs
> -
> 
>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/BasicServer.java 
> a148436 
>   exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillConnectionImpl.java 
> 30279e6 
> 
> Diff: https://reviews.apache.org/r/34071/diff/
> 
> 
> Testing
> ---
> 
> Manually tested in SQLLine and debugger.
> 
> Ran regular tests; no new failures.
> 
> 
> Thanks,
> 
> Daniel Barclay
> 
>

community hangout, now.

[Hanifi Gunes]

Stop by to say hi, introduce yourself!

https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc

Re: Review Request 34075: DRILL-3033: Add memory leak fixes found so far in DRILL-1942 to 1.0.0

[Chris Westin]

> On May 12, 2015, 9:13 a.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java, 
> > line 218
> > 
> >
> > Shouldn't this and above be target.clear() rather than reaching inside?

I go back and forth on it. As I've mentioned before, I'd like to see us move to 
not treating the empty buffer specially. One of the changes there is to have a 
close() that is different from clear(), which does a data.release(1), and then 
sets data to null. I tried that out, and found that some vectors are 
resurrected after being closed, and are used again. So I started going with the 
check for null.

I can change it to clear() if you like. What's your preference?


- Chris


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34075/#review83413
---


On May 12, 2015, 8:22 a.m., Chris Westin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34075/
> ---
> 
> (Updated May 12, 2015, 8:22 a.m.)
> 
> 
> Review request for drill and Jacques Nadeau.
> 
> 
> Bugs: DRILL-3033
> https://issues.apache.org/jira/browse/DRILL-3033
> 
> 
> Repository: drill-git
> 
> 
> Description
> ---
> 
> Fixes some missing buffer retains() and missing vector clears().
> 
> 
> Diffs
> -
> 
>   exec/java-exec/src/main/codegen/templates/FixedValueVectors.java 7d85810 
>   exec/java-exec/src/main/codegen/templates/NullableValueVectors.java 9d03efb 
>   exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java 
> 659d99b 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/cache/VectorAccessibleSerializable.java
>  8e2ce96 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/memory/BufferAllocator.java
>  c233ac5 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/PriorityQueueTemplate.java
>  369c0ec 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/TopNBatch.java
>  c3e70f5 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatch.java
>  0430f1b 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatchBuilder.java
>  1187bd6 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java
>  5d990f0 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/orderedpartitioner/OrderedPartitionRecordBatch.java
>  ca6d83c 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/sort/SortBatch.java
>  8748aaf 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/sort/SortRecordBatchBuilder.java
>  e559ece 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/ExternalSortBatch.java
>  3159811 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/MSortTemplate.java
>  9b97e1c 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/PriorityQueueCopierTemplate.java
>  f7786b7 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/record/WritableBatch.java 
> 308a8bc 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java 
> ae5fad5 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/AbstractMapVector.java
>  78846dc 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/MapVector.java
>  b615b66 
>   
> exec/java-exec/src/test/java/org/apache/drill/exec/record/vector/TestValueVector.java
>  2b1dff0 
> 
> Diff: https://reviews.apache.org/r/34075/diff/
> 
> 
> Testing
> ---
> 
> mvn install
> precommit suite
> 
> 
> Thanks,
> 
> Chris Westin
> 
>

Re: Review Request 34075: DRILL-3033: Add memory leak fixes found so far in DRILL-1942 to 1.0.0

[Jacques Nadeau]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34075/#review83413
---


One quick comment, otherwise looks good.


exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java


Shouldn't this and above be target.clear() rather than reaching inside?


- Jacques Nadeau


On May 12, 2015, 3:22 p.m., Chris Westin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34075/
> ---
> 
> (Updated May 12, 2015, 3:22 p.m.)
> 
> 
> Review request for drill and Jacques Nadeau.
> 
> 
> Bugs: DRILL-3033
> https://issues.apache.org/jira/browse/DRILL-3033
> 
> 
> Repository: drill-git
> 
> 
> Description
> ---
> 
> Fixes some missing buffer retains() and missing vector clears().
> 
> 
> Diffs
> -
> 
>   exec/java-exec/src/main/codegen/templates/FixedValueVectors.java 7d85810 
>   exec/java-exec/src/main/codegen/templates/NullableValueVectors.java 9d03efb 
>   exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java 
> 659d99b 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/cache/VectorAccessibleSerializable.java
>  8e2ce96 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/memory/BufferAllocator.java
>  c233ac5 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/PriorityQueueTemplate.java
>  369c0ec 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/TopNBatch.java
>  c3e70f5 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatch.java
>  0430f1b 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatchBuilder.java
>  1187bd6 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java
>  5d990f0 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/orderedpartitioner/OrderedPartitionRecordBatch.java
>  ca6d83c 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/sort/SortBatch.java
>  8748aaf 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/sort/SortRecordBatchBuilder.java
>  e559ece 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/ExternalSortBatch.java
>  3159811 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/MSortTemplate.java
>  9b97e1c 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/PriorityQueueCopierTemplate.java
>  f7786b7 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/record/WritableBatch.java 
> 308a8bc 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java 
> ae5fad5 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/AbstractMapVector.java
>  78846dc 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/MapVector.java
>  b615b66 
>   
> exec/java-exec/src/test/java/org/apache/drill/exec/record/vector/TestValueVector.java
>  2b1dff0 
> 
> Diff: https://reviews.apache.org/r/34075/diff/
> 
> 
> Testing
> ---
> 
> mvn install
> precommit suite
> 
> 
> Thanks,
> 
> Chris Westin
> 
>

[jira] [Resolved] (DRILL-1980) Create table with a Cast to interval day results in a file which cannot be read

[Mehant Baid (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-1980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mehant Baid resolved DRILL-1980.

Resolution: Pending Closed

Fixed in d10769f478900ff1868d206086874bdd67a45e7d

> Create table with a Cast to interval day results in a file which cannot be 
> read
> ---
>
> Key: DRILL-1980
> URL: https://issues.apache.org/jira/browse/DRILL-1980
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Parquet
>Affects Versions: 0.7.0
>Reporter: Ramana Inukonda Nagaraj
>Assignee: Mehant Baid
> Fix For: 1.0.0
>
> Attachments: DRILL-1980.patch, alltypes.json, 
> parquet_all_types.parquet
>
>
> Created a parquet file from a json file with all types listed in it.
> {code}
> 0: jdbc:drill:> CREATE TABLE parquet_all_types AS SELECT cast( INT_col as 
> int) INT_col,cast( BIGINT_col as bigint) BIGINT_col,cast( DECIMAL9_col as 
> decimal) DECIMAL9_col,cast( DECIMAL18_col as decimal(18,9)) 
> DECIMAL18_col,cast( DECIMAL28SPARSE_col as decimal(28, 14)) 
> DECIMAL28SPARSE_col,cast( DECIMAL38SPARSE_col as decimal(38, 19)) 
> DECIMAL38SPARSE_col,cast( DATE_col as date) DATE_col,cast( TIME_col as time) 
> TIME_col,cast( TIMESTAMP_col as timestamp) TIMESTAMP_col,cast( FLOAT4_col as 
> float) FLOAT4_col,cast( FLOAT8_col as double) FLOAT8_col,cast( BIT_col as 
> boolean) BIT_col,cast( VARCHAR_col as varchar(65000)) VARCHAR_col,cast( 
> VAR16CHAR_col as varchar(65000)) VAR16CHAR_col,cast( VARBINARY_col as 
> varbinary(65000)) VARBINARY_col,cast( INTERVALYEAR_col as interval year) 
> INTERVALYEAR_col,cast( INTERVALDAY_col as interval day) INTERVALDAY_col FROM 
> `/user/root/alltypes.json`;
> ++---+
> |  Fragment  | Number of records written |
> ++---+
> | 0_0| 8 |
> ++---+
> 1 row selected (0.595 seconds)
> {code}
> Tried reading created parquet file from drill. Fails with
> {code}
> 0: jdbc:drill:> explain plan for select * from 
> `/parquet_all_types/0_0_0.parquet`;
> Query failed: Query failed: Unexpected exception during fragment 
> initialization: Internal error: Error while applying rule DrillTableRule, 
> args [rel#6060:EnumerableTableAccessRel.ENUMERABLE.ANY([]).[](table=[dfs, 
> root, /parquet_all_types/0_0_0.parquet])]
> Error: exception while executing query: Failure while executing query. 
> (state=,code=0)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 34071: DRILL-3034: Apply UserException to port-binding; handle in embedded-Drill case.

[abdelhakim deneche]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34071/#review83408
---



exec/java-exec/src/main/java/org/apache/drill/exec/rpc/BasicServer.java


build() already logs user exceptions at INFO level. Are you sure you want 
to log the same exception twice ?



exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillConnectionImpl.java


this will alter the user exception's message. Sometimes this will clutter 
the original error message making it less useful to the user


- abdelhakim deneche


On May 12, 2015, 5:59 a.m., Daniel Barclay wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34071/
> ---
> 
> (Updated May 12, 2015, 5:59 a.m.)
> 
> 
> Review request for drill, abdelhakim deneche, Mehant Baid, and Parth Chandra.
> 
> 
> Bugs: DRILL-3034
> https://issues.apache.org/jira/browse/DRILL-3034
> 
> 
> Repository: drill-git
> 
> 
> Description
> ---
> 
> Applied UserException to can't-bind-to-port error.  [BasicServer]
> Added specific handling of UserException (above case or other) in 
> SQLException wrapping.  [DrillConnectionImpl]
> 
> 
> Diffs
> -
> 
>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/BasicServer.java 
> a148436 
>   exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillConnectionImpl.java 
> 30279e6 
> 
> Diff: https://reviews.apache.org/r/34071/diff/
> 
> 
> Testing
> ---
> 
> Manually tested in SQLLine and debugger.
> 
> Ran regular tests; no new failures.
> 
> 
> Thanks,
> 
> Daniel Barclay
> 
>

Re: Review Request 34075: DRILL-3033: Add memory leak fixes found so far in DRILL-1942 to 1.0.0

[Chris Westin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34075/
---

(Updated May 12, 2015, 8:22 a.m.)


Review request for drill and Jacques Nadeau.


Bugs: DRILL-3033
https://issues.apache.org/jira/browse/DRILL-3033


Repository: drill-git


Description
---

Fixes some missing buffer retains() and missing vector clears().


Diffs
-

  exec/java-exec/src/main/codegen/templates/FixedValueVectors.java 7d85810 
  exec/java-exec/src/main/codegen/templates/NullableValueVectors.java 9d03efb 
  exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java 659d99b 
  
exec/java-exec/src/main/java/org/apache/drill/exec/cache/VectorAccessibleSerializable.java
 8e2ce96 
  
exec/java-exec/src/main/java/org/apache/drill/exec/memory/BufferAllocator.java 
c233ac5 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/PriorityQueueTemplate.java
 369c0ec 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/TopNBatch.java
 c3e70f5 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatch.java
 0430f1b 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatchBuilder.java
 1187bd6 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java
 5d990f0 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/orderedpartitioner/OrderedPartitionRecordBatch.java
 ca6d83c 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/sort/SortBatch.java
 8748aaf 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/sort/SortRecordBatchBuilder.java
 e559ece 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/ExternalSortBatch.java
 3159811 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/MSortTemplate.java
 9b97e1c 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/PriorityQueueCopierTemplate.java
 f7786b7 
  exec/java-exec/src/main/java/org/apache/drill/exec/record/WritableBatch.java 
308a8bc 
  exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java 
ae5fad5 
  
exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/AbstractMapVector.java
 78846dc 
  
exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/MapVector.java
 b615b66 
  
exec/java-exec/src/test/java/org/apache/drill/exec/record/vector/TestValueVector.java
 2b1dff0 

Diff: https://reviews.apache.org/r/34075/diff/


Testing (updated)
---

mvn install
precommit suite


Thanks,

Chris Westin

Re: Review Request 34028: DRILL-3020: Copy cause exception's message to thrown SQLException's message.

[Mehant Baid]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34028/#review83400
---

Ship it!


Ship It!

- Mehant Baid


On May 11, 2015, 6:06 p.m., Daniel Barclay wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34028/
> ---
> 
> (Updated May 11, 2015, 6:06 p.m.)
> 
> 
> Review request for drill, Mehant Baid and Parth Chandra.
> 
> 
> Bugs: DRILL-3020
> https://issues.apache.org/jira/browse/DRILL-3020
> 
> 
> Repository: drill-git
> 
> 
> Description
> ---
> 
> Changed SQLException construction to set SQLException's message to the cause 
> exception's toString().
> 
> Also:
> - Narrowed one SQLException to SQLNonTransientConnectionException (not other
>   two, since unclear whether transient, non-transient, or varied).
> - Clarified/simplified exception messages.
> - Fixed message typo.
> 
> 
> Diffs
> -
> 
>   exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillConnectionImpl.java 
> 30279e6 
>   exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillCursor.java 30c85eb 
> 
> Diff: https://reviews.apache.org/r/34028/diff/
> 
> 
> Testing
> ---
> 
> Manually tested in SQLLine.
> 
> Ran regular tests; no new failures.
> 
> 
> Thanks,
> 
> Daniel Barclay
> 
>

[jira] [Created] (DRILL-3037) Unable to query on hdfs after moving to 0.9.0 version

[Sharath Akinapally (JIRA)]

Sharath Akinapally created DRILL-3037:
-

 Summary: Unable to query on hdfs after moving to 0.9.0 version
 Key: DRILL-3037
 URL: https://issues.apache.org/jira/browse/DRILL-3037
 Project: Apache Drill
  Issue Type: Bug
  Components: Storage - Other
Affects Versions: 0.9.0
Reporter: Sharath Akinapally
Assignee: Jacques Nadeau


I  recently moved from drill 0.8.0 to 0.9.0. Since then,I am unable to query on 
json files present in hadoop file system( queries working fine on other storage 
plugins cp and dfs) 

I had registered a storage plugin named hadoop in drill. I had all my files in 
/user/hadoop in hdfs. 

When I query "use hadoop;" it returns "true". But from drill when I query "show 
files in hadoop.`/user/hadoop/`;" gives me following error

Query failed: SYSTEM ERROR: Failure handling SQL.
Error: exception while executing query: Failure while executing query. 
(state=,code=0)

And when I do query "select * from hadoop.`/user/hadoop/donuts.json`;" gives me 
the following error

Query failed: PARSE ERROR: From line 1, column 15 to line 1, column 20: Table 
'hadoop./home/hadoop/donuts.json' not found

Same queries worked fine when I use apache drill 0.8.0.

So are there any extra things I should configure in drill 0.9.0 or is it a bug?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-3036) Mongo plugin doesn't pushdown filter with a project

[Adam Gilmore (JIRA)]

Adam Gilmore created DRILL-3036:
---

 Summary: Mongo plugin doesn't pushdown filter with a project
 Key: DRILL-3036
 URL: https://issues.apache.org/jira/browse/DRILL-3036
 Project: Apache Drill
  Issue Type: Bug
  Components: Storage - MongoDB
Affects Versions: 0.9.0
Reporter: Adam Gilmore
Assignee: B Anil Kumar


Exacerbated by DRILL-2732, if a project exists between the scan and the filter, 
the current Mongo pushdown filter rule will not push the filter down into the 
scan.

As implemented in DRILL-1950 and for the partition prune optimizer rule, the 
pushdown filter should handle the above scenario (by implementing the rule for 
both "filter/scan" and "filter/project/scan" scenarios).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)