[jira] [Commented] (DRILL-6025) Execution time of a running query shown as 'NOT AVAILABLE'

[Prasad Nagaraj Subramanya (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16287225#comment-16287225
 ] 

Prasad Nagaraj Subramanya commented on DRILL-6025:
--

No, the total time shows the duration since the query was started

I am talking of the case when the query state is 'RUNNING'. In such a case we 
could show the execution time till that point in time, similar to total time of 
a query in execution.

> Execution time of a running query shown as 'NOT AVAILABLE'
> --
>
> Key: DRILL-6025
> URL: https://issues.apache.org/jira/browse/DRILL-6025
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Client - HTTP
>Affects Versions: 1.11.0
>Reporter: Prasad Nagaraj Subramanya
>Assignee: Prasad Nagaraj Subramanya
>
> When a query is in 'RUNNING' state, the execution time is shown as 'NOT 
> AVAILABLE'
> We could show the execution duration till the current time



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5973) Support injection of time-bound pauses in server

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16287204#comment-16287204
 ] 

ASF GitHub Bot commented on DRILL-5973:
---

Github user kkhatua commented on the issue:

https://github.com/apache/drill/pull/1055
  
@parthchandra  Passed all regression and unit tests. Travis build is 
failing due to the GraceFullShutdown tests. Let me know if I should squash and 
rebase.


> Support injection of time-bound pauses in server
> 
>
> Key: DRILL-5973
> URL: https://issues.apache.org/jira/browse/DRILL-5973
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
> Fix For: 1.13.0
>
>
> While working on DRILL-3640 , when creating a unit test for a server-induced 
> timeout, the injecting a pause leaves the JUnit framework's DrillClient 
> without a handle to the query on the server. This is because we injected the 
> pause to occur before the server could send back a query ID, so the 
> DrillClient has no way to unpause the server.
> The workaround to support this unit test is to allow for injecting pauses 
> with a defined time-bound, after which the server would resume.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-6026) dependency missing for javax.annotation.concurrent

[Kunal Khatua (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16287203#comment-16287203
 ] 

Kunal Khatua commented on DRILL-6026:
-

I'm not seeing this error. Can you confirm where are you pulling the sources 
from, i.e. version/commit? 
The package referenced is part of the FindBugs project which was introduced 7 
months ago with DRILL-5545 . 
Can you clean your Maven cache and retry building?

> dependency missing for javax.annotation.concurrent
> --
>
> Key: DRILL-6026
> URL: https://issues.apache.org/jira/browse/DRILL-6026
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Mitchel Labonte
>Priority: Minor
>
> When building from sources, the following error pops up in module 
> exec/memory/base:
> exec/memory/base/src/main/java/org/apache/drill/exec/memory/Accountant.java[22,34]
>  error: package javax.annotation.concurrent does not exist
> exec/memory/base/src/main/java/org/apache/drill/exec/memory/Accountant.java[34,1]
>  error: cannot find symbol



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-6025) Execution time of a running query shown as 'NOT AVAILABLE'

[Kunal Khatua (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16287198#comment-16287198
 ] 

Kunal Khatua commented on DRILL-6025:
-

It was historically chosen as _Not Available_ for the reason that the query is 
inflight and this represented the post-planning part of the query. 
Is the Total time also showing up as Not Available?

> Execution time of a running query shown as 'NOT AVAILABLE'
> --
>
> Key: DRILL-6025
> URL: https://issues.apache.org/jira/browse/DRILL-6025
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Client - HTTP
>Affects Versions: 1.11.0
>Reporter: Prasad Nagaraj Subramanya
>Assignee: Prasad Nagaraj Subramanya
>
> When a query is in 'RUNNING' state, the execution time is shown as 'NOT 
> AVAILABLE'
> We could show the execution duration till the current time



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-3936) We don't handle out of memory condition during build phase of hash join

[Boaz Ben-Zvi (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16287046#comment-16287046
 ] 

Boaz Ben-Zvi commented on DRILL-3936:
-

  See DRILL-6027


> We don't handle out of memory condition during build phase of hash join
> ---
>
> Key: DRILL-3936
> URL: https://issues.apache.org/jira/browse/DRILL-3936
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Relational Operators
>Reporter: Victoria Markman
>Assignee: Boaz Ben-Zvi
>
> It looks like we just fall through ( see excerpt from HashJoinBatch.java 
> below )
> {code:java}
>   public void executeBuildPhase() throws SchemaChangeException, 
> ClassTransformationException, IOException {
> //Setup the underlying hash table
> // skip first batch if count is zero, as it may be an empty schema batch
> if (right.getRecordCount() == 0) {
>   for (final VectorWrapper w : right) {
> w.clear();
>   }
>   rightUpstream = next(right);
> }
> boolean moreData = true;
> while (moreData) {
>   switch (rightUpstream) {
>   case OUT_OF_MEMORY:
>   case NONE:
>   case NOT_YET:
>   case STOP:
> moreData = false;
> continue;
> ...
> {code}
> We don't handle it later either:
> {code:java}
>   public IterOutcome innerNext() {
> try {
>   /* If we are here for the first time, execute the build phase of the
>* hash join and setup the run time generated class for the probe side
>*/
>   if (state == BatchState.FIRST) {
> // Build the hash table, using the build side record batches.
> executeBuildPhase();
> //IterOutcome next = next(HashJoinHelper.LEFT_INPUT, 
> left);
> hashJoinProbe.setupHashJoinProbe(context, hyperContainer, left, 
> left.getRecordCount(), this, hashTable,
> hjHelper, joinType);
> // Update the hash table related stats for the operator
> updateStats(this.hashTable);
>   }
> 
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-6027) Implement spill to disk for the Hash Join

[Boaz Ben-Zvi (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16287061#comment-16287061
 ] 

Boaz Ben-Zvi commented on DRILL-6027:
-

 Also a draft for the functional spec is available:  
https://docs.google.com/document/d/1bPAddVCRxKHxi2RjqUvISIWXNqLdbRzZT9CWmanh4h0/edit


> Implement spill to disk for the Hash Join
> -
>
> Key: DRILL-6027
> URL: https://issues.apache.org/jira/browse/DRILL-6027
> Project: Apache Drill
>  Issue Type: New Feature
>  Components: Execution - Relational Operators
>Affects Versions: 1.11.0
>Reporter: Boaz Ben-Zvi
>Assignee: Boaz Ben-Zvi
> Fix For: 1.13.0
>
>
>   Implement the spill memory to disk (as needed) feature for the Hash Join 
> operator (similar to the prior work on the Hash Aggregate).
>   A design draft document has been published:
>   
> https://docs.google.com/document/d/1-c_oGQY4E5d58qJYv_zc7ka834hSaB3wDQwqKcMoSAI/edit?usp=sharing



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (DRILL-6027) Implement spill to disk for the Hash Join

[Boaz Ben-Zvi (JIRA)]

Boaz Ben-Zvi created DRILL-6027:
---

 Summary: Implement spill to disk for the Hash Join
 Key: DRILL-6027
 URL: https://issues.apache.org/jira/browse/DRILL-6027
 Project: Apache Drill
  Issue Type: New Feature
  Components: Execution - Relational Operators
Affects Versions: 1.11.0
Reporter: Boaz Ben-Zvi
Assignee: Boaz Ben-Zvi
 Fix For: 1.13.0


  Implement the spill memory to disk (as needed) feature for the Hash Join 
operator (similar to the prior work on the Hash Aggregate).

  A design draft document has been published:

  
https://docs.google.com/document/d/1-c_oGQY4E5d58qJYv_zc7ka834hSaB3wDQwqKcMoSAI/edit?usp=sharing





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5578) Drill fails on date functions in 'where clause' when queried on a JDBC Storage plugin

[Julian Hyde (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16287010#comment-16287010
 ] 

Julian Hyde commented on DRILL-5578:


These are the lines in Calcite JdbcImplementor.java:

{code}
switch (literal.getTypeName().getFamily()) {
case CHARACTER:
  return SqlLiteral.createCharString((String) literal.getValue2(), POS);
...
case ANY:
case NULL:
  switch (literal.getTypeName()) {
  case NULL:
return SqlLiteral.createNull(POS);
  // fall through
  }
default:
  throw new AssertionError(literal + ": " + literal.getTypeName());
}
{code}

So it appears that Calcite can't convert an interval literal from a RexNode to 
a SQL string. Possible explanations: (1) Calcite usually represents interval 
literals as numerics - but Drill is doing things differently; (2) we've not 
tested before because we don't know of any standard SQL functions that accept 
intervals as arguments.

> Drill fails on date functions in 'where clause' when queried on a JDBC 
> Storage plugin
> -
>
> Key: DRILL-5578
> URL: https://issues.apache.org/jira/browse/DRILL-5578
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.9.0, 1.10.0
>Reporter: Rahul Raj
>
> Drill 1.9/1.10 fails on any date manipulation in the where part while 
> querying on a table from JDBC Storage plugin. Issue happens on postgres and 
> Oracle.
> The following two queries error out:
> select * from config_1.APP.EXECUTIONSTEP  where DATE_ADD(CAST(STARTED_AT as 
> DATE),interval '1' second) < CAST(CURRENT_DATE as DATE)
> select `id` from (select * from config_1.public.project_release) where 
> CAST(DATE_ADD(`start_date`,interval '19800' second(5)) AS DATE) = DATE 
> '2011-11-11'
> However date manipulation succeeds when date function are applied on the 
> selected fields
>  
> select DATE_ADD(CAST(STARTED_AT as DATE),interval '1' second) < 
> CAST(CURRENT_DATE as DATE) from config_1.APP.EXECUTIONSTEP 
> select `id` from (select * from config_1.public.project_release) where 
> CAST(`start_date` AS DATE) = DATE '2011-11-11'
> Error:
> [Error Id: 048fe4b9-ecab-40fe-aca9-d57eb2df9b0c on vpc12.o3c.in:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 
> AssertionError: 1000: INTERVAL_DAY_TIME
> [Error Id: 048fe4b9-ecab-40fe-aca9-d57eb2df9b0c on vpc12.o3c.in:31010]
>   at 
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:543)
>  ~[drill-common-1.9.0.jar:1.9.0]
>   at 
> org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:825)
>  [drill-java-exec-1.9.0.jar:1.9.0]
>   at 
> org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:935) 
> [drill-java-exec-1.9.0.jar:1.9.0]
>   at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:281) 
> [drill-java-exec-1.9.0.jar:1.9.0]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_60]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_60]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
> Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected 
> exception during fragment initialization: 1000: INTERVAL_DAY_TIME
>   ... 4 common frames omitted
> Caused by: java.lang.AssertionError: 1000: INTERVAL_DAY_TIME
>   at 
> org.apache.calcite.adapter.jdbc.JdbcImplementor$Context.toSql(JdbcImplementor.java:179)
>  ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
>   at 
> org.apache.calcite.adapter.jdbc.JdbcImplementor$Context.toSql(JdbcImplementor.java:268)
>  ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
>   at 
> org.apache.calcite.adapter.jdbc.JdbcImplementor$Context.toSql(JdbcImplementor.java:212)
>  ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
>   at 
> org.apache.calcite.adapter.jdbc.JdbcImplementor$Context.toSql(JdbcImplementor.java:268)
>  ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
>   at 
> org.apache.calcite.adapter.jdbc.JdbcImplementor$Context.toSql(JdbcImplementor.java:212)
>  ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
>   at 
> org.apache.calcite.adapter.jdbc.JdbcImplementor$Context.toSql(JdbcImplementor.java:268)
>  ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
>   at 
> org.apache.calcite.adapter.jdbc.JdbcImplementor$Context.toSql(JdbcImplementor.java:212)
>  ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
>   at 
> org.apache.calcite.adapter.jdbc.JdbcRules$JdbcFilter.implement(JdbcRules.java:658)
>  ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
>   at 
> org.apache.calcite.adapter.jdbc.JdbcImplementor.visit

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286898#comment-16286898
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user ilooner commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156245506
  
--- Diff: 
contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java
 ---
@@ -91,9 +90,9 @@ public MongoRecordReader(MongoSubScan.MongoSubScanSpec 
subScanSpec, List Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests: 
>   TestMetadataProvider.tables:153 expected: but was:
>   TestMetadataProvider.tablesWithTableNameFilter:212 expected: but 
> was:
>   TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but 
> was:
>   TestMetadataProvider.tablesWithTableFilter:176 expected: but 
> was:
> Tests in error: 
>   TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR: 
> URISyntaxExcepti...
>   TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR: 
> URISyn...
>   TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth » 
> UserRemote
>   TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344 
> » Rpc
>   TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 » 
> Rpc o...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286899#comment-16286899
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user ilooner commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156245600
  
--- Diff: 
contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java
 ---
@@ -144,13 +143,12 @@ private void init(MongoSubScan.MongoSubScanSpec 
subScanSpec) {
 }
 MongoClient client = plugin.getClient(addresses);
 MongoDatabase db = client.getDatabase(subScanSpec.getDbName());
-this.unionEnabled = 
fragmentContext.getOptions().getOption(ExecConstants.ENABLE_UNION_TYPE);
+this.unionEnabled = 
fragmentContext.getOptionSet().getOption(ExecConstants.ENABLE_UNION_TYPE);
--- End diff --

done


> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests: 
>   TestMetadataProvider.tables:153 expected: but was:
>   TestMetadataProvider.tablesWithTableNameFilter:212 expected: but 
> was:
>   TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but 
> was:
>   TestMetadataProvider.tablesWithTableFilter:176 expected: but 
> was:
> Tests in error: 
>   TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR: 
> URISyntaxExcepti...
>   TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR: 
> URISyn...
>   TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth » 
> UserRemote
>   TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344 
> » Rpc
>   TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 » 
> Rpc o...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286892#comment-16286892
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user ilooner commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156245196
  
--- Diff: 
contrib/storage-kafka/src/main/java/org/apache/drill/exec/store/kafka/KafkaRecordReader.java
 ---
@@ -66,13 +66,12 @@
   public KafkaRecordReader(KafkaSubScan.KafkaSubScanSpec subScanSpec, 
List projectedColumns,
   FragmentContext context, KafkaStoragePlugin plugin) {
 setColumns(projectedColumns);
-this.enableAllTextMode = 
context.getOptions().getOption(ExecConstants.KAFKA_ALL_TEXT_MODE).bool_val;
-this.readNumbersAsDouble = context.getOptions()
-
.getOption(ExecConstants.KAFKA_READER_READ_NUMBERS_AS_DOUBLE).bool_val;
-OptionManager options = context.getOptions();
-this.unionEnabled = options.getOption(ExecConstants.ENABLE_UNION_TYPE);
-this.kafkaMsgReader = 
options.getOption(ExecConstants.KAFKA_RECORD_READER).string_val;
-this.kafkaPollTimeOut = 
options.getOption(ExecConstants.KAFKA_POLL_TIMEOUT).num_val;
+final OptionSet optionSet = context.getOptionSet();
+this.enableAllTextMode = 
optionSet.getOption(ExecConstants.KAFKA_ALL_TEXT_MODE).bool_val;
+this.readNumbersAsDouble = 
optionSet.getOption(ExecConstants.KAFKA_READER_READ_NUMBERS_AS_DOUBLE).bool_val;
+this.unionEnabled = 
optionSet.getOption(ExecConstants.ENABLE_UNION_TYPE);
+this.kafkaMsgReader = 
optionSet.getOption(ExecConstants.KAFKA_RECORD_READER).string_val;
+this.kafkaPollTimeOut = 
optionSet.getOption(ExecConstants.KAFKA_POLL_TIMEOUT).num_val;
--- End diff --

done


> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests: 
>   TestMetadataProvider.tables:153 expected: but was:
>   TestMetadataProvider.tablesWithTableNameFilter:212 expected: but 
> was:
>   TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but 
> was:
>   TestMetadataProvider.tablesWithTableFilter:176 expected: but 
> was:
> Tests in error: 
>   TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR: 
> URISyntaxExcepti...
>   TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR: 
> URISyn...
>   TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth » 
> UserRemote
>   TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344 
> » Rpc
>   TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 » 
> Rpc o...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5967) Memory leak by HashPartitionSender

[Timothy Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286823#comment-16286823
 ] 

Timothy Farkas commented on DRILL-5967:
---

[~dechanggu] Please try commit *a15e80c1bc225b9fe745b4bfeaedbc89b9f56262* .

After looking at the code again I saw my previous two fixes actually didn't fix 
anything, which is probably when they didn't work :). The root fragment was 
already clearing data stored by the partitioners and waiting for asynchronously 
sent batches to be freed when it's close method was called. I suspect the issue 
is that a new OrderedPartitionRecordBatch is created in 
OrderedPartitionSenderCreator and passed to the PartitionSenderRootExec. Since 
the OrderedPartitionRecordBatch is created in the batch creator it is never 
added to the list of operators that need to be closed by the root, so the 
memory it allocates is never freed.

> Memory leak by HashPartitionSender
> --
>
> Key: DRILL-5967
> URL: https://issues.apache.org/jira/browse/DRILL-5967
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> The error found by [~cch...@maprtech.com] and [~dechanggu]
> {code}
> 2017-10-25 15:43:28,658 [260eec84-7de3-03ec-300f-7fdbc111fb7c:frag:2:9] ERROR 
> o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: IllegalStateException: 
> Memory was leaked by query. Memory leaked: (9216)
> Allocator(op:2:9:0:HashPartitionSender) 100/9216/12831744/100 
> (res/actual/peak/limit)
> Fragment 2:9
> [Error Id: 7eae6c2a-868c-49f8-aad8-b690243ffe9b on mperf113.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 
> IllegalStateException: Memory was leaked by query. Memory leaked: (9216)
> Allocator(op:2:9:0:HashPartitionSender) 100/9216/12831744/100 
> (res/actual/peak/limit)
> Fragment 2:9
> [Error Id: 7eae6c2a-868c-49f8-aad8-b690243ffe9b on mperf113.qa.lab:31010]
> at 
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:586)
>  ~[drill-common-1.11.0-mapr.jar:1.11.0-mapr]
> at 
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:301)
>  [drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
> at 
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160)
>  [drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
> at 
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:267)
>  [drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
> at 
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
>  [drill-common-1.11.0-mapr.jar:1.11.0-mapr]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_121]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> Caused by: java.lang.IllegalStateException: Memory was leaked by query. 
> Memory leaked: (9216)
> Allocator(op:2:9:0:HashPartitionSender) 100/9216/12831744/100 
> (res/actual/peak/limit)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286865#comment-16286865
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user ilooner commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156243457
  
--- Diff: 
contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/MaprDBJsonRecordReader.java
 ---
@@ -117,7 +116,7 @@ public MaprDBJsonRecordReader(MapRDBSubScanSpec 
subScanSpec,
 
 disableCountOptimization = 
formatPluginConfig.disableCountOptimization();
 setColumns(projectedColumns);
-unionEnabled = 
context.getOptions().getOption(ExecConstants.ENABLE_UNION_TYPE);
+unionEnabled = 
context.getOptionSet().getOption(ExecConstants.ENABLE_UNION_TYPE);
--- End diff --

Sounds good


> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests: 
>   TestMetadataProvider.tables:153 expected: but was:
>   TestMetadataProvider.tablesWithTableNameFilter:212 expected: but 
> was:
>   TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but 
> was:
>   TestMetadataProvider.tablesWithTableFilter:176 expected: but 
> was:
> Tests in error: 
>   TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR: 
> URISyntaxExcepti...
>   TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR: 
> URISyn...
>   TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth » 
> UserRemote
>   TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344 
> » Rpc
>   TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 » 
> Rpc o...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286864#comment-16286864
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user ilooner commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156243416
  
--- Diff: 
contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/MaprDBJsonRecordReader.java
 ---
@@ -117,7 +116,7 @@ public MaprDBJsonRecordReader(MapRDBSubScanSpec 
subScanSpec,
 
 disableCountOptimization = 
formatPluginConfig.disableCountOptimization();
 setColumns(projectedColumns);
-unionEnabled = 
context.getOptions().getOption(ExecConstants.ENABLE_UNION_TYPE);
+unionEnabled = 
context.getOptionSet().getOption(ExecConstants.ENABLE_UNION_TYPE);
--- End diff --

Done


> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests: 
>   TestMetadataProvider.tables:153 expected: but was:
>   TestMetadataProvider.tablesWithTableNameFilter:212 expected: but 
> was:
>   TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but 
> was:
>   TestMetadataProvider.tablesWithTableFilter:176 expected: but 
> was:
> Tests in error: 
>   TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR: 
> URISyntaxExcepti...
>   TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR: 
> URISyn...
>   TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth » 
> UserRemote
>   TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344 
> » Rpc
>   TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 » 
> Rpc o...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286876#comment-16286876
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user ilooner commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156244298
  
--- Diff: 
contrib/storage-jdbc/src/main/java/org/apache/drill/exec/store/jdbc/JdbcBatchCreator.java
 ---
@@ -33,10 +32,10 @@
 public class JdbcBatchCreator implements BatchCreator {
   @Override
   public ScanBatch getBatch(FragmentContext context, JdbcSubScan config,
-  List children) throws ExecutionSetupException {
+List children) throws 
ExecutionSetupException {
 Preconditions.checkArgument(children.isEmpty());
 JdbcStoragePlugin plugin = config.getPlugin();
-RecordReader reader = new JdbcRecordReader(context, 
plugin.getSource(), config.getSql(), plugin.getName());
+RecordReader reader = new JdbcRecordReader(plugin.getSource(), 
config.getSql(), plugin.getName());
--- End diff --

Which file did I remove the POP config from? I can't seem to find it.


> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests: 
>   TestMetadataProvider.tables:153 expected: but was:
>   TestMetadataProvider.tablesWithTableNameFilter:212 expected: but 
> was:
>   TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but 
> was:
>   TestMetadataProvider.tablesWithTableFilter:176 expected: but 
> was:
> Tests in error: 
>   TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR: 
> URISyntaxExcepti...
>   TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR: 
> URISyn...
>   TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth » 
> UserRemote
>   TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344 
> » Rpc
>   TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 » 
> Rpc o...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5922) Intermittent Memory Leaks in the ROOT allocator

[Timothy Farkas (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Timothy Farkas updated DRILL-5922:
--
Labels: ready-to-commit  (was: )

> Intermittent Memory Leaks in the ROOT allocator  
> -
>
> Key: DRILL-5922
> URL: https://issues.apache.org/jira/browse/DRILL-5922
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Minor
>  Labels: ready-to-commit
>
> This issue was originall found by [~ben-zvi]. I am able to consistently 
> reproduce the error on my laptop by running the following unit test:
> org.apache.drill.exec.DrillSeparatePlanningTest#testMultiMinorFragmentComplexQuery
> {code}
> java.lang.IllegalStateException: Allocator[ROOT] closed with outstanding 
> child allocators.
> Allocator(ROOT) 0/1048576/10113536/3221225472 (res/actual/peak/limit)
>   child allocators: 1
> Allocator(query:26049b50-0cec-0a92-437c-bbe486e1fcbf) 
> 1048576/0/0/268435456 (res/actual/peak/limit)
>   child allocators: 0
>   ledgers: 0
>   reservations: 0
>   ledgers: 0
>   reservations: 0
>   at 
> org.apache.drill.exec.memory.BaseAllocator.close(BaseAllocator.java:496) 
> ~[classes/:na]
>   at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76) 
> [classes/:na]
>   at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64) 
> [classes/:na]
>   at 
> org.apache.drill.exec.server.BootStrapContext.close(BootStrapContext.java:256)
>  ~[classes/:na]
>   at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76) 
> [classes/:na]
>   at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64) 
> [classes/:na]
>   at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:205) 
> [classes/:na]
>   at org.apache.drill.BaseTestQuery.closeClient(BaseTestQuery.java:315) 
> [test-classes/:na]
>   at 
> org.apache.drill.BaseTestQuery.updateTestCluster(BaseTestQuery.java:157) 
> [test-classes/:na]
>   at 
> org.apache.drill.BaseTestQuery.updateTestCluster(BaseTestQuery.java:148) 
> [test-classes/:na]
>   at 
> org.apache.drill.exec.DrillSeparatePlanningTest.getFragmentsHelper(DrillSeparatePlanningTest.java:185)
>  [test-classes/:na]
>   at 
> org.apache.drill.exec.DrillSeparatePlanningTest.testMultiMinorFragmentComplexQuery(DrillSeparatePlanningTest.java:108)
>  [test-classes/:na]
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
> ~[na:1.8.0_144]
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> ~[na:1.8.0_144]
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[na:1.8.0_144]
>   at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_144]
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
>  [junit-4.11.jar:na]
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>  [junit-4.11.jar:na]
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
>  [junit-4.11.jar:na]
>   at 
> mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.executeTestMethod(JUnit4TestRunnerDecorator.java:120)
>  [jmockit-1.3.jar:na]
>   at 
> mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.invokeExplosively(JUnit4TestRunnerDecorator.java:65)
>  [jmockit-1.3.jar:na]
>   at 
> mockit.integration.junit4.internal.MockFrameworkMethod.invokeExplosively(MockFrameworkMethod.java:29)
>  [jmockit-1.3.jar:na]
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
> ~[na:1.8.0_144]
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> ~[na:1.8.0_144]
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[na:1.8.0_144]
>   at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_144]
>   at 
> mockit.internal.util.MethodReflection.invokeWithCheckedThrows(MethodReflection.java:95)
>  [jmockit-1.3.jar:na]
>   at 
> mockit.internal.annotations.MockMethodBridge.callMock(MockMethodBridge.java:76)
>  [jmockit-1.3.jar:na]
>   at 
> mockit.internal.annotations.MockMethodBridge.invoke(MockMethodBridge.java:41) 
> [jmockit-1.3.jar:na]
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java)
>  [junit-4.11.jar:na]
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>  [junit-4.11.jar:na]
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
>  [junit-4.11.jar:na]
> {code}



--
This message was sent by Atlassian JIRA
(v6.4

[jira] [Commented] (DRILL-5922) Intermittent Memory Leaks in the ROOT allocator

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286667#comment-16286667
 ] 

ASF GitHub Bot commented on DRILL-5922:
---

Github user ilooner commented on the issue:

https://github.com/apache/drill/pull/1023
  
@paul-rogers Applied review comments. I'll squash commits and mark this as 
ready to commit



> Intermittent Memory Leaks in the ROOT allocator  
> -
>
> Key: DRILL-5922
> URL: https://issues.apache.org/jira/browse/DRILL-5922
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Minor
>
> This issue was originall found by [~ben-zvi]. I am able to consistently 
> reproduce the error on my laptop by running the following unit test:
> org.apache.drill.exec.DrillSeparatePlanningTest#testMultiMinorFragmentComplexQuery
> {code}
> java.lang.IllegalStateException: Allocator[ROOT] closed with outstanding 
> child allocators.
> Allocator(ROOT) 0/1048576/10113536/3221225472 (res/actual/peak/limit)
>   child allocators: 1
> Allocator(query:26049b50-0cec-0a92-437c-bbe486e1fcbf) 
> 1048576/0/0/268435456 (res/actual/peak/limit)
>   child allocators: 0
>   ledgers: 0
>   reservations: 0
>   ledgers: 0
>   reservations: 0
>   at 
> org.apache.drill.exec.memory.BaseAllocator.close(BaseAllocator.java:496) 
> ~[classes/:na]
>   at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76) 
> [classes/:na]
>   at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64) 
> [classes/:na]
>   at 
> org.apache.drill.exec.server.BootStrapContext.close(BootStrapContext.java:256)
>  ~[classes/:na]
>   at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76) 
> [classes/:na]
>   at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64) 
> [classes/:na]
>   at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:205) 
> [classes/:na]
>   at org.apache.drill.BaseTestQuery.closeClient(BaseTestQuery.java:315) 
> [test-classes/:na]
>   at 
> org.apache.drill.BaseTestQuery.updateTestCluster(BaseTestQuery.java:157) 
> [test-classes/:na]
>   at 
> org.apache.drill.BaseTestQuery.updateTestCluster(BaseTestQuery.java:148) 
> [test-classes/:na]
>   at 
> org.apache.drill.exec.DrillSeparatePlanningTest.getFragmentsHelper(DrillSeparatePlanningTest.java:185)
>  [test-classes/:na]
>   at 
> org.apache.drill.exec.DrillSeparatePlanningTest.testMultiMinorFragmentComplexQuery(DrillSeparatePlanningTest.java:108)
>  [test-classes/:na]
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
> ~[na:1.8.0_144]
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> ~[na:1.8.0_144]
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[na:1.8.0_144]
>   at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_144]
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
>  [junit-4.11.jar:na]
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>  [junit-4.11.jar:na]
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
>  [junit-4.11.jar:na]
>   at 
> mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.executeTestMethod(JUnit4TestRunnerDecorator.java:120)
>  [jmockit-1.3.jar:na]
>   at 
> mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.invokeExplosively(JUnit4TestRunnerDecorator.java:65)
>  [jmockit-1.3.jar:na]
>   at 
> mockit.integration.junit4.internal.MockFrameworkMethod.invokeExplosively(MockFrameworkMethod.java:29)
>  [jmockit-1.3.jar:na]
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
> ~[na:1.8.0_144]
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> ~[na:1.8.0_144]
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[na:1.8.0_144]
>   at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_144]
>   at 
> mockit.internal.util.MethodReflection.invokeWithCheckedThrows(MethodReflection.java:95)
>  [jmockit-1.3.jar:na]
>   at 
> mockit.internal.annotations.MockMethodBridge.callMock(MockMethodBridge.java:76)
>  [jmockit-1.3.jar:na]
>   at 
> mockit.internal.annotations.MockMethodBridge.invoke(MockMethodBridge.java:41) 
> [jmockit-1.3.jar:na]
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java)
>  [junit-4.11.jar:na]
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>  [junit-4.11.jar:na]
>   at 
> org.

[jira] [Updated] (DRILL-6026) dependency missing for javax.annotation.concurrent

[Mitchel Labonte (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-6026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mitchel Labonte updated DRILL-6026:
---
Description: 
When building from sources, the following error pops up in module 
exec/memory/base:
exec/memory/base/src/main/java/org/apache/drill/exec/memory/Accountant.java[22,34]
 error: package javax.annotation.concurrent does not exist
exec/memory/base/src/main/java/org/apache/drill/exec/memory/Accountant.java[34,1]
 error: cannot find symbol

  was:
When building from sources, the following error pops up:
exec/memory/base/src/main/java/org/apache/drill/exec/memory/Accountant.java[22,34]
 error: package javax.annotation.concurrent does not exist
exec/memory/base/src/main/java/org/apache/drill/exec/memory/Accountant.java[34,1]
 error: cannot find symbol


> dependency missing for javax.annotation.concurrent
> --
>
> Key: DRILL-6026
> URL: https://issues.apache.org/jira/browse/DRILL-6026
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Mitchel Labonte
>Priority: Minor
>
> When building from sources, the following error pops up in module 
> exec/memory/base:
> exec/memory/base/src/main/java/org/apache/drill/exec/memory/Accountant.java[22,34]
>  error: package javax.annotation.concurrent does not exist
> exec/memory/base/src/main/java/org/apache/drill/exec/memory/Accountant.java[34,1]
>  error: cannot find symbol



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (DRILL-6026) dependency missing for javax.annotation.concurrent

[Mitchel Labonte (JIRA)]

Mitchel Labonte created DRILL-6026:
--

 Summary: dependency missing for javax.annotation.concurrent
 Key: DRILL-6026
 URL: https://issues.apache.org/jira/browse/DRILL-6026
 Project: Apache Drill
  Issue Type: Bug
Reporter: Mitchel Labonte
Priority: Minor


When building from sources, the following error pops up:
exec/memory/base/src/main/java/org/apache/drill/exec/memory/Accountant.java[22,34]
 error: package javax.annotation.concurrent does not exist
exec/memory/base/src/main/java/org/apache/drill/exec/memory/Accountant.java[34,1]
 error: cannot find symbol



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286622#comment-16286622
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user ilooner commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156208221
  
--- Diff: 
common/src/main/java/org/apache/drill/common/config/DrillConfig.java ---
@@ -45,13 +45,11 @@
 
   private final ImmutableList startupArguments;
 
-  public static final boolean ON_OSX = 
System.getProperty("os.name").contains("OS X");
-
   @SuppressWarnings("restriction")
   private static final long MAX_DIRECT_MEMORY = 
sun.misc.VM.maxDirectMemory();
 
   @VisibleForTesting
-  public DrillConfig(Config config, boolean enableServerConfigs) {
+  public DrillConfig(Config config) {
--- End diff --

The flag is actually an unused argument. Maybe it did something at one 
point but the code that used it is long gone.


> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests: 
>   TestMetadataProvider.tables:153 expected: but was:
>   TestMetadataProvider.tablesWithTableNameFilter:212 expected: but 
> was:
>   TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but 
> was:
>   TestMetadataProvider.tablesWithTableFilter:176 expected: but 
> was:
> Tests in error: 
>   TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR: 
> URISyntaxExcepti...
>   TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR: 
> URISyn...
>   TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth » 
> UserRemote
>   TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344 
> » Rpc
>   TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 » 
> Rpc o...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286548#comment-16286548
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156191611
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java ---
@@ -17,483 +17,216 @@
  */
 package org.apache.drill.exec.ops;
 
+import java.io.IOException;
+import java.util.Collection;
 import java.util.List;
-import java.util.Map;
-import java.util.concurrent.Executor;
+import java.util.concurrent.ExecutorService;
 
+import com.google.common.annotations.VisibleForTesting;
 import org.apache.calcite.schema.SchemaPlus;
 import org.apache.drill.common.config.DrillConfig;
-import org.apache.drill.common.exceptions.ExecutionSetupException;
-import org.apache.drill.common.exceptions.UserException;
-import org.apache.drill.common.types.TypeProtos.MinorType;
-import org.apache.drill.exec.ExecConstants;
 import org.apache.drill.exec.compile.CodeCompiler;
-import org.apache.drill.exec.exception.OutOfMemoryException;
+import org.apache.drill.exec.coord.ClusterCoordinator;
+import org.apache.drill.exec.exception.ClassTransformationException;
+import org.apache.drill.exec.expr.ClassGenerator;
+import org.apache.drill.exec.expr.CodeGenerator;
 import org.apache.drill.exec.expr.fn.FunctionImplementationRegistry;
-import org.apache.drill.exec.expr.holders.ValueHolder;
 import org.apache.drill.exec.memory.BufferAllocator;
 import org.apache.drill.exec.physical.base.PhysicalOperator;
-import org.apache.drill.exec.planner.physical.PlannerSettings;
-import org.apache.drill.exec.proto.BitControl.PlanFragment;
-import org.apache.drill.exec.proto.CoordinationProtos.DrillbitEndpoint;
-import org.apache.drill.exec.proto.ExecProtos.FragmentHandle;
-import org.apache.drill.exec.proto.GeneralRPCProtos.Ack;
-import org.apache.drill.exec.proto.helper.QueryIdHelper;
-import org.apache.drill.exec.rpc.RpcException;
-import org.apache.drill.exec.rpc.RpcOutcomeListener;
-import org.apache.drill.exec.rpc.UserClientConnection;
-import org.apache.drill.exec.rpc.control.ControlTunnel;
-import org.apache.drill.exec.rpc.control.WorkEventBus;
-import org.apache.drill.exec.server.DrillbitContext;
-import org.apache.drill.exec.server.options.FragmentOptionManager;
-import org.apache.drill.exec.server.options.OptionList;
-import org.apache.drill.exec.server.options.OptionManager;
+import org.apache.drill.exec.physical.impl.OperatorCreatorRegistry;
+import org.apache.drill.exec.planner.PhysicalPlanReader;
+import org.apache.drill.exec.proto.CoordinationProtos;
+import org.apache.drill.exec.proto.ExecProtos;
+import org.apache.drill.exec.rpc.control.Controller;
 import org.apache.drill.exec.server.options.OptionSet;
-import org.apache.drill.exec.store.PartitionExplorer;
-import org.apache.drill.exec.store.SchemaConfig;
 import org.apache.drill.exec.testing.ExecutionControls;
-import org.apache.drill.exec.util.ImpersonationUtil;
-import org.apache.drill.exec.work.batch.IncomingBuffers;
-
-import com.google.common.annotations.VisibleForTesting;
-import com.google.common.base.Function;
-import com.google.common.base.Preconditions;
-import com.google.common.collect.Lists;
-import com.google.common.collect.Maps;
 
 import io.netty.buffer.DrillBuf;
+import org.apache.drill.exec.work.batch.IncomingBuffers;
 
 /**
- * Contextual objects required for execution of a particular fragment.
- * This is the implementation; use FragmentContextInterface
- * in code to allow tests to use test-time implementations.
+ * Fragment context interface: separates implementation from definition.
+ * Allows unit testing by mocking or reimplementing services with
+ * test-time versions. The name is awkward, chosen to avoid renaming
+ * the implementation class which is used in many places in legacy code.
+ * New code should use this interface, and the names should eventually
+ * be swapped with {@link FragmentContextImpl} becoming
+ * FragmentContextImpl and this interface becoming
+ * {@link FragmentContextImpl}.
  */
 
-public class FragmentContext extends BaseFragmentContext implements 
AutoCloseable, UdfUtilities, FragmentContextInterface {
-  private static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(FragmentContext.class);
-
-  private final Map tunnels = 
Maps.newHashMap();
-  private final List contexts = Lists.newLinkedList();
-
-  private final DrillbitContext context;
-  private final UserC

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286531#comment-16286531
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156180661
  
--- Diff: 
contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java
 ---
@@ -144,13 +143,12 @@ private void init(MongoSubScan.MongoSubScanSpec 
subScanSpec) {
 }
 MongoClient client = plugin.getClient(addresses);
 MongoDatabase db = client.getDatabase(subScanSpec.getDbName());
-this.unionEnabled = 
fragmentContext.getOptions().getOption(ExecConstants.ENABLE_UNION_TYPE);
+this.unionEnabled = 
fragmentContext.getOptionSet().getOption(ExecConstants.ENABLE_UNION_TYPE);
--- End diff --

Another one


> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests: 
>   TestMetadataProvider.tables:153 expected: but was:
>   TestMetadataProvider.tablesWithTableNameFilter:212 expected: but 
> was:
>   TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but 
> was:
>   TestMetadataProvider.tablesWithTableFilter:176 expected: but 
> was:
> Tests in error: 
>   TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR: 
> URISyntaxExcepti...
>   TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR: 
> URISyn...
>   TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth » 
> UserRemote
>   TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344 
> » Rpc
>   TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 » 
> Rpc o...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286545#comment-16286545
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156180593
  
--- Diff: 
contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java
 ---
@@ -91,9 +90,9 @@ public MongoRecordReader(MongoSubScan.MongoSubScanSpec 
subScanSpec, List Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests: 
>   TestMetadataProvider.tables:153 expected: but was:
>   TestMetadataProvider.tablesWithTableNameFilter:212 expected: but 
> was:
>   TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but 
> was:
>   TestMetadataProvider.tablesWithTableFilter:176 expected: but 
> was:
> Tests in error: 
>   TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR: 
> URISyntaxExcepti...
>   TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR: 
> URISyn...
>   TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth » 
> UserRemote
>   TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344 
> » Rpc
>   TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 » 
> Rpc o...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286542#comment-16286542
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156192043
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java ---
@@ -17,483 +17,216 @@
  */
 package org.apache.drill.exec.ops;
 
+import java.io.IOException;
+import java.util.Collection;
 import java.util.List;
-import java.util.Map;
-import java.util.concurrent.Executor;
+import java.util.concurrent.ExecutorService;
 
+import com.google.common.annotations.VisibleForTesting;
 import org.apache.calcite.schema.SchemaPlus;
 import org.apache.drill.common.config.DrillConfig;
-import org.apache.drill.common.exceptions.ExecutionSetupException;
-import org.apache.drill.common.exceptions.UserException;
-import org.apache.drill.common.types.TypeProtos.MinorType;
-import org.apache.drill.exec.ExecConstants;
 import org.apache.drill.exec.compile.CodeCompiler;
-import org.apache.drill.exec.exception.OutOfMemoryException;
+import org.apache.drill.exec.coord.ClusterCoordinator;
+import org.apache.drill.exec.exception.ClassTransformationException;
+import org.apache.drill.exec.expr.ClassGenerator;
+import org.apache.drill.exec.expr.CodeGenerator;
 import org.apache.drill.exec.expr.fn.FunctionImplementationRegistry;
-import org.apache.drill.exec.expr.holders.ValueHolder;
 import org.apache.drill.exec.memory.BufferAllocator;
 import org.apache.drill.exec.physical.base.PhysicalOperator;
-import org.apache.drill.exec.planner.physical.PlannerSettings;
-import org.apache.drill.exec.proto.BitControl.PlanFragment;
-import org.apache.drill.exec.proto.CoordinationProtos.DrillbitEndpoint;
-import org.apache.drill.exec.proto.ExecProtos.FragmentHandle;
-import org.apache.drill.exec.proto.GeneralRPCProtos.Ack;
-import org.apache.drill.exec.proto.helper.QueryIdHelper;
-import org.apache.drill.exec.rpc.RpcException;
-import org.apache.drill.exec.rpc.RpcOutcomeListener;
-import org.apache.drill.exec.rpc.UserClientConnection;
-import org.apache.drill.exec.rpc.control.ControlTunnel;
-import org.apache.drill.exec.rpc.control.WorkEventBus;
-import org.apache.drill.exec.server.DrillbitContext;
-import org.apache.drill.exec.server.options.FragmentOptionManager;
-import org.apache.drill.exec.server.options.OptionList;
-import org.apache.drill.exec.server.options.OptionManager;
+import org.apache.drill.exec.physical.impl.OperatorCreatorRegistry;
+import org.apache.drill.exec.planner.PhysicalPlanReader;
+import org.apache.drill.exec.proto.CoordinationProtos;
+import org.apache.drill.exec.proto.ExecProtos;
+import org.apache.drill.exec.rpc.control.Controller;
 import org.apache.drill.exec.server.options.OptionSet;
-import org.apache.drill.exec.store.PartitionExplorer;
-import org.apache.drill.exec.store.SchemaConfig;
 import org.apache.drill.exec.testing.ExecutionControls;
-import org.apache.drill.exec.util.ImpersonationUtil;
-import org.apache.drill.exec.work.batch.IncomingBuffers;
-
-import com.google.common.annotations.VisibleForTesting;
-import com.google.common.base.Function;
-import com.google.common.base.Preconditions;
-import com.google.common.collect.Lists;
-import com.google.common.collect.Maps;
 
 import io.netty.buffer.DrillBuf;
+import org.apache.drill.exec.work.batch.IncomingBuffers;
 
 /**
- * Contextual objects required for execution of a particular fragment.
- * This is the implementation; use FragmentContextInterface
- * in code to allow tests to use test-time implementations.
+ * Fragment context interface: separates implementation from definition.
+ * Allows unit testing by mocking or reimplementing services with
+ * test-time versions. The name is awkward, chosen to avoid renaming
+ * the implementation class which is used in many places in legacy code.
+ * New code should use this interface, and the names should eventually
+ * be swapped with {@link FragmentContextImpl} becoming
+ * FragmentContextImpl and this interface becoming
+ * {@link FragmentContextImpl}.
  */
 
-public class FragmentContext extends BaseFragmentContext implements 
AutoCloseable, UdfUtilities, FragmentContextInterface {
-  private static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(FragmentContext.class);
-
-  private final Map tunnels = 
Maps.newHashMap();
-  private final List contexts = Lists.newLinkedList();
-
-  private final DrillbitContext context;
-  private final UserC

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286544#comment-16286544
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156192227
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java ---
@@ -17,483 +17,216 @@
  */
 package org.apache.drill.exec.ops;
 
+import java.io.IOException;
+import java.util.Collection;
 import java.util.List;
-import java.util.Map;
-import java.util.concurrent.Executor;
+import java.util.concurrent.ExecutorService;
 
+import com.google.common.annotations.VisibleForTesting;
 import org.apache.calcite.schema.SchemaPlus;
 import org.apache.drill.common.config.DrillConfig;
-import org.apache.drill.common.exceptions.ExecutionSetupException;
-import org.apache.drill.common.exceptions.UserException;
-import org.apache.drill.common.types.TypeProtos.MinorType;
-import org.apache.drill.exec.ExecConstants;
 import org.apache.drill.exec.compile.CodeCompiler;
-import org.apache.drill.exec.exception.OutOfMemoryException;
+import org.apache.drill.exec.coord.ClusterCoordinator;
+import org.apache.drill.exec.exception.ClassTransformationException;
+import org.apache.drill.exec.expr.ClassGenerator;
+import org.apache.drill.exec.expr.CodeGenerator;
 import org.apache.drill.exec.expr.fn.FunctionImplementationRegistry;
-import org.apache.drill.exec.expr.holders.ValueHolder;
 import org.apache.drill.exec.memory.BufferAllocator;
 import org.apache.drill.exec.physical.base.PhysicalOperator;
-import org.apache.drill.exec.planner.physical.PlannerSettings;
-import org.apache.drill.exec.proto.BitControl.PlanFragment;
-import org.apache.drill.exec.proto.CoordinationProtos.DrillbitEndpoint;
-import org.apache.drill.exec.proto.ExecProtos.FragmentHandle;
-import org.apache.drill.exec.proto.GeneralRPCProtos.Ack;
-import org.apache.drill.exec.proto.helper.QueryIdHelper;
-import org.apache.drill.exec.rpc.RpcException;
-import org.apache.drill.exec.rpc.RpcOutcomeListener;
-import org.apache.drill.exec.rpc.UserClientConnection;
-import org.apache.drill.exec.rpc.control.ControlTunnel;
-import org.apache.drill.exec.rpc.control.WorkEventBus;
-import org.apache.drill.exec.server.DrillbitContext;
-import org.apache.drill.exec.server.options.FragmentOptionManager;
-import org.apache.drill.exec.server.options.OptionList;
-import org.apache.drill.exec.server.options.OptionManager;
+import org.apache.drill.exec.physical.impl.OperatorCreatorRegistry;
+import org.apache.drill.exec.planner.PhysicalPlanReader;
+import org.apache.drill.exec.proto.CoordinationProtos;
+import org.apache.drill.exec.proto.ExecProtos;
+import org.apache.drill.exec.rpc.control.Controller;
 import org.apache.drill.exec.server.options.OptionSet;
-import org.apache.drill.exec.store.PartitionExplorer;
-import org.apache.drill.exec.store.SchemaConfig;
 import org.apache.drill.exec.testing.ExecutionControls;
-import org.apache.drill.exec.util.ImpersonationUtil;
-import org.apache.drill.exec.work.batch.IncomingBuffers;
-
-import com.google.common.annotations.VisibleForTesting;
-import com.google.common.base.Function;
-import com.google.common.base.Preconditions;
-import com.google.common.collect.Lists;
-import com.google.common.collect.Maps;
 
 import io.netty.buffer.DrillBuf;
+import org.apache.drill.exec.work.batch.IncomingBuffers;
 
 /**
- * Contextual objects required for execution of a particular fragment.
- * This is the implementation; use FragmentContextInterface
- * in code to allow tests to use test-time implementations.
+ * Fragment context interface: separates implementation from definition.
+ * Allows unit testing by mocking or reimplementing services with
+ * test-time versions. The name is awkward, chosen to avoid renaming
+ * the implementation class which is used in many places in legacy code.
+ * New code should use this interface, and the names should eventually
+ * be swapped with {@link FragmentContextImpl} becoming
+ * FragmentContextImpl and this interface becoming
+ * {@link FragmentContextImpl}.
  */
 
-public class FragmentContext extends BaseFragmentContext implements 
AutoCloseable, UdfUtilities, FragmentContextInterface {
-  private static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(FragmentContext.class);
-
-  private final Map tunnels = 
Maps.newHashMap();
-  private final List contexts = Lists.newLinkedList();
-
-  private final DrillbitContext context;
-  private final UserC

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286547#comment-16286547
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156193025
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java ---
@@ -17,483 +17,216 @@
  */
 package org.apache.drill.exec.ops;
 
+import java.io.IOException;
+import java.util.Collection;
 import java.util.List;
-import java.util.Map;
-import java.util.concurrent.Executor;
+import java.util.concurrent.ExecutorService;
 
+import com.google.common.annotations.VisibleForTesting;
 import org.apache.calcite.schema.SchemaPlus;
 import org.apache.drill.common.config.DrillConfig;
-import org.apache.drill.common.exceptions.ExecutionSetupException;
-import org.apache.drill.common.exceptions.UserException;
-import org.apache.drill.common.types.TypeProtos.MinorType;
-import org.apache.drill.exec.ExecConstants;
 import org.apache.drill.exec.compile.CodeCompiler;
-import org.apache.drill.exec.exception.OutOfMemoryException;
+import org.apache.drill.exec.coord.ClusterCoordinator;
+import org.apache.drill.exec.exception.ClassTransformationException;
+import org.apache.drill.exec.expr.ClassGenerator;
+import org.apache.drill.exec.expr.CodeGenerator;
 import org.apache.drill.exec.expr.fn.FunctionImplementationRegistry;
-import org.apache.drill.exec.expr.holders.ValueHolder;
 import org.apache.drill.exec.memory.BufferAllocator;
 import org.apache.drill.exec.physical.base.PhysicalOperator;
-import org.apache.drill.exec.planner.physical.PlannerSettings;
-import org.apache.drill.exec.proto.BitControl.PlanFragment;
-import org.apache.drill.exec.proto.CoordinationProtos.DrillbitEndpoint;
-import org.apache.drill.exec.proto.ExecProtos.FragmentHandle;
-import org.apache.drill.exec.proto.GeneralRPCProtos.Ack;
-import org.apache.drill.exec.proto.helper.QueryIdHelper;
-import org.apache.drill.exec.rpc.RpcException;
-import org.apache.drill.exec.rpc.RpcOutcomeListener;
-import org.apache.drill.exec.rpc.UserClientConnection;
-import org.apache.drill.exec.rpc.control.ControlTunnel;
-import org.apache.drill.exec.rpc.control.WorkEventBus;
-import org.apache.drill.exec.server.DrillbitContext;
-import org.apache.drill.exec.server.options.FragmentOptionManager;
-import org.apache.drill.exec.server.options.OptionList;
-import org.apache.drill.exec.server.options.OptionManager;
+import org.apache.drill.exec.physical.impl.OperatorCreatorRegistry;
+import org.apache.drill.exec.planner.PhysicalPlanReader;
+import org.apache.drill.exec.proto.CoordinationProtos;
+import org.apache.drill.exec.proto.ExecProtos;
+import org.apache.drill.exec.rpc.control.Controller;
 import org.apache.drill.exec.server.options.OptionSet;
-import org.apache.drill.exec.store.PartitionExplorer;
-import org.apache.drill.exec.store.SchemaConfig;
 import org.apache.drill.exec.testing.ExecutionControls;
-import org.apache.drill.exec.util.ImpersonationUtil;
-import org.apache.drill.exec.work.batch.IncomingBuffers;
-
-import com.google.common.annotations.VisibleForTesting;
-import com.google.common.base.Function;
-import com.google.common.base.Preconditions;
-import com.google.common.collect.Lists;
-import com.google.common.collect.Maps;
 
 import io.netty.buffer.DrillBuf;
+import org.apache.drill.exec.work.batch.IncomingBuffers;
 
 /**
- * Contextual objects required for execution of a particular fragment.
- * This is the implementation; use FragmentContextInterface
- * in code to allow tests to use test-time implementations.
+ * Fragment context interface: separates implementation from definition.
+ * Allows unit testing by mocking or reimplementing services with
+ * test-time versions. The name is awkward, chosen to avoid renaming
+ * the implementation class which is used in many places in legacy code.
+ * New code should use this interface, and the names should eventually
+ * be swapped with {@link FragmentContextImpl} becoming
+ * FragmentContextImpl and this interface becoming
+ * {@link FragmentContextImpl}.
  */
 
-public class FragmentContext extends BaseFragmentContext implements 
AutoCloseable, UdfUtilities, FragmentContextInterface {
-  private static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(FragmentContext.class);
-
-  private final Map tunnels = 
Maps.newHashMap();
-  private final List contexts = Lists.newLinkedList();
-
-  private final DrillbitContext context;
-  private final UserC

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286549#comment-16286549
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156192365
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java ---
@@ -17,483 +17,216 @@
  */
 package org.apache.drill.exec.ops;
 
+import java.io.IOException;
+import java.util.Collection;
 import java.util.List;
-import java.util.Map;
-import java.util.concurrent.Executor;
+import java.util.concurrent.ExecutorService;
 
+import com.google.common.annotations.VisibleForTesting;
 import org.apache.calcite.schema.SchemaPlus;
 import org.apache.drill.common.config.DrillConfig;
-import org.apache.drill.common.exceptions.ExecutionSetupException;
-import org.apache.drill.common.exceptions.UserException;
-import org.apache.drill.common.types.TypeProtos.MinorType;
-import org.apache.drill.exec.ExecConstants;
 import org.apache.drill.exec.compile.CodeCompiler;
-import org.apache.drill.exec.exception.OutOfMemoryException;
+import org.apache.drill.exec.coord.ClusterCoordinator;
+import org.apache.drill.exec.exception.ClassTransformationException;
+import org.apache.drill.exec.expr.ClassGenerator;
+import org.apache.drill.exec.expr.CodeGenerator;
 import org.apache.drill.exec.expr.fn.FunctionImplementationRegistry;
-import org.apache.drill.exec.expr.holders.ValueHolder;
 import org.apache.drill.exec.memory.BufferAllocator;
 import org.apache.drill.exec.physical.base.PhysicalOperator;
-import org.apache.drill.exec.planner.physical.PlannerSettings;
-import org.apache.drill.exec.proto.BitControl.PlanFragment;
-import org.apache.drill.exec.proto.CoordinationProtos.DrillbitEndpoint;
-import org.apache.drill.exec.proto.ExecProtos.FragmentHandle;
-import org.apache.drill.exec.proto.GeneralRPCProtos.Ack;
-import org.apache.drill.exec.proto.helper.QueryIdHelper;
-import org.apache.drill.exec.rpc.RpcException;
-import org.apache.drill.exec.rpc.RpcOutcomeListener;
-import org.apache.drill.exec.rpc.UserClientConnection;
-import org.apache.drill.exec.rpc.control.ControlTunnel;
-import org.apache.drill.exec.rpc.control.WorkEventBus;
-import org.apache.drill.exec.server.DrillbitContext;
-import org.apache.drill.exec.server.options.FragmentOptionManager;
-import org.apache.drill.exec.server.options.OptionList;
-import org.apache.drill.exec.server.options.OptionManager;
+import org.apache.drill.exec.physical.impl.OperatorCreatorRegistry;
+import org.apache.drill.exec.planner.PhysicalPlanReader;
+import org.apache.drill.exec.proto.CoordinationProtos;
+import org.apache.drill.exec.proto.ExecProtos;
+import org.apache.drill.exec.rpc.control.Controller;
 import org.apache.drill.exec.server.options.OptionSet;
-import org.apache.drill.exec.store.PartitionExplorer;
-import org.apache.drill.exec.store.SchemaConfig;
 import org.apache.drill.exec.testing.ExecutionControls;
-import org.apache.drill.exec.util.ImpersonationUtil;
-import org.apache.drill.exec.work.batch.IncomingBuffers;
-
-import com.google.common.annotations.VisibleForTesting;
-import com.google.common.base.Function;
-import com.google.common.base.Preconditions;
-import com.google.common.collect.Lists;
-import com.google.common.collect.Maps;
 
 import io.netty.buffer.DrillBuf;
+import org.apache.drill.exec.work.batch.IncomingBuffers;
 
 /**
- * Contextual objects required for execution of a particular fragment.
- * This is the implementation; use FragmentContextInterface
- * in code to allow tests to use test-time implementations.
+ * Fragment context interface: separates implementation from definition.
+ * Allows unit testing by mocking or reimplementing services with
+ * test-time versions. The name is awkward, chosen to avoid renaming
+ * the implementation class which is used in many places in legacy code.
+ * New code should use this interface, and the names should eventually
+ * be swapped with {@link FragmentContextImpl} becoming
+ * FragmentContextImpl and this interface becoming
+ * {@link FragmentContextImpl}.
  */
 
-public class FragmentContext extends BaseFragmentContext implements 
AutoCloseable, UdfUtilities, FragmentContextInterface {
-  private static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(FragmentContext.class);
-
-  private final Map tunnels = 
Maps.newHashMap();
-  private final List contexts = Lists.newLinkedList();
-
-  private final DrillbitContext context;
-  private final UserC

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286532#comment-16286532
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156176534
  
--- Diff: 
contrib/storage-jdbc/src/main/java/org/apache/drill/exec/store/jdbc/JdbcBatchCreator.java
 ---
@@ -33,10 +32,10 @@
 public class JdbcBatchCreator implements BatchCreator {
   @Override
   public ScanBatch getBatch(FragmentContext context, JdbcSubScan config,
-  List children) throws ExecutionSetupException {
+List children) throws 
ExecutionSetupException {
 Preconditions.checkArgument(children.isEmpty());
 JdbcStoragePlugin plugin = config.getPlugin();
-RecordReader reader = new JdbcRecordReader(context, 
plugin.getSource(), config.getSql(), plugin.getName());
+RecordReader reader = new JdbcRecordReader(plugin.getSource(), 
config.getSql(), plugin.getName());
--- End diff --

For readers that don't need the fragment context, it is certainly helpful, 
for unit testing, to omit it. However, operator context seems fair game. The 
operator context provides the Drill file system, the physical operator 
definition (the "config" here) and so on. Unfortunately, it is created inside 
the `ScanBatch`, to late to pass to the reader constructor.

The new scan framework from the "batch size" project addresses these 
issues; but only for those few readers that we choose to upgrade; most readers 
will continue to use the changes you are making here.

One other point: in an earlier file, you removed the physical operator 
definition (AKA "POP config") from the `ScanBatch` constructor. Here you leave 
it. Will this cause unexpected modes in which sometimes the operator definition 
is available, other times not?


> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests: 
>   TestMetadataProvider.tables:153 expected: but was:
>   TestMetadataProvider.tablesWithTableNameFilter:212 expected: but 
> was:
>   TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but 
> was:
>   TestMetadataProvider.tablesWithTableFilter:176 expected: but 
> was:
> Tests in error: 
>   TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR: 
> URISyntaxExcepti...
>   TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR: 
> URISyn...
>   TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth » 
> UserRemote
>   TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344 
> » Rpc
>   TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 » 
> Rpc o...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286541#comment-16286541
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156181381
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/ops/BaseOperatorContext.java 
---
@@ -39,27 +39,29 @@
  * version of the operator context and the full production-time context
  * that includes network services.
  */
-
 public abstract class BaseOperatorContext implements OperatorContext {
   static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(BaseOperatorContext.class);
 
-  protected final FragmentContextInterface context;
+  protected final FragmentContext context;
   protected final BufferAllocator allocator;
   protected final PhysicalOperator popConfig;
   protected final BufferManager manager;
+  protected OperatorStatReceiver statsWriter;
--- End diff --

Didn't I deprecate this in a recent commit? Turned out to be more trouble 
than it was worth. The original class could be made to work in a test 
environment.


> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests: 
>   TestMetadataProvider.tables:153 expected: but was:
>   TestMetadataProvider.tablesWithTableNameFilter:212 expected: but 
> was:
>   TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but 
> was:
>   TestMetadataProvider.tablesWithTableFilter:176 expected: but 
> was:
> Tests in error: 
>   TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR: 
> URISyntaxExcepti...
>   TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR: 
> URISyn...
>   TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth » 
> UserRemote
>   TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344 
> » Rpc
>   TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 » 
> Rpc o...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286538#comment-16286538
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156191431
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java ---
@@ -17,483 +17,216 @@
  */
 package org.apache.drill.exec.ops;
 
+import java.io.IOException;
+import java.util.Collection;
 import java.util.List;
-import java.util.Map;
-import java.util.concurrent.Executor;
+import java.util.concurrent.ExecutorService;
 
+import com.google.common.annotations.VisibleForTesting;
 import org.apache.calcite.schema.SchemaPlus;
 import org.apache.drill.common.config.DrillConfig;
-import org.apache.drill.common.exceptions.ExecutionSetupException;
-import org.apache.drill.common.exceptions.UserException;
-import org.apache.drill.common.types.TypeProtos.MinorType;
-import org.apache.drill.exec.ExecConstants;
 import org.apache.drill.exec.compile.CodeCompiler;
-import org.apache.drill.exec.exception.OutOfMemoryException;
+import org.apache.drill.exec.coord.ClusterCoordinator;
+import org.apache.drill.exec.exception.ClassTransformationException;
+import org.apache.drill.exec.expr.ClassGenerator;
+import org.apache.drill.exec.expr.CodeGenerator;
 import org.apache.drill.exec.expr.fn.FunctionImplementationRegistry;
-import org.apache.drill.exec.expr.holders.ValueHolder;
 import org.apache.drill.exec.memory.BufferAllocator;
 import org.apache.drill.exec.physical.base.PhysicalOperator;
-import org.apache.drill.exec.planner.physical.PlannerSettings;
-import org.apache.drill.exec.proto.BitControl.PlanFragment;
-import org.apache.drill.exec.proto.CoordinationProtos.DrillbitEndpoint;
-import org.apache.drill.exec.proto.ExecProtos.FragmentHandle;
-import org.apache.drill.exec.proto.GeneralRPCProtos.Ack;
-import org.apache.drill.exec.proto.helper.QueryIdHelper;
-import org.apache.drill.exec.rpc.RpcException;
-import org.apache.drill.exec.rpc.RpcOutcomeListener;
-import org.apache.drill.exec.rpc.UserClientConnection;
-import org.apache.drill.exec.rpc.control.ControlTunnel;
-import org.apache.drill.exec.rpc.control.WorkEventBus;
-import org.apache.drill.exec.server.DrillbitContext;
-import org.apache.drill.exec.server.options.FragmentOptionManager;
-import org.apache.drill.exec.server.options.OptionList;
-import org.apache.drill.exec.server.options.OptionManager;
+import org.apache.drill.exec.physical.impl.OperatorCreatorRegistry;
+import org.apache.drill.exec.planner.PhysicalPlanReader;
+import org.apache.drill.exec.proto.CoordinationProtos;
+import org.apache.drill.exec.proto.ExecProtos;
+import org.apache.drill.exec.rpc.control.Controller;
 import org.apache.drill.exec.server.options.OptionSet;
-import org.apache.drill.exec.store.PartitionExplorer;
-import org.apache.drill.exec.store.SchemaConfig;
 import org.apache.drill.exec.testing.ExecutionControls;
-import org.apache.drill.exec.util.ImpersonationUtil;
-import org.apache.drill.exec.work.batch.IncomingBuffers;
-
-import com.google.common.annotations.VisibleForTesting;
-import com.google.common.base.Function;
-import com.google.common.base.Preconditions;
-import com.google.common.collect.Lists;
-import com.google.common.collect.Maps;
 
 import io.netty.buffer.DrillBuf;
+import org.apache.drill.exec.work.batch.IncomingBuffers;
 
 /**
- * Contextual objects required for execution of a particular fragment.
- * This is the implementation; use FragmentContextInterface
- * in code to allow tests to use test-time implementations.
+ * Fragment context interface: separates implementation from definition.
+ * Allows unit testing by mocking or reimplementing services with
+ * test-time versions. The name is awkward, chosen to avoid renaming
+ * the implementation class which is used in many places in legacy code.
+ * New code should use this interface, and the names should eventually
+ * be swapped with {@link FragmentContextImpl} becoming
+ * FragmentContextImpl and this interface becoming
+ * {@link FragmentContextImpl}.
  */
 
-public class FragmentContext extends BaseFragmentContext implements 
AutoCloseable, UdfUtilities, FragmentContextInterface {
-  private static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(FragmentContext.class);
-
-  private final Map tunnels = 
Maps.newHashMap();
-  private final List contexts = Lists.newLinkedList();
-
-  private final DrillbitContext context;
-  private final UserC

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286543#comment-16286543
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156180422
  
--- Diff: 
contrib/storage-kafka/src/main/java/org/apache/drill/exec/store/kafka/KafkaRecordReader.java
 ---
@@ -66,13 +66,12 @@
   public KafkaRecordReader(KafkaSubScan.KafkaSubScanSpec subScanSpec, 
List projectedColumns,
   FragmentContext context, KafkaStoragePlugin plugin) {
 setColumns(projectedColumns);
-this.enableAllTextMode = 
context.getOptions().getOption(ExecConstants.KAFKA_ALL_TEXT_MODE).bool_val;
-this.readNumbersAsDouble = context.getOptions()
-
.getOption(ExecConstants.KAFKA_READER_READ_NUMBERS_AS_DOUBLE).bool_val;
-OptionManager options = context.getOptions();
-this.unionEnabled = options.getOption(ExecConstants.ENABLE_UNION_TYPE);
-this.kafkaMsgReader = 
options.getOption(ExecConstants.KAFKA_RECORD_READER).string_val;
-this.kafkaPollTimeOut = 
options.getOption(ExecConstants.KAFKA_POLL_TIMEOUT).num_val;
+final OptionSet optionSet = context.getOptionSet();
+this.enableAllTextMode = 
optionSet.getOption(ExecConstants.KAFKA_ALL_TEXT_MODE).bool_val;
+this.readNumbersAsDouble = 
optionSet.getOption(ExecConstants.KAFKA_READER_READ_NUMBERS_AS_DOUBLE).bool_val;
+this.unionEnabled = 
optionSet.getOption(ExecConstants.ENABLE_UNION_TYPE);
+this.kafkaMsgReader = 
optionSet.getOption(ExecConstants.KAFKA_RECORD_READER).string_val;
+this.kafkaPollTimeOut = 
optionSet.getOption(ExecConstants.KAFKA_POLL_TIMEOUT).num_val;
--- End diff --

See above note. Replace:
```
this.kafkaPollTimeOut = 
optionSet.getOption(ExecConstants.KAFKA_POLL_TIMEOUT).num_val;
```

With
```
kafkaPollTimeOut = optionSet.getLong(ExecConstants.KAFKA_POLL_TIMEOUT);
```

And so on for the other options.

As it is, the above will cause an NPE if ever the option types don't match 
what the code expects.


> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests: 
>   TestMetadataProvider.tables:153 expected: but was:
>   TestMetadataProvider.tablesWithTableNameFilter:212 expected: but 
> was:
>   TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but 
> was:
>   TestMetadataProvider.tablesWithTableFilter:176 expected: but 
> was:
> Tests in error: 
>   TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR: 
> URISyntaxExcepti...
>   TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR: 
> URISyn...
>   TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth » 
> UserRemote
>   TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344 
> » Rpc
>   TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 » 
> Rpc o...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286529#comment-16286529
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156191898
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java ---
@@ -17,483 +17,216 @@
  */
 package org.apache.drill.exec.ops;
 
+import java.io.IOException;
+import java.util.Collection;
 import java.util.List;
-import java.util.Map;
-import java.util.concurrent.Executor;
+import java.util.concurrent.ExecutorService;
 
+import com.google.common.annotations.VisibleForTesting;
 import org.apache.calcite.schema.SchemaPlus;
 import org.apache.drill.common.config.DrillConfig;
-import org.apache.drill.common.exceptions.ExecutionSetupException;
-import org.apache.drill.common.exceptions.UserException;
-import org.apache.drill.common.types.TypeProtos.MinorType;
-import org.apache.drill.exec.ExecConstants;
 import org.apache.drill.exec.compile.CodeCompiler;
-import org.apache.drill.exec.exception.OutOfMemoryException;
+import org.apache.drill.exec.coord.ClusterCoordinator;
+import org.apache.drill.exec.exception.ClassTransformationException;
+import org.apache.drill.exec.expr.ClassGenerator;
+import org.apache.drill.exec.expr.CodeGenerator;
 import org.apache.drill.exec.expr.fn.FunctionImplementationRegistry;
-import org.apache.drill.exec.expr.holders.ValueHolder;
 import org.apache.drill.exec.memory.BufferAllocator;
 import org.apache.drill.exec.physical.base.PhysicalOperator;
-import org.apache.drill.exec.planner.physical.PlannerSettings;
-import org.apache.drill.exec.proto.BitControl.PlanFragment;
-import org.apache.drill.exec.proto.CoordinationProtos.DrillbitEndpoint;
-import org.apache.drill.exec.proto.ExecProtos.FragmentHandle;
-import org.apache.drill.exec.proto.GeneralRPCProtos.Ack;
-import org.apache.drill.exec.proto.helper.QueryIdHelper;
-import org.apache.drill.exec.rpc.RpcException;
-import org.apache.drill.exec.rpc.RpcOutcomeListener;
-import org.apache.drill.exec.rpc.UserClientConnection;
-import org.apache.drill.exec.rpc.control.ControlTunnel;
-import org.apache.drill.exec.rpc.control.WorkEventBus;
-import org.apache.drill.exec.server.DrillbitContext;
-import org.apache.drill.exec.server.options.FragmentOptionManager;
-import org.apache.drill.exec.server.options.OptionList;
-import org.apache.drill.exec.server.options.OptionManager;
+import org.apache.drill.exec.physical.impl.OperatorCreatorRegistry;
+import org.apache.drill.exec.planner.PhysicalPlanReader;
+import org.apache.drill.exec.proto.CoordinationProtos;
+import org.apache.drill.exec.proto.ExecProtos;
+import org.apache.drill.exec.rpc.control.Controller;
 import org.apache.drill.exec.server.options.OptionSet;
-import org.apache.drill.exec.store.PartitionExplorer;
-import org.apache.drill.exec.store.SchemaConfig;
 import org.apache.drill.exec.testing.ExecutionControls;
-import org.apache.drill.exec.util.ImpersonationUtil;
-import org.apache.drill.exec.work.batch.IncomingBuffers;
-
-import com.google.common.annotations.VisibleForTesting;
-import com.google.common.base.Function;
-import com.google.common.base.Preconditions;
-import com.google.common.collect.Lists;
-import com.google.common.collect.Maps;
 
 import io.netty.buffer.DrillBuf;
+import org.apache.drill.exec.work.batch.IncomingBuffers;
 
 /**
- * Contextual objects required for execution of a particular fragment.
- * This is the implementation; use FragmentContextInterface
- * in code to allow tests to use test-time implementations.
+ * Fragment context interface: separates implementation from definition.
+ * Allows unit testing by mocking or reimplementing services with
+ * test-time versions. The name is awkward, chosen to avoid renaming
+ * the implementation class which is used in many places in legacy code.
+ * New code should use this interface, and the names should eventually
+ * be swapped with {@link FragmentContextImpl} becoming
+ * FragmentContextImpl and this interface becoming
+ * {@link FragmentContextImpl}.
  */
 
-public class FragmentContext extends BaseFragmentContext implements 
AutoCloseable, UdfUtilities, FragmentContextInterface {
-  private static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(FragmentContext.class);
-
-  private final Map tunnels = 
Maps.newHashMap();
-  private final List contexts = Lists.newLinkedList();
-
-  private final DrillbitContext context;
-  private final UserC

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286537#comment-16286537
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156192269
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java ---
@@ -17,483 +17,216 @@
  */
 package org.apache.drill.exec.ops;
 
+import java.io.IOException;
+import java.util.Collection;
 import java.util.List;
-import java.util.Map;
-import java.util.concurrent.Executor;
+import java.util.concurrent.ExecutorService;
 
+import com.google.common.annotations.VisibleForTesting;
 import org.apache.calcite.schema.SchemaPlus;
 import org.apache.drill.common.config.DrillConfig;
-import org.apache.drill.common.exceptions.ExecutionSetupException;
-import org.apache.drill.common.exceptions.UserException;
-import org.apache.drill.common.types.TypeProtos.MinorType;
-import org.apache.drill.exec.ExecConstants;
 import org.apache.drill.exec.compile.CodeCompiler;
-import org.apache.drill.exec.exception.OutOfMemoryException;
+import org.apache.drill.exec.coord.ClusterCoordinator;
+import org.apache.drill.exec.exception.ClassTransformationException;
+import org.apache.drill.exec.expr.ClassGenerator;
+import org.apache.drill.exec.expr.CodeGenerator;
 import org.apache.drill.exec.expr.fn.FunctionImplementationRegistry;
-import org.apache.drill.exec.expr.holders.ValueHolder;
 import org.apache.drill.exec.memory.BufferAllocator;
 import org.apache.drill.exec.physical.base.PhysicalOperator;
-import org.apache.drill.exec.planner.physical.PlannerSettings;
-import org.apache.drill.exec.proto.BitControl.PlanFragment;
-import org.apache.drill.exec.proto.CoordinationProtos.DrillbitEndpoint;
-import org.apache.drill.exec.proto.ExecProtos.FragmentHandle;
-import org.apache.drill.exec.proto.GeneralRPCProtos.Ack;
-import org.apache.drill.exec.proto.helper.QueryIdHelper;
-import org.apache.drill.exec.rpc.RpcException;
-import org.apache.drill.exec.rpc.RpcOutcomeListener;
-import org.apache.drill.exec.rpc.UserClientConnection;
-import org.apache.drill.exec.rpc.control.ControlTunnel;
-import org.apache.drill.exec.rpc.control.WorkEventBus;
-import org.apache.drill.exec.server.DrillbitContext;
-import org.apache.drill.exec.server.options.FragmentOptionManager;
-import org.apache.drill.exec.server.options.OptionList;
-import org.apache.drill.exec.server.options.OptionManager;
+import org.apache.drill.exec.physical.impl.OperatorCreatorRegistry;
+import org.apache.drill.exec.planner.PhysicalPlanReader;
+import org.apache.drill.exec.proto.CoordinationProtos;
+import org.apache.drill.exec.proto.ExecProtos;
+import org.apache.drill.exec.rpc.control.Controller;
 import org.apache.drill.exec.server.options.OptionSet;
-import org.apache.drill.exec.store.PartitionExplorer;
-import org.apache.drill.exec.store.SchemaConfig;
 import org.apache.drill.exec.testing.ExecutionControls;
-import org.apache.drill.exec.util.ImpersonationUtil;
-import org.apache.drill.exec.work.batch.IncomingBuffers;
-
-import com.google.common.annotations.VisibleForTesting;
-import com.google.common.base.Function;
-import com.google.common.base.Preconditions;
-import com.google.common.collect.Lists;
-import com.google.common.collect.Maps;
 
 import io.netty.buffer.DrillBuf;
+import org.apache.drill.exec.work.batch.IncomingBuffers;
 
 /**
- * Contextual objects required for execution of a particular fragment.
- * This is the implementation; use FragmentContextInterface
- * in code to allow tests to use test-time implementations.
+ * Fragment context interface: separates implementation from definition.
+ * Allows unit testing by mocking or reimplementing services with
+ * test-time versions. The name is awkward, chosen to avoid renaming
+ * the implementation class which is used in many places in legacy code.
+ * New code should use this interface, and the names should eventually
+ * be swapped with {@link FragmentContextImpl} becoming
+ * FragmentContextImpl and this interface becoming
+ * {@link FragmentContextImpl}.
  */
 
-public class FragmentContext extends BaseFragmentContext implements 
AutoCloseable, UdfUtilities, FragmentContextInterface {
-  private static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(FragmentContext.class);
-
-  private final Map tunnels = 
Maps.newHashMap();
-  private final List contexts = Lists.newLinkedList();
-
-  private final DrillbitContext context;
-  private final UserC

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286535#comment-16286535
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156186475
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/ops/BaseOperatorContext.java 
---
@@ -161,16 +163,13 @@ public void close() {
 }
 try {
   if (fs != null) {
-fs.close();
-fs = null;
+  fs.close();
--- End diff --

Don't we like two space indentations?


> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests: 
>   TestMetadataProvider.tables:153 expected: but was:
>   TestMetadataProvider.tablesWithTableNameFilter:212 expected: but 
> was:
>   TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but 
> was:
>   TestMetadataProvider.tablesWithTableFilter:176 expected: but 
> was:
> Tests in error: 
>   TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR: 
> URISyntaxExcepti...
>   TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR: 
> URISyn...
>   TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth » 
> UserRemote
>   TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344 
> » Rpc
>   TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 » 
> Rpc o...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286546#comment-16286546
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156186386
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/ops/BaseOperatorContext.java 
---
@@ -39,27 +39,29 @@
  * version of the operator context and the full production-time context
  * that includes network services.
  */
-
 public abstract class BaseOperatorContext implements OperatorContext {
   static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(BaseOperatorContext.class);
 
-  protected final FragmentContextInterface context;
+  protected final FragmentContext context;
   protected final BufferAllocator allocator;
   protected final PhysicalOperator popConfig;
   protected final BufferManager manager;
+  protected OperatorStatReceiver statsWriter;
   private DrillFileSystem fs;
   private ControlsInjector injector;
 
-  public BaseOperatorContext(FragmentContextInterface context, 
BufferAllocator allocator,
-   PhysicalOperator popConfig) {
+  public BaseOperatorContext(FragmentContext context, BufferAllocator 
allocator,
+ PhysicalOperator popConfig,
+ OperatorStatReceiver stats) {
 this.context = context;
 this.allocator = allocator;
 this.popConfig = popConfig;
 this.manager = new BufferManagerImpl(allocator);
+this.statsWriter = stats;
--- End diff --

This member was removed in a recent commit. Since the test and production 
versions need different implementation, the derived classes manage the operator 
stats.


> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests: 
>   TestMetadataProvider.tables:153 expected: but was:
>   TestMetadataProvider.tablesWithTableNameFilter:212 expected: but 
> was:
>   TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but 
> was:
>   TestMetadataProvider.tablesWithTableFilter:176 expected: but 
> was:
> Tests in error: 
>   TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR: 
> URISyntaxExcepti...
>   TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR: 
> URISyn...
>   TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth » 
> UserRemote
>   TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344 
> » Rpc
>   TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 » 
> Rpc o...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286539#comment-16286539
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156181112
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/ops/BaseOperatorContext.java 
---
@@ -39,27 +39,29 @@
  * version of the operator context and the full production-time context
  * that includes network services.
  */
-
 public abstract class BaseOperatorContext implements OperatorContext {
   static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(BaseOperatorContext.class);
 
-  protected final FragmentContextInterface context;
+  protected final FragmentContext context;
--- End diff --

You are a brave man. I elected not to do this renaming because I didn't 
think I'd get anyone to review it. Thanks for doing it so I can do the review 
:-)


> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests: 
>   TestMetadataProvider.tables:153 expected: but was:
>   TestMetadataProvider.tablesWithTableNameFilter:212 expected: but 
> was:
>   TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but 
> was:
>   TestMetadataProvider.tablesWithTableFilter:176 expected: but 
> was:
> Tests in error: 
>   TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR: 
> URISyntaxExcepti...
>   TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR: 
> URISyn...
>   TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth » 
> UserRemote
>   TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344 
> » Rpc
>   TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 » 
> Rpc o...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286530#comment-16286530
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156192175
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java ---
@@ -17,483 +17,216 @@
  */
 package org.apache.drill.exec.ops;
 
+import java.io.IOException;
+import java.util.Collection;
 import java.util.List;
-import java.util.Map;
-import java.util.concurrent.Executor;
+import java.util.concurrent.ExecutorService;
 
+import com.google.common.annotations.VisibleForTesting;
 import org.apache.calcite.schema.SchemaPlus;
 import org.apache.drill.common.config.DrillConfig;
-import org.apache.drill.common.exceptions.ExecutionSetupException;
-import org.apache.drill.common.exceptions.UserException;
-import org.apache.drill.common.types.TypeProtos.MinorType;
-import org.apache.drill.exec.ExecConstants;
 import org.apache.drill.exec.compile.CodeCompiler;
-import org.apache.drill.exec.exception.OutOfMemoryException;
+import org.apache.drill.exec.coord.ClusterCoordinator;
+import org.apache.drill.exec.exception.ClassTransformationException;
+import org.apache.drill.exec.expr.ClassGenerator;
+import org.apache.drill.exec.expr.CodeGenerator;
 import org.apache.drill.exec.expr.fn.FunctionImplementationRegistry;
-import org.apache.drill.exec.expr.holders.ValueHolder;
 import org.apache.drill.exec.memory.BufferAllocator;
 import org.apache.drill.exec.physical.base.PhysicalOperator;
-import org.apache.drill.exec.planner.physical.PlannerSettings;
-import org.apache.drill.exec.proto.BitControl.PlanFragment;
-import org.apache.drill.exec.proto.CoordinationProtos.DrillbitEndpoint;
-import org.apache.drill.exec.proto.ExecProtos.FragmentHandle;
-import org.apache.drill.exec.proto.GeneralRPCProtos.Ack;
-import org.apache.drill.exec.proto.helper.QueryIdHelper;
-import org.apache.drill.exec.rpc.RpcException;
-import org.apache.drill.exec.rpc.RpcOutcomeListener;
-import org.apache.drill.exec.rpc.UserClientConnection;
-import org.apache.drill.exec.rpc.control.ControlTunnel;
-import org.apache.drill.exec.rpc.control.WorkEventBus;
-import org.apache.drill.exec.server.DrillbitContext;
-import org.apache.drill.exec.server.options.FragmentOptionManager;
-import org.apache.drill.exec.server.options.OptionList;
-import org.apache.drill.exec.server.options.OptionManager;
+import org.apache.drill.exec.physical.impl.OperatorCreatorRegistry;
+import org.apache.drill.exec.planner.PhysicalPlanReader;
+import org.apache.drill.exec.proto.CoordinationProtos;
+import org.apache.drill.exec.proto.ExecProtos;
+import org.apache.drill.exec.rpc.control.Controller;
 import org.apache.drill.exec.server.options.OptionSet;
-import org.apache.drill.exec.store.PartitionExplorer;
-import org.apache.drill.exec.store.SchemaConfig;
 import org.apache.drill.exec.testing.ExecutionControls;
-import org.apache.drill.exec.util.ImpersonationUtil;
-import org.apache.drill.exec.work.batch.IncomingBuffers;
-
-import com.google.common.annotations.VisibleForTesting;
-import com.google.common.base.Function;
-import com.google.common.base.Preconditions;
-import com.google.common.collect.Lists;
-import com.google.common.collect.Maps;
 
 import io.netty.buffer.DrillBuf;
+import org.apache.drill.exec.work.batch.IncomingBuffers;
 
 /**
- * Contextual objects required for execution of a particular fragment.
- * This is the implementation; use FragmentContextInterface
- * in code to allow tests to use test-time implementations.
+ * Fragment context interface: separates implementation from definition.
+ * Allows unit testing by mocking or reimplementing services with
+ * test-time versions. The name is awkward, chosen to avoid renaming
+ * the implementation class which is used in many places in legacy code.
+ * New code should use this interface, and the names should eventually
+ * be swapped with {@link FragmentContextImpl} becoming
+ * FragmentContextImpl and this interface becoming
+ * {@link FragmentContextImpl}.
  */
 
-public class FragmentContext extends BaseFragmentContext implements 
AutoCloseable, UdfUtilities, FragmentContextInterface {
-  private static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(FragmentContext.class);
-
-  private final Map tunnels = 
Maps.newHashMap();
-  private final List contexts = Lists.newLinkedList();
-
-  private final DrillbitContext context;
-  private final UserC

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286540#comment-16286540
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156178698
  
--- Diff: 
common/src/main/java/org/apache/drill/common/config/DrillConfig.java ---
@@ -45,13 +45,11 @@
 
   private final ImmutableList startupArguments;
 
-  public static final boolean ON_OSX = 
System.getProperty("os.name").contains("OS X");
-
   @SuppressWarnings("restriction")
   private static final long MAX_DIRECT_MEMORY = 
sun.misc.VM.maxDirectMemory();
 
   @VisibleForTesting
-  public DrillConfig(Config config, boolean enableServerConfigs) {
+  public DrillConfig(Config config) {
--- End diff --

I'm going to guess that the same `DrillConfig` is used (sadly) in the Drill 
client and Drill server. (Sadly because it is odd for a client application to 
need to provide a config file in order to connect to a Drill server. Just adds 
unnecessary complexity, especially for third-party apps that happen to include 
a Drill JDBC driver. Sigh...)

In the client mode, we don't want to load all the internal server configs 
because we'll never use them.

Not sure if this is actually how this flag has been used (maybe it was 
added, then forgotten.) But, it is worth checking the client code to see if it 
was ever used.


> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests: 
>   TestMetadataProvider.tables:153 expected: but was:
>   TestMetadataProvider.tablesWithTableNameFilter:212 expected: but 
> was:
>   TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but 
> was:
>   TestMetadataProvider.tablesWithTableFilter:176 expected: but 
> was:
> Tests in error: 
>   TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR: 
> URISyntaxExcepti...
>   TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR: 
> URISyn...
>   TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth » 
> UserRemote
>   TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344 
> » Rpc
>   TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 » 
> Rpc o...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286534#comment-16286534
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156177751
  
--- Diff: 
contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/MaprDBJsonRecordReader.java
 ---
@@ -117,7 +116,7 @@ public MaprDBJsonRecordReader(MapRDBSubScanSpec 
subScanSpec,
 
 disableCountOptimization = 
formatPluginConfig.disableCountOptimization();
 setColumns(projectedColumns);
-unionEnabled = 
context.getOptions().getOption(ExecConstants.ENABLE_UNION_TYPE);
+unionEnabled = 
context.getOptionSet().getOption(ExecConstants.ENABLE_UNION_TYPE);
--- End diff --

If you are cleaning up this code, consider using a newly added feature of 
the option manager, the ability to get the typed result. Rather than:

```
  public static final BooleanValidator ENABLE_UNION_TYPE = new 
BooleanValidator(ENABLE_UNION_TYPE_KEY);
...
unionEnabled = 
context.getOptionSet().getOption(ExecConstants.ENABLE_UNION_TYPE);
```

We can now do:
```
  public static final String ENABLE_UNION_TYPE_KEY = 
"exec.enable_union_type";
...
unionEnabled = 
context.getOptionSet().getBoolean(ExecConstants.ENABLE_UNION_TYPE_KEY);
```

This method handles value lookup, but also enforces types. (The existing 
code will just trigger an NPE if types don't match.) This new form also hides 
the details of the option validator as recent changes have ensured that the 
validators are already visible to the option manager internals.


> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests: 
>   TestMetadataProvider.tables:153 expected: but was:
>   TestMetadataProvider.tablesWithTableNameFilter:212 expected: but 
> was:
>   TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but 
> was:
>   TestMetadataProvider.tablesWithTableFilter:176 expected: but 
> was:
> Tests in error: 
>   TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR: 
> URISyntaxExcepti...
>   TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR: 
> URISyn...
>   TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth » 
> UserRemote
>   TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344 
> » Rpc
>   TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 » 
> Rpc o...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286536#comment-16286536
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156187558
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java ---
@@ -17,483 +17,216 @@
  */
 package org.apache.drill.exec.ops;
 
+import java.io.IOException;
--- End diff --

Github is very confused by this double rename. I presume the new file just 
has the same contents, but a new name? Used an online diff checker to compare 
the two interfaces. Turns out there are important differences. Comments below.


> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests: 
>   TestMetadataProvider.tables:153 expected: but was:
>   TestMetadataProvider.tablesWithTableNameFilter:212 expected: but 
> was:
>   TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but 
> was:
>   TestMetadataProvider.tablesWithTableFilter:176 expected: but 
> was:
> Tests in error: 
>   TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR: 
> URISyntaxExcepti...
>   TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR: 
> URISyn...
>   TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth » 
> UserRemote
>   TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344 
> » Rpc
>   TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 » 
> Rpc o...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286533#comment-16286533
 ] 

ASF GitHub Bot commented on DRILL-5730:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1045#discussion_r156179325
  
--- Diff: 
contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/MaprDBJsonRecordReader.java
 ---
@@ -117,7 +116,7 @@ public MaprDBJsonRecordReader(MapRDBSubScanSpec 
subScanSpec,
 
 disableCountOptimization = 
formatPluginConfig.disableCountOptimization();
 setColumns(projectedColumns);
-unionEnabled = 
context.getOptions().getOption(ExecConstants.ENABLE_UNION_TYPE);
+unionEnabled = 
context.getOptionSet().getOption(ExecConstants.ENABLE_UNION_TYPE);
--- End diff --

Kind of funny that you have converted all this code to use `OptionSet`. I 
was in the process of deprecating that class since the `OptionManager` seemed 
to work in tests. (The previous dependencies on the Drill server had been 
removed.)

But, since you've done all this work, we might as well embrace `OptionSet` 
as the read-only view of options used by fragments and operators. 
`OptionManager` is used only by code that alters options.


> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests: 
>   TestMetadataProvider.tables:153 expected: but was:
>   TestMetadataProvider.tablesWithTableNameFilter:212 expected: but 
> was:
>   TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but 
> was:
>   TestMetadataProvider.tablesWithTableFilter:176 expected: but 
> was:
> Tests in error: 
>   TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR: 
> URISyntaxExcepti...
>   TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR: 
> URISyn...
>   TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth » 
> UserRemote
>   TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344 
> » Rpc
>   TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 » 
> Rpc o...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5996) Unable to re-run queries from Profiles tab with impersonation and without authentication

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-5996:

Reviewer: Paul Rogers  (was: Karthikeyan Manivannan)

> Unable to re-run queries from Profiles tab with impersonation and without 
> authentication 
> -
>
> Key: DRILL-5996
> URL: https://issues.apache.org/jira/browse/DRILL-5996
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - CLI
>Affects Versions: 1.12.0
>Reporter: Krystal
>Assignee: Arina Ielchiieva
>  Labels: ready-to-commit
> Fix For: 1.13.0
>
>
> On drillbit, enable impersonation without authentication.  From WebUI, click 
> on a query from Profiles link -> click on the "Edit Query" tab -> click on 
> the "Re-Run Query" button.  The query would fail with the following error:
> "HTTP ERROR 412
> Problem accessing /query. Reason:
> User-Name header is not set"
> We should also provide a username dialog box for this tab.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5996) Unable to re-run queries from Profiles tab with impersonation and without authentication

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-5996:

Labels: ready-to-commit  (was: )

> Unable to re-run queries from Profiles tab with impersonation and without 
> authentication 
> -
>
> Key: DRILL-5996
> URL: https://issues.apache.org/jira/browse/DRILL-5996
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - CLI
>Affects Versions: 1.12.0
>Reporter: Krystal
>Assignee: Arina Ielchiieva
>  Labels: ready-to-commit
> Fix For: 1.13.0
>
>
> On drillbit, enable impersonation without authentication.  From WebUI, click 
> on a query from Profiles link -> click on the "Edit Query" tab -> click on 
> the "Re-Run Query" button.  The query would fail with the following error:
> "HTTP ERROR 412
> Problem accessing /query. Reason:
> User-Name header is not set"
> We should also provide a username dialog box for this tab.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (DRILL-6025) Execution time of a running query shown as 'NOT AVAILABLE'

[Prasad Nagaraj Subramanya (JIRA)]

Prasad Nagaraj Subramanya created DRILL-6025:


 Summary: Execution time of a running query shown as 'NOT AVAILABLE'
 Key: DRILL-6025
 URL: https://issues.apache.org/jira/browse/DRILL-6025
 Project: Apache Drill
  Issue Type: Bug
  Components: Client - HTTP
Affects Versions: 1.11.0
Reporter: Prasad Nagaraj Subramanya
Assignee: Prasad Nagaraj Subramanya


When a query is in 'RUNNING' state, the execution time is shown as 'NOT 
AVAILABLE'

We could show the execution duration till the current time



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-6025) Execution time of a running query shown as 'NOT AVAILABLE'

[Prasad Nagaraj Subramanya (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-6025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasad Nagaraj Subramanya updated DRILL-6025:
-
Issue Type: Improvement  (was: Bug)

> Execution time of a running query shown as 'NOT AVAILABLE'
> --
>
> Key: DRILL-6025
> URL: https://issues.apache.org/jira/browse/DRILL-6025
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Client - HTTP
>Affects Versions: 1.11.0
>Reporter: Prasad Nagaraj Subramanya
>Assignee: Prasad Nagaraj Subramanya
>
> When a query is in 'RUNNING' state, the execution time is shown as 'NOT 
> AVAILABLE'
> We could show the execution duration till the current time



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)