[jira] [Resolved] (DRILL-6816) NPE - Concurrent query execution using PreparedStatement
[
https://issues.apache.org/jira/browse/DRILL-6816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vitalii Diravka resolved DRILL-6816.
Resolution: Invalid
Assignee: Vitalii Diravka
I have already checked and I confirm {{stmt.executeQuery()}} works just fine.
> NPE - Concurrent query execution using PreparedStatement
> -
>
> Key: DRILL-6816
> URL: https://issues.apache.org/jira/browse/DRILL-6816
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.14.0
>Reporter: Khurram Faraaz
>Assignee: Vitalii Diravka
>Priority: Major
> Attachments: test_tbl.json
>
>
> Concurrent query execution from JDBC program using PreparedStatement results
> in NPE.
> Queries that were executed concurrently are (part of a query file),
> {noformat}
> select id from `test_tbl.json`
> select count(id) from `test_tbl.json`
> select count(*) from `test_tbl.json`
> select * from `test_tbl.json`
> {noformat}
> Drill 1.14.0
> git.commit.id=35a1ae23c9b280b9e73cb0f6f01808c996515454
> MapR version => 6.1.0.20180911143226.GA (secure cluster)
> JDBC driver used was org.apache.drill.jdbc.Driver
> Executing the above queries concurrently using a Statement object results in
> successful query execution.
> {noformat}
> Statement stmt = conn.createStatement();
> ResultSet rs = stmt.executeQuery(query);
> {noformat}
> However, when the same queries listed above are executed using a
> PreparedStatement object we see an NPE
> {noformat}
> PreparedStatement prdstmnt = conn.prepareStatement(query);
> prdstmnt.executeUpdate();
> {noformat}
> Stack trace from drillbit.log
> {noformat}
> 17:04:32.941 [pool-1-thread-3] DEBUG o.a.d.j.impl.DrillStatementRegistry -
> Adding to open-statements registry:
> org.apache.drill.jdbc.impl.DrillJdbc41Factory$DrillJdbc41PreparedStatement@35757005
> 17:04:32.941 [pool-1-thread-2] DEBUG o.a.d.j.impl.DrillStatementRegistry -
> Adding to open-statements registry:
> org.apache.drill.jdbc.impl.DrillJdbc41Factory$DrillJdbc41PreparedStatement@d4413b8
> 17:04:32.956 [pool-1-thread-1] DEBUG o.a.d.j.impl.DrillStatementRegistry -
> Adding to open-statements registry:
> org.apache.drill.jdbc.impl.DrillJdbc41Factory$DrillJdbc41PreparedStatement@5eb3b9ab
> 17:04:32.956 [pool-1-thread-4] DEBUG o.a.d.j.impl.DrillStatementRegistry -
> Adding to open-statements registry:
> org.apache.drill.jdbc.impl.DrillJdbc41Factory$DrillJdbc41PreparedStatement@d9367d0
> java.lang.NullPointerException
> at java.util.Objects.requireNonNull(Objects.java:203)
> at org.apache.calcite.avatica.Meta$MetaResultSet.create(Meta.java:577)
> at org.apache.drill.jdbc.impl.DrillMetaImpl.execute(DrillMetaImpl.java:1143)
> at org.apache.drill.jdbc.impl.DrillMetaImpl.execute(DrillMetaImpl.java:1150)
> at
> org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:511)
> at
> org.apache.calcite.avatica.AvaticaPreparedStatement.executeLargeUpdate(AvaticaPreparedStatement.java:146)
> at
> org.apache.drill.jdbc.impl.DrillPreparedStatementImpl.executeLargeUpdate(DrillPreparedStatementImpl.java:512)
> at
> org.apache.calcite.avatica.AvaticaPreparedStatement.executeUpdate(AvaticaPreparedStatement.java:142)
> at RunQuery.executeQuery(RunQuery.java:61)
> at RunQuery.run(RunQuery.java:30)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6691) Unify checkstyle-config.xml files
[
https://issues.apache.org/jira/browse/DRILL-6691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690209#comment-16690209
]
Vitalii Diravka commented on DRILL-6691:
[~hanu.ncr] What is the status of it? I suppose we can just remove
\{{checkstyle-config.xml}} for \{{format-maprdb}} package, so then the only one
check-style will be applicable for the whole project.
> Unify checkstyle-config.xml files
> -
>
> Key: DRILL-6691
> URL: https://issues.apache.org/jira/browse/DRILL-6691
> Project: Apache Drill
> Issue Type: Task
>Reporter: Volodymyr Vysotskyi
>Assignee: Hanumath Rao Maduri
>Priority: Minor
> Fix For: 1.15.0
>
>
> Currently, `drill-root` and `format-maprdb` modules contain
> `checkstyle-config.xml` own files.
> They should be unified to apply the same checkstyle rules for all modules.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6806) Start moving code for handling a partition in HashAgg into a separate class
[ https://issues.apache.org/jira/browse/DRILL-6806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690208#comment-16690208 ] ASF GitHub Bot commented on DRILL-6806: --- ilooner commented on issue #1515: DRILL-6806: Moving code for a HashAgg partition into separate class. URL: https://github.com/apache/drill/pull/1515#issuecomment-439581445 @vdiravka still need to address @Ben-Zvi 's comments. I will try to get it done early next week. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Start moving code for handling a partition in HashAgg into a separate class > --- > > Key: DRILL-6806 > URL: https://issues.apache.org/jira/browse/DRILL-6806 > Project: Apache Drill > Issue Type: Sub-task >Reporter: Timothy Farkas >Assignee: Timothy Farkas >Priority: Major > Fix For: 1.15.0 > > > Since this involves a lot of refactoring this will be a multiple PR effort. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6253) HashAgg Unit Testing And Refactoring
[ https://issues.apache.org/jira/browse/DRILL-6253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690201#comment-16690201 ] Vitalii Diravka commented on DRILL-6253: [~timothyfarkas] Do we want to have all sub-tasks to be included in 1.15 release? > HashAgg Unit Testing And Refactoring > > > Key: DRILL-6253 > URL: https://issues.apache.org/jira/browse/DRILL-6253 > Project: Apache Drill > Issue Type: Improvement >Reporter: Timothy Farkas >Assignee: Timothy Farkas >Priority: Major > Fix For: 1.15.0 > > > This is a parent issue to hold all the subtasks required to refactor HashAgg > to make it unit testable. Design doc > https://docs.google.com/document/d/110BAWg3QXMfdmuqB0p3HuaoKpPGY-lqCRtHFxdh53Ds/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6253) HashAgg Unit Testing And Refactoring
[ https://issues.apache.org/jira/browse/DRILL-6253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vitalii Diravka updated DRILL-6253: --- Labels: (was: ready-to-commit) > HashAgg Unit Testing And Refactoring > > > Key: DRILL-6253 > URL: https://issues.apache.org/jira/browse/DRILL-6253 > Project: Apache Drill > Issue Type: Improvement >Reporter: Timothy Farkas >Assignee: Timothy Farkas >Priority: Major > Fix For: 1.15.0 > > > This is a parent issue to hold all the subtasks required to refactor HashAgg > to make it unit testable. Design doc > https://docs.google.com/document/d/110BAWg3QXMfdmuqB0p3HuaoKpPGY-lqCRtHFxdh53Ds/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6806) Start moving code for handling a partition in HashAgg into a separate class
[ https://issues.apache.org/jira/browse/DRILL-6806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690200#comment-16690200 ] ASF GitHub Bot commented on DRILL-6806: --- vdiravka commented on issue #1515: DRILL-6806: Moving code for a HashAgg partition into separate class. URL: https://github.com/apache/drill/pull/1515#issuecomment-439579902 @ilooner Could we proceed with this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Start moving code for handling a partition in HashAgg into a separate class > --- > > Key: DRILL-6806 > URL: https://issues.apache.org/jira/browse/DRILL-6806 > Project: Apache Drill > Issue Type: Sub-task >Reporter: Timothy Farkas >Assignee: Timothy Farkas >Priority: Major > Fix For: 1.15.0 > > > Since this involves a lot of refactoring this will be a multiple PR effort. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6844) Query with ORDER BY DESC on indexed column does not pick secondary index
[
https://issues.apache.org/jira/browse/DRILL-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690183#comment-16690183
]
ASF GitHub Bot commented on DRILL-6844:
---
vdiravka closed pull request #1534: DRILL-6844: Query with ORDER BY DESC on
indexed column does not pick …
URL: https://github.com/apache/drill/pull/1534
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/contrib/format-maprdb/src/main/java/org/apache/drill/exec/planner/index/MapRDBIndexDiscover.java
b/contrib/format-maprdb/src/main/java/org/apache/drill/exec/planner/index/MapRDBIndexDiscover.java
index f828ba02daf..d949634c815 100644
---
a/contrib/format-maprdb/src/main/java/org/apache/drill/exec/planner/index/MapRDBIndexDiscover.java
+++
b/contrib/format-maprdb/src/main/java/org/apache/drill/exec/planner/index/MapRDBIndexDiscover.java
@@ -293,9 +293,9 @@ private LogicalExpression getIndexExpression(IndexFieldDesc
desc) throws Invalid
if (direction != null) {
// assume null direction of NULLS UNSPECIFIED for now until MapR-DB
adds that to the APIs
RelFieldCollation.NullDirection nulldir =
-desc.getMissingAndNullOrdering() ==
MissingAndNullOrdering.MissingAndNullFirst ? NullDirection.FIRST :
-(desc.getMissingAndNullOrdering() ==
MissingAndNullOrdering.MissingAndNullLast ?
-NullDirection.LAST : NullDirection.UNSPECIFIED);
+direction == RelFieldCollation.Direction.ASCENDING ?
NullDirection.LAST :
+(direction == RelFieldCollation.Direction.DESCENDING ?
+NullDirection.FIRST : NullDirection.UNSPECIFIED);
RelFieldCollation c = new RelFieldCollation(i++, direction, nulldir);
fieldCollations.add(c);
} else {
diff --git
a/contrib/format-maprdb/src/test/java/com/mapr/drill/maprdb/tests/index/IndexPlanTest.java
b/contrib/format-maprdb/src/test/java/com/mapr/drill/maprdb/tests/index/IndexPlanTest.java
index 67542207118..fd3b5cca169 100644
---
a/contrib/format-maprdb/src/test/java/com/mapr/drill/maprdb/tests/index/IndexPlanTest.java
+++
b/contrib/format-maprdb/src/test/java/com/mapr/drill/maprdb/tests/index/IndexPlanTest.java
@@ -106,6 +106,7 @@ public static void setupTableIndexes() throws Exception {
{"i_ssn", "id.ssn", "contact.phone",
"i_state_city", "address.state,address.city",
"name.fname,name.lname",//mainly for composite key test
"i_age", "personal.age", "",
+"i_age_desc", "personal.age:desc", "",
"i_income", "personal.income", "",
"i_lic", "driverlicense", "reverseid",
"i_state_city_dl", "address.state,address.city", "driverlicense",
@@ -1935,27 +1936,58 @@ public void testRowkeyJoinPushdown_12() throws
Exception {
public void testRowkeyJoinPushdown_13() throws Exception {
// Check option planner.rowkeyjoin_conversion_using_hashjoin works as
expected!
String query = "select t1.id.ssn as ssn from hbase.`index_test_primary` t1
where _id in (select t2._id " +
-" from hbase.`index_test_primary` t2 where
cast(t2.activity.irs.firstlogin as timestamp) = " +
-" to_timestamp('2013-02-04 22:34:38.0', '-MM-dd HH:mm:ss.S'))";
+" from hbase.`index_test_primary` t2 where
cast(t2.activity.irs.firstlogin as timestamp) = " +
+" to_timestamp('2013-02-04 22:34:38.0', '-MM-dd HH:mm:ss.S'))";
try {
test(incrRowKeyJoinConvSelThreshold + ";" +
lowNonCoveringSelectivityThreshold + ";");
- PlanTestBase.testPlanMatchingPatterns(query, new String[]
{"RowKeyJoin"}, new String[] {});
+ PlanTestBase.testPlanMatchingPatterns(query, new String[]{"RowKeyJoin"},
new String[]{});
testBuilder()
- .sqlQuery(query)
- .ordered()
- .baselineColumns("ssn").baselineValues("17423")
- .go();
+ .sqlQuery(query)
+ .ordered()
+ .baselineColumns("ssn").baselineValues("17423")
+ .go();
test(incrRowKeyJoinConvSelThreshold + ";" +
lowNonCoveringSelectivityThreshold + ";" +
- forceRowKeyJoinConversionUsingHashJoin + ";");
- PlanTestBase.testPlanMatchingPatterns(query, new String[] {"HashJoin"},
new String[] {"RowKeyJoin"});
+ forceRowKeyJoinConversionUsingHashJoin + ";");
+ PlanTestBase.testPlanMatchingPatterns(query, new String[]{"HashJoin"},
new String[]{"RowKeyJoin"});
testBuilder()
- .sqlQuery(query)
- .ordered()
- .baselineColumns("ssn").baselineValues("17423")
- .go();
+ .sqlQuery(query)
+ .ordered()
+
[jira] [Updated] (DRILL-6845) Eliminate duplicates for Semi Hash Join
[ https://issues.apache.org/jira/browse/DRILL-6845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vitalii Diravka updated DRILL-6845: --- Issue Type: Improvement (was: Sub-task) Parent: (was: DRILL-6735) > Eliminate duplicates for Semi Hash Join > --- > > Key: DRILL-6845 > URL: https://issues.apache.org/jira/browse/DRILL-6845 > Project: Apache Drill > Issue Type: Improvement > Components: Execution - Relational Operators >Affects Versions: 1.14.0 >Reporter: Boaz Ben-Zvi >Assignee: Boaz Ben-Zvi >Priority: Minor > Fix For: 1.15.0 > > > Following DRILL-6735: The performance of the new Semi Hash Join may degrade > if the build side contains excessive number of join-key duplicate rows; this > mainly a result of the need to store all those rows first, before the hash > table is built. > Proposed solution: For Semi, the Hash Agg would create a Hash-Table > initially, and use it to eliminate key-duplicate rows as they arrive. > Proposed extra: That Hash-Table has an added cost (e.g. resizing). So > perform "runtime stats" – Check initial number of incoming rows (e.g. 32k), > and if the number of duplicates is less than some threshold (e.g. %20) – > cancel that "early" hash table. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6839) Failed to plan (aggregate + Hash or NL join) when slice target is low
[
https://issues.apache.org/jira/browse/DRILL-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690149#comment-16690149
]
Aman Sinha commented on DRILL-6839:
---
[~vvysotskyi] it is certainly possible to change the `StreamAggPrule` to
generate single-phase agg if for some reason the 2 phase agg is not possible.
It would be a workaround but the performance will be poor because after a
`NestedLoopJoin` the output row count would increase substantially due to the
cross join, so we should ideally do local aggregation first before sending all
those rows to the foreman node. After doing local aggregation, we are only
sending M rows to the foreman where M is the number of minor fragments, whereas
otherwise it will send `O(N^2)` rows to the foreman where N is the number of
input rows.
So, it would be useful to determine why the 2 phase StreamingAgg was not
created in this example. Looking at the code, it is creating the distribution
traits using the distribution traits of the child. If there is no GROUP-BY in
the aggregation, then I would expect that DrillDistributionTrait.ANY should be
sufficient. It would be good to check if that is happening.
> Failed to plan (aggregate + Hash or NL join) when slice target is low
> --
>
> Key: DRILL-6839
> URL: https://issues.apache.org/jira/browse/DRILL-6839
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Igor Guzenko
>Priority: Major
> Fix For: 1.16.0
>
>
> *Case 1.* When nested loop join is about to be used:
> - Option "_planner.enable_nljoin_for_scalar_only_" is set to false
> - Option "_planner.slice_target_" is set to low value for imitation of big
> input tables
>
> {code:java}
> @Category(SqlTest.class)
> public class CrossJoinTest extends ClusterTest {
> @BeforeClass
> public static void setUp() throws Exception {
> startCluster(ClusterFixture.builder(dirTestWatcher));
> }
> @Test
> public void testCrossJoinSucceedsForLowSliceTarget() throws Exception {
>try {
> client.alterSession(PlannerSettings.NLJOIN_FOR_SCALAR.getOptionName(),
> false);
> client.alterSession(ExecConstants.SLICE_TARGET, 1);
> queryBuilder().sql(
> "SELECT COUNT(l.nation_id) " +
> "FROM cp.`tpch/nation.parquet` l " +
> ", cp.`tpch/region.parquet` r")
> .run();
>} finally {
> client.resetSession(ExecConstants.SLICE_TARGET);
> client.resetSession(PlannerSettings.NLJOIN_FOR_SCALAR.getOptionName());
>}
> }
> }{code}
>
> *Case 2.* When hash join is about to be used:
> - Option "planner.enable_mergejoin" is set to false, so hash join will be
> used instead
> - Option "planner.slice_target" is set to low value for imitation of big
> input tables
> - Comment out //ruleList.add(HashJoinPrule.DIST_INSTANCE); in
> PlannerPhase.getPhysicalRules method
> {code:java}
> @Category(SqlTest.class)
> public class CrossJoinTest extends ClusterTest {
> @BeforeClass
> public static void setUp() throws Exception {
>startCluster(ClusterFixture.builder(dirTestWatcher));
> }
> @Test
> public void testInnerJoinSucceedsForLowSliceTarget() throws Exception {
>try {
> client.alterSession(PlannerSettings.MERGEJOIN.getOptionName(), false);
> client.alterSession(ExecConstants.SLICE_TARGET, 1);
> queryBuilder().sql(
> "SELECT COUNT(l.nation_id) " +
> "FROM cp.`tpch/nation.parquet` l " +
> "INNER JOIN cp.`tpch/region.parquet` r " +
> "ON r.nation_id = l.nation_id")
> .run();
>} finally {
> client.resetSession(ExecConstants.SLICE_TARGET);
> client.resetSession(PlannerSettings.MERGEJOIN.getOptionName());
>}
> }
> }
> {code}
>
> *Workaround:* To avoid the exception we need to set option
> "_planner.enable_multiphase_agg_" to false. By doing this we avoid
> unsuccessful attempts to create 2 phase aggregation plan in StreamAggPrule
> and guarantee that logical aggregate will be converted to physical one.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6816) NPE - Concurrent query execution using PreparedStatement
[
https://issues.apache.org/jira/browse/DRILL-6816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690157#comment-16690157
]
Khurram Faraaz commented on DRILL-6816:
---
[~vitalii] I haven't tried with executeQuery method yet, I will run the test
using executeQuery and share results here.
> NPE - Concurrent query execution using PreparedStatement
> -
>
> Key: DRILL-6816
> URL: https://issues.apache.org/jira/browse/DRILL-6816
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.14.0
>Reporter: Khurram Faraaz
>Priority: Major
> Attachments: test_tbl.json
>
>
> Concurrent query execution from JDBC program using PreparedStatement results
> in NPE.
> Queries that were executed concurrently are (part of a query file),
> {noformat}
> select id from `test_tbl.json`
> select count(id) from `test_tbl.json`
> select count(*) from `test_tbl.json`
> select * from `test_tbl.json`
> {noformat}
> Drill 1.14.0
> git.commit.id=35a1ae23c9b280b9e73cb0f6f01808c996515454
> MapR version => 6.1.0.20180911143226.GA (secure cluster)
> JDBC driver used was org.apache.drill.jdbc.Driver
> Executing the above queries concurrently using a Statement object results in
> successful query execution.
> {noformat}
> Statement stmt = conn.createStatement();
> ResultSet rs = stmt.executeQuery(query);
> {noformat}
> However, when the same queries listed above are executed using a
> PreparedStatement object we see an NPE
> {noformat}
> PreparedStatement prdstmnt = conn.prepareStatement(query);
> prdstmnt.executeUpdate();
> {noformat}
> Stack trace from drillbit.log
> {noformat}
> 17:04:32.941 [pool-1-thread-3] DEBUG o.a.d.j.impl.DrillStatementRegistry -
> Adding to open-statements registry:
> org.apache.drill.jdbc.impl.DrillJdbc41Factory$DrillJdbc41PreparedStatement@35757005
> 17:04:32.941 [pool-1-thread-2] DEBUG o.a.d.j.impl.DrillStatementRegistry -
> Adding to open-statements registry:
> org.apache.drill.jdbc.impl.DrillJdbc41Factory$DrillJdbc41PreparedStatement@d4413b8
> 17:04:32.956 [pool-1-thread-1] DEBUG o.a.d.j.impl.DrillStatementRegistry -
> Adding to open-statements registry:
> org.apache.drill.jdbc.impl.DrillJdbc41Factory$DrillJdbc41PreparedStatement@5eb3b9ab
> 17:04:32.956 [pool-1-thread-4] DEBUG o.a.d.j.impl.DrillStatementRegistry -
> Adding to open-statements registry:
> org.apache.drill.jdbc.impl.DrillJdbc41Factory$DrillJdbc41PreparedStatement@d9367d0
> java.lang.NullPointerException
> at java.util.Objects.requireNonNull(Objects.java:203)
> at org.apache.calcite.avatica.Meta$MetaResultSet.create(Meta.java:577)
> at org.apache.drill.jdbc.impl.DrillMetaImpl.execute(DrillMetaImpl.java:1143)
> at org.apache.drill.jdbc.impl.DrillMetaImpl.execute(DrillMetaImpl.java:1150)
> at
> org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:511)
> at
> org.apache.calcite.avatica.AvaticaPreparedStatement.executeLargeUpdate(AvaticaPreparedStatement.java:146)
> at
> org.apache.drill.jdbc.impl.DrillPreparedStatementImpl.executeLargeUpdate(DrillPreparedStatementImpl.java:512)
> at
> org.apache.calcite.avatica.AvaticaPreparedStatement.executeUpdate(AvaticaPreparedStatement.java:142)
> at RunQuery.executeQuery(RunQuery.java:61)
> at RunQuery.run(RunQuery.java:30)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-3933) Error execute select command line sqlline -u -q
[ https://issues.apache.org/jira/browse/DRILL-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vitalii Diravka updated DRILL-3933: --- Labels: doc-impacting ready-to-commit (was: ready-to-commit) > Error execute select command line sqlline -u -q > --- > > Key: DRILL-3933 > URL: https://issues.apache.org/jira/browse/DRILL-3933 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.1.0 >Reporter: Jon >Assignee: Arina Ielchiieva >Priority: Major > Labels: doc-impacting, ready-to-commit > Fix For: 1.15.0 > > > I'm newbie with Drill and Jira, so sorry if this is not the correct site. > When I query : "sqlline -u 'jdbc:drill:drillbit=localhost' -q 'select * from > hive.database.table;' " return: > "select anaconda-ks.cfg build.out install.log install.log.syslog > ranger_tutorial sandbox.info start_ambari.sh start_hbase.sh start_solr.sh > stop_solr.sh from hive.database.table;" > Error: PARSE ERROR: Encountered "." at line 1, column 29. > Was expecting one of: > "FROM" ... > "," ... > So, to fix this, i should type all columns to do this one work. > But, if I used UI, in localhost:8047/query, the query works. Drill is > connected to Hive with plugin of course. Is this a bug or something? Bad > conf.? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6844) Query with ORDER BY DESC on indexed column does not pick secondary index
[
https://issues.apache.org/jira/browse/DRILL-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689927#comment-16689927
]
ASF GitHub Bot commented on DRILL-6844:
---
HanumathRao commented on issue #1534: DRILL-6844: Query with ORDER BY DESC on
indexed column does not pick …
URL: https://github.com/apache/drill/pull/1534#issuecomment-439502816
@vdiravka I have resolved the conflict and done the rebase.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Query with ORDER BY DESC on indexed column does not pick secondary index
>
>
> Key: DRILL-6844
> URL: https://issues.apache.org/jira/browse/DRILL-6844
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning Optimization
>Affects Versions: 1.14.0
>Reporter: Hanumath Rao Maduri
>Assignee: Hanumath Rao Maduri
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.15.0
>
>
> Query with ORDER BY DESC on indexed column does not pick secondary index
> {noformat}
> // Query that uses the secondary index defined on ts.
> 0: jdbc:drill:schema=dfs.tmp> explain plan for
> . . . . . . . . . . . . . . > select ts from dfs.`/c8/test3` order by ts
> limit 1;
> +--+--+
> | text | json |
> +--+--+
> | 00-00 Screen
> 00-01 Project(ts=[$0])
> 00-02 SelectionVectorRemover
> 00-03 Limit(fetch=[1])
> 00-04 Scan(table=[[dfs, /c8/test3]], groupscan=[JsonTableGroupScan
> [ScanSpec=JsonScanSpec [tableName=maprfs:///c8/test3, condition=null,
> indexName=ts], columns=[`ts`], limit=1, maxwidth=125]])
> {noformat}
> // Same query with ORDER BY ts DESC does not use the secondary index defined
> on ts.
> 0: jdbc:drill:schema=dfs.tmp> explain plan for
> . . . . . . . . . . . . . . > select ts from dfs.`/c8/test3` order by ts desc
> limit 1;
> +--+--+
> | text | json |
> +--+--+
> | 00-00 Screen
> 00-01 Project(ts=[$0])
> 00-02 SelectionVectorRemover
> 00-03 Limit(fetch=[1])
> 00-04 SingleMergeExchange(sort0=[0 DESC])
> 01-01 OrderedMuxExchange(sort0=[0 DESC])
> 02-01 SelectionVectorRemover
> 02-02 Limit(fetch=[1])
> 02-03 SelectionVectorRemover
> 02-04 TopN(limit=[1])
> 02-05 HashToRandomExchange(dist0=[[$0]])
> 03-01 Scan(table=[[dfs, /c8/test3]], groupscan=[JsonTableGroupScan
> [ScanSpec=JsonScanSpec [tableName=maprfs:///c8/test3, condition=null],
> columns=[`ts`], maxwidth=8554]])
> {noformat}
> { noformat}
> Index definition is,
> maprcli table index list -path /c8/test3 -json
> {
> "timestamp":1538066303932,
> "timeofday":"2018-09-27 04:38:23.932 GMT+ PM",
> "status":"OK",
> "total":2,
> "data":[
> {
> "cluster":"c8",
> "type":"maprdb.si",
> "indexFid":"2176.68.131294",
> "indexName":"ts",
> "hashed":false,
> "indexState":"REPLICA_STATE_REPLICATING",
> "idx":1,
> "indexedFields":"ts:ASC",
> "isUptodate":false,
> "minPendingTS":1538066077,
> "maxPendingTS":1538066077,
> "bytesPending":0,
> "putsPending":0,
> "bucketsPending":1,
> "copyTableCompletionPercentage":100,
> "numTablets":32,
> "numRows":80574368,
> "totalSize":4854052160
> },
> {
> "cluster":"c8",
> "type":"maprdb.si",
> "indexFid":"2176.72.131302",
> "indexName":"ts_desc",
> "hashed":false,
> "indexState":"REPLICA_STATE_REPLICATING",
> "idx":2,
> "indexedFields":"ts:DESC",
> "isUptodate":false,
> "minPendingTS":1538066077,
> "maxPendingTS":1538066077,
> "bytesPending":0,
> "putsPending":0,
> "bucketsPending":1,
> "copyTableCompletionPercentage":100,
> "numTablets":32,
> "numRows":80081344,
> "totalSize":4937154560
> }
> ]
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6039) drillbit.sh graceful_stop does not wait for fragments to complete before stopping the drillbit
[
https://issues.apache.org/jira/browse/DRILL-6039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689908#comment-16689908
]
ASF GitHub Bot commented on DRILL-6039:
---
dvjyothsna commented on a change in pull request #1536: DRILL-6039: Fixed
drillbit.sh script to do graceful shutdown
URL: https://github.com/apache/drill/pull/1536#discussion_r234318353
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/server/Drillbit.java
##
@@ -285,6 +286,10 @@ public synchronized void close() {
if (storeProvider != profileStoreProvider) {
AutoCloseables.close(profileStoreProvider);
}
+ File f = new File(System.getenv("DRILL_PID_DIR") + "/.graceful");
Review comment:
Can we get that config param in the script?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> drillbit.sh graceful_stop does not wait for fragments to complete before
> stopping the drillbit
> --
>
> Key: DRILL-6039
> URL: https://issues.apache.org/jira/browse/DRILL-6039
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
>Affects Versions: 1.3.0
>Reporter: Krystal
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
> Fix For: 1.15.0
>
>
> git.commit.id.abbrev=eb0c403
> I have 3-nodes cluster with drillbits running on each node. I kicked off a
> long running query. In the middle of the query, I did a "./drillbit.sh
> graceful_stop" on one of the non-foreman node. The node was stopped within a
> few seconds and the query failed with error:
> Error: SYSTEM ERROR: IOException: Filesystem closed
> Fragment 4:15
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6039) drillbit.sh graceful_stop does not wait for fragments to complete before stopping the drillbit
[
https://issues.apache.org/jira/browse/DRILL-6039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689793#comment-16689793
]
ASF GitHub Bot commented on DRILL-6039:
---
sohami commented on a change in pull request #1536: DRILL-6039: Fixed
drillbit.sh script to do graceful shutdown
URL: https://github.com/apache/drill/pull/1536#discussion_r234070327
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/server/Drillbit.java
##
@@ -285,6 +286,10 @@ public synchronized void close() {
if (storeProvider != profileStoreProvider) {
AutoCloseables.close(profileStoreProvider);
}
+ File f = new File(System.getenv("DRILL_PID_DIR") + "/.graceful");
Review comment:
Provide this as a configuration parameter with default value being `graceful`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> drillbit.sh graceful_stop does not wait for fragments to complete before
> stopping the drillbit
> --
>
> Key: DRILL-6039
> URL: https://issues.apache.org/jira/browse/DRILL-6039
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
>Affects Versions: 1.3.0
>Reporter: Krystal
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
> Fix For: 1.15.0
>
>
> git.commit.id.abbrev=eb0c403
> I have 3-nodes cluster with drillbits running on each node. I kicked off a
> long running query. In the middle of the query, I did a "./drillbit.sh
> graceful_stop" on one of the non-foreman node. The node was stopped within a
> few seconds and the query failed with error:
> Error: SYSTEM ERROR: IOException: Filesystem closed
> Fragment 4:15
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6039) drillbit.sh graceful_stop does not wait for fragments to complete before stopping the drillbit
[ https://issues.apache.org/jira/browse/DRILL-6039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689795#comment-16689795 ] ASF GitHub Bot commented on DRILL-6039: --- sohami commented on a change in pull request #1536: DRILL-6039: Fixed drillbit.sh script to do graceful shutdown URL: https://github.com/apache/drill/pull/1536#discussion_r233586484 ## File path: distribution/src/resources/drillbit.sh ## @@ -94,11 +94,24 @@ waitForProcessEnd() commandName=$2 kill_drillbit=$3 processedAt=`date +%s` + triggered_shutdown=false + FILE=$DRILL_PID_DIR/.graceful Review comment: Let's not make it a hidden file. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > drillbit.sh graceful_stop does not wait for fragments to complete before > stopping the drillbit > -- > > Key: DRILL-6039 > URL: https://issues.apache.org/jira/browse/DRILL-6039 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow >Affects Versions: 1.3.0 >Reporter: Krystal >Assignee: Venkata Jyothsna Donapati >Priority: Major > Fix For: 1.15.0 > > > git.commit.id.abbrev=eb0c403 > I have 3-nodes cluster with drillbits running on each node. I kicked off a > long running query. In the middle of the query, I did a "./drillbit.sh > graceful_stop" on one of the non-foreman node. The node was stopped within a > few seconds and the query failed with error: > Error: SYSTEM ERROR: IOException: Filesystem closed > Fragment 4:15 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6039) drillbit.sh graceful_stop does not wait for fragments to complete before stopping the drillbit
[
https://issues.apache.org/jira/browse/DRILL-6039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689794#comment-16689794
]
ASF GitHub Bot commented on DRILL-6039:
---
sohami commented on a change in pull request #1536: DRILL-6039: Fixed
drillbit.sh script to do graceful shutdown
URL: https://github.com/apache/drill/pull/1536#discussion_r234070509
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/server/Drillbit.java
##
@@ -285,6 +286,10 @@ public synchronized void close() {
if (storeProvider != profileStoreProvider) {
AutoCloseables.close(profileStoreProvider);
}
+ File f = new File(System.getenv("DRILL_PID_DIR") + "/.graceful");
+ if (f.exists()) {
+f.delete();
Review comment:
check and log warning based on the return value of `f.delete()`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> drillbit.sh graceful_stop does not wait for fragments to complete before
> stopping the drillbit
> --
>
> Key: DRILL-6039
> URL: https://issues.apache.org/jira/browse/DRILL-6039
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
>Affects Versions: 1.3.0
>Reporter: Krystal
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
> Fix For: 1.15.0
>
>
> git.commit.id.abbrev=eb0c403
> I have 3-nodes cluster with drillbits running on each node. I kicked off a
> long running query. In the middle of the query, I did a "./drillbit.sh
> graceful_stop" on one of the non-foreman node. The node was stopped within a
> few seconds and the query failed with error:
> Error: SYSTEM ERROR: IOException: Filesystem closed
> Fragment 4:15
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6039) drillbit.sh graceful_stop does not wait for fragments to complete before stopping the drillbit
[
https://issues.apache.org/jira/browse/DRILL-6039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689792#comment-16689792
]
ASF GitHub Bot commented on DRILL-6039:
---
sohami commented on a change in pull request #1536: DRILL-6039: Fixed
drillbit.sh script to do graceful shutdown
URL: https://github.com/apache/drill/pull/1536#discussion_r234069987
##
File path: distribution/src/resources/drillbit.sh
##
@@ -94,11 +94,24 @@ waitForProcessEnd()
commandName=$2
kill_drillbit=$3
processedAt=`date +%s`
+ triggered_shutdown=false
+ FILE=$DRILL_PID_DIR/.graceful
origcnt=${DRILL_STOP_TIMEOUT:-120}
while kill -0 $pidKilled > /dev/null 2>&1;
do
echo -n "."
sleep 1;
+ #Incase of graceful shutdown, create .graceful file and wait till it is
deleted to trigger kill command.
+ if [ "$kill_drillbit" = false ]; then
+ if [ "$triggered_shutdown" = false ]; then
+ touch $DRILL_PID_DIR/.graceful
+ triggered_shutdown=true
+ else
+ if [ ! -f "$FILE" ]; then
+ kill $pidKilled > /dev/null 2>&1;
Review comment:
this shouldn't be required since after `close` Drillbit process should exit.
If not then we should find out the reason behind it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> drillbit.sh graceful_stop does not wait for fragments to complete before
> stopping the drillbit
> --
>
> Key: DRILL-6039
> URL: https://issues.apache.org/jira/browse/DRILL-6039
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
>Affects Versions: 1.3.0
>Reporter: Krystal
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
> Fix For: 1.15.0
>
>
> git.commit.id.abbrev=eb0c403
> I have 3-nodes cluster with drillbits running on each node. I kicked off a
> long running query. In the middle of the query, I did a "./drillbit.sh
> graceful_stop" on one of the non-foreman node. The node was stopped within a
> few seconds and the query failed with error:
> Error: SYSTEM ERROR: IOException: Filesystem closed
> Fragment 4:15
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6039) drillbit.sh graceful_stop does not wait for fragments to complete before stopping the drillbit
[
https://issues.apache.org/jira/browse/DRILL-6039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689791#comment-16689791
]
ASF GitHub Bot commented on DRILL-6039:
---
sohami commented on a change in pull request #1536: DRILL-6039: Fixed
drillbit.sh script to do graceful shutdown
URL: https://github.com/apache/drill/pull/1536#discussion_r234301751
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/work/WorkManager.java
##
@@ -433,6 +442,22 @@ public void run() {
public StatusThread() {
// assume this thread is created by a non-daemon thread
setName("WorkManager.StatusThread");
+ }
+ private void pollShutdown(Drillbit drillbit) throws IOException,
InterruptedException {
+ final Path path =
FileSystems.getDefault().getPath(System.getenv("DRILL_PID_DIR"));
+ try (final WatchService watchService =
FileSystems.getDefault().newWatchService()) {
+path.register(watchService, StandardWatchEventKinds.ENTRY_MODIFY,
StandardWatchEventKinds.ENTRY_CREATE);
+while (true) {
+ final WatchKey wk = watchService.take();
Review comment:
This design is not correct and has issues.
- You are using `WorkManager` `StatusThread` for registering a
`WatchService` and then doing a blocking call `take()` which will make this
thread stuck until there is any event seen by `WatchService`. Hence
`StatusThread` will not be able to perform it's actual job of sending status of
`runningFragments` to it's `Foreman`.
- The `pollShutdown` method is calling `Drillbit.close()` which is in
`StatusThread`. Now `Drillbit.close() `calls close on `WorkManager` and which
calls `interrupt` on `StatusThread`. There is a cycle here. What it means to
call interrupt on itself ? Usually interrupt is used by another thread to wake
up a blocking thread. I think based on interrupt call the interrupt flag of
status thread will be set. Then when it comes out of `pollShutdown` and calls
sleep it will hit interrupted exception. It might work but I don't like the
idea where WorkManager which is part of Drillbit has a thread which calling
`Drillbit.close()`.
Please create a separate thread within Drillbit class itself to do this. But
in `Drillbit::close()` method you need to still `interrupt` this new thread for
cases when `close` it called from some other path. But you can maintain a flag
which will indicate if `Thread.interrupt()` should be called or not.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> drillbit.sh graceful_stop does not wait for fragments to complete before
> stopping the drillbit
> --
>
> Key: DRILL-6039
> URL: https://issues.apache.org/jira/browse/DRILL-6039
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
>Affects Versions: 1.3.0
>Reporter: Krystal
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
> Fix For: 1.15.0
>
>
> git.commit.id.abbrev=eb0c403
> I have 3-nodes cluster with drillbits running on each node. I kicked off a
> long running query. In the middle of the query, I did a "./drillbit.sh
> graceful_stop" on one of the non-foreman node. The node was stopped within a
> few seconds and the query failed with error:
> Error: SYSTEM ERROR: IOException: Filesystem closed
> Fragment 4:15
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6852) Adapt current Parquet Metadata cache implementation to use Drill Metastore API
[ https://issues.apache.org/jira/browse/DRILL-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Volodymyr Vysotskyi updated DRILL-6852: --- Description: According to the design document for DRILL-6552, existing metadata cache API should be adapted to use generalized API for metastore and parquet metadata cache will be presented as the implementation of metastore API. The aim of this Jira is to refactor Parquet Metadata cache implementation and adapt it to use Drill Metastore API. Execution plan: - Refactor AbstractParquetGroupScan and its implementations to use metastore metadata classes. - Rework logic in AbstractParquetGroupScan to allow filtering at different metadata layers: partition, file, row group, etc. The same for pushing the limit. - Implement logic to populate metastore metadata classes from existing parquet metadata. - Rework EasyGroupScan to use the same logic as for refactored AbstractParquetGroupScan if metadata is available. - TBA... was: According to the design document for DRILL-6552, existing metadata cache API should be adapted to use generalized API for metastore and parquet metadata cache will be presented as the implementation of metastore API. The aim of this Jira is to refactor Parquet Metadata cache implementation and adapt it to use Drill Metastore API. > Adapt current Parquet Metadata cache implementation to use Drill Metastore API > -- > > Key: DRILL-6852 > URL: https://issues.apache.org/jira/browse/DRILL-6852 > Project: Apache Drill > Issue Type: Sub-task >Reporter: Volodymyr Vysotskyi >Assignee: Volodymyr Vysotskyi >Priority: Major > Fix For: 1.16.0 > > > According to the design document for DRILL-6552, existing metadata cache API > should be adapted to use generalized API for metastore and parquet metadata > cache will be presented as the implementation of metastore API. > The aim of this Jira is to refactor Parquet Metadata cache implementation and > adapt it to use Drill Metastore API. > Execution plan: > - Refactor AbstractParquetGroupScan and its implementations to use metastore > metadata classes. > - Rework logic in AbstractParquetGroupScan to allow filtering at different > metadata layers: partition, file, row group, etc. The same for pushing the > limit. > - Implement logic to populate metastore metadata classes from existing > parquet metadata. > - Rework EasyGroupScan to use the same logic as for refactored > AbstractParquetGroupScan if metadata is available. > - TBA... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6349) Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
[
https://issues.apache.org/jira/browse/DRILL-6349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689317#comment-16689317
]
ASF GitHub Bot commented on DRILL-6349:
---
oleg-zinovev commented on issue #1446: DRILL-6349: Drill JDBC driver fails on
Java 1.9+ with NoClassDefFoundError: sun/misc/VM
URL: https://github.com/apache/drill/pull/1446#issuecomment-439370857
@vvysotskyi , done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
> ---
>
> Key: DRILL-6349
> URL: https://issues.apache.org/jira/browse/DRILL-6349
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - JDBC
>Affects Versions: 1.13.0
> Environment: 16:23 apache-drill-1.13.0$ uname -a
> Darwin bix.local 17.5.0 Darwin Kernel Version 17.5.0: Mon Mar 5 22:24:32 PST
> 2018; root:xnu-4570.51.1~1/RELEASE_X86_64 x86_64
>
>Reporter: Marc Prud'hommeaux
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.15.0
>
>
> I'm surprised to be unable to find this issue logged elsewhere, but a quick
> search yields nothing. Trying to connect with the JDBC driver raises a
> NoClassDefFoundError for sun.misc.VM, which was removed in Java 9. It is
> understandable that the full drillbit (or its many dependencies) may have
> difficult-to-extract dependencies on some sun.misc.* classes, but the JDBC
> driver should be able to be loaded by any JVM.
>
> Looking at
> ./common/src/main/java/org/apache/drill/common/config/DrillConfig.java, it
> appears that a quick workaround could be to make sun.misc.VM.maxDirectMemory
> called lazily from the one place that uses it: getMaxDirectMemory()
>
>
>
> {{}}{{16:21 apache-drill-1.13.0$
> JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk-9.0.1.jdk
> ./bin/drill-localhost }}
> ***Connecting to jdbc:drill:drillbit=localhost*
> WARNING: An illegal reflective access operation has occurred
> WARNING: Illegal reflective access by
> org.apache.calcite.avatica.com.google.protobuf.UnsafeUtil
> (file:/Users/marc/Downloads/apache-drill-1.13.0/jars/3rdparty/avatica-1.10.0.jar)
> to field java.nio.Buffer.address
> WARNING: Please consider reporting this to the maintainers of
> org.apache.calcite.avatica.com.google.protobuf.UnsafeUtil
> WARNING: Use --illegal-access=warn to enable warnings of further illegal
> reflective access operations
> WARNING: All illegal access operations will be denied in a future release
> {{java.lang.NoClassDefFoundError: sun/misc/VM}}
> {{ at
> org.apache.drill.common.config.DrillConfig.(DrillConfig.java:49)}}
> {{ at
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:155)}}
> {{ at
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:73)}}
> {{ at
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)}}
> {{ at
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:138)}}
> {{ at org.apache.drill.jdbc.Driver.connect(Driver.java:72)}}
> {{ at sqlline.DatabaseConnection.connect(DatabaseConnection.java:168)}}
> {{ at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:214)}}
> {{ at sqlline.Commands.connect(Commands.java:1083)}}
> {{ at sqlline.Commands.connect(Commands.java:1015)}}
> {{ at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)}}
> {{ at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)}}
> {{ at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}}
> {{ at java.base/java.lang.reflect.Method.invoke(Method.java:564)}}
> {{ at
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)}}
> {{ at sqlline.SqlLine.dispatch(SqlLine.java:742)}}
> {{ at sqlline.SqlLine.initArgs(SqlLine.java:528)}}
> {{ at sqlline.SqlLine.begin(SqlLine.java:596)}}
> {{ at sqlline.SqlLine.start(SqlLine.java:375)}}
> {{ at sqlline.SqlLine.main(SqlLine.java:268)}}
> {{Caused by: java.lang.ClassNotFoundException: sun.misc.VM}}
> {{ at
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:582)}}
> {{ at
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:185)}}
> {{ at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:496)}}
> {{... 20 more}}
> apache drill 1.13.0
> "just drill it"
> 0:
