[GitHub] Ben-Zvi commented on issue #1373: DRILL-6517: Hash-Join: handling uninitialized vector container
Ben-Zvi commented on issue #1373: DRILL-6517: Hash-Join: handling uninitialized vector container URL: https://github.com/apache/drill/pull/1373#issuecomment-416800370 @ilooner please review -- This PR was changed, reduced from its original scope as other work (e.g. DRILL-6453) handled the main issue (a STOP outcome with uninitialized container). The main addition is the test file, checking that incoming uninitialized input containers (for STOP or NONE outcomes) are handled correctly. (Basically the same test as before, only now calling `next()` twice (as the first call always returns OK_NEW_SCHEMA)). A small fix was needed for the hash join, as in the case the first real incoming batch was NONE, the second `next()` was returning an OK (with 0 rows). With the change it returns NONE. And a minor change in the MockRecordBatch to avoid getting record count on uninitialized containers. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] Ben-Zvi commented on a change in pull request #1450: DRILL-6717: lower and upper functions not works with national characters
Ben-Zvi commented on a change in pull request #1450: DRILL-6717: lower and
upper functions not works with national characters
URL: https://github.com/apache/drill/pull/1450#discussion_r213521381
##
File path:
exec/java-exec/src/test/java/org/apache/drill/exec/expr/fn/impl/TestStringFunctions.java
##
@@ -1441,11 +1442,12 @@ public void testUpper() throws Exception {
"upper('AbC aBc') as col_space,\n" +
"upper('123ABC$!abc123.') as col_special,\n" +
"upper('') as col_empty,\n" +
-"upper(cast(null as varchar(10))) as col_null\n" +
+"upper(cast(null as varchar(10))) as col_null,\n" +
+"upper('привет, мир!') as col_national\n" +
Review comment:
Also added here
```+ "upper('ßß') as col_ss\n" + ```
which should yield
```""```
as uppercase (even without Locale.GERMAN) . This was in part to check for a
buffer overflow
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] Ben-Zvi commented on a change in pull request #1450: DRILL-6717: lower and upper functions not works with national characters
Ben-Zvi commented on a change in pull request #1450: DRILL-6717: lower and
upper functions not works with national characters
URL: https://github.com/apache/drill/pull/1450#discussion_r213520981
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/StringFunctions.java
##
@@ -529,7 +538,16 @@ public void eval() {
Review comment:
Isn't this too complicated ? I tried and changed this code by adding here:
```
String orig = new String(buffer.unsafeGetMemory(0,out.end));
String upp = orig.toUpperCase(/*Locale.GERMAN*/);
buffer.setBytes(0,upp.getBytes(),0,out.end);
```
and then commented out the whole following *for* loop, and the test passed!
(not sure which one performs better)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
Re: [ANNOUNCE] New PMC member: Volodymyr Vysotskyi
Congrats Vova ! On Tue, Aug 28, 2018 at 6:43 AM Vitalii Diravka wrote: > Congrats Vova! Well deserved! > > On Tue, Aug 28, 2018, 16:04 Volodymyr Vysotskyi > wrote: > > > Thank everyone for nice words, it's a great honor for me to become a > Drill > > PMC member! > > > > Kind regards, > > Volodymyr Vysotskyi > > > > > > On Sun, Aug 26, 2018 at 9:55 AM Robert Hou wrote: > > > > > Congratulations, Volodymyr! Thank you for all your work on Drill. > > > > > > On Sat, Aug 25, 2018 at 2:38 PM, Timothy Farkas > > wrote: > > > > > > > Congratulations, Volodymyr! > > > > > > > > On Sat, Aug 25, 2018 at 9:00 AM, Kunal Khatua > > wrote: > > > > > > > > > Congratulations, Volodymyr! > > > > > On 8/25/2018 6:32:07 AM, weijie tong > > wrote: > > > > > Congratulations Volodymyr! > > > > > > > > > > On Sat, Aug 25, 2018 at 8:30 AM salim achouche wrote: > > > > > > > > > > > Congrats Volodymyr! > > > > > > > > > > > > On Fri, Aug 24, 2018 at 11:32 AM Gautam Parai wrote: > > > > > > > > > > > > > Congratulations Vova! > > > > > > > > > > > > > > Gautam > > > > > > > > > > > > > > On Fri, Aug 24, 2018 at 10:59 AM, Khurram Faraaz > > > > > > wrote: > > > > > > > > > > > > > > > Congratulations Volodymyr! > > > > > > > > > > > > > > > > Regards, > > > > > > > > Khurram > > > > > > > > > > > > > > > > On Fri, Aug 24, 2018 at 10:25 AM, Hanumath Rao Maduri > > > > > > > hanu@gmail.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Congratulations Volodymyr! > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > -Hanu > > > > > > > > > > > > > > > > > > On Fri, Aug 24, 2018 at 10:22 AM Paul Rogers > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Congratulations Volodymyr! > > > > > > > > > > Thanks, > > > > > > > > > > - Paul > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Friday, August 24, 2018, 5:53:25 AM PDT, Arina > > Ielchiieva > > > > > > > > > > ar...@apache.org> wrote: > > > > > > > > > > > > > > > > > > > > I am pleased to announce that Drill PMC invited Volodymyr > > > > > > Vysotskyi > > > > > > > to > > > > > > > > > the > > > > > > > > > > PMC and he has accepted the invitation. > > > > > > > > > > > > > > > > > > > > Congratulations Vova and thanks for your contributions! > > > > > > > > > > > > > > > > > > > > - Arina > > > > > > > > > > (on behalf of Drill PMC) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
[GitHub] sohami commented on a change in pull request #1296: DRILL-5365: Prevent plugin config from changing default fs. Make DrillFileSystem Immutable.
sohami commented on a change in pull request #1296: DRILL-5365: Prevent plugin
config from changing default fs. Make DrillFileSystem Immutable.
URL: https://github.com/apache/drill/pull/1296#discussion_r213507759
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/DrillFileSystem.java
##
@@ -79,32 +77,56 @@
private final ConcurrentMap
openedFiles = Maps.newConcurrentMap();
+ private final Configuration fsConf;
private final FileSystem underlyingFs;
private final OperatorStats operatorStats;
private final CompressionCodecFactory codecFactory;
+ private boolean initialized = false;
+
public DrillFileSystem(Configuration fsConf) throws IOException {
this(fsConf, null);
}
public DrillFileSystem(Configuration fsConf, OperatorStats operatorStats)
throws IOException {
-this.underlyingFs = FileSystem.get(fsConf);
-this.codecFactory = new CompressionCodecFactory(fsConf);
+// Configuration objects are mutable, and the underlying FileSystem object
may directly use a passed in Configuration.
+// In order to avoid scenarios where a Configuration can change after a
DrillFileSystem is created, we make a copy
+// of the Configuration.
+this.fsConf = new Configuration(fsConf);
+this.underlyingFs = FileSystem.get(this.fsConf);
+this.codecFactory = new CompressionCodecFactory(this.fsConf);
this.operatorStats = operatorStats;
+this.initialized = true;
+ }
+
+ private void throwUnsupported() {
+throw new
UnsupportedOperationException(DrillFileSystem.class.getCanonicalName() + " is
immutable and should not be changed after creation.");
}
+ /**
+ * This method should never be used on {@link DrillFileSystem} since {@link
DrillFileSystem} is immutable.
+ * {@inheritDoc}
+ * @throws UnsupportedOperationException when called.
+ */
@Override
public void setConf(Configuration conf) {
-// Guard against setConf(null) call that is called as part of superclass
constructor (Configured) of the
-// DrillFileSystem, at which point underlyingFs is null.
-if (conf != null && underlyingFs != null) {
- underlyingFs.setConf(conf);
+if (!initialized) {
Review comment:
We can still use `(underlyingFs != null)` check and get rid of the
`initialized` flag. New check can be:
```
if (underlyingFs != null) {
throwUnsupported();
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] sohami commented on a change in pull request #1296: DRILL-5365: Prevent plugin config from changing default fs. Make DrillFileSystem Immutable.
sohami commented on a change in pull request #1296: DRILL-5365: Prevent plugin
config from changing default fs. Make DrillFileSystem Immutable.
URL: https://github.com/apache/drill/pull/1296#discussion_r213508890
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/DrillFileSystem.java
##
@@ -79,32 +77,56 @@
private final ConcurrentMap
openedFiles = Maps.newConcurrentMap();
+ private final Configuration fsConf;
private final FileSystem underlyingFs;
private final OperatorStats operatorStats;
private final CompressionCodecFactory codecFactory;
+ private boolean initialized = false;
+
public DrillFileSystem(Configuration fsConf) throws IOException {
this(fsConf, null);
}
public DrillFileSystem(Configuration fsConf, OperatorStats operatorStats)
throws IOException {
-this.underlyingFs = FileSystem.get(fsConf);
-this.codecFactory = new CompressionCodecFactory(fsConf);
+// Configuration objects are mutable, and the underlying FileSystem object
may directly use a passed in Configuration.
+// In order to avoid scenarios where a Configuration can change after a
DrillFileSystem is created, we make a copy
+// of the Configuration.
+this.fsConf = new Configuration(fsConf);
Review comment:
We can convert `this.fsConf` to be a local variable and in `getConf()` do
`return new Configuration(underlyingFs.getConf());`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] sohami opened a new pull request #1451: DRILL-6710: Drill C++ Client does not handle scale = 0 properly for d…
sohami opened a new pull request #1451: DRILL-6710: Drill C++ Client does not handle scale = 0 properly for d… URL: https://github.com/apache/drill/pull/1451 …ecimal @vvysotskyi - Please help to review this change. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kkhatua commented on issue #1279: DRILL-5735: Allow search/sort in the Options webUI
kkhatua commented on issue #1279: DRILL-5735: Allow search/sort in the Options webUI URL: https://github.com/apache/drill/pull/1279#issuecomment-416734773 Rebased again due to Guava shading. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] vrozov commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
vrozov commented on a change in pull request #1446: DRILL-6349: Drill JDBC
driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
URL: https://github.com/apache/drill/pull/1446#discussion_r213464182
##
File path:
common/src/main/java/org/apache/drill/common/util/DrillPlatformDependent.java
##
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.common.util;
+
+import io.netty.util.internal.PlatformDependent;
+
+import java.lang.reflect.Method;
+
+public class DrillPlatformDependent {
+ private final static long MAX_DIRECT_MEMORY = maxDirectMemory0();
+
+ public static long maxDirectMemory() {
+return MAX_DIRECT_MEMORY;
+ }
+
+
+ private static long maxDirectMemory0() {
Review comment:
Drill delegates to netty all large memory allocation and netty internally
tracks how much direct memory is available. If netty incorrectly determines max
direct memory size or Drill and netty max direct memory sizes are different, it
is easy to get into a situation where netty will fail allocation when memory is
available or it will try to allocate when memory is not available.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Resolved] (DRILL-6563) TPCDS query 10 has regressed
[ https://issues.apache.org/jira/browse/DRILL-6563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Khurram Faraaz resolved DRILL-6563. --- Resolution: Fixed > TPCDS query 10 has regressed > - > > Key: DRILL-6563 > URL: https://issues.apache.org/jira/browse/DRILL-6563 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization >Affects Versions: 1.14.0 >Reporter: Khurram Faraaz >Assignee: Pritesh Maker >Priority: Major > Fix For: 1.15.0 > > Attachments: 24ca3c6c-90e1-a4bf-6c6f-3f981fa2d043.sys.drill, > query10.fast_plan_old_commit, tpcds_query_10_plan_slow_140d09e.pdf, > tpcds_query_plan_10_140d09e.txt > > > TPC-DS query 10 has regressed in performance from taking 3.5 seconds to > execute on Apache Drill 1.14.0 commit b92f599 , to 07 min 51.851 sec to > complete execution on Apache Drill 1.14.0 commit 140d09e. Query was executed > over SF1 parquet views on a 4 node cluster. > Query plan from old and newer commit is attached here, with the query profile > from newer commit. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] kkhatua commented on a change in pull request #1437: DRILL-6702: Disable CPU Reporting for non-HotSpot JDKs
kkhatua commented on a change in pull request #1437: DRILL-6702: Disable CPU
Reporting for non-HotSpot JDKs
URL: https://github.com/apache/drill/pull/1437#discussion_r213418278
##
File path: common/src/main/java/org/apache/drill/exec/metrics/CpuGaugeSet.java
##
@@ -32,13 +32,22 @@
*/
@SuppressWarnings("restriction")
public class CpuGaugeSet implements MetricSet {
+ private static final org.slf4j.Logger logger =
org.slf4j.LoggerFactory.getLogger(CpuGaugeSet.class);
- private OperatingSystemMXBean osMXBean;
- private RuntimeMXBean rtMXBean;
+ private final OperatingSystemMXBean osMXBean;
+ private final RuntimeMXBean rtMXBean;
public CpuGaugeSet() {
-this.osMXBean = (OperatingSystemMXBean)
ManagementFactory.getOperatingSystemMXBean();
this.rtMXBean = ManagementFactory.getRuntimeMXBean();
+//DRILL-6702: Instead of worrying about compiling with IBM JDK, for now,
we shall provide no CPU metrics for non-HotSpot JVMs
+OperatingSystemMXBean tempOSMXBean;
+try {
+ tempOSMXBean = (OperatingSystemMXBean)
ManagementFactory.getOperatingSystemMXBean();
+} catch (ClassCastException cce) {
+ logger.warn("{}. Detected non-Supported JVM [{}]. CPU Metrics in the
WebUI will not be available!", cce.getMessage(),
System.getProperty("java.vm.name"));
Review comment:
@arina-ielchiieva the only message you get is
``` java.lang.ClassCastException:
com.ibm.lang.management.ExtendedOperatingSystem incompatible with
com.sun.management.OperatingSystemMXBean```
Not very useful, but you still get the simplified message without stack
trace. There isn't much value in adding more information than what is already
there, because the issue only emerged when JDBC drivers were being built using
the IBM JDK, something that a regular user will not face. (my testing showed
that an OracleJDK compiled class ran without issue for an IBM JDK Runtime).
I'll apply Vlad's suggestion & squash the commits.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
Re: Some questions about sorting pushdown to custom plugins
If I understand it correctly, you may need to write new storage plugin
rules (as similar to that of other storage plugin rules) to support
projection pushdown and limit pushdown for your custom storage plugin. This
might help in reading only the required fields from the storage.
Thanks,
On Tue, Aug 28, 2018 at 10:19 AM yang zhang
wrote:
> Hi：
> I have two questions to ask.
>
> There is already a custom plugin. Query sql:
> Select * from indexr.face_image_mess where device_id =
> '3E04846B-6B69-1A4D-0569-ED0813853348' and org_id in (11,12) order by
> short_time desc limit 4 offset 9;
> plan:
> {
> "head" : {
> "version" : 1,
> "generator" : {
> "type" : "DefaultSqlHandler",
> "info" : ""
> },
> "type" : "APACHE_DRILL_PHYSICAL",
> "options" : [ {
> "kind" : "LONG",
> "accessibleScopes" : "ALL",
> "name" : "planner.width.max_per_node",
> "num_val" : 1,
> "scope" : "SESSION"
> } ],
> "queue" : 0,
> "hasResourcePlan" : false,
> "resultMode" : "EXEC"
> },
> "graph" : [ {
> "pop" : "indexr-scan",
> "@id" : 7,
> "userName" : "conn",
> "indexrScanSpec" : {
> "tableName" : "face_image_mess",
> "rsFilter" : {
> "type" : "and",
> "children" : [ {
> "type" : "equal",
> "attr" : {
> "name" : "device_id",
> "type" : "VARCHAR"
> },
> "numValue" : 0,
> "strValue" : "3E04846B-6B69-1A4D-0569-ED0813853348",
> "type" : "equal"
> }, {
> "type" : "or",
> "children" : [ {
> "type" : "equal",
> "attr" : {
> "name" : "org_id",
> "type" : "VARCHAR"
> },
> "numValue" : 0,
> "strValue" : "11",
> "type" : "equal"
> }, {
> "type" : "equal",
> "attr" : {
> "name" : "org_id",
> "type" : "VARCHAR"
> },
> "numValue" : 0,
> "strValue" : "12",
> "type" : "equal"
> } ],
> "type" : "or"
> } ],
> "type" : "and"
> }
> },
> "storage" : {
> "type" : "indexr",
> "enabled" : true
> },
> "columns" : [ "`**`" ],
> "limitScanRows" : 9223372036854775807,
> "scanId" : "dc889ee0-6675-42bb-b7a5-2bc413d6372d",
> "cost" : 0.0
> }, {
> "pop" : "top-n",
> "@id" : 6,
> "child" : 7,
> "orderings" : [ {
> "order" : "DESC",
> "expr" : "`short_time`",
> "nullDirection" : "FIRST"
> } ],
> "reverse" : false,
> "limit" : 13,
> "initialAllocation" : 100,
> "maxAllocation" : 100,
> "cost" : 1.0
> }, {
> "pop" : "selection-vector-remover",
> "@id" : 5,
> "child" : 6,
> "initialAllocation" : 100,
> "maxAllocation" : 100,
> "cost" : 1.0
> }, {
> "pop" : "limit",
> "@id" : 4,
> "child" : 5,
> "first" : 9,
> "last" : 13,
> "initialAllocation" : 100,
> "maxAllocation" : 100,
> "cost" : 13.0
> }, {
> "pop" : "limit",
> "@id" : 3,
> "child" : 4,
> "first" : 9,
> "last" : 13,
> "initialAllocation" : 100,
> "maxAllocation" : 100,
> "cost" : 13.0
> }, {
> "pop" : "selection-vector-remover",
> "@id" : 2,
> "child" : 3,
> "initialAllocation" : 100,
> "maxAllocation" : 100,
> "cost" : 13.0
> }, {
> "pop" : "project",
> "@id" : 1,
> "exprs" : [ {
> "ref" : "`id`",
> "expr" : "`id`"
> }
> .
> .
> .
> ],
> "child" : 2,
> "outputProj" : true,
> "initialAllocation" : 100,
> "maxAllocation" : 100,
> "cost" : 13.0
> }, {
> "pop" : "screen",
> "@id" : 0,
> "child" : 1,
> "initialAllocation" : 100,
> "maxAllocation" : 100,
> "cost" : 13.0
> } ]
> }
>
>
> The version of drill-1.13.0 is used.
> 1. Is order by pushdown supported?
> Does the drill execution plan push down to the plugin when it
> supports getting the order by field 'short_time' and the select field (all
> fields)?
> This table (face_image_mess) has 70 fields, one data has 1KB; this
> query hits 2; currently can't get the order by field and select
> field pushed down to the plugin, only scan all the field values of the hit;
> the calculation of the drill engine is 2 * 1KB = 190.7GB.
>Custom plugin I designed a Long type rowid, which points to the physical
> address.
>If the plugin can get the pushed by order field and the select field to
> filter out the fields that do not need to participate in the calculation,
> only four fields that actually participate in the calculation are 'rowid',
> 'short_time', 'device_id', 'org_id', then Minimize the calculation of the
> drill engine. The actual
[GitHub] oleg-zinovev opened a new pull request #1450: DRILL-6717: lower and upper functions not works with national characters
oleg-zinovev opened a new pull request #1450: DRILL-6717: lower and upper functions not works with national characters URL: https://github.com/apache/drill/pull/1450 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (DRILL-6717) lower and upper functions not works with national charactes
Oleg Zinoviev created DRILL-6717:
Summary: lower and upper functions not works with national
charactes
Key: DRILL-6717
URL: https://issues.apache.org/jira/browse/DRILL-6717
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.14.0, 1.15.0
Reporter: Oleg Zinoviev
Attachments: result.csv, sample.json
upper() and lower() functions works only with a ascii characters.
sample dataset: [^sample.json]
query:
{code:sql}
select
upper(ascii_l) as upper_ascii,
upper(national_l) as upper_national,
lower(ascii_u) as lower_ascii,
lower(national_u) as lower_national
from dfs.`sample.json`
{code}
drill response: [^result.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Apache drill High availability
Unlike other databases or data engines drill doesn't store data in its own storage engine. So high availability of the data when using Drill means the storage engine needs to support high availability. High availability can also mean that when running a query even though some of the fragments/ nodes might fail, query needs to succeed. In this scenario Drill reports an error to the user and expects the user to re-run the query. On re-running the query again, it might be successful (of-course depending upon load of the system). If a node is crashed or not responding then the ZK will not consider the node during planning stage itself. Hence, no fragments are executed on the node (and query performance might degrade in this scenario). Thanks, On Tue, Aug 28, 2018 at 9:34 AM salim achouche wrote: > You need to clarify your definition of HA as there can be multiple faults > at play: > - A Drill cluster can handle nodes going down (and new ones joining the > cluster) > - Though, running queries (which are executed in a distributed manner) > might fail if they had minor-fragments running on a faulty node > - Similarly, Drill has some built-in resilience to network disconnects > albeit it is not always transparent (I believe, queries might fail if > network disconnect happened during a connection exchange) > > Regards, > > On Mon, Aug 27, 2018 at 10:48 PM pujari Satish > wrote: > > > Hi Team, > > > > Good Morning. I am trying to do drill high avilability using Haproxy load > > balancer. > > Is drill supports for high availability ? > > > > > > please let me know in this. > > > > > > -Thanks, > > Satish > > >
Some questions about sorting pushdown to custom plugins
Hi：
I have two questions to ask.
There is already a custom plugin. Query sql:
Select * from indexr.face_image_mess where device_id =
'3E04846B-6B69-1A4D-0569-ED0813853348' and org_id in (11,12) order by
short_time desc limit 4 offset 9;
plan:
{
"head" : {
"version" : 1,
"generator" : {
"type" : "DefaultSqlHandler",
"info" : ""
},
"type" : "APACHE_DRILL_PHYSICAL",
"options" : [ {
"kind" : "LONG",
"accessibleScopes" : "ALL",
"name" : "planner.width.max_per_node",
"num_val" : 1,
"scope" : "SESSION"
} ],
"queue" : 0,
"hasResourcePlan" : false,
"resultMode" : "EXEC"
},
"graph" : [ {
"pop" : "indexr-scan",
"@id" : 7,
"userName" : "conn",
"indexrScanSpec" : {
"tableName" : "face_image_mess",
"rsFilter" : {
"type" : "and",
"children" : [ {
"type" : "equal",
"attr" : {
"name" : "device_id",
"type" : "VARCHAR"
},
"numValue" : 0,
"strValue" : "3E04846B-6B69-1A4D-0569-ED0813853348",
"type" : "equal"
}, {
"type" : "or",
"children" : [ {
"type" : "equal",
"attr" : {
"name" : "org_id",
"type" : "VARCHAR"
},
"numValue" : 0,
"strValue" : "11",
"type" : "equal"
}, {
"type" : "equal",
"attr" : {
"name" : "org_id",
"type" : "VARCHAR"
},
"numValue" : 0,
"strValue" : "12",
"type" : "equal"
} ],
"type" : "or"
} ],
"type" : "and"
}
},
"storage" : {
"type" : "indexr",
"enabled" : true
},
"columns" : [ "`**`" ],
"limitScanRows" : 9223372036854775807,
"scanId" : "dc889ee0-6675-42bb-b7a5-2bc413d6372d",
"cost" : 0.0
}, {
"pop" : "top-n",
"@id" : 6,
"child" : 7,
"orderings" : [ {
"order" : "DESC",
"expr" : "`short_time`",
"nullDirection" : "FIRST"
} ],
"reverse" : false,
"limit" : 13,
"initialAllocation" : 100,
"maxAllocation" : 100,
"cost" : 1.0
}, {
"pop" : "selection-vector-remover",
"@id" : 5,
"child" : 6,
"initialAllocation" : 100,
"maxAllocation" : 100,
"cost" : 1.0
}, {
"pop" : "limit",
"@id" : 4,
"child" : 5,
"first" : 9,
"last" : 13,
"initialAllocation" : 100,
"maxAllocation" : 100,
"cost" : 13.0
}, {
"pop" : "limit",
"@id" : 3,
"child" : 4,
"first" : 9,
"last" : 13,
"initialAllocation" : 100,
"maxAllocation" : 100,
"cost" : 13.0
}, {
"pop" : "selection-vector-remover",
"@id" : 2,
"child" : 3,
"initialAllocation" : 100,
"maxAllocation" : 100,
"cost" : 13.0
}, {
"pop" : "project",
"@id" : 1,
"exprs" : [ {
"ref" : "`id`",
"expr" : "`id`"
}
.
.
.
],
"child" : 2,
"outputProj" : true,
"initialAllocation" : 100,
"maxAllocation" : 100,
"cost" : 13.0
}, {
"pop" : "screen",
"@id" : 0,
"child" : 1,
"initialAllocation" : 100,
"maxAllocation" : 100,
"cost" : 13.0
} ]
}
The version of drill-1.13.0 is used.
1. Is order by pushdown supported?
Does the drill execution plan push down to the plugin when it
supports getting the order by field 'short_time' and the select field (all
fields)?
This table (face_image_mess) has 70 fields, one data has 1KB; this
query hits 2; currently can't get the order by field and select
field pushed down to the plugin, only scan all the field values of the hit;
the calculation of the drill engine is 2 * 1KB = 190.7GB.
Custom plugin I designed a Long type rowid, which points to the physical
address.
If the plugin can get the pushed by order field and the select field to
filter out the fields that do not need to participate in the calculation,
only four fields that actually participate in the calculation are 'rowid',
'short_time', 'device_id', 'org_id', then Minimize the calculation of the
drill engine. The actual calculation is 190.7GB * 4 ÷ 70 = 10.9GB, which
greatly improves the query performance. After the calculation is completed,
the topN detailed records are obtained according to the rowid.
2. Is limit pushdown supported?
The current limit 4 offset 9 is actually the data that satisfies the
where condition and then the limit 4 offset 9; whether to support the limit
4 offset 9 push to the plugin, each drillbit node plugin handles the topN
return, only jump to the last one Does the page need all the data for
'where conditions'?
[GitHub] agozhiy opened a new pull request #1449: DRILL-6544: Timestamp value in Drill UI showed inconsistently with th…
agozhiy opened a new pull request #1449: DRILL-6544: Timestamp value in Drill UI showed inconsistently with th… URL: https://github.com/apache/drill/pull/1449 …e same value retrieved from sqline Added a formatting of timestamp values instead of just calling toString(). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill
JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
URL: https://github.com/apache/drill/pull/1446#discussion_r213392442
##
File path:
common/src/main/java/org/apache/drill/common/util/DrillPlatformDependent.java
##
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.common.util;
+
+import io.netty.util.internal.PlatformDependent;
+
+import java.lang.reflect.Method;
+
+public class DrillPlatformDependent {
+ private final static long MAX_DIRECT_MEMORY = maxDirectMemory0();
+
+ public static long maxDirectMemory() {
+return MAX_DIRECT_MEMORY;
+ }
+
+
+ private static long maxDirectMemory0() {
Review comment:
Looks like it can cause OOM:
In Drill version of `PooledByteBufAllocatorL` exists a reflective access to
`PooledByteBufAllocator.directArenas` field.
It's size depends on
` DEFAULT_NUM_DIRECT_ARENA = Math.max(0,
SystemPropertyUtil.getInt(
"io.netty.allocator.numDirectArenas",
(int) Math.min(
defaultMinNumArena,
PlatformDependent.maxDirectMemory() /
defaultChunkSize / 2 / 3)));`
Should i use a netty `PlatformDependent`, or "override" it (as
`PooledByteBufAllocatorL`) ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill
JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
URL: https://github.com/apache/drill/pull/1446#discussion_r213392442
##
File path:
common/src/main/java/org/apache/drill/common/util/DrillPlatformDependent.java
##
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.common.util;
+
+import io.netty.util.internal.PlatformDependent;
+
+import java.lang.reflect.Method;
+
+public class DrillPlatformDependent {
+ private final static long MAX_DIRECT_MEMORY = maxDirectMemory0();
+
+ public static long maxDirectMemory() {
+return MAX_DIRECT_MEMORY;
+ }
+
+
+ private static long maxDirectMemory0() {
Review comment:
Looks like it can cause OOM:
In Drill version of `PooledByteBufAllocatorL` exists a reflective access to
`PooledByteBufAllocator.directArenas` field.
It's size depends on
` DEFAULT_NUM_DIRECT_ARENA = Math.max(0,
SystemPropertyUtil.getInt(
"io.netty.allocator.numDirectArenas",
(int) Math.min(
defaultMinNumArena,
*PlatformDependent.maxDirectMemory()* /
defaultChunkSize / 2 / 3)));`
Should i use a netty `PlatformDependent`, or "override" it (as
`PooledByteBufAllocatorL`) ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill
JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
URL: https://github.com/apache/drill/pull/1446#discussion_r213392442
##
File path:
common/src/main/java/org/apache/drill/common/util/DrillPlatformDependent.java
##
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.common.util;
+
+import io.netty.util.internal.PlatformDependent;
+
+import java.lang.reflect.Method;
+
+public class DrillPlatformDependent {
+ private final static long MAX_DIRECT_MEMORY = maxDirectMemory0();
+
+ public static long maxDirectMemory() {
+return MAX_DIRECT_MEMORY;
+ }
+
+
+ private static long maxDirectMemory0() {
Review comment:
Looks like it can cause OOM:
In Drill version of `PooledByteBufAllocatorL` exists a reflective access to
`PooledByteBufAllocator.directArenas` field.
It's size depends on
` DEFAULT_NUM_DIRECT_ARENA = Math.max(0,
SystemPropertyUtil.getInt(
"io.netty.allocator.numDirectArenas",
(int) Math.min(
defaultMinNumArena,
PlatformDependent.maxDirectMemory() /
defaultChunkSize / 2 / 3)));`
Should i use a netty `PlatformDependent`, or "override" it (as
`PooledByteBufAllocatorL`) ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill
JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
URL: https://github.com/apache/drill/pull/1446#discussion_r213392442
##
File path:
common/src/main/java/org/apache/drill/common/util/DrillPlatformDependent.java
##
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.common.util;
+
+import io.netty.util.internal.PlatformDependent;
+
+import java.lang.reflect.Method;
+
+public class DrillPlatformDependent {
+ private final static long MAX_DIRECT_MEMORY = maxDirectMemory0();
+
+ public static long maxDirectMemory() {
+return MAX_DIRECT_MEMORY;
+ }
+
+
+ private static long maxDirectMemory0() {
Review comment:
Looks like it can cause OOM:
In Drill version of `PooledByteBufAllocatorL` exists a reflective access to
`PooledByteBufAllocator.directArenas` field. It's size depends on
` DEFAULT_NUM_DIRECT_ARENA = Math.max(0,
SystemPropertyUtil.getInt(
"io.netty.allocator.numDirectArenas",
(int) Math.min(
defaultMinNumArena,
PlatformDependent.maxDirectMemory() /
defaultChunkSize / 2 / 3)));`
Should i use a netty `PlatformDependent`, or "override" it (as
`PooledByteBufAllocatorL`) ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill
JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
URL: https://github.com/apache/drill/pull/1446#discussion_r213392442
##
File path:
common/src/main/java/org/apache/drill/common/util/DrillPlatformDependent.java
##
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.common.util;
+
+import io.netty.util.internal.PlatformDependent;
+
+import java.lang.reflect.Method;
+
+public class DrillPlatformDependent {
+ private final static long MAX_DIRECT_MEMORY = maxDirectMemory0();
+
+ public static long maxDirectMemory() {
+return MAX_DIRECT_MEMORY;
+ }
+
+
+ private static long maxDirectMemory0() {
Review comment:
Looks like it can cause OOM:
In Drill version of `PooledByteBufAllocatorL` exists a reflective access to
`PooledByteBufAllocator.directArenas` field. It's size depends on
` DEFAULT_NUM_DIRECT_ARENA = Math.max(0,
SystemPropertyUtil.getInt(
"io.netty.allocator.numDirectArenas",
(int) Math.min(
defaultMinNumArena,
PlatformDependent.maxDirectMemory() /
defaultChunkSize / 2 / 3)));`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill
JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
URL: https://github.com/apache/drill/pull/1446#discussion_r213386245
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/FileSystemPartitionDescriptor.java
##
@@ -179,7 +179,7 @@ protected void createPartitionSublists() {
// build a list of DFSDirPartitionLocation.
for (final List dirs : dirToFileMap.keySet()) {
- locations.add( new DFSDirPartitionLocation((String [])dirs.toArray(),
dirToFileMap.get(dirs)));
+ locations.add( new DFSDirPartitionLocation(dirs.toArray(new String [0]),
dirToFileMap.get(dirs)));
Review comment:
According to
https://stackoverflow.com/questions/174093/toarraynew-myclass0-or-toarraynew-myclassmylist-size
`.toArray(new T[0])` faster
+ https://github.com/naver/pinpoint/issues/4138
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill
JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
URL: https://github.com/apache/drill/pull/1446#discussion_r213386245
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/FileSystemPartitionDescriptor.java
##
@@ -179,7 +179,7 @@ protected void createPartitionSublists() {
// build a list of DFSDirPartitionLocation.
for (final List dirs : dirToFileMap.keySet()) {
- locations.add( new DFSDirPartitionLocation((String [])dirs.toArray(),
dirToFileMap.get(dirs)));
+ locations.add( new DFSDirPartitionLocation(dirs.toArray(new String [0]),
dirToFileMap.get(dirs)));
Review comment:
According to
https://stackoverflow.com/questions/174093/toarraynew-myclass0-or-toarraynew-myclassmylist-size
`.toArray(new T[0])` faster
+https://github.com/naver/pinpoint/issues/4138
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill
JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
URL: https://github.com/apache/drill/pull/1446#discussion_r213386245
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/FileSystemPartitionDescriptor.java
##
@@ -179,7 +179,7 @@ protected void createPartitionSublists() {
// build a list of DFSDirPartitionLocation.
for (final List dirs : dirToFileMap.keySet()) {
- locations.add( new DFSDirPartitionLocation((String [])dirs.toArray(),
dirToFileMap.get(dirs)));
+ locations.add( new DFSDirPartitionLocation(dirs.toArray(new String [0]),
dirToFileMap.get(dirs)));
Review comment:
According to
https://stackoverflow.com/questions/174093/toarraynew-myclass0-or-toarraynew-myclassmylist-size
`.toArray(new T[0])` faster
+ https://github.com/naver/pinpoint/issues/4138
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM URL: https://github.com/apache/drill/pull/1446#discussion_r213387093 ## File path: pom.xml ## @@ -733,6 +734,11 @@ -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -XX:+CMSClassUnloadingEnabled -ea + -XX:+IgnoreUnrecognizedVMOptions + "--add-modules java.se" + -Djdk.attach.allowAttachSelf=true + -Duser.language=en Review comment: -Djdk.attach.allowAttachSelf=true - for jmockit -Duser.language=en - format tests fails on non-english system locale -XX:+IgnoreUnrecognizedVMOptions "--add-modules java.se" - revalidate it tomorrow This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill
JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
URL: https://github.com/apache/drill/pull/1446#discussion_r213386245
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/FileSystemPartitionDescriptor.java
##
@@ -179,7 +179,7 @@ protected void createPartitionSublists() {
// build a list of DFSDirPartitionLocation.
for (final List dirs : dirToFileMap.keySet()) {
- locations.add( new DFSDirPartitionLocation((String [])dirs.toArray(),
dirToFileMap.get(dirs)));
+ locations.add( new DFSDirPartitionLocation(dirs.toArray(new String [0]),
dirToFileMap.get(dirs)));
Review comment:
According to
https://stackoverflow.com/questions/174093/toarraynew-myclass0-or-toarraynew-myclassmylist-size
`.toArray(new T[0])` faster
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] vrozov commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
vrozov commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM URL: https://github.com/apache/drill/pull/1446#discussion_r213386021 ## File path: pom.xml ## @@ -733,6 +734,11 @@ -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -XX:+CMSClassUnloadingEnabled -ea + -XX:+IgnoreUnrecognizedVMOptions + "--add-modules java.se" + -Djdk.attach.allowAttachSelf=true + -Duser.language=en Review comment: Why is this necessary? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
Re: Apache drill High availability
You need to clarify your definition of HA as there can be multiple faults at play: - A Drill cluster can handle nodes going down (and new ones joining the cluster) - Though, running queries (which are executed in a distributed manner) might fail if they had minor-fragments running on a faulty node - Similarly, Drill has some built-in resilience to network disconnects albeit it is not always transparent (I believe, queries might fail if network disconnect happened during a connection exchange) Regards, On Mon, Aug 27, 2018 at 10:48 PM pujari Satish wrote: > Hi Team, > > Good Morning. I am trying to do drill high avilability using Haproxy load > balancer. > Is drill supports for high availability ? > > > please let me know in this. > > > -Thanks, > Satish >
[GitHub] vrozov commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
vrozov commented on a change in pull request #1446: DRILL-6349: Drill JDBC
driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
URL: https://github.com/apache/drill/pull/1446#discussion_r213384680
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/FileSystemPartitionDescriptor.java
##
@@ -179,7 +179,7 @@ protected void createPartitionSublists() {
// build a list of DFSDirPartitionLocation.
for (final List dirs : dirToFileMap.keySet()) {
- locations.add( new DFSDirPartitionLocation((String [])dirs.toArray(),
dirToFileMap.get(dirs)));
+ locations.add( new DFSDirPartitionLocation(dirs.toArray(new String [0]),
dirToFileMap.get(dirs)));
Review comment:
use `dirs.size()` instead of 0.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] vrozov commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
vrozov commented on a change in pull request #1446: DRILL-6349: Drill JDBC
driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
URL: https://github.com/apache/drill/pull/1446#discussion_r213382649
##
File path:
common/src/main/java/org/apache/drill/common/util/DrillPlatformDependent.java
##
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.common.util;
+
+import io.netty.util.internal.PlatformDependent;
+
+import java.lang.reflect.Method;
+
+public class DrillPlatformDependent {
+ private final static long MAX_DIRECT_MEMORY = maxDirectMemory0();
+
+ public static long maxDirectMemory() {
+return MAX_DIRECT_MEMORY;
+ }
+
+
+ private static long maxDirectMemory0() {
Review comment:
If netty `PlatformDependent` uses wrong max direct memory, will not netty
memory allocation be affected that may cause OOM in Drill?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
oleg-zinovev commented on a change in pull request #1446: DRILL-6349: Drill
JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
URL: https://github.com/apache/drill/pull/1446#discussion_r213346042
##
File path:
common/src/main/java/org/apache/drill/common/util/DrillPlatformDependent.java
##
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.common.util;
+
+import io.netty.util.internal.PlatformDependent;
+
+import java.lang.reflect.Method;
+
+public class DrillPlatformDependent {
+ private final static long MAX_DIRECT_MEMORY = maxDirectMemory0();
+
+ public static long maxDirectMemory() {
+return MAX_DIRECT_MEMORY;
+ }
+
+
+ private static long maxDirectMemory0() {
Review comment:
netty PlatformDependent do not try to access jdk.internal.misc.VM.
By "copy-paste" i mean only a method name
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] vrozov commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
vrozov commented on a change in pull request #1446: DRILL-6349: Drill JDBC
driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
URL: https://github.com/apache/drill/pull/1446#discussion_r213343540
##
File path:
exec/java-exec/src/test/java/org/apache/drill/exec/udf/dynamic/JarBuilder.java
##
@@ -69,6 +69,8 @@ public String build(String jarName, String buildDirectory,
String includeFiles,
List params = new LinkedList<>();
params.add("clean");
params.add("package");
+ params.add("-Dmaven.compiler.source=1.8");
Review comment:
use pom.xml
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] vrozov commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
vrozov commented on a change in pull request #1446: DRILL-6349: Drill JDBC
driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
URL: https://github.com/apache/drill/pull/1446#discussion_r213342013
##
File path: distribution/src/resources/sqlline.bat
##
@@ -215,6 +215,12 @@ SET JAVA_CMD=%JAVA_HOME%\bin\%JAVA_EXE%
if "%JAVA_HOME%" == "" (set JAVA_CMD=%JAVA_EXE%)
set ERROR_CODE=0
+for /f tokens^=2-5^ delims^=.-_^" %%j in ('"%JAVA_CMD%" -fullversion 2^>^&1')
do set "version=%%j%%k"
Review comment:
LF/CR?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] oleg-zinovev commented on issue #1447: DRILL-6712: Creation of jdbc storage plugin fails with NoSuchMethod
oleg-zinovev commented on issue #1447: DRILL-6712: Creation of jdbc storage plugin fails with NoSuchMethod URL: https://github.com/apache/drill/pull/1447#issuecomment-416608686 I do not see any breaking changes in version 2.6.0 (https://commons.apache.org/proper/commons-pool/changes-report.html#a2.6.0), so it's really better to upgrade to the latest version This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] vrozov commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
vrozov commented on a change in pull request #1446: DRILL-6349: Drill JDBC
driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
URL: https://github.com/apache/drill/pull/1446#discussion_r213336202
##
File path:
common/src/main/java/org/apache/drill/common/util/DrillPlatformDependent.java
##
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.common.util;
+
+import io.netty.util.internal.PlatformDependent;
+
+import java.lang.reflect.Method;
+
+public class DrillPlatformDependent {
+ private final static long MAX_DIRECT_MEMORY = maxDirectMemory0();
+
+ public static long maxDirectMemory() {
+return MAX_DIRECT_MEMORY;
+ }
+
+
+ private static long maxDirectMemory0() {
Review comment:
Why is copy/paste necessary (why not to use netty `PlatformDependent`
directly)?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
Re: [ANNOUNCE] New PMC member: Volodymyr Vysotskyi
Congrats Vova! Well deserved! On Tue, Aug 28, 2018, 16:04 Volodymyr Vysotskyi wrote: > Thank everyone for nice words, it's a great honor for me to become a Drill > PMC member! > > Kind regards, > Volodymyr Vysotskyi > > > On Sun, Aug 26, 2018 at 9:55 AM Robert Hou wrote: > > > Congratulations, Volodymyr! Thank you for all your work on Drill. > > > > On Sat, Aug 25, 2018 at 2:38 PM, Timothy Farkas > wrote: > > > > > Congratulations, Volodymyr! > > > > > > On Sat, Aug 25, 2018 at 9:00 AM, Kunal Khatua > wrote: > > > > > > > Congratulations, Volodymyr! > > > > On 8/25/2018 6:32:07 AM, weijie tong > wrote: > > > > Congratulations Volodymyr! > > > > > > > > On Sat, Aug 25, 2018 at 8:30 AM salim achouche wrote: > > > > > > > > > Congrats Volodymyr! > > > > > > > > > > On Fri, Aug 24, 2018 at 11:32 AM Gautam Parai wrote: > > > > > > > > > > > Congratulations Vova! > > > > > > > > > > > > Gautam > > > > > > > > > > > > On Fri, Aug 24, 2018 at 10:59 AM, Khurram Faraaz > > > > > wrote: > > > > > > > > > > > > > Congratulations Volodymyr! > > > > > > > > > > > > > > Regards, > > > > > > > Khurram > > > > > > > > > > > > > > On Fri, Aug 24, 2018 at 10:25 AM, Hanumath Rao Maduri > > > > > > hanu@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > Congratulations Volodymyr! > > > > > > > > > > > > > > > > Thanks, > > > > > > > > -Hanu > > > > > > > > > > > > > > > > On Fri, Aug 24, 2018 at 10:22 AM Paul Rogers > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Congratulations Volodymyr! > > > > > > > > > Thanks, > > > > > > > > > - Paul > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Friday, August 24, 2018, 5:53:25 AM PDT, Arina > Ielchiieva > > > > > > > > > ar...@apache.org> wrote: > > > > > > > > > > > > > > > > > > I am pleased to announce that Drill PMC invited Volodymyr > > > > > Vysotskyi > > > > > > to > > > > > > > > the > > > > > > > > > PMC and he has accepted the invitation. > > > > > > > > > > > > > > > > > > Congratulations Vova and thanks for your contributions! > > > > > > > > > > > > > > > > > > - Arina > > > > > > > > > (on behalf of Drill PMC) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: [ANNOUNCE] New PMC member: Volodymyr Vysotskyi
Thank everyone for nice words, it's a great honor for me to become a Drill PMC member! Kind regards, Volodymyr Vysotskyi On Sun, Aug 26, 2018 at 9:55 AM Robert Hou wrote: > Congratulations, Volodymyr! Thank you for all your work on Drill. > > On Sat, Aug 25, 2018 at 2:38 PM, Timothy Farkas wrote: > > > Congratulations, Volodymyr! > > > > On Sat, Aug 25, 2018 at 9:00 AM, Kunal Khatua wrote: > > > > > Congratulations, Volodymyr! > > > On 8/25/2018 6:32:07 AM, weijie tong wrote: > > > Congratulations Volodymyr! > > > > > > On Sat, Aug 25, 2018 at 8:30 AM salim achouche wrote: > > > > > > > Congrats Volodymyr! > > > > > > > > On Fri, Aug 24, 2018 at 11:32 AM Gautam Parai wrote: > > > > > > > > > Congratulations Vova! > > > > > > > > > > Gautam > > > > > > > > > > On Fri, Aug 24, 2018 at 10:59 AM, Khurram Faraaz > > > > wrote: > > > > > > > > > > > Congratulations Volodymyr! > > > > > > > > > > > > Regards, > > > > > > Khurram > > > > > > > > > > > > On Fri, Aug 24, 2018 at 10:25 AM, Hanumath Rao Maduri > > > > > hanu@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > Congratulations Volodymyr! > > > > > > > > > > > > > > Thanks, > > > > > > > -Hanu > > > > > > > > > > > > > > On Fri, Aug 24, 2018 at 10:22 AM Paul Rogers > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Congratulations Volodymyr! > > > > > > > > Thanks, > > > > > > > > - Paul > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Friday, August 24, 2018, 5:53:25 AM PDT, Arina Ielchiieva > > > > > > > > ar...@apache.org> wrote: > > > > > > > > > > > > > > > > I am pleased to announce that Drill PMC invited Volodymyr > > > > Vysotskyi > > > > > to > > > > > > > the > > > > > > > > PMC and he has accepted the invitation. > > > > > > > > > > > > > > > > Congratulations Vova and thanks for your contributions! > > > > > > > > > > > > > > > > - Arina > > > > > > > > (on behalf of Drill PMC) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
[GitHub] arina-ielchiieva commented on issue #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
arina-ielchiieva commented on issue #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM URL: https://github.com/apache/drill/pull/1446#issuecomment-416534459 @oleg-zinovev besides comment in the PR I suggest a couple more things: 1. File a Jira for failed tests (indicating what and why fails - basically copy info from the PR description + indicate affected JDKs). 2. As I understand the purpose of this PR is allow build on JDK 9 and 10. Maybe it's better to rename commit message to highlight this. Description of the changes (which was really useful during the review, thanks for that!) can be included the commit message as well. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] arina-ielchiieva commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
arina-ielchiieva commented on a change in pull request #1446: DRILL-6349: Drill
JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
URL: https://github.com/apache/drill/pull/1446#discussion_r213260907
##
File path:
common/src/main/java/org/apache/drill/common/util/DrillPlatformDependent.java
##
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.common.util;
+
+import io.netty.util.internal.PlatformDependent;
+
+import java.lang.reflect.Method;
+
+public class DrillPlatformDependent {
+ private final static long MAX_DIRECT_MEMORY = maxDirectMemory0();
+
+ public static long maxDirectMemory() {
+return MAX_DIRECT_MEMORY;
+ }
+
+
+ private static long maxDirectMemory0() {
Review comment:
@oleg-zinovev
1. Please add javadoc for this class explaining its purpose.
2. I still suggest to rename the method.
3. Maybe add trace for the ignored exception, just in case we might need to
find out what has happened.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] arina-ielchiieva commented on a change in pull request #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
arina-ielchiieva commented on a change in pull request #1446: DRILL-6349: Drill
JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
URL: https://github.com/apache/drill/pull/1446#discussion_r213260907
##
File path:
common/src/main/java/org/apache/drill/common/util/DrillPlatformDependent.java
##
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.common.util;
+
+import io.netty.util.internal.PlatformDependent;
+
+import java.lang.reflect.Method;
+
+public class DrillPlatformDependent {
+ private final static long MAX_DIRECT_MEMORY = maxDirectMemory0();
+
+ public static long maxDirectMemory() {
+return MAX_DIRECT_MEMORY;
+ }
+
+
+ private static long maxDirectMemory0() {
Review comment:
1. Please add javadoc for this class explaining its purpose.
2. I still suggest to rename the method.
3. Maybe add trace for the ignored exception, just in case we might need to
find out what has happened.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] vvysotskyi commented on issue #1445: DRILL-6706: fixed null pointer exception in HashJoin
vvysotskyi commented on issue #1445: DRILL-6706: fixed null pointer exception in HashJoin URL: https://github.com/apache/drill/pull/1445#issuecomment-416532411 @ilooner, I don't have any objection regarding this PR. If this bug is critical, it may be merged and DRILL-6714 with more proper fix may be done later. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] arina-ielchiieva commented on a change in pull request #1437: DRILL-6702: Disable CPU Reporting for non-HotSpot JDKs
arina-ielchiieva commented on a change in pull request #1437: DRILL-6702:
Disable CPU Reporting for non-HotSpot JDKs
URL: https://github.com/apache/drill/pull/1437#discussion_r213256408
##
File path: common/src/main/java/org/apache/drill/exec/metrics/CpuGaugeSet.java
##
@@ -32,13 +32,22 @@
*/
@SuppressWarnings("restriction")
public class CpuGaugeSet implements MetricSet {
+ private static final org.slf4j.Logger logger =
org.slf4j.LoggerFactory.getLogger(CpuGaugeSet.class);
- private OperatingSystemMXBean osMXBean;
- private RuntimeMXBean rtMXBean;
+ private final OperatingSystemMXBean osMXBean;
+ private final RuntimeMXBean rtMXBean;
public CpuGaugeSet() {
-this.osMXBean = (OperatingSystemMXBean)
ManagementFactory.getOperatingSystemMXBean();
this.rtMXBean = ManagementFactory.getRuntimeMXBean();
+//DRILL-6702: Instead of worrying about compiling with IBM JDK, for now,
we shall provide no CPU metrics for non-HotSpot JVMs
+OperatingSystemMXBean tempOSMXBean;
+try {
+ tempOSMXBean = (OperatingSystemMXBean)
ManagementFactory.getOperatingSystemMXBean();
+} catch (ClassCastException cce) {
+ logger.warn("{}. Detected non-Supported JVM [{}]. CPU Metrics in the
WebUI will not be available!", cce.getMessage(),
System.getProperty("java.vm.name"));
Review comment:
@kkhatua
1. Could you please give example of warn message? I am not sure that
`e.getMessage` will give enough information.
2. Vlad's suggestion to use ternary operator should be also implemented.
3. Please the squash the commits.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] arina-ielchiieva commented on issue #1447: DRILL-6712: Creation of jdbc storage plugin fails with NoSuchMethod
arina-ielchiieva commented on issue #1447: DRILL-6712: Creation of jdbc storage plugin fails with NoSuchMethod URL: https://github.com/apache/drill/pull/1447#issuecomment-416528031 @oleg-zinovev restarted the build one more time, though it was failed log showed that it has passed successful but failed in the end, so it's critical, happens sometimes with Travis. Also still waiting your feedback about upgrading to 2.6 instead of 2.5. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] oleg-zinovev edited a comment on issue #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
oleg-zinovev edited a comment on issue #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM URL: https://github.com/apache/drill/pull/1446#issuecomment-416520694 build works with Java 9, except hive and kafka tests (HBase tests surprisely works) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] oleg-zinovev commented on issue #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
oleg-zinovev commented on issue #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM URL: https://github.com/apache/drill/pull/1446#issuecomment-416520694 On java 9 fails only hive and kafka tests (HBase tests surprisely works) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (DRILL-6716) NullPointerException for select query with alias on * symbol
Denys Ordynskiy created DRILL-6716: -- Summary: NullPointerException for select query with alias on * symbol Key: DRILL-6716 URL: https://issues.apache.org/jira/browse/DRILL-6716 Project: Apache Drill Issue Type: Bug Affects Versions: 1.14.0 Reporter: Denys Ordynskiy Fix For: 1.15.0 When I added alias for * in SELECT query in Drill WebUI and in sqlline: SELECT * as testAlias FROM cp.`employee.json` LIMIT 2; I got an error: -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] kkhatua commented on issue #1437: DRILL-6702: Disable CPU Reporting for non-HotSpot JDKs
kkhatua commented on issue #1437: DRILL-6702: Disable CPU Reporting for non-HotSpot JDKs URL: https://github.com/apache/drill/pull/1437#issuecomment-416463479 @arina-ielchiieva / @vrozov The fix has been verified. Let me know if I need to do any other changes, else I can squash and push the commit. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kkhatua commented on a change in pull request #1437: DRILL-6702: Disable CPU Reporting for non-HotSpot JDKs
kkhatua commented on a change in pull request #1437: DRILL-6702: Disable CPU
Reporting for non-HotSpot JDKs
URL: https://github.com/apache/drill/pull/1437#discussion_r213193819
##
File path: common/src/main/java/org/apache/drill/exec/metrics/CpuGaugeSet.java
##
@@ -32,13 +32,23 @@
*/
@SuppressWarnings("restriction")
public class CpuGaugeSet implements MetricSet {
+ private static final org.slf4j.Logger logger =
org.slf4j.LoggerFactory.getLogger(CpuGaugeSet.class);
- private OperatingSystemMXBean osMXBean;
- private RuntimeMXBean rtMXBean;
+ private final OperatingSystemMXBean osMXBean;
+ private final RuntimeMXBean rtMXBean;
public CpuGaugeSet() {
-this.osMXBean = (OperatingSystemMXBean)
ManagementFactory.getOperatingSystemMXBean();
this.rtMXBean = ManagementFactory.getRuntimeMXBean();
+//DRILL-6702: Instead of worrying about compiling with IBM JDK, for now,
we shall provide no CPU metrics for non-HotSpot JVMs
+OperatingSystemMXBean tempOSMXBean = null;
+try {
+ tempOSMXBean = (OperatingSystemMXBean)
ManagementFactory.getOperatingSystemMXBean();
+} catch (ClassCastException cce) {
+ logger.warn("Detected non-HotSpot JVM [{}]. CPU Metrics in the WebUI
will not be available!", System.getProperty("java.vm.name"));
+ tempOSMXBean = null;
+}
+this.osMXBean = tempOSMXBean;
+System.out.println("osMXBean::" + osMXBean);
Review comment:
Agreed. The fix has been verified and I've done the clean up.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
