date:20190909

[GitHub] [incubator-druid] stale[bot] commented on issue #6578: can not infer which one is the count aggregator from segment metadata query result

2019-09-09 Thread GitBox

stale[bot] commented on issue #6578: can not infer which one is the count 
aggregator from segment metadata query result
URL: 
https://github.com/apache/incubator-druid/issues/6578#issuecomment-529783198
 
 
   This issue has been closed due to lack of activity. If you think that is 
incorrect, or the issue requires additional review, you can revive the issue at 
any time.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] stale[bot] closed issue #6578: can not infer which one is the count aggregator from segment metadata query result

2019-09-09 Thread GitBox

stale[bot] closed issue #6578: can not infer which one is the count aggregator 
from segment metadata query result
URL: https://github.com/apache/incubator-druid/issues/6578
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] clintropolis opened a new pull request #8499: [Backport] Fix dependency analyze warnings

2019-09-09 Thread GitBox

clintropolis opened a new pull request #8499: [Backport] Fix dependency analyze 
warnings
URL: https://github.com/apache/incubator-druid/pull/8499
 
 
   Backport of #8230 to 0.16.0-incubating.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[incubator-druid] branch 0.16.0-incubating updated: Exit JVM on curator unhandled errors (#8458) (#8498)

2019-09-09 Thread cwylie

This is an automated email from the ASF dual-hosted git repository.

cwylie pushed a commit to branch 0.16.0-incubating
in repository https://gitbox.apache.org/repos/asf/incubator-druid.git


The following commit(s) were added to refs/heads/0.16.0-incubating by this push:
 new 65f5ac2  Exit JVM on curator unhandled errors (#8458) (#8498)
65f5ac2 is described below

commit 65f5ac23021e00c14d7e53396396b3ad9d6f08a2
Author: Clint Wylie 
AuthorDate: Mon Sep 9 20:53:22 2019 -0700

Exit JVM on curator unhandled errors (#8458) (#8498)

* Exit JVM on curator unhandled errors

If an unhandled error occurs when curator is talking to ZooKeeper, exit
the JVM in addition to stopping the lifecycle to prevent the process
from being left in a zombie state. With this change,
BoundedExponentialBackoffRetryWithQuit is no longer needed as when
curator exceeds the configured retries, it triggers its unhandled error
listeners. A new "connectionTimeoutMs" CuratorConfig setting is added
mostly to facilitate testing curator unhandled errors, but it may be
useful for users as well.

* Address review comments
---
 .../druid/testing/junit/LoggerCaptureRule.java | 106 
 docs/configuration/index.md|   2 +-
 pom.xml|   6 ++
 server/pom.xml |   5 +
 .../BoundedExponentialBackoffRetryWithQuit.java|  63 
 .../org/apache/druid/curator/CuratorConfig.java|  43 
 .../org/apache/druid/curator/CuratorModule.java|  82 ---
 ...BoundedExponentialBackoffRetryWithQuitTest.java | 111 -
 .../apache/druid/curator/CuratorModuleTest.java| 104 +++
 9 files changed, 242 insertions(+), 280 deletions(-)

diff --git 
a/core/src/test/java/org/apache/druid/testing/junit/LoggerCaptureRule.java 
b/core/src/test/java/org/apache/druid/testing/junit/LoggerCaptureRule.java
new file mode 100644
index 000..79c5643
--- /dev/null
+++ b/core/src/test/java/org/apache/druid/testing/junit/LoggerCaptureRule.java
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.testing.junit;
+
+import org.apache.logging.log4j.Level;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.LoggerContext;
+import org.apache.logging.log4j.core.appender.AbstractAppender;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.LoggerConfig;
+import org.junit.rules.ExternalResource;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+/**
+ * JUnit rule to capture a class's logger output to an in-memory buffer to 
allow verification of log messages in tests.
+ */
+public class LoggerCaptureRule extends ExternalResource
+{
+  private final Class targetClass;
+
+  private InMemoryAppender inMemoryAppender;
+  private LoggerConfig targetClassLoggerConfig;
+
+  public LoggerCaptureRule(Class targetClass)
+  {
+this.targetClass = targetClass;
+  }
+
+  @Override
+  protected void before()
+  {
+inMemoryAppender = new InMemoryAppender();
+LoggerContext loggerContext = (LoggerContext) LogManager.getContext(false);
+Configuration configuration = loggerContext.getConfiguration();
+targetClassLoggerConfig = 
configuration.getLoggerConfig(targetClass.getName());
+targetClassLoggerConfig.addAppender(inMemoryAppender, Level.ALL, null);
+  }
+
+  @Override
+  protected void after()
+  {
+clearLogEvents();
+targetClassLoggerConfig.removeAppender(InMemoryAppender.NAME);
+  }
+
+  public List getLogEvents()
+  {
+return inMemoryAppender.getLogEvents();
+  }
+
+  public void clearLogEvents()
+  {
+inMemoryAppender.clearLogEvents();
+  }
+
+  private static class InMemoryAppender extends AbstractAppender
+  {
+static final String NAME = InMemoryAppender.class.getName();
+
+private final List logEvents;
+
+InMemoryAppender()
+{
+  super(NAME, null, null);
+  logEvents = new ArrayList<>();
+}
+
+@Override
+

[GitHub] [incubator-druid] clintropolis merged pull request #8498: [Backport] Exit JVM on curator unhandled errors

2019-09-09 Thread GitBox

clintropolis merged pull request #8498: [Backport] Exit JVM on curator 
unhandled errors
URL: https://github.com/apache/incubator-druid/pull/8498
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] jon-wei commented on a change in pull request #6972: Support LDAP authentication/authorization

2019-09-09 Thread GitBox

jon-wei commented on a change in pull request #6972: Support LDAP 
authentication/authorization
URL: https://github.com/apache/incubator-druid/pull/6972#discussion_r322485924
 
 

 ##
 File path: docs/content/development/extensions-core/druid-basic-security.md
 ##
 @@ -97,6 +125,9 @@ druid.escalator.authorizerName=MyBasicAuthorizer
 |`druid.escalator.internalClientUsername`|The escalator will use this username 
for requests made as the internal systerm user.|n/a|Yes|
 |`druid.escalator.internalClientPassword`|The escalator will use this 
[Password Provider](../../operations/password-provider.html) for requests made 
as the internal system user.|n/a|Yes|
 |`druid.escalator.authorizerName`|Authorizer that requests should be directed 
to.|n/a|Yes|
+|`druid.escalator.enableCacheNotifications`|If true, the Coordinator will 
notify Druid processes whenever a configuration change to this Escalator 
occurs, allowing them to immediately update their state without waiting for 
polling.|true|No|
 
 Review comment:
   These escalator params are no longer used, let's remove them


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] jon-wei commented on a change in pull request #6972: Support LDAP authentication/authorization

2019-09-09 Thread GitBox

jon-wei commented on a change in pull request #6972: Support LDAP 
authentication/authorization
URL: https://github.com/apache/incubator-druid/pull/6972#discussion_r322518732
 
 

 ##
 File path: 
extensions-core/druid-basic-security/src/main/java/org/apache/druid/security/basic/authentication/db/cache/CoordinatorBasicAuthenticatorCacheNotifier.java
 ##
 @@ -42,9 +42,9 @@
 @ManageLifecycle
 public class CoordinatorBasicAuthenticatorCacheNotifier implements 
BasicAuthenticatorCacheNotifier
 {
-
   private final LifecycleLock lifecycleLock = new LifecycleLock();
-  private CommonCacheNotifier cacheNotifier;
+  private final CommonCacheNotifier userCacheNotifier;
+  private final CommonCacheNotifier configCacheNotifier;
 
 Review comment:
   is `configCacheNotifier` still used?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] jon-wei commented on a change in pull request #6972: Support LDAP authentication/authorization

2019-09-09 Thread GitBox

jon-wei commented on a change in pull request #6972: Support LDAP 
authentication/authorization
URL: https://github.com/apache/incubator-druid/pull/6972#discussion_r322515347
 
 

 ##
 File path: docs/content/development/extensions-core/druid-basic-security.md
 ##
 @@ -156,6 +190,27 @@ Example request body:
 }
 ```
 
+`GET(/druid-ext/basic-security/authentication/db/{authenticatorName}/config)`
 
 Review comment:
   These endpoints are not used anymore, let's delete


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] jon-wei commented on a change in pull request #6972: Support LDAP authentication/authorization

2019-09-09 Thread GitBox

jon-wei commented on a change in pull request #6972: Support LDAP 
authentication/authorization
URL: https://github.com/apache/incubator-druid/pull/6972#discussion_r322485750
 
 

 ##
 File path: docs/content/development/extensions-core/druid-basic-security.md
 ##
 @@ -44,42 +44,70 @@ These configuration properties should be added to the 
common runtime properties
 ### Properties
 |Property|Description|Default|required|
 ||---|---||
-|`druid.auth.basic.common.pollingPeriod`|Defines in milliseconds how often 
processes should poll the Coordinator for the current authenticator/authorizer 
database state.|6|No|
+|`druid.auth.basic.common.pollingPeriod`|Defines in milliseconds how often 
processes should poll the Coordinator for the current 
escalator/authenticator/authorizer database state.|6|No|
 |`druid.auth.basic.common.maxRandomDelay`|Defines in milliseconds the amount 
of random delay to add to the pollingPeriod, to spread polling requests across 
time.|6000|No|
 |`druid.auth.basic.common.maxSyncRetries`|Determines how many times a service 
will retry if the authentication/authorization database state sync with the 
Coordinator fails.|10|No|
 |`druid.auth.basic.common.cacheDirectory`|If defined, snapshots of the basic 
Authenticator and Authorizer database caches will be stored on disk in this 
directory. If this property is defined, when a service is starting, it will 
attempt to initialize its caches from these on-disk snapshots, if the service 
is unable to initialize its state by communicating with the 
Coordinator.|null|No|
 
 
-### Creating an Authenticator
+### Creating an Authenticator that uses database to lookup and validate 
credentials
 
 Review comment:
   Suggest `the Druid metadata store` instead of `database` here and elsewhere 
in the docs


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid-website-src] asdf2014 opened a new pull request #46: Remove broken urls

2019-09-09 Thread GitBox

asdf2014 opened a new pull request #46: Remove broken urls
URL: https://github.com/apache/incubator-druid-website-src/pull/46
 
 
   Remove broken urls:
   * 
https://jini.site/jin-yumafengwobig-data-in-travel-real-time-analytics-with-kafka-spark-and-druid_269053
   * http://liquidm.com/reporting-at-liquidm/


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[incubator-druid-website-src] branch remove_borken_url created (now 4bf7fe2)

2019-09-09 Thread asdf2014

This is an automated email from the ASF dual-hosted git repository.

asdf2014 pushed a change to branch remove_borken_url
in repository 
https://gitbox.apache.org/repos/asf/incubator-druid-website-src.git.


  at 4bf7fe2  Remove broken url

This branch includes the following new commits:

 new 4bf7fe2  Remove broken url

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.



-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[incubator-druid-website-src] 01/01: Remove broken url

2019-09-09 Thread asdf2014

This is an automated email from the ASF dual-hosted git repository.

asdf2014 pushed a commit to branch remove_borken_url
in repository 
https://gitbox.apache.org/repos/asf/incubator-druid-website-src.git

commit 4bf7fe279700c80f3f4273689df36de24f5d5975
Author: asdf2014 
AuthorDate: Tue Sep 10 00:22:32 2019 +0800

Remove broken url
---
 druid-powered.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/druid-powered.md b/druid-powered.md
index d0d6911..3bd0f33 100644
--- a/druid-powered.md
+++ b/druid-powered.md
@@ -186,7 +186,7 @@ LifeBuzz is a popular web property that serves tens of 
millions of pageviews per
 
 LiquidM uses Druid for real-time drill-down reporting. LiquidM is also 
contributing back to the community by creating and maintaining a ruby client 
library for interacting with Druid located at 
.
 
-* [Reporting at LiquidM](http://liquidm.com/reporting-at-liquidm/)
+* Reporting at LiquidM
 
 ## Lyft
 
@@ -201,7 +201,7 @@ which they can do that is in a big part thanks to Druid!
 
 ## Mafengwo
 
-* [Real-time Analytics with Kafka, Spark, and 
Druid](https://jini.site/jin-yumafengwobig-data-in-travel-real-time-analytics-with-kafka-spark-and-druid_269053)
+* Real-time Analytics with Kafka, Spark, and Druid
 
 ## MakeMyTrip
 


-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] wc188996 commented on issue #5340: Exceptions in broker: Channel disconnected

2019-09-09 Thread GitBox

wc188996 commented on issue #5340: Exceptions in broker: Channel disconnected
URL: 
https://github.com/apache/incubator-druid/issues/5340#issuecomment-529743458
 
 
   I have the same issue when using imply--2.8.20, My solution is increase the 
druid.processing.numThreads ,the path is 
imply-2.8.20/conf/druid/historical/runtime.properties.
   the occurrence of this error is t0o many queries in one time, and not enough 
threads to work at this time.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] clintropolis opened a new pull request #8498: [Backport] Exit JVM on curator unhandled errors

2019-09-09 Thread GitBox

clintropolis opened a new pull request #8498: [Backport] Exit JVM on curator 
unhandled errors
URL: https://github.com/apache/incubator-druid/pull/8498
 
 
   Backport of #8458 to 0.16.0-incubating.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] clintropolis closed pull request #8497: [Backport] Fix dependency analyze warnings (#8230)

2019-09-09 Thread GitBox

clintropolis closed pull request #8497: [Backport] Fix dependency analyze 
warnings (#8230)
URL: https://github.com/apache/incubator-druid/pull/8497
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] jihoonson commented on issue #8492: Improper Appenderator.add() calls concurrent with persist

2019-09-09 Thread GitBox

jihoonson commented on issue #8492: Improper Appenderator.add() calls 
concurrent with persist
URL: 
https://github.com/apache/incubator-druid/issues/8492#issuecomment-529728974
 
 
   Thank you for creating this issue.
   
   During ingestion, `persist()` can happen multiple times for the same 
segment. `Sink` is a logical representation of a single segment and can have 
multiple `FireHydrant` which could hold an `IncrementalIndex` in memory or a 
`QueryableIndex` in disk. Note that each of these `IncrementalIndex` or 
`QueryableIndex` is a part of the same segment and should be merged before it's 
pushed to deep storage. In `AppenderatorImpl.persistAll()`, it persists all 
parts of the segments in memory into disk. How the persist happens is, the 
"main" thread first [creates a new `FireHydrant` with a new 
`IncrementalIndex`](https://github.com/apache/incubator-druid/blob/master/server/src/main/java/org/apache/druid/segment/realtime/appenderator/AppenderatorImpl.java#L561),
 and then the "persist" thread [performs the persist in 
background](https://github.com/apache/incubator-druid/blob/master/server/src/main/java/org/apache/druid/segment/realtime/appenderator/AppenderatorImpl.java#L579).
 As a result, when the "main" thread calls `AppenderatorImpl.add()` next time, 
the new row will be added to the new `Firehydrant` instead of the one which is 
being persisted.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] clintropolis opened a new pull request #8497: [Backport] Fix dependency analyze warnings (#8230)

2019-09-09 Thread GitBox

clintropolis opened a new pull request #8497: [Backport] Fix dependency analyze 
warnings (#8230)
URL: https://github.com/apache/incubator-druid/pull/8497
 
 
   Backport of #8405 to 0.16.0-incubating.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[incubator-druid] branch 0.16.0-incubating updated: Bump ORC library to 1.5.6 (#8405) (#8496)

2019-09-09 Thread cwylie

This is an automated email from the ASF dual-hosted git repository.

cwylie pushed a commit to branch 0.16.0-incubating
in repository https://gitbox.apache.org/repos/asf/incubator-druid.git


The following commit(s) were added to refs/heads/0.16.0-incubating by this push:
 new 2ff5532  Bump ORC library to 1.5.6 (#8405) (#8496)
2ff5532 is described below

commit 2ff5532daf68c457942da79f48bf34014d43fe26
Author: Clint Wylie 
AuthorDate: Mon Sep 9 18:20:48 2019 -0700

Bump ORC library to 1.5.6 (#8405) (#8496)

Changelog at:
https://orc.apache.org/docs/releases.html#current-release---156
---
 extensions-core/orc-extensions/pom.xml | 2 +-
 licenses.yaml  | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/extensions-core/orc-extensions/pom.xml 
b/extensions-core/orc-extensions/pom.xml
index f6d9d62..97c62b0 100644
--- a/extensions-core/orc-extensions/pom.xml
+++ b/extensions-core/orc-extensions/pom.xml
@@ -33,7 +33,7 @@
 
 4.0.0
 
-1.5.5
+1.5.6
 
 
 
diff --git a/licenses.yaml b/licenses.yaml
index 776612a..f223d7f 100644
--- a/licenses.yaml
+++ b/licenses.yaml
@@ -3880,7 +3880,7 @@ name: Apache ORC libraries
 license_category: binary
 module: extensions/druid-orc-extensions
 license_name: Apache License version 2.0
-version: 1.5.5
+version: 1.5.6
 libraries:
   - org.apache.orc: orc-mapreduce
   - org.apache.orc: orc-core


-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] clintropolis commented on issue #8496: [Backport] Bump ORC library to 1.5.6

2019-09-09 Thread GitBox

clintropolis commented on issue #8496: [Backport] Bump ORC library to 1.5.6
URL: https://github.com/apache/incubator-druid/pull/8496#issuecomment-529727700
 
 
   skipping CI for single dependency change, will let the next one run


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] clintropolis merged pull request #8496: [Backport] Bump ORC library to 1.5.6

2019-09-09 Thread GitBox

clintropolis merged pull request #8496: [Backport] Bump ORC library to 1.5.6
URL: https://github.com/apache/incubator-druid/pull/8496
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] clintropolis opened a new pull request #8496: [Backport] Bump ORC library to 1.5.6

2019-09-09 Thread GitBox

clintropolis opened a new pull request #8496: [Backport] Bump ORC library to 
1.5.6
URL: https://github.com/apache/incubator-druid/pull/8496
 
 
   Backport of #8405 to 0.16.0-incubating.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] clintropolis commented on issue #8230: Fix dependency analyze warnings

2019-09-09 Thread GitBox

clintropolis commented on issue #8230: Fix dependency analyze warnings
URL: https://github.com/apache/incubator-druid/pull/8230#issuecomment-529727243
 
 
   This seems like it would clean up distribution jars so I'm going to backport 
to 0.16-incubating, which also involves pulling in #8405 since it went in first.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] clintropolis commented on a change in pull request #8495: Check targetCompactionSizeBytes to search for candidate segments in auto compaction

2019-09-09 Thread GitBox

clintropolis commented on a change in pull request #8495: Check 
targetCompactionSizeBytes to search for candidate segments in auto compaction
URL: https://github.com/apache/incubator-druid/pull/8495#discussion_r322510975
 
 

 ##
 File path: 
server/src/main/java/org/apache/druid/server/coordinator/helper/SegmentCompactorUtil.java
 ##
 @@ -20,21 +20,36 @@
 package org.apache.druid.server.coordinator.helper;
 
 import com.google.common.base.Preconditions;
+import org.apache.druid.timeline.DataSegment;
 import org.joda.time.Interval;
 
+import javax.annotation.Nullable;
+import java.util.List;
+
 /**
  * Util class used by {@link DruidCoordinatorSegmentCompactor} and {@link 
CompactionSegmentSearchPolicy}.
  */
 class SegmentCompactorUtil
 {
-  static boolean isCompactibleSize(long targetBytes, long currentTotalBytes, 
long additionalBytes)
-  {
-return currentTotalBytes + additionalBytes <= targetBytes;
-  }
+  /**
+   * The allowed error rate of the segment size after compaction.
+   * Its value is determined experimentally.
+   */
+  private static final double ALLOWED_ERROR_OF_SEGMENT_SIZE = .2;
 
-  static boolean isCompactibleNum(int numTargetSegments, int 
numCurrentSegments, int numAdditionalSegments)
+  static boolean needsCompaction(@Nullable Long targetCompactionSizeBytes, 
List candidates)
   {
-return numCurrentSegments + numAdditionalSegments <= numTargetSegments;
+if (targetCompactionSizeBytes == null) {
+  // If targetCompactionSizeBytes is null, we have no way to check that 
the given segments need compaction or not.
+  return true;
+}
+final double minAllowedSize = targetCompactionSizeBytes * (1 - 
ALLOWED_ERROR_OF_SEGMENT_SIZE);
+final double maxAllowedSize = targetCompactionSizeBytes * (1 + 
ALLOWED_ERROR_OF_SEGMENT_SIZE);
+
+return candidates
+.stream()
+.filter(segment -> segment.getSize() < minAllowedSize || 
segment.getSize() > maxAllowedSize)
+.count() > 1;
 
 Review comment:
   ~nit: I think this should use `anyMatch` instead of `filter` + `count` since 
the former might not have to materialize the entire set upon reaching first 
match~


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] clintropolis commented on a change in pull request #8495: Check targetCompactionSizeBytes to search for candidate segments in auto compaction

2019-09-09 Thread GitBox

clintropolis commented on a change in pull request #8495: Check 
targetCompactionSizeBytes to search for candidate segments in auto compaction
URL: https://github.com/apache/incubator-druid/pull/8495#discussion_r322511684
 
 

 ##
 File path: 
server/src/main/java/org/apache/druid/server/coordinator/helper/SegmentCompactorUtil.java
 ##
 @@ -20,21 +20,36 @@
 package org.apache.druid.server.coordinator.helper;
 
 import com.google.common.base.Preconditions;
+import org.apache.druid.timeline.DataSegment;
 import org.joda.time.Interval;
 
+import javax.annotation.Nullable;
+import java.util.List;
+
 /**
  * Util class used by {@link DruidCoordinatorSegmentCompactor} and {@link 
CompactionSegmentSearchPolicy}.
  */
 class SegmentCompactorUtil
 {
-  static boolean isCompactibleSize(long targetBytes, long currentTotalBytes, 
long additionalBytes)
-  {
-return currentTotalBytes + additionalBytes <= targetBytes;
-  }
+  /**
+   * The allowed error rate of the segment size after compaction.
+   * Its value is determined experimentally.
+   */
+  private static final double ALLOWED_ERROR_OF_SEGMENT_SIZE = .2;
 
-  static boolean isCompactibleNum(int numTargetSegments, int 
numCurrentSegments, int numAdditionalSegments)
+  static boolean needsCompaction(@Nullable Long targetCompactionSizeBytes, 
List candidates)
   {
-return numCurrentSegments + numAdditionalSegments <= numTargetSegments;
+if (targetCompactionSizeBytes == null) {
+  // If targetCompactionSizeBytes is null, we have no way to check that 
the given segments need compaction or not.
+  return true;
+}
+final double minAllowedSize = targetCompactionSizeBytes * (1 - 
ALLOWED_ERROR_OF_SEGMENT_SIZE);
+final double maxAllowedSize = targetCompactionSizeBytes * (1 + 
ALLOWED_ERROR_OF_SEGMENT_SIZE);
+
+return candidates
+.stream()
+.filter(segment -> segment.getSize() < minAllowedSize || 
segment.getSize() > maxAllowedSize)
+.count() > 1;
 
 Review comment:
   ignore ^


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] clintropolis commented on a change in pull request #8495: Check targetCompactionSizeBytes to search for candidate segments in auto compaction

2019-09-09 Thread GitBox

clintropolis commented on a change in pull request #8495: Check 
targetCompactionSizeBytes to search for candidate segments in auto compaction
URL: https://github.com/apache/incubator-druid/pull/8495#discussion_r322510975
 
 

 ##
 File path: 
server/src/main/java/org/apache/druid/server/coordinator/helper/SegmentCompactorUtil.java
 ##
 @@ -20,21 +20,36 @@
 package org.apache.druid.server.coordinator.helper;
 
 import com.google.common.base.Preconditions;
+import org.apache.druid.timeline.DataSegment;
 import org.joda.time.Interval;
 
+import javax.annotation.Nullable;
+import java.util.List;
+
 /**
  * Util class used by {@link DruidCoordinatorSegmentCompactor} and {@link 
CompactionSegmentSearchPolicy}.
  */
 class SegmentCompactorUtil
 {
-  static boolean isCompactibleSize(long targetBytes, long currentTotalBytes, 
long additionalBytes)
-  {
-return currentTotalBytes + additionalBytes <= targetBytes;
-  }
+  /**
+   * The allowed error rate of the segment size after compaction.
+   * Its value is determined experimentally.
+   */
+  private static final double ALLOWED_ERROR_OF_SEGMENT_SIZE = .2;
 
-  static boolean isCompactibleNum(int numTargetSegments, int 
numCurrentSegments, int numAdditionalSegments)
+  static boolean needsCompaction(@Nullable Long targetCompactionSizeBytes, 
List candidates)
   {
-return numCurrentSegments + numAdditionalSegments <= numTargetSegments;
+if (targetCompactionSizeBytes == null) {
+  // If targetCompactionSizeBytes is null, we have no way to check that 
the given segments need compaction or not.
+  return true;
+}
+final double minAllowedSize = targetCompactionSizeBytes * (1 - 
ALLOWED_ERROR_OF_SEGMENT_SIZE);
+final double maxAllowedSize = targetCompactionSizeBytes * (1 + 
ALLOWED_ERROR_OF_SEGMENT_SIZE);
+
+return candidates
+.stream()
+.filter(segment -> segment.getSize() < minAllowedSize || 
segment.getSize() > maxAllowedSize)
+.count() > 1;
 
 Review comment:
   nit: I think this should use `anyMatch` instead of `filter` + `count` since 
the former might not have to materialize the entire set upon reaching first 
match


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] jihoonson opened a new pull request #8495: Check targetCompactionSizeBytes to search for candidate segments in auto compaction

2019-09-09 Thread GitBox

jihoonson opened a new pull request #8495: Check targetCompactionSizeBytes to 
search for candidate segments in auto compaction
URL: https://github.com/apache/incubator-druid/pull/8495
 
 
   ### Description
   
   This PR is to resolve the release blocker by temporarily addressing the bug 
reported in https://github.com/apache/incubator-druid/issues/8481. The auto 
compaction will check `targetCompactionSizeBytes` with this PR. The real 
solution should be something better such as 
https://github.com/apache/incubator-druid/issues/8489.
   
   It should be mentioned that the auto compaction can get stuck if 
`maxRowsPerSegment` or `maxTotalRows` is set instead of 
`targetCompactionSizeBytes`.
   
   
   
   This PR has:
   - [x] been self-reviewed.
   - [x] added Javadocs for most classes and all non-trivial methods. Linked 
related entities via Javadoc links.
   - [x] added unit tests or modified existing tests to cover new code paths.
   - [x] been tested in a test Druid cluster.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] clintropolis commented on issue #8230: Fix dependency analyze warnings

2019-09-09 Thread GitBox

clintropolis commented on issue #8230: Fix dependency analyze warnings
URL: https://github.com/apache/incubator-druid/pull/8230#issuecomment-529682971
 
 
   I tested that orc hadoop indexing is still functional after these changes, 
:+1:


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[incubator-druid] branch master updated (58e2634 -> 5f61374)

2019-09-09 Thread jonwei

This is an automated email from the ASF dual-hosted git repository.

jonwei pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-druid.git.


from 58e2634  Update RoaringBitmap version to 0.8.11 (#8490)
 add 5f61374  Fix dependency analyze warnings (#8230)

No new revisions were added by this update.

Summary of changes:
 .travis.yml|  22 +++
 benchmarks/pom.xml |  61 --
 cloud/aws-common/pom.xml   |  29 ++-
 cloud/gcp-common/pom.xml   |  18 ++
 codestyle/druid-forbidden-apis.txt |   1 +
 core/pom.xml   |  58 --
 extensions-contrib/ambari-metrics-emitter/pom.xml  |  41 
 extensions-contrib/azure-extensions/pom.xml|  49 +
 extensions-contrib/cassandra-storage/pom.xml   |  34 
 extensions-contrib/cloudfiles-extensions/pom.xml   |  64 +--
 extensions-contrib/distinctcount/pom.xml   |  46 +
 extensions-contrib/graphite-emitter/pom.xml|  47 +
 extensions-contrib/influx-extensions/pom.xml   |  39 ++--
 extensions-contrib/influxdb-emitter/pom.xml|  46 +++--
 extensions-contrib/kafka-emitter/pom.xml   |  37 
 .../materialized-view-maintenance/pom.xml  |  63 +-
 .../materialized-view-selection/pom.xml|  65 ++-
 extensions-contrib/momentsketch/pom.xml|  26 +++
 extensions-contrib/moving-average-query/pom.xml|  46 +
 extensions-contrib/opentsdb-emitter/pom.xml|  40 
 extensions-contrib/redis-cache/pom.xml |  26 +++
 .../sqlserver-metadata-storage/pom.xml |  25 +++
 extensions-contrib/statsd-emitter/pom.xml  |  30 +++
 extensions-contrib/tdigestsketch/pom.xml   |   5 +
 extensions-contrib/thrift-extensions/pom.xml   |  44 +
 extensions-contrib/time-min-max/pom.xml|  35 
 extensions-contrib/virtual-columns/pom.xml |  30 +++
 extensions-core/avro-extensions/pom.xml|  81 +++-
 extensions-core/datasketches/pom.xml   |  46 -
 extensions-core/druid-basic-security/pom.xml   |  71 +++
 extensions-core/druid-bloom-filter/pom.xml |  52 +
 extensions-core/druid-kerberos/pom.xml | 105 ++
 extensions-core/ec2-extensions/pom.xml |  42 
 extensions-core/google-extensions/pom.xml  |  66 +++
 extensions-core/hdfs-storage/pom.xml   | 124 +++-
 extensions-core/histogram/pom.xml  |  57 ++
 extensions-core/kafka-extraction-namespace/pom.xml |  56 ++
 extensions-core/kafka-indexing-service/pom.xml |  73 +++
 extensions-core/kinesis-indexing-service/pom.xml   |  54 +-
 extensions-core/lookups-cached-global/pom.xml  |  64 +++
 extensions-core/lookups-cached-single/pom.xml  |  36 
 extensions-core/mysql-metadata-storage/pom.xml |  30 +++
 extensions-core/orc-extensions/pom.xml |  92 -
 extensions-core/parquet-extensions/pom.xml |  88 -
 .../postgresql-metadata-storage/pom.xml|  30 +++
 extensions-core/protobuf-extensions/pom.xml|  32 
 extensions-core/s3-extensions/pom.xml  |  61 ++
 extensions-core/simple-client-sslcontext/pom.xml   |  20 ++
 extensions-core/stats/pom.xml  |  47 +
 hll/pom.xml|  12 +-
 indexing-hadoop/pom.xml|  50 -
 indexing-service/pom.xml   | 152 +++
 integration-tests/pom.xml  |  92 -
 .../tests/indexer/AbstractITBatchIndexTest.java|   8 +-
 licenses.yaml  |  70 +--
 pom.xml| 213 -
 processing/pom.xml |  46 -
 server/pom.xml | 196 +--
 services/pom.xml   |  90 +
 .../java/org/apache/druid/cli/ExportMetadata.java  |  22 +--
 sql/pom.xml|  86 -
 .../druid/sql/calcite/schema/DruidSchema.java  |   2 +-
 62 files changed, 3170 insertions(+), 223 deletions(-)


-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] jon-wei merged pull request #8230: Fix dependency analyze warnings

2019-09-09 Thread GitBox

jon-wei merged pull request #8230: Fix dependency analyze warnings
URL: https://github.com/apache/incubator-druid/pull/8230
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] jihoonson commented on issue #8494: Fix batch index task naming convention to accommodate simultaneous tasks

2019-09-09 Thread GitBox

jihoonson commented on issue #8494: Fix batch index task naming convention to 
accommodate simultaneous tasks
URL: 
https://github.com/apache/incubator-druid/issues/8494#issuecomment-529674088
 
 
   Ah thanks. It sound good!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] jon-wei commented on issue #6972: Support LDAP authentication/authorization

2019-09-09 Thread GitBox

jon-wei commented on issue #6972: Support LDAP authentication/authorization
URL: https://github.com/apache/incubator-druid/pull/6972#issuecomment-529673502
 
 
   @mohammadjkhan @nishantmonu51 @capistrant 
   
   Sorry for the late review, I finally have some free cycles now, I think this 
would most likely make it into 0.17.0


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] himanshug merged pull request #8490: Update RoaringBitmap version to 0.8.11

2019-09-09 Thread GitBox

himanshug merged pull request #8490: Update RoaringBitmap version to 0.8.11
URL: https://github.com/apache/incubator-druid/pull/8490
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[incubator-druid] branch master updated (2b04c22 -> 58e2634)

2019-09-09 Thread himanshug

This is an automated email from the ASF dual-hosted git repository.

himanshug pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-druid.git.


from 2b04c22  split segment view query into two (#8485)
 add 58e2634  Update RoaringBitmap version to 0.8.11 (#8490)

No new revisions were added by this update.

Summary of changes:
 licenses.yaml | 2 +-
 pom.xml   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)


-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] a2l007 commented on issue #8494: Fix batch index task naming convention to accommodate simultaneous tasks

2019-09-09 Thread GitBox

a2l007 commented on issue #8494: Fix batch index task naming convention to 
accommodate simultaneous tasks
URL: 
https://github.com/apache/incubator-druid/issues/8494#issuecomment-529642411
 
 
   @jihoonson Oops sorry, I meant to remove the timestamp from the new 
convention. I've updated the description accordingly.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] jihoonson commented on issue #8494: Fix batch index task naming convention to accommodate simultaneous tasks

2019-09-09 Thread GitBox

jihoonson commented on issue #8494: Fix batch index task naming convention to 
accommodate simultaneous tasks
URL: 
https://github.com/apache/incubator-druid/issues/8494#issuecomment-529637423
 
 
   @a2l007 thanks for raising this issue! I like it. One question. Do we still 
need the timestamp at the end of the id? I guess its intention was also to 
avoid duplicate task IDs.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] a2l007 opened a new issue #8494: Fix batch index task naming convention to accommodate simultaneous tasks

2019-09-09 Thread GitBox

a2l007 opened a new issue #8494: Fix batch index task naming convention to 
accommodate simultaneous tasks
URL: https://github.com/apache/incubator-druid/issues/8494
 
 
   ### Description
   Modify the batch indexing task ID generation logic from 
`__timestamp` to 
`___timestamp`
   This would impact native index tasks as well as hadoop index tasks.
   
   ### Motivation
   With the existing task naming convention, if there are multiple indexing 
tasks submitted at the same time for a datasource, only one of the tasks gets 
accepted and the remaining tasks fail with  `Task[xxx] already exists`. I'd 
like to modify the naming convention to the one mentioned above, so that this 
issue can be avoided.
   @jihoonson Any comments regarding this?
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] renevan10 commented on a change in pull request #8484: Web console: Druid status displayed in a table

2019-09-09 Thread GitBox

renevan10 commented on a change in pull request #8484: Web console: Druid 
status displayed in a table
URL: https://github.com/apache/incubator-druid/pull/8484#discussion_r322384359
 
 

 ##
 File path: web-console/src/dialogs/status-dialog/status-dialog.tsx
 ##
 @@ -16,29 +16,97 @@
  * limitations under the License.
  */
 
-import { Button, Classes, Dialog, Intent } from '@blueprintjs/core';
+import { Button, Classes, Dialog, InputGroup, Intent } from 
'@blueprintjs/core';
+import axios from 'axios';
 import React from 'react';
+import ReactTable from 'react-table';
 
-import { ShowJson } from '../../components';
 import { UrlBaser } from '../../singletons/url-baser';
+import { QueryManager } from '../../utils';
 
 import './status-dialog.scss';
 
 interface StatusDialogProps {
   onClose: () => void;
 }
 
-export class StatusDialog extends React.PureComponent {
+interface StatusDialogState {
+  response: any;
+  loading: boolean;
+  version: string;
+}
+
+export class StatusDialog extends React.PureComponent {
+  private showStatusQueryManager: QueryManager;
+  constructor(props: StatusDialogProps, context: any) {
+super(props, context);
+this.state = {
+  response: [],
+  loading: false,
+  version: '',
+};
+this.showStatusQueryManager = new QueryManager({
+  processQuery: async () => {
+const endpoint = UrlBaser.base(`/status`);
+const resp = await axios.get(endpoint);
+this.setState({ version: 'Version ' + resp.data.version });
+return resp.data.modules;
+  },
+  onStateChange: ({ result, loading }) => {
+this.setState({
+  loading,
+  response: result,
+});
+  },
+});
+  }
+
+  componentDidMount(): void {
+this.showStatusQueryManager.runQuery(null);
+  }
+
   render(): JSX.Element {
 const { onClose } = this.props;
-
+const { response, loading, version } = this.state;
 return (
   
 
-  
+  
+

[GitHub] [incubator-druid] leventov opened a new pull request #8493: Add new items to concurrency code review checklist

2019-09-09 Thread GitBox

leventov opened a new pull request #8493: Add new items to concurrency code 
review checklist
URL: https://github.com/apache/incubator-druid/pull/8493
 
 
   Adds many new checklist items (reflects changes in the reference repo: 
https://github.com/code-review-checklists/java-concurrency), and clarifies the 
wording of some existing questions.
   
   
   
   This PR has:
   - [x] been self-reviewed.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] ccaominh commented on issue #8490: Update RoaringBitmap version to 0.8.11

2019-09-09 Thread GitBox

ccaominh commented on issue #8490: Update RoaringBitmap version to 0.8.11
URL: https://github.com/apache/incubator-druid/pull/8490#issuecomment-529597088
 
 
   The entry in 
https://github.com/apache/incubator-druid/blob/master/licenses.yaml#L1872 needs 
to be updated to have the new version number as well.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] vogievetsky commented on a change in pull request #8484: Web console: Druid status displayed in a table

2019-09-09 Thread GitBox

vogievetsky commented on a change in pull request #8484: Web console: Druid 
status displayed in a table
URL: https://github.com/apache/incubator-druid/pull/8484#discussion_r322360577
 
 

 ##
 File path: web-console/src/dialogs/status-dialog/status-dialog.tsx
 ##
 @@ -16,29 +16,97 @@
  * limitations under the License.
  */
 
-import { Button, Classes, Dialog, Intent } from '@blueprintjs/core';
+import { Button, Classes, Dialog, InputGroup, Intent } from 
'@blueprintjs/core';
+import axios from 'axios';
 import React from 'react';
+import ReactTable from 'react-table';
 
-import { ShowJson } from '../../components';
 import { UrlBaser } from '../../singletons/url-baser';
+import { QueryManager } from '../../utils';
 
 import './status-dialog.scss';
 
 interface StatusDialogProps {
   onClose: () => void;
 }
 
-export class StatusDialog extends React.PureComponent {
+interface StatusDialogState {
+  response: any;
+  loading: boolean;
+  version: string;
+}
+
+export class StatusDialog extends React.PureComponent {
+  private showStatusQueryManager: QueryManager;
+  constructor(props: StatusDialogProps, context: any) {
+super(props, context);
+this.state = {
+  response: [],
+  loading: false,
+  version: '',
+};
+this.showStatusQueryManager = new QueryManager({
+  processQuery: async () => {
+const endpoint = UrlBaser.base(`/status`);
+const resp = await axios.get(endpoint);
+this.setState({ version: 'Version ' + resp.data.version });
 
 Review comment:
   do not call `setState` in the `processQuery` function it is assumes to be a 
pure function


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] vogievetsky commented on a change in pull request #8484: Web console: Druid status displayed in a table

2019-09-09 Thread GitBox

vogievetsky commented on a change in pull request #8484: Web console: Druid 
status displayed in a table
URL: https://github.com/apache/incubator-druid/pull/8484#discussion_r322361257
 
 

 ##
 File path: web-console/src/dialogs/status-dialog/status-dialog.tsx
 ##
 @@ -16,29 +16,97 @@
  * limitations under the License.
  */
 
-import { Button, Classes, Dialog, Intent } from '@blueprintjs/core';
+import { Button, Classes, Dialog, InputGroup, Intent } from 
'@blueprintjs/core';
+import axios from 'axios';
 import React from 'react';
+import ReactTable from 'react-table';
 
-import { ShowJson } from '../../components';
 import { UrlBaser } from '../../singletons/url-baser';
+import { QueryManager } from '../../utils';
 
 import './status-dialog.scss';
 
 interface StatusDialogProps {
   onClose: () => void;
 }
 
-export class StatusDialog extends React.PureComponent {
+interface StatusDialogState {
+  response: any;
+  loading: boolean;
+  version: string;
+}
+
+export class StatusDialog extends React.PureComponent {
+  private showStatusQueryManager: QueryManager;
 
 Review comment:
   the return type should be more complex than a `string` so you could return 
the version and the modules at the same time and now need to call setState 
where it is not needed


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] vogievetsky commented on a change in pull request #8484: Web console: Druid status displayed in a table

2019-09-09 Thread GitBox

vogievetsky commented on a change in pull request #8484: Web console: Druid 
status displayed in a table
URL: https://github.com/apache/incubator-druid/pull/8484#discussion_r322359910
 
 

 ##
 File path: web-console/src/dialogs/status-dialog/status-dialog.tsx
 ##
 @@ -16,29 +16,97 @@
  * limitations under the License.
  */
 
-import { Button, Classes, Dialog, Intent } from '@blueprintjs/core';
+import { Button, Classes, Dialog, InputGroup, Intent } from 
'@blueprintjs/core';
+import axios from 'axios';
 import React from 'react';
+import ReactTable from 'react-table';
 
-import { ShowJson } from '../../components';
 import { UrlBaser } from '../../singletons/url-baser';
+import { QueryManager } from '../../utils';
 
 import './status-dialog.scss';
 
 interface StatusDialogProps {
   onClose: () => void;
 }
 
-export class StatusDialog extends React.PureComponent {
+interface StatusDialogState {
+  response: any;
+  loading: boolean;
+  version: string;
+}
+
+export class StatusDialog extends React.PureComponent {
+  private showStatusQueryManager: QueryManager;
+  constructor(props: StatusDialogProps, context: any) {
+super(props, context);
+this.state = {
+  response: [],
+  loading: false,
+  version: '',
+};
+this.showStatusQueryManager = new QueryManager({
+  processQuery: async () => {
+const endpoint = UrlBaser.base(`/status`);
+const resp = await axios.get(endpoint);
+this.setState({ version: 'Version ' + resp.data.version });
+return resp.data.modules;
+  },
+  onStateChange: ({ result, loading }) => {
+this.setState({
+  loading,
+  response: result,
+});
+  },
+});
+  }
+
+  componentDidMount(): void {
+this.showStatusQueryManager.runQuery(null);
+  }
+
   render(): JSX.Element {
 const { onClose } = this.props;
-
+const { response, loading, version } = this.state;
 return (
   
 
-  
+  
+

[GitHub] [incubator-druid] leventov opened a new issue #8492: Improper Appenderator.add() calls concurrent with persist

2019-09-09 Thread GitBox

leventov opened a new issue #8492: Improper Appenderator.add() calls concurrent 
with persist
URL: https://github.com/apache/incubator-druid/issues/8492
 
 
   This issue is to track NPE reported in 
https://github.com/RoaringBitmap/RoaringBitmap/issues/358 but it doesn't seem 
that it may be caused by a bug in RoaringBitmap. Hence I think it's caused by a 
bug in Druid.
   
   @jihoonson 
   
   > the read operation (persist operation in Druid) happens only when all 
write operations (indexing operation in Druid) are finished.
   
   Could you demonstrate which synchronization mechanism or a happens-before 
edge or a casualty edge specifically ensure this? I cannot find it. It seems to 
happen somewhere in `SeekableStreamIndexTaskRunner.runInternal()` (related: 
https://github.com/apache/incubator-druid/issues/7360).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] waywtd commented on issue #8491: Fix missing space in string literal and spurious Javadoc @param tags from LGTM

2019-09-09 Thread GitBox

waywtd commented on issue #8491: Fix missing space in string literal and 
spurious Javadoc @param tags from LGTM
URL: https://github.com/apache/incubator-druid/pull/8491#issuecomment-529367695
 
 
   666


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] lgtm-com[bot] commented on issue #8491: Fix missing space in string literal and spurious Javadoc @param tags from LGTM

2019-09-09 Thread GitBox

lgtm-com[bot] commented on issue #8491: Fix missing space in string literal and 
spurious Javadoc @param tags from LGTM
URL: https://github.com/apache/incubator-druid/pull/8491#issuecomment-529346507
 
 
   This pull request **fixes 4 alerts** when merging 
aad95c37bfb34a1975c465bbe4c85b869b6ad943 into 
2b04c22bfd157601b8ca5d705ffdd89b930e9a05 - [view on 
LGTM.com](https://lgtm.com/projects/g/apache/incubator-druid/rev/pr-daf3563992637e8b8738813fda6519adae8e7184)
   
   **fixed alerts:**
   
   * 2 for Spurious Javadoc @param tags
   * 2 for Missing space in string literal


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

[GitHub] [incubator-druid] asdf2014 opened a new pull request #8491: Fix missing space in string literal and spurious Javadoc @param tags from LGTM

2019-09-09 Thread GitBox

asdf2014 opened a new pull request #8491: Fix missing space in string literal 
and spurious Javadoc @param tags from LGTM
URL: https://github.com/apache/incubator-druid/pull/8491
 
 
   
   
   
   
   
   
   
   
   ### Description
   
   
   
   
   
   
   
   Fix missing space in string literal and spurious Javadoc @param tags from 
LGTM
   
   
   
   
   
   
   
   
   
   
   
   
   This PR has:
   - [x] been self-reviewed.
  - [x] using the [concurrency 
checklist](https://github.com/apache/incubator-druid/blob/master/dev/code-review/concurrency.md)
 (Remove this item if the PR doesn't have any relation to concurrency.)
   - [ ] added documentation for new or modified features or behaviors.
   - [ ] added Javadocs for most classes and all non-trivial methods. Linked 
related entities via Javadoc links.
   - [ ] added or updated version, license, or notice information in 
[licenses.yaml](https://github.com/apache/incubator-druid/blob/master/licenses.yaml)
   - [ ] added comments explaining the "why" and the intent of the code 
wherever would not be obvious for an unfamiliar reader.
   - [ ] added unit tests or modified existing tests to cover new code paths.
   - [ ] added integration tests.
   - [ ] been tested in a test Druid cluster.
   
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

46 matches

Mail list logo