[GitHub] [druid] fan-code commented on pull request #9579: Add Apache Ranger Authorization

2020-09-04 Thread GitBox
fan-code commented on pull request #9579: URL: https://github.com/apache/druid/pull/9579#issuecomment-686983951 @bolkedebruin How long can druid with ranger's test case be used? See the document, I find it's diffucut to use ranger inside the druid ---

[GitHub] [druid] abhishekagarwal87 commented on a change in pull request #10338: Vectorized ANY aggregators

2020-09-04 Thread GitBox
abhishekagarwal87 commented on a change in pull request #10338: URL: https://github.com/apache/druid/pull/10338#discussion_r483453361 ## File path: processing/src/main/java/org/apache/druid/query/aggregation/any/NumericAnyVectorAggregator.java ## @@ -0,0 +1,128 @@ +/* + * Lice

[GitHub] [druid] abhishekagarwal87 commented on a change in pull request #10338: Vectorized ANY aggregators

2020-09-04 Thread GitBox
abhishekagarwal87 commented on a change in pull request #10338: URL: https://github.com/apache/druid/pull/10338#discussion_r483453937 ## File path: processing/src/main/java/org/apache/druid/query/aggregation/any/NumericAnyVectorAggregator.java ## @@ -0,0 +1,128 @@ +/* + * Lice

[GitHub] [druid] abhishekagarwal87 commented on a change in pull request #10338: Vectorized ANY aggregators

2020-09-04 Thread GitBox
abhishekagarwal87 commented on a change in pull request #10338: URL: https://github.com/apache/druid/pull/10338#discussion_r483458719 ## File path: processing/src/main/java/org/apache/druid/query/aggregation/any/NumericAnyVectorAggregator.java ## @@ -0,0 +1,128 @@ +/* + * Lice

[GitHub] [druid] abhishekagarwal87 commented on a change in pull request #10338: Vectorized ANY aggregators

2020-09-04 Thread GitBox
abhishekagarwal87 commented on a change in pull request #10338: URL: https://github.com/apache/druid/pull/10338#discussion_r483460415 ## File path: processing/src/main/java/org/apache/druid/query/aggregation/any/NumericAnyVectorAggregator.java ## @@ -0,0 +1,128 @@ +/* + * Lice

[GitHub] [druid] abhishekagarwal87 commented on a change in pull request #10338: Vectorized ANY aggregators

2020-09-04 Thread GitBox
abhishekagarwal87 commented on a change in pull request #10338: URL: https://github.com/apache/druid/pull/10338#discussion_r483462463 ## File path: processing/src/main/java/org/apache/druid/query/aggregation/any/NumericAnyVectorAggregator.java ## @@ -0,0 +1,128 @@ +/* + * Lice

[GitHub] [druid] abhishekagarwal87 commented on a change in pull request #10338: Vectorized ANY aggregators

2020-09-04 Thread GitBox
abhishekagarwal87 commented on a change in pull request #10338: URL: https://github.com/apache/druid/pull/10338#discussion_r483462463 ## File path: processing/src/main/java/org/apache/druid/query/aggregation/any/NumericAnyVectorAggregator.java ## @@ -0,0 +1,128 @@ +/* + * Lice

[GitHub] [druid] abhishekagarwal87 commented on a change in pull request #10338: Vectorized ANY aggregators

2020-09-04 Thread GitBox
abhishekagarwal87 commented on a change in pull request #10338: URL: https://github.com/apache/druid/pull/10338#discussion_r483467418 ## File path: processing/src/main/java/org/apache/druid/query/aggregation/any/NumericAnyVectorAggregator.java ## @@ -0,0 +1,128 @@ +/* + * Lice

[GitHub] [druid] abhishekagarwal87 commented on a change in pull request #10338: Vectorized ANY aggregators

2020-09-04 Thread GitBox
abhishekagarwal87 commented on a change in pull request #10338: URL: https://github.com/apache/druid/pull/10338#discussion_r483470098 ## File path: processing/src/main/java/org/apache/druid/query/aggregation/any/NumericAnyVectorAggregator.java ## @@ -0,0 +1,128 @@ +/* + * Lice

[GitHub] [druid] abhishekagarwal87 commented on a change in pull request #10338: Vectorized ANY aggregators

2020-09-04 Thread GitBox
abhishekagarwal87 commented on a change in pull request #10338: URL: https://github.com/apache/druid/pull/10338#discussion_r483474959 ## File path: processing/src/main/java/org/apache/druid/query/aggregation/any/NumericAnyVectorAggregator.java ## @@ -0,0 +1,128 @@ +/* + * Lice

[GitHub] [druid] abhishekagarwal87 commented on a change in pull request #10338: Vectorized ANY aggregators

2020-09-04 Thread GitBox
abhishekagarwal87 commented on a change in pull request #10338: URL: https://github.com/apache/druid/pull/10338#discussion_r483475430 ## File path: processing/src/main/java/org/apache/druid/query/aggregation/any/NumericAnyVectorAggregator.java ## @@ -0,0 +1,128 @@ +/* + * Lice

[GitHub] [druid] dylwylie opened a new pull request #10356: Avoid large limits causing int overflow in buffer size checks

2020-09-04 Thread GitBox
dylwylie opened a new pull request #10356: URL: https://github.com/apache/druid/pull/10356 ### Description The size check in LimitedBufferHashGrouper can overflow and hence pass inadvertently. This results in the limited grouper being used when the buffer is too small and so the

[GitHub] [druid] abhishekagarwal87 commented on a change in pull request #10338: Vectorized ANY aggregators

2020-09-04 Thread GitBox
abhishekagarwal87 commented on a change in pull request #10338: URL: https://github.com/apache/druid/pull/10338#discussion_r483476704 ## File path: processing/src/main/java/org/apache/druid/query/aggregation/any/NumericAnyVectorAggregator.java ## @@ -0,0 +1,128 @@ +/* + * Lice

[GitHub] [druid] abhishekagarwal87 commented on a change in pull request #10338: Vectorized ANY aggregators

2020-09-04 Thread GitBox
abhishekagarwal87 commented on a change in pull request #10338: URL: https://github.com/apache/druid/pull/10338#discussion_r483476704 ## File path: processing/src/main/java/org/apache/druid/query/aggregation/any/NumericAnyVectorAggregator.java ## @@ -0,0 +1,128 @@ +/* + * Lice

[GitHub] [druid] abhishekagarwal87 commented on a change in pull request #10338: Vectorized ANY aggregators

2020-09-04 Thread GitBox
abhishekagarwal87 commented on a change in pull request #10338: URL: https://github.com/apache/druid/pull/10338#discussion_r483477223 ## File path: processing/src/main/java/org/apache/druid/query/aggregation/any/NumericAnyVectorAggregator.java ## @@ -0,0 +1,128 @@ +/* + * Lice

[GitHub] [druid] abhishekagarwal87 commented on a change in pull request #10338: Vectorized ANY aggregators

2020-09-04 Thread GitBox
abhishekagarwal87 commented on a change in pull request #10338: URL: https://github.com/apache/druid/pull/10338#discussion_r483478720 ## File path: processing/src/main/java/org/apache/druid/query/aggregation/any/StringAnyAggregatorFactory.java ## @@ -77,6 +81,33 @@ public Buff

[GitHub] [druid] abhishekagarwal87 commented on a change in pull request #10338: Vectorized ANY aggregators

2020-09-04 Thread GitBox
abhishekagarwal87 commented on a change in pull request #10338: URL: https://github.com/apache/druid/pull/10338#discussion_r483467418 ## File path: processing/src/main/java/org/apache/druid/query/aggregation/any/NumericAnyVectorAggregator.java ## @@ -0,0 +1,128 @@ +/* + * Lice

[GitHub] [druid] lgtm-com[bot] commented on pull request #10356: Avoid large limits causing int overflow in buffer size checks

2020-09-04 Thread GitBox
lgtm-com[bot] commented on pull request #10356: URL: https://github.com/apache/druid/pull/10356#issuecomment-687047936 This pull request **introduces 1 alert** when merging 06cabfd7da436cfcbd69ddb788877c0f16a73502 into d7fcff3aba47dbd840c1cdca24f4802168ce1d9c - [view on LGTM.com](https://

[GitHub] [druid] FrankChen021 commented on issue #10348: druid.segmentCache.locationSelectorStrategy config raise Exception

2020-09-04 Thread GitBox
FrankChen021 commented on issue #10348: URL: https://github.com/apache/druid/issues/10348#issuecomment-687059953 According to the code, the configuration item should be `druid.segmentCache.locationSelectorStrategy.type` instead of `druid.segmentCache.locationSelectorStrategy`. But

[GitHub] [druid] a2l007 commented on pull request #10341: Fix result-level caching

2020-09-04 Thread GitBox
a2l007 commented on pull request #10341: URL: https://github.com/apache/druid/pull/10341#issuecomment-687131555 > I believe we need integration tests for this fix, but I'm not sure how to add it without introducing a metricsMonitor for integration tests. Curious, but are we trying to avo

[GitHub] [druid] a2l007 edited a comment on pull request #10341: Fix result-level caching

2020-09-04 Thread GitBox
a2l007 edited a comment on pull request #10341: URL: https://github.com/apache/druid/pull/10341#issuecomment-687131555 > I believe we need integration tests for this fix, but I'm not sure how to add it without introducing a metricsMonitor for integration tests. Curious, but are we tr

[GitHub] [druid] suneet-s commented on pull request #10336: More structured way to handle parse exceptions

2020-09-04 Thread GitBox
suneet-s commented on pull request #10336: URL: https://github.com/apache/druid/pull/10336#issuecomment-687199069 > ``` > 279 L rowIngestionMeters.incrementProcessed(); > 280 F | L | B(0/2)} else if (addResult.hasParseException()) { > 281 L p

[GitHub] [druid] suneet-s commented on a change in pull request #10338: Vectorized ANY aggregators

2020-09-04 Thread GitBox
suneet-s commented on a change in pull request #10338: URL: https://github.com/apache/druid/pull/10338#discussion_r483674764 ## File path: processing/src/main/java/org/apache/druid/query/aggregation/any/NumericAnyVectorAggregator.java ## @@ -0,0 +1,128 @@ +/* + * Licensed to t

[GitHub] [druid] suneet-s commented on a change in pull request #10338: Vectorized ANY aggregators

2020-09-04 Thread GitBox
suneet-s commented on a change in pull request #10338: URL: https://github.com/apache/druid/pull/10338#discussion_r483681502 ## File path: processing/src/main/java/org/apache/druid/query/aggregation/any/NumericAnyVectorAggregator.java ## @@ -0,0 +1,128 @@ +/* + * Licensed to t

[GitHub] [druid] abhishekagarwal87 commented on a change in pull request #10338: Vectorized ANY aggregators

2020-09-04 Thread GitBox
abhishekagarwal87 commented on a change in pull request #10338: URL: https://github.com/apache/druid/pull/10338#discussion_r483719357 ## File path: processing/src/main/java/org/apache/druid/query/aggregation/any/StringAnyAggregatorFactory.java ## @@ -77,6 +81,33 @@ public Buff

[GitHub] [druid] kroeders closed pull request #10129: allow retention rules to be imported from other data sources

2020-09-04 Thread GitBox
kroeders closed pull request #10129: URL: https://github.com/apache/druid/pull/10129 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [druid] kroeders commented on pull request #10129: allow retention rules to be imported from other data sources

2020-09-04 Thread GitBox
kroeders commented on pull request #10129: URL: https://github.com/apache/druid/pull/10129#issuecomment-687258911 pending design issues and conflicts need to be resolved, so I created a related issue to discuss alternative approaches and can reopen this after fixing conflicts if this solut

[GitHub] [druid] jihoonson commented on pull request #10341: Fix result-level caching

2020-09-04 Thread GitBox
jihoonson commented on pull request #10341: URL: https://github.com/apache/druid/pull/10341#issuecomment-687273952 @a2l007 It's the former. Also, I'm not sure if we can write deterministic integration tests using monitors. Maybe possible by polling it.

[GitHub] [druid] AshishKapoor opened a new pull request #10357: [web-console] Update README.md

2020-09-04 Thread GitBox
AshishKapoor opened a new pull request #10357: URL: https://github.com/apache/druid/pull/10357 Added a step to compile scss files before npm start step. ### Description Compile the scss files before npm start step. This PR

[GitHub] [druid] jihoonson commented on pull request #10336: More structured way to handle parse exceptions

2020-09-04 Thread GitBox
jihoonson commented on pull request #10336: URL: https://github.com/apache/druid/pull/10336#issuecomment-687276293 Ah I didn't check the code coverage. I manually checked them instead. Well, not only in `IndexTaskTest`, but there are also many tests in `KafkaIndexTaskTest` and `KinesisInde

[GitHub] [druid] xsqian commented on issue #9780: Support directly reading and writing Druid data from Spark

2020-09-04 Thread GitBox
xsqian commented on issue #9780: URL: https://github.com/apache/druid/issues/9780#issuecomment-687289068 > @mangrrua and @JulianJaffePinterest I am very thankful to your guys for the work of spark connector to druid. Can I package this jar and make it working with current druid 0.18.1 vers

[GitHub] [druid] ccaominh opened a new issue #10358: Update cassandra-storage extension to use the Datastax java driver instead of Astyanax

2020-09-04 Thread GitBox
ccaominh opened a new issue #10358: URL: https://github.com/apache/druid/issues/10358 The cassandra-storage extension uses Astyanax, which has been retired and is no longer under active development. The latest version of Astyanax uses an old version of libthrift (0.6.1), which has a few hi

[GitHub] [druid] ccaominh commented on a change in pull request #10353: Ignore CVEs from htrace and ambari transitive deps

2020-09-04 Thread GitBox
ccaominh commented on a change in pull request #10353: URL: https://github.com/apache/druid/pull/10353#discussion_r483802950 ## File path: owasp-dependency-check-suppressions.xml ## @@ -156,6 +156,28 @@ CVE-2018-1320 CVE-2019-0205 + + Review comment:

[GitHub] [druid] 2bethere commented on a change in pull request #10339: Security doc revamp

2020-09-04 Thread GitBox
2bethere commented on a change in pull request #10339: URL: https://github.com/apache/druid/pull/10339#discussion_r483817698 ## File path: docs/operations/security-overview.md ## @@ -0,0 +1,228 @@ +--- +id: security-overview +title: "Security overview" +--- + + +## Overview + +

[GitHub] [druid] 2bethere commented on a change in pull request #10339: Security doc revamp

2020-09-04 Thread GitBox
2bethere commented on a change in pull request #10339: URL: https://github.com/apache/druid/pull/10339#discussion_r483818138 ## File path: docs/operations/security-overview.md ## @@ -0,0 +1,228 @@ +--- +id: security-overview +title: "Security overview" +--- + + +## Overview + +

[GitHub] [druid] 2bethere commented on a change in pull request #10339: Security doc revamp

2020-09-04 Thread GitBox
2bethere commented on a change in pull request #10339: URL: https://github.com/apache/druid/pull/10339#discussion_r483819133 ## File path: docs/operations/security-overview.md ## @@ -0,0 +1,228 @@ +--- +id: security-overview +title: "Security overview" +--- + + +## Overview + +

[GitHub] [druid] 2bethere commented on a change in pull request #10339: Security doc revamp

2020-09-04 Thread GitBox
2bethere commented on a change in pull request #10339: URL: https://github.com/apache/druid/pull/10339#discussion_r483819133 ## File path: docs/operations/security-overview.md ## @@ -0,0 +1,228 @@ +--- +id: security-overview +title: "Security overview" +--- + + +## Overview + +

[GitHub] [druid] 2bethere commented on a change in pull request #10339: Security doc revamp

2020-09-04 Thread GitBox
2bethere commented on a change in pull request #10339: URL: https://github.com/apache/druid/pull/10339#discussion_r483819719 ## File path: docs/operations/security-overview.md ## @@ -0,0 +1,228 @@ +--- +id: security-overview +title: "Security overview" +--- + + +## Overview + +

[GitHub] [druid] gianm merged pull request #10325: fix NPE in StringGroupByColumnSelectorStrategy#bufferComparator

2020-09-04 Thread GitBox
gianm merged pull request #10325: URL: https://github.com/apache/druid/pull/10325 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[druid] branch master updated: fix NPE in StringGroupByColumnSelectorStrategy#bufferComparator (#10325)

2020-09-04 Thread gian
This is an automated email from the ASF dual-hosted git repository. gian pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid.git The following commit(s) were added to refs/heads/master by this push: new 91a1538 fix NPE in StringGroupByColumnSelectorStr

[druid] branch master updated: Ignore CVEs from htrace and ambari transitive deps (#10353)

2020-09-04 Thread ccaominh
This is an automated email from the ASF dual-hosted git repository. ccaominh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid.git The following commit(s) were added to refs/heads/master by this push: new 176b715 Ignore CVEs from htrace and ambari tra

[GitHub] [druid] ccaominh merged pull request #10353: Ignore CVEs from htrace and ambari transitive deps

2020-09-04 Thread GitBox
ccaominh merged pull request #10353: URL: https://github.com/apache/druid/pull/10353 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [druid] gianm commented on pull request #10355: consolidate ValueType and ExprType

2020-09-04 Thread GitBox
gianm commented on pull request #10355: URL: https://github.com/apache/druid/pull/10355#issuecomment-687443737 Because the expression system doesn't support all of the normal Druid ValueTypes, it might actually be better to keep them separate, since it makes things more type-safe. (We know

[GitHub] [druid] jihoonson opened a new pull request #10359: Add shuffle metrics for parallel indexing

2020-09-04 Thread GitBox
jihoonson opened a new pull request #10359: URL: https://github.com/apache/druid/pull/10359 ### Description Part of https://github.com/apache/druid/issues/10352. This PR adds these metrics for middleManagers. These metrics have the `supervisorTaskId` as their dimension. - `sh

[GitHub] [druid] meicool commented on issue #10348: druid.segmentCache.locationSelectorStrategy config raise Exception

2020-09-04 Thread GitBox
meicool commented on issue #10348: URL: https://github.com/apache/druid/issues/10348#issuecomment-687542284 Thank you for your help,I replaced the configuration item with druid.segmentCache.locationSelectorStrategy.type,but the same error persisted

[GitHub] [druid] vogievetsky opened a new pull request #10360: Web console: improve query manager (convert to React hook)

2020-09-04 Thread GitBox
vogievetsky opened a new pull request #10360: URL: https://github.com/apache/druid/pull/10360 ### Description This PR at the heart creates a useQueryManager hook that lets allows us to convert a bunch of web console components to be functional instead of Component class based. This

[GitHub] [druid] lgtm-com[bot] commented on pull request #10360: Web console: improve query manager (convert to React hook)

2020-09-04 Thread GitBox
lgtm-com[bot] commented on pull request #10360: URL: https://github.com/apache/druid/pull/10360#issuecomment-687543535 This pull request **introduces 3 alerts** when merging cb52080840fe5781403c0d9443bcbdc6bed4cee4 into 176b7156249fdcd2a148c9d825f1f828df44709a - [view on LGTM.com](https:/