[druid] branch master updated: Web console: add a nice UI for overlord dynamic configs and improve the docs (#13993)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid.git The following commit(s) were added to refs/heads/master by this push: new 981662e9f4 Web console: add a nice UI for overlord dynamic configs and improve the docs (#13993) 981662e9f4 is described below commit 981662e9f407093162d5649c9e8a223c131d8c78 Author: Vadim Ogievetsky AuthorDate: Fri Mar 31 10:12:25 2023 -0700 Web console: add a nice UI for overlord dynamic configs and improve the docs (#13993) * in progress * better form * doc updates * doc changes * add inline docs * fix tests * Update docs/configuration/index.md Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> * Update docs/configuration/index.md Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> * Update docs/configuration/index.md Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> * Update docs/configuration/index.md Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> * Update docs/configuration/index.md Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> * Update docs/configuration/index.md Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> * Update docs/configuration/index.md Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> * Update docs/configuration/index.md Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> * Update docs/configuration/index.md Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> * Update docs/configuration/index.md Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> * Update docs/configuration/index.md Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> * Update docs/configuration/index.md Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> * Update docs/configuration/index.md Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> * Update docs/configuration/index.md Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> * Update docs/configuration/index.md Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> * Update docs/configuration/index.md Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> * Update docs/configuration/index.md Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> * Update docs/configuration/index.md Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> * Update docs/configuration/index.md Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> * Update docs/configuration/index.md Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> * final fixes * fix case * Update docs/configuration/index.md Co-authored-by: Kashif Faraz * Update docs/configuration/index.md Co-authored-by: Kashif Faraz * Update docs/configuration/index.md Co-authored-by: Kashif Faraz * Update docs/configuration/index.md Co-authored-by: Kashif Faraz * Update docs/configuration/index.md Co-authored-by: Kashif Faraz * Update docs/configuration/index.md Co-authored-by: Kashif Faraz * Update docs/configuration/index.md Co-authored-by: Kashif Faraz * Update docs/configuration/index.md Co-authored-by: Kashif Faraz * Update docs/configuration/index.md Co-authored-by: Kashif Faraz * Update docs/configuration/index.md Co-authored-by: Kashif Faraz * fix overflow * fix spelling - Co-authored-by: 317brian <53799971+317br...@users.noreply.github.com> Co-authored-by: Kashif Faraz --- docs/configuration/index.md| 152 ++-- .../form-group-with-info/form-group-with-info.scss | 8 + .../overload-dynamic-config-dialog.spec.tsx.snap | 1 + .../overlord-dynamic-config-dialog.scss| 10 -- .../overlord-dynamic-config-dialog.tsx | 33 +++- .../overlord-dynamic-config.tsx| 196 - 6 files changed, 320 insertions(+), 80 deletions(-) diff --git a/docs/configuration/index.md b/docs/configuration/index.md index 5fbf4b0c07..4f86695b95 100644 --- a/do
[druid-website-src] branch master updated: Add LI to community list
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new 83aab945 Add LI to community list new 12414cab Merge pull request #351 from petermarshallio/202212-addLinkedIn 83aab945 is described below commit 83aab945a5196af8808a0e92016f8f1f2bd16350 Author: Peter Marshall AuthorDate: Thu Dec 1 11:44:50 2022 + Add LI to community list --- community/index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/community/index.md b/community/index.md index 04b7a3ea..cdfcbe9b 100644 --- a/community/index.md +++ b/community/index.md @@ -24,6 +24,7 @@ Also check out: * **User mailing list:** [druid-u...@googlegroups.com](https://groups.google.com/forum/#!forum/druid-user) for general discussion, questions, and announcements. +* **LinkedIn** Connect with other Apache Druid professionals in the [LinkedIn group](https://www.linkedin.com/groups/8791983/) * **Meetups:** Check out [Apache Druid on meetup.com](https://www.meetup.com/topics/apache-druid/) for links to regular meetups in cities all over the world. * **StackOverflow:** While the user mailing list is the primary resource for asking questions, if you prefer - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated (31dc9004bd -> fd6cfcb8fb)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git from 31dc9004bd Auto-reload TLS certs for druid endpoints (#12933) add fd6cfcb8fb Web console: fix pagination, add error delimiters (#12969) No new revisions were added by this update. Summary of changes: web-console/lib/keywords.js| 1 + .../react-table-pagination.tsx | 2 +- .../execution-error-pane.spec.tsx.snap | 16 +-- .../execution-error-pane/execution-error-pane.scss | 2 ++ .../execution-error-pane/execution-error-pane.tsx | 31 +++--- .../recent-query-task-panel.tsx| 2 +- 6 files changed, 28 insertions(+), 26 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated: Docs: fix doc footer (#12943)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new c1a75fca3c Docs: fix doc footer (#12943)
c1a75fca3c is described below
commit c1a75fca3cb2bd76d4db5510daa02e4df318fd6c
Author: Vadim Ogievetsky
AuthorDate: Wed Aug 24 06:47:58 2022 -0700
Docs: fix doc footer (#12943)
* fix doc footer
* move the logic to the fix-path script
---
website/script/fix-path.js | 7 +++
1 file changed, 7 insertions(+)
diff --git a/website/script/fix-path.js b/website/script/fix-path.js
index f6b8c26222..b2d43c35c7 100644
--- a/website/script/fix-path.js
+++ b/website/script/fix-path.js
@@ -27,6 +27,13 @@ var urlVersion = process.argv[2];
var druidVersion = process.argv[3];
try {
+ // Change any explict "/docs/latest" links to "/docs/" to be picked up by
the next replace
+ replace.sync({
+files: './build/ApacheDruid/docs/**/*.html',
+from: /"\/docs\/latest"/g,
+to: `"/docs/"`,
+ });
+
// Fix doc paths
replace.sync({
files: './build/ApacheDruid/docs/**/*.html',
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated: Do no create filters on right side table columns while join to filter conversion (#12899)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new b26ab678b9 Do no create filters on right side table columns while join
to filter conversion (#12899)
b26ab678b9 is described below
commit b26ab678b9e3b9724a667914234fe30e9a71038e
Author: Rohan Garg <7731512+rohang...@users.noreply.github.com>
AuthorDate: Sun Aug 14 21:05:23 2022 +0530
Do no create filters on right side table columns while join to filter
conversion (#12899)
---
.../druid/segment/join/JoinableFactoryWrapper.java | 14 +++--
.../segment/join/JoinableFactoryWrapperTest.java | 36 ++
2 files changed, 48 insertions(+), 2 deletions(-)
diff --git
a/processing/src/main/java/org/apache/druid/segment/join/JoinableFactoryWrapper.java
b/processing/src/main/java/org/apache/druid/segment/join/JoinableFactoryWrapper.java
index 0971e1abae..5530d7243d 100644
---
a/processing/src/main/java/org/apache/druid/segment/join/JoinableFactoryWrapper.java
+++
b/processing/src/main/java/org/apache/druid/segment/join/JoinableFactoryWrapper.java
@@ -58,6 +58,7 @@ import java.util.Optional;
import java.util.Set;
import java.util.concurrent.atomic.AtomicLong;
import java.util.function.Function;
+import java.util.stream.Collectors;
/**
* A wrapper class over {@link JoinableFactory} for working with {@link
Joinable} related classes.
@@ -233,6 +234,7 @@ public class JoinableFactoryWrapper
}
}
+Set rightPrefixes =
clauses.stream().map(JoinableClause::getPrefix).collect(Collectors.toSet());
// Walk through the list of clauses, picking off any from the start of the
list that can be converted to filters.
boolean atStart = true;
for (JoinableClause clause : clauses) {
@@ -246,7 +248,8 @@ public class JoinableFactoryWrapper
convertJoinToFilter(
clause,
Sets.union(requiredColumns,
columnsRequiredByJoinClauses.elementSet()),
-maxNumFilterValues
+maxNumFilterValues,
+rightPrefixes
);
// add the converted filter to the filter list
@@ -287,7 +290,8 @@ public class JoinableFactoryWrapper
static JoinClauseToFilterConversion convertJoinToFilter(
final JoinableClause clause,
final Set requiredColumns,
- final int maxNumFilterValues
+ final int maxNumFilterValues,
+ final Set rightPrefixes
)
{
if (clause.getJoinType() == JoinType.INNER
@@ -305,6 +309,12 @@ public class JoinableFactoryWrapper
return new JoinClauseToFilterConversion(null, false);
}
+// don't add a filter on any right side table columns. only filter on
left base table is supported as of now.
+if (rightPrefixes.stream().anyMatch(leftColumn::startsWith)) {
+ joinClauseFullyConverted = false;
+ continue;
+}
+
Joinable.ColumnValuesWithUniqueFlag columnValuesWithUniqueFlag =
clause.getJoinable().getNonNullColumnValues(condition.getRightColumn(),
numValues);
// For an empty values set, isAllUnique flag will be true only if the
column had no non-null values.
diff --git
a/processing/src/test/java/org/apache/druid/segment/join/JoinableFactoryWrapperTest.java
b/processing/src/test/java/org/apache/druid/segment/join/JoinableFactoryWrapperTest.java
index 49d159d585..8f5b699ee7 100644
---
a/processing/src/test/java/org/apache/druid/segment/join/JoinableFactoryWrapperTest.java
+++
b/processing/src/test/java/org/apache/druid/segment/join/JoinableFactoryWrapperTest.java
@@ -833,6 +833,42 @@ public class JoinableFactoryWrapperTest extends
NullHandlingTest
);
}
+ @Test
+ public void
test_convertJoinsToFilters_dontConvertJoinsDependedOnByLaterJoins()
+ {
+// in this multi-join, a join matching two right sides is kept first to
ensure :
+// 1. there is no filter on the right side table column j.k
+// 2. the right side matching join gets considered for join conversion
always (since it is the first join clause)
+final ImmutableList clauses = ImmutableList.of(
+new JoinableClause(
+"_j.",
+LookupJoinable.wrap(new MapLookupExtractor(TEST_LOOKUP, false)),
+JoinType.INNER,
+JoinConditionAnalysis.forExpression("\"j.k\" == \"_j.k\"", "_j.",
ExprMacroTable.nil())
+),
+new JoinableClause(
+"j.",
+LookupJoinable.wrap(new MapLookupExtractor(TEST_LOOKUP, false)),
+JoinType.INNER,
+JoinConditionAnalysis.forExpression("x == \"j.k\"", "j.",
ExprMacroTable.nil())
+)
+);
+
+final Pair, List> conversion =
JoinableFactoryWrapper.convertJoinsToFilters(
+cl
[druid-website-src] branch master updated: Update index.md
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new 9c46edb Update index.md new 0987218 Merge pull request #282 from petermarshallio/202201-channels-0druidForum 9c46edb is described below commit 9c46edbd3df9b63c523a62c2e4f4a9e51cf468ea Author: Peter Marshall AuthorDate: Sat Jan 8 14:08:39 2022 + Update index.md Added Druid Forum (by Imply) to the list. --- community/index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/community/index.md b/community/index.md index 2e94a6a..29ffd82 100644 --- a/community/index.md +++ b/community/index.md @@ -13,6 +13,7 @@ Druid development, or just stay up to date: * **User mailing list:** [druid-u...@googlegroups.com](https://groups.google.com/forum/#!forum/druid-user) for general discussion, questions, and announcements. This is the perfect place to ask for help if you need it! +* **Druid Forum:** Ask and answer questions, read and submit tips and tricks, and find links to Apache Druid articles and videos in the [Druid Forum](https://www.druidforum.org/) (by Imply). * **Development mailing list:** [d...@druid.apache.org](https://lists.apache.org/list.html?d...@druid.apache.org) for discussion about project development. * **GitHub:** Star us at [apache/druid](https://github.com/apache/druid) and use this to follow Druid development, raise - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: Add an public article link under Shopee section
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new e437157 Add an public article link under Shopee section new cdd2424 Merge pull request #283 from FrankChen021/shopee_article_link e437157 is described below commit e43715717fb274d87a83aa85bf96dd53d8405caa Author: Frank Chen AuthorDate: Fri Jan 14 10:18:26 2022 +0800 Add an public article link under Shopee section Signed-off-by: Frank Chen --- druid-powered.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/druid-powered.md b/druid-powered.md index db0e7d4..eb10a66 100644 --- a/druid-powered.md +++ b/druid-powered.md @@ -660,6 +660,8 @@ Scalac migrated to Apache Druid and Apache Kafka (from Apache Cassandra and Apac At [Shopee](http://shopee.com/) Data Engineer Group, we use Druid for real-time analytics of users'(including seller and buyer) interaction with its popular e-commerce site. +* [Apache Druid in Shopee](https://mp.weixin.qq.com/s/WFiPDPvxMUi_QVfBDqgVcQ)(in Chinese). Published on 2022-01-13 by [yuanlihan](https://github.com/yuanlihan), a contributor of Apache Druid. + ## Shopify Shopify uses Druid, along with Apache Flink and Airflow, to create an interactive analytics platform for realtime sales data with enormous volume and variety. - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated (0abdc3c -> fbabaac)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git. from 0abdc3c Merge pull request #277 from jelenazanko/patch-36 new 3c33fce Add how to get slack invitation new 435ba11 Update index.md new fbabaac Merge pull request #278 from abhishekagarwal87/druid-slack-invite The 578 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: community/index.md | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] 01/01: new title
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch nnn in repository https://gitbox.apache.org/repos/asf/druid-website-src.git commit 2d96031e5684f18af3c4a486a4a6d003da012373 Author: fjy AuthorDate: Tue Nov 16 15:26:39 2021 -0800 new title --- index.html | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/index.html b/index.html index bc8c63e..ac9be9f09 100644 --- a/index.html +++ b/index.html @@ -1,6 +1,6 @@ --- published: true -title: Interactive Analytics at Scale +title: Database for modern analytics applications layout: html_page id: home canonical: 'https://druid.apache.org/' @@ -12,7 +12,7 @@ canonical: 'https://druid.apache.org/' -Apache Druid is a high performance real-time analytics database. +Apache Druid is a real-time database to power modern analytics applications. Download https://github.com/apache/druid/;> GitHub @@ -30,7 +30,7 @@ canonical: 'https://druid.apache.org/' - Build fast, modern data applications + Build fast, modern data analytics applications Druid is designed for workflows where fast ad-hoc analytics, instant data visibility, or supporting high concurrency is important. As such, Druid is often used to power UIs where an interactive, consistent user experience is desired. - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch nnn created (now 2d96031)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch nnn in repository https://gitbox.apache.org/repos/asf/druid-website-src.git. at 2d96031 new title This branch includes the following new commits: new 2d96031 new title The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated (659bf1a -> c0407c8)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git. from 659bf1a Merge pull request #263 from godzig/master new 041a6de Update featured.yml new f3176cb Update featured.yml new c0407c8 Merge pull request #261 from petermarshallio/202110-fArticles The 549 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: _data/featured.yml | 18 ++ 1 file changed, 18 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: Fix broken links, adding Triton Digital
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new 306683b Fix broken links, adding Triton Digital new 659bf1a Merge pull request #263 from godzig/master 306683b is described below commit 306683bfb73214b881e714bc7ec27d372932cddc Author: Mike Godwin <1851282+god...@users.noreply.github.com> AuthorDate: Thu Oct 14 15:20:02 2021 -0400 Fix broken links, adding Triton Digital Fixed broken links for: Adikteev, Italiaonline.it, Paypal, Zuoyebang. Removed links for Sweet Couch which is no longer in business. Added link to Rill Data. Added Triton Digital entry. --- druid-powered.md | 18 -- 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/druid-powered.md b/druid-powered.md index 3f89d19..7b14eed 100644 --- a/druid-powered.md +++ b/druid-powered.md @@ -44,7 +44,7 @@ Druid is widely used in different business units of Alibaba Group. ## Allegro -[Allegro](https://allegro.tech/about-us/) is the most popular Polish shopping destination with about 14 million users monthly and over 1.5 million items sold on the platform daily. +[Allegro](https://allegro.tech/) is the most popular Polish shopping destination with about 14 million users monthly and over 1.5 million items sold on the platform daily. We have been using Druid since 2016 as business intelligence platform to power our clickstream analytics, marketing automation, anomaly detection, technical metrics and more. Our cluster (432 CPUs, 1300GB of RAM on historicals) processes billions of realtime events loaded from [Kafka](https://kafka.apache.org/) and hundreds batch indexing jobs on daily basis. @@ -313,7 +313,7 @@ At ININ we're using Druid within a Lambda architecture to drive cloud based call ## Italiaonline -Italiaonline exploits Druid for Internet trends and analytics management inside its new [Data-Driven Contents Management System](http://www.italiaonline.it/en/supereva-a-new-online-publishing-experience/). +Italiaonline exploits Druid for Internet trends and analytics management inside its new [Data-Driven Contents Management System](https://www.italiaonline.it/corporate/en/2016/02/17/supereva-a-new-online-publishing-experience/). Italiaonline is the first Italian internet company, with the two most visited web portals, Libero and Virgilio, and the most used email service of the country @libero.it. Italiaonline features 16.8 million unique users per month, 4.8 billion impressions per month, 10.2 million active email accounts and a 58% active reach. * Source: Audiweb View, powered by Nielsen, TDA 2H 2015; Internal data, December 2015* @@ -522,7 +522,7 @@ is what they have to say: > seen a Open Source Community providing such a very high level of > responsiveness for ANY > issue related to Druid setup and tuning. -* [PayPal merchant ecosystem using Apache Spark, Hive, Druid, and HBase](https://dataworkssummit.com/san-jose-2018/session/paypal-merchant-ecosystem-using-apache-spark-hive-druid-and-hbase/) +* [PayPal merchant ecosystem using Apache Spark, Hive, Druid, and HBase](https://www.youtube.com/watch?v=HJvuU0CQS44) ## PayU @@ -580,7 +580,7 @@ Druid is a critical component in our advertising infrastructure, where it serves Retargetly is a Data Management Platform that enables publishers and advertisers to manage their first party user data, mix it with second and third party data from others providers and activate it into advertising campaigns (direct, programmatic, etc.). Druid enables us to show real time audience insights. It also provides a lot of flexibility on ad-hoc queries with low latency. We provide default graphs and metrics to our clients but they also have the possibility to make their own int [...] ## Rill Data -Rill Data uses Druid to power its truly elastic, fully managed cloud service. +[Rill Data](https://rilldata.com) uses Druid to power its truly elastic, fully managed cloud service. Rill uses Druid to deliver operational intelligence to business stakeholders with zero DevOps overhead. Rill's team operates the first and longest continuously running Druid service. @@ -695,7 +695,7 @@ Sugo is a company that focus on realtime multi-dimension analytics and mining on ## Sweet Couch -[Sweet Couch](https://www.sweetcouch.com) is a place to discover unique products which are buyable online. Druid powers [Sweet Couch harvest](https://www.sweetcouch.com/harvest/womens) which is an open analytics platform for tracking performance of online shops based out in India. All end user events are tracked and analysed using Druid for business insights. +Sweet Couch was a place to discover unique products which are buyable online. Druid powered Sweet Couch harvest which was an open analytics platfor
[druid-website-src] branch nn created (now 625fb6f)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch nn in repository https://gitbox.apache.org/repos/asf/druid-website-src.git. at 625fb6f small changes to messaging This branch includes the following new commits: new 625fb6f small changes to messaging The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] 01/01: small changes to messaging
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch nn in repository https://gitbox.apache.org/repos/asf/druid-website-src.git commit 625fb6fdfd2670337bef147fde7bb503d78fa12d Author: fjy AuthorDate: Tue Sep 14 13:28:40 2021 -0700 small changes to messaging --- index.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/index.html b/index.html index d819db8..bc8c63e 100644 --- a/index.html +++ b/index.html @@ -30,7 +30,7 @@ canonical: 'https://druid.apache.org/' - A fast, modern analytics database + Build fast, modern data applications Druid is designed for workflows where fast ad-hoc analytics, instant data visibility, or supporting high concurrency is important. As such, Druid is often used to power UIs where an interactive, consistent user experience is desired. - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: Update druid-powered.md to add societe generale
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new 724169e Update druid-powered.md to add societe generale new e608489 Merge pull request #215 from jelenazanko/patch-3 724169e is described below commit 724169e81d5da75a7fe9fe3ec254da3f28f93b9e Author: Jelena Zanko <59612355+jelenaza...@users.noreply.github.com> AuthorDate: Mon Mar 22 09:28:28 2021 -0500 Update druid-powered.md to add societe generale --- druid-powered.md | 6 ++ 1 file changed, 6 insertions(+) diff --git a/druid-powered.md b/druid-powered.md index 2184cd6..f267220 100644 --- a/druid-powered.md +++ b/druid-powered.md @@ -619,6 +619,12 @@ Smyte provides an API and UI for detecting and blocking bad actors on the intern * [Data Analytics and Processing at Snap](https://www.slideshare.net/CharlesAllen9/data-analytics-and-processing-at-snap-druid-meetup-la-september-2018) +## Societe Generale + +Societe Generale, one of Europe's leading financial services groups and a major player in the economy for over 150 years, supports 29 million clients every day with 138,000 staff in 62 countries. + +Within the Societe Generale IT department, Apache Druid is used as Time Series Database in order to store performance metrics generated in real-time by thousands of servers, databases, middlewares. These data are stored in multiple Druid clusters in multiple regions (+840 vCPUs, +7000GB of RAM, +300 billions of events) and are used for many purposes, such as dashboarding and predictive maintenance use cases. + ## Splunk We went through the journey of deploying Apache Druid clusters on Kubernetes and created a [druid-operator](https://github.com/druid-io/druid-operator). We use this operator to deploy Druid clusters at Splunk. - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: add autonomic, mobiauto, apollo graphql
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new ed2a29f add autonomic, mobiauto, apollo graphql new 14dc5b3 Merge pull request #201 from druid-matt/patch-44 ed2a29f is described below commit ed2a29fdb3d748afa1115a5975145f943503c8e1 Author: Matt Sarrel <59710709+druid-m...@users.noreply.github.com> AuthorDate: Wed Jan 20 12:06:04 2021 -0800 add autonomic, mobiauto, apollo graphql --- druid-powered.md | 12 1 file changed, 12 insertions(+) diff --git a/druid-powered.md b/druid-powered.md index c6ba5c0..9b8c083 100644 --- a/druid-powered.md +++ b/druid-powered.md @@ -36,6 +36,10 @@ Today’s consumers move across screens with ease. But advertisers and media com Amobee provides advertising solutions across tv, digital and social platforms. Amobee has massive incoming data, with customers requiring reporting analytics across unbounded combinations of use cases. Amobee uses Druid to power such intense customer needs via quick response dashboards and offline reports in Amobee’s Platform UI, as well as queries from demanding automated systems. Specifically, we love that Druid allows convergence of many data streams via its robust ingestion, parallel [...] +## Apollo GraphQL + +Apollo uses Druid to power our "Graph Manager" SaaS service, which helps application developers manage, validate and secure their organization’s data graph. In addition to managing the evolution of their GraphQL schema, users can draw insights from the operations metadata and execution trace data that are sent to our service, indexed, and stored in Druid. + ## Appsflyer Druid is the major player in the real-time analytics pipeline at Appsflyer, and it serves as customer facing analytics database for dashboard reporting. @@ -53,6 +57,10 @@ At [Athena Health](https://www.athenahealth.com/), we are creating a new perform Atomx is a new media exchange that connects networks, DSPs, SSPs, and other parties. Atomx uses Druid for it's advanced realtime reporting system. Using the Google Cloud modifications Atomx contributed to Druid, it can easily scale Druid with the fast growing platform. +## Autonomic + +Autonomic Transportation Mobility Cloud (TMC) is the first industry open cloud platform for transportation and mobility data. One of our missions is to enable our customers to easily explore a tremendous amount of data and draw valuable insights in a timely fashion. To meet these needs, we deploy Apache Druid in our platform to aggregate and process time series data. Each component in Druid is running as an individual microservices, and can be easily deployed, managed, and scaled indepen [...] + ## Avesta Technologies At [Avesta](https://avestatechnologies.com/), we use Druid as a central component in our cloud data platform to provide real-time analytics solutions to our clients. We are using Druid for Customer Data Analytics and extending it for Industrial IoT and Market Automation use cases. Druid has not only proven itself to be resilient and performant but has also helped our clients save enormous amounts in cloud and licensing costs. @@ -341,6 +349,10 @@ Mindhouse streams data from Apache Kafka into Druid and runs dashboards as well * [Mindhouse Achieves Monitoring Nirvana with Apache Druid](https://imply.io/post/mindhouse-achieves-monitoring-nirvana-with-apache-druid) +## Mobiauto + +Druid has helped Mobiauto ingest and analyze millions of real time events generated on our online car sales website. We use this data to generate real time analytics used by the commercial team, and create APIs for our backend team that can be used to insert real time data to the website that helps our clients make informed decisions. + ## Monetate Druid is a critical component in Monetate's personalization platform, where it - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: added Salesforce
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new 1c38c21 added Salesforce new ae43d8c Merge pull request #182 from druid-matt/patch-28 1c38c21 is described below commit 1c38c218763659b4774cdfeda6f81453b62c385e Author: Matt Sarrel <59710709+druid-m...@users.noreply.github.com> AuthorDate: Thu Oct 22 10:14:12 2020 -0700 added Salesforce --- druid-powered.md | 6 ++ 1 file changed, 6 insertions(+) diff --git a/druid-powered.md b/druid-powered.md index 6dc034d..15dd1e7 100644 --- a/druid-powered.md +++ b/druid-powered.md @@ -492,6 +492,12 @@ We introduced Druid to our technology stack while developing our new supply chai We are using Druid as our single source of truth for both realtime statistics and data analysis. Our clients can see and filter detailed metrics in our [self-service DSP](https://www.sage-archer.com). The DSP is used for mobile and digital-out-of-home advertising campaigns. Druid is powering both the front-end and the optimization algorithms within the system. +## Salesforce + +Salesforce's Edge Intelligence team uses Apache Druid as a real-time analytical database to store application performance metrics extracted from log lines. Our customers within Salesforce, including engineers, product owners, customer service representatives, etc., use our service to define the way they want to ingest and query their data and obtain insights such as performance analysis, trend analysis, release comparison, issue triage, and troubleshooting. We chose Druid because it give [...] + +[Delivering High-Quality Insights Interactively Using Apache Druid at Salesforce](https://engineering.salesforce.com/delivering-high-quality-insights-interactively-using-apache-druid-at-salesforce-7a038f0fe3d1) + ## Sift Science Sift Science provides an online trust platform that online businesses use to prevent fraud and abuse. We use Druid as a tool to gain real-time insights about our data and machine learning models. - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: add Avesta Technologies
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new 69ad718 add Avesta Technologies new 5d36138 Merge pull request #181 from druid-matt/patch-27 69ad718 is described below commit 69ad718996a92d68ce6de742dfd24f454e061e7d Author: Matt Sarrel <59710709+druid-m...@users.noreply.github.com> AuthorDate: Tue Oct 20 13:02:25 2020 -0700 add Avesta Technologies --- druid-powered.md | 4 1 file changed, 4 insertions(+) diff --git a/druid-powered.md b/druid-powered.md index 0aea5e3..6dc034d 100644 --- a/druid-powered.md +++ b/druid-powered.md @@ -47,6 +47,10 @@ At [Athena Health](https://www.athenahealth.com/), we are creating a new perform Atomx is a new media exchange that connects networks, DSPs, SSPs, and other parties. Atomx uses Druid for it's advanced realtime reporting system. Using the Google Cloud modifications Atomx contributed to Druid, it can easily scale Druid with the fast growing platform. +## Avesta Technologies + +At [Avesta](https://avestatechnologies.com/), we use Druid as a central component in our cloud data platform to provide real-time analytics solutions to our clients. We are using Druid for Customer Data Analytics and extending it for Industrial IoT and Market Automation use cases. Druid has not only proven itself to be resilient and performant but has also helped our clients save enormous amounts in cloud and licensing costs. + ## Bannerflow [Bannerflow](https://www.bannerflow.com) is the leading display ad production platform. We use Druid to power our customer facing analytics system and for internal reporting, monitoring and ad-hoc data exploration. - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: update favicon
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new 2527ca9 update favicon new 7ccaba8 Merge pull request #176 from implydata/better-favicon 2527ca9 is described below commit 2527ca94c3ef118087e45407711fbc96f80c2a8e Author: Vadim Ogievetsky AuthorDate: Thu Oct 8 12:31:30 2020 -0700 update favicon --- img/favicon.png | Bin 4514 -> 1156 bytes 1 file changed, 0 insertions(+), 0 deletions(-) diff --git a/img/favicon.png b/img/favicon.png index caf8e68..96b9aff 100644 Binary files a/img/favicon.png and b/img/favicon.png differ - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch c updated (dd553a5 -> cde3af8)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch c in repository https://gitbox.apache.org/repos/asf/druid-website-src.git. from dd553a5 fix add cde3af8 fix No new revisions were added by this update. Summary of changes: .gitignore | 2 -- 1 file changed, 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch c updated (97b38b5 -> dd553a5)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch c in repository https://gitbox.apache.org/repos/asf/druid-website-src.git. from 97b38b5 fix add dd553a5 fix No new revisions were added by this update. Summary of changes: index.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch c updated (97b38b5 -> dd553a5)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch c in repository https://gitbox.apache.org/repos/asf/druid-website-src.git. from 97b38b5 fix add dd553a5 fix No new revisions were added by this update. Summary of changes: index.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch c updated (b53688f -> 97b38b5)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch c in repository https://gitbox.apache.org/repos/asf/druid-website-src.git. from b53688f fix add 97b38b5 fix No new revisions were added by this update. Summary of changes: .gitignore| 1 + package-lock.json | 724 -- 2 files changed, 318 insertions(+), 407 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch c updated (9cb0088 -> b53688f)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch c in repository https://gitbox.apache.org/repos/asf/druid-website-src.git. from 9cb0088 fix add b53688f fix No new revisions were added by this update. Summary of changes: .gitignore | 1 + package.json | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch c updated (56ff30a -> 9cb0088)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch c in repository https://gitbox.apache.org/repos/asf/druid-website-src.git. from 56ff30a more changes add 9cb0088 fix No new revisions were added by this update. Summary of changes: index.html| 2 +- technology.md | 2 +- use-cases.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: Added Deep.Explorer to the UIs section
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new e9dfb74 Added Deep.Explorer to the UIs section new fde3626 Merge pull request #174 from hishamitani/patch-3 e9dfb74 is described below commit e9dfb74b99353d282ac4efd77f47d25280b7e79b Author: Hisham Itani <47254768+hishamit...@users.noreply.github.com> AuthorDate: Tue Oct 6 09:35:15 2020 +0200 Added Deep.Explorer to the UIs section I saw all the rest have a github page but assumed it wasn't a requirement since those github pages in many cases just redirected to the commercial link. If that's not the case please let me know! --- libraries.md | 1 + 1 file changed, 1 insertion(+) diff --git a/libraries.md b/libraries.md index 3f9f206..534c420 100644 --- a/libraries.md +++ b/libraries.md @@ -73,6 +73,7 @@ UIs --- * [airbnb/superset](https://github.com/airbnb/superset) - A web application to slice, dice and visualize data out of Druid. Formerly Caravel and Panoramix +* [Deep.Explorer](https://www.deep.bi/solutions/apache-druid) - A UI built for slice & dice analytics, adhoc queries and powerful, easy data visualizations * [Grafana](https://github.com/societe-generale/druidplugin) - A plugin for [Grafana](http://grafana.org/) * [grafana](https://github.com/Quantiply/grafana-plugins/tree/master/features/druid) - A plugin for [Grafana](http://grafana.org/) * [Pivot](https://github.com/implydata/pivot) - An exploratory analytics UI for Druid - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch c updated (d26d5d5 -> 56ff30a)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch c in repository https://gitbox.apache.org/repos/asf/druid-website-src.git. from d26d5d5 wording changes add 56ff30a more changes No new revisions were added by this update. Summary of changes: index.html| 8 +- package-lock.json | 724 ++ package.json | 2 +- technology.md | 2 +- 4 files changed, 413 insertions(+), 323 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch c created (now d26d5d5)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch c in repository https://gitbox.apache.org/repos/asf/druid-website-src.git. at d26d5d5 wording changes This branch includes the following new commits: new d26d5d5 wording changes The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] 01/01: wording changes
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch c in repository https://gitbox.apache.org/repos/asf/druid-website-src.git commit d26d5d5fa8abe20933d528c6eaef8f802b84b375 Author: fjy AuthorDate: Mon Oct 5 13:58:58 2020 -0700 wording changes --- index.html| 18 +- technology.md | 2 +- use-cases.md | 2 +- 3 files changed, 11 insertions(+), 11 deletions(-) diff --git a/index.html b/index.html index 03f16c9..d3934ba 100644 --- a/index.html +++ b/index.html @@ -12,7 +12,7 @@ canonical: 'https://druid.apache.org/' -Apache Druid is a high performance real-time analytics database. +Apache Druid is a high performance analytics database. Download https://github.com/apache/druid/;> GitHub @@ -30,35 +30,35 @@ canonical: 'https://druid.apache.org/' - A modern cloud-native, stream-native, analytics database + Power data applications and products -Druid is designed for workflows where fast queries and ingest really matter. Druid excels at instant data visibility, ad-hoc queries, operational analytics, and handling high concurrency. Consider Druid as an open source alternative to data warehouses for a variety of use cases. +Druid is designed for workflows where instant data visibility, fast ad-hoc queries, powerful analytics, and/or handling high query concurrency really matter. As such, Druid is often used to power UIs where interactive, consistent responses are desired for end users. Easy integration with your existing data pipelines -Druid can natively stream data from message buses such as Kafka, Amazon Kinesis, and more, and batch load files from data lakes such as HDFS, Amazon S3, and more. +Druid streams data from message buses such as Kafka, Amazon Kinesis, and batch load files from data lakes such as HDFS, Amazon S3. Druid supports most popular file formats for structured and semi-structured data. - Up to 100x faster than traditional solutions + Fast, consistent queries at high concurrency -Druid has been benchmarked to greatly outperform legacy solutions for data ingestion and data querying. Druid's novel architecture combines the best of data warehouses, timeseries databases, and search systems. +Druid has been benchmarked to greatly outperform legacy solutions for data ingestion and data querying. Druid combines novel storage ideas, indexing structures, and both exact and approximate queries so that most results can return in under a second. - Unlock new workflows + Broad applicability -Druid unlocks new types of queries and workflows for clickstream, APM, supply chain, network telemetry, digital marketing, and many other forms of event-driven data. Druid is purpose built for rapid, ad-hoc queries on both real-time and historical data. +Druid unlocks new types of queries and workflows for clickstream, APM, supply chain, network telemetry, digital marketing, risk/fraud, and many other types of data. Druid is purpose built for rapid, ad-hoc queries on both real-time and historical data. - Deploy in AWS/GCP/Azure, hybrid clouds, Kubernetes, and bare metal + Deploy in public, private, and hybrid clouds Druid can be deployed in any *NIX environment on commodity hardware, both in the cloud and on premise. Deploying Druid is easy: scaling up and down is as simple as adding and removing Druid services. diff --git a/technology.md b/technology.md index e516692..30f8d20 100644 --- a/technology.md +++ b/technology.md @@ -6,7 +6,7 @@ canonical: 'https://druid.apache.org/technology' --- Apache Druid is an open source distributed data store. -Druid’s core design combines ideas from [data warehouses](https://en.wikipedia.org/wiki/Data_warehouse), [timeseries databases](https://en.wikipedia.org/wiki/Time_series_database), and [search systems](https://en.wikipedia.org/wiki/Full-text_search) to create a unified system for real-time analytics for a broad range of [use cases](/use-cases). Druid merges key characteristics of each of the 3 systems into its ingestion layer, storage format, querying layer, and core architecture. +Druid’s core design combines ideas from [data warehouses](https://en.wikipedia.org/wiki/Data_warehouse), [timeseries databases](https://en.wikipedia.org/wiki/Time_series_database), and [search systems](https://en.wikipedia.org/wiki/Full-text_search) to create a powerful backend for a broad range of [use cases](/use-cases). Druid merges key characteristics of each of the 3 syst
[druid] branch 0.20.0 updated: fix array types from escaping into wider query engine (#10460) (#10474)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch 0.20.0
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/0.20.0 by this push:
new 239e9f0 fix array types from escaping into wider query engine
(#10460) (#10474)
239e9f0 is described below
commit 239e9f006e021a9c2ba81107a0c65071a0c7a59d
Author: Jonathan Wei
AuthorDate: Sat Oct 3 19:50:21 2020 -0700
fix array types from escaping into wider query engine (#10460) (#10474)
* fix array types from escaping into wider query engine
* oops
* adjust
* fix lgtm
Co-authored-by: Clint Wylie
---
.../apache/druid/math/expr/BinaryOperatorExpr.java | 8 +-
.../org/apache/druid/math/expr/ConstantExpr.java | 11 ++
.../main/java/org/apache/druid/math/expr/Expr.java | 6 +
.../java/org/apache/druid/math/expr/ExprType.java | 104 --
.../apache/druid/math/expr/ExprTypeConversion.java | 159 +
.../java/org/apache/druid/math/expr/Function.java | 33 +++--
.../org/apache/druid/math/expr/OutputTypeTest.java | 142 ++
.../druid/segment/virtual/ExpressionPlanner.java | 12 +-
.../segment/virtual/ExpressionVirtualColumn.java | 10 +-
.../druid/sql/calcite/expression/Expressions.java | 17 ---
.../builtin/ReductionOperatorConversionHelper.java | 3 +-
.../apache/druid/sql/calcite/planner/Calcites.java | 9 +-
.../apache/druid/sql/calcite/rel/QueryMaker.java | 16 ++-
.../apache/druid/sql/calcite/CalciteQueryTest.java | 30 +++-
14 files changed, 354 insertions(+), 206 deletions(-)
diff --git
a/core/src/main/java/org/apache/druid/math/expr/BinaryOperatorExpr.java
b/core/src/main/java/org/apache/druid/math/expr/BinaryOperatorExpr.java
index 128a780..20ecc5d 100644
--- a/core/src/main/java/org/apache/druid/math/expr/BinaryOperatorExpr.java
+++ b/core/src/main/java/org/apache/druid/math/expr/BinaryOperatorExpr.java
@@ -83,7 +83,13 @@ abstract class BinaryOpExprBase implements Expr
@Override
public ExprType getOutputType(InputBindingTypes inputTypes)
{
-return ExprType.operatorAutoTypeConversion(left.getOutputType(inputTypes),
right.getOutputType(inputTypes));
+if (left.isNullLiteral()) {
+ return right.getOutputType(inputTypes);
+}
+if (right.isNullLiteral()) {
+ return left.getOutputType(inputTypes);
+}
+return ExprTypeConversion.operator(left.getOutputType(inputTypes),
right.getOutputType(inputTypes));
}
@Override
diff --git a/core/src/main/java/org/apache/druid/math/expr/ConstantExpr.java
b/core/src/main/java/org/apache/druid/math/expr/ConstantExpr.java
index 279600d..57ae900 100644
--- a/core/src/main/java/org/apache/druid/math/expr/ConstantExpr.java
+++ b/core/src/main/java/org/apache/druid/math/expr/ConstantExpr.java
@@ -99,6 +99,11 @@ abstract class NullNumericConstantExpr extends ConstantExpr
}
+ @Override
+ public boolean isNullLiteral()
+ {
+return true;
+ }
}
class LongExpr extends ConstantExpr
@@ -429,6 +434,12 @@ class StringExpr extends ConstantExpr
}
@Override
+ public boolean isNullLiteral()
+ {
+return value == null;
+ }
+
+ @Override
public String toString()
{
return value;
diff --git a/core/src/main/java/org/apache/druid/math/expr/Expr.java
b/core/src/main/java/org/apache/druid/math/expr/Expr.java
index be0a32e..ff646fe 100644
--- a/core/src/main/java/org/apache/druid/math/expr/Expr.java
+++ b/core/src/main/java/org/apache/druid/math/expr/Expr.java
@@ -53,6 +53,12 @@ public interface Expr
return false;
}
+ default boolean isNullLiteral()
+ {
+// Overridden by things that are null literals.
+return false;
+ }
+
/**
* Returns the value of expr if expr is a literal, or throws an exception
otherwise.
*
diff --git a/core/src/main/java/org/apache/druid/math/expr/ExprType.java
b/core/src/main/java/org/apache/druid/math/expr/ExprType.java
index e11b8ace..ebdf64a 100644
--- a/core/src/main/java/org/apache/druid/math/expr/ExprType.java
+++ b/core/src/main/java/org/apache/druid/math/expr/ExprType.java
@@ -19,7 +19,6 @@
package org.apache.druid.math.expr;
-import org.apache.druid.java.util.common.IAE;
import org.apache.druid.java.util.common.ISE;
import org.apache.druid.segment.column.ValueType;
@@ -169,107 +168,4 @@ public enum ExprType
return elementType;
}
- /**
- * Given 2 'input' types, choose the most appropriate combined type, if
possible
- *
- * arrays must be the same type
- * if both types are {@link #STRING}, the output type will be preserved as
string
- * if both types are {@link #LONG}, the output type will be preserved as long
- *
- */
- @Nullable
- public static ExprType operatorAutoTypeConversion(@Nullable ExprType type,
@Nullable ExprType other)
- {
-if (type == null || other == null) {
- // cannot auto conversion unknown types
- return
[druid] branch 0.20.0 updated: Fix the offset setting in GoogleStorage#get (#10449) (#10471)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch 0.20.0
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/0.20.0 by this push:
new 41ba126 Fix the offset setting in GoogleStorage#get (#10449) (#10471)
41ba126 is described below
commit 41ba12648dcc0d30f1eaddec4ad5f02077da3af6
Author: Jonathan Wei
AuthorDate: Sat Oct 3 19:50:03 2020 -0700
Fix the offset setting in GoogleStorage#get (#10449) (#10471)
* Fix the offset in get of GCP object
* upgrade compute dependency
* fix version
* review comments
* missed
Co-authored-by: Abhishek Agarwal
<1477457+abhishekagarwa...@users.noreply.github.com>
---
extensions-contrib/gce-extensions/pom.xml | 2 +-
extensions-core/google-extensions/pom.xml | 4 -
.../apache/druid/storage/google/GoogleStorage.java | 8 +-
.../druid/storage/google/GoogleTaskLogs.java | 5 +-
.../druid/storage/google/GoogleStorageTest.java| 90 ++
.../druid/storage/google/GoogleTaskLogsTest.java | 23 +++---
.../druid/storage/google/GoogleTestUtils.java | 10 +++
licenses.yaml | 8 +-
pom.xml| 5 +-
9 files changed, 126 insertions(+), 29 deletions(-)
diff --git a/extensions-contrib/gce-extensions/pom.xml
b/extensions-contrib/gce-extensions/pom.xml
index 0d05bfc..db64e27 100644
--- a/extensions-contrib/gce-extensions/pom.xml
+++ b/extensions-contrib/gce-extensions/pom.xml
@@ -88,7 +88,7 @@
com.google.apis
google-api-services-compute
- v1-rev214-1.25.0
+ ${com.google.apis.compute.version}
compile
diff --git a/extensions-core/google-extensions/pom.xml
b/extensions-core/google-extensions/pom.xml
index 2105153..b14e916 100644
--- a/extensions-core/google-extensions/pom.xml
+++ b/extensions-core/google-extensions/pom.xml
@@ -33,10 +33,6 @@
../../pom.xml
-
-
v1-rev158-${com.google.apis.client.version}
-
-
org.apache.druid
diff --git
a/extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleStorage.java
b/extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleStorage.java
index ee57966..5a63183 100644
---
a/extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleStorage.java
+++
b/extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleStorage.java
@@ -51,11 +51,9 @@ public class GoogleStorage
public InputStream get(final String bucket, final String path, long start)
throws IOException
{
final Get get = storage.objects().get(bucket, path);
-if (start > 0) {
- get.getMediaHttpDownloader().setBytesDownloaded(start);
-}
-get.getMediaHttpDownloader().setDirectDownloadEnabled(false);
-return get.executeMediaAsInputStream();
+InputStream inputStream = get.executeMediaAsInputStream();
+inputStream.skip(start);
+return inputStream;
}
public void delete(final String bucket, final String path) throws IOException
diff --git
a/extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleTaskLogs.java
b/extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleTaskLogs.java
index 1b3c531..d6d9ff5 100644
---
a/extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleTaskLogs.java
+++
b/extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleTaskLogs.java
@@ -142,10 +142,7 @@ public class GoogleTaskLogs implements TaskLogs
start = 0;
}
-InputStream stream = new GoogleByteSource(storage,
config.getBucket(), taskKey).openStream();
-stream.skip(start);
-
-return stream;
+return new GoogleByteSource(storage, config.getBucket(),
taskKey).openStream(start);
}
catch (Exception e) {
throw new IOException(e);
diff --git
a/extensions-core/google-extensions/src/test/java/org/apache/druid/storage/google/GoogleStorageTest.java
b/extensions-core/google-extensions/src/test/java/org/apache/druid/storage/google/GoogleStorageTest.java
new file mode 100644
index 000..c915748
--- /dev/null
+++
b/extensions-core/google-extensions/src/test/java/org/apache/druid/storage/google/GoogleStorageTest.java
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not u
[druid-website-src] branch master updated: add unity, pollfish
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new 7feeb60 add unity, pollfish new ea379d5 Merge pull request #168 from druid-matt/patch-21 7feeb60 is described below commit 7feeb608f4f588dcb3a4e7e8f77ce86016d25f9d Author: Matt Sarrel <59710709+druid-m...@users.noreply.github.com> AuthorDate: Thu Sep 24 11:23:52 2020 -0700 add unity, pollfish --- druid-powered.md | 10 ++ 1 file changed, 10 insertions(+) diff --git a/druid-powered.md b/druid-powered.md index 022832f..9ec6627 100644 --- a/druid-powered.md +++ b/druid-powered.md @@ -428,6 +428,10 @@ is what they have to say: * [Introduction to Druid, fast distributed data store](https://youtu.be/rx6c7D9QmyU) * [Storing and accessing trillions of events](https://plumbr.eu/blog/programming/storing-and-querying-trillions-of-events) +## Pollfish + +The [Pollfish](https://www.pollfish.com/) real-time analytics platform enables researchers to analyze survey data from over half-a-billion consumers across the world, informing advertising, branding, market research, content, and product strategies. Druid is used alongside Apache Kafka, Spark, Flink, Akka Streams, Finatra / Finagle microservices, Cassandra, PostgreSQL, Hive, and Scruid – an open source Scala library for Apache Druid. + ## PubNative PubNative uses Druid for its real-time reports and analysis of millions of daily ad views, clicks, conversions and other events. @@ -578,6 +582,12 @@ In order to continue empowering decision making as Twitch scaled, we turned to u * [Interactive Analytics at MoPub: Querying Terabytes of Data in Seconds](https://blog.twitter.com/engineering/en_us/topics/infrastructure/2019/interactive-analytics-at-mopub.html) +## Unity + +Unity's monetization business generates billions of in-game events in a multi-sided marketplace, which creates complexity, slowness, and overhead for reporting. To work around these issues, Unity deploys a Kafka, Spark, and Druid-based ingestion and aggregation pipeline. + +* [Getting Started with Druid (Imply)](https://medium.com/@linda0511ny/getting-started-with-druid-imply-17d194762574) + ## VideoAmp At VideoAmp, Druid is a key component of our Big Data stack. It powers our real-time video advertising analytics at low granularity and huge scale. Druid has helped us minimized the time between event, insight and action. - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: change meetup data
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new d7fc0c9 change meetup data new 1d0c98e Merge pull request #167 from druid-matt/patch-20 d7fc0c9 is described below commit d7fc0c9a3b2c6598a82db8478c7e89cbec68a207 Author: Matt Sarrel <59710709+druid-m...@users.noreply.github.com> AuthorDate: Wed Sep 23 13:10:13 2020 -0700 change meetup data --- _data/events.yml | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/_data/events.yml b/_data/events.yml index 28b24ea..569d842 100644 --- a/_data/events.yml +++ b/_data/events.yml @@ -1,14 +1,14 @@ -- date: 2020-09-23 - name: "Benchmarking Apache Druid 2: What's Under the Hood?" - info: "Bay Area Apache Druid Meetup" - link: https://www.meetup.com/druidio/events/272938350/ - - date: 2020-09-28 name: "September Office Hours: All About Apache Druid" info: "Bay Area Apache Druid Meetup" link: https://www.meetup.com/druidio/events/273120070/ +- date: 2020-10-01 + name: "Benchmarking Apache Druid 2: What's Under the Hood?" + info: "Bay Area Apache Druid Meetup" + link: https://www.meetup.com/druidio/events/272938350/ + - date: 2020-10-07 name: "Virtual Druid Summit III" info: "A free half-day online event." - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated (49a0930 -> 7cc0a7b)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git.
from 49a0930 Issue fix for CSV loading with header and skip header not
parsing well. (#10398)
add 7cc0a7b Web console: clean up styling imports (#10410)
No new revisions were added by this update.
Summary of changes:
.../parse-time-table.scss => blueprint-overrides/_index.scss}| 4 +---
.../blueprint-overrides/common/_colors.scss} | 9 +
web-console/src/components/auto-form/auto-form.scss | 4 ++--
.../src/dialogs/lookup-edit-dialog/lookup-edit-dialog.scss | 2 +-
web-console/src/dialogs/spec-dialog/spec-dialog.scss | 2 +-
web-console/src/entry.scss | 5 +++--
web-console/src/views/load-data-view/load-data-view.scss | 7 ++-
web-console/src/views/query-view/query-input/query-input.scss| 2 +-
8 files changed, 20 insertions(+), 15 deletions(-)
copy
web-console/src/{views/load-data-view/parse-time-table/parse-time-table.scss =>
blueprint-overrides/_index.scss} (94%)
copy web-console/{jest.common.config.js =>
src/blueprint-overrides/common/_colors.scss} (87%)
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: remove druid summit, add virtual druid summit
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new 1d948f1 remove druid summit, add virtual druid summit new e3e841e Merge pull request #166 from druid-matt/patch-19 1d948f1 is described below commit 1d948f147a1362159e1999289a9796ea82ecfd1a Author: Matt Sarrel <59710709+druid-m...@users.noreply.github.com> AuthorDate: Mon Sep 21 12:54:20 2020 -0700 remove druid summit, add virtual druid summit --- _data/events.yml | 13 - 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/_data/events.yml b/_data/events.yml index 1410bc3..28b24ea 100644 --- a/_data/events.yml +++ b/_data/events.yml @@ -1,9 +1,4 @@ -- date: 2020-09-17 - name: "September Virtual Apache Druid AGA (Ask Gian Anything)" - info: "Bay Area Apache Druid Meetup" - link: https://www.meetup.com/druidio/events/273011362/ - - date: 2020-09-23 name: "Benchmarking Apache Druid 2: What's Under the Hood?" info: "Bay Area Apache Druid Meetup" @@ -14,7 +9,7 @@ info: "Bay Area Apache Druid Meetup" link: https://www.meetup.com/druidio/events/273120070/ -- date: 2020-11-02 - name: "Druid Summit" - info: "San Francisco Airport Marriott Waterfront, 1800 Old Bayshore Highway, Burlingame, CA" - link: https://druidsummit.org/ +- date: 2020-10-07 + name: "Virtual Druid Summit III" + info: "A free half-day online event." + link: https://go.imply.io/Virtual-Druid-Summit-III-Registration.html - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: add to powered by
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new bce2aae add to powered by new e0f2a31 Merge pull request #165 from druid-matt/patch-18 bce2aae is described below commit bce2aae65800a3b045348607de922f28074e5ff1 Author: Matt Sarrel <59710709+druid-m...@users.noreply.github.com> AuthorDate: Mon Sep 21 12:48:02 2020 -0700 add to powered by added Razorpay, Sage+Archer, SuperAwesome linked talk, and Dream 11 linked talk --- druid-powered.md | 12 1 file changed, 12 insertions(+) diff --git a/druid-powered.md b/druid-powered.md index d427585..022832f 100644 --- a/druid-powered.md +++ b/druid-powered.md @@ -142,6 +142,7 @@ Didi uses Druid as a core component of our real-time bigdata processing pipeline Dream11 is consuming data from Apache Druid, Amazon Redshift, and Athena and building models on Looker to report user concurrency, to track user journeys, and to build interaction events based funnels. * [Data Highway — Dream11’s Inhouse Analytics Platform — The Burden and Benefits](https://medium.com/@D11Engg/data-highway-dream11s-inhouse-analytics-platform-the-burden-and-benefits-90b8777d282) +* [Real-Time data and analytics for fantasy sports](https://imply.io/post/real-time-analytics-for-fantasy-sports) ## DripStat @@ -441,6 +442,12 @@ Raygun is a full stack software intelligence platform that monitors your applica * [Using the Druid GroupBy V2 engine for real-time analytics](https://raygun.com/blog/2016/12/druid-groupby-v2-engine/) +## Razorpay + +We have successfully deployed Druid at [Razorpay](https://razorpay.com/) for our use cases and see continued growth in its footprint. We were able to achieve p90, p95 values of less than 5s and 10s respectively and our dashboard performances have improved by at least 10x when compared to Presto. + +* [How Razorpay uses Druid for seamless analytics and product insights?](https://medium.com/@birendra.sahu_77409/how-razorpay-uses-druid-for-seamless-analytics-and-product-insights-364c01b87f1e) + ## Redborder redBorder is an open source, scale out, cybersecurity analytics platform based on Druid. We hope its full-blown web interface, dashboard and report systems, and ready-to-use real-time pipeline foster other Druid users to create a strong community around it. To see more, please visit redborder.org @@ -449,6 +456,10 @@ redBorder is an open source, scale out, cybersecurity analytics platform based o Retargetly is a Data Management Platform that enables publishers and advertisers to manage their first party user data, mix it with second and third party data from others providers and activate it into advertising campaigns (direct, programmatic, etc.). Druid enables us to show real time audience insights. It also provides a lot of flexibility on ad-hoc queries with low latency. We provide default graphs and metrics to our clients but they also have the possibility to make their own int [...] +## Sage + Archer + +We are using Druid as our single source of truth for both realtime statistics and data analysis. Our clients can see and filter detailed metrics in our [self-service DSP](https://www.sage-archer.com). The DSP is used for mobile and digital-out-of-home advertising campaigns. Druid is powering both the front-end and the optimization algorithms within the system. + ## Sift Science Sift Science provides an online trust platform that online businesses use to prevent fraud and abuse. We use Druid as a tool to gain real-time insights about our data and machine learning models. @@ -512,6 +523,7 @@ Sugo is a company that focus on realtime multi-dimension analytics and mining on [SuperAwesome](https://www.superawesome.com)’s mission is to make the internet safer for kids. At the core of SuperAwesome’s analytics is Apache Druid, which helps us relay key insights and data points back to our stakeholders and customers, as well as use this data to power our products themselves. This all happens in a kid-safe way and enables us to deliver the best level of service to billions of children and teens every month. * [How we use Apache Druid’s real-time analytics to power kidtech at SuperAwesome](https://medium.com/superawesome-engineering/how-we-use-apache-druids-real-time-analytics-to-power-kidtech-at-superawesome-8da6a0fb28b1) +* [Virtual Apache Druid Meetup featuring SuperAwesome](https://imply.io/videos/virtual-apache-druid-meetup-featuring-superswesome) ## Sweet Couch - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: add companies from Virtual Druid Summit
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new ea113c1 add companies from Virtual Druid Summit new 8cbdbe2 Merge pull request #163 from druid-matt/patch-16 ea113c1 is described below commit ea113c1489c7c96126f4d9eeeda33be88be35299 Author: Matt Sarrel <59710709+druid-m...@users.noreply.github.com> AuthorDate: Wed Sep 16 14:51:50 2020 -0700 add companies from Virtual Druid Summit --- druid-powered.md | 27 +++ 1 file changed, 27 insertions(+) diff --git a/druid-powered.md b/druid-powered.md index 9970b20..43dae77 100644 --- a/druid-powered.md +++ b/druid-powered.md @@ -38,6 +38,11 @@ Druid is the major player in the real-time analytics pipeline at Appsflyer, and Druid is used to power dynamic analytics and charting in Archive-It Reports. Reports help Archive-It partners understand what content they captured, why some content did not get captured, and assists with quality assurance and crawl scoping to ensure they are building the highest quality collections. +## Athena Health +At [Athena Health](https://www.athenahealth.com/), we are creating a new performance management application for our clients, and one of its key components is Apache Druid. + +* [Automating CI/CD for Druid Clusters at Athena Health](https://imply.io/virtual-druid-summit/automating-ci-cd-for-druid-clusters-at-athena-health) + ## Atomx Atomx is a new media exchange that connects networks, DSPs, SSPs, and other parties. Atomx uses Druid for it's advanced realtime reporting system. Using the Google Cloud modifications Atomx contributed to Druid, it can easily scale Druid with the fast growing platform. @@ -62,6 +67,11 @@ Billy Mobile is a mobile advertising platform, excelling in the performance-base Branch uses Druid as their trusted analytics engine to power all of their data analysis needs. This ranges from the user-facing Branch Dashboard analytics that our partners rely on to gain insight into the performance of their links; to the data-driven business decisions that we need to make internally to build a sustainable business. +## British Telecom (BT) + +* [British Telecom's Apache Druid Architecture](https://imply.io/videos/british-telecoms-apache-druid-architecture) +* [Why British Telecom Chose Druid Over Cassandra](https://imply.io/videos/why-british-telecom-chose-druid-over-cassandra) + ## Central Bank of the Republic of Turkey The Central Bank of the Republic of Turkey addresses the challenge of analyzing and interpreting real-time tick data related to money market instruments using Apache Druid and Superset. @@ -97,6 +107,11 @@ With more than 1 billion unique users reached per month, 3 billion ads displayed * [Real Time Metrics on Tracker Calls](http://labs.criteo.com/2016/02/real-time-metrics-on-tracker-calls/) +## DBS Bank + +[DBS](https://www.dbs.com) is using Apache Druid to handle the AML investigation for the compliance team. The AML (anti-money laundering) workflow generates alerts which are tracked within Druid. The transactional data is ingested from RDBMS to S3 and ingested back to Druid at regular intervals. Investigators can now slice and dice over millions of data with low latency. + +* [Apache Druid for Anti-Money Laundering (AML) at DBS Bank](https://imply.io/virtual-druid-summit/apache-druid-anti-money-laundering-dbs-bank ## Datumo @@ -495,6 +510,12 @@ Sugo is a company that focus on realtime multi-dimension analytics and mining on [TalkingData](http://www.talkingdata.com/) is China’s largest independent Big Data service platform. TalkingData uses Druid with Atomcube, an extension for enhancement, to power analyze online application and advertising data. +## Target + +Apache Druid’s speed and flexibility allow us to provide interactive analytics to front-line, edge-of-business consumers to address hundreds of unique use-cases across several business units. + +* [Enterprise Scale Analytics Platform Powered by Druid at Target](https://imply.io/virtual-druid-summit/enterprise-scale-analytics-platform-powered-by-druid-at-target) + ## Tencent Tencent SCRM product use Druid for customer behavior analysis. @@ -524,8 +545,14 @@ Trendyol, which is the largest e-commerce company in Turkey, uses Druid for real TripleLift uses Druid to provide insights into performance aspects of its native programmatic exchange for sales/business development opportunities, and to provide reporting used by advertisers and publishers. +## Twitch + +In order to continue empowering decision making as Twitch scaled, we turned to using Druid and Imply to provide self service analytics to both our technical and non technical staff allowing them to drill into high level metrics in lieu of reading generated r
[druid-website-src] branch master updated: add superawesome and ibotta
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new b4c2493 add superawesome and ibotta new 27b3115 Merge pull request #162 from druid-matt/patch-15 b4c2493 is described below commit b4c2493f040eeb4230fdb941ded5d99b34924058 Author: Matt Sarrel <59710709+druid-m...@users.noreply.github.com> AuthorDate: Wed Sep 16 11:46:31 2020 -0700 add superawesome and ibotta add superawesome and ibotta --- druid-powered.md | 11 +++ 1 file changed, 11 insertions(+) diff --git a/druid-powered.md b/druid-powered.md index 9970b20..0df27e9 100644 --- a/druid-powered.md +++ b/druid-powered.md @@ -179,6 +179,12 @@ At Hulu, we use Druid to power our analytics platform that enables us to interac [ICSOC Co., Ltd.](https://www.icsoc.net) is the leading call center service provider in China. In our big data department, as traditional ways cannot meet our requirements, eventually we found that Druid, designed in lambda, fully satisfies our requires. It simplified developing statistics or real-time products and highly speeded up OLAP queries. +## Ibotta + +[Ibotta](https://home.ibotta.com/) is a free cash back rewards and payments app. The company has partnered with more than 1,500 brands and retailers to pay consumers cash for their purchases. Ibotta has paid out over $600 million in cash rewards to more than 35 million users since its founding in 2012. + +* [Combatting fraud at Ibotta with Imply](https://imply.io/post/combating-fraud-at-ibotta-with-imply) + ## Imply Imply uses Druid to power public demos and to power our internal monitoring and metrics. @@ -487,6 +493,11 @@ Streamlyzer uses Druid as a next generation online video analytics for online vi Sugo is a company that focus on realtime multi-dimension analytics and mining on big data. We build our platform based on Druid, and developed our own extensions to make it more powerful. +## SuperAwesome +[SuperAwesome](https://www.superawesome.com)’s mission is to make the internet safer for kids. At the core of SuperAwesome’s analytics is Apache Druid, which helps us relay key insights and data points back to our stakeholders and customers, as well as use this data to power our products themselves. This all happens in a kid-safe way and enables us to deliver the best level of service to billions of children and teens every month. + +* [How we use Apache Druid’s real-time analytics to power kidtech at SuperAwesome](https://medium.com/superawesome-engineering/how-we-use-apache-druids-real-time-analytics-to-power-kidtech-at-superawesome-8da6a0fb28b1) + ## Sweet Couch [Sweet Couch](https://www.sweetcouch.com) is a place to discover unique products which are buyable online. Druid powers [Sweet Couch harvest](https://www.sweetcouch.com/harvest/womens) which is an open analytics platform for tracking performance of online shops based out in India. All end user events are tracked and analysed using Druid for business insights. - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated (f71ba6f -> 5751d0e)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from f71ba6f Vectorized ANY aggregators (#10338) add 5751d0e Skip coverage check for tag builds (#10397) No new revisions were added by this update. Summary of changes: .travis.yml | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: remove 1st week of Sept events, add new events
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new 23771a5 remove 1st week of Sept events, add new events new b613177 Merge pull request #159 from druid-matt/patch-12 23771a5 is described below commit 23771a53e06a20cc4517cb8ff3efd29655b3fdb1 Author: Matt Sarrel <59710709+druid-m...@users.noreply.github.com> AuthorDate: Tue Sep 8 10:35:04 2020 -0700 remove 1st week of Sept events, add new events --- _data/events.yml | 13 - 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/_data/events.yml b/_data/events.yml index 7508014..61edc77 100644 --- a/_data/events.yml +++ b/_data/events.yml @@ -1,13 +1,8 @@ -- date: 2020-09-01 - name: "Apache Druid: the open source, performant, real-time, analytical datastore" - info: "DataOps Polska" - link: https://www.meetup.com/DataOps-Polska/events/271935936/ - -- date: 2020-09-02 - name: "Virtual Druid Summit II" - info: "Free Online Half-Day Event" - link: https://go.imply.io/Virtual-Druid-Summit-II-Registration.html +- date: 2020-09-17 + name: "September Virtual Apache Druid AGA (Ask Gian Anything)" + info: "Bay Area Apache Druid Meetup" + link: https://www.meetup.com/druidio/events/273011362/ - date: 2020-09-23 name: "Benchmarking Apache Druid 2: What's Under the Hood?" - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: Update libraries.md - Async Rust client
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new 6dabe3c Update libraries.md - Async Rust client new dd2daee Merge pull request #158 from bugzmanov/patch-1 6dabe3c is described below commit 6dabe3c48ae3f40386ab129e829efb0dfd4c8849 Author: Rafael Bagmanov AuthorDate: Tue Sep 8 06:26:36 2020 -0400 Update libraries.md - Async Rust client --- libraries.md | 4 1 file changed, 4 insertions(+) diff --git a/libraries.md b/libraries.md index 111a5fd..3f9f206 100644 --- a/libraries.md +++ b/libraries.md @@ -59,6 +59,10 @@ Some great folks have written their own libraries to interact with Apache Druid. * [raygun/druid4net](https://github.com/MindscapeHQ/druid4net) - A .NET client for Druid written in C#. Supports the .NET full framework and .NET Core. + Rust + +* [bugzmanov/druid-io-rs](https://github.com/bugzmanov/druid-io-rs) - Fully asynchronous, future-enabled Apache Druid client library for rust programming language. + Other Druid Distributions - * [eBay/embedded-druid](https://github.com/eBay/embedded-druid) - Leveraging Druid capabilities in stand alone application - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated (a5cd5f1 -> a3bb6ee)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from a5cd5f1 Fix VARIANCE aggregator comparator (#10340) add a3bb6ee Add missing comma between JSON members in data-formats.md (#10343) No new revisions were added by this update. Summary of changes: docs/ingestion/data-formats.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated (a3bb6ee -> d7fcff3)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from a3bb6ee Add missing comma between JSON members in data-formats.md (#10343) add d7fcff3 StringFirstAggregatorFactory: Fix incorrect "combine" method. (#10351) No new revisions were added by this update. Summary of changes: .../first/StringFirstAggregatorFactory.java | 2 +- .../first/StringFirstAggregationTest.java | 21 +++-- 2 files changed, 20 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: update events add sept 23 meetup
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new e8bfcb7 update events add sept 23 meetup new 263c35a Merge pull request #157 from druid-matt/patch-11 e8bfcb7 is described below commit e8bfcb7cb08c54a0044a04c02835cc907f3dd5ba Author: Matt Sarrel <59710709+druid-m...@users.noreply.github.com> AuthorDate: Mon Aug 31 11:30:18 2020 -0700 update events add sept 23 meetup --- _data/events.yml | 5 + 1 file changed, 5 insertions(+) diff --git a/_data/events.yml b/_data/events.yml index 91ad3d8..7508014 100644 --- a/_data/events.yml +++ b/_data/events.yml @@ -9,6 +9,11 @@ info: "Free Online Half-Day Event" link: https://go.imply.io/Virtual-Druid-Summit-II-Registration.html +- date: 2020-09-23 + name: "Benchmarking Apache Druid 2: What's Under the Hood?" + info: "Bay Area Apache Druid Meetup" + link: https://www.meetup.com/druidio/events/272938350/ + - date: 2020-11-02 name: "Druid Summit" info: "San Francisco Airport Marriott Waterfront, 1800 Old Bayshore Highway, Burlingame, CA" - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated: Remove NUMERIC_HASHING_THRESHOLD (#10313)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new a9de00d Remove NUMERIC_HASHING_THRESHOLD (#10313)
a9de00d is described below
commit a9de00d43ab34f6d9c5c1ba749617b7f2e1f1559
Author: Suneet Saldanha
AuthorDate: Tue Aug 25 20:05:39 2020 -0700
Remove NUMERIC_HASHING_THRESHOLD (#10313)
* Make NUMERIC_HASHING_THRESHOLD configurable
Change the default numeric hashing threshold to 1 and make it configurable.
Benchmarks attached to this PR show that binary searches are not more faster
than doing a set contains check. The attached flamegraph shows the amount of
time a query spent in the binary search. Given the benchmarks, we can expect
to see roughly a 2x speed up in this part of the query which works out to
~ a 10% faster query in this instance.
* Remove NUMERIC_HASHING_THRESHOLD
* Remove stale docs
---
.../apache/druid/benchmark/ContainsBenchmark.java | 103 +
.../org/apache/druid/query/filter/InDimFilter.java | 59 ++--
.../apache/druid/query/topn/TopNQueryBuilder.java | 2 +-
.../segment/join/filter/JoinFilterAnalyzer.java| 8 +-
.../filter/FloatAndDoubleFilteringTest.java| 9 +-
.../druid/segment/filter/LongFilteringTest.java| 9 +-
.../druid/segment/filter/TimeFilteringTest.java| 5 +-
7 files changed, 145 insertions(+), 50 deletions(-)
diff --git
a/benchmarks/src/test/java/org/apache/druid/benchmark/ContainsBenchmark.java
b/benchmarks/src/test/java/org/apache/druid/benchmark/ContainsBenchmark.java
new file mode 100644
index 000..4e6eb95
--- /dev/null
+++ b/benchmarks/src/test/java/org/apache/druid/benchmark/ContainsBenchmark.java
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.benchmark;
+
+import it.unimi.dsi.fastutil.longs.LongArraySet;
+import it.unimi.dsi.fastutil.longs.LongOpenHashSet;
+import org.openjdk.jmh.annotations.Benchmark;
+import org.openjdk.jmh.annotations.BenchmarkMode;
+import org.openjdk.jmh.annotations.Fork;
+import org.openjdk.jmh.annotations.Measurement;
+import org.openjdk.jmh.annotations.Mode;
+import org.openjdk.jmh.annotations.OutputTimeUnit;
+import org.openjdk.jmh.annotations.Scope;
+import org.openjdk.jmh.annotations.Setup;
+import org.openjdk.jmh.annotations.State;
+import org.openjdk.jmh.annotations.Warmup;
+import org.openjdk.jmh.infra.Blackhole;
+
+import java.util.Arrays;
+import java.util.concurrent.ThreadLocalRandom;
+import java.util.concurrent.TimeUnit;
+
+@State(Scope.Benchmark)
+@BenchmarkMode(Mode.AverageTime)
+@Warmup(iterations = 5)
+@Measurement(iterations = 10)
+@Fork(value = 1)
+public class ContainsBenchmark
+{
+ private static final long[] LONGS;
+ private static final long[] SORTED_LONGS;
+ private static final LongOpenHashSet LONG_HASH_SET;
+ private static final LongArraySet LONG_ARRAY_SET;
+
+ private long worstSearchValue;
+ private long worstSearchValueBin;
+
+ static {
+LONGS = new long[16];
+for (int i = 0; i < LONGS.length; i++) {
+ LONGS[i] = ThreadLocalRandom.current().nextInt(Short.MAX_VALUE);
+}
+
+LONG_HASH_SET = new LongOpenHashSet(LONGS);
+LONG_ARRAY_SET = new LongArraySet(LONGS);
+SORTED_LONGS = Arrays.copyOf(LONGS, LONGS.length);
+
+Arrays.sort(SORTED_LONGS);
+
+ }
+
+ @Setup
+ public void setUp()
+ {
+worstSearchValue = LONGS[LONGS.length - 1];
+worstSearchValueBin = SORTED_LONGS[(SORTED_LONGS.length - 1) >>> 1];
+ }
+
+ @Benchmark
+ @BenchmarkMode(Mode.AverageTime)
+ @OutputTimeUnit(TimeUnit.NANOSECONDS)
+ public void linearSearch(Blackhole blackhole)
+ {
+boolean found = LONG_ARRAY_SET.contains(worstSearchValue);
+blackhole.consume(found);
+ }
+
+ @Benchmark
+ @BenchmarkMode(Mode.AverageTime)
+ @OutputTimeUnit(TimeUnit.NANOSECONDS)
+ public void hashSetSearch(Blackhole blackhole)
+ {
+
+boolean found = LONG_HASH_SET.contains(worstSearchValueBin);
+blackhole.consume(found);
+ }
+
[druid-website-src] branch master updated: Update druid-powered.md
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new 8e21b5d Update druid-powered.md 8e21b5d is described below commit 8e21b5d15e7fe9d148c9ad3e871c5f3a45c19839 Author: Fangjin Yang AuthorDate: Thu Aug 20 14:52:35 2020 -0700 Update druid-powered.md reorder --- druid-powered.md | 17 - 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/druid-powered.md b/druid-powered.md index 14205fb..9970b20 100644 --- a/druid-powered.md +++ b/druid-powered.md @@ -46,6 +46,10 @@ Atomx is a new media exchange that connects networks, DSPs, SSPs, and other part [Bannerflow](https://www.bannerflow.com) is the leading display ad production platform. We use Druid to power our customer facing analytics system and for internal reporting, monitoring and ad-hoc data exploration. +## BIGO + +BIGO selects Druid as an OLAP engine to analyze app(Like, Bigolive, IMO, etc.) data in bigdata team. Typical analysis in Druid includes apm, A/B testing, push, funnel model, etc. Besides, Superset is used as dashboard for deep integration with Druid. + ## Billy Mobile Billy Mobile is a mobile advertising platform, excelling in the performance-based optimisation segment. We use Druid to power our real-time analytics dashboards, in which our publishers, advertisers and staff can get insights on how their campaigns, offers and traffic are performing, with sub-second query time and minute granularity . We are using a lambda-architecture aproach, ingesting the traffic in real time with Tranquility and Storm, and a batch layer via a tight integration with H [...] @@ -252,6 +256,10 @@ which they can do that is in a big part thanks to Druid! * [Data Insights Engine @ MakeMyTrip](https://medium.com/makemytrip-engineering/data-insights-engine-makemytrip-900bd353d99c) +## MAKESENS + +[MAKESENS](http://www.battery-doctor.cn/) use Druid to store and analyze battery data. + ## Marchex Marchex uses Druid to provide data for Marchex Call Analytics' new customer facing Speech Analytics dashboards. @@ -600,14 +608,5 @@ China Youzan is a SaaS company which principally engaged in retail science and t [Zuoyebang](http://www.zuoyebang.com/) is the most used K12 education platform, 7 out of every 10 K12 users are using Zuoyebang. At Zuoyebang Data Platform Group, we use the Druid in the advertising scene, mainly related to advertising display, click, billing, and other functions. The performance and timeliness of druid can meet our OLAP queries very well. -## BIGO - -BIGO selects Druid as an OLAP engine to analyze app(Like, Bigolive, IMO, etc.) data in bigdata team. Typical analysis in Druid includes apm, A/B testing, push, funnel model, etc. Besides, Superset is used as dashboard for deep integration with Druid. - - -## MAKESENS - -[MAKESENS](http://www.battery-doctor.cn/) use Druid to store and analyze battery data. - [Add Your Company](https://github.com/apache/druid-website-src/blob/master/druid-powered.md) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated: Add "offset" parameter to GroupBy query. (#10235)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new b6aaf59 Add "offset" parameter to GroupBy query. (#10235)
b6aaf59 is described below
commit b6aaf59e8cdc4b2965ec9f54d8b824a51baaa594
Author: Gian Merlino
AuthorDate: Wed Aug 5 15:39:58 2020 -0700
Add "offset" parameter to GroupBy query. (#10235)
* Add "offset" parameter to GroupBy query.
It works by doing the query as normal and then throwing away the first
"offset" number of rows on the broker.
* Stabilize GroupBy sorts.
* Fix inspections.
* Fix suppression.
* Fixups.
* Move TopNSequence to druid-core.
* Addl comments.
* NumberedElement equals verification.
* Changes from review.
---
codestyle/spotbugs-exclude.xml | 10 +
.../druid/collections/StableLimitingSorter.java| 136 +++
.../druid/java/util/common/guava/Sequence.java | 14 ++
.../druid/java/util/common/guava/Sequences.java| 11 +-
.../java/util/common/guava/SkippingSequence.java | 104
.../java/util/common/guava}/TopNSequence.java | 55 ++---
.../collections/StableLimitingSorterTest.java | 32 +++
.../util/common/guava/SkippingSequenceTest.java| 160 +
.../java/util/common/guava/TopNSequenceTest.java | 150
docs/querying/limitspec.md | 17 +-
.../apache/druid/query/groupby/GroupByQuery.java | 10 +-
.../epinephelinae/GroupByQueryEngineV2.java| 3 +
.../groupby/epinephelinae/SpillingGrouper.java | 4 +
.../query/groupby/orderby/DefaultLimitSpec.java| 266 ++---
.../query/groupby/strategy/GroupByStrategyV2.java | 9 +-
.../druid/query/MultiValuedDimensionTest.java | 13 +-
.../query/groupby/GroupByQueryRunnerTest.java | 178 +-
.../groupby/orderby/DefaultLimitSpecTest.java | 37 +++
.../query/groupby/orderby/TopNSequenceTest.java| 101
.../apache/druid/sql/calcite/rel/DruidQuery.java | 1 +
.../apache/druid/sql/calcite/CalciteQueryTest.java | 5 +-
21 files changed, 1066 insertions(+), 250 deletions(-)
diff --git a/codestyle/spotbugs-exclude.xml b/codestyle/spotbugs-exclude.xml
index d0b7ee2..973aa7a 100644
--- a/codestyle/spotbugs-exclude.xml
+++ b/codestyle/spotbugs-exclude.xml
@@ -28,6 +28,16 @@
Reference: https://github.com/apache/druid/pull/7894/files
-->
+
+
+
+
+
+
+
+
+
+
diff --git
a/core/src/main/java/org/apache/druid/collections/StableLimitingSorter.java
b/core/src/main/java/org/apache/druid/collections/StableLimitingSorter.java
new file mode 100644
index 000..318751c
--- /dev/null
+++ b/core/src/main/java/org/apache/druid/collections/StableLimitingSorter.java
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.collections;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.MinMaxPriorityQueue;
+import com.google.common.collect.Ordering;
+
+import java.util.Comparator;
+import java.util.Iterator;
+import java.util.Objects;
+
+/**
+ * Simultaneously sorts and limits its input.
+ *
+ * The sort is stable, meaning that equal elements (as determined by the
comparator) will not be reordered.
+ *
+ * Not thread-safe.
+ *
+ * Note: this class doesn't have its own unit tests. It is tested along with
+ * {@link org.apache.druid.java.util.common.guava.TopNSequence} in
"TopNSequenceTest".
+ */
+public class StableLimitingSorter
+{
+ private final MinMaxPriorityQueue> queue;
+
+ private long count = 0;
+
+ public StableLimitingSorter(final Comparator comparator, final int limit)
+ {
+this.queue = MinMaxPriorityQueue
+.orderedBy(
+Ordering.from(
+Comparator.,
T>comparing(NumberedElement::getElem
[druid-website-src] branch master updated: MAKESENS
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new 77f0ed4 MAKESENS new 33bb4b4 Merge pull request #145 from liujunhui/patch-1 77f0ed4 is described below commit 77f0ed4cc70bca7b3e8c951f6be3bff01a035b9f Author: 刘峻辉 <970023...@qq.com> AuthorDate: Sat Jul 25 16:33:00 2020 +0800 MAKESENS ADD USE CASE --- druid-powered.md | 6 ++ 1 file changed, 6 insertions(+) diff --git a/druid-powered.md b/druid-powered.md index a2bb556..3a5abfd 100644 --- a/druid-powered.md +++ b/druid-powered.md @@ -595,4 +595,10 @@ China Youzan is a SaaS company which principally engaged in retail science and t BIGO selects Druid as an OLAP engine to analyze app(Like, Bigolive, IMO, etc.) data in bigdata team. Typical analysis in Druid includes apm, A/B testing, push, funnel model, etc. Besides, Superset is used as dashboard for deep integration with Druid. + +## MAKESENS + +[MAKESENS](http://www.battery-doctor.cn/) use Druid to store and analyze battery data. + + [Add Your Company](https://github.com/apache/druid-website-src/blob/master/druid-powered.md) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated: JettyTest.testNumConnectionsMetricHttp is rarely flaky (#10169)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new 454eed0 JettyTest.testNumConnectionsMetricHttp is rarely flaky
(#10169)
454eed0 is described below
commit 454eed06a872e369e4d2fc2e3b955457ffa081ee
Author: Maytas Monsereenusorn
AuthorDate: Tue Jul 14 10:36:46 2020 -0700
JettyTest.testNumConnectionsMetricHttp is rarely flaky (#10169)
---
.../test/java/org/apache/druid/server/initialization/JettyTest.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git
a/server/src/test/java/org/apache/druid/server/initialization/JettyTest.java
b/server/src/test/java/org/apache/druid/server/initialization/JettyTest.java
index 2ab9f7c..0e7459b 100644
--- a/server/src/test/java/org/apache/druid/server/initialization/JettyTest.java
+++ b/server/src/test/java/org/apache/druid/server/initialization/JettyTest.java
@@ -511,13 +511,13 @@ public class JettyTest extends BaseJettyTest
{
// it can take a bit to close the connection, so maybe sleep for a while
and hope it closes
final int sleepTimeMills = 10;
-final int totalSleeps = 10_000 / sleepTimeMills;
+final int totalSleeps = 15_000 / sleepTimeMills;
int count = 0;
while (jsm.getActiveConnections() > 0 && count++ < totalSleeps) {
Thread.sleep(sleepTimeMills);
}
if (jsm.getActiveConnections() > 0) {
- throw new RuntimeException("Connections greater than 0");
+ throw new RuntimeException("Connections greater than 0.
activeConnections=" + jsm.getActiveConnections() + " port=" + port);
}
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated: Fix flaky tests in DruidCoordinatorTest (#10157)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new 54a8fb8 Fix flaky tests in DruidCoordinatorTest (#10157)
54a8fb8 is described below
commit 54a8fb827df6ca2d7b0f1e86656d0cd5a978f273
Author: Maytas Monsereenusorn
AuthorDate: Wed Jul 8 20:03:52 2020 -0700
Fix flaky tests in DruidCoordinatorTest (#10157)
* Fix flaky tests in DruidCoordinatorTest
* Imporve fail msg
* Fix flaky tests in DruidCoordinatorTest
---
.../druid/server/coordinator/DruidCoordinatorTest.java | 18 ++
1 file changed, 6 insertions(+), 12 deletions(-)
diff --git
a/server/src/test/java/org/apache/druid/server/coordinator/DruidCoordinatorTest.java
b/server/src/test/java/org/apache/druid/server/coordinator/DruidCoordinatorTest.java
index 0077fc1..51308cb 100644
---
a/server/src/test/java/org/apache/druid/server/coordinator/DruidCoordinatorTest.java
+++
b/server/src/test/java/org/apache/druid/server/coordinator/DruidCoordinatorTest.java
@@ -103,12 +103,10 @@ public class DruidCoordinatorTest extends CuratorTestBase
private ObjectMapper objectMapper;
private DruidNode druidNode;
private LatchableServiceEmitter serviceEmitter = new
LatchableServiceEmitter();
- private boolean serverAddedCountExpected = true;
@Before
public void setUp() throws Exception
{
-serverAddedCountExpected = true;
druidServer = EasyMock.createMock(DruidServer.class);
serverInventoryView = EasyMock.createMock(SingleServerInventoryView.class);
segmentsMetadataManager =
EasyMock.createNiceMock(SegmentsMetadataManager.class);
@@ -385,7 +383,6 @@ public class DruidCoordinatorTest extends CuratorTestBase
druidServer
);
assignSegmentLatch.await();
-Assert.assertTrue(serverAddedCountExpected);
final CountDownLatch coordinatorRunLatch = new CountDownLatch(2);
serviceEmitter.latch = coordinatorRunLatch;
@@ -486,7 +483,6 @@ public class DruidCoordinatorTest extends CuratorTestBase
final CountDownLatch assignSegmentLatchCold =
createCountDownLatchAndSetPathChildrenCacheListenerWithLatch(1,
pathChildrenCacheCold, dataSegments, coldServer);
assignSegmentLatchHot.await();
assignSegmentLatchCold.await();
-Assert.assertTrue(serverAddedCountExpected);
final CountDownLatch coordinatorRunLatch = new CountDownLatch(2);
serviceEmitter.latch = coordinatorRunLatch;
@@ -646,7 +642,6 @@ public class DruidCoordinatorTest extends CuratorTestBase
assignSegmentLatchBroker1.await();
assignSegmentLatchBroker2.await();
assignSegmentLatchPeon.await();
-Assert.assertTrue(serverAddedCountExpected);
final CountDownLatch coordinatorRunLatch = new CountDownLatch(2);
serviceEmitter.latch = coordinatorRunLatch;
@@ -679,16 +674,15 @@ public class DruidCoordinatorTest extends CuratorTestBase
pathChildrenCache.getListenable().addListener(
(CuratorFramework client, PathChildrenCacheEvent event) -> {
if (CuratorUtils.isChildAdded(event)) {
-if (countDownLatch.getCount() > 0) {
- DataSegment segment = findSegmentRelatedToCuratorEvent(segments,
event);
- if (segment != null) {
+DataSegment segment = findSegmentRelatedToCuratorEvent(segments,
event);
+if (segment != null && server.getSegment(segment.getId()) == null)
{
+ if (countDownLatch.getCount() > 0) {
server.addDataSegment(segment);
curator.delete().guaranteed().forPath(event.getData().getPath());
+countDownLatch.countDown();
+ } else {
+Assert.fail("The segment path " + event.getData().getPath() +
" is not expected");
}
- countDownLatch.countDown();
-} else {
- // The segment is assigned to the server more times than expected
- serverAddedCountExpected = false;
}
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated (6160bb0 -> 4888d6e)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git. from 6160bb0 Merge pull request #141 from apache/update_events add ea2d26c Add Dream11 to powered by page add cd88850 Merge branch 'master' of github.com:apache/druid-website-src into powered-by new 4888d6e Merge pull request #142 from apache/powered-by The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: druid-powered.md | 6 ++ 1 file changed, 6 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] 01/01: Merge pull request #142 from apache/powered-by
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git commit 4888d6e147d88f42e41e98e95ffe2ac1640b46f2 Merge: 6160bb0 cd88850 Author: Fangjin Yang AuthorDate: Mon Jul 6 10:33:36 2020 -0700 Merge pull request #142 from apache/powered-by Update Powered by page druid-powered.md | 6 ++ 1 file changed, 6 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] 01/01: Merge pull request #141 from apache/update_events
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git commit 6160bb083ef7279e8d1cf5c2cf41570ffddfea77 Merge: e53430c 93abf8a Author: Fangjin Yang AuthorDate: Mon Jul 6 10:24:30 2020 -0700 Merge pull request #141 from apache/update_events Add upcoming events _data/events.yml | 10 ++ 1 file changed, 10 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated (e53430c -> 6160bb0)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git. from e53430c Merge pull request #140 from apache/edit add 93abf8a Add upcoming events new 6160bb0 Merge pull request #141 from apache/update_events The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: _data/events.yml | 10 ++ 1 file changed, 10 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated: VersionedIntervalTimeline: Fix thread-unsafe call to "lookup". (#10130)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new ddda2a4 VersionedIntervalTimeline: Fix thread-unsafe call to
"lookup". (#10130)
ddda2a4 is described below
commit ddda2a4f18597257633b55536d8a54e1a3f61eb3
Author: Gian Merlino
AuthorDate: Sun Jul 5 09:32:18 2020 -0700
VersionedIntervalTimeline: Fix thread-unsafe call to "lookup". (#10130)
---
.../apache/druid/timeline/VersionedIntervalTimeline.java | 15 +--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git
a/core/src/main/java/org/apache/druid/timeline/VersionedIntervalTimeline.java
b/core/src/main/java/org/apache/druid/timeline/VersionedIntervalTimeline.java
index 783bf8b..376a97f 100644
---
a/core/src/main/java/org/apache/druid/timeline/VersionedIntervalTimeline.java
+++
b/core/src/main/java/org/apache/druid/timeline/VersionedIntervalTimeline.java
@@ -164,8 +164,18 @@ public class VersionedIntervalTimeline findNonOvershadowedObjectsInInterval(Interval
interval, Partitions completeness)
{
+final List> holders;
+
+lock.readLock().lock();
+try {
+ holders = lookup(interval, completeness);
+}
+finally {
+ lock.readLock().unlock();
+}
+
return FluentIterable
-.from(lookup(interval, completeness))
+.from(holders)
.transformAndConcat(TimelineObjectHolder::getObject)
.transform(PartitionChunk::getObject)
.toSet();
@@ -273,7 +283,8 @@ public class VersionedIntervalTimeline findEntry(Interval interval,
VersionType version)
+ @Nullable
+ public PartitionHolder findEntry(Interval interval, VersionType
version)
{
lock.readLock().lock();
try {
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated: More prominent instructions on code coverage failure (#10060)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new b7d771f More prominent instructions on code coverage failure (#10060)
b7d771f is described below
commit b7d771f633fb3c54490ea1a1f8df6691ea4bb4e1
Author: Suneet Saldanha
AuthorDate: Thu Jun 25 19:48:30 2020 -0700
More prominent instructions on code coverage failure (#10060)
* More prominent instructions on code coverage failure
* Update .travis.yml
---
.travis.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.travis.yml b/.travis.yml
index 9b3c755..61450ef 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -183,7 +183,7 @@ jobs:
--log-template "totals-complete"
--log-template "errors"
--
- || { printf "\nDiff code coverage check failed. To view coverage
report, run 'mvn clean test jacoco:report' and open
'target/site/jacoco/index.html'\n" && false; }
+ || { printf "\n\nFAILED\nDiff code coverage check failed. To
view coverage report, run 'mvn clean test jacoco:report' and open
'target/site/jacoco/index.html'\nFor more details on how to run code coverage
locally, follow instructions here -
https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md#running-code-coverage-locally\n\n;
&& false; }
fi
after_success:
# retry in case of network error
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website] branch asf-site updated: Autobuild (#85)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/druid-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 005a07c Autobuild (#85) 005a07c is described below commit 005a07cc966bf55d576f68605e9ca93e96830358 Author: Vadim Ogievetsky AuthorDate: Thu Jun 18 10:25:17 2020 -0700 Autobuild (#85) * Autobuild * restore files --- community/index.html | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/community/index.html b/community/index.html index 6df99dc..b76f71b 100644 --- a/community/index.html +++ b/community/index.html @@ -317,7 +317,7 @@ by others. https://github.com/b-slim;>Mohamed Slim Bouguerra -https://hortonworks.com/;>Hortonworks +https://www.linkedin.com/;>LinkedIn https://github.com/navis;>Navis Ryu @@ -329,7 +329,7 @@ by others. https://github.com/nishantmonu51;>Nishant Bangarwa -https://hortonworks.com/;>Hortonworks +Independent https://github.com/pjain1;>Parag Jain - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch u created (now 24d14c2)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch u in repository https://gitbox.apache.org/repos/asf/druid-website-src.git. at 24d14c2 update This branch includes the following new commits: new 24d14c2 update The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] 01/01: update
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch u in repository https://gitbox.apache.org/repos/asf/druid-website-src.git commit 24d14c22b67c423f82ee27545e97be668c129bc8 Author: fjy AuthorDate: Thu Jun 18 10:20:51 2020 -0700 update --- community/index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/community/index.md b/community/index.md index 285da1e..ab922d3 100644 --- a/community/index.md +++ b/community/index.md @@ -113,10 +113,10 @@ You don't need to be a committer to contribute- pull requests are welcome from a | [Lijin Bin](https://github.com/binlijin) | [Alibaba](https://www.alibaba.com/)| | [Maxime Beauchemin](https://github.com/mistercrunch) | [Lyft](http://lyft.com/) | | [Mingming Qiu](https://github.com/QiuMM) | [Bytedance](https://bytedance.com/)| -| [Mohamed Slim Bouguerra](https://github.com/b-slim) | [Hortonworks](https://hortonworks.com/)| +| [Mohamed Slim Bouguerra](https://github.com/b-slim) | [LinkedIn](https://www.linkedin.com/) | | [Navis Ryu](https://github.com/navis) | [SK Telecom](https://www.sktelecom.com/index.html) | | [Niketh Sabbineni](https://github.com/niketh) | [Oath](https://www.oath.com/) | -| [Nishant Bangarwa](https://github.com/nishantmonu51) | [Hortonworks](https://hortonworks.com/)| +| [Nishant Bangarwa](https://github.com/nishantmonu51) | Independent | | [Parag Jain](https://github.com/pjain1) | [Lyft](http://lyft.com/) | | [P. Taylor Goetz](https://github.com/ptgoetz) | [EPAM](https://www.epam.com/) | | [Roman Leventov](https://github.com/leventov) | [Snap](https://www.snap.com/en-US/)| - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated: make phaser of ReferenceCountingCloseableObject protected instead of private so subclasses can do stuff with it (#10035)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid.git The following commit(s) were added to refs/heads/master by this push: new 9468df4 make phaser of ReferenceCountingCloseableObject protected instead of private so subclasses can do stuff with it (#10035) 9468df4 is described below commit 9468df4721c478a9c96ed2ce713c4ee79f2bb99c Author: Clint Wylie AuthorDate: Mon Jun 15 19:56:49 2020 -0700 make phaser of ReferenceCountingCloseableObject protected instead of private so subclasses can do stuff with it (#10035) --- .../java/org/apache/druid/segment/ReferenceCountingCloseableObject.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/processing/src/main/java/org/apache/druid/segment/ReferenceCountingCloseableObject.java b/processing/src/main/java/org/apache/druid/segment/ReferenceCountingCloseableObject.java index db37fc5..af0e1c7 100644 --- a/processing/src/main/java/org/apache/druid/segment/ReferenceCountingCloseableObject.java +++ b/processing/src/main/java/org/apache/druid/segment/ReferenceCountingCloseableObject.java @@ -41,7 +41,7 @@ public abstract class ReferenceCountingCloseableObject
[druid] branch master updated (7282e2f -> 5da78d1)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from 7282e2f Simplify CompressedVSizeColumnarIntsSupplierTest (#10003) add 5da78d1 Update password-provider.md (#9857) No new revisions were added by this update. Summary of changes: docs/operations/password-provider.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated (d4d0004 -> 7282e2f)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from d4d0004 Fix failed tests in TimestampParserTest when running locally (#9997) add 7282e2f Simplify CompressedVSizeColumnarIntsSupplierTest (#10003) No new revisions were added by this update. Summary of changes: .../segment/data/CompressedVSizeColumnarIntsSupplierTest.java | 7 +-- 1 file changed, 1 insertion(+), 6 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated: Fix failed tests in TimestampParserTest when running locally (#9997)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new d4d0004 Fix failed tests in TimestampParserTest when running locally
(#9997)
d4d0004 is described below
commit d4d00043383172bf393513fb33e95918807bb10c
Author: BIGrey
AuthorDate: Thu Jun 11 00:19:38 2020 +0800
Fix failed tests in TimestampParserTest when running locally (#9997)
* fix failed tests in TimestampPaserTest due to timezone
* remove unneeded -Duser.country=US
Co-authored-by: huagnhui.bigrey
---
core/pom.xml | 1 +
1 file changed, 1 insertion(+)
diff --git a/core/pom.xml b/core/pom.xml
index 627b3af..839d19b 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -394,6 +394,7 @@
@{jacocoArgLine}
-Djava.library.path=${project.build.directory}/hyperic-sigar-${sigar.base.version}/sigar-bin/lib/
+-Duser.language=en
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated (3111e45 -> 0be1c30)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git. from 3111e45 Merge pull request #134 from spaghettifunk/patch-1 add d85ae46 update events and poweredby pages new 0be1c30 Merge pull request #135 from apache/updates The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: _data/events.yml | 8 druid-powered.md | 1 + 2 files changed, 5 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] 01/01: Merge pull request #135 from apache/updates
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git commit 0be1c300b5bf5d6be6c30f77802ac102fef2d2de Merge: 3111e45 d85ae46 Author: Fangjin Yang AuthorDate: Fri Jun 5 17:48:55 2020 -0700 Merge pull request #135 from apache/updates update events and poweredby pages _data/events.yml | 8 druid-powered.md | 1 + 2 files changed, 5 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: Update libraries.md
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new a93ef25 Update libraries.md new 3111e45 Merge pull request #134 from spaghettifunk/patch-1 a93ef25 is described below commit a93ef25c01b3287bb5122ab8b6420af1aa77976a Author: Davide Berdin AuthorDate: Sun May 31 15:24:30 2020 +0200 Update libraries.md Add [druid-prometheus-exporter](https://github.com/spaghettifunk/druid-prometheus-exporter) in the `tools` section --- libraries.md | 1 + 1 file changed, 1 insertion(+) diff --git a/libraries.md b/libraries.md index 3fc03ff..bdef576 100644 --- a/libraries.md +++ b/libraries.md @@ -80,6 +80,7 @@ Tools * [Insert Segments](/docs/latest/operations/insert-segment-to-db.html) - A tool that can insert segments' metadata into Druid metadata storage. * [liquidm/druid-dumbo](https://github.com/liquidm/druid-dumbo) - Scripts to help generate batch configs for the ingestion of data into Druid * [housejester/druid-test-harness](https://github.com/housejester/druid-test-harness) - A set of scripts to simplify standing up some servers and seeing how things work +* [spaghettifunk/druid-prometheus-exporter](https://github.com/spaghettifunk/druid-prometheus-exporter) - A HTTP service for collecting Druid metrics and exposing them as Prometheus metrics Community Extensions - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: Adding BIGO Company usage
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new 2f6f5a7 Adding BIGO Company usage new 783ae28 Merge pull request #128 from sekingme/addBigo 2f6f5a7 is described below commit 2f6f5a7cbe418bc11086366dcdc3642adc33edf7 Author: jiangshequan AuthorDate: Mon Apr 27 15:18:22 2020 +0800 Adding BIGO Company usage --- druid-powered.md | 4 1 file changed, 4 insertions(+) diff --git a/druid-powered.md b/druid-powered.md index 556d052..8659be6 100644 --- a/druid-powered.md +++ b/druid-powered.md @@ -584,4 +584,8 @@ China Youzan is a SaaS company which principally engaged in retail science and t [Zuoyebang](http://www.zuoyebang.com/) is the most used K12 education platform, 7 out of every 10 K12 users are using Zuoyebang. At Zuoyebang Data Platform Group, we use the Druid in the advertising scene, mainly related to advertising display, click, billing, and other functions. The performance and timeliness of druid can meet our OLAP queries very well. +## BIGO + +BIGO selects Druid as an OLAP engine to analyze app(Like, Bigolive, IMO, etc.) data in bigdata team. Typical analysis in Druid includes apm, A/B testing, push, funnel model, etc. Besides, Superset is used as dashboard for deep integration with Druid. + [Add Your Company](https://github.com/apache/druid-website-src/blob/master/druid-powered.md) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated (b279e04 -> a107ee3)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from b279e04 table fix (#9769) add a107ee3 Fix problem when running single integration test using -Dit.test= (#9778) No new revisions were added by this update. Summary of changes: integration-tests/pom.xml | 12 +++ .../java/org/testng/DruidTestRunnerFactory.java| 105 + 2 files changed, 117 insertions(+) create mode 100644 integration-tests/src/main/java/org/testng/DruidTestRunnerFactory.java - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated (a107ee3 -> 6bc64b7)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from a107ee3 Fix problem when running single integration test using -Dit.test= (#9778) add 6bc64b7 Improve "waiting for tasks complete" logic in integration tests (#9759) No new revisions were added by this update. Summary of changes: .../tests/coordinator/duty/ITAutoCompactionTest.java | 2 +- .../apache/druid/tests/indexer/AbstractIndexerTest.java | 16 ++-- 2 files changed, 7 insertions(+), 11 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated (e7e41e3 -> b279e04)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from e7e41e3 Adding support for autoscaling in GCE (#8987) add b279e04 table fix (#9769) No new revisions were added by this update. Summary of changes: docs/querying/sql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] 01/01: Merge pull request #126 from apache/featured-content
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git commit a41ac0cbe15c057709d1653763b656ca603d6983 Merge: 5b57ba6 e7b0973 Author: Fangjin Yang AuthorDate: Mon Apr 20 10:18:48 2020 -0700 Merge pull request #126 from apache/featured-content update featured content _data/featured.yml | 42 ++ 1 file changed, 42 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated (5b57ba6 -> a41ac0c)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git. from 5b57ba6 Merge pull request #120 from apache/update-powered-by add e7b0973 update featured content: add virtual Druid summit talks new a41ac0c Merge pull request #126 from apache/featured-content The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: _data/featured.yml | 42 ++ 1 file changed, 42 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] 01/01: Merge pull request #120 from apache/update-powered-by
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git commit 5b57ba6318c48eaf609d1d0654d20fe7ab4c9f0d Merge: a9623d6 73a6b50 Author: Fangjin Yang AuthorDate: Fri Apr 17 16:46:38 2020 -0700 Merge pull request #120 from apache/update-powered-by update wikimedia foundation powered by druid-powered.md | 1 + 1 file changed, 1 insertion(+) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: Adding Fanatics Company usage
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new 2f80b15 Adding Fanatics Company usage new a9623d6 Merge pull request #125 from im2kul/patch-1 2f80b15 is described below commit 2f80b15f6c3b48c7914a0190d90ae870b9407b00 Author: im2kul AuthorDate: Fri Apr 17 16:34:47 2020 -0700 Adding Fanatics Company usage --- druid-powered.md | 4 1 file changed, 4 insertions(+) diff --git a/druid-powered.md b/druid-powered.md index eec24d8..8833ca2 100644 --- a/druid-powered.md +++ b/druid-powered.md @@ -138,6 +138,10 @@ eBay uses Druid to aggregate multiple data streams for real-time user behavior a One of our key goals at [FacilityConneX](https://www.facilityconnex.com/) is to offer real-time insights that help our customers optimize their equipment or processes, reduce cost, or prevent incidents, to ultimately improve our customers business. This *real-time* requirement has always been a major technical challenge to scale within our SaaS environment. FacilityConneX has looked to **Druid** to help solve many of these challenging performance and growth issues. +## Fanatics + +Druid powers our applications related metrics and helps us slice and dice analytics on both historical and realtime-time metrics. It significantly reduces latency of analytic queries and help people to get insights more interactively. Through druid data we can now do anomaly detection as well + ## FullContact * [Enrich API Brings Higher Match Rates with Multi-Field Enrichment Capabilities](https://www.fullcontact.com/blog/enrich-api-brings-higher-match-rates-with-multi-field-enrichment-capabilities/) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch 0.18.0 updated: fix flaky jetty test (#9633) (#9680)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch 0.18.0
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/0.18.0 by this push:
new c11cd3e fix flaky jetty test (#9633) (#9680)
c11cd3e is described below
commit c11cd3e16ad91e8e7224ec5b97de17cc0cb0a44e
Author: Jihoon Son
AuthorDate: Fri Apr 10 23:38:15 2020 -0700
fix flaky jetty test (#9633) (#9680)
Co-authored-by: Clint Wylie
---
.../java/org/apache/druid/server/initialization/JettyTest.java | 10 +-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git
a/server/src/test/java/org/apache/druid/server/initialization/JettyTest.java
b/server/src/test/java/org/apache/druid/server/initialization/JettyTest.java
index 64ccd14..206c318 100644
--- a/server/src/test/java/org/apache/druid/server/initialization/JettyTest.java
+++ b/server/src/test/java/org/apache/druid/server/initialization/JettyTest.java
@@ -78,6 +78,7 @@ import java.util.Map;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;
+import java.util.concurrent.ThreadLocalRandom;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicLong;
import java.util.zip.GZIPInputStream;
@@ -186,6 +187,8 @@ public class JettyTest extends BaseJettyTest
throw new RuntimeException(e);
}
+final int ephemeralPort = ThreadLocalRandom.current().nextInt(49152,
65535);
+
latchedRequestState = new LatchedRequestStateHolder();
injector = Initialization.makeInjectorWithModules(
GuiceInjectors.makeStartupInjector(),
@@ -198,7 +201,7 @@ public class JettyTest extends BaseJettyTest
JsonConfigProvider.bindInstance(
binder,
Key.get(DruidNode.class, Self.class),
-new DruidNode("test", "localhost", false, 9988, ,
true, true)
+new DruidNode("test", "localhost", false, ephemeralPort,
ephemeralPort + 1, true, true)
);
binder.bind(TLSServerConfig.class).toInstance(tlsConfig);
binder.bind(JettyServerInitializer.class).to(JettyServerInit.class).in(LazySingleton.class);
@@ -450,6 +453,7 @@ public class JettyTest extends BaseJettyTest
JettyServerModule jsm = injector.getInstance(JettyServerModule.class);
latchedRequestState.reset();
+waitForJettyServerModuleActiveConnectionsZero(jsm);
Assert.assertEquals(0, jsm.getActiveConnections());
ListenableFuture go = client.go(
@@ -489,6 +493,7 @@ public class JettyTest extends BaseJettyTest
JettyServerModule jsm = injector.getInstance(JettyServerModule.class);
latchedRequestState.reset();
+waitForJettyServerModuleActiveConnectionsZero(jsm);
Assert.assertEquals(0, jsm.getActiveConnections());
ListenableFuture go = client.go(
request,
@@ -511,5 +516,8 @@ public class JettyTest extends BaseJettyTest
while (jsm.getActiveConnections() > 0 && count++ < totalSleeps) {
Thread.sleep(sleepTimeMills);
}
+if (jsm.getActiveConnections() > 0) {
+ throw new RuntimeException("Connections greater than 0");
+}
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch 0.18.0 updated: SQL: More straightforward handling of join planning. (#9648) (#9659)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch 0.18.0
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/0.18.0 by this push:
new 95a4d2a SQL: More straightforward handling of join planning. (#9648)
(#9659)
95a4d2a is described below
commit 95a4d2adde64572b57b744ab36a54672d7f265ed
Author: Gian Merlino
AuthorDate: Fri Apr 10 09:06:05 2020 -0700
SQL: More straightforward handling of join planning. (#9648) (#9659)
* SQL: More straightforward handling of join planning.
Two changes that simplify how joins are planned:
1) Stop using JoinProjectTransposeRule as a way of guiding subquery
removal. Instead, add logic to DruidJoinRule that identifies removable
subqueries and removes them at the point of creating a DruidJoinQueryRel.
This approach reduces the size of the planning space and allows the
planner to complete quickly.
2) Remove rules that reorder joins. Not because of an impact on the
planning time (it seems minimal), but because the decisions that the
planner was making in the new tests were sometimes worse than the
user-provided order. I think we'll need to go with the user-provided
order for now, and revisit reordering when we can add more smarts to
the cost estimator.
A third change updates numeric ExprEval classes to store their
value as a boxed type that corresponds to what it is supposed to be.
This is useful because it affects the behavior of "asString", and
is included in this patch because it is needed for the new test
"testInnerJoinTwoLookupsToTableUsingNumericColumnInReverse". This
test relies on CAST('6', 'DOUBLE') stringifying to "6.0" like an
actual double would.
Fixes #9646.
* Fix comments.
* Fix tests.
---
.../java/org/apache/druid/math/expr/ExprEval.java | 4 +-
.../org/apache/druid/math/expr/FunctionTest.java | 6 +-
.../apache/druid/sql/calcite/planner/Rules.java| 40 +-
.../druid/sql/calcite/rel/CostEstimates.java | 14 +-
.../druid/sql/calcite/rel/DruidJoinQueryRel.java | 90 ++---
.../druid/sql/calcite/rel/DruidOuterQueryRel.java | 6 -
.../apache/druid/sql/calcite/rel/DruidQuery.java | 3 +-
.../druid/sql/calcite/rel/DruidQueryRel.java | 6 -
.../org/apache/druid/sql/calcite/rel/DruidRel.java | 7 -
.../druid/sql/calcite/rel/DruidUnionRel.java | 6 -
.../druid/sql/calcite/rule/DruidJoinRule.java | 294 +--
.../apache/druid/sql/calcite/CalciteQueryTest.java | 408 -
12 files changed, 737 insertions(+), 147 deletions(-)
diff --git a/core/src/main/java/org/apache/druid/math/expr/ExprEval.java
b/core/src/main/java/org/apache/druid/math/expr/ExprEval.java
index b6f1f58..a993790 100644
--- a/core/src/main/java/org/apache/druid/math/expr/ExprEval.java
+++ b/core/src/main/java/org/apache/druid/math/expr/ExprEval.java
@@ -244,7 +244,7 @@ public abstract class ExprEval
{
private DoubleExprEval(@Nullable Number value)
{
- super(value == null ? NullHandling.defaultDoubleValue() : value);
+ super(value == null ? NullHandling.defaultDoubleValue() : (Double)
value.doubleValue());
}
@Override
@@ -304,7 +304,7 @@ public abstract class ExprEval
{
private LongExprEval(@Nullable Number value)
{
- super(value == null ? NullHandling.defaultLongValue() : value);
+ super(value == null ? NullHandling.defaultLongValue() : (Long)
value.longValue());
}
@Override
diff --git a/core/src/test/java/org/apache/druid/math/expr/FunctionTest.java
b/core/src/test/java/org/apache/druid/math/expr/FunctionTest.java
index 9239e77..2fe5249 100644
--- a/core/src/test/java/org/apache/druid/math/expr/FunctionTest.java
+++ b/core/src/test/java/org/apache/druid/math/expr/FunctionTest.java
@@ -175,7 +175,7 @@ public class FunctionTest extends
InitializedNullHandlingTest
public void testArrayLength()
{
assertExpr("array_length([1,2,3])", 3L);
-assertExpr("array_length(a)", 4);
+assertExpr("array_length(a)", 4L);
}
@Test
@@ -199,7 +199,7 @@ public class FunctionTest extends
InitializedNullHandlingTest
{
assertExpr("array_offset_of([1, 2, 3], 3)", 2L);
assertExpr("array_offset_of([1, 2, 3], 4)",
NullHandling.replaceWithDefault() ? -1L : null);
-assertExpr("array_offset_of(a, 'baz')", 2);
+assertExpr("array_offset_of(a, 'baz')", 2L);
}
@Test
@@ -207,7 +207,7 @@ public class FunctionTest extends
InitializedNullHandlingTest
{
assertExpr("array_ordinal_of([1, 2, 3], 3)", 3L);
assertExpr("array_ordinal_of([1, 2, 3], 4)",
NullHandling.replaceWithDefault() ? -1L : null);
-assertExpr("array_ordinal_of(a, 'baz')", 3);
+
[druid] branch master updated: Fix wrong cardinality computation in BufferArrayGrouper (#9655)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new e157fb0 Fix wrong cardinality computation in BufferArrayGrouper
(#9655)
e157fb0 is described below
commit e157fb089a8f1863b49ea14841d7653582a60117
Author: Jihoon Son
AuthorDate: Fri Apr 10 09:05:38 2020 -0700
Fix wrong cardinality computation in BufferArrayGrouper (#9655)
* Fix wrong cardinality computation in BufferArrayGrouper
* fix javadoc
---
.../query/groupby/epinephelinae/BufferArrayGrouper.java | 16
.../groupby/epinephelinae/GroupByQueryEngineV2.java | 6 +-
.../groupby/epinephelinae/BufferArrayGrouperTest.java| 11 ---
3 files changed, 21 insertions(+), 12 deletions(-)
diff --git
a/processing/src/main/java/org/apache/druid/query/groupby/epinephelinae/BufferArrayGrouper.java
b/processing/src/main/java/org/apache/druid/query/groupby/epinephelinae/BufferArrayGrouper.java
index 9b912e4..864b6a3 100644
---
a/processing/src/main/java/org/apache/druid/query/groupby/epinephelinae/BufferArrayGrouper.java
+++
b/processing/src/main/java/org/apache/druid/query/groupby/epinephelinae/BufferArrayGrouper.java
@@ -69,12 +69,20 @@ public class BufferArrayGrouper implements VectorGrouper,
IntGrouper
@Nullable
private int[] vAggregationRows = null;
- static long requiredBufferCapacity(
- int cardinality,
- AggregatorFactory[] aggregatorFactories
- )
+ /**
+ * Computes required buffer capacity for a grouping key of the given
cardinaltiy and aggregatorFactories.
+ * This method assumes that the given cardinality doesn't count nulls.
+ *
+ * Returns -1 if cardinality + 1 (for null) > Integer.MAX_VALUE. Returns
computed required buffer capacity
+ * otherwise.
+ */
+ static long requiredBufferCapacity(int cardinality, AggregatorFactory[]
aggregatorFactories)
{
final long cardinalityWithMissingValue =
computeCardinalityWithMissingValue(cardinality);
+// Cardinality should be in the integer range. See
DimensionDictionarySelector.
+if (cardinalityWithMissingValue > Integer.MAX_VALUE) {
+ return -1;
+}
final long recordSize = Arrays.stream(aggregatorFactories)
.mapToLong(AggregatorFactory::getMaxIntermediateSizeWithNulls)
.sum();
diff --git
a/processing/src/main/java/org/apache/druid/query/groupby/epinephelinae/GroupByQueryEngineV2.java
b/processing/src/main/java/org/apache/druid/query/groupby/epinephelinae/GroupByQueryEngineV2.java
index fd3213f..8e3fe05 100644
---
a/processing/src/main/java/org/apache/druid/query/groupby/epinephelinae/GroupByQueryEngineV2.java
+++
b/processing/src/main/java/org/apache/druid/query/groupby/epinephelinae/GroupByQueryEngineV2.java
@@ -300,7 +300,11 @@ public class GroupByQueryEngineV2
);
// Check that all keys and aggregated values can be contained in the
buffer
- return requiredBufferCapacity <= buffer.capacity() ? cardinality : -1;
+ if (requiredBufferCapacity < 0 || requiredBufferCapacity >
buffer.capacity()) {
+return -1;
+ } else {
+return cardinality;
+ }
} else {
return -1;
}
diff --git
a/processing/src/test/java/org/apache/druid/query/groupby/epinephelinae/BufferArrayGrouperTest.java
b/processing/src/test/java/org/apache/druid/query/groupby/epinephelinae/BufferArrayGrouperTest.java
index c05d976..d61700a 100644
---
a/processing/src/test/java/org/apache/druid/query/groupby/epinephelinae/BufferArrayGrouperTest.java
+++
b/processing/src/test/java/org/apache/druid/query/groupby/epinephelinae/BufferArrayGrouperTest.java
@@ -110,20 +110,17 @@ public class BufferArrayGrouperTest
final long[] requiredSizes;
if (NullHandling.sqlCompatible()) {
- // We need additional size to store nullability information.
- requiredSizes = new long[]{19, 101, 19595788279L, 19595788288L};
+ // We need additional space to store nulls.
+ requiredSizes = new long[]{19, 101, 19595788279L, -1};
} else {
- requiredSizes = new long[]{17, 90, 17448304632L, 17448304640L};
+ requiredSizes = new long[]{17, 90, 17448304632L, -1};
}
for (int i = 0; i < cardinalityArray.length; i++) {
Assert.assertEquals(
StringUtils.format("cardinality[%d]", cardinalityArray[i]),
requiredSizes[i],
- BufferArrayGrouper.requiredBufferCapacity(
- cardinalityArray[i],
- aggregatorFactories
- )
+ BufferArrayGrouper.requiredBufferCapacity(cardinalityArray[i],
aggregatorFactories)
);
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional
[druid] branch master updated: fix flaky jetty test (#9633)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new 7bf2dfa fix flaky jetty test (#9633)
7bf2dfa is described below
commit 7bf2dfa3b1a1bb324cc6cb802964ddb13d188d2d
Author: Clint Wylie
AuthorDate: Wed Apr 8 10:07:06 2020 -0700
fix flaky jetty test (#9633)
---
.../java/org/apache/druid/server/initialization/JettyTest.java | 10 +-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git
a/server/src/test/java/org/apache/druid/server/initialization/JettyTest.java
b/server/src/test/java/org/apache/druid/server/initialization/JettyTest.java
index 64ccd14..206c318 100644
--- a/server/src/test/java/org/apache/druid/server/initialization/JettyTest.java
+++ b/server/src/test/java/org/apache/druid/server/initialization/JettyTest.java
@@ -78,6 +78,7 @@ import java.util.Map;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;
+import java.util.concurrent.ThreadLocalRandom;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicLong;
import java.util.zip.GZIPInputStream;
@@ -186,6 +187,8 @@ public class JettyTest extends BaseJettyTest
throw new RuntimeException(e);
}
+final int ephemeralPort = ThreadLocalRandom.current().nextInt(49152,
65535);
+
latchedRequestState = new LatchedRequestStateHolder();
injector = Initialization.makeInjectorWithModules(
GuiceInjectors.makeStartupInjector(),
@@ -198,7 +201,7 @@ public class JettyTest extends BaseJettyTest
JsonConfigProvider.bindInstance(
binder,
Key.get(DruidNode.class, Self.class),
-new DruidNode("test", "localhost", false, 9988, ,
true, true)
+new DruidNode("test", "localhost", false, ephemeralPort,
ephemeralPort + 1, true, true)
);
binder.bind(TLSServerConfig.class).toInstance(tlsConfig);
binder.bind(JettyServerInitializer.class).to(JettyServerInit.class).in(LazySingleton.class);
@@ -450,6 +453,7 @@ public class JettyTest extends BaseJettyTest
JettyServerModule jsm = injector.getInstance(JettyServerModule.class);
latchedRequestState.reset();
+waitForJettyServerModuleActiveConnectionsZero(jsm);
Assert.assertEquals(0, jsm.getActiveConnections());
ListenableFuture go = client.go(
@@ -489,6 +493,7 @@ public class JettyTest extends BaseJettyTest
JettyServerModule jsm = injector.getInstance(JettyServerModule.class);
latchedRequestState.reset();
+waitForJettyServerModuleActiveConnectionsZero(jsm);
Assert.assertEquals(0, jsm.getActiveConnections());
ListenableFuture go = client.go(
request,
@@ -511,5 +516,8 @@ public class JettyTest extends BaseJettyTest
while (jsm.getActiveConnections() > 0 && count++ < totalSleeps) {
Thread.sleep(sleepTimeMills);
}
+if (jsm.getActiveConnections() > 0) {
+ throw new RuntimeException("Connections greater than 0");
+}
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch 0.18.0 updated: Fix NPE in RemoteTaskRunner event handler causes JVM shutdown (#9610) (#9641)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch 0.18.0
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/0.18.0 by this push:
new 325f299 Fix NPE in RemoteTaskRunner event handler causes JVM shutdown
(#9610) (#9641)
325f299 is described below
commit 325f29977740b1ff446f01b98179e377760e47f4
Author: Maytas Monsereenusorn <52679095+mayt...@users.noreply.github.com>
AuthorDate: Tue Apr 7 16:58:27 2020 -1000
Fix NPE in RemoteTaskRunner event handler causes JVM shutdown (#9610)
(#9641)
* Fix NPE in RemoteTaskRunner event handler causes JVM shutdown
* address comments
* fix compile
* fix checkstyle
* fix lgtm
* fix merge
* fix test
* fix tests
* change scope
* address comments
* address comments
---
extensions-contrib/ambari-metrics-emitter/pom.xml | 12 ++
.../druid/indexing/overlord/RemoteTaskRunner.java | 232 -
.../indexing/overlord/RemoteTaskRunnerTest.java| 36
licenses.yaml | 2 +-
pom.xml| 2 +-
.../druid/curator/discovery/DiscoveryModule.java | 12 ++
6 files changed, 191 insertions(+), 105 deletions(-)
diff --git a/extensions-contrib/ambari-metrics-emitter/pom.xml
b/extensions-contrib/ambari-metrics-emitter/pom.xml
index 7cd69ea..fe94d09 100644
--- a/extensions-contrib/ambari-metrics-emitter/pom.xml
+++ b/extensions-contrib/ambari-metrics-emitter/pom.xml
@@ -131,6 +131,18 @@
JUnitParams
test
+
+ org.codehaus.jackson
+ jackson-core-asl
+ ${codehaus.jackson.version}
+ test
+
+
+ org.codehaus.jackson
+ jackson-mapper-asl
+ ${codehaus.jackson.version}
+ test
+
diff --git
a/indexing-service/src/main/java/org/apache/druid/indexing/overlord/RemoteTaskRunner.java
b/indexing-service/src/main/java/org/apache/druid/indexing/overlord/RemoteTaskRunner.java
index fe6e8be..dbaadf9 100644
---
a/indexing-service/src/main/java/org/apache/druid/indexing/overlord/RemoteTaskRunner.java
+++
b/indexing-service/src/main/java/org/apache/druid/indexing/overlord/RemoteTaskRunner.java
@@ -45,6 +45,7 @@ import com.google.common.util.concurrent.SettableFuture;
import org.apache.commons.lang.mutable.MutableInt;
import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.recipes.cache.PathChildrenCache;
+import org.apache.curator.framework.recipes.cache.PathChildrenCacheListener;
import org.apache.curator.utils.ZKPaths;
import org.apache.druid.concurrent.LifecycleLock;
import org.apache.druid.curator.CuratorUtils;
@@ -969,116 +970,141 @@ public class RemoteTaskRunner implements
WorkerTaskRunner, TaskLogStreamer
);
// Add status listener to the watcher for status changes
- zkWorker.addListener(
- (client, event) -> {
-final String taskId;
-final RemoteTaskRunnerWorkItem taskRunnerWorkItem;
-synchronized (statusLock) {
- try {
-switch (event.getType()) {
- case CHILD_ADDED:
- case CHILD_UPDATED:
-taskId =
ZKPaths.getNodeFromPath(event.getData().getPath());
-final TaskAnnouncement announcement = jsonMapper.readValue(
-event.getData().getData(), TaskAnnouncement.class
-);
-
-log.info(
-"Worker[%s] wrote %s status for task [%s] on [%s]",
-zkWorker.getWorker().getHost(),
-announcement.getTaskStatus().getStatusCode(),
-taskId,
-announcement.getTaskLocation()
-);
-
-// Synchronizing state with ZK
-statusLock.notifyAll();
-
-final RemoteTaskRunnerWorkItem tmp;
-if ((tmp = runningTasks.get(taskId)) != null) {
- taskRunnerWorkItem = tmp;
-} else {
- final RemoteTaskRunnerWorkItem newTaskRunnerWorkItem =
new RemoteTaskRunnerWorkItem(
- taskId,
- announcement.getTaskType(),
- zkWorker.getWorker(),
- TaskLocation.unknown(),
- announcement.getTaskDataSource()
- );
- final RemoteTaskRunnerWorkItem existingItem =
runningTasks.putIfAbsent(
- taskId,
- newTaskRunnerWorkItem
- );
- if (existingItem == null) {
-log.warn(
-"Worker[%s] announced a st
[druid] branch 0.18.0 updated: check paths used for shuffle intermediary data manager get and delete (#9630) (#9640)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch 0.18.0
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/0.18.0 by this push:
new b46d06a check paths used for shuffle intermediary data manager get
and delete (#9630) (#9640)
b46d06a is described below
commit b46d06abc3cc07909b9a52ecb18074937dbcf449
Author: Clint Wylie
AuthorDate: Tue Apr 7 14:33:46 2020 -0700
check paths used for shuffle intermediary data manager get and delete
(#9630) (#9640)
* check paths used for shuffle intermediary data manager get and delete
* add test
* newline
* meh
---
.../java/org/apache/druid/indexer/TaskIdUtils.java | 63
.../org/apache/druid/indexer/TaskIdUtilsTest.java | 111 +
.../druid/indexing/kafka/KafkaConsumerConfigs.java | 4 +-
.../indexing/kafka/supervisor/KafkaSupervisor.java | 4 +-
.../kinesis/supervisor/KinesisSupervisor.java | 4 +-
.../druid/indexing/common/task/AbstractTask.java | 4 +-
.../indexing/common/task/utils/RandomIdUtils.java | 34 ---
.../supervisor/SeekableStreamSupervisor.java | 6 +-
.../indexing/worker/IntermediaryDataManager.java | 3 +
...ermediaryDataManagerManualAddAndDeleteTest.java | 64 +++-
.../tests/indexer/AbstractKafkaIndexerTest.java| 4 +-
server/pom.xml | 6 ++
.../apache/druid/segment/indexing/DataSchema.java | 14 +--
.../druid/segment/indexing/DataSchemaTest.java | 66 ++--
14 files changed, 316 insertions(+), 71 deletions(-)
diff --git a/core/src/main/java/org/apache/druid/indexer/TaskIdUtils.java
b/core/src/main/java/org/apache/druid/indexer/TaskIdUtils.java
new file mode 100644
index 000..a88341b
--- /dev/null
+++ b/core/src/main/java/org/apache/druid/indexer/TaskIdUtils.java
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.indexer;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+import org.apache.druid.java.util.common.StringUtils;
+
+import java.util.concurrent.ThreadLocalRandom;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+public class TaskIdUtils
+{
+ private static final Pattern INVALIDCHARS = Pattern.compile("(?s).*[^\\S
].*");
+
+ public static void validateId(String thingToValidate, String
stringToValidate)
+ {
+Preconditions.checkArgument(
+!Strings.isNullOrEmpty(stringToValidate),
+StringUtils.format("%s cannot be null or empty. Please provide a %s.",
thingToValidate, thingToValidate)
+);
+Preconditions.checkArgument(
+!stringToValidate.startsWith("."),
+StringUtils.format("%s cannot start with the '.' character.",
thingToValidate)
+);
+Preconditions.checkArgument(
+!stringToValidate.contains("/"),
+StringUtils.format("%s cannot contain the '/' character.",
thingToValidate)
+);
+Matcher m = INVALIDCHARS.matcher(stringToValidate);
+Preconditions.checkArgument(
+!m.matches(),
+StringUtils.format("%s cannot contain whitespace character except
space.", thingToValidate)
+);
+ }
+
+ public static String getRandomId()
+ {
+final StringBuilder suffix = new StringBuilder(8);
+for (int i = 0; i < Integer.BYTES * 2; ++i) {
+ suffix.append((char) ('a' + ((ThreadLocalRandom.current().nextInt() >>>
(i * 4)) & 0x0F)));
+}
+return suffix.toString();
+ }
+}
diff --git a/core/src/test/java/org/apache/druid/indexer/TaskIdUtilsTest.java
b/core/src/test/java/org/apache/druid/indexer/TaskIdUtilsTest.java
new file mode 100644
index 000..5fed8fb
--- /dev/null
+++ b/core/src/test/java/org/apache/druid/indexer/TaskIdUtilsTest.java
@@ -0,0 +1,111 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The AS
[druid] branch 0.18.0 updated: Reuse transformer in stream indexing (#9625) (#9639)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch 0.18.0
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/0.18.0 by this push:
new 4df4eac Reuse transformer in stream indexing (#9625) (#9639)
4df4eac is described below
commit 4df4eacf231501175b61cb830038a1d6ee769601
Author: Jihoon Son
AuthorDate: Tue Apr 7 14:19:30 2020 -0700
Reuse transformer in stream indexing (#9625) (#9639)
* Reuse transformer in stream indexing
* remove unused method
* memoize complied pattern
---
.../druid/data/input/impl/RegexInputFormat.java| 29 +++-
.../apache/druid/data/input/impl/RegexReader.java | 7 +-
.../data/input/impl/RegexInputFormatTest.java | 72 +++
.../SeekableStreamIndexTaskRunner.java | 54 --
.../seekablestream/SettableByteEntityReader.java | 84 ++
.../indexing/seekablestream/StreamChunkParser.java | 81 +
.../druid/segment/transform/TransformSpec.java | 6 --
7 files changed, 283 insertions(+), 50 deletions(-)
diff --git
a/core/src/main/java/org/apache/druid/data/input/impl/RegexInputFormat.java
b/core/src/main/java/org/apache/druid/data/input/impl/RegexInputFormat.java
index 1165516..c3745bc 100644
--- a/core/src/main/java/org/apache/druid/data/input/impl/RegexInputFormat.java
+++ b/core/src/main/java/org/apache/druid/data/input/impl/RegexInputFormat.java
@@ -20,7 +20,10 @@
package org.apache.druid.data.input.impl;
import com.fasterxml.jackson.annotation.JsonCreator;
+import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.annotation.JsonProperty;
+import com.google.common.base.Supplier;
+import com.google.common.base.Suppliers;
import org.apache.druid.data.input.InputEntity;
import org.apache.druid.data.input.InputEntityReader;
import org.apache.druid.data.input.InputFormat;
@@ -29,12 +32,15 @@ import org.apache.druid.data.input.InputRowSchema;
import javax.annotation.Nullable;
import java.io.File;
import java.util.List;
+import java.util.regex.Pattern;
public class RegexInputFormat implements InputFormat
{
private final String pattern;
private final String listDelimiter;
private final List columns;
+ @JsonIgnore
+ private final Supplier compiledPatternSupplier;
@JsonCreator
public RegexInputFormat(
@@ -46,6 +52,27 @@ public class RegexInputFormat implements InputFormat
this.pattern = pattern;
this.listDelimiter = listDelimiter;
this.columns = columns;
+this.compiledPatternSupplier = Suppliers.memoize(() ->
Pattern.compile(pattern));
+ }
+
+ @JsonProperty
+ public String getPattern()
+ {
+return pattern;
+ }
+
+ @Nullable
+ @JsonProperty
+ public String getListDelimiter()
+ {
+return listDelimiter;
+ }
+
+ @Nullable
+ @JsonProperty
+ public List getColumns()
+ {
+return columns;
}
@Override
@@ -57,6 +84,6 @@ public class RegexInputFormat implements InputFormat
@Override
public InputEntityReader createReader(InputRowSchema inputRowSchema,
InputEntity source, File temporaryDirectory)
{
-return new RegexReader(inputRowSchema, source, pattern, listDelimiter,
columns);
+return new RegexReader(inputRowSchema, source, pattern,
compiledPatternSupplier.get(), listDelimiter, columns);
}
}
diff --git
a/core/src/main/java/org/apache/druid/data/input/impl/RegexReader.java
b/core/src/main/java/org/apache/druid/data/input/impl/RegexReader.java
index 2962ebd..e29e1df 100644
--- a/core/src/main/java/org/apache/druid/data/input/impl/RegexReader.java
+++ b/core/src/main/java/org/apache/druid/data/input/impl/RegexReader.java
@@ -42,7 +42,7 @@ import java.util.regex.Pattern;
public class RegexReader extends TextReader
{
private final String pattern;
- private final Pattern compiled;
+ private final Pattern compiledPattern;
private final Function multiValueFunction;
private List columns;
@@ -51,13 +51,14 @@ public class RegexReader extends TextReader
InputRowSchema inputRowSchema,
InputEntity source,
String pattern,
+ Pattern compiledPattern,
@Nullable String listDelimiter,
@Nullable List columns
)
{
super(inputRowSchema, source);
this.pattern = pattern;
-this.compiled = Pattern.compile(pattern);
+this.compiledPattern = compiledPattern;
final String finalListDelimeter = listDelimiter == null ?
Parsers.DEFAULT_LIST_DELIMITER : listDelimiter;
this.multiValueFunction =
ParserUtils.getMultiValueFunction(finalListDelimeter,
Splitter.on(finalListDelimeter));
this.columns = columns;
@@ -78,7 +79,7 @@ public class RegexReader extends TextReader
private Map parseLine(String line)
{
try {
- final Matcher matcher = compiled.matcher(line);
+ final Matcher matcher = compiledPattern.matcher(line);
if (!matcher.matc
[druid] branch 0.18.0 updated: pac4j: be noop if a previous authenticator in chain has successfully authenticated (#9620) (#9627)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch 0.18.0
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/0.18.0 by this push:
new bc472e8 pac4j: be noop if a previous authenticator in chain has
successfully authenticated (#9620) (#9627)
bc472e8 is described below
commit bc472e85f57216902cd598d4a158f7196c4d254c
Author: Himanshu
AuthorDate: Mon Apr 6 21:10:58 2020 -0700
pac4j: be noop if a previous authenticator in chain has successfully
authenticated (#9620) (#9627)
---
.../src/main/java/org/apache/druid/security/pac4j/Pac4jFilter.java | 7 +++
1 file changed, 7 insertions(+)
diff --git
a/extensions-core/druid-pac4j/src/main/java/org/apache/druid/security/pac4j/Pac4jFilter.java
b/extensions-core/druid-pac4j/src/main/java/org/apache/druid/security/pac4j/Pac4jFilter.java
index 7a9eff7..4463e43 100644
---
a/extensions-core/druid-pac4j/src/main/java/org/apache/druid/security/pac4j/Pac4jFilter.java
+++
b/extensions-core/druid-pac4j/src/main/java/org/apache/druid/security/pac4j/Pac4jFilter.java
@@ -79,6 +79,13 @@ public class Pac4jFilter implements Filter
public void doFilter(ServletRequest servletRequest, ServletResponse
servletResponse, FilterChain filterChain)
throws IOException, ServletException
{
+// If there's already an auth result, then we have authenticated already,
skip this or else caller
+// could get HTTP redirect even if one of the druid authenticators in
chain has successfully authenticated.
+if (servletRequest.getAttribute(AuthConfig.DRUID_AUTHENTICATION_RESULT) !=
null) {
+ filterChain.doFilter(servletRequest, servletResponse);
+ return;
+}
+
HttpServletRequest httpServletRequest = (HttpServletRequest)
servletRequest;
HttpServletResponse httpServletResponse = (HttpServletResponse)
servletResponse;
J2EContext context = new J2EContext(httpServletRequest,
httpServletResponse, sessionStore);
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch 0.18.0 updated: Eliminate common subfilters when converting it to a CNF (#9608) (#9626)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch 0.18.0
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/0.18.0 by this push:
new 908462c Eliminate common subfilters when converting it to a CNF
(#9608) (#9626)
908462c is described below
commit 908462c6a36cecd1c294af7276fe133952e198b4
Author: Jihoon Son
AuthorDate: Mon Apr 6 21:10:30 2020 -0700
Eliminate common subfilters when converting it to a CNF (#9608) (#9626)
---
.../druid/benchmark/FilterPartitionBenchmark.java | 4 +-
.../apache/druid/query/filter/BooleanFilter.java | 5 +-
.../apache/druid/query/filter/DimFilterUtils.java | 4 +-
.../apache/druid/query/filter/TrueDimFilter.java | 8 +-
.../segment/QueryableIndexStorageAdapter.java | 9 +-
.../org/apache/druid/segment/filter/AndFilter.java | 33 ++--
.../org/apache/druid/segment/filter/Filters.java | 56 +++---
.../org/apache/druid/segment/filter/NotFilter.java | 32 +++-
.../org/apache/druid/segment/filter/OrFilter.java | 28 ++-
.../apache/druid/segment/filter/TrueFilter.java| 9 +-
.../segment/join/filter/JoinFilterAnalyzer.java| 8 +-
.../druid/query/filter/AndDimFilterTest.java | 29 ++-
...dDimFilterTest.java => DimFilterTestUtils.java} | 34 ++--
.../apache/druid/query/filter/OrDimFilterTest.java | 53 +
.../druid/segment/filter/BaseFilterTest.java | 2 +-
.../druid/segment/filter/FilterPartitionTest.java | 4 +-
.../filter/FilterTestUtils.java} | 38 ++--
.../apache/druid/segment/filter/FiltersTest.java | 213 -
...tFilterTest.java => NotFilterEvaluateTest.java} | 6 +-
.../apache/druid/segment/filter/NotFilterTest.java | 84 ++--
20 files changed, 473 insertions(+), 186 deletions(-)
diff --git
a/benchmarks/src/test/java/org/apache/druid/benchmark/FilterPartitionBenchmark.java
b/benchmarks/src/test/java/org/apache/druid/benchmark/FilterPartitionBenchmark.java
index b52cde1..547784e 100644
---
a/benchmarks/src/test/java/org/apache/druid/benchmark/FilterPartitionBenchmark.java
+++
b/benchmarks/src/test/java/org/apache/druid/benchmark/FilterPartitionBenchmark.java
@@ -377,7 +377,7 @@ public class FilterPartitionBenchmark
Filter orFilter = new OrFilter(Arrays.asList(filter, filter2));
StorageAdapter sa = new QueryableIndexStorageAdapter(qIndex);
-Sequence cursors = makeCursors(sa, Filters.convertToCNF(orFilter));
+Sequence cursors = makeCursors(sa, Filters.toCNF(orFilter));
readCursors(cursors, blackhole);
}
@@ -451,7 +451,7 @@ public class FilterPartitionBenchmark
);
StorageAdapter sa = new QueryableIndexStorageAdapter(qIndex);
-Sequence cursors = makeCursors(sa,
Filters.convertToCNF(dimFilter3.toFilter()));
+Sequence cursors = makeCursors(sa,
Filters.toCNF(dimFilter3.toFilter()));
readCursors(cursors, blackhole);
}
diff --git
a/processing/src/main/java/org/apache/druid/query/filter/BooleanFilter.java
b/processing/src/main/java/org/apache/druid/query/filter/BooleanFilter.java
index 9bbbdb6..e11153e 100644
--- a/processing/src/main/java/org/apache/druid/query/filter/BooleanFilter.java
+++ b/processing/src/main/java/org/apache/druid/query/filter/BooleanFilter.java
@@ -23,12 +23,13 @@ import org.apache.druid.segment.ColumnSelector;
import org.apache.druid.segment.ColumnSelectorFactory;
import java.util.HashSet;
-import java.util.List;
import java.util.Set;
public interface BooleanFilter extends Filter
{
- List getFilters();
+ ValueMatcher[] EMPTY_VALUE_MATCHER_ARRAY = new ValueMatcher[0];
+
+ Set getFilters();
/**
* Get a ValueMatcher that applies this filter to row values.
diff --git
a/processing/src/main/java/org/apache/druid/query/filter/DimFilterUtils.java
b/processing/src/main/java/org/apache/druid/query/filter/DimFilterUtils.java
index 00a84fc..c980adb 100644
--- a/processing/src/main/java/org/apache/druid/query/filter/DimFilterUtils.java
+++ b/processing/src/main/java/org/apache/druid/query/filter/DimFilterUtils.java
@@ -52,9 +52,9 @@ public class DimFilterUtils
static final byte COLUMN_COMPARISON_CACHE_ID = 0xD;
static final byte EXPRESSION_CACHE_ID = 0xE;
static final byte TRUE_CACHE_ID = 0xF;
- public static byte BLOOM_DIM_FILTER_CACHE_ID = 0x10;
- public static final byte STRING_SEPARATOR = (byte) 0xFF;
+ public static final byte BLOOM_DIM_FILTER_CACHE_ID = 0x10;
+ public static final byte STRING_SEPARATOR = (byte) 0xFF;
static byte[] computeCacheKey(byte cacheIdKey, List filters)
{
diff --git
a/processing/src/main/java/org/apache/druid/query/filter/TrueDimFilter.java
b/processing/src/main/java/org/apache/druid/query/filter/TrueDimFilter.java
index d10e6d9..2254358 100644
--- a/processing/src/main/java/org/apache/druid/query/filter/TrueDimFilter.java
+++ b/processing/src/main/java/org/apache/druid/query/filter/TrueDimFilte
[druid-website-src] branch master updated (f81a5d4 -> e162e18)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git. from f81a5d4 Merge pull request #116 from apache/0.17.1-docs add f0641ee Add to featured content new e162e18 Merge pull request #118 from apache/add-featured-content The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: _data/featured.yml | 12 1 file changed, 12 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] 01/01: Merge pull request #118 from apache/add-featured-content
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git commit e162e18e6831a6c82e4d80004ca446451c387bd6 Merge: f81a5d4 f0641ee Author: Fangjin Yang AuthorDate: Fri Apr 3 14:44:45 2020 -0700 Merge pull request #118 from apache/add-featured-content update featured content _data/featured.yml | 12 1 file changed, 12 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated: Fix HDFS input source split (#9574)
This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new c0195a1 Fix HDFS input source split (#9574)
c0195a1 is described below
commit c0195a19e4e9bef50dd1bb5f1b63d03b4f3e318f
Author: Chi Cao Minh
AuthorDate: Sat Mar 28 15:45:57 2020 -0700
Fix HDFS input source split (#9574)
Fixes an issue where splitting an HDFS input source for use in native
parallel batch ingestion would cause the subtasks to get a split with an
invalid HDFS path.
---
.../apache/druid/inputsource/hdfs/HdfsInputSource.java | 7 +--
.../druid/inputsource/hdfs/HdfsInputSourceTest.java | 16
2 files changed, 21 insertions(+), 2 deletions(-)
diff --git
a/extensions-core/hdfs-storage/src/main/java/org/apache/druid/inputsource/hdfs/HdfsInputSource.java
b/extensions-core/hdfs-storage/src/main/java/org/apache/druid/inputsource/hdfs/HdfsInputSource.java
index be7f3c8..b8c798a 100644
---
a/extensions-core/hdfs-storage/src/main/java/org/apache/druid/inputsource/hdfs/HdfsInputSource.java
+++
b/extensions-core/hdfs-storage/src/main/java/org/apache/druid/inputsource/hdfs/HdfsInputSource.java
@@ -22,6 +22,7 @@ package org.apache.druid.inputsource.hdfs;
import com.fasterxml.jackson.annotation.JacksonInject;
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;
+import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Preconditions;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.Iterators;
@@ -142,8 +143,9 @@ public class HdfsInputSource extends AbstractInputSource
implements SplittableIn
}
}
+ @VisibleForTesting
@JsonProperty(PROP_PATHS)
- private List getInputPaths()
+ List getInputPaths()
{
return inputPaths;
}
@@ -199,7 +201,8 @@ public class HdfsInputSource extends AbstractInputSource
implements SplittableIn
@Override
public SplittableInputSource> withSplit(InputSplit>
split)
{
-return new HdfsInputSource(split.get().toString(), configuration);
+List paths = split.get().stream().map(path ->
path.toString()).collect(Collectors.toList());
+return new HdfsInputSource(paths, configuration);
}
@Override
diff --git
a/extensions-core/hdfs-storage/src/test/java/org/apache/druid/inputsource/hdfs/HdfsInputSourceTest.java
b/extensions-core/hdfs-storage/src/test/java/org/apache/druid/inputsource/hdfs/HdfsInputSourceTest.java
index d2c7820..044930f 100644
---
a/extensions-core/hdfs-storage/src/test/java/org/apache/druid/inputsource/hdfs/HdfsInputSourceTest.java
+++
b/extensions-core/hdfs-storage/src/test/java/org/apache/druid/inputsource/hdfs/HdfsInputSourceTest.java
@@ -22,6 +22,7 @@ package org.apache.druid.inputsource.hdfs;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.databind.InjectableValues;
import com.fasterxml.jackson.databind.ObjectMapper;
+import com.google.common.collect.Iterables;
import org.apache.druid.data.input.InputFormat;
import org.apache.druid.data.input.InputRow;
import org.apache.druid.data.input.InputRowSchema;
@@ -277,6 +278,21 @@ public class HdfsInputSourceTest extends
InitializedNullHandlingTest
int numSplits = target.estimateNumSplits(null, new
MaxSizeSplitHintSpec(1L));
Assert.assertEquals(NUM_FILE, numSplits);
}
+
+@Test
+public void createCorrectInputSourceWithSplit() throws Exception
+{
+ // Set maxSplitSize to 1 so that each inputSplit has only one object
+ List>> splits = target.createSplits(null, new
MaxSizeSplitHintSpec(1L))
+
.collect(Collectors.toList());
+
+ for (InputSplit> split : splits) {
+String expectedPath = Iterables.getOnlyElement(split.get()).toString();
+HdfsInputSource inputSource = (HdfsInputSource)
target.withSplit(split);
+String actualPath =
Iterables.getOnlyElement(inputSource.getInputPaths());
+Assert.assertEquals(expectedPath, actualPath);
+ }
+}
}
public static class EmptyPathsTest
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: Update Added Deep.BI
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new 1e882cf Update Added Deep.BI new b977522 Merge pull request #115 from hishamitani/patch-1 1e882cf is described below commit 1e882cf8d22e1f1c800469f63cb94a8d9f8d8db6 Author: hishamitani <47254768+hishamit...@users.noreply.github.com> AuthorDate: Wed Mar 25 05:13:51 2020 +0100 Update Added Deep.BI Hello, thank you for your consideration! The phrasing I have put in there might have been better in terms of making it clear but the main point is that our platform is built on Druid (on which our UI platform is built as well) as the core with others such as Flink and Cassandra that allow us to augment the way we use Druid. Thanks in advance! --- druid-powered.md | 8 1 file changed, 8 insertions(+) diff --git a/druid-powered.md b/druid-powered.md index e3bdade..eec24d8 100644 --- a/druid-powered.md +++ b/druid-powered.md @@ -101,6 +101,14 @@ We are co-authors of [Turnilo](https://allegro.tech/2018/10/turnilo-lets-change- Our analytics platform [Storyteller](https://datumo.io/#product) uses power of Apache Druid and Turnilo to help our clients analyze large datasets in interactive manner. Our biggest Druid deployment is responsible for real-time monitoring and data processing of a large video streaming platform (60 000 Events Per Second). +## Deep.BI + +At [Deep.BI](www.deep.bi) we track user habits, engagement, product and content performance — processing up to terabytes or billions of events of data daily. Our goal is to provide real-time insights based on custom metrics from a variety of self-created dimensions. To accomplish this, our system collects every user interaction. We use Apache Flink for event enrichment, custom transformations, aggregations and serving machine learning models. The processed data is then indexed by Apache [...] + +Our talk from FlinkForward conference in Berlin 2019: + +[Real-time Stream Analytics and User Scoring Using Apache Druid, Flink & Cassandra at Deep.BI] (https://medium.com/deep-bi/real-time-stream-analytics-and-user-scoring-using-apache-druid-flink-cassandra-at-deep-bi-38e5dc40958d?source=friends_link=d9c85da0cb060280bfeb8f3c1c166e84) + ## Delta Projects [Delta Projects](https://www.deltaprojects.com) uses Druid to power real-time audience and advertising analytics interfaces. - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: Update events.yml
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new 2790662 Update events.yml new de37960 Merge pull request #114 from rachelpedreschi/patch-9 2790662 is described below commit 2790662feafe0a5e8717bab990b88ccee411c661 Author: Rachel Pedreschi <53063158+rachelpedres...@users.noreply.github.com> AuthorDate: Tue Mar 24 10:18:30 2020 -0700 Update events.yml --- _data/events.yml | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/_data/events.yml b/_data/events.yml index 5715c09..f3bfffe 100644 --- a/_data/events.yml +++ b/_data/events.yml @@ -1,9 +1,9 @@ -- date: 2020-03-04 - name: "Analytics at speed - the rise of the Druid" - info: "Castlereagh Boutique Hotel, 169 Castlereagh St, Sydney" - link: https://www.meetup.com/Big-Data-Analytics/events/268442945/ +- date: 2020-03-24 + name: "Virtual Meetup- AGA (*Ask Gian Anything*)" + info: "Hosted by the Bay Area Apache Druid Meetup Group" + link: https://www.meetup.com/druidio/events/269554006/ - date: 2020-04-13 - name: "Druid Summit" - info: "San Francisco Airport Marriott Waterfront, 1800 Old Bayshore Highway, Burlingame, CA" + name: "Virtual Druid Summit" + info: "The comfort of your own laptop" link: https://druidsummit.org/ - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated (3082b92 -> ff59d2e)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from 3082b92 Fix NPE when using IndexedTable and all left rows are filtered out (#9490) add ff59d2e Move RowSignature from druid-sql to druid-processing and make use of it. (#9508) No new revisions were added by this update. Summary of changes: .../movingaverage/AveragerFactoryWrapper.java | 7 +- .../sql/TDigestGenerateSketchSqlAggregator.java| 2 +- .../sql/TDigestSketchQuantileSqlAggregator.java| 2 +- .../TimestampGroupByAggregationTest.java | 4 +- .../hll/sql/HllSketchBaseSqlAggregator.java| 4 +- .../sql/HllSketchEstimateOperatorConversion.java | 2 +- ...hEstimateWithErrorBoundsOperatorConversion.java | 2 +- .../sql/HllSketchSetUnionOperatorConversion.java | 2 +- .../sql/HllSketchToStringOperatorConversion.java | 2 +- .../DoublesSketchApproxQuantileSqlAggregator.java | 2 +- ...DoublesSketchListArgBaseOperatorConversion.java | 2 +- .../sql/DoublesSketchObjectSqlAggregator.java | 2 +- ...ublesSketchSingleArgBaseOperatorConversion.java | 2 +- .../DoublesSketchSummaryOperatorConversion.java| 2 +- .../theta/sql/ThetaSketchBaseSqlAggregator.java| 5 +- .../sql/ThetaSketchEstimateOperatorConversion.java | 2 +- ...hEstimateWithErrorBoundsOperatorConversion.java | 2 +- .../sql/ThetaSketchSetBaseOperatorConversion.java | 2 +- .../bloom/sql/BloomFilterSqlAggregator.java| 2 +- .../filter/sql/BloomFilterOperatorConversion.java | 2 +- ...FixedBucketsHistogramQuantileSqlAggregator.java | 2 +- .../histogram/sql/QuantileSqlAggregator.java | 4 +- .../variance/sql/BaseVarianceSqlAggregator.java| 2 +- .../org/apache/druid/query/InlineDataSource.java | 14 +- .../org/apache/druid/query/QueryToolChest.java | 11 +- .../druid/query/aggregation/AggregatorFactory.java | 20 +- .../apache/druid/query/groupby/GroupByQuery.java | 65 ++--- .../druid/query/groupby/GroupByQueryHelper.java| 39 --- .../query/groupby/GroupByQueryQueryToolChest.java | 5 +- .../org/apache/druid/query/groupby/ResultRow.java | 8 +- .../epinephelinae/RowBasedGrouperHelper.java | 7 +- .../query/groupby/having/DimFilterHavingSpec.java | 2 +- .../having/DimensionSelectorHavingSpec.java| 2 +- .../query/groupby/having/EqualToHavingSpec.java| 2 +- .../groupby/having/GreaterThanHavingSpec.java | 2 +- .../query/groupby/having/LessThanHavingSpec.java | 2 +- .../query/groupby/orderby/DefaultLimitSpec.java| 8 +- .../query/groupby/strategy/GroupByStrategyV2.java | 2 +- .../apache/druid/query/lookup/LookupSegment.java | 11 +- .../druid/query/scan/ScanQueryQueryToolChest.java | 39 ++- .../timeseries/TimeseriesQueryQueryToolChest.java | 23 +- .../druid/query/topn/TopNQueryQueryToolChest.java | 24 +- .../segment/RowBasedColumnSelectorFactory.java | 22 +- .../org/apache/druid/segment/RowBasedCursor.java | 5 +- .../org/apache/druid/segment/RowBasedSegment.java | 5 +- .../druid/segment/RowBasedStorageAdapter.java | 9 +- .../apache/druid/segment/column/RowSignature.java | 283 + .../org/apache/druid/segment/column/ValueType.java | 3 +- .../segment/incremental/IncrementalIndex.java | 3 +- .../druid/segment/join/table/IndexedTable.java | 19 +- .../table/IndexedTableColumnSelectorFactory.java | 6 +- .../join/table/IndexedTableJoinMatcher.java| 9 +- .../segment/join/table/IndexedTableJoinable.java | 8 +- .../segment/join/table/RowBasedIndexedTable.java | 42 ++- .../druid/segment/transform/Transformer.java | 3 +- .../apache/druid/query/InlineDataSourceTest.java | 13 +- .../groupby/GroupByQueryQueryToolChestTest.java| 26 +- .../query/groupby/GroupByQueryRunnerTest.java | 8 +- .../groupby/GroupByQueryRunnerTestHelper.java | 4 +- .../druid/query/groupby/GroupByQueryTest.java | 3 +- .../query/scan/ScanQueryQueryToolChestTest.java| 23 +- .../TimeseriesQueryQueryToolChestTest.java | 22 +- .../query/topn/TopNQueryQueryToolChestTest.java| 14 +- .../org/apache/druid/segment/IndexBuilder.java | 24 +- .../druid/segment/RowBasedStorageAdapterTest.java | 33 ++- .../druid/segment/filter/BaseFilterTest.java | 9 +- .../join/HashJoinSegmentStorageAdapterTest.java| 4 +- .../apache/druid/segment/join/JoinTestHelper.java | 31 +-- .../druid/segment/join/JoinableClauseTest.java | 2 +- .../join/table/IndexedTableJoinableTest.java | 2 +- .../join/table/RowBasedIndexedTableTest.java | 31 +-- .../virtual/ExpressionVirtualColumnTest.java | 9 +- .../druid/client/CachingClusteredClientTest.java | 2 +- .../segment/join/InlineJoinableFactoryTest.java| 2 +- .../druid/sql/calcite
[druid-website-src] branch master updated: Adding a new site to the list
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new 3f0f596 Adding a new site to the list new d6627cb Merge pull request #113 from AlexNodex/patch-1 3f0f596 is described below commit 3f0f596a1a24ef1e0c5380b018d94586ee14f7ac Author: AlexNodex AuthorDate: Wed Mar 11 09:41:41 2020 + Adding a new site to the list Added Nodex to the "Powered by" list --- druid-powered.md | 4 1 file changed, 4 insertions(+) diff --git a/druid-powered.md b/druid-powered.md index 80ec39d..e3bdade 100644 --- a/druid-powered.md +++ b/druid-powered.md @@ -297,6 +297,10 @@ Nielsen Marketing Cloud uses Druid as it's core real-time analytics tool to help * [Counting Unique Users in Real-Time: Here’s a Challenge for You! slides](https://www.slideshare.net/Hadoop_Summit/counting-unique-users-in-realtime-heres-a-challenge-for-you-139142580) * [Data Retention and Deletion in Apache Druid](https://medium.com/nmc-techblog/data-retention-and-deletion-in-apache-druid-74ffd12398a8) +## Nodex + +[Nodex](https://www.nodex.co.uk) uses Druid in its real time analytics pipeline to deliver insights and analytics across a wide range of recruitment CRM software, Job boards and recruitment career portals. We ingest large amounts of data and needed something capable of real time metrics to offer KPI's to our clients to allow them to make better business decisions based on actual usage data. + ## NTT Com * [Kappa architecture at NTT Com: Building a streaming analytics stack with Druid and Kafka](https://imply.io/post/kappa-architecture-at-ntt) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: Update druid-powered.md
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new fd5f7cf Update druid-powered.md new 3344618 Merge pull request #112 from bsyk/patch-1 fd5f7cf is described below commit fd5f7cfc94bd9eed310ed53b2333e73690f95c7d Author: Ben Sykes AuthorDate: Wed Mar 4 15:09:04 2020 -0800 Update druid-powered.md Add link to recent blog post about Netflix's use of Druid for Realtime logs --- druid-powered.md | 1 + 1 file changed, 1 insertion(+) diff --git a/druid-powered.md b/druid-powered.md index a96c0f4..80ec39d 100644 --- a/druid-powered.md +++ b/druid-powered.md @@ -278,6 +278,7 @@ activity and content flow. * [Druid Meetup @ Netflix slides](https://www.slideshare.net/JaebinYoon/druid-meetup-netflix-11142018/) * [Netflix & Amazon Case Study](https://aws.amazon.com/solutions/case-studies/netflix-kinesis-streams/) * [Announcing Suro: Backbone of Netflix's Data Pipeline](http://techblog.netflix.com/2013/12/announcing-suro-backbone-of-netflixs.html) +* [How Netflix uses Druid for Real-time Insights to Ensure a High-Quality Experience](https://netflixtechblog.com/how-netflix-uses-druid-for-real-time-insights-to-ensure-a-high-quality-experience-19e1e8568d06) ## Netsil - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: Added FacilityConneX to druid-powered.md
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new e544041 Added FacilityConneX to druid-powered.md new 59a7e73 Merge pull request #111 from jpritchard-fcx/patch-1 e544041 is described below commit e544041d9e32cddd70ef5ed4298d746123b02117 Author: Jake Pritchard AuthorDate: Tue Mar 3 10:56:46 2020 -0500 Added FacilityConneX to druid-powered.md --- druid-powered.md | 4 1 file changed, 4 insertions(+) diff --git a/druid-powered.md b/druid-powered.md index 938b8bb..a96c0f4 100644 --- a/druid-powered.md +++ b/druid-powered.md @@ -126,6 +126,10 @@ eBay uses Druid to aggregate multiple data streams for real-time user behavior a * [Druid and Pulsar](http://blog.csdn.net/ebay/article/details/50205611) * [embedded-druid: Leveraging Druid Capabilities in Stand-alone Applications](http://www.ebaytechblog.com/2016/02/05/embedded-druid-leveraging-druid-capabilities-in-stand-alone-applications/) +## FacilityConneX + +One of our key goals at [FacilityConneX](https://www.facilityconnex.com/) is to offer real-time insights that help our customers optimize their equipment or processes, reduce cost, or prevent incidents, to ultimately improve our customers business. This *real-time* requirement has always been a major technical challenge to scale within our SaaS environment. FacilityConneX has looked to **Druid** to help solve many of these challenging performance and growth issues. + ## FullContact * [Enrich API Brings Higher Match Rates with Multi-Field Enrichment Capabilities](https://www.fullcontact.com/blog/enrich-api-brings-higher-match-rates-with-multi-field-enrichment-capabilities/) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated (7fc99ee -> f619903)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from 7fc99ee Add common optional dependencies for extensions (#9399) add f619903 Updated the configuration documentation on coordinator kill tasks to clarify whether they delete only unused segments. (#9400) No new revisions were added by this update. Summary of changes: docs/configuration/index.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: Update events.yml
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new ee810ca Update events.yml new 4d9e412 Merge pull request #110 from rachelpedreschi/patch-8 ee810ca is described below commit ee810ca70158acf622ecc0c87d4f91039ea6aebc Author: Rachel Pedreschi <53063158+rachelpedres...@users.noreply.github.com> AuthorDate: Wed Feb 19 15:59:17 2020 -0800 Update events.yml --- _data/events.yml | 4 1 file changed, 4 insertions(+) diff --git a/_data/events.yml b/_data/events.yml index a44b6d8..5715c09 100644 --- a/_data/events.yml +++ b/_data/events.yml @@ -1,3 +1,7 @@ +- date: 2020-03-04 + name: "Analytics at speed - the rise of the Druid" + info: "Castlereagh Boutique Hotel, 169 Castlereagh St, Sydney" + link: https://www.meetup.com/Big-Data-Analytics/events/268442945/ - date: 2020-04-13 name: "Druid Summit" - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated (6d52edd -> b1be88d)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from 6d52edd Remove references to Docker Machine (#9366) add b1be88d fix Expressions.toQueryGranularity to be more correct, improve javadocs of Expr.getIdentifierIfIdentifier and Expr.getBindingIfIdentifier (#9363) No new revisions were added by this update. Summary of changes: core/src/main/java/org/apache/druid/math/expr/Expr.java| 7 +-- .../java/org/apache/druid/sql/calcite/expression/Expressions.java | 2 +- 2 files changed, 6 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated (bcf8f91 -> 4221573)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from bcf8f91 Add unit tests for CoordinatorRuleManager (#9318) add 4221573 Update web-console dependencies (#9361) No new revisions were added by this update. Summary of changes: web-console/package-lock.json | 170 +- web-console/package.json | 4 +- 2 files changed, 39 insertions(+), 135 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated (7968524 -> ebd199d)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from 7968524 Add Pig-specific file handling to Avro parser (#9258) add ebd199d docs: extensions-core: mysql: fix MySQL connector library Maven Central URL (#9344) No new revisions were added by this update. Summary of changes: docs/development/extensions-core/mysql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid-website-src] branch master updated: Add Trendyol to powered list
This is an automated email from the ASF dual-hosted git repository. fjy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid-website-src.git The following commit(s) were added to refs/heads/master by this push: new 2e45bf6 Add Trendyol to powered list new 86fd6a4 Merge pull request #107 from mangrrua/patch-1 2e45bf6 is described below commit 2e45bf6c482cd966146f1d80445b1a937b44dcbf Author: Oğuzhan Mangır AuthorDate: Fri Feb 7 16:14:42 2020 +0300 Add Trendyol to powered list --- druid-powered.md | 4 1 file changed, 4 insertions(+) diff --git a/druid-powered.md b/druid-powered.md index 6ae8522..938b8bb 100644 --- a/druid-powered.md +++ b/druid-powered.md @@ -471,6 +471,10 @@ Learn more about TrafficGuard’s comprehensive fraud mitigation at (https://www * [Using Druid to fight ad fraud](https://imply.io/post/using-druid-to-fight-ad-fraud) +## Trendyol.com - Alibaba Group Company + +Trendyol, which is the largest e-commerce company in Turkey, uses Druid for real-time analytics. It is mostly used for providing insights to their suppliers. + ## TripleLift TripleLift uses Druid to provide insights into performance aspects of its native programmatic exchange for sales/business development opportunities, and to provide reporting used by advertisers and publishers. - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated (53bb45f -> b55657c)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from 53bb45f Forbid easily misused HashSet and HashMap constructors (#9165) add b55657c fix protobuf extension packaging and docs (#9320) No new revisions were added by this update. Summary of changes: docs/development/extensions-core/protobuf.md | 109 --- examples/quickstart/protobuf/pb_publisher.py | 2 + extensions-core/protobuf-extensions/pom.xml | 2 +- 3 files changed, 68 insertions(+), 45 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated (33a97df -> 5c541f5)
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from 33a97df Guicify druid sql module (#9279) add 5c541f5 remove log.info from FixedBucketsHistogramAggregator aggregate method (#9309) No new revisions were added by this update. Summary of changes: .../query/aggregation/histogram/FixedBucketsHistogramAggregator.java | 4 .../druid/query/aggregation/histogram/FixedBucketsHistogramSerde.java | 3 --- 2 files changed, 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
