Re: Spark Druid connectors, take 2

For proper disclosure, it's been a while since I used Druid, but here's my 2 cents w.r.t Will's question (based on what I originally wrote in this design doc ): 1. *Spark writes to Druid*: 1. Based on

Re: [DISCUSS] Hadoop 3, dropping support for Hadoop 2.x

Hey Clint, I think it's definitely a step in the right direction. One thing I would suggest, since the are several deployments using Hadoop (either for deep storage and/or for ingestion), is to let the wider community know in advance that Hadoop 2.x support is going to be dropped in favor of 3.x

Re: Propose a scheme for Coordinator to pull metadata incrementally

Hey, I'm not a Druid developer, so it's quite possible I'm missing many considerations here, but from a first glance, I like your offer, as it resembles the *tsColumn *in JDBC lookups ( https://druid.apache.org/docs/latest/development/extensions-core/lookups-cached-global.html#jdbc-lookup ).

Re: Draft April ASF Board Report

ntioned those in a previous report (our last > one > > was just a month ago — so this one covers the last month). After this > > report they'll start being quarterly and covering 3 months. > > > > On Wed, Apr 8, 2020 at 1:18 PM itai yaffe wrote: > > > > > Hey

Re: Draft April ASF Board Report

Hey, Not sure it matters, but we actually had at least 3 in-person meetups during Q1 organized by the Druid community: 1. January 15th in London - https://www.meetup.com/Apache-Druid-London/events/267380924/ 2. January 28th in Athens -

Re: Spark-based ingestion into Druid

Gian’s review. BTW - the doc is currently open for everyone to edit, let me know if you think I should change that. On 2020/03/11 22:33:19, itai yaffe wrote: > Hey Rajiv, > Can you please provide some details on the use-case of querying Druid from > Spark (e.g what type of queries

Re: Spark-based ingestion into Druid

On 2020/03/05 20:19:38, Julian Jaffe > wrote: > > Yeah, I think the primary objective here is a standalone writer from > Spark > > to Druid. > > > > On Thu, Mar 5, 2020 at 11:43 AM itai yaffe > wrote: > > > > > Thanks Julian! >

Re: Spark-based ingestion into Druid

egmentMetadataQueries to get location info). > > On Thu, Mar 5, 2020 at 8:04 AM itai yaffe wrote: > > > I'll let Julian answer, but in the meantime, I just wanted to point out we > > might be able to draw some inspiration from this Spark-Redshift connector ( > > ht

Re: Spark-based ingestion into Druid

riends will taking time for > that. Julian, can you share your experiences for that? After that, we can > write our aims, requirements and flows easily. > > On 2020/02/26 13:26:13, itai yaffe wrote: > > Hey, > > Per Gian's proposal, and following this thread in Druid user group (

Spark-based ingestion into Druid

Hey, Per Gian's proposal, and following this thread in Druid user group ( https://groups.google.com/forum/#!topic/druid-user/FqAuDGc-rUM) and this thread in Druid Slack channel ( https://the-asf.slack.com/archives/CJ8D1JTB8/p1581452302483600), I'd like to start discussing the options of having