+1 for removing all the standalone realtime node stuff regarding internal discovery, I really would like to see BatchServerInventoryView , CuratorLoadQueuePeon and RemoteTaskRunner go away at some point which will also remove a lot of code And , as to running cluster without zookeeper I have some thoughts in my mind but haven't gotten around to finishing a non-zookeeper extension just yet.
On Mon, Jun 24, 2019 at 6:08 PM Clint Wylie <cwy...@apache.org> wrote: > +1 from me for getting rid of all of those firehoses too. I kept bumping > into 'RealtimeManager' while scoping out what it would take to do the "In a > later release, remove the dead code used for existing internal > announcements and discovery" part of > https://github.com/apache/incubator-druid/pull/4634, in the hope that > someday we can make the quickstart tutorial run on a single host without > zk. I didn't really want to bother fixing up the realtime node, which is > what motivated me to start this in the first place. > > I will remove these firehose classes in a follow-up to the initial PR I > have raised if everyone else is on board with this. > > On Sun, Jun 23, 2019 at 2:11 PM Gian Merlino <g...@apache.org> wrote: > > > Hey Druids, > > > > Realtime nodes (the original Druid streaming ingestion mechanism, > predating > > both realtime tasks and the Kafka/Kinesis indexing services) have been > > deprecated for years. Recently, Clint opened a PR to remove them: > > https://github.com/apache/incubator-druid/pull/7915 > > > > I think this is a great idea, as they are pretty well obsolete at this > > point. I would also suggest removing firehoses that are only really > useful > > in realtime nodes. (Even though firehoses are still a relevant concept, > > since batch index tasks use them, any 'unbounded' firehoses aren't really > > useful for batch index tasks.) > > > > This includes: > > > > - IrcFirehoseFactory > > - KafkaEightFirehoseFactory > > - RabbitMQFirehoseFactory > > - RocketMQFirehoseFactory > > - TwitterSpritzerFirehoseFactory > > - The FirehoseFactoryV2 and FirehoseV2 interfaces, which are only used by > > Realtime nodes, and all implementations of them > > > > I'm writing this on the dev list, in addition to the PR, for some > > additional visibility since this is a relatively large deletion of code > (it > > would remove at least 3 entire extensions). > > >
