Hi all,

I spoke at the London Kafka meetup last night about searching streaming data 
(write-up here: 
http://www.flax.co.uk/blog/2016/04/14/apache-kafka-london-meetup-real-time-search-insights/),
 and as part of the preparation for the talk I tried porting some Samza code I 
have to the KStreams library.

My first impressions are that kstreams is very nice, particularly when it comes 
to deploying and testing (no need for Yarn, yay!).  I have a couple of 
questions though:

- I couldn't work out to open a stream and read it from the beginning.  My 
usecase here is a cached search engine, where stored queries are run over 
documents in a stream.  Both queries and documents are stored in Kafka topics, 
and when the processor starts it needs to read the query topic from the 
beginning to construct its cache.  StreamBuilder.table() and 
StreamBuilder.stream() seem to create consumers that join the end of topics 
only.

- There doesn't seem to be be a nice way of closing resources on shutdown.  Is 
there a plan to add shutdown hooks, or maybe a KafkaStreams.join() method that 
waits for the internal threads to be interrupted?

My (broken!) code can be found here: 
https://github.com/romseygeek/luwak-kstream/blob/master/src/main/java/com/flaxsearch/luwak_kstreams/StreamMonitor.java

Thanks,

Alan Woodward
www.flax.co.uk


Reply via email to