Hello Joel, This definitely seems like a good feature to add. In fact I was also looking to push data into Solr from Kafka and this can be good feature to have.
I have created JIRA https://issues.apache.org/jira/browse/SOLR-10086 and can contribute as well. Thanks, Susheel On Tue, Jan 31, 2017 at 6:28 PM, John Bickerstaff <j...@johnbickerstaff.com> wrote: > I wrote a simple java microservice that did this about a year ago... It > was pretty simple - and the kafka topic served as a way to re-create my > collection from scratch without hitting the database again in the event of > the Solr servers going down. > > The code just read from Kafka topic one by one and shipped to Solr in > batches of 500 (between commits) > > (It was a small data set, I was lucky that way) > > On Tue, Jan 31, 2017 at 3:41 PM, Joel Bernstein <joels...@gmail.com> > wrote: > > > This would make a great Streaming Expression as well. If you're > interested > > in working on this I'll help out along the way. Here is an example Stream > > that connects to a JDBC data source: > > > > https://github.com/apache/lucene-solr/blob/master/solr/ > > solrj/src/java/org/apache/solr/client/solrj/io/stream/JDBCStream.java > > > > Joel Bernstein > > http://joelsolr.blogspot.com/ > > > > On Tue, Jan 31, 2017 at 12:07 PM, Mike Thomsen <mikerthom...@gmail.com> > > wrote: > > > > > Probably not, but writing your own little Java process to do it would > be > > > trivial with Kafka 0.9.X or 0.10.X. You can also look at the Confluent > > > Platform as they have tons of connectors for Kafka to directly feed > into > > > other systems. > > > > > > On Mon, Jan 30, 2017 at 3:05 AM, Mahmoud Almokadem < > > prog.mahm...@gmail.com > > > > > > > wrote: > > > > > > > Hello, > > > > > > > > Is there a way to get SolrCloud to pull data from a topic in Kafak > > > > periodically using Dataimport Handler? > > > > > > > > Thanks > > > > Mahmoud > > > > > >
