> On April 22, 2015, 5:49 p.m., Naveen Somasundaram wrote: > > samza-elasticsearch/src/main/java/org/apache/samza/system/elasticsearch/ElasticsearchSystemProducer.java, > > line 116 > > <https://reviews.apache.org/r/33297/diff/3/?file=939646#file939646line116> > > > > If the previous send has failed, i.e., sendFailed is true here, > > shouldn't we throw an exception here ? Otherwise it might result in message > > develivered out of order. > > Navina Ramesh wrote: > I think the messages are only buffered and not sent until we invoke flush > (Line 122). Is that right, Dan ? > > Dan Harvey wrote: > Yes the send method puts the message onto a queue inside the > BulkProcessor, if Elasticsearch fails to commit a document in a batch that > will cause flush to raise an exception. There for no later messages will be > sent. I need to double check that the BulkProcessor won't send two batches at > the same time and that batches are processed in order. I've not considered > what happens if you update a document twice in a batch to ensure they get > updated in the correct order. Elasticsearch documents can have version > numbers that help with this.
As mentioned below the BulkProcessor has a concurrent requests option, so if we want to ensure ordering we need to set this to 0. - Dan ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33297/#review81184 ----------------------------------------------------------- On April 22, 2015, 5:12 p.m., Dan Harvey wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/33297/ > ----------------------------------------------------------- > > (Updated April 22, 2015, 5:12 p.m.) > > > Review request for samza. > > > Repository: samza > > > Description > ------- > > [SAMZA-654] Added ElasticsearchSystemProducer and Factory to output messages > into Elasticseach indexes. > > > Diffs > ----- > > build.gradle 97de3a28f6379e3862eec845da87587b1d4f742e > gradle/dependency-versions.gradle ee6dfc411b7ab90b187df79f109884127953862e > > samza-elasticsearch/src/main/java/org/apache/samza/config/ElasticsearchConfig.java > PRE-CREATION > > samza-elasticsearch/src/main/java/org/apache/samza/system/elasticsearch/DefaultIndexRequestFactory.java > PRE-CREATION > > samza-elasticsearch/src/main/java/org/apache/samza/system/elasticsearch/ElasticsearchSystemAdmin.java > PRE-CREATION > > samza-elasticsearch/src/main/java/org/apache/samza/system/elasticsearch/ElasticsearchSystemFactory.java > PRE-CREATION > > samza-elasticsearch/src/main/java/org/apache/samza/system/elasticsearch/ElasticsearchSystemProducer.java > PRE-CREATION > > samza-elasticsearch/src/main/java/org/apache/samza/system/elasticsearch/IndexRequestFactory.java > PRE-CREATION > settings.gradle bb07a3b84b14dcef94da1bb166eab6aa3d0026bb > > Diff: https://reviews.apache.org/r/33297/diff/ > > > Testing > ------- > > > Thanks, > > Dan Harvey > >
