Louro (Hortonworks)
- [20m] – Rethinking the Storm 2.0 Worker - Roshan Naik (Hortonworks)
- [57m] – Storm in Retail Context: Catalog data processing using
Kafka, Storm & Microservices - Karthik Deivasigamani (WalMart Labs)
- [1h: 54m:45sec] – Schema Regi
Here is a case of data duplication that should be avoidable.
It is observed when leadership of partition changes from the current leader
back to preferred leader.
Steps to reproduce:
- Using 3 broker setup.
- Create topic with 1 partition, replication factor=3, ISR count=2 and
Thanks Jay and Ewen for the response.
@Jay
3. This has a built in notion of parallelism throughout.
It was not obvious how it will look like or differ from existing systemsŠ
since all of existing ones do parallelize data movement.
@Ewen,
Import: Flume is just one of many similar systems
My initial thoughts:
Although it is kind of discussed very broadly, I did struggle a bit to properly
grasp the value add this adds over the alternative approaches that are
available today (or need a little work to accomplish) in specific use cases. I
feel its better to take specific common
For some reason the HTML formatting is being dropped from my email..
Making it harder to read the measurements table.
On 4/29/15 8:32 PM, Roshan Naik ros...@hortonworks.com wrote:
@Jay,
My bad. I mistook the batch.size to be number of messages instead of
bytes. Below are revised measurements
@Jay,
My bad. I mistook the batch.size to be number of messages instead of bytes.
Below are revised measurements based on computing the batch.size in bytes .
@Jun,
With explicit flush()... linger should not impact. Isn't it ?
@Wang,
Larger batches are not necessarily giving better
Based on recent suggestion by Joel, I am experimenting with using flush() to
simulate batched-sync behavior.
The essence of my single threaded producer code is :
for (int i = 0; i numRecords;) {
// 1- Send a batch
for(int batchCounter=0; batchCounterbatchSz;
:58:43AM +, Roshan Naik wrote:
Based on recent suggestion by Joel, I am experimenting with using
flush() to simulate batched-sync behavior.
The essence of my single threaded producer code is :
for (int i = 0; i numRecords;) {
// 1- Send a batch
for(int batchCounter=0
