PutDataBaseRecord performance issues.

2019-10-15 Thread 王磊 
I am using CaptureChangeMySQL to extract bin log and do some transformations 
and then write to another database using  PutDataBaseRecord.  Now the 
PutDataBaseRecord  processor is a performance bottleneck
If i set the PutDataBaseRecord  processor concurrency lager than 1,  there will 
be ordering issues. The ordering the binlog to the destination database will 
not be the same as they comming. But with  one concurrency, the TPS is only 
about 80/s 
Even I add a MergeRecord before PutDataBaseRecord, the TPS is no more than 300
Anybody have any idea about this?


Thanks,
Lei

MergeRecord can not guarantee the ordering of the input sequence?

2019-10-15 Thread 王磊 
If  FlowFile A, B, C enter the MergeRecord sequentially, the output should be 
one FlowFile {A, B, C}
However, when testing with  large data volume, sometimes the output order will 
be not the same as they enter. And this result is nondeterministic


This really confuses me a lot.
Anybody has any insight on this?


Thanks，
Lei