Re: Unable to access flowfile content

2019-12-19 Thread wangl...@geekplus.com.cn
Hi Purushotham, Since you are using cluster mode, just delete the flow.xml.gz file and restart the node, the flow file will be synced from the other two nodes. Regards, Lei wangl...@geekplus.com.cn From: Purushotham Pushpavanthar Date: 2019-12-19 17:05 To: dev Subject: Unable to access

Re: Re: NiFi backpressure not work

2019-10-16 Thread wangl...@geekplus.com.cn
the backpressure mechanism, the processor must do something? In my case, what it will happen if the queue continue to grow(for example to 100G)? Something bad will happen? Thanks, Lei wangl...@geekplus.com.cn From: josef.zahn...@swisscom.com Date: 2019-10-16 14:17 To: us...@nifi.apache.org; dev

NiFi backpressure not work

2019-10-16 Thread wangl...@geekplus.com.cn
As the picture shows. I have set the backpressure object threshold to 1, but picture shows there's more than 1,000,000. Why this happens? Thanks, Lei wangl...@geekplus.com.cn

Re: Re: MergeRecord can not guarantee the ordering of the input sequence?

2019-10-15 Thread wangl...@geekplus.com.cn
This is nondeterministic. I think I should look up the MergeRecord code and do further debug. Thanks, Lei wangl...@geekplus.com.cn From: Koji Kawamura Date: 2019-10-16 09:46 To: users CC: dev Subject: Re: MergeRecord can not guarantee the ordering of the input sequence? Hi Lei, How about

PutDataBaseRecord performance issues.

2019-10-15 Thread wangl...@geekplus.com.cn
issues. The ordering the binlog to the destination database will not be the same as they comming. But with one concurrency, the TPS is only about 80/s Even I add a MergeRecord before PutDataBaseRecord, the TPS is no more than 300 Anybody have any idea about this? Thanks, Lei wangl

Data inconsistency happens when using CDC to replicate my database

2019-10-14 Thread wangl...@geekplus.com.cn
wangl...@geekplus.com.cn

Can CaptureChangeMySQL be scheduled to all nodes instead of primary node?

2019-10-09 Thread wangl...@geekplus.com.cn
on the primary node. Is there any method than i can distribute the CaptureChangeMySQL processors to all nodes instead of only to primary node? Thanks, Lei wangl...@geekplus.com.cn

Memory issues when using NiFi to replicate mysql database with CaptureChangeMySQL

2019-09-30 Thread wangl...@geekplus.com.cn
to optimize the memory usage of NiFi? Thanks, Lei wangl...@geekplus.com.cn

CaptureChangeMySQL table name map error if one update statement changed two tables

2019-09-26 Thread wangl...@geekplus.com.cn
TableMap BMap UPDATE AUPDATE UPDATE BUPDATE If a DistributedCacheClient is used, the two UPDATE EVENT will be mapped to the same table(the latter BMap). It is an expected behavior according to the source code. I think it is a bug, we can fix it Thanks, Lei wangl

Re: RE: CaptureChangeMySQL throws communication failure error every 30 seconds

2019-09-16 Thread wangl...@geekplus.com.cn
:3306 at mysql-bin.000675/707993476 (sid:10091, cid:5560080) I have stopped the processor, why it still retry to connect to mysql server? Thanks, Lei wangl...@geekplus.com.cn From: Williams, Jim Date: 2019-09-11 21:42 To: us...@nifi.apache.org Subject: RE: CaptureChangeMySQL throws

NiFi cluster heavy cpu usage

2019-09-16 Thread wangl...@geekplus.com.cn
- - 3593 00:39:05 There's some threads consumes cpu. I pick one and jstack it, often it is in waiting state. Any insight on this? Thanks, Lei wangl...@geekplus.com.cn

回复: CaptureChangeMySQL bit(1) type field is parsed to '{}'

2019-09-03 Thread wangl...@geekplus.com.cn
Anybody has any insight on this? wangl...@geekplus.com.cn 发件人: wangl...@geekplus.com.cn 发送时间: 2019-08-22 19:31 收件人: users 主题: CaptureChangeMySQL bit(1) type field is parsed to '{}' CaptureChangeMySQL -> ConvertJSONToSQL -> ExecuteSQL to replicate my database. But when the field type

Still have Network traffic after stopped the CaptureChangeMySQL processor.

2019-08-30 Thread wangl...@geekplus.com.cn
Using CaptureChangeMySQL to parse binlog from mysql server. Surely there will be network traffic transmission from the MySQL server to NiFi Server. But after I stop the CaptureChangeMySQL processor, why the network traffic transmission still exists? Thanks, Lei wangl