ZihanLi58 commented on PR #3549: URL: https://github.com/apache/gobblin/pull/3549#issuecomment-1251711184
@arjun4084346 here is the answer 1) The normal mysqlSpecStore treat update the same as add operation, and the add statement accepts add when key is existed. We used only have one host talk to mysql so this is durable. But when we enable warm standby, multi host can access mysql for the same entry at the same time. I want to fail one of the request when such thing happens to make sure they are not over write with each other. This require a new mysql catalog where we treat update and add different. Add will fail when the key already exist, update will fail when it try to update one non-existed key or the key is recently updated after we receive the request (where concurrent write happens) 2) new resources handler is specifically for warm standby where we do not care whether it's leader/follower, and directly compile and update the flow catalog. Also it needs to call update method for mysql catalog when it's a update request 3) those two are both for warmstandby, and as new resource handler needs to handle the race condition, we introduce the new mysql catalog to handle that -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
