featzhang opened a new issue, #7072:
URL: https://github.com/apache/inlong/issues/7072

   ### Description
   
   Hudi flink jobs often have unreasonable resource allocation. Too much 
allocation will lead to a waste of resources, and too little will lead to back 
pressure or OOM.
   
   When allocating resources, you first need to determine the concurrency of 
the source side to ensure that there is no data backlog upstream when reading. 
Here is a general configuration situation, such as partitioning by day, with 
about 15 billion data per day, and about 50 concurrent configurations. Other 
data volumes can be converted appropriately.
   
   After determining the concurrency on the source side, you can configure the 
concurrency of write according to the ratio of 1:1.5 or 1:2.
   
   ### Use case
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes, I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to