nicknezis commented on pull request #3741: URL: https://github.com/apache/incubator-heron/pull/3741#issuecomment-985778347
> > If we want to lower the default Request/Limit values for the manager pod, we can always do that in a future PR. > > You would need a large and diverse dataset to form a baseline for the default resource values. I feel we would need to solicit statistics from users for topologies with varying configurations (bolts and spouts) as well as data velocity and volume to form a baseline. > So the bolts and spouts and strmgr processes all live in the executor pods. Data velocity and stuff like that is all in the executor. Manager just has a few processes that collect metrics and manage coordination of checkpointing (if used). Also if the physical plan changes it will coordinate changes through zookeeper. (It does more, but trying to give examples of the types of operations). I still think we can default to smaller value with low risk, but agree the risk is not 0%. After this is merged, I'll do some analysis on workloads at work to see what could be a good default. @windhamwong provided numbers also give me confidence because they match what I've observed. But I'll do a more rigorous process to capture numbers. But what I said above doesn't counter the decision to merge as is. I agree with @surahman decision. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
