Folks, does anyone have experience they can share regarding memory allocation for Samza tasks? Out of the box, it looks like the ApplicationManager defaults to 1GB of RAM for its container and 1GB per YARN container for each TaskRunner.
Some of my Samza tasks are pretty simple and (I think) use very little runtime memory per partition — essentially following a pattern of read message, process, commit result to a database or a stream output, repeat. For these kinds of tasks, I’m assuming I can safely scale down the container memory bounds. What about ApplicationManager? Does it need a full GB per Samza task? Thanks! —T
