Hi everyone, I apologize in advance if that is not the right mailing list for my question. If there is a better place for it, please let me know.
Basically, I wanted to ask if you have some statement about the hardware requirements of Flink to process larger amounts of data beginning from, say, 20 GBs. Currently, I am facing issues in my jobs, e.g., there are not enough buffers for safe execution of some operations. Since the machines that run my TaskTrackers have unfortunately very limited main memory, I cannot increase the number of buffers (and heap space in general) too much. Currently, I assigned them 1.5 GB. So, the exact questions are: * Do you have experiences with a suitable HW setup for crunching larger amounts of data, maybe from the TU cluster? * Are there any configuration tips, you can provide, e.g. pertaining to the buffer configuration? * Are there any general statements on the growth of Flink's memory requirements wrt. to the size of the input data? Thanks for your help! Sebastian
