Hello I was running a program with 'parallelism.default' of 384 as I read in the documentation on Flink's official page that 'parallelism.default' is "the total number of CPUs in the cluster". I have four machines with 96 cores on each of them. So 96*4=384. But the program thew an error saying:
Caused by: java.io.IOException: Insufficient number of network buffers: required 384, but only 298 available. The total number of network buffers is currently set to 2048. You can increase this number by setting the configuration key 'taskmanager.network.numberOfBuffers'. What does this mean? And how to choose a proper value for parallelism? -- Thank You Regards Punit Naik
