Hi Will, How are you starting your cluster/executing your program? From what I can gather you are using Beam, is that right? The line about the FlinkMiniCluster seems strange because this would hint at the fact that the Runner is trying to execute in local mode.
Best, Aljoscha > On 6. Jun 2017, at 02:13, Will Walters <wwalt...@yahoo-inc.com> wrote: > > Hello, > > I'm having issues editing the default Flink memory settings. I'm attempting > to run a Flink task on a cluster at scale. The log shows my edited config > settings having been read into the program, but they're having no effect. > Here's the trace: > > 17/06/05 23:45:41 INFO flink.FlinkRunner: Starting execution of Flink program. > 17/06/05 23:45:41 INFO java.ExecutionEnvironment: The job has 0 registered > types and 0 default Kryo serializers > 17/06/05 23:45:41 INFO configuration.GlobalConfiguration: Loading > configuration property: taskmanager.memory.size, 100 > 17/06/05 23:45:41 INFO configuration.GlobalConfiguration: Loading > configuration property: taskmanager.heap.mb, 256 > 17/06/05 23:45:41 INFO configuration.GlobalConfiguration: Loading > configuration property: taskmanager.debug.memory.startLogThread, true > 17/06/05 23:45:41 INFO minicluster.FlinkMiniCluster: Disabled queryable state > server > 17/06/05 23:45:41 INFO minicluster.FlinkMiniCluster: Starting > FlinkMiniCluster. > . > . > . > 17/06/05 23:45:41 INFO network.NetworkEnvironment: Starting the network > environment and its components. > 17/06/05 23:45:41 INFO taskmanager.TaskManager: Limiting managed memory to > 17592186044406 MB, memory will be allocated lazily. > 17/06/05 23:45:41 ERROR flink.FlinkRunner: Pipeline execution failed > java.lang.IllegalArgumentException: Size of total memory must be positive. > > And here's the config file I'm using: > > taskmanager.memory.size: 100 > taskmanager.heap.mb: 256 > taskmanager.debug.memory.startLogThread: true > > In the shell script I'm using to run the task, I've edited the FLINK_CONF_DIR > to direct to the config file that I created. > > If anyone has any advice or inputs it would be much appreciated. > > Thanks! > Will Walters.
