My gut feeling is your "vmArgs" does not take effect. Best, Yangze Guo
On Mon, Dec 7, 2020 at 10:32 AM Yangze Guo <karma...@gmail.com> wrote: > > Hi, Rex, > > Can you share more logs for it. Did you see something like "The > configuration option taskmanager.cpu.cores required for local > execution is not set, setting it to" in your logs? > > Best, > Yangze Guo > > Best, > Yangze Guo > > > On Sat, Dec 5, 2020 at 6:53 PM David Anderson <dander...@apache.org> wrote: > > > > taskmanager.cpu.cores is intended for internal use only -- you aren't meant > > to set this option. What happens if you leave it alone? > > > > Regards, > > David > > > > > > On Sat, Dec 5, 2020 at 8:04 AM Rex Fenley <r...@remind101.com> wrote: > >> > >> We're running this in a local environment so that may be contributing to > >> what we're seeing. > >> > >> On Fri, Dec 4, 2020 at 10:41 PM Rex Fenley <r...@remind101.com> wrote: > >>> > >>> Hello, > >>> > >>> I'm tuning flink for parallelism right now and when I look at the > >>> JobManager I see > >>> taskmanager.cpu.cores1.7976931348623157E308 > >>> Which looks like the maximum double number. > >>> > >>> We have 8 cpu cores, so we figured we'd bump to 16 for hyper threading. > >>> We have 37 operators so we rounded up and set 40 task slots. > >>> > >>> Here is our configuration > >>> > >>> "vmArgs": "-Xmx16g -Xms16g -XX:MaxDirectMemorySize=1207959552 > >>> -XX:MaxMetaspaceSize=268435456 -Dlog.file=/tmp/flink.log > >>> -Dtaskmanager.memory.framework.off-heap.size=134217728b > >>> -Dtaskmanager.memory.network.max=1073741824b > >>> -Dtaskmanager.memory.network.min=1073741824b > >>> -Dtaskmanager.memory.framework.heap.size=134217728b > >>> -Dtaskmanager.memory.managed.size=6335076856b > >>> -Dtaskmanager.memory.task.heap.size=8160437768b > >>> -Dtaskmanager.memory.task.off-heap.size=0b > >>> -Dtaskmanager.numberOfTaskSlots=40 -Dtaskmanager.cpu.cores=16.0" > >>> > >>> We then tried with -Dtaskmanager.cpu.cores=7.0 and still ended up with > >>> that very odd value for cpu cores. > >>> > >>> How do we correctly adjust this? > >>> > >>> Thanks! > >>> -- > >>> > >>> Rex Fenley | Software Engineer - Mobile and Backend > >>> > >>> > >>> Remind.com | BLOG | FOLLOW US | LIKE US > >> > >> > >> > >> -- > >> > >> Rex Fenley | Software Engineer - Mobile and Backend > >> > >> > >> Remind.com | BLOG | FOLLOW US | LIKE US
