[
https://issues.apache.org/jira/browse/FLINK-32909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Wu updated FLINK-32909:
----------------------------
Description:
I' m trying to use the jobmanager.sh script to create a jobmanager instance
manually, and I need to pass arugments to the script dynamically, rather than
through flink-conf.yaml. But I found that I didn't succeed in doing that when I
commented out all configurations in the flink-conf.yaml, I typed command like:
{code:java}
./bin/jobmanager.sh start -D jobmanager.memory.flink.size=1024m -D
jobmanager.rpc.address=xx.xx.xx.xx -D jobmanager.rpc.port=xxx -D
jobmanager.bind-host=0.0.0.0 -D rest.address=xx.xx.xx.xx -D rest.port=xxx -D
rest.bind-address=0.0.0.0{code}
but I got some errors below:
{code:java}
[ERROR] The execution result is empty.
[ERROR] Could not get JVM parameters and dynamic configurations properly.
[ERROR] Raw output from BashJavaUtils:
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will
impact performance.
Exception in thread "main"
org.apache.flink.configuration.IllegalConfigurationException: JobManager memory
configuration failed: Either required fine-grained memory
(jobmanager.memory.heap.size), or Total Flink Memory size (Key:
'jobmanager.memory.flink.size' , default: null (fallback keys: [])), or Total
Process Memory size (Key: 'jobmanager.memory.process.size' , default: null
(fallback keys: [])) need to be configured explicitly.
at
org.apache.flink.runtime.jobmanager.JobManagerProcessUtils.processSpecFromConfigWithNewOptionToInterpretLegacyHeap(JobManagerProcessUtils.java:78)
at
org.apache.flink.runtime.util.bash.BashJavaUtils.getJmResourceParams(BashJavaUtils.java:98)
at
org.apache.flink.runtime.util.bash.BashJavaUtils.runCommand(BashJavaUtils.java:69)
at
org.apache.flink.runtime.util.bash.BashJavaUtils.main(BashJavaUtils.java:56)
Caused by: org.apache.flink.configuration.IllegalConfigurationException: Either
required fine-grained memory (jobmanager.memory.heap.size), or Total Flink
Memory size (Key: 'jobmanager.memory.flink.size' , default: null (fallback
keys: [])), or Total Process Memory size (Key: 'jobmanager.memory.process.size'
, default: null (fallback keys: [])) need to be configured explicitly.
at
org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils.failBecauseRequiredOptionsNotConfigured(ProcessMemoryUtils.java:129)
at
org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils.memoryProcessSpecFromConfig(ProcessMemoryUtils.java:86)
at
org.apache.flink.runtime.jobmanager.JobManagerProcessUtils.processSpecFromConfig(JobManagerProcessUtils.java:83)
at
org.apache.flink.runtime.jobmanager.JobManagerProcessUtils.processSpecFromConfigWithNewOptionToInterpretLegacyHeap(JobM
{code}
It seems to remind me to configure memory for jobmanager instance explicitly,
but I had already passed the jobmanager.memory.flink.size parameter. So I debug
the script, and found a spelling error in the jobmanager.sh script at line 54:
{code:java}
parseJmArgsAndExportLogs "${ARGS[@]}"
{code}
the uppercase "$\{ARGS[@]}" is a wrong variable name here from a contextual
perspective, and causing an empty string passed to the function. I changed to
"$\{args[@]}" and It works fine.
was:
I' m trying to use the jobmanager.sh script to create a jobmanager instance
manually, and I need to pass arugments to the script dynamically, rather than
through flink-conf.yaml. But I found that I didn't succeed in doing that when I
commented out all configurations in the flink-conf.yaml, I typed command like:
{code:java}
./bin/jobmanager.sh start -D jobmanager.memory.flink.size=1024m -D
jobmanager.rpc.address=xx.xx.xx.xx -D jobmanager.rpc.port=xxx -D
jobmanager.bind-host=0.0.0.0 -Drest.address=xx.xx.xx.xx -Drest.port=xxx
-Drest.bind-address=0.0.0.0{code}
but I got some errors below:
{code:java}
[ERROR] The execution result is empty.
[ERROR] Could not get JVM parameters and dynamic configurations properly.
[ERROR] Raw output from BashJavaUtils:
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will
impact performance.
Exception in thread "main"
org.apache.flink.configuration.IllegalConfigurationException: JobManager memory
configuration failed: Either required fine-grained memory
(jobmanager.memory.heap.size), or Total Flink Memory size (Key:
'jobmanager.memory.flink.size' , default: null (fallback keys: [])), or Total
Process Memory size (Key: 'jobmanager.memory.process.size' , default: null
(fallback keys: [])) need to be configured explicitly.
at
org.apache.flink.runtime.jobmanager.JobManagerProcessUtils.processSpecFromConfigWithNewOptionToInterpretLegacyHeap(JobManagerProcessUtils.java:78)
at
org.apache.flink.runtime.util.bash.BashJavaUtils.getJmResourceParams(BashJavaUtils.java:98)
at
org.apache.flink.runtime.util.bash.BashJavaUtils.runCommand(BashJavaUtils.java:69)
at
org.apache.flink.runtime.util.bash.BashJavaUtils.main(BashJavaUtils.java:56)
Caused by: org.apache.flink.configuration.IllegalConfigurationException: Either
required fine-grained memory (jobmanager.memory.heap.size), or Total Flink
Memory size (Key: 'jobmanager.memory.flink.size' , default: null (fallback
keys: [])), or Total Process Memory size (Key: 'jobmanager.memory.process.size'
, default: null (fallback keys: [])) need to be configured explicitly.
at
org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils.failBecauseRequiredOptionsNotConfigured(ProcessMemoryUtils.java:129)
at
org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils.memoryProcessSpecFromConfig(ProcessMemoryUtils.java:86)
at
org.apache.flink.runtime.jobmanager.JobManagerProcessUtils.processSpecFromConfig(JobManagerProcessUtils.java:83)
at
org.apache.flink.runtime.jobmanager.JobManagerProcessUtils.processSpecFromConfigWithNewOptionToInterpretLegacyHeap(JobM
{code}
It seems to remind me to configure memory for jobmanager instance explicitly,
but I had already passed the jobmanager.memory.flink.size parameter. So I debug
the script, and found a spelling error in the jobmanager.sh script at line 54:
{code:java}
parseJmArgsAndExportLogs "${ARGS[@]}"
{code}
the uppercase "$\{ARGS[@]}" is a wrong variable name here from a contextual
perspective, and causing an empty string passed to the function. I changed to
"$\{args[@]}" and It works fine.
> The jobmanager.sh pass arguments failed
> ---------------------------------------
>
> Key: FLINK-32909
> URL: https://issues.apache.org/jira/browse/FLINK-32909
> Project: Flink
> Issue Type: Bug
> Components: Deployment / Scripts
> Affects Versions: 1.16.2, 1.18.0, 1.17.1
> Reporter: Alex Wu
> Priority: Major
>
> I' m trying to use the jobmanager.sh script to create a jobmanager instance
> manually, and I need to pass arugments to the script dynamically, rather than
> through flink-conf.yaml. But I found that I didn't succeed in doing that when
> I commented out all configurations in the flink-conf.yaml, I typed command
> like:
>
> {code:java}
> ./bin/jobmanager.sh start -D jobmanager.memory.flink.size=1024m -D
> jobmanager.rpc.address=xx.xx.xx.xx -D jobmanager.rpc.port=xxx -D
> jobmanager.bind-host=0.0.0.0 -D rest.address=xx.xx.xx.xx -D rest.port=xxx -D
> rest.bind-address=0.0.0.0{code}
> but I got some errors below:
>
> {code:java}
> [ERROR] The execution result is empty.
> [ERROR] Could not get JVM parameters and dynamic configurations properly.
> [ERROR] Raw output from BashJavaUtils:
> WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will
> impact performance.
> Exception in thread "main"
> org.apache.flink.configuration.IllegalConfigurationException: JobManager
> memory configuration failed: Either required fine-grained memory
> (jobmanager.memory.heap.size), or Total Flink Memory size (Key:
> 'jobmanager.memory.flink.size' , default: null (fallback keys: [])), or Total
> Process Memory size (Key: 'jobmanager.memory.process.size' , default: null
> (fallback keys: [])) need to be configured explicitly.
> at
> org.apache.flink.runtime.jobmanager.JobManagerProcessUtils.processSpecFromConfigWithNewOptionToInterpretLegacyHeap(JobManagerProcessUtils.java:78)
> at
> org.apache.flink.runtime.util.bash.BashJavaUtils.getJmResourceParams(BashJavaUtils.java:98)
> at
> org.apache.flink.runtime.util.bash.BashJavaUtils.runCommand(BashJavaUtils.java:69)
> at
> org.apache.flink.runtime.util.bash.BashJavaUtils.main(BashJavaUtils.java:56)
> Caused by: org.apache.flink.configuration.IllegalConfigurationException:
> Either required fine-grained memory (jobmanager.memory.heap.size), or Total
> Flink Memory size (Key: 'jobmanager.memory.flink.size' , default: null
> (fallback keys: [])), or Total Process Memory size (Key:
> 'jobmanager.memory.process.size' , default: null (fallback keys: [])) need to
> be configured explicitly.
> at
> org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils.failBecauseRequiredOptionsNotConfigured(ProcessMemoryUtils.java:129)
> at
> org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils.memoryProcessSpecFromConfig(ProcessMemoryUtils.java:86)
> at
> org.apache.flink.runtime.jobmanager.JobManagerProcessUtils.processSpecFromConfig(JobManagerProcessUtils.java:83)
> at
> org.apache.flink.runtime.jobmanager.JobManagerProcessUtils.processSpecFromConfigWithNewOptionToInterpretLegacyHeap(JobM
> {code}
> It seems to remind me to configure memory for jobmanager instance explicitly,
> but I had already passed the jobmanager.memory.flink.size parameter. So I
> debug the script, and found a spelling error in the jobmanager.sh script at
> line 54:
>
> {code:java}
> parseJmArgsAndExportLogs "${ARGS[@]}"
> {code}
> the uppercase "$\{ARGS[@]}" is a wrong variable name here from a contextual
> perspective, and causing an empty string passed to the function. I changed to
> "$\{args[@]}" and It works fine.
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
