Thanks a lot for posting the full output.

It seems that Flink is passing an invalid list of arguments to the JVM.
Can you
- set the root log level in conf/log4j-yarn-session.properties to DEBUG
- then launch the YARN session
- share the log file of the yarn session on the mailing list?

I'm particularly interested in the line printed here, as it shows the JVM
invocation.
https://github.com/apache/flink/blob/release-1.6/flink-yarn/src/main/java/org/apache/flink/yarn/AbstractYarnClusterDescriptor.java#L1630


On Mon, May 11, 2020 at 9:56 PM Jacky D <jacky.du0...@gmail.com> wrote:

> Hi,Robert
>
> Yes , I tried to retrieve more log info from yarn UI , the full logs
> showing below , this happens when I try to create a flink yarn session on
> emr when set up jitwatch configuration .
>
> 2020-05-11 19:06:09,552 ERROR
> org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - Error while
> running the Flink Yarn session.
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1862)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> at
> org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:813)
> Caused by: org.apache.flink.client.deployment.ClusterDeploymentException:
> Couldn't deploy Yarn session cluster
> at
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploySessionCluster(AbstractYarnClusterDescriptor.java:429)
> at
> org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:610)
> at
> org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$2(FlinkYarnSessionCli.java:813)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
> ... 2 more
> Caused by:
> org.apache.flink.yarn.AbstractYarnClusterDescriptor$YarnDeploymentException:
> The YARN application unexpectedly switched to state FAILED during
> deployment.
> Diagnostics from YARN: Application application_1584459865196_0165 failed 1
> times (global limit =2; local limit is =1) due to AM Container for
> appattempt_1584459865196_0165_000001 exited with  exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1584459865196_0165_01_000001
> Exit code: 1
> Exception message: Usage: java [-options] class [args...]
>            (to execute a class)
>    or  java [-options] -jar jarfile [args...]
>            (to execute a jar file)
> where options include:
>     -d32   use a 32-bit data model if available
>     -d64   use a 64-bit data model if available
>     -server   to select the "server" VM
>                   The default VM is server,
>                   because you are running on a server-class machine.
>
>
>     -cp <class search path of directories and zip/jar files>
>     -classpath <class search path of directories and zip/jar files>
>                   A : separated list of directories, JAR archives,
>                   and ZIP archives to search for class files.
>     -D<name>=<value>
>                   set a system property
>     -verbose:[class|gc|jni]
>                   enable verbose output
>     -version      print product version and exit
>     -version:<value>
>                   Warning: this feature is deprecated and will be removed
>                   in a future release.
>                   require the specified version to run
>     -showversion  print product version and continue
>     -jre-restrict-search | -no-jre-restrict-search
>                   Warning: this feature is deprecated and will be removed
>                   in a future release.
>                   include/exclude user private JREs in the version search
>     -? -help      print this help message
>     -X            print help on non-standard options
>     -ea[:<packagename>...|:<classname>]
>     -enableassertions[:<packagename>...|:<classname>]
>                   enable assertions with specified granularity
>     -da[:<packagename>...|:<classname>]
>     -disableassertions[:<packagename>...|:<classname>]
>                   disable assertions with specified granularity
>     -esa | -enablesystemassertions
>                   enable system assertions
>     -dsa | -disablesystemassertions
>                   disable system assertions
>     -agentlib:<libname>[=<options>]
>                   load native agent library <libname>, e.g. -agentlib:hprof
>                   see also, -agentlib:jdwp=help and -agentlib:hprof=help
>     -agentpath:<pathname>[=<options>]
>                   load native agent library by full pathname
>     -javaagent:<jarpath>[=<options>]
>                   load Java programming language agent, see
> java.lang.instrument
>     -splash:<imagepath>
>                   show splash screen with specified image
> See http://www.oracle.com/technetwork/java/javase/documentation/index.html
> for more details.
>
> Thanks
> Jacky
>
> Robert Metzger <rmetz...@apache.org> 于2020年5月11日周一 下午3:42写道:
>
>> Hey Jacky,
>>
>> The error says "The YARN application unexpectedly switched to state
>> FAILED during deployment.".
>> Have you tried retrieving the YARN application logs?
>> Does the YARN UI / resource manager logs reveal anything on the reason
>> for the deployment to fail?
>>
>> Best,
>> Robert
>>
>>
>> On Mon, May 11, 2020 at 9:34 PM Jacky D <jacky.du0...@gmail.com> wrote:
>>
>>>
>>>
>>> ---------- Forwarded message ---------
>>> 发件人: Jacky D <jacky.du0...@gmail.com>
>>> Date: 2020年5月11日周一 下午3:12
>>> Subject: Re: Flink Memory analyze on AWS EMR
>>> To: Khachatryan Roman <khachatryan.ro...@gmail.com>
>>>
>>>
>>> Hi, Roman
>>>
>>> Thanks for quick response , I tried without logFIle option but failed
>>> with same error , I'm currently using flink 1.6
>>> https://ci.apache.org/projects/flink/flink-docs-release-1.6/monitoring/application_profiling.html,
>>> so I can only use Jitwatch or JMC .  I guess those tools only available on
>>> Standalone cluster ? as document mentioned "Each standalone JobManager,
>>> TaskManager, HistoryServer, and ZooKeeper daemon redirects stdout and
>>> stderr to a file with a .out filename suffix and writes internal
>>> logging to a file with a .log suffix. Java options configured by the
>>> user in env.java.opts" ?
>>>
>>> Thanks
>>> Jacky
>>>
>>

Reply via email to