[
https://issues.apache.org/jira/browse/HIVE-20548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16613015#comment-16613015
]
zhangbutao commented on HIVE-20548:
-----------------------------------
hi，yes，we use ambari to start llap. Do you have some tutorials to setup llap
via yarn service ? Thanks !
Email: zhangbu...@cmss.chinamobile.com
发件人: Gopal V (JIRA)
时间: 2018/09/13(星期四)12:24
收件人: zhangbutao;
主题: [jira] [Commented] (HIVE-20548) Can not start llp via yarn service
[
https://issues.apache.org/jira/browse/HIVE-20548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612999#comment-16612999
]
Gopal V commented on HIVE-20548:
--------------------------------
This is a YARN setup problem - are you using Ambari? Ambari does this while
setting up YARN.
https://github.com/apache/ambari/blob/5460e8952729854f1c032a781c9a8de608ba4475/ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py#L213
> Can not start llp via yarn service
> ----------------------------------
>
> Key: HIVE-20548
> URL: https://issues.apache.org/jira/browse/HIVE-20548
> Project: Hive
> Issue Type: Bug
> Components: llap
> Affects Versions: 3.1.0
> Reporter: zhangbutao
> Priority: Major
>
> We start llap through yarn service instead of slider, and some problems
> happen as follows:
> {code:java}
> 2018-09-12 19:32:48,629 - LLAP start command:
> /usr/bch/current/hive-server2/bin/hive --service llap --size 10930m
> --startImmediately --name llap0 --cache 0m --xmx 8m --loglevel INFO --output
> /var/lib/ambari-agent/tmp/llap-yarn-service_2018-09-12_11-32-48
> --service-placement 4 --skiphadoopversion --skiphbasecp --instances 1
> --logger query-routing --args " -XX:+AlwaysPreTouch -XX:+UseG1GC
> -XX:TLABSize=8m -XX:+ResizeTLAB -XX:+UseNUMA -XX:+AggressiveOpts
> -XX:InitiatingHeapOccupancyPercent=70 -XX:+UnlockExperimentalVMOptions
> -XX:G1MaxNewSizePercent=40 -XX:G1ReservePercent=20 -XX:MaxGCPauseMillis=200
> -XX:MetaspaceSize=1024m"
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/bch/3.0.0/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/bch/3.0.0/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> WARN conf.HiveConf: HiveConf of name hive.hook.proto.base-directory does not
> exist
> WARN conf.HiveConf: HiveConf of name hive.strict.managed.tables does not exist
> WARN conf.HiveConf: HiveConf of name hive.stats.fetch.partition.stats does
> not exist
> WARN conf.HiveConf: HiveConf of name hive.heapsize does not exist
> WARN conf.HiveConf: HiveConf of name hive.druid.select.distribute does not
> exist
> WARN cli.LlapServiceDriver: Ignoring unknown llap server parameter:
> [hive.aux.jars.path]
> WARN cli.LlapServiceDriver: Java versions might not match :
> JAVA_HOME=[/usr/jdk64/jdk1.8.0_112],process jre=[/usr/jdk64/jdk1.8.0_112/jre]
> WARN conf.HiveConf: HiveConf of name hive.hook.proto.base-directory does not
> exist
> WARN conf.HiveConf: HiveConf of name hive.strict.managed.tables does not exist
> WARN conf.HiveConf: HiveConf of name hive.stats.fetch.partition.stats does
> not exist
> WARN conf.HiveConf: HiveConf of name hive.heapsize does not exist
> WARN conf.HiveConf: HiveConf of name hive.druid.select.distribute does not
> exist
> WARN conf.HiveConf: HiveConf of name hive.hook.proto.base-directory does not
> exist
> WARN conf.HiveConf: HiveConf of name hive.strict.managed.tables does not exist
> WARN conf.HiveConf: HiveConf of name hive.stats.fetch.partition.stats does
> not exist
> WARN conf.HiveConf: HiveConf of name hive.heapsize does not exist
> WARN conf.HiveConf: HiveConf of name hive.druid.select.distribute does not
> exist
> 11:32:54 Running as a child of LlapServiceDriver
> 11:32:54 Prepared the files
> 11:33:13 Packaged the files
> WARN curator.CuratorZookeeperClient: session timeout [10000] is less than
> connection timeout [15000]
> ERROR client.ServiceClient: Error on destroy 'llap0': not found.
> WARN client.ServiceClient: Property yarn.service.framework.path has a value
> /bch/apps/3.0.0/yarn/service-dep.tar.gz, but is not a valid file
> 2018-09-12 19:33:17,385 -
> 2018-09-12 19:33:17,385 - LLAP status command :
> /usr/bch/current/hive-server2/bin/hive --service llapstatus -w -r 0.8 -i 2 -t
> 400
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/bch/3.0.0/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/bch/3.0.0/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> WARN conf.HiveConf: HiveConf of name hive.hook.proto.base-directory does not
> exist
> WARN conf.HiveConf: HiveConf of name hive.strict.managed.tables does not exist
> WARN conf.HiveConf: HiveConf of name hive.stats.fetch.partition.stats does
> not exist
> WARN conf.HiveConf: HiveConf of name hive.heapsize does not exist
> WARN conf.HiveConf: HiveConf of name hive.druid.select.distribute does not
> exist
> LLAPSTATUS WatchMode with timeout=400 s
> --------------------------------------------------------------------------------
> LLAP Starting up with AppId=application_1536745653378_0002.
> --------------------------------------------------------------------------------
> WARN cli.LlapStatusServiceDriver: COMPLETE state reached while waiting for
> RUNNING state. Failing.
> Final diagnostics: null
> LLAP Application already complete.
> ApplicationId=application_1536745653378_0002
> --------------------------------------------------------------------------------
> {
> "amInfo" : {
> "appName" : "llap0",
> "appType" : "yarn-service",
> "appId" : "application_1536745653378_0002"
> },
> "state" : "COMPLETE",
> "appStartTime" : 1536751995975,
> "appFinishTime" : 1536752013756,
> "runningThresholdAchieved" : false
> }
> 2018-09-12 19:33:34,521 - LLAP app 'llap0' current state is COMPLETE.
> 2018-09-12 19:33:34,521 - LLAP app 'llap0' current state is COMPLETE.
> 2018-09-12 19:33:34,521 - LLAP app 'llap0' deployment unsuccessful.
> Command failed after 1 tries
> {code}
> We find the error logs:
> {code:java}
> ERROR client.ServiceClient: Error on destroy 'llap0': not found.
> WARN client.ServiceClient: Property yarn.service.framework.path has a value
> /bch/apps/3.0.0/yarn/service-dep.tar.gz, but is not a valid file
> {code}
> but we can't make sure if the yarn service or llap has some problems. In
> addition, hadoop version is 3.1.0.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
> Can not start llp via yarn service
> ----------------------------------
>
> Key: HIVE-20548
> URL: https://issues.apache.org/jira/browse/HIVE-20548
> Project: Hive
> Issue Type: Bug
> Components: llap
> Affects Versions: 3.1.0
> Reporter: zhangbutao
> Priority: Major
>
> We start llap through yarn service instead of slider, and some problems
> happen as follows:
> {code:java}
> 2018-09-12 19:32:48,629 - LLAP start command:
> /usr/bch/current/hive-server2/bin/hive --service llap --size 10930m
> --startImmediately --name llap0 --cache 0m --xmx 8m --loglevel INFO --output
> /var/lib/ambari-agent/tmp/llap-yarn-service_2018-09-12_11-32-48
> --service-placement 4 --skiphadoopversion --skiphbasecp --instances 1
> --logger query-routing --args " -XX:+AlwaysPreTouch -XX:+UseG1GC
> -XX:TLABSize=8m -XX:+ResizeTLAB -XX:+UseNUMA -XX:+AggressiveOpts
> -XX:InitiatingHeapOccupancyPercent=70 -XX:+UnlockExperimentalVMOptions
> -XX:G1MaxNewSizePercent=40 -XX:G1ReservePercent=20 -XX:MaxGCPauseMillis=200
> -XX:MetaspaceSize=1024m"
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/bch/3.0.0/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/bch/3.0.0/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> WARN conf.HiveConf: HiveConf of name hive.hook.proto.base-directory does not
> exist
> WARN conf.HiveConf: HiveConf of name hive.strict.managed.tables does not exist
> WARN conf.HiveConf: HiveConf of name hive.stats.fetch.partition.stats does
> not exist
> WARN conf.HiveConf: HiveConf of name hive.heapsize does not exist
> WARN conf.HiveConf: HiveConf of name hive.druid.select.distribute does not
> exist
> WARN cli.LlapServiceDriver: Ignoring unknown llap server parameter:
> [hive.aux.jars.path]
> WARN cli.LlapServiceDriver: Java versions might not match :
> JAVA_HOME=[/usr/jdk64/jdk1.8.0_112],process jre=[/usr/jdk64/jdk1.8.0_112/jre]
> WARN conf.HiveConf: HiveConf of name hive.hook.proto.base-directory does not
> exist
> WARN conf.HiveConf: HiveConf of name hive.strict.managed.tables does not exist
> WARN conf.HiveConf: HiveConf of name hive.stats.fetch.partition.stats does
> not exist
> WARN conf.HiveConf: HiveConf of name hive.heapsize does not exist
> WARN conf.HiveConf: HiveConf of name hive.druid.select.distribute does not
> exist
> WARN conf.HiveConf: HiveConf of name hive.hook.proto.base-directory does not
> exist
> WARN conf.HiveConf: HiveConf of name hive.strict.managed.tables does not exist
> WARN conf.HiveConf: HiveConf of name hive.stats.fetch.partition.stats does
> not exist
> WARN conf.HiveConf: HiveConf of name hive.heapsize does not exist
> WARN conf.HiveConf: HiveConf of name hive.druid.select.distribute does not
> exist
> 11:32:54 Running as a child of LlapServiceDriver
> 11:32:54 Prepared the files
> 11:33:13 Packaged the files
> WARN curator.CuratorZookeeperClient: session timeout [10000] is less than
> connection timeout [15000]
> ERROR client.ServiceClient: Error on destroy 'llap0': not found.
> WARN client.ServiceClient: Property yarn.service.framework.path has a value
> /bch/apps/3.0.0/yarn/service-dep.tar.gz, but is not a valid file
> 2018-09-12 19:33:17,385 -
> 2018-09-12 19:33:17,385 - LLAP status command :
> /usr/bch/current/hive-server2/bin/hive --service llapstatus -w -r 0.8 -i 2 -t
> 400
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/bch/3.0.0/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/bch/3.0.0/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> WARN conf.HiveConf: HiveConf of name hive.hook.proto.base-directory does not
> exist
> WARN conf.HiveConf: HiveConf of name hive.strict.managed.tables does not exist
> WARN conf.HiveConf: HiveConf of name hive.stats.fetch.partition.stats does
> not exist
> WARN conf.HiveConf: HiveConf of name hive.heapsize does not exist
> WARN conf.HiveConf: HiveConf of name hive.druid.select.distribute does not
> exist
> LLAPSTATUS WatchMode with timeout=400 s
> --------------------------------------------------------------------------------
> LLAP Starting up with AppId=application_1536745653378_0002.
> --------------------------------------------------------------------------------
> WARN cli.LlapStatusServiceDriver: COMPLETE state reached while waiting for
> RUNNING state. Failing.
> Final diagnostics: null
> LLAP Application already complete.
> ApplicationId=application_1536745653378_0002
> --------------------------------------------------------------------------------
> {
> "amInfo" : {
> "appName" : "llap0",
> "appType" : "yarn-service",
> "appId" : "application_1536745653378_0002"
> },
> "state" : "COMPLETE",
> "appStartTime" : 1536751995975,
> "appFinishTime" : 1536752013756,
> "runningThresholdAchieved" : false
> }
> 2018-09-12 19:33:34,521 - LLAP app 'llap0' current state is COMPLETE.
> 2018-09-12 19:33:34,521 - LLAP app 'llap0' current state is COMPLETE.
> 2018-09-12 19:33:34,521 - LLAP app 'llap0' deployment unsuccessful.
> Command failed after 1 tries
> {code}
> We find the error logs:
> {code:java}
> ERROR client.ServiceClient: Error on destroy 'llap0': not found.
> WARN client.ServiceClient: Property yarn.service.framework.path has a value
> /bch/apps/3.0.0/yarn/service-dep.tar.gz, but is not a valid file
> {code}
> but we can't make sure if the yarn service or llap has some problems. In
> addition, hadoop version is 3.1.0.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
