Aman Raj created HIVE-26939:
-------------------------------

             Summary: Hive LLAP Application Master fails to come up with Hadoop 
3.3.4
                 Key: HIVE-26939
                 URL: https://issues.apache.org/jira/browse/HIVE-26939
             Project: Hive
          Issue Type: Bug
          Components: Hive
            Reporter: Aman Raj
            Assignee: Aman Raj


When current oss master hive tries to bring up the LLAP Application Master, it 
fails with this issue :
{code:java}
Executing the launch command\nINFO client.ServiceClient: Loading service 
definition from local FS: 
/var/lib/ambari-agent/tmp/llap-yarn-service_2023-01-10_07-56-46/Yarnfile\nERROR 
utils.JsonSerDeser: Exception while parsing json input 
stream\ncom.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot 
deserialize value of type 
`org.apache.hadoop.yarn.service.api.records.PlacementScope` from String 
\"NODE\": not one of the values accepted for Enum class: [node, rack]\n at 
[Source: (org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream); 
line: 31, column: 22] (through reference chain: 
org.apache.hadoop.yarn.service.api.records.Service[\"components\"]->java.util.ArrayList[0]->org.apache.hadoop.yarn.service.api.records.Component[\"placement_policy\"]->org.apache.hadoop.yarn.service.api.records.PlacementPolicy[\"constraints\"]->java.util.ArrayList[0]->org.apache.hadoop.yarn.service.api.records.PlacementConstraint[\"scope\"])\n\tat
 
com.fasterxml.jackson.databind.exc.InvalidFormatException.from(InvalidFormatException.java:67)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.DeserializationContext.weirdStringException(DeserializationContext.java:1851)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.DeserializationContext.handleWeirdStringValue(DeserializationContext.java:1079)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.std.EnumDeserializer._deserializeAltString(EnumDeserializer.java:339)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.std.EnumDeserializer._fromString(EnumDeserializer.java:214)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.std.EnumDeserializer.deserialize(EnumDeserializer.java:188)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:324)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:355)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:324)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:324)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:355)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:324)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4593)
 ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3585) 
~[jackson-databind-2.12.7.jar:2.12.7]\n\tat 
org.apache.hadoop.yarn.service.utils.JsonSerDeser.fromStream(JsonSerDeser.java:137)
 [hadoop-yarn-services-core-3.3.4.5.1-SNAPSHOT.jar:?]\n\tat 
org.apache.hadoop.yarn.service.utils.JsonSerDeser.load(JsonSerDeser.java:179) 
[hadoop-yarn-services-core-3.3.4.5.1-SNAPSHOT.jar:?]\n\tat 
org.apache.hadoop.yarn.service.client.ServiceClient.loadAppJsonFromLocalFS(ServiceClient.java:208)
 [hadoop-yarn-services-core-3.3.4.5.1-SNAPSHOT.jar:?]\n\tat 
org.apache.hadoop.yarn.service.client.ServiceClient.actionLaunch(ServiceClient.java:546)
 [hadoop-yarn-services-core-3.3.4.5.1-SNAPSHOT.jar:?]\n\tat 
org.apache.hadoop.hive.llap.cli.LlapSliderUtils.startCluster(LlapSliderUtils.java:102)
 [hive-llap-server-3.1.2.5.1-SNAPSHOT.jar:3.1.2.5.1-SNAPSHOT]\n\tat 
org.apache.hadoop.hive.llap.cli.LlapServiceDriver.run(LlapServiceDriver.java:601)
 [hive-llap-server-3.1.2.5.1-SNAPSHOT.jar:3.1.2.5.1-SNAPSHOT]\n\tat 
org.apache.hadoop.hive.llap.cli.LlapServiceDriver.main(LlapServiceDriver.java:119)
 [hive-llap-server-3.1.2.5.1-SNAPSHOT.jar:3.1.2.5.1-SNAPSHOT]\n {code}
This is because llap_server uses templates.py yarnfile to bringup the LLAP 
Application Master which has the placement_policy as "NODE". Hadoop has started 
using 2.12.7 jackson version which has issues in parsing NODE to either [node, 
rack]. Therefore it is not able to cast the placement policy correctly.

 

Resolution :

We will have to send placement_policy as "node" from the templates.py of Hive 
Llap Server module. Will raise a PR for this issue :



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to