Hi, I am using Hive 0.13 in Amazon EMR. I am stuck up with the problem of hive-exec jar being bundled with older protobuf buffers java of version 2.5. Please help me in getting myself unblocked from this problem.
We have developed a Cusom SerDe which is in turn dependent on protobuf-buffers-java package of version 2.6 and the jar of this SerDe is built by bundling the protobuf-buffers-java of version 2.6. When I try to create a table using my custom serde, it fails in parsing the data with the following error. This is because the method that is used from protobuf in my custom serde is not available in protobuf version 2.5. Even though i have bundled protobuf of version 2.6 with my custom serde, it does not seem to use that rather uses the protobuf of version 2.5 that comes bundled with hive-exec jar. I tried adding the protobuf 2.6 jar explicitly on hive console and that does not help. How do i make my serde use the protobuf version bundled within my serde? *Exception in thread "main" java.lang.NoSuchMethodError: com.google.protobuf.LazyStringList.getUnmodifiableView()Lcom/google/protobuf/LazyStringList;* * at com.amazon.transportation.ate.aas.AddressAttributeTypes$GateCodeAttribute.<init>(AddressAttributeTypes.java:3451)* * at com.amazon.transportation.ate.aas.AddressAttributeTypes$GateCodeAttribute.<init>(AddressAttributeTypes.java:3384)* * at com.amazon.transportation.ate.aas.AddressAttributeTypes$GateCodeAttribute$1.parsePartialFrom(AddressAttributeTypes.java:3475)* * at com.amazon.transportation.ate.aas.AddressAttributeTypes$GateCodeAttribute$1.parsePartialFrom(AddressAttributeTypes.java:3470)* * at com.google.protobuf.CodedInputStream.readMessage(CodedInputStream.java:309)* * at com.amazon.transportation.ate.aas.AddressAttributeTypes$Attribute.<init>(AddressAttributeTypes.java:961)* * at com.amazon.transportation.ate.aas.AddressAttributeTypes$Attribute.<init>(AddressAttributeTypes.java:862)* * at com.amazon.transportation.ate.aas.AddressAttributeTypes$Attribute$1.parsePartialFrom(AddressAttributeTypes.java:1051)* * at com.amazon.transportation.ate.aas.AddressAttributeTypes$Attribute$1.parsePartialFrom(AddressAttributeTypes.java:1046)* * at com.google.protobuf.CodedInputStream.readMessage(CodedInputStream.java:309)* * at com.amazon.transportation.ate.aas.AtePlacesExchanges$AteAddressAttributeNotification.<init>(AtePlacesExchanges.java:883)* * at com.amazon.transportation.ate.aas.AtePlacesExchanges$AteAddressAttributeNotification.<init>(AtePlacesExchanges.java:811)* * at com.amazon.transportation.ate.aas.AtePlacesExchanges$AteAddressAttributeNotification$1.parsePartialFrom(AtePlacesExchanges.java:919)* * at com.amazon.transportation.ate.aas.AtePlacesExchanges$AteAddressAttributeNotification$1.parsePartialFrom(AtePlacesExchanges.java:914)* * at com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:141)* * at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:176)* * at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:188)* * at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:193)* * at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:49)* * at com.amazon.transportation.ate.aas.AtePlacesExchanges$AteAddressAttributeNotification.parseFrom(AtePlacesExchanges.java:1134)* * at com.amazon.places.protobuf.decoder.NotificationsProtoDecoder.<init>(NotificationsProtoDecoder.java:57)* * at com.amazon.places.serde.AtePlacesNotificationSerde.deserialize(AtePlacesNotificationSerde.java:103)* * at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:620)* * at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:534)* * at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:137)* * at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1519)* * at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:292)* * at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:227)* * at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:430)* * at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:803)* * at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:697)* * at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:636)* * at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)* * at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)* * at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)* * at java.lang.reflect.Method.invoke(Method.java:606)* * at org.apache.hadoop.util.RunJar.main(RunJar.java:212)* Please help me with the solution to my problem. Thanks, Kalai
