Maziyar PANAHI created SPARK-22380:
--------------------------------------
Summary: Upgrade protobuf-java (com.google.protobuf) version from
2.5.0 to 3.4.0
Key: SPARK-22380
URL: https://issues.apache.org/jira/browse/SPARK-22380
Project: Spark
Issue Type: Dependency upgrade
Components: Deploy
Affects Versions: 2.2.0, 1.6.1
Environment: Cloudera 5.13.x
Spark 2.2.0.cloudera1-1.cdh5.12.0.p0.142354
And anything beyond Spark 2.2.0
Reporter: Maziyar PANAHI
Priority: Blocker
Hi,
This upgrade is needed when we try to use CoreNLP 3.8 with Spark (1.6+ and
2.2+) due to incompatibilities in the protobuf version used by
com.google.protobuf and the one is used in latest Stanford CoreNLP (3.8). The
version of protobuf has been set to 2.5.0 in the global properties, and this is
stated in the pom.xml file.
The error that refers to this dependency:
{code:java}
java.lang.VerifyError: Bad type on operand stack
Exception Details:
Location:
com/google/protobuf/GeneratedMessageV3$ExtendableMessage.getExtension(Lcom/google/protobuf/GeneratedMessage$GeneratedExtension;I)Ljava/lang/Object;
@3: invokevirtual
Reason:
Type 'com/google/protobuf/GeneratedMessage$GeneratedExtension' (current
frame, stack[1]) is not assignable to 'com/google/protobuf/ExtensionLite'
Current Frame:
bci: @3
flags: { }
locals: { 'com/google/protobuf/GeneratedMessageV3$ExtendableMessage',
'com/google/protobuf/GeneratedMessage$GeneratedExtension', integer }
stack: { 'com/google/protobuf/GeneratedMessageV3$ExtendableMessage',
'com/google/protobuf/GeneratedMessage$GeneratedExtension', integer }
Bytecode:
0x0000000: 2a2b 1cb6 0024 b0
at edu.stanford.nlp.simple.Document.<init>(Document.java:433)
at edu.stanford.nlp.simple.Sentence.<init>(Sentence.java:118)
at edu.stanford.nlp.simple.Sentence.<init>(Sentence.java:126)
... 56 elided
{code}
Is it possible to upgrade this dependency to the latest (3.4) or any workaround
besides manually removing protobuf-java-2.5.0.jar and adding
protobuf-java-3.4.0.jar?
You can follow the discussion of how this upgrade would fix the issue:
https://github.com/stanfordnlp/CoreNLP/issues/556
Many thanks,
Maziyar
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
