Why does consuming a RESTful web service (using javax.ws.rs.* and Jsersey) work in unit test but not when submitted to Spark?

[Emre Sevinc]

Hello,

I have a piece of code that runs inside Spark Streaming and tries to get
some data from a RESTful web service (that runs locally on my machine). The
code snippet in question is:

     Client client = ClientBuilder.newClient();
     WebTarget target = client.target("http://localhost:2222/rest";);
     target = target.path("annotate")
                 .queryParam("text",
UrlEscapers.urlFragmentEscaper().escape(spotlightSubmission))
                 .queryParam("confidence", "0.3");

      logger.warn("!!! DEBUG !!! target: {}", target.getUri().toString());

      String response =
target.request().accept(MediaType.APPLICATION_JSON_TYPE).get(String.class);

      logger.warn("!!! DEBUG !!! Spotlight response: {}", response);

When run inside a unit test as follows:

     mvn clean test -Dtest=SpotlightTest#testCountWords

it contacts the RESTful web service and retrieves some data as expected.
But when the same code is run as part of the application that is submitted
to Spark, using spark-submit script I receive the following error:

      java.lang.NoSuchMethodError:
javax.ws.rs.core.MultivaluedMap.addAll(Ljava/lang/Object;[Ljava/lang/Object;)V

I'm using Spark 1.1.0 and for consuming the web service I'm using Jersey in
my project's pom.xml:

     <dependency>
      <groupId>org.glassfish.jersey.containers</groupId>
      <artifactId>jersey-container-servlet-core</artifactId>
      <version>2.14</version>
    </dependency>

So I suspect that when the application is submitted to Spark, somehow
there's a different JAR in the environment that uses a different version of
Jersey / javax.ws.rs.*

Does anybody know which version of Jersey / javax.ws.rs.*  is used in the
Spark environment, or how to solve this conflict?


-- 
Emre Sevinç
https://be.linkedin.com/in/emresevinc/