I've created a pull request for the fix: https://github.com/apache/flink/pull/3556
It would be nice if one of the issue reporters could validate that the cassandra connector works after the fix. If it is a valid fix, I would like to include it into the upcoming 1.2.1 release. On Thu, Mar 16, 2017 at 6:08 PM, Robert Metzger <rmetz...@apache.org> wrote: > Yep, this is definitively a bug / misconfiguration in the build system. > > The cassandra client defines metrics-core as a dependency, but the shading > is dropping the dependency when building the dependency reduced pom. > To resolve the issue, we need to add the following line into the shading > config of the cassandra module: > > <promoteTransitiveDependencies>true</promoteTransitiveDependencies> > > This makes the metrics dependency appear again in the dep red pom. > > > I've filed a JIRA: https://issues.apache.org/jira/browse/FLINK-6084 and > will open a PR. > > On Thu, Mar 16, 2017 at 1:08 PM, Stephan Ewen <se...@apache.org> wrote: > >> Can we improve the Flink experience here by adding this dependency >> directly to the cassandra connector pom.xml (so that user jars always pull >> it in via transitivity)? >> >> On Wed, Mar 15, 2017 at 4:09 PM, Nico <nico.franz...@gmail.com> wrote: >> >>> Hi @all, >>> >>> I came back to this issue today... >>> >>> @Robert: >>> "com/codahale/metrics/Metric" class was not available in the user code >>> jar >>> >>> Even after adding the metric class into the build-jar profile of the pom >>> file, more "class not found" errors occur. So the only solution was to add >>> the whole dependency: >>> >>> <dependency> >>> <groupId>com.codahale.metrics</groupId> >>> <artifactId>metrics-core</artifactId> >>> <version>3.0.2</version> >>> </dependency> >>> >>> >>> This worked for me. >>> >>> Best, >>> Nico >>> >>> >>> >>> 2017-03-06 11:46 GMT+01:00 Chesnay Schepler <ches...@apache.org>: >>> >>>> Hello, >>>> >>>> i believe the cassandra connector is not shading it's dependencies >>>> properly. This didn't cause issues in the >>>> past since flink used to have a dependency on codahale metrics as well. >>>> >>>> Please open a JIRA for this issue. >>>> >>>> Regards, >>>> Chesnay >>>> >>>> >>>> On 06.03.2017 11:32, Tarandeep Singh wrote: >>>> >>>> Hi Robert & Nico, >>>> >>>> I am facing the same problem (java.lang.NoClassDefFoundError: >>>> com/codahale/metrics/Metric) >>>> Can you help me identify shading issue in pom.xml file. >>>> >>>> My pom.xml content- >>>> --------------------------------------------- >>>> >>>> <project xmlns="http://maven.apache.org/POM/4.0.0"; >>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >>>> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 >>>> http://maven.apache.org/xsd/maven-4.0.0.xsd";> >>>> <modelVersion>4.0.0</modelVersion> >>>> <groupId>rfk-dataplatform</groupId> >>>> <artifactId>stream-processing</artifactId> <version>0.1.0</version> >>>> <packaging>jar</packaging> <name>Stream processing</name> <properties> >>>> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> >>>> <flink.version>1.2.0</flink.version> >>>> <slf4j.version>1.7.7</slf4j.version> >>>> <log4j.version>1.2.17</log4j.version> </properties> <dependencies> >>>> <dependency> <groupId>org.apache.flink</groupId> >>>> <artifactId>flink-streaming-java_2.10</artifactId> >>>> <version>${flink.version}</version> </dependency> <dependency> >>>> <groupId>org.apache.flink</groupId> >>>> <artifactId>flink-clients_2.10</artifactId> >>>> <version>${flink.version}</version> </dependency> <dependency> >>>> <groupId>org.apache.flink</groupId> >>>> <artifactId>flink-connector-cassandra_2.10</artifactId> >>>> <version>1.2.0</version> </dependency> <dependency> >>>> <groupId>org.apache.flink</groupId> >>>> <artifactId>flink-statebackend-rocksdb_2.10</artifactId> >>>> <version>1.2.0</version> </dependency> <dependency> >>>> <groupId>org.slf4j</groupId> >>>> <artifactId>slf4j-log4j12</artifactId> >>>> <version>${slf4j.version}</version> </dependency> <dependency> >>>> <groupId>log4j</groupId> <artifactId>log4j</artifactId> >>>> <version>${log4j.version}</version> </dependency> >>>> <dependency> <groupId>org.apache.avro</groupId> >>>> <artifactId>avro</artifactId> <version>1.8.1</version> >>>> </dependency> <dependency> <groupId>org.testng</groupId> >>>> <artifactId>testng</artifactId> >>>> <version>6.8</version> <scope>test</scope> </dependency> >>>> <!-- flink kafka connector --> <dependency> >>>> <groupId>org.apache.flink</groupId> >>>> <artifactId>flink-connector-kafka-0.8_2.10</artifactId> >>>> <version>1.2.0</version> </dependency> <!-- java influxdb >>>> client --> <dependency> <groupId>org.influxdb</groupId> >>>> <artifactId>influxdb-java</artifactId> >>>> <version>2.5</version> </dependency> </dependencies> >>>> <profiles> <profile> <!-- Profile for packaging correct JAR >>>> files --> <id>build-jar</id> <activation> >>>> <activeByDefault>false</activeByDefault> </activation> >>>> <dependencies> <dependency> >>>> <groupId>org.apache.flink</groupId> >>>> <artifactId>flink-java</artifactId> >>>> <version>${flink.version}</version> <scope>provided</scope> >>>> </dependency> <dependency> >>>> <groupId>org.apache.flink</groupId> >>>> <artifactId>flink-streaming-java_2.10</artifactId> >>>> <version>${flink.version}</version> <scope>provided</scope> >>>> </dependency> <dependency> >>>> <groupId>org.apache.flink</groupId> >>>> <artifactId>flink-clients_2.10</artifactId> >>>> <version>${flink.version}</version> <scope>provided</scope> >>>> </dependency> <dependency> >>>> <groupId>org.slf4j</groupId> >>>> <artifactId>slf4j-log4j12</artifactId> >>>> <version>${slf4j.version}</version> <scope>provided</scope> >>>> </dependency> <dependency> >>>> <groupId>log4j</groupId> <artifactId>log4j</artifactId> >>>> <version>${log4j.version}</version> >>>> <scope>provided</scope> </dependency> </dependencies> >>>> <build> <plugins> <!-- disable the >>>> exclusion rules --> <plugin> >>>> <groupId>org.apache.maven.plugins</groupId> >>>> <artifactId>maven-shade-plugin</artifactId> >>>> <version>2.4.1</version> <executions> >>>> <execution> <phase>package</phase> >>>> <goals> <goal>shade</goal> >>>> </goals> <configuration> >>>> <artifactSet> <excludes >>>> combine.self="override"></excludes> >>>> </artifactSet> </configuration> >>>> </execution> </executions> </plugin> >>>> </plugins> </build> </profile> </profiles> <build> >>>> <plugins> <plugin> >>>> <groupId>org.apache.maven.plugins</groupId> >>>> <artifactId>maven-shade-plugin</artifactId> >>>> <version>2.4.1</version> <executions> <!-- Run >>>> shade goal on package phase --> <execution> >>>> <phase>package</phase> <goals> >>>> <goal>shade</goal> </goals> >>>> <configuration> <artifactSet> >>>> <excludes> <!-- This list contains all >>>> dependencies of flink-dist Everything else will >>>> be packaged into the fat-jar --> >>>> <exclude>org.apache.flink:flink-annotations</exclude> >>>> <exclude>org.apache.flink:flink-shaded-hadoop2</exclude> >>>> >>>> <exclude>org.apache.flink:flink-shaded-curator-recipes</exclude> >>>> <exclude>org.apache.flink:flink-core</exclude> >>>> <exclude>org.apache.flink:flink-java</exclude> >>>> <exclude>org.apache.flink:flink-scala_2.10</exclude> >>>> <exclude>org.apache.flink:flink-runtime_2.10</exclude> >>>> >>>> <exclude>org.apache.flink:flink-optimizer_2.10</exclude> >>>> <exclude>org.apache.flink:flink-clients_2.10</exclude> >>>> <exclude>org.apache.flink:flink-avro_2.10</exclude> >>>> >>>> <exclude>org.apache.flink:flink-examples-batch_2.10</exclude> >>>> >>>> <exclude>org.apache.flink:flink-examples-streaming_2.10</exclude> >>>> >>>> <exclude>org.apache.flink:flink-streaming-java_2.10</exclude> >>>> >>>> <exclude>org.apache.flink:flink-streaming-scala_2.10</exclude> >>>> <exclude>org.apache.flink:flink-scala-shell_2.10</exclude> >>>> <exclude>org.apache.flink:flink-python</exclude> >>>> >>>> <exclude>org.apache.flink:flink-metrics-core</exclude> >>>> <exclude>org.apache.flink:flink-metrics-jmx</exclude> >>>> >>>> <exclude>org.apache.flink:flink-statebackend-rocksdb_2.10</exclude> >>>> <!-- Also exclude very big transitive dependencies of >>>> Flink WARNING: You have to remove these excludes >>>> if your code relies on other versions of these >>>> dependencies. --> >>>> <exclude>log4j:log4j</exclude> >>>> <exclude>org.scala-lang:scala-library</exclude> >>>> <exclude>org.scala-lang:scala-compiler</exclude> >>>> <exclude>org.scala-lang:scala-reflect</exclude> >>>> <exclude>com.data-artisans:flakka-actor_*</exclude> >>>> <exclude>com.data-artisans:flakka-remote_*</exclude> >>>> <exclude>com.data-artisans:flakka-slf4j_*</exclude> >>>> <exclude>io.netty:netty-all</exclude> >>>> <exclude>io.netty:netty</exclude> >>>> <exclude>commons-fileupload:commons-fileupload</exclude> >>>> <exclude>org.apache.avro:avro</exclude> >>>> <exclude>commons-collections:commons-collections</exclude> >>>> <exclude>org.codehaus.jackson:jackson-core-asl</exclude> >>>> >>>> <exclude>org.codehaus.jackson:jackson-mapper-asl</exclude> >>>> <exclude>com.thoughtworks.paranamer:paranamer</exclude> >>>> <exclude>org.xerial.snappy:snappy-java</exclude> >>>> <exclude>org.apache.commons:commons-compress</exclude> >>>> <exclude>org.tukaani:xz</exclude> >>>> <exclude>com.esotericsoftware.kryo:kryo</exclude> >>>> <exclude>com.esotericsoftware.minlog:minlog</exclude> >>>> <exclude>org.objenesis:objenesis</exclude> >>>> <exclude>com.twitter:chill_*</exclude> >>>> <exclude>com.twitter:chill-java</exclude> >>>> <exclude>commons-lang:commons-lang</exclude> >>>> <exclude>junit:junit</exclude> >>>> <exclude>org.apache.commons:commons-lang3</exclude> >>>> <exclude>org.slf4j:slf4j-api</exclude> >>>> <exclude>org.slf4j:slf4j-log4j12</exclude> >>>> <exclude>log4j:log4j</exclude> >>>> <exclude>org.apache.commons:commons-math</exclude> >>>> <exclude>org.apache.sling:org.apache.sling.commons.json</exclude> >>>> <exclude>commons-logging:commons-logging</exclude> >>>> <exclude>commons-codec:commons-codec</exclude> >>>> >>>> <exclude>com.fasterxml.jackson.core:jackson-core</exclude> >>>> <exclude>com.fasterxml.jackson.core:jackson-databind</exclude> >>>> >>>> <exclude>com.fasterxml.jackson.core:jackson-annotations</exclude> >>>> <exclude>stax:stax-api</exclude> >>>> <exclude>com.typesafe:config</exclude> >>>> <exclude>org.uncommons.maths:uncommons-maths</exclude> >>>> <exclude>com.github.scopt:scopt_*</exclude> >>>> <exclude>commons-io:commons-io</exclude> >>>> <exclude>commons-cli:commons-cli</exclude> >>>> </excludes> </artifactSet> >>>> <filters> <filter> >>>> <artifact>org.apache.flink:*</artifact> >>>> <excludes> <!-- exclude shaded google but >>>> include shaded curator --> >>>> <exclude>org/apache/flink/shaded/com/**</exclude> >>>> <exclude>web-docs/**</exclude> </excludes> >>>> </filter> <filter> >>>> <artifact>*:*</artifact> >>>> <excludes> <exclude>META-INF/*.SF</exclude> >>>> <exclude>META-INF/*.DSA</exclude> >>>> <exclude>META-INF/*.RSA</exclude> >>>> </excludes> </filter> >>>> </filters> >>>> <createDependencyReducedPom>false</createDependencyReducedPom> >>>> </configuration> </execution> </executions> >>>> </plugin> <plugin> >>>> <groupId>org.apache.maven.plugins</groupId> >>>> <artifactId>maven-compiler-plugin</artifactId> >>>> <version>3.1</version> <configuration> >>>> <source>1.8</source> <target>1.8</target> >>>> </configuration> </plugin> <plugin> >>>> <groupId>org.apache.avro</groupId> >>>> <artifactId>avro-maven-plugin</artifactId> >>>> <version>1.8.1</version> <executions> >>>> <execution> <phase>generate-sources</phase> >>>> <goals> <goal>schema</goal> >>>> </goals> <configuration> >>>> <stringType>String</stringType> >>>> <fieldVisibility>PRIVATE</fieldVisibility> >>>> <sourceDirectory>${project.basedir}/src/main/schema/</sourceDirectory> >>>> >>>> <outputDirectory>${project.basedir}/src/main/java/</outputDirectory> >>>> </configuration> </execution> >>>> </executions> </plugin> <plugin> >>>> <groupId>org.apache.maven.plugins</groupId> >>>> <artifactId>maven-compiler-plugin</artifactId> >>>> <configuration> <source>1.8</source> >>>> <target>1.8</target> </configuration> </plugin> >>>> </plugins> </build></project> >>>> >>>> >>>> --------------------------------------------- >>>> >>>> >>>> On Sun, Feb 12, 2017 at 1:56 AM, Robert Metzger <rmetz...@apache.org> >>>> wrote: >>>> >>>>> Hi Nico, >>>>> The cassandra connector should be available on Maven central: >>>>> http://search.maven.org/#artifactdetails%7Corg.apac >>>>> he.flink%7Cflink-connector-cassandra_2.10%7C1.2.0%7Cjar >>>>> >>>>> Potentially, the issue you've mentioned is due to some shading issue. >>>>> Is the "com/codahale/metrics/Metric" class in your user code jar? >>>>> >>>>> On Thu, Feb 9, 2017 at 2:56 PM, Nico < <nico.franz...@gmail.com> >>>>> nico.franz...@gmail.com> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I would like to upgrade to the new stable version 1.2 - but i get an >>>>>> ClassNotFound exception when i start the application. >>>>>> >>>>>> Caused by: java.lang.NoClassDefFoundError: >>>>>> com/codahale/metrics/Metric >>>>>> at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1 >>>>>> 367) >>>>>> at com.datastax.driver.core.Cluster.init(Cluster.java:162) >>>>>> at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:3 >>>>>> 33) >>>>>> at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:3 >>>>>> 08) >>>>>> at com.datastax.driver.core.Cluster.connect(Cluster.java:250) >>>>>> at org.apache.flink.streaming.connectors.cassandra.CassandraSin >>>>>> kBase.open(CassandraSinkBase.java:67) >>>>>> at org.apache.flink.streaming.connectors.cassandra.CassandraTup >>>>>> leSink.open(CassandraTupleSink.java:42) >>>>>> at org.apache.flink.api.common.functions.util.FunctionUtils.ope >>>>>> nFunction(FunctionUtils.java:36) >>>>>> at org.apache.flink.streaming.api.operators.AbstractUdfStreamOp >>>>>> erator.open(AbstractUdfStreamOperator.java:112) >>>>>> at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllO >>>>>> perators(StreamTask.java:386) >>>>>> at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(S >>>>>> treamTask.java:262) >>>>>> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:655) >>>>>> at java.lang.Thread.run(Thread.java:745) >>>>>> >>>>>> >>>>>> So I think the cassandra connector is the reason for it. Moreover, i >>>>>> don't see a version 1.2 in the maven repository for the connector as >>>>>> mentioned in the doc. >>>>>> >>>>>> <dependency> >>>>>> <groupId>org.apache.flink</groupId> >>>>>> <artifactId>flink-connector-cassandra_2.10</artifactId> >>>>>> <version>1.2.0</version> >>>>>> </dependency> >>>>>> >>>>>> Is there a plan to release a new version? >>>>>> >>>>>> Best, >>>>>> Nico >>>>>> >>>>> >>>>> >>>> >>>> >>> >> >
