Of course, 15 minutes after I give up and decide to email the mailing list, I figure it out - my flink App was using the CollectionsEnvironment and not the proper RemoteEnvironment.
It is still the case, however, that the `flink-table` JAR built by the standard commands doesn't include the dependencies it requires, and so I'd be curious to hear what the proper procedure is for linking against `flink-table` if you want to avoid the bug I highlighted in the aforementioned JIRA. Thank you and sorry for the extra noise! Justin On Tue, Mar 7, 2017 at 7:21 PM, Justin Yan <justin....@remitly.com> wrote: > Hello! > > We are attempting to use the Flink Table API, but are running into a few > issues. > > We initially started with our dependencies looking something like this: > > libraryDependencies ++= Seq( > "org.apache.flink" %% "flink-scala" % "1.2.0" % "provided", > "org.apache.flink" %% "flink-clients" % "1.2.0" % "provided", > "org.apache.flink" %% "flink-table" % "1.2.0", > Libraries.specs2, > ...) > > However, this is mildly annoying since flink-table declares dependencies > on the flink core modules, and thus brings everything in *anyway*. On > top of that, we saw this JIRA: https://issues.apache. > org/jira/browse/FLINK-5227, which we found concerning, so we decided to > follow the advice - we downloaded and built Flink-1.2 from source (java 7, > mvn 3.3) using the following (We're also using the Kinesis connector): > > tools/change-scala-version.sh 2.11 > mvn clean install -Pinclude-kinesis -DskipTests > cd flink-dist > mvn clean install -Pinclude-kinesis -DskipTests > > Once this was done, we took the JAR in "/flink-libraries/flink-table/target/" > and copied it over to the taskManager "/lib" directory. Finally, we marked > our `flink-table` dependency as "provided". Everything compiles, but when > I try to actually run a simple job, I get the following error at runtime: > > java.lang.NoClassDefFoundError: org/codehaus/commons/compiler/ > CompileException > > Indeed, when I peek inside of the `flink-table` JAR, I can't find that > particular package (and similarly, it isn't in the flink-dist JAR either) > > $ jar tf flink-table_2.11-1.2.0.jar | grep codehaus > $ > > I then attempted to include this library in my user code by adding: > > "org.codehaus.janino" % "janino" % "3.0.6", > > to my list of dependencies. When I run a `jar tf myjar.jar | grep > CompileException` - I see the class. However, when I run my flink > application in this fat JAR, I *continue to get the same error*, even > though I am positive this class is included in the fat JAR. I eventually > got around this by placing this jar in the `flink/lib` directory, but I am > very confused as to how this class cannot be found when I have included it > in the fat JAR that I am submitting with the Flink CLI to a YARN cluster. > I mostly wanted to mention this in case it is a bug, but mostly to see if > anyone else has had trouble with the Table API, and if not, if I have > structured my project incorrectly to cause these troubles. > > Thanks! > > Justin > > > >
