Hey Guys need your help again,
I am currently having problems with the multiple linear regression from the
flink-ml on the HDFS.
Locally it works fine with the 0.9-SNAPSHOT. The cluster runs with the
0.10-SNAPSHOT. The code is the following:
// set linear regression with parameters:
val mlr = MultipleLinearRegression()
.setStepsize(0.001)
.setIterations(1000000000)
.setConvergenceThreshold(0.001)
// do linear regression and time the method
val model = mlr.fit(transformTrain)
// The fitted model can now be used to make predictions
val predictions = mlr.predict(tranformTest)
The dataset transformTrain has the following form (filled with doubles):
LabeledVector(numList(0), DenseVector(numList(1),numList(2)))
Mainly the line where the fit method (mlr.fit) is called causes the following
error:
An error occurred while invoking the program:
The program caused an error:
java.lang.NoClassDefFoundError: breeze/storage/Zero
at org.apache.flink.ml.pipeline.Estimator$class.fit(Estimator.scala:53)
at
org.apache.flink.ml.regression.MultipleLinearRegression.fit(MultipleLinearRegression.scala:88)
at Regression2$.buildModelRegression(Regression2.scala:37)
at
Regression2$$anonfun$mainRegression$1.apply$mcVI$sp(Regression2.scala:116)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141)
at Regression2$.mainRegression(Regression2.scala:103)
at MainClass$.main(MainClass.scala:47)
at MainClass.main(MainClass.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:437)
at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:353)
at
org.apache.flink.client.program.Client.getOptimizedPlan(Client.java:192)
at org.apache.flink.client.CliFrontend.info(CliFrontend.java:399)
at
org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:959)
at
org.apache.flink.client.web.JobSubmissionServlet.doGet(JobSubmissionServlet.java:174)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:734)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:532)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:965)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:388)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:187)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:901)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at
org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:47)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:113)
at org.eclipse.jetty.server.Server.handle(Server.java:352)
at
org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:596)
at
org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1048)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:549)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:211)
at
org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:425)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:489)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:436)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: breeze.storage.Zero
at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 39 more
Thanks for any help.
Best wishes,
Flo
