Tried to implement spark package in 2.0 https://spark-packages.org/package/rotationsymmetry/sparkxgboost but it is throwing the error:
error: not found: type SparkXGBoostClassifier On Tue, Sep 6, 2016 at 11:26 AM, janardhan shetty <janardhan...@gmail.com> wrote: > Is this merged to Spark ML ? If so which version ? > > On Tue, Sep 6, 2016 at 12:58 AM, Takeshi Yamamuro <linguin....@gmail.com> > wrote: > >> Hi, >> >> Sorry to bother you, but I'ld like to inform you our activities. >> We'll start incubating our product, Hivemall, in Apache and this is a >> scalable ML library >> for Hive/Spark/Pig. >> >> - http://wiki.apache.org/incubator/HivemallProposal >> - http://markmail.org/thread/mjwyyd4btthk3626 >> >> I made a pr for XGBoost integration on DataFrame/Spark(https://github >> .com/myui/hivemall/pull/281) >> and this pr has already been merged in a master. >> I wrote how to use the integration on my gist: >> https://gist.github.com/maropu/33794b293ee937e99b8fb0788843fa3f >> >> If you are interested in the integration, could you please you try using >> it and >> let me know the issues that you get stuck in? >> >> Best regards, >> takeshi >> >> // maropu >> >> >> >> On Mon, Aug 15, 2016 at 1:04 PM, Brandon White <bwwintheho...@gmail.com> >> wrote: >> >>> The XGBoost integration with Spark is currently only supported for RDDs, >>> there is a ticket for dataframe and folks calm to be working on it. >>> >>> On Aug 14, 2016 8:15 PM, "Jacek Laskowski" <ja...@japila.pl> wrote: >>> >>>> Hi, >>>> >>>> I've never worked with the library and speaking about sbt setup only. >>>> >>>> It appears that the project didn't release 2.11-compatible jars (only >>>> 2.10) [1] so you need to build the project yourself and uber-jar it >>>> (using sbt-assembly plugin). >>>> >>>> [1] https://spark-packages.org/package/rotationsymmetry/sparkxgboost >>>> >>>> Pozdrawiam, >>>> Jacek Laskowski >>>> ---- >>>> https://medium.com/@jaceklaskowski/ >>>> Mastering Apache Spark 2.0 http://bit.ly/mastering-apache-spark >>>> Follow me at https://twitter.com/jaceklaskowski >>>> >>>> >>>> On Sun, Aug 14, 2016 at 7:13 AM, janardhan shetty >>>> <janardhan...@gmail.com> wrote: >>>> > Any leads how to do acheive this? >>>> > >>>> > On Aug 12, 2016 6:33 PM, "janardhan shetty" <janardhan...@gmail.com> >>>> wrote: >>>> >> >>>> >> I tried using sparkxgboost package in build.sbt file but it failed. >>>> >> Spark 2.0 >>>> >> Scala 2.11.8 >>>> >> >>>> >> Error: >>>> >> [warn] >>>> >> http://dl.bintray.com/spark-packages/maven/rotationsymmetry/ >>>> sparkxgboost/0.2.1-s_2.10/sparkxgboost-0.2.1-s_2.10-javadoc.jar >>>> >> [warn] :::::::::::::::::::::::::::::::::::::::::::::: >>>> >> [warn] :: FAILED DOWNLOADS :: >>>> >> [warn] :: ^ see resolution messages for details ^ :: >>>> >> [warn] :::::::::::::::::::::::::::::::::::::::::::::: >>>> >> [warn] :: >>>> >> rotationsymmetry#sparkxgboost;0.2.1-s_2.10!sparkxgboost.jar(src) >>>> >> [warn] :: >>>> >> rotationsymmetry#sparkxgboost;0.2.1-s_2.10!sparkxgboost.jar(doc) >>>> >> >>>> >> build.sbt: >>>> >> >>>> >> scalaVersion := "2.11.8" >>>> >> >>>> >> libraryDependencies ++= { >>>> >> val sparkVersion = "2.0.0-preview" >>>> >> Seq( >>>> >> "org.apache.spark" %% "spark-core" % sparkVersion % "provided", >>>> >> "org.apache.spark" %% "spark-sql" % sparkVersion % "provided", >>>> >> "org.apache.spark" %% "spark-streaming" % sparkVersion % >>>> "provided", >>>> >> "org.apache.spark" %% "spark-mllib" % sparkVersion % "provided" >>>> >> ) >>>> >> } >>>> >> >>>> >> resolvers += "Spark Packages Repo" at >>>> >> "http://dl.bintray.com/spark-packages/maven" >>>> >> >>>> >> libraryDependencies += "rotationsymmetry" % "sparkxgboost" % >>>> >> "0.2.1-s_2.10" >>>> >> >>>> >> assemblyMergeStrategy in assembly := { >>>> >> case PathList("META-INF", "MANIFEST.MF") => >>>> >> MergeStrategy.discard >>>> >> case PathList("javax", "servlet", xs @ _*) => >>>> >> MergeStrategy.first >>>> >> case PathList(ps @ _*) if ps.last endsWith ".html" => >>>> >> MergeStrategy.first >>>> >> case "application.conf" => >>>> >> MergeStrategy.concat >>>> >> case "unwanted.txt" => >>>> >> MergeStrategy.discard >>>> >> >>>> >> case x => val oldStrategy = (assemblyMergeStrategy in >>>> assembly).value >>>> >> oldStrategy(x) >>>> >> >>>> >> } >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> On Fri, Aug 12, 2016 at 3:35 PM, janardhan shetty < >>>> janardhan...@gmail.com> >>>> >> wrote: >>>> >>> >>>> >>> Is there a dataframe version of XGBoost in spark-ml ?. >>>> >>> Has anyone used sparkxgboost package ? >>>> >> >>>> >> >>>> > >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >>>> >>>> >> >> >> -- >> --- >> Takeshi Yamamuro >> > >