Is this merged to Spark ML ? If so which version ? On Tue, Sep 6, 2016 at 12:58 AM, Takeshi Yamamuro <linguin....@gmail.com> wrote:
> Hi, > > Sorry to bother you, but I'ld like to inform you our activities. > We'll start incubating our product, Hivemall, in Apache and this is a > scalable ML library > for Hive/Spark/Pig. > > - http://wiki.apache.org/incubator/HivemallProposal > - http://markmail.org/thread/mjwyyd4btthk3626 > > I made a pr for XGBoost integration on DataFrame/Spark(https:// > github.com/myui/hivemall/pull/281) > and this pr has already been merged in a master. > I wrote how to use the integration on my gist: > https://gist.github.com/maropu/33794b293ee937e99b8fb0788843fa3f > > If you are interested in the integration, could you please you try using > it and > let me know the issues that you get stuck in? > > Best regards, > takeshi > > // maropu > > > > On Mon, Aug 15, 2016 at 1:04 PM, Brandon White <bwwintheho...@gmail.com> > wrote: > >> The XGBoost integration with Spark is currently only supported for RDDs, >> there is a ticket for dataframe and folks calm to be working on it. >> >> On Aug 14, 2016 8:15 PM, "Jacek Laskowski" <ja...@japila.pl> wrote: >> >>> Hi, >>> >>> I've never worked with the library and speaking about sbt setup only. >>> >>> It appears that the project didn't release 2.11-compatible jars (only >>> 2.10) [1] so you need to build the project yourself and uber-jar it >>> (using sbt-assembly plugin). >>> >>> [1] https://spark-packages.org/package/rotationsymmetry/sparkxgboost >>> >>> Pozdrawiam, >>> Jacek Laskowski >>> ---- >>> https://medium.com/@jaceklaskowski/ >>> Mastering Apache Spark 2.0 http://bit.ly/mastering-apache-spark >>> Follow me at https://twitter.com/jaceklaskowski >>> >>> >>> On Sun, Aug 14, 2016 at 7:13 AM, janardhan shetty >>> <janardhan...@gmail.com> wrote: >>> > Any leads how to do acheive this? >>> > >>> > On Aug 12, 2016 6:33 PM, "janardhan shetty" <janardhan...@gmail.com> >>> wrote: >>> >> >>> >> I tried using sparkxgboost package in build.sbt file but it failed. >>> >> Spark 2.0 >>> >> Scala 2.11.8 >>> >> >>> >> Error: >>> >> [warn] >>> >> http://dl.bintray.com/spark-packages/maven/rotationsymmetry/ >>> sparkxgboost/0.2.1-s_2.10/sparkxgboost-0.2.1-s_2.10-javadoc.jar >>> >> [warn] :::::::::::::::::::::::::::::::::::::::::::::: >>> >> [warn] :: FAILED DOWNLOADS :: >>> >> [warn] :: ^ see resolution messages for details ^ :: >>> >> [warn] :::::::::::::::::::::::::::::::::::::::::::::: >>> >> [warn] :: >>> >> rotationsymmetry#sparkxgboost;0.2.1-s_2.10!sparkxgboost.jar(src) >>> >> [warn] :: >>> >> rotationsymmetry#sparkxgboost;0.2.1-s_2.10!sparkxgboost.jar(doc) >>> >> >>> >> build.sbt: >>> >> >>> >> scalaVersion := "2.11.8" >>> >> >>> >> libraryDependencies ++= { >>> >> val sparkVersion = "2.0.0-preview" >>> >> Seq( >>> >> "org.apache.spark" %% "spark-core" % sparkVersion % "provided", >>> >> "org.apache.spark" %% "spark-sql" % sparkVersion % "provided", >>> >> "org.apache.spark" %% "spark-streaming" % sparkVersion % >>> "provided", >>> >> "org.apache.spark" %% "spark-mllib" % sparkVersion % "provided" >>> >> ) >>> >> } >>> >> >>> >> resolvers += "Spark Packages Repo" at >>> >> "http://dl.bintray.com/spark-packages/maven"; >>> >> >>> >> libraryDependencies += "rotationsymmetry" % "sparkxgboost" % >>> >> "0.2.1-s_2.10" >>> >> >>> >> assemblyMergeStrategy in assembly := { >>> >> case PathList("META-INF", "MANIFEST.MF") => >>> >> MergeStrategy.discard >>> >> case PathList("javax", "servlet", xs @ _*) => >>> >> MergeStrategy.first >>> >> case PathList(ps @ _*) if ps.last endsWith ".html" => >>> >> MergeStrategy.first >>> >> case "application.conf" => >>> >> MergeStrategy.concat >>> >> case "unwanted.txt" => >>> >> MergeStrategy.discard >>> >> >>> >> case x => val oldStrategy = (assemblyMergeStrategy in >>> assembly).value >>> >> oldStrategy(x) >>> >> >>> >> } >>> >> >>> >> >>> >> >>> >> >>> >> On Fri, Aug 12, 2016 at 3:35 PM, janardhan shetty < >>> janardhan...@gmail.com> >>> >> wrote: >>> >>> >>> >>> Is there a dataframe version of XGBoost in spark-ml ?. >>> >>> Has anyone used sparkxgboost package ? >>> >> >>> >> >>> > >>> >>> --------------------------------------------------------------------- >>> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >>> >>> > > > -- > --- > Takeshi Yamamuro >
