+1 Tested on OSX
Tested Scala 2.10.3, SparkSQL with Hive 0.12 / Hadoop 2.5, Thrift Server,
MLLib SVD
On Fri Dec 12 2014 at 8:57:16 PM Mark Hamstra
wrote:
> +1
>
> On Fri, Dec 12, 2014 at 8:00 PM, Josh Rosen wrote:
> >
> > +1. Tested using spark-perf and the Spark EC2 scripts. I didn’t notic
+1
On Fri, Dec 12, 2014 at 8:00 PM, Josh Rosen wrote:
>
> +1. Tested using spark-perf and the Spark EC2 scripts. I didn’t notice
> any performance regressions that could not be attributed to changes of
> default configurations. To be more specific, when running Spark 1.2.0 with
> the Spark 1.1
+1. Tested using spark-perf and the Spark EC2 scripts. I didn’t notice any
performance regressions that could not be attributed to changes of default
configurations. To be more specific, when running Spark 1.2.0 with the Spark
1.1.0 settings of spark.shuffle.manager=hash and
spark.shuffle.bl
Do we have one-hot encoding in spark MLLib 1.1.1 or 1.2.0 ? It wasn't
available in 1.1.0.
Thanks.
-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org
Hi Sam,
We developed the Spark Kernel with a focus on the newest version of the
IPython message protocol (5.0) for the upcoming IPython 3.0 release.
We are building around Apache Spark's REPL, which is used in the current
Spark Shell implementation.
The Spark Kernel was designed to be extensibl
Okay, I got it. In Estimator, fit(dataset: SchemaRDD, paramMaps:
Array[ParamMap]): Seq[M] can be overwritten to implement
regularization path. Correct me if I'm wrong.
Sincerely,
DB Tsai
---
My Blog: https://www.dbtsai.com
LinkedIn: https://www.
protobuf comes from missing -Phadoop2.3
On Fri, Dec 12, 2014 at 2:34 PM, Sean Owen wrote:
>
> What errors do you see? protobuf errors usually mean you didn't build
> for the right version of Hadoop, but if you are using -Phadoop-2.3 or
> better -Phadoop-2.4 that should be fine. Yes, a stack trace
What errors do you see? protobuf errors usually mean you didn't build
for the right version of Hadoop, but if you are using -Phadoop-2.3 or
better -Phadoop-2.4 that should be fine. Yes, a stack trace would be
good. I'm still not sure what error you are seeing.
On Fri, Dec 12, 2014 at 10:32 PM, Gan
Hi Sean - I should clarify : I was able to build the master but when running I
hit really random looking protobuf errors (just starting up a spark shell), I
can try doing a build later today and give the exact stack trace.
I know that 5.2 is running 1.1 but I believe the latest and greatest Ml L
We are happy to announce a developer preview of the Spark Kernel which
enables remote applications to dynamically interact with Spark. You can
think of the Spark Kernel as a remote Spark Shell that uses the IPython
notebook interface to provide a common entrypoint for any application. The
Spark
Could you specify what problems you're seeing? there is nothing
special about the CDH distribution at all.
The latest and greatest is 1.1, and that is what is in CDH 5.2. You
can certainly compile even master for CDH and get it to work though.
The safest build flags should be "-Phadoop-2.4 -Dhado
For CDH this works well for me...tested till 5.1...
./make-distribution -Dhadoop.version=2.3.0-cdh5.1.0 -Phadoop-2.3 -Pyarn
-Phive -DskipTests
To build with hive thriftserver support for spark-sql
On Fri, Dec 12, 2014 at 1:41 PM, Ganelin, Ilya
wrote:
>
> Hi all – we’re running CDH 5.2 and would
Hi all – we’re running CDH 5.2 and would be interested in having the latest and
greatest ML Lib version on our cluster (with YARN). Could anyone help me out in
terms of figuring out what build profiles to use to get this to play well? Will
I be able to update ML-Lib independently of updating the
Hi Xiangrui,
It seems that it's stateless so will be hard to implement
regularization path. Any suggestion to extend it? Thanks.
Sincerely,
DB Tsai
---
My Blog: https://www.dbtsai.com
LinkedIn: https://www.linkedin.com/in/dbtsai
--
Hey York - I'm sending some feedback off-list, feel free to open a PR as well.
On Tue, Dec 9, 2014 at 12:05 PM, York, Brennon
wrote:
> Patrick, I¹ve nearly completed a basic build out for the SPARK-4501 issue
> (at https://github.com/brennonyork/spark/tree/SPARK-4501) and would be
> great to get
ok, we're back up w/all new jenkins workers. i'll be keeping an eye on
these pretty closely today for any build failures caused by the new
systems, and if things look bleak, i'll switch back to the original five.
thanks for your patience!
On Fri, Dec 12, 2014 at 8:47 AM, shane knapp wrote:
> d
downtime is extended to 10am PST so that i can finish testing the numpy
upgrade... besides that, everything looks good and the system updates and
reboots went off w/o a hitch.
shane
On Fri, Dec 12, 2014 at 7:26 AM, shane knapp wrote:
> reminder: jenkins is going down NOW.
>
> On Thu, Dec 11,
Junfeng, by off the heap solution, did you mean "rdd.persist(OFF_HEAP)"?
That feature is different from the lineage feature. You can use this
feature (rdd.persist(OFF_HEAP)) now for any Spark version later than 1.0.0
with Tachyon without a problem.
Regarding Reynold's last email, those are good po
reminder: jenkins is going down NOW.
On Thu, Dec 11, 2014 at 3:08 PM, shane knapp wrote:
> here's the plan... reboots, of course, come last. :)
>
> pause build queue at 7am, kill off (and eventually retrigger) any
> stragglers at 8am. then begin maintenance:
>
> all systems:
> * yum update a
I think the linage is the key feature of tachyon to reproduce the RDD when
any error happen. Otherwise, there have to be some data replica among
tachyon nodes to ensure the data redundancy for fault tolerant - I think
tachyon is avoiding to go to this path. Dose it mean the off-heap solution
is
Hello there.
I am running the Internet Technologies Lab at http://new.huji.ac.il/en";>HUJI.
A team of my students would like to contribute a JavaScript run-time (
node.js/v8 based) to Spark.
I wonder
(1) What do you think about the necessity of such a project?
(2) Where should we get started? We h
21 matches
Mail list logo