I started building Spark / running Spark tests this weekend and on maybe
5-10 occasions have run into a compiler crash while compiling
DataTypeConversions.scala.
Here https://gist.github.com/ryan-williams/7673d7da928570907f4d is a full
gist of an innocuous test command (mvn test -Dsuites
encountering this issue.
Typically you would have changed one or more of the profiles/options -
which leads to this occurring.
2014-10-22 22:00 GMT-07:00 Ryan Williams ryan.blake.willi...@gmail.com:
I started building Spark / running Spark tests this weekend and on maybe
5-10 occasions have run
TD's portion seems to start at 27:24: http://youtu.be/jcJq3ZalXD8?t=27m24s
On Tue Dec 16 2014 at 7:13:43 AM Gerard Maas gerard.m...@gmail.com wrote:
Hi Jeniba,
The second part of this meetup recording has a very good answer to your
question. TD explains the current behavior and the on-going
If anyone is curious to try exporting Spark metrics to Graphite, I just
published a post about my experience doing that, building dashboards in
Grafana http://grafana.org/, and using them to monitor Spark jobs:
http://www.hammerlab.org/2015/02/27/monitoring-spark-with-graphite-and-grafana/
Code
:149)
Best Regards,
Shixiong Zhu
2015-06-03 0:08 GMT+08:00 Ryan Williams ryan.blake.willi...@gmail.com:
I think this is causing issues upgrading ADAM
https://github.com/bigdatagenomics/adam to Spark 1.3.1 (cf. adam#690
https://github.com/bigdatagenomics/adam/pull/690#issuecomment-107769383
I think this is causing issues upgrading ADAM
https://github.com/bigdatagenomics/adam to Spark 1.3.1 (cf. adam#690
https://github.com/bigdatagenomics/adam/pull/690#issuecomment-107769383);
attempting to build against Hadoop 1.0.4 yields errors like:
2015-06-02 15:57:44 ERROR Executor:96 -
Probably relevant to people on this list: on Friday I released a clone of
the Spark web UI built using Meteor https://www.meteor.com/ so that
everything updates in real-time, saving you from endlessly refreshing the
page while jobs are running :) It can also serve as the UI for running as
well as
Hey Jeff, in addition to what Sandy said, there are two more reasons that
this might not be as bad as it seems; I may be incorrect in my
understanding though.
First, the "additional step" you're referring to is not likely to be adding
any overhead; the "extra map" is really just materializing the