iScala or Scala-notebook

2014-07-17 Thread ericjohnston1989
Hey everyone, I know this was asked before but I'm wondering if there have since been any updates. Are there any plans to integrate iScala/Scala-notebook with spark in the near future? This seems like something a lot of people would find very useful, so I was just wondering if anyone has started

Re: Doubts about MLlib.linalg in python

2014-06-12 Thread ericjohnston1989
Hi Congrui Yi, Spark is implemented in Scala, so all Scala features are first available in Scala/Java. PySpark is a python wrapper for the Scala code, so it won't always have the latest features. This is especially true for the Machine learning library. Eric -- View this message in context:

Increase storage.MemoryStore size

2014-06-12 Thread ericjohnston1989
Hey everyone, I'm having some trouble increasing the default storage size for a broadcast variable. It looks like it defaults to a little less than 512MB every time, and I can't figure out which configuration to change to increase this. INFO storage.MemoryStore: Block broadcast_0 stored as values

Calliope Frame size larger than max length

2014-04-18 Thread ericjohnston1989
Hey all, I'm working with Calliope to run jobs on a Cassandra cluster in standalone mode. On some larger jobs I run into the following error: java.lang.RuntimeException: Frame size (20667866) larger than max length (15728640)! at org.apache.cassandra.hadoop.cql3.CqlPagingRecordReader$RowI