At 2014-08-23 08:33:48 -0700, Denis RP <qq378789...@gmail.com> wrote: > Bottleneck seems to be I/O, the CPU usage ranges 10%~15% most time per VM. > The caching is maintained by pregel, should be reliable. Storage level is > MEMORY_AND_DISK_SER.
I'd suggest trying the DISK_ONLY storage level and possibly increasing the number of partitions. I did a local test with a 2G heap, 1M vertices, 126M edges, and 100 edge partitions, and MEMORY_AND_DISK_SER failed with OutOfMemoryErrors while DISK_ONLY succeeded. Ankur --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
