Re: About Memory usage in the Spark UI

2014-10-23 Thread Patrick Wendell
It shows the amount of memory used to store RDD blocks, which are created when you run .cache()/.persist() on an RDD. On Wed, Oct 22, 2014 at 10:07 PM, Haopu Wang hw...@qilinsoft.com wrote: Hi, please take a look at the attached screen-shot. I wonders what's the Memory Used column mean. I

RE: About Memory usage in the Spark UI

2014-10-23 Thread Haopu Wang
! From: Patrick Wendell [mailto:pwend...@gmail.com] Sent: 2014年10月23日 14:00 To: Haopu Wang Cc: user Subject: Re: About Memory usage in the Spark UI It shows the amount of memory used to store RDD blocks, which are created when you run .cache()/.persist() on an RDD. On Wed

Re: About Memory usage in the Spark UI

2014-10-23 Thread Tathagata Das
again for the answer! -- *From:* Patrick Wendell [mailto:pwend...@gmail.com] *Sent:* 2014年10月23日 14:00 *To:* Haopu Wang *Cc:* user *Subject:* Re: About Memory usage in the Spark UI It shows the amount of memory used to store RDD blocks, which are created when

RE: About Memory usage in the Spark UI

2014-10-23 Thread Haopu Wang
: 2014年10月24日 8:07 To: Haopu Wang Cc: Patrick Wendell; user Subject: Re: About Memory usage in the Spark UI The memory usage of blocks of data received through Spark Streaming is not reflected in the Spark UI. It only shows the memory usage due to cached RDDs. I didnt find a JIRA for this, so I

About Memory usage in the Spark UI

2014-10-22 Thread Haopu Wang
Hi, please take a look at the attached screen-shot. I wonders what's the Memory Used column mean. I give 2GB memory to the driver process and 12GB memory to the executor process. Thank you!