Re: Identify the performance bottleneck from hardware prospective

2015-03-05 Thread Julaiti Alafate
, then drop close to 0... Do you know more? thanks, Mitch Gusat (IBM research) On Tue, Feb 17, 2015 at 11:20 AM, Julaiti Alafate jalaf...@eng.ucsd.edu wrote: Hi there, I am trying to scale up the data size that my application is handling. This application is running on a cluster with 16

Re: Identify the performance bottleneck from hardware prospective

2015-02-17 Thread Julaiti Alafate
is the length of the sentence. It is about 63 billions. Thanks, Julaiti On Tue, Feb 17, 2015 at 2:44 AM, Arush Kharbanda ar...@sigmoidanalytics.com wrote: Hi How big is your dataset? Thanks Arush On Tue, Feb 17, 2015 at 4:06 PM, Julaiti Alafate jalaf...@eng.ucsd.edu wrote: Thank you very

Re: Identify the performance bottleneck from hardware prospective

2015-02-17 Thread Julaiti Alafate
locally on one of the worker and hence it has to collect it from others, which is a lot of Serialization and data transfer across your cluster. Thanks Best Regards On Tue, Feb 17, 2015 at 11:20 AM, Julaiti Alafate jalaf...@eng.ucsd.edu wrote: Hi there, I am trying to scale up the data

Identify the performance bottleneck from hardware prospective

2015-02-16 Thread Julaiti Alafate
Hi there, I am trying to scale up the data size that my application is handling. This application is running on a cluster with 16 slave nodes. Each slave node has 60GB memory. It is running in standalone mode. The data is coming from HDFS that also in same local network. In order to have an