RE: YARN timelineserver process taking 600% CPU

2015-09-30 Thread Naganarasimha G R (Naga)
Hi , Whats the size of Store Files? Since when is it running ? how many applications have been run since it has been started ? Whats the value of "yarn.timeline-service.leveldb-timeline-store.ttl-interval-ms" ? + Naga From: Krzysztof Zarzycki [k.zarzy...@gmail.c

Re: YARN timelineserver process taking 600% CPU

2015-09-30 Thread Krzysztof Zarzycki
Hi Naga, I see the following size: $ sudo du --max=1 /var/lib/hadoop/yarn/timeline 36 /var/lib/hadoop/yarn/timeline/timeline-state-store.ldb 3307772 /var/lib/hadoop/yarn/timeline/leveldb-timeline-store.ldb 3307812 /var/lib/hadoop/yarn/timeline The timeline service has been multiple times rest

Re: YARN timelineserver process taking 600% CPU

2015-09-30 Thread Naganarasimha Garla
Hi Krzysiek, seems like the size is around 3 MB which seems to be fine. , Could you try enabling in debug and share the logs of ATS/AHS and also if possible the jstack output for the AHS process + Naga On Wed, Sep 30, 2015 at 10:27 PM, Krzysztof Zarzycki wrote: > Hi Naga, > I see the following

Re: YARN timelineserver process taking 600% CPU

2015-10-05 Thread Naganarasimha Garla
Hi Krzysiek, Oops My mistake, 3 Gb seems to be on little higher side. And from the jstack it seems like there were no major activity other than puts seems like around 16 concurrent puts were happening which tries to get the timeline Entity hence hitting the native call. >From the logs it seems lik

Re: YARN timelineserver process taking 600% CPU

2015-11-05 Thread Krzysztof Zarzycki
Thanks Naga for your input, (I'm sorry for a late response, I was out for some time). So you believe that Spark is actually doing the PUTs? There are currently 8 Spark Streaming jobs constantly running, each 3 with 1 second batch, 5 x 10 s. I believe these are the jobs that publish to ATS. How c

RE: YARN timelineserver process taking 600% CPU

2015-11-05 Thread Naganarasimha G R (Naga)
che.org Subject: Re: YARN timelineserver process taking 600% CPU Thanks Naga for your input, (I'm sorry for a late response, I was out for some time). So you believe that Spark is actually doing the PUTs? There are currently 8 Spark Streaming jobs constantly running, each 3 with 1 second bat

Re: YARN timelineserver process taking 600% CPU

2015-12-05 Thread 郭士伟
--------- > *From:* Krzysztof Zarzycki [k.zarzy...@gmail.com] > *Sent:* Thursday, November 05, 2015 19:51 > *To:* user@hadoop.apache.org > *Subject:* Re: YARN timelineserver process taking 600% CPU > > Thanks Naga for your input, (I'm sorry for a late response

Re: YARN timelineserver process taking 600% CPU

2015-12-07 Thread Krzysztof Zarzycki
rom other company who have used or supported ATSV1 might be >> able to tell the ATSV1 scale better ! >> >> >> Regards, >> >> + Naga >> -- >> *From:* Krzysztof Zarzycki [k.zarzy...@gmail.com] >> *Sent:* Thursday, November