Re: Review Request 35067: SAMZA-546: Add config option max.in.flight.requests.per.connection
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35067/ --- (Updated June 15, 2015, 1:42 p.m.) Review request for samza. Repository: samza Description (updated) --- Fixed test Diffs (updated) - samza-kafka/src/main/scala/org/apache/samza/config/KafkaConfig.scala a1de88781a8ff2a3fe50389ae7a59062289404fc samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala 82006960adc45a92a79e28f078ebd5eb01d3205e Diff: https://reviews.apache.org/r/35067/diff/ Testing --- Thanks, Robert Zuljevic
Re: 3 processed message per incoming message
Hi Shekar, a. How do I detect it? -- You can go to Yarn's monitoring page, usually, localhost:8088. May see what are running. b. How do I kill redundant jobs? -- Just like what you do for any Yarn jobs. *yarn application -kill applicationId* . If you are using the similar structure as the hello-samza, there is a script, samza/bin/kill-job.sh which you can use. Thanks, Fang, Yan yanfang...@gmail.com On Mon, Jun 15, 2015 at 10:36 PM, Shekar Tippur ctip...@gmail.com wrote: Hello, I have a scenario where for each incoming message, I see 3 corresponding processed message. I am suspecting that I may have submitted the same job multiple number of times to Yarn. If this is the case, a. How do I detect it b. How do I kill redundant jobs? Could there be other reasons for this? Appreciate your input. - Shekar
3 processed message per incoming message
Hello, I have a scenario where for each incoming message, I see 3 corresponding processed message. I am suspecting that I may have submitted the same job multiple number of times to Yarn. If this is the case, a. How do I detect it b. How do I kill redundant jobs? Could there be other reasons for this? Appreciate your input. - Shekar
Re: Combine two web servlets in the AM?
Thanks, Navina. Those two classes have been there for a while. So I think this is not really related to JobCoordinator/CoordinatorStream. Will just create a separate ticket for this. Of course, we can finish it within SAMZA-614. Cheers, Fang, Yan yanfang...@gmail.com On Mon, Jun 15, 2015 at 11:39 AM, Navina Ramesh nram...@linkedin.com.invalid wrote: +1 for adding to the documentation. I didn¹t know about this difference. This information is perhaps part of the JobCoordinator/CoordinatorStream documentation - SAMZA-614? On 6/15/15, 11:27 AM, Yan Fang yanfang...@gmail.com wrote: All right. Sorry for bothering. I figured it out. ApplicationMasterRestServlet is for the RPC and ApplicationMasterWebServlet is for tracking. The AM dashboard is from the tracking url, not he RPC url. Maybe we should mention the RPC url in the doc as well, since not many people realize this. Fang, Yan yanfang...@gmail.com On Fri, Jun 12, 2015 at 5:09 PM, Yan Fang yanfang...@gmail.com wrote: Hi guys, I am looking at the code. Do we have any special reason that we want to keep two web servlets in the AM? One is for the Rest, the other one is for the AM dashboard. Can we combine them together? This can reduce some duplicated code. Because it's good to have the AM dashboard also show the information from the coordinatorStream, which actually has been done in the Rest servlet. I think this also reduces the resource usage, though the reduction maybe not very significant. Cheers, Fang, Yan yanfang...@gmail.com
Re: Combine two web servlets in the AM?
All right. Sorry for bothering. I figured it out. ApplicationMasterRestServlet is for the RPC and ApplicationMasterWebServlet is for tracking. The AM dashboard is from the tracking url, not he RPC url. Maybe we should mention the RPC url in the doc as well, since not many people realize this. Fang, Yan yanfang...@gmail.com On Fri, Jun 12, 2015 at 5:09 PM, Yan Fang yanfang...@gmail.com wrote: Hi guys, I am looking at the code. Do we have any special reason that we want to keep two web servlets in the AM? One is for the Rest, the other one is for the AM dashboard. Can we combine them together? This can reduce some duplicated code. Because it's good to have the AM dashboard also show the information from the coordinatorStream, which actually has been done in the Rest servlet. I think this also reduces the resource usage, though the reduction maybe not very significant. Cheers, Fang, Yan yanfang...@gmail.com
Re: Combine two web servlets in the AM?
+1 for adding to the documentation. I didn¹t know about this difference. This information is perhaps part of the JobCoordinator/CoordinatorStream documentation - SAMZA-614? On 6/15/15, 11:27 AM, Yan Fang yanfang...@gmail.com wrote: All right. Sorry for bothering. I figured it out. ApplicationMasterRestServlet is for the RPC and ApplicationMasterWebServlet is for tracking. The AM dashboard is from the tracking url, not he RPC url. Maybe we should mention the RPC url in the doc as well, since not many people realize this. Fang, Yan yanfang...@gmail.com On Fri, Jun 12, 2015 at 5:09 PM, Yan Fang yanfang...@gmail.com wrote: Hi guys, I am looking at the code. Do we have any special reason that we want to keep two web servlets in the AM? One is for the Rest, the other one is for the AM dashboard. Can we combine them together? This can reduce some duplicated code. Because it's good to have the AM dashboard also show the information from the coordinatorStream, which actually has been done in the Rest servlet. I think this also reduces the resource usage, though the reduction maybe not very significant. Cheers, Fang, Yan yanfang...@gmail.com
Re: Review Request 35067: SAMZA-546: Add config option max.in.flight.requests.per.connection
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35067/#review87963 --- @Robert: Can you please add a unit test in TestKafkaConfig to verify that these properties get overridden correctly ? Sorry about this afterthought request. - Navina Ramesh On June 15, 2015, 1:42 p.m., Robert Zuljevic wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35067/ --- (Updated June 15, 2015, 1:42 p.m.) Review request for samza. Repository: samza Description --- Fixed test Diffs - samza-kafka/src/main/scala/org/apache/samza/config/KafkaConfig.scala a1de88781a8ff2a3fe50389ae7a59062289404fc samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala 82006960adc45a92a79e28f078ebd5eb01d3205e Diff: https://reviews.apache.org/r/35067/diff/ Testing --- Thanks, Robert Zuljevic
