Re:Re:Need help in log4j.xml externalization

2015-10-26 Thread Yan Fang
" When I keep my log4j.xml in //$PWD/deploy/alice/config/log4j.xml and do not set any "samza.container.name" then it is not generating any log files for me. " == this is weird. The run-am/run-container script should automatically create the samba.container.name if it is not set... At

Snapshot metrics stop getting scheduled in an exception occurs

2015-10-26 Thread Roger Hoover
Hi Samza devs, I ran into an issue with Samza 0.9.1 where I had a serialization exception thrown in the MetricsSnapshotReporter. It's very hard to find because nothing is logged and the metrics just stop getting scheduled. Samza should catch all exceptions in that thread, log them, and suppress

Re:Re:Need help in log4j.xml externalization

2015-10-26 Thread Yan Fang
Hi Ankush, In current situation, if you run the Yarn client and the Yarn application in different machines, the logs will not be mixed. Otherwise, there is no other ways, because they share the same environment variable, if the environment variable is set, they both will use it. You may want t

Re:Re: questions of partition and task of Samza

2015-10-26 Thread Yan Fang
Hi Selina, Your understanding is correct. Yes, you "need to consumer the original input and send it back to Kafka and reset the* Key to departmentName *and then consume it again to count in Samza" if you want to count the number of students in the same departmentName. This is a typical aggreg

Re: Review Request 39252: SAMZA-626 - tool to read the RocksDb in a running job (Yan's patch)

2015-10-26 Thread Navina Ramesh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/39252/ --- (Updated Oct. 26, 2015, 6:45 p.m.) Review request for samza, Yan Fang, Chinmay

Re: Review Request 39252: SAMZA-626 - tool to read the RocksDb in a running job (Yan's patch)

2015-10-26 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/39252/#review104029 --- Except the check for the key variable arguments and minor comments

Re:Need help in log4j.xml externalization

2015-10-26 Thread Patni, Ankush
Hi Yan, Small doubt : Is it possible to set the log4j.xml in somewhere so that it will not get mix with yarn client and will be available for yarn application only. With this I will be able to get the logs for my individual task in their respective container. Ankush

Re:Need help in log4j.xml externalization

2015-10-26 Thread Patni, Ankush
Hi Yang, Thanks a lot for help. I have couple of doubts and want to clear them : I am starting samza task from same machine where it is running. My problem is : I am not using StreamAppender as suggested by you in previous reply. But still DailyRollingFileAppender uses the ${samza.container.nam