Re: When I system.out.println() in a map or reduce, where does it go?
Please check userlogs directory - Original Message - From: David Coe [EMAIL PROTECTED] To: core-user@hadoop.apache.org Sent: Thursday, December 11, 2008 5:31 AM Subject: When I system.out.println() in a map or reduce, where does it go? I've noticed that if I put a system.out.println in the run() method I see the result on my console. If I put it in the map or reduce class, I never see the result. Where does it go? Is there a way to get this result easily (eg dump it in a log file)? David
Re: Seeking Someone to Review Hadoop Article
Dear Tom, Here is one more writtien by our core data warehouse team. Appreciate if you can add it in Hadoop article links, so that community is more benefitted:- http://www.javaworld.com/javaworld/jw-09-2008/jw-09-hadoop.html Best, Ravion - Original Message - From: Tom Wheeler [EMAIL PROTECTED] To: core-user@hadoop.apache.org Sent: Wednesday, November 05, 2008 4:57 AM Subject: Re: Seeking Someone to Review Hadoop Article On Tue, Nov 4, 2008 at 3:46 PM, Milind Bhandarkar [EMAIL PROTECTED] wrote: Please consider adding it to: http://wiki.apache.org/hadoop/HadoopArticles Great suggestion -- I've just linked it there as you request. -- Tom Wheeler http://www.tomwheeler.com/
Re: Seeking Someone to Review Hadoop Article
Great and thank you!! Best Ravion - Original Message - From: Tom Wheeler [EMAIL PROTECTED] To: core-user@hadoop.apache.org Sent: Wednesday, November 05, 2008 5:47 AM Subject: Re: Seeking Someone to Review Hadoop Article Done. I also added a link to the article that Amit Kumar Saha wrote just a few weeks ago for linux.com. On Tue, Nov 4, 2008 at 4:37 PM, Ravion [EMAIL PROTECTED] wrote: Dear Tom, Here is one more writtien by our core data warehouse team. Appreciate if you can add it in Hadoop article links, so that community is more benefitted:- -- Tom Wheeler http://www.tomwheeler.com/
Re: Can jobs be configured to be sequential
Hi Paco, Thanks - This is exactly what I was looking for.. Regards, Ravi - Original Message - From: Paco NATHAN [EMAIL PROTECTED] To: core-user@hadoop.apache.org Sent: Saturday, October 18, 2008 9:46 AM Subject: Re: Can jobs be configured to be sequential Hi Ravion, The problem you are describing sounds like a workflow where you must be careful to verify certain conditions before proceeding to a next step. We have similar kinds of use cases for Hadoop apps at work, which are essentially ETL. I recommend that you look at http://cascading.org as an abstraction layer for managing these kinds of workflows. We've found it quite useful. Best, Paco On Fri, Oct 17, 2008 at 8:29 PM, Ravion [EMAIL PROTECTED] wrote: Dear all, We have in our Data Warehouse System, about 600 ETL( Extract Transform Load) jobs to create interim data model. SOme jobs are dependent on completion of others. Assume that I create a group id intdependent jobs. Say a group G1 contains 100 jobs , G2 contains another 200 jobs which are dependent on completion of Group G1 and so on. Can we leverage on Haddop so that Hadoop executed G1 first, on failure it wont execute G2 otherwise will continue for G2 and so on.. ? Or do I need to configure N ( where N = total number of groups) Haddop jobs independently and handle by ourselves? Please share your thoughts, thanks Warmest regards, Ravion
Can jobs be configured to be sequential
Dear all, We have in our Data Warehouse System, about 600 ETL( Extract Transform Load) jobs to create interim data model. SOme jobs are dependent on completion of others. Assume that I create a group id intdependent jobs. Say a group G1 contains 100 jobs , G2 contains another 200 jobs which are dependent on completion of Group G1 and so on. Can we leverage on Haddop so that Hadoop executed G1 first, on failure it wont execute G2 otherwise will continue for G2 and so on.. ? Or do I need to configure N ( where N = total number of groups) Haddop jobs independently and handle by ourselves? Please share your thoughts, thanks Warmest regards, Ravion
