Re: questions about hadoop map reduce and compute intensive related applications

2011-05-02 Thread gmackey
Integrating MPI with map-reduce is currently difficult and/or very ugly, however. Not impossible and there are hackish ways to do the job, but they are hacks. There is an project out of Sandia National Lab that puts MR and MPI together in a library if you're interested --

[hadoop] Hadoop Summit 2011 by Yahoo!: June 29th, Santa Clara Convention Center. Register and submit abstract for presentation: www.hadoopsummit.org

2011-05-02 Thread Avik Dey
Hi Folks, The Hadoop Summit 2011 abstract submission closes on May 6th i.e. this Friday. I know there are still some of you that are working on the presentation abstract for the Summit, thought I would send a gentle reminder. :-) See you at the Summit. Avik

Re: Discussions - Re: [VOTE] Release candidate 0.20.203.0-rc0

2011-05-02 Thread James Seigel
Hello! I guess I am concerned as a user of hadoop that the only way to get an “endorsed” up-to-date version of hadoop one has to abandon the community and “trust” a commercial release with its special sauce. I am just hoping that the community can put together a nice stable up-to-date patched

Re: [VOTE] Release candidate 0.20.203.0-rc0

2011-05-02 Thread Arun C Murthy
Doug, On May 2, 2011, at 1:40 PM, Doug Cutting wrote: Also note that, on the common-dev thread, Eli Tom have both noted a number of inconsistencies between this set of patches and trunk, 0.22 and even prior 0.20 branches and releases. In addition to the lack of community involvement in patch

Re: [VOTE] Release candidate 0.20.203.0-rc0

2011-05-02 Thread Doug Cutting
On 05/02/2011 02:05 PM, Arun C Murthy wrote: As I noted before you were the first one to propose this release off Yahoo security patch-set in April, 2010: http://s.apache.org/5Gv What has changed since? Clearly, the same situation exists today. I have absolutely no objection in principle to

Re: [VOTE] Release candidate 0.20.203.0-rc0

2011-05-02 Thread Arun C Murthy
On May 2, 2011, at 2:21 PM, Doug Cutting wrote: On 05/02/2011 02:05 PM, Arun C Murthy wrote: As I noted before you were the first one to propose this release off Yahoo security patch-set in April, 2010: http://s.apache.org/5Gv What has changed since? Clearly, the same situation exists today.

Re: [VOTE] Release candidate 0.20.203.0-rc0

2011-05-02 Thread Ian Holsman
On May 3, 2011, at 7:33 AM, Arun C Murthy wrote: This patchset started from 0.20.1 has is complete superset of 0.20.1. We will work towards ensuring it is a complete superset of the last stable release: 0.20.2. so are you intending to make it a superset for 203? or for a future release?

Re: [VOTE] Release candidate 0.20.203.0-rc0

2011-05-02 Thread Doug Cutting
On 05/02/2011 02:33 PM, Arun C Murthy wrote: We will work towards ensuring it is a complete superset of the last stable release: 0.20.2. Great! Who's 'we'? Do you want any help with this? Doug

Re: [VOTE] Release candidate 0.20.203.0-rc0

2011-05-02 Thread Andrew Purtell
Most points in this thread are valid, having to do with the process of how the contribution was assembled; and specific technical aspects of it, e.g. JIRAs missing from branch 0.20.203 relative to branch 0.20. However, From: Doug Cutting cutt...@apache.org Assuming the technical

Re: [VOTE] Release candidate 0.20.203.0-rc0

2011-05-02 Thread Eli Collins
On Mon, May 2, 2011 at 3:15 PM, Arun C Murthy a...@yahoo-inc.com wrote: On May 2, 2011, at 3:05 PM, Andrew Purtell wrote: Some technical concerns seem reasonable. Regarding that: From: Stack st...@duboce.net How hard would it be to get the patches Tom lists below into

Re: [VOTE] Release candidate 0.20.203.0-rc0

2011-05-02 Thread Arun C Murthy
On May 2, 2011, at 2:49 PM, Ian Holsman wrote: On May 3, 2011, at 7:33 AM, Arun C Murthy wrote: This patchset started from 0.20.1 has is complete superset of 0.20.1. We will work towards ensuring it is a complete superset of the last stable release: 0.20.2. so are you intending to

Re: [VOTE] Release candidate 0.20.203.0-rc0

2011-05-02 Thread Doug Cutting
On 05/02/2011 03:05 PM, Andrew Purtell wrote: What strikes me, as an observer to this discussion, is that here community does not seem equated with Yahoo by implication. Perhaps I misread. Nevertheless, Yahoo retains a good percentage of active Core developers with standing as both committers

Re: [VOTE] Release candidate 0.20.203.0-rc0

2011-05-02 Thread Jake Cornelius
Doug Cutting cutt...@apache.org wrote: On 05/02/2011 03:05 PM, Andrew Purtell wrote: What strikes me, as an observer to this discussion, is that here community does not seem equated with Yahoo by implication. Perhaps I misread. Nevertheless, Yahoo retains a good percentage of active Core

Re: Discussions - Re: [VOTE] Release candidate 0.20.203.0-rc0

2011-05-02 Thread Roy T. Fielding
On May 2, 2011, at 12:15 PM, Ian Holsman wrote: moving this thread to general@ On May 3, 2011, at 3:58 AM, Doug Cutting wrote: Should we release http://people.apache.org/~omalley/hadoop-0.20.203.0-rc0/? The patch selection process for this branch did not appear to be a community

Re: Discussions - Re: [VOTE] Release candidate 0.20.203.0-rc0

2011-05-02 Thread Milind Bhandarkar
It is perfectly reasonable for Doug (or anyone else) to vote on a release based on a lack of version history, adequate description of the sweet meats, or anything else that others might consider non-technical. This is a release vote! It does not require consensus. It requires minimal review

Re: questions about hadoop map reduce and compute intensive related applications

2011-05-02 Thread elton sky
thanks gmackey, There is an project out of Sandia National Lab that puts MR and MPI together in a library if you're interested -- http://www.sandia.gov/~sjplimp/mapreduce.html That is a implementation of MR using MPI. I saw that as well but haven't tried it out. I am actually looking at