I have tried what you suggest (well sort of) a goof example would help alot - My reducer is set to among other things emit the local os and user.dir - when I try running from my windows box these appear on hdfs but show the windows os and user.dir leading me to believe that the reducer is still running on my windows machine - I will check the values but a working example would be very useful
On Sun, May 29, 2011 at 6:19 AM, Ferdy Galema <ferdy.gal...@kalooga.com>wrote: > Would it not also be possible for a Windows machine to submit the job > directly from a Java process? This way you don't need Cygwin / a full local > copy of the installation (correct my if I'm wrong). The steps would then > just be: > 1) Create a basic Java project, add minimum required libraries > (Hadoop/logging) > 2) Set the essential properties (at least this would be the jobtracker and > the filesystem) > 3) Implement the Tool > 4) Run the process (from either the IDE or stand-alone jar) > > Steps 1-3 could technically be implemented on another machine, if you > choose to compile a stand-alone jar. > > Ferdy. > > > On 05/29/2011 04:50 AM, Harsh J wrote: > >> Keep a local Hadoop installation with a mirror-copy config, and use >> "hadoop jar<jar>" to submit as usual (since the config points to the >> right areas, the jobs go there). >> >> For Windows you'd need Cygwin installed, however. >> >> On Sun, May 29, 2011 at 12:56 AM, Steve Lewis<lordjoe2...@gmail.com> >> wrote: >> >>> When I want to launch a hadoop job I use SCP to execute a command on the >>> Name node machine. I an wondering if there is >>> a way to launch a Hadoop job from a machine that is not on the cluster. >>> How >>> to do this on a Windows box or a Mac would be >>> of special interest. >>> >>> -- >>> Steven M. Lewis PhD >>> 4221 105th Ave NE >>> Kirkland, WA 98033 >>> 206-384-1340 (cell) >>> Skype lordjoe_com >>> >>> >> >> -- Steven M. Lewis PhD 4221 105th Ave NE Kirkland, WA 98033 206-384-1340 (cell) Skype lordjoe_com
