One question: JobClient sets replication of job.jar to 10, but without waiting for namenode to actually do the replication, submits job. The replication requests are sent to datanodes in 3-4 seconds after namenode receives it. Mostly, the job already starts running in those 3-4 seconds. Should JobClient wait for the namenode to actually replicate job.jar before submitting job ?
- Milind On 8/28/07 2:41 PM, "Doug Cutting" <[EMAIL PROTECTED]> wrote: > Torsten Curdt wrote: >> So far we never increased the replication. > > JobClient increases the replication of submitted job.xml and job.jar > files, to give them increased availability within the cluster. So that > may be what triggers this. You could try setting > mapred.submit.replication to three to effectively disable this. > > Doug -- Milind Bhandarkar 408-349-2136 ([EMAIL PROTECTED])
