Thank you very much Prashant. Date: Thu, 24 Apr 2014 01:24:39 -0700 From: ml-node+s1001560n4739...@n3.nabble.com To: gyz...@hotmail.com Subject: Re: Need help about how hadoop works.
It is the same file and hadoop library that we use for splitting takes care of assigning the right split to each node.Prashant Sharma On Thu, Apr 24, 2014 at 1:36 PM, Carter <[hidden email]> wrote: Thank you very much for your help Prashant. Sorry I still have another question about your answer: "however if the file("/home/scalatest.txt") is present on the same path on all systems it will be processed on all nodes." When presenting the file to the same path on all nodes, do we just simply copy the same file to all nodes, or do we need to split the original file into different parts (each part is still with the same file name "scalatest.txt"), and copy each part to a different node for parallelization? Thank you very much. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Need-help-about-how-hadoop-works-tp4638p4738.html Sent from the Apache Spark User List mailing list archive at Nabble.com. If you reply to this email, your message will be added to the discussion below: http://apache-spark-user-list.1001560.n3.nabble.com/Need-help-about-how-hadoop-works-tp4638p4739.html To unsubscribe from Need help about how hadoop works., click here. NAML -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Need-help-about-how-hadoop-works-tp4638p4746.html Sent from the Apache Spark User List mailing list archive at Nabble.com.