My point is could Hadoop go wrong about one Spark execution ? meaning that it gets confused (given the concurrent distributed tasks) and then adds wrong instruction to the program, or maybe does execute an instruction not at its right order (shuffling the order of execution by executing previous ones, while it shouldn't) ? Before finishing and returning the results from one node it returns the results of the other in a wrong way for example.
Le lun. 24 janv. 2022 à 15:31, Sean Owen <sro...@gmail.com> a écrit : > Not clear what you mean here. A Spark program is a program, so what are > the alternatives here? program execution order is still program execution > order. You are not guaranteed anything about order of concurrent tasks. > Failed tasks can be reexecuted so should be idempotent. I think the answer > is 'no' but not sure what you are thinking of here. > > On Mon, Jan 24, 2022 at 7:10 AM sam smith <qustacksm2123...@gmail.com> > wrote: > >> Hello guys, >> >> I hope my question does not sound weird, but could a Spark execution on >> Hadoop cluster give different output than the program actually does ? I >> mean by that, the execution order is messed by hadoop, or an instruction >> executed twice..; ? >> >> Thanks for your enlightenment >> >
