Hey AN-TRUONG I have got some articles about this subject that should help. E.g. https://khalidmammadov.github.io/spark/spark_internals_rdd.html
Also check other Spark Internals on web. Regards Khalid On Fri, 31 Mar 2023, 16:29 AN-TRUONG Tran Phan, <tr.phan.tru...@gmail.com> wrote: > Thank you for your information, > > I have tracked the spark history server on port 18080 and the spark UI on > port 4040. I see the result of these two tools as similar right? > > I want to know what each Task ID (Example Task ID 0, 1, 3, 4, 5, ....) in > the images does, is it possible? > https://i.stack.imgur.com/Azva4.png > > Best regards, > > An - Truong > > > On Fri, Mar 31, 2023 at 9:38 PM Mich Talebzadeh <mich.talebza...@gmail.com> > wrote: > >> Are you familiar with spark GUI default on port 4040? >> >> have a look. >> >> HTH >> >> Mich Talebzadeh, >> Lead Solutions Architect/Engineering Lead >> Palantir Technologies Limited >> >> >> view my Linkedin profile >> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >> >> >> https://en.everybodywiki.com/Mich_Talebzadeh >> >> >> >> *Disclaimer:* Use it at your own risk. Any and all responsibility for >> any loss, damage or destruction of data or any other property which may >> arise from relying on this email's technical content is explicitly >> disclaimed. The author will in no case be liable for any monetary damages >> arising from such loss, damage or destruction. >> >> >> >> >> On Fri, 31 Mar 2023 at 15:15, AN-TRUONG Tran Phan < >> tr.phan.tru...@gmail.com> wrote: >> >>> Hi, >>> >>> I am learning about Apache Spark and want to know the meaning of each >>> Task created on the Jobs recorded on Spark history. >>> >>> For example, the application I write creates 17 jobs, in which job 0 >>> runs for 10 minutes, there are 2384 small tasks and I want to learn about >>> the meaning of these 2384, is it possible? >>> >>> I found a picture of DAG in the Jobs and want to know the relationship >>> between DAG and Task, is it possible (Specifically from the attached file >>> DAG and 2384 tasks below)? >>> >>> Thank you very much, have a nice day everyone. >>> >>> Best regards, >>> >>> An-Trường. >>> >>> --------------------------------------------------------------------- >>> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >> >> > > -- > Trân Trọng, > > An Trường. >
