Thanks Li and Imran for providing us an overview about one of the complex module in spark 👍 Excellent sharing.
Regards Sujith. On Mon, 4 Feb 2019 at 10:54 PM, Xiao Li <gatorsm...@gmail.com> wrote: > Thank you, Imran! > > Also, I attached the slides of "Deep Dive: Scheduler of Apache Spark". > > Cheers, > > Xiao > > > > John Zhuge <jzh...@apache.org> 于2019年2月4日周一 上午8:59写道： > >> Thanks Imran! >> >> On Mon, Feb 4, 2019 at 8:42 AM Imran Rashid <iras...@cloudera.com.invalid> >> wrote: >> >>> The scheduler has been pretty error-prone and hard to work on, and I >>> feel like there may be a dwindling core of active experts. I'm sure its >>> very discouraging to folks trying to make what seem like simple changes, >>> and then find they are in a rats nest of complex issues they weren't >>> expecting. But for those who are still trying, THANK YOU! more >>> involvement and more folks becoming experts is definitely needed. >>> >>> I put together a doc going over the architecture of the scheduler, and >>> things I've seen us get bitten by in the past. Its sort of a brain dump, >>> but I'm hopeful it'll help orient new folks to the scheduler. I also hope >>> more experts will chime in -- there are places in the doc I know I've >>> missed things, and called that out, but there are probably even more that >>> should be discussed, & mistakes I've made. All input welcome. >>> >>> >>> https://docs.google.com/document/d/1oiE21t-8gXLXk5evo-t-BXpO5Hdcob5D-Ps40hogsp8/edit?usp=sharing >>> >> >> >> -- >> John Zhuge >> > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
