For this case I’d agree with Ryan. I haven’t followed this thread and the details of the change since it’s way too much for me to consume “in my free time” (which is 0 nowadays) but I’m pretty sure the existing behavior works for us and very likely we don’t want it to change because of some proxy magic we do behind the scene.
I’d also agree config flag is not always the best way but in this case the existing established behavior doesn’t seem broken... I could be wrong though. ________________________________ From: Ryan Blue <rb...@netflix.com.invalid> Sent: Friday, February 8, 2019 4:39 PM To: Sean Owen Cc: Jungtaek Lim; dev Subject: Re: [DISCUSS] Change default executor log URLs for YARN I'm not sure that many people need this, so it is hard to make a decision. I'm reluctant to change the current behavior if the result is a new papercut to 99% of users and a win for 1%. The suggested change will work for 100% of users, so if we don't want a flag then we should go with that. But I would certainly want to turn it off in our environment because it doesn't provide any value for us and would annoy our users. On Fri, Feb 8, 2019 at 4:18 PM Sean Owen <sro...@gmail.com<mailto:sro...@gmail.com>> wrote: Is a flag needed? You know me, I think flags are often failures of design, or disagreement punted to the user. I can understand retaining old behavior under a flag where the behavior change could be problematic for some users or facilitate migration, but this is just a change to some UI links no? the underlying links don't change. On Fri, Feb 8, 2019 at 5:41 PM Ryan Blue <rb...@netflix.com<mailto:rb...@netflix.com>> wrote: > > I suggest using the current behavior as the default and add a flag to > implement the behavior you're suggesting: to link to the logs path in YARN > instead of directly to stderr and stdout. > > On Fri, Feb 8, 2019 at 3:33 PM Jungtaek Lim > <kabh...@gmail.com<mailto:kabh...@gmail.com>> wrote: >> >> Ryan, >> >> actually I'm not clear about your suggestion. For me three possible options >> here: >> >> 1. If we want to let users be able to completely rewrite log urls, that's >> SPARK-26792. For SHS we already addressed it. >> 2. We could let users turning on/off flag option to just get one url or >> default two stdout/stderr urls. >> 3. We could let users enumerate file names they want to link, and create log >> links for each file. >> >> Which one do you suggest? > -- Ryan Blue Software Engineer Netflix
