Clarification on what "[id=#]" refers to in Physical Plan Exchange hashpartitioning

2024-04-04 Thread Tahj Anderson
Hello, While looking through spark physical plans generated by the spark history server log to find any bottle necks in my code, I stumbled across an ID that shows up in a partitioning stage. My goal is to use the history server log to provide meaningful analysis on my spark system

Clarification on what "[id=#]" refers to in Physical Plan Exchange hashpartitioning

2024-04-04 Thread Tahj Anderson
Hello, While looking through spark physical plans generated by the spark history server log to find any bottle necks in my code, I stumbled across an ID that shows up in a partitioning stage. My goal is to use the history server log to provide meaningful analysis on my spark system