Is dataframe write blocking? what can be done for fair scheduler?

Hi, I have noticed that in fair scheduler setting, if i block on dataframe write to complete, using AwaitResult, the API call ends up returning, whereas that is not what I intend to do as it can cause inconsistencies later in the pipeline. Is there a way to make the dataframe write call blocking?

Is Dataframe write blocking?

HI, I have some futures setup to operate in stages, where I expect one stage to complete before another begins. I was hoping that dataframe write call is blocking, whereas the behavior i see is that the call returns before data is persisted. This can cause unintended consequences. I am also using