I just double checked. 3.0.1 is the latest one that has my fix.
2020년 12월 31일 (목) 오전 9:21, Hyukjin Kwon 님이 작성:
> Nice, yeah, 3.0.1 should have all fixes needed.
>
> 2020년 12월 31일 (목) 오전 5:23, Felix Cheung 님이 작성:
>
>> We could just submit the latest release with the fix again. I would not
>>
Nice, yeah, 3.0.1 should have all fixes needed.
2020년 12월 31일 (목) 오전 5:23, Felix Cheung 님이 작성:
> We could just submit the latest release with the fix again. I would not
> recommend waiting, often time there are some external changes that are not
> caught, and a fix will need to go through a
We could just submit the latest release with the fix again. I would not
recommend waiting, often time there are some external changes that are not
caught, and a fix will need to go through a release vote.
What is the latest release with your fix? 3.0.1? I can put it in but will
need to make sure
Hi,
Sorry. A false alarm. Got mistaken with what IDEA calls "unused" may
not really be unused. It is (re)assigned in StageInfo.fromStage for
a ShuffleMapStage [1] and then caught in ExecutorMonitor [2] (since it's a
SparkListener).
[1]
Hi,
It's been a while. Glad to be back Sparkians!
I've been exploring ExecutorMonitor.onJobStart in 3.0.1 and noticed that it
uses StageInfo.shuffleDepId [1] that is None by default and moreover never
"written to" according to IntelliJ IDEA.
Is this the case and intentional?
I'm wondering how
