Where should this *coalesce* come from ? Is it related to the partition manipulation coalesce method ? Thanks !
Le lun. 20 avr. 2015 à 22:48, Reynold Xin <[email protected]> a écrit : > Ah ic. You can do something like > > > df.select(coalesce(df("a"), lit(0.0))) > > On Mon, Apr 20, 2015 at 1:44 PM, Olivier Girardot < > [email protected]> wrote: > >> From PySpark it seems to me that the fillna is relying on Java/Scala >> code, that's why I was wondering. >> Thank you for answering :) >> >> Le lun. 20 avr. 2015 à 22:22, Reynold Xin <[email protected]> a écrit : >> >>> You can just create fillna function based on the 1.3.1 implementation of >>> fillna, no? >>> >>> >>> On Mon, Apr 20, 2015 at 2:48 AM, Olivier Girardot < >>> [email protected]> wrote: >>> >>>> a UDF might be a good idea no ? >>>> >>>> Le lun. 20 avr. 2015 à 11:17, Olivier Girardot < >>>> [email protected]> a écrit : >>>> >>>> > Hi everyone, >>>> > let's assume I'm stuck in 1.3.0, how can I benefit from the *fillna* >>>> API >>>> > in PySpark, is there any efficient alternative to mapping the records >>>> > myself ? >>>> > >>>> > Regards, >>>> > >>>> > Olivier. >>>> > >>>> >>> >>> >
