Hi Bjorn I have been looking for spark transform for a while. Can you send me a link to the pyspark function?
I assume pandas transform is not really an option. I think it will try to pull the entire dataframe into the drivers memory. Kind regards Andy p.s. My real problem is that spark does not allow you to bind columns. You can use union() to bind rows. I could get the equivalent of cbind() using union().transform() From: Bjørn Jørgensen <bjornjorgen...@gmail.com> Date: Tuesday, March 15, 2022 at 10:37 AM To: Mich Talebzadeh <mich.talebza...@gmail.com> Cc: "user @spark" <user@spark.apache.org> Subject: Re: pivoting panda dataframe https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.transpose.html we have that transpose in pandas api for spark to. You also have stack() and multilevel https://pandas.pydata.org/pandas-docs/stable/user_guide/reshaping.html tir. 15. mar. 2022 kl. 17:50 skrev Mich Talebzadeh <mich.talebza...@gmail.com<mailto:mich.talebza...@gmail.com>>: hi, Is it possible to pivot a panda dataframe by making the row column heading? thanks [Image removed by sender.] view my Linkedin profile<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> https://en.everybodywiki.com/Mich_Talebzadeh Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. -- Bjørn Jørgensen Vestre Aspehaug 4, 6010 Ålesund Norge +47 480 94 297
