Re: pivoting panda dataframe

2022-03-16 Thread ayan guha
c(x) -> ps.Series[np.int32]: >>>>> ... return x + sum(x) >>>>> >>>>> Parameters >>>>> -- >>>>> func : function >>>>> Function to use for transforming the data. It must work when pan

Re: pivoting panda dataframe

2022-03-16 Thread Mich Talebzadeh
gating type operations. >>>> DataFrame.apply : Invoke function on DataFrame. >>>> Series.transform : The equivalent function for Series. >>>> >>>> Examples >>>> >>>> >>> df = ps.DataFrame({'A': range(3), 'B': ran

Re: pivoting panda dataframe

2022-03-15 Thread Mich Talebzadeh
t;> 1 1 2 >>> 2 2 3 >>> >>> >>> def square(x) -> ps.Series[np.int32]: >>> ... return x ** 2 >>> >>> df.transform(square) >>>A B >>> 0 0 1 >>> 1 1 4 >>> 2 4 9 >>&g

Re: pivoting panda dataframe

2022-03-15 Thread Bjørn Jørgensen
ørgensen > *Date: *Tuesday, March 15, 2022 at 2:19 PM > *To: *Andrew Davidson > *Cc: *Mich Talebzadeh , "user @spark" < > user@spark.apache.org> > *Subject: *Re: pivoting panda dataframe > > > > Hi Andrew. Mitch asked, and I an

Re: pivoting panda dataframe

2022-03-15 Thread Bjørn Jørgensen
nsform(square) # doctest: +NORMALIZE_WHITESPACE >>X >>A B >> 0 0 1 >> 1 1 4 >> 2 4 9 >> >> >>> (df * -1).transform(abs) # doctest: +NORMALIZE_WHITESPACE >>X >>A B >> 0 0 1 >> 1 1

Re: pivoting panda dataframe

2022-03-15 Thread Andrew Davidson
: Tuesday, March 15, 2022 at 2:19 PM To: Andrew Davidson Cc: Mich Talebzadeh , "user @spark" Subject: Re: pivoting panda dataframe Hi Andrew. Mitch asked, and I answered transpose() https://spark.apache.org/docs/latest/api/python/reference/pyspark.pandas/api/pyspark.pandas.DataFrame.tran

Re: pivoting panda dataframe

2022-03-15 Thread Mich Talebzadeh
21 > 121 1044 > 2 1044 59069File: /opt/spark/python/pyspark/pandas/frame.pyType: > method > > > > > > tir. 15. mar. 2022 kl. 19:33 skrev Andrew Davidson : > >> Hi Bjorn >> >> >> >> I have been looking for spa

Re: pivoting panda dataframe

2022-03-15 Thread Bjørn Jørgensen
t; Andy > > > > p.s. My real problem is that spark does not allow you to bind columns. You > can use union() to bind rows. I could get the equivalent of cbind() using > union().transform() > > > > *From: *Bjørn Jørgensen > *Date: *Tuesday, March 15, 2022 at 10:37 AM > *To:

Re: pivoting panda dataframe

2022-03-15 Thread Andrew Davidson
does not allow you to bind columns. You can use union() to bind rows. I could get the equivalent of cbind() using union().transform() From: Bjørn Jørgensen Date: Tuesday, March 15, 2022 at 10:37 AM To: Mich Talebzadeh Cc: "user @spark" Subject: Re: pivoting panda datafr

Re: pivoting panda dataframe

2022-03-15 Thread Bjørn Jørgensen
https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.transpose.html we have that transpose in pandas api for spark to. You also have stack() and multilevel https://pandas.pydata.org/pandas-docs/stable/user_guide/reshaping.html tir. 15. mar. 2022 kl. 17:50 skrev Mich Talebzadeh <

pivoting panda dataframe

2022-03-15 Thread Mich Talebzadeh
hi, Is it possible to pivot a panda dataframe by making the row column heading? thanks view my Linkedin profile https://en.everybodywiki.com/Mich_Talebzadeh *Disclaimer:* Use it at your own risk. Any and all responsibility