from pyspark.sql import SparkSession from pyspark.sql.functions import stddev_samp, stddev_pop
spark = SparkSession.builder.getOrCreate() data = [(52.7,), (45.3,), (60.2,), (53.8,), (49.1,), (44.6,), (58.0,), (56.5,), (47.9,), (50.3,)] df = spark.createDataFrame(data, ["value"]) df.select(stddev_samp("value").alias("sample_stddev")).show() +-----------------+ | sample_stddev| +-----------------+ |5.320025062597606| +-----------------+ In MS Excel 365 Norwegian [image: image.png] =STDAVVIKA(B1:B10) =STDAV.S(B1:B10) They both prints 5,32002506 Which is the same as pyspark does. tir. 19. sep. 2023 kl. 14:15 skrev Helene Bøe <helene.b...@hydro.com.invalid >: > Hi! > > > > I am applying the stddev function (so actually stddev_samp), however when > comparing with the sample standard deviation in Excel the resuls do not > match. > > I cannot find in your documentation any more specifics on how the sample > standard deviation is calculated, so I cannot compare the difference toward > excel, which uses > > . > > I am trying to avoid using Excel at all costs, but if the stddev_samp > function is not calculating the standard deviation correctly I have a > problem. > > I hope you can help me resolve this issue. > > > > Kindest regards, > > > > *Helene Bøe* > *Graduate Project Engineer* > Recycling Process & Support > > M: +47 980 00 887 > helene.b...@hydro.com > <https://intra.hydro.com/EPiServer/CMS/Content/en/%2c%2c9/?epieditmode=False> > > Norsk Hydro ASA > Drammensveien 264 > NO-0283 Oslo, Norway > www.hydro.com > <https://intra.hydro.com/EPiServer/CMS/Content/en/%2c%2c9/?epieditmode=False> > > > NOTICE: This e-mail transmission, and any documents, files or previous > e-mail messages attached to it, may contain confidential or privileged > information. If you are not the intended recipient, or a person responsible > for delivering it to the intended recipient, you are hereby notified that > any disclosure, copying, distribution or use of any of the information > contained in or attached to this message is STRICTLY PROHIBITED. If you > have received this transmission in error, please immediately notify the > sender and delete the e-mail and attached documents. Thank you. > -- Bjørn Jørgensen Vestre Aspehaug 4, 6010 Ålesund Norge +47 480 94 297