Re: [go-nuts] Re: recommended statistics package? Requirements: ANOVA, Brown-Forsythe

[王富民awaw]

Hi Jason

Thanks for your suggestion.
I have renamed the function Levene to BrownForsythe, as well as added a 
comment to note that it assumes its input to be sorted ascendingly.

On Tuesday, 31 October 2023 at 02:18:51 UTC+8 Jason E. Aten wrote:

>
> On Monday, October 30, 2023 at 3:14:29 AM UTC 王富民awaw wrote:
>
> Therefore, perhaps I should not be following scipy and rename my function 
> from Levene to BrownForsythe?
>
>  
> Since you have made it available to the public Go community, yes, it would 
> be a good idea to name it BrownForsythe,
> as it is using the deviations from the group medians.
>
> Also the function makes the assumption, as does Median, that each group's 
> samples[j] is already sorted. You
> should document that assumption, or sort.Slice() each group's sample 
> before calling Median.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/e12012dc-36c5-43e2-a165-02b0a480cce0n%40googlegroups.com.

Re: [go-nuts] Re: recommended statistics package? Requirements: ANOVA, Brown-Forsythe

[Jason E. Aten]

On Monday, October 30, 2023 at 3:14:29 AM UTC 王富民awaw wrote:

Therefore, perhaps I should not be following scipy and rename my function 
from Levene to BrownForsythe?

 
Since you have made it available to the public Go community, yes, it would 
be a good idea to name it BrownForsythe,
as it is using the deviations from the group medians.

Also the function makes the assumption, as does Median, that each group's 
samples[j] is already sorted. You
should document that assumption, or sort.Slice() each group's sample before 
calling Median.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/2722f546-62e3-4cb1-80bb-1104d19ced4an%40googlegroups.com.

Re: [go-nuts] Re: recommended statistics package? Requirements: ANOVA, Brown-Forsythe

[王富民awaw]

Hi Jason

Yes, I concur that the function "Levene" named in my package should really 
be called Brown-Forsthe.

I was vacillating between this correct naming versus following the 
nomenclature of scipy 
from
 
which my implementation is numerically tested against.

I looked at a couple of popular statistics software, and it seems that most 
differentiate between Levene and Brown-Forsythe

* JMP 

* SPSS 

* R 

Therefore, perhaps I should not be following scipy and rename my function 
from Levene to BrownForsythe?
I admit I am no statistician and would appreciate some professional 
guidance.
On Monday, 30 October 2023 at 10:41:23 UTC+8 Jason E. Aten wrote:

> 王富民awaw: Very nice.
>
> The Levene function, though, is confusingly named. The Levene test is 
> distinct from the Brown-Forsthe test, so mixing up the names
> seems odd.[1]
>
> [1] quoting 
> https://www.statisticshowto.com/brown-forsythe-test/#:~:text=The%20Levene%20test%20uses%20deviations,a%20test%20that's%20more%20robust.
>   
> below: (see the underlined sentence, underline and bold there is mine)
>
> How the Test Works
>
> Both the Levene and B-F tests transform dependent variables 
>  for use 
> in an ANOVA  
> test.
>  
> The only difference between the two tests is in how those transformed 
> variables are constructed. The Levene test uses deviations from group 
> means , which usually results in a 
> highly-skewed set of data; This violates the assumption of normality 
> .* The 
> Brown-Forsythe test attempts to correct for this skewness  
> by
>  
> using deviations from group medians 
> .*
>  
> The result is a test that’s more robust 
> . In other words, the 
> B-F test is less likely than the Levene test to incorrectly declare that 
> the assumption of equal variances has been violated.
>
> The test works as follows:
>
>- The median is calculated for each factor level group.
>- The median value is subtracted from each dependent variable in the 
>group.
>- An ANOVA 
>
> 
>  is 
>run with the transformed variables. If a factor’s p-value 
>
> 
>  is 
>less than the significance level 
>
> 
>  (usually 
>5%), the population variances are not equal.
>
> W or F statistic?
>
> The test statistic used in a regular ANOVA is an F-statistic 
> .
>  
> The statistic used in an ANOVA with transformed variables is sometimes 
> called a W-Statistic — but it’s really just an F-Statistic with a 
> different name. It should not be confused with the coefficient of 
> concordance W-statistic 
> , 
> which is used to assess agreement between raters.
> Cautions
>
> For the most part, the B-F test is thought to perform as well as or better 
> than other available tests for equal variances. However, Glass and Hopkins 
> (1996 p. 436) state that the Levene and B-F tests are “fatally flawed”; It 
> isn’t clear how robust they are when there is significant differences in 
> variances and unequal sample sizes. Hill et. al (2006) advise repeating the 
> test using a non-parametric 
> 
>  method.
>
> On Monday, October 30, 2023 at 1:08:19 AM UTC 王富民awaw wrote:
>
>> Hi Jan,  Martin, and Jason
>>
>> Thanks for your tips and encouragement for rolling our sleeves.
>> I have put together a small statistics library:
>>
>> stat package - github.com/fumin/stat - Go Packages 
>> 
>>
>> For my own needs, in addition to Brown-Forsythe, I also needed the Welch 
>> t-test and tools for multiple testing, so these are what's in the above 
>> package.
>>
>> On Monday, 30 October 2023 at 07:25:50 UTC+8 Jason E. Aten wrote:
>>
>>> For the ANOVA, I usually 

Re: [go-nuts] Re: recommended statistics package? Requirements: ANOVA, Brown-Forsythe

[Jason E. Aten]

王富民awaw: Very nice.

The Levene function, though, is confusingly named. The Levene test is 
distinct from the Brown-Forsthe test, so mixing up the names
seems odd.[1]

[1] 
quoting 
https://www.statisticshowto.com/brown-forsythe-test/#:~:text=The%20Levene%20test%20uses%20deviations,a%20test%20that's%20more%20robust.
  
below: (see the underlined sentence, underline and bold there is mine)

How the Test Works

Both the Levene and B-F tests transform dependent variables 
 for use in 
an ANOVA  
test.
 
The only difference between the two tests is in how those transformed 
variables are constructed. The Levene test uses deviations from group means 
, which usually results in a 
highly-skewed set of data; This violates the assumption of normality 
.* The 
Brown-Forsythe test attempts to correct for this skewness  
by
 
using deviations from group medians 
.*
 
The result is a test that’s more robust 
. In other words, the 
B-F test is less likely than the Levene test to incorrectly declare that 
the assumption of equal variances has been violated.

The test works as follows:

   - The median is calculated for each factor level group.
   - The median value is subtracted from each dependent variable in the 
   group.
   - An ANOVA 
   

 is 
   run with the transformed variables. If a factor’s p-value 
   

 is 
   less than the significance level 
   

 (usually 
   5%), the population variances are not equal.

W or F statistic?

The test statistic used in a regular ANOVA is an F-statistic 
.
 
The statistic used in an ANOVA with transformed variables is sometimes 
called a W-Statistic — but it’s really just an F-Statistic with a different 
name. It should not be confused with the coefficient of concordance 
W-statistic 
, 
which is used to assess agreement between raters.
Cautions

For the most part, the B-F test is thought to perform as well as or better 
than other available tests for equal variances. However, Glass and Hopkins 
(1996 p. 436) state that the Levene and B-F tests are “fatally flawed”; It 
isn’t clear how robust they are when there is significant differences in 
variances and unequal sample sizes. Hill et. al (2006) advise repeating the 
test using a non-parametric 

 method.

On Monday, October 30, 2023 at 1:08:19 AM UTC 王富民awaw wrote:

> Hi Jan,  Martin, and Jason
>
> Thanks for your tips and encouragement for rolling our sleeves.
> I have put together a small statistics library:
>
> stat package - github.com/fumin/stat - Go Packages 
> 
>
> For my own needs, in addition to Brown-Forsythe, I also needed the Welch 
> t-test and tools for multiple testing, so these are what's in the above 
> package.
>
> On Monday, 30 October 2023 at 07:25:50 UTC+8 Jason E. Aten wrote:
>
>> For the ANOVA, I usually just call from Go into R for such things -- at 
>> least until I 
>> can validate if its the right thing to do/ meets the sensitivity/power 
>> needs of the analysis.
>>
>> https://statsandr.com/blog/anova-in-r/
>>
>>
>> https://github.com/glycerine/rmq#and-the-reverse-embedding-r-inside-your-golang-program
>>
>>
>> On Wednesday, October 25, 2023 at 6:48:56 AM UTC+1 Jan wrote:
>>
>>> So cool!
>>> On Monday, October 23, 2023 at 5:44:29 PM UTC+2 Martin Schnabel wrote:
>>>
 Hi, 

 I attempted to translate the linked JS implementation for fun. Maybe 
 someone can use it as a starting point and correct or verify its 
 correctness. 

 https://go.dev/play/p/Wrw2yDRof0z 

 Have fun! 

 On 10/23/23 07:38, Jan wrote: 
 > hi, I did a quick search and I didn't find anything in Go. But 
 looking 
 > at the definition and at one implementation in JS 
 > <
 https://github.com/lukem512/brown-forsythe-test/blob/master/src/brown-forsythe.js>,
  
 it sounds something relatively easy to write and share :)  You can use the 
 R implementation to create some test 

Re: [go-nuts] Re: recommended statistics package? Requirements: ANOVA, Brown-Forsythe

[王富民awaw]

Hi Jan,  Martin, and Jason

Thanks for your tips and encouragement for rolling our sleeves.
I have put together a small statistics library:

stat package - github.com/fumin/stat - Go Packages 


For my own needs, in addition to Brown-Forsythe, I also needed the Welch 
t-test and tools for multiple testing, so these are what's in the above 
package.

On Monday, 30 October 2023 at 07:25:50 UTC+8 Jason E. Aten wrote:

> For the ANOVA, I usually just call from Go into R for such things -- at 
> least until I 
> can validate if its the right thing to do/ meets the sensitivity/power 
> needs of the analysis.
>
> https://statsandr.com/blog/anova-in-r/
>
>
> https://github.com/glycerine/rmq#and-the-reverse-embedding-r-inside-your-golang-program
>
>
> On Wednesday, October 25, 2023 at 6:48:56 AM UTC+1 Jan wrote:
>
>> So cool!
>> On Monday, October 23, 2023 at 5:44:29 PM UTC+2 Martin Schnabel wrote:
>>
>>> Hi, 
>>>
>>> I attempted to translate the linked JS implementation for fun. Maybe 
>>> someone can use it as a starting point and correct or verify its 
>>> correctness. 
>>>
>>> https://go.dev/play/p/Wrw2yDRof0z 
>>>
>>> Have fun! 
>>>
>>> On 10/23/23 07:38, Jan wrote: 
>>> > hi, I did a quick search and I didn't find anything in Go. But looking 
>>> > at the definition and at one implementation in JS 
>>> > <
>>> https://github.com/lukem512/brown-forsythe-test/blob/master/src/brown-forsythe.js>,
>>>  
>>> it sounds something relatively easy to write and share :)  You can use the 
>>> R implementation to create some test datasets. Maybe gonum/stat <
>>> https://godocs.io/gonum.org/v1/gonum/stat> would be a potential home 
>>> for such a function ? What do you think ? 
>>> > 
>>> > cheers 
>>> > On Friday, October 20, 2023 at 10:54:55 AM UTC+2 王富民awaw wrote: 
>>> > 
>>> > Hi follow Gophers 
>>> > 
>>> > I wonder is there a canonical, verifiably correct Go package for 
>>> > statistics? 
>>> > In particular, Go code that does the Brown-Forsythe test of equal 
>>> > variance. 
>>> > Ideally in pure Go, but linking with CGo is OK. 
>>> > 
>>> > A search on Google and pkg.go.dev  does not 
>>> > return helpful results. 
>>> > I wonder is there anything that the community could share? 
>>> > 
>>> > -- 
>>> > You received this message because you are subscribed to the Google 
>>> > Groups "golang-nuts" group. 
>>> > To unsubscribe from this group and stop receiving emails from it, send 
>>> > an email to golang-nuts...@googlegroups.com 
>>> > . 
>>> > To view this discussion on the web visit 
>>> > 
>>> https://groups.google.com/d/msgid/golang-nuts/7ee10c0f-8af6-4b31-baaf-ce2ccb9c0211n%40googlegroups.com
>>>  
>>> <
>>> https://groups.google.com/d/msgid/golang-nuts/7ee10c0f-8af6-4b31-baaf-ce2ccb9c0211n%40googlegroups.com?utm_medium=email_source=footer>.
>>>  
>>>
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/f0437c9c-5ed5-42ec-ba4d-5dc3ee43ad40n%40googlegroups.com.

Re: [go-nuts] Re: recommended statistics package? Requirements: ANOVA, Brown-Forsythe

[Jason E. Aten]

For the ANOVA, I usually just call from Go into R for such things -- at 
least until I 
can validate if its the right thing to do/ meets the sensitivity/power 
needs of the analysis.

https://statsandr.com/blog/anova-in-r/

https://github.com/glycerine/rmq#and-the-reverse-embedding-r-inside-your-golang-program


On Wednesday, October 25, 2023 at 6:48:56 AM UTC+1 Jan wrote:

> So cool!
> On Monday, October 23, 2023 at 5:44:29 PM UTC+2 Martin Schnabel wrote:
>
>> Hi, 
>>
>> I attempted to translate the linked JS implementation for fun. Maybe 
>> someone can use it as a starting point and correct or verify its 
>> correctness. 
>>
>> https://go.dev/play/p/Wrw2yDRof0z 
>>
>> Have fun! 
>>
>> On 10/23/23 07:38, Jan wrote: 
>> > hi, I did a quick search and I didn't find anything in Go. But looking 
>> > at the definition and at one implementation in JS 
>> > <
>> https://github.com/lukem512/brown-forsythe-test/blob/master/src/brown-forsythe.js>,
>>  
>> it sounds something relatively easy to write and share :)  You can use the 
>> R implementation to create some test datasets. Maybe gonum/stat <
>> https://godocs.io/gonum.org/v1/gonum/stat> would be a potential home for 
>> such a function ? What do you think ? 
>> > 
>> > cheers 
>> > On Friday, October 20, 2023 at 10:54:55 AM UTC+2 王富民awaw wrote: 
>> > 
>> > Hi follow Gophers 
>> > 
>> > I wonder is there a canonical, verifiably correct Go package for 
>> > statistics? 
>> > In particular, Go code that does the Brown-Forsythe test of equal 
>> > variance. 
>> > Ideally in pure Go, but linking with CGo is OK. 
>> > 
>> > A search on Google and pkg.go.dev  does not 
>> > return helpful results. 
>> > I wonder is there anything that the community could share? 
>> > 
>> > -- 
>> > You received this message because you are subscribed to the Google 
>> > Groups "golang-nuts" group. 
>> > To unsubscribe from this group and stop receiving emails from it, send 
>> > an email to golang-nuts...@googlegroups.com 
>> > . 
>> > To view this discussion on the web visit 
>> > 
>> https://groups.google.com/d/msgid/golang-nuts/7ee10c0f-8af6-4b31-baaf-ce2ccb9c0211n%40googlegroups.com
>>  
>> <
>> https://groups.google.com/d/msgid/golang-nuts/7ee10c0f-8af6-4b31-baaf-ce2ccb9c0211n%40googlegroups.com?utm_medium=email_source=footer>.
>>  
>>
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/c4b3018b-08cd-4591-8ba6-ff2ffd40ecc7n%40googlegroups.com.

Re: [go-nuts] Re: recommended statistics package? Requirements: ANOVA, Brown-Forsythe

[Jan]

So cool!
On Monday, October 23, 2023 at 5:44:29 PM UTC+2 Martin Schnabel wrote:

> Hi,
>
> I attempted to translate the linked JS implementation for fun. Maybe 
> someone can use it as a starting point and correct or verify its 
> correctness.
>
> https://go.dev/play/p/Wrw2yDRof0z
>
> Have fun!
>
> On 10/23/23 07:38, Jan wrote:
> > hi, I did a quick search and I didn't find anything in Go. But looking 
> > at the definition and at one implementation in JS 
> > <
> https://github.com/lukem512/brown-forsythe-test/blob/master/src/brown-forsythe.js>,
>  
> it sounds something relatively easy to write and share :)  You can use the 
> R implementation to create some test datasets. Maybe gonum/stat <
> https://godocs.io/gonum.org/v1/gonum/stat> would be a potential home for 
> such a function ? What do you think ?
> > 
> > cheers
> > On Friday, October 20, 2023 at 10:54:55 AM UTC+2 王富民awaw wrote:
> > 
> > Hi follow Gophers
> > 
> > I wonder is there a canonical, verifiably correct Go package for
> > statistics?
> > In particular, Go code that does the Brown-Forsythe test of equal
> > variance.
> > Ideally in pure Go, but linking with CGo is OK.
> > 
> > A search on Google and pkg.go.dev  does not
> > return helpful results.
> > I wonder is there anything that the community could share?
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> > Groups "golang-nuts" group.
> > To unsubscribe from this group and stop receiving emails from it, send 
> > an email to golang-nuts...@googlegroups.com 
> > .
> > To view this discussion on the web visit 
> > 
> https://groups.google.com/d/msgid/golang-nuts/7ee10c0f-8af6-4b31-baaf-ce2ccb9c0211n%40googlegroups.com
>  
> <
> https://groups.google.com/d/msgid/golang-nuts/7ee10c0f-8af6-4b31-baaf-ce2ccb9c0211n%40googlegroups.com?utm_medium=email_source=footer
> >.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0b10338e-2fe8-4e55-b6c6-a7c2870df382n%40googlegroups.com.

Re: [go-nuts] Re: recommended statistics package? Requirements: ANOVA, Brown-Forsythe

[Martin Schnabel]

Hi,

I attempted to translate the linked JS implementation for fun. Maybe 
someone can use it as a starting point and correct or verify its 
correctness.


https://go.dev/play/p/Wrw2yDRof0z

Have fun!

On 10/23/23 07:38, Jan wrote:
hi, I did a quick search and I didn't find anything in Go. But looking 
at the definition and at one implementation in JS 
, it sounds something relatively easy to write and share :)  You can use the R implementation to create some test datasets. Maybe gonum/stat  would be a potential home for such a function ? What do you think ?


cheers
On Friday, October 20, 2023 at 10:54:55 AM UTC+2 王富民awaw wrote:

Hi follow Gophers

I wonder is there a canonical, verifiably correct Go package for
statistics?
In particular, Go code that does the Brown-Forsythe test of equal
variance.
Ideally in pure Go, but linking with CGo is OK.

A search on Google and pkg.go.dev  does not
return helpful results.
I wonder is there anything that the community could share?

--
You received this message because you are subscribed to the Google 
Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to golang-nuts+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/7ee10c0f-8af6-4b31-baaf-ce2ccb9c0211n%40googlegroups.com .


--
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/7fe194f0-2d29-4b44-8654-e857365ba5cc%40mb0.org.

[go-nuts] Re: recommended statistics package? Requirements: ANOVA, Brown-Forsythe

[Jan]

hi, I did a quick search and I didn't find anything in Go. But looking at 
the definition and at one implementation in JS 
,
 
it sounds something relatively easy to write and share :)  You can use the 
R implementation to create some test datasets. Maybe gonum/stat 
 would be a potential home for 
such a function ? What do you think ?

cheers 
On Friday, October 20, 2023 at 10:54:55 AM UTC+2 王富民awaw wrote:

> Hi follow Gophers
>
> I wonder is there a canonical, verifiably correct Go package for 
> statistics?
> In particular, Go code that does the Brown-Forsythe test of equal variance.
> Ideally in pure Go, but linking with CGo is OK.
>
> A search on Google and pkg.go.dev does not return helpful results.
> I wonder is there anything that the community could share?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/7ee10c0f-8af6-4b31-baaf-ce2ccb9c0211n%40googlegroups.com.