[spark] branch branch-3.0 updated: [MINOR][3.1][3.0] Improve flaky NaiveBayes test

[ruifengz]

This is an automated email from the ASF dual-hosted git repository.

ruifengz pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.0 by this push:
     new 4cbc177  [MINOR][3.1][3.0] Improve flaky NaiveBayes test
4cbc177 is described below

commit 4cbc17792a2a31bd0c04cc5840294663646f81c5
Author: Weichen Xu <weichen...@databricks.com>
AuthorDate: Mon Jan 11 18:14:52 2021 +0800

    [MINOR][3.1][3.0] Improve flaky NaiveBayes test
    
    ### What changes were proposed in this pull request?
    
    Current test may sometimes fail under different BLAS library. Due to some 
absTol check. Error like
    ```
    Expected 0.7 and 0.6485507246376814 to be within 0.05 using absolute 
tolerance...
    
    ```
    
    * Change absTol to relTol: The `absTol 0.05` in some cases (such as compare 
0.1 and 0.05) is a big difference
    * Remove the `exp` when comparing params. The `exp` will amplify the 
relative error.
    
    ### Why are the changes needed?
    Flaky test
    
    ### Does this PR introduce _any_ user-facing change?
    no
    
    ### How was this patch tested?
    N/A
    
    Closes #31004 from WeichenXu123/improve_bayes_tests.
    
    Authored-by: Weichen Xu <weichen.xudatabricks.com>
    Signed-off-by: Ruifeng Zheng <ruifengzfoxmail.com>
    
    ### What changes were proposed in this pull request?
    
    ### Why are the changes needed?
    
    ### Does this PR introduce _any_ user-facing change?
    
    ### How was this patch tested?
    
    Closes #31123 from WeichenXu123/bp-3.1-nb-test.
    
    Authored-by: Weichen Xu <weichen...@databricks.com>
    Signed-off-by: Ruifeng Zheng <ruife...@foxmail.com>
    (cherry picked from commit d33f0d44ec7c58a86c11c40a8f5933b333175ceb)
    Signed-off-by: Ruifeng Zheng <ruife...@foxmail.com>
---
 .../scala/org/apache/spark/mllib/classification/NaiveBayesSuite.scala | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git 
a/mllib/src/test/scala/org/apache/spark/mllib/classification/NaiveBayesSuite.scala
 
b/mllib/src/test/scala/org/apache/spark/mllib/classification/NaiveBayesSuite.scala
index 47dac3e..c206747 100644
--- 
a/mllib/src/test/scala/org/apache/spark/mllib/classification/NaiveBayesSuite.scala
+++ 
b/mllib/src/test/scala/org/apache/spark/mllib/classification/NaiveBayesSuite.scala
@@ -107,9 +107,9 @@ class NaiveBayesSuite extends SparkFunSuite with 
MLlibTestSparkContext {
     val modelIndex = piData.indices.zip(model.labels.map(_.toInt))
     try {
       for (i <- modelIndex) {
-        assert(math.exp(piData(i._2)) ~== math.exp(model.pi(i._1)) absTol 0.05)
+        assert(piData(i._2) ~== model.pi(i._1) relTol 0.15)
         for (j <- thetaData(i._2).indices) {
-          assert(math.exp(thetaData(i._2)(j)) ~== 
math.exp(model.theta(i._1)(j)) absTol 0.05)
+          assert(thetaData(i._2)(j) ~== model.theta(i._1)(j) relTol 0.15)
         }
       }
     } catch {


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org