GitHub user shahidki31 opened a pull request:
https://github.com/apache/spark/pull/22659
[SPARK-25623][TEST] Reduce test time of LogisticRegressionSuite:
multinomial logistic regression....
...with intercept with L1 regularization
## What changes were proposed in this pull request?
In the test, "multinomial logistic regression with intercept with L1
regularization" in the "LogisticRegressionSuite", taking more than a minute due
to training of 2 logistic regression model.
However after analysing the training cost over iteration, we can reduce the
computation time by 50%.
Training cost vs iteration for mode1
So, model1 is converging after iteration 150.
Training cost vs iteration for model2
After around 100 iteration, model2 is converging.
So, if we give maximum iteration for model1 and model2 as 175 and 125
respectively, we can reduce the computation time by half.
## How was this patch tested?
Computation time in local setup :
Before change:
~53 sec
After change:
~26 sec
Please review http://spark.apache.org/contributing.html before opening a
pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shahidki31/spark SPARK-25623
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22659.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22659
----
commit 2040ada029bc8f8b894b724706acb0450c2874b5
Author: Shahid <shahidki31@...>
Date: 2018-10-06T16:55:28Z
[SPARK-25623]LogisticRegressionSuite: multinomial logistic regressioN with
intercept with L1 regularization 1 min 10 sec
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
