[GitHub] cjolivier01 commented on a change in pull request #7559: nightly build stochastically choose optimizer
cjolivier01 commented on a change in pull request #7559: nightly build
stochastically choose optimizer
URL: https://github.com/apache/incubator-mxnet/pull/7559#discussion_r134611795
##
File path: tests/nightly/test_all.sh
##
@@ -72,9 +80,15 @@ check_val() {
example_dir=../../example/image-classification
# python: lenet + mnist
test_lenet() {
-python $example_dir/train_mnist.py \
---data-dir `pwd`/data/mnist/ --network lenet --gpus $gpus --num-epochs
10 \
-2>&1 | tee log
+optimizer=$(get_random_optimizer)
Review comment:
(Naveen does not believe in stochastic testing methodology) :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] cjolivier01 commented on a change in pull request #7559: nightly build stochastically choose optimizer
cjolivier01 commented on a change in pull request #7559: nightly build
stochastically choose optimizer
URL: https://github.com/apache/incubator-mxnet/pull/7559#discussion_r134585168
##
File path: tests/nightly/test_all.sh
##
@@ -72,9 +80,15 @@ check_val() {
example_dir=../../example/image-classification
# python: lenet + mnist
test_lenet() {
-python $example_dir/train_mnist.py \
---data-dir `pwd`/data/mnist/ --network lenet --gpus $gpus --num-epochs
10 \
-2>&1 | tee log
+optimizer=$(get_random_optimizer)
Review comment:
ok, that's two people who want them all, so adding :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] cjolivier01 commented on a change in pull request #7559: nightly build stochastically choose optimizer
cjolivier01 commented on a change in pull request #7559: nightly build
stochastically choose optimizer
URL: https://github.com/apache/incubator-mxnet/pull/7559#discussion_r134561760
##
File path: tests/nightly/test_all.sh
##
@@ -72,9 +80,15 @@ check_val() {
example_dir=../../example/image-classification
# python: lenet + mnist
test_lenet() {
-python $example_dir/train_mnist.py \
---data-dir `pwd`/data/mnist/ --network lenet --gpus $gpus --num-epochs
10 \
-2>&1 | tee log
+optimizer=$(get_random_optimizer)
Review comment:
Yes, it would be caught by any optimizer other than sgd. When the test is
run, the full set of options is displayed, so copy-pasting that will
reproduce.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] cjolivier01 commented on a change in pull request #7559: nightly build stochastically choose optimizer
cjolivier01 commented on a change in pull request #7559: nightly build
stochastically choose optimizer
URL: https://github.com/apache/incubator-mxnet/pull/7559#discussion_r134561760
##
File path: tests/nightly/test_all.sh
##
@@ -72,9 +80,15 @@ check_val() {
example_dir=../../example/image-classification
# python: lenet + mnist
test_lenet() {
-python $example_dir/train_mnist.py \
---data-dir `pwd`/data/mnist/ --network lenet --gpus $gpus --num-epochs
10 \
-2>&1 | tee log
+optimizer=$(get_random_optimizer)
Review comment:
Yes, it would be caught by any optimizer other than sgd. When he test is
run, the full set of options is displayed, so copy-pasting that will
reproduce.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] cjolivier01 commented on a change in pull request #7559: nightly build stochastically choose optimizer
cjolivier01 commented on a change in pull request #7559: nightly build
stochastically choose optimizer
URL: https://github.com/apache/incubator-mxnet/pull/7559#discussion_r134547748
##
File path: tests/nightly/test_all.sh
##
@@ -72,9 +80,15 @@ check_val() {
example_dir=../../example/image-classification
# python: lenet + mnist
test_lenet() {
-python $example_dir/train_mnist.py \
---data-dir `pwd`/data/mnist/ --network lenet --gpus $gpus --num-epochs
10 \
-2>&1 | tee log
+optimizer=$(get_random_optimizer)
Review comment:
You mean back-to-back or as part of the stochastic set? Back-to-back would
be too slow.
The individual optimizers themselves are tested as part of the unit tests in
test_optimizer.py (which doesn't use fit.py logic).
I selected these three because they're the most common and are mostly just
testing the fit.py logic. They all have much different parameters from each
other, so should catch problems with parameters (which is what caused this
problem). My logic was by adding too many optimizers, the listed optimizers
could go a very long time (more than a week) without being tested, and failures
would seem much more random. For example, SGD is the default and several
optimizers are derived from SGD.
It's certainly possible to test them all, however.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
