[chromium-dev] Re: Running UI test in parallel (experimental)
On Wed, Aug 19, 2009 at 9:54 PM, Huan Ren hu...@google.com wrote:
On Thu, Aug 6, 2009 at 1:06 PM, John Abd-El-Malek j...@chromium.orgwrote:
This is very cool, but I ran into a few problems when I tried to run it:
a:\chrome2\src\chrometools\test\smoketests.py --tests=ui
You must have your local path of trunk/src/tools/python added to your
PYTHONPATH.
Traceback (most recent call last):
File A:\chrome2\src\chrome\tools\test\smoketests.py, line 32, in
module
import google.httpd_utils
ImportError: No module named google.httpd_utils
hmmm, never needed PYTHONPATH set before. Can't this script set it
itself? Setting it manually will fail when I move depot locations etc..
But anyways, I set it and then
a:\chrome2\src\chromeset PYTHONPATH=a:\chrome\src\tools\python
a:\chrome2\src\chrometools\test\smoketests.py --tests=ui
Traceback (most recent call last):
File A:\chrome2\src\chrome\tools\test\smoketests.py, line 274, in
module
result = main(options, args)
File A:\chrome2\src\chrome\tools\test\smoketests.py, line 155, in main
sys.path.insert(0, _BuildbotScriptPath('slave'))
File A:\chrome2\src\chrome\tools\test\smoketests.py, line 84, in
_BuildbotScriptPath
'scripts', sub_dir)
File a:\chrome\src\tools\python\google\path_utils.py, line 72, in
FindUpward
parent = FindUpwardParent(start_dir, *desired_list)
File a:\chrome\src\tools\python\google\path_utils.py, line 55, in
FindUpwardParent
(desired_path, start_dir))
google.path_utils.PathNotFound: Unable to find
tools\buildbot\scripts\slave above A:\chrome2\src\chrome\tools\test
tools\buildbot isn't in the public tree I think, since I don't find it
here: http://src.chromium.org/viewvc/chrome/trunk/src/chrome/tools/. So
external contributors can't use this? Also, why should this script depend
on the buildbot scripts? I don't have them so I can't try this out.
It is externally available.
http://src.chromium.org/viewvc/chrome/trunk/tools/buildbot/
http://src.chromium.org/viewvc/chrome/trunk/tools/buildbot/
Can't we just have a minimal python script that runs ui_tests in parallel?
Pam wrote the original smoketests.py. Pam, is it easy to drop those
dependencies? Otherwise, I will write a minimal python script.
I'll take a look. At a quick glance, it looks like the buildbot slave
scripts are only needed if you're running layout tests, so I'll try to
extract that.
- Pam
Huan
On Wed, Jul 29, 2009 at 3:28 PM, Huan Ren hu...@google.com wrote:
I just checked in a change to run ui_tests in parallel based on
sharding mechanism provided by GTest. Each ui_tests instance has its
own user data dir, and the number of ui_tests instances is
NUMBER_OF_PROCESSORS. I have updated
src/chrome/tools/test/smoketests.py so you can run it through command
line:
python.exe smoketests.py --tests=ui [--verbose]
Running ui_tests.exe directly is still the old behavior of
sequentially running. On my 4 core machine, the running time has been
reduced by half, from 832 secs to 443 secs. But I need to make sure
all tests can run reliably in this parallel fashion. So if you try it
out, I will be interested to know how fast the performance is improved
and what additional tests are failing.
Huan
P.S. this change is for Windows platform as I think Linux/Mac is
already using GTest sharding.
--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com
View archives, change email options, or unsubscribe:
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---
[chromium-dev] Re: Running UI test in parallel (experimental)
On Thu, Aug 6, 2009 at 1:06 PM, John Abd-El-Malek j...@chromium.org wrote:
This is very cool, but I ran into a few problems when I tried to run it:
a:\chrome2\src\chrometools\test\smoketests.py --tests=ui
You must have your local path of trunk/src/tools/python added to your
PYTHONPATH.
Traceback (most recent call last):
File A:\chrome2\src\chrome\tools\test\smoketests.py, line 32, in
module
import google.httpd_utils
ImportError: No module named google.httpd_utils
hmmm, never needed PYTHONPATH set before. Can't this script set it itself?
Setting it manually will fail when I move depot locations etc.. But
anyways, I set it and then
a:\chrome2\src\chromeset PYTHONPATH=a:\chrome\src\tools\python
a:\chrome2\src\chrometools\test\smoketests.py --tests=ui
Traceback (most recent call last):
File A:\chrome2\src\chrome\tools\test\smoketests.py, line 274, in
module
result = main(options, args)
File A:\chrome2\src\chrome\tools\test\smoketests.py, line 155, in main
sys.path.insert(0, _BuildbotScriptPath('slave'))
File A:\chrome2\src\chrome\tools\test\smoketests.py, line 84, in
_BuildbotScriptPath
'scripts', sub_dir)
File a:\chrome\src\tools\python\google\path_utils.py, line 72, in
FindUpward
parent = FindUpwardParent(start_dir, *desired_list)
File a:\chrome\src\tools\python\google\path_utils.py, line 55, in
FindUpwardParent
(desired_path, start_dir))
google.path_utils.PathNotFound: Unable to find tools\buildbot\scripts\slave
above A:\chrome2\src\chrome\tools\test
tools\buildbot isn't in the public tree I think, since I don't find it
here: http://src.chromium.org/viewvc/chrome/trunk/src/chrome/tools/. So
external contributors can't use this? Also, why should this script depend
on the buildbot scripts? I don't have them so I can't try this out.
It is externally available.
http://src.chromium.org/viewvc/chrome/trunk/tools/buildbot/
http://src.chromium.org/viewvc/chrome/trunk/tools/buildbot/
Can't we just have a minimal python script that runs ui_tests in parallel?
Pam wrote the original smoketests.py. Pam, is it easy to drop those
dependencies? Otherwise, I will write a minimal python script.
Huan
On Wed, Jul 29, 2009 at 3:28 PM, Huan Ren hu...@google.com wrote:
I just checked in a change to run ui_tests in parallel based on
sharding mechanism provided by GTest. Each ui_tests instance has its
own user data dir, and the number of ui_tests instances is
NUMBER_OF_PROCESSORS. I have updated
src/chrome/tools/test/smoketests.py so you can run it through command
line:
python.exe smoketests.py --tests=ui [--verbose]
Running ui_tests.exe directly is still the old behavior of
sequentially running. On my 4 core machine, the running time has been
reduced by half, from 832 secs to 443 secs. But I need to make sure
all tests can run reliably in this parallel fashion. So if you try it
out, I will be interested to know how fast the performance is improved
and what additional tests are failing.
Huan
P.S. this change is for Windows platform as I think Linux/Mac is
already using GTest sharding.
--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com
View archives, change email options, or unsubscribe:
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---
[chromium-dev] Re: Running UI test in parallel (experimental)
This is very cool, but I ran into a few problems when I tried to run it:
a:\chrome2\src\chrometools\test\smoketests.py --tests=ui
You must have your local path of trunk/src/tools/python added to your
PYTHONPATH.
Traceback (most recent call last):
File A:\chrome2\src\chrome\tools\test\smoketests.py, line 32, in
module
import google.httpd_utils
ImportError: No module named google.httpd_utils
hmmm, never needed PYTHONPATH set before. Can't this script set it itself?
Setting it manually will fail when I move depot locations etc.. But
anyways, I set it and then
a:\chrome2\src\chromeset PYTHONPATH=a:\chrome\src\tools\python
a:\chrome2\src\chrometools\test\smoketests.py --tests=ui
Traceback (most recent call last):
File A:\chrome2\src\chrome\tools\test\smoketests.py, line 274, in
module
result = main(options, args)
File A:\chrome2\src\chrome\tools\test\smoketests.py, line 155, in main
sys.path.insert(0, _BuildbotScriptPath('slave'))
File A:\chrome2\src\chrome\tools\test\smoketests.py, line 84, in
_BuildbotScriptPath
'scripts', sub_dir)
File a:\chrome\src\tools\python\google\path_utils.py, line 72, in
FindUpward
parent = FindUpwardParent(start_dir, *desired_list)
File a:\chrome\src\tools\python\google\path_utils.py, line 55, in
FindUpwardParent
(desired_path, start_dir))
google.path_utils.PathNotFound: Unable to find tools\buildbot\scripts\slave
above A:\chrome2\src\chrome\tools\test
tools\buildbot isn't in the public tree I think, since I don't find it
here: http://src.chromium.org/viewvc/chrome/trunk/src/chrome/tools/. So
external contributors can't use this? Also, why should this script depend
on the buildbot scripts? I don't have them so I can't try this out.
Can't we just have a minimal python script that runs ui_tests in parallel?
On Wed, Jul 29, 2009 at 3:28 PM, Huan Ren hu...@google.com wrote:
I just checked in a change to run ui_tests in parallel based on
sharding mechanism provided by GTest. Each ui_tests instance has its
own user data dir, and the number of ui_tests instances is
NUMBER_OF_PROCESSORS. I have updated
src/chrome/tools/test/smoketests.py so you can run it through command
line:
python.exe smoketests.py --tests=ui [--verbose]
Running ui_tests.exe directly is still the old behavior of
sequentially running. On my 4 core machine, the running time has been
reduced by half, from 832 secs to 443 secs. But I need to make sure
all tests can run reliably in this parallel fashion. So if you try it
out, I will be interested to know how fast the performance is improved
and what additional tests are failing.
Huan
P.S. this change is for Windows platform as I think Linux/Mac is
already using GTest sharding.
--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com
View archives, change email options, or unsubscribe:
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---
[chromium-dev] Re: Running UI test in parallel (experimental)
On Thu, Aug 6, 2009 at 1:06 PM, John Abd-El-Malek j...@chromium.org wrote:
This is very cool, but I ran into a few problems when I tried to run it:
a:\chrome2\src\chrometools\test\smoketests.py --tests=ui
You must have your local path of trunk/src/tools/python added to your
PYTHONPATH.
Traceback (most recent call last):
File A:\chrome2\src\chrome\tools\test\smoketests.py, line 32, in
module
import google.httpd_utils
ImportError: No module named google.httpd_utils
You get this error when you use a python that is different from the one in
src\third_party\python_24\python.exe
hmmm, never needed PYTHONPATH set before. Can't this script set it itself?
Setting it manually will fail when I move depot locations etc.. But
anyways, I set it and then
a:\chrome2\src\chromeset PYTHONPATH=a:\chrome\src\tools\python
a:\chrome2\src\chrometools\test\smoketests.py --tests=ui
Traceback (most recent call last):
File A:\chrome2\src\chrome\tools\test\smoketests.py, line 274, in
module
result = main(options, args)
File A:\chrome2\src\chrome\tools\test\smoketests.py, line 155, in main
sys.path.insert(0, _BuildbotScriptPath('slave'))
File A:\chrome2\src\chrome\tools\test\smoketests.py, line 84, in
_BuildbotScriptPath
'scripts', sub_dir)
File a:\chrome\src\tools\python\google\path_utils.py, line 72, in
FindUpward
parent = FindUpwardParent(start_dir, *desired_list)
File a:\chrome\src\tools\python\google\path_utils.py, line 55, in
FindUpwardParent
(desired_path, start_dir))
google.path_utils.PathNotFound: Unable to find tools\buildbot\scripts\slave
above A:\chrome2\src\chrome\tools\test
tools\buildbot isn't in the public tree I think, since I don't find it
here: http://src.chromium.org/viewvc/chrome/trunk/src/chrome/tools/. So
external contributors can't use this? Also, why should this script depend
on the buildbot scripts? I don't have them so I can't try this out.
http://src.chromium.org/viewvc/chrome/trunk/tools/buildbot/
Not sure why it needs it though.
Can't we just have a minimal python script that runs ui_tests in parallel?
Nicolas
On Wed, Jul 29, 2009 at 3:28 PM, Huan Ren hu...@google.com wrote:
I just checked in a change to run ui_tests in parallel based on
sharding mechanism provided by GTest. Each ui_tests instance has its
own user data dir, and the number of ui_tests instances is
NUMBER_OF_PROCESSORS. I have updated
src/chrome/tools/test/smoketests.py so you can run it through command
line:
python.exe smoketests.py --tests=ui [--verbose]
Running ui_tests.exe directly is still the old behavior of
sequentially running. On my 4 core machine, the running time has been
reduced by half, from 832 secs to 443 secs. But I need to make sure
all tests can run reliably in this parallel fashion. So if you try it
out, I will be interested to know how fast the performance is improved
and what additional tests are failing.
Huan
P.S. this change is for Windows platform as I think Linux/Mac is
already using GTest sharding.
--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com
View archives, change email options, or unsubscribe:
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---
[chromium-dev] Re: Running UI test in parallel (experimental)
Does each shard get its own profile? Or do they share? Does this mean that when you shard they're sharing the same browser process? Erik On Wed, Jul 29, 2009 at 3:28 PM, Huan Ren hu...@google.com wrote: I just checked in a change to run ui_tests in parallel based on sharding mechanism provided by GTest. Each ui_tests instance has its own user data dir, and the number of ui_tests instances is NUMBER_OF_PROCESSORS. I have updated src/chrome/tools/test/smoketests.py so you can run it through command line: python.exe smoketests.py --tests=ui [--verbose] Running ui_tests.exe directly is still the old behavior of sequentially running. On my 4 core machine, the running time has been reduced by half, from 832 secs to 443 secs. But I need to make sure all tests can run reliably in this parallel fashion. So if you try it out, I will be interested to know how fast the performance is improved and what additional tests are failing. Huan P.S. this change is for Windows platform as I think Linux/Mac is already using GTest sharding. --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Running UI test in parallel (experimental)
Each gets it own profile thus different browser process. Huan On Wed, Jul 29, 2009 at 6:01 PM, Erik Kayerik...@chromium.org wrote: Does each shard get its own profile? Or do they share? Does this mean that when you shard they're sharing the same browser process? Erik On Wed, Jul 29, 2009 at 3:28 PM, Huan Ren hu...@google.com wrote: I just checked in a change to run ui_tests in parallel based on sharding mechanism provided by GTest. Each ui_tests instance has its own user data dir, and the number of ui_tests instances is NUMBER_OF_PROCESSORS. I have updated src/chrome/tools/test/smoketests.py so you can run it through command line: python.exe smoketests.py --tests=ui [--verbose] Running ui_tests.exe directly is still the old behavior of sequentially running. On my 4 core machine, the running time has been reduced by half, from 832 secs to 443 secs. But I need to make sure all tests can run reliably in this parallel fashion. So if you try it out, I will be interested to know how fast the performance is improved and what additional tests are failing. Huan P.S. this change is for Windows platform as I think Linux/Mac is already using GTest sharding. --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
