Re: [chromium-dev] Running multiple Google Chrome channels side-by-side (for testing)

[Huan Ren]

It involves quite a bit code and process change, but we plan to make it
happen.

http://code.google.com/p/chromium/issues/detail?id=27931

http://code.google.com/p/chromium/issues/detail?id=27933

Huan

On Thu, Dec 17, 2009 at 10:27 PM, Aaron Boodman  wrote:

> On Windows, is there a way to hack the registry to allow multiple
> channels of Chrome to be installed simultaneously and let them update?
> It would be really useful for testing.
>
> - a
>
> --
> Chromium Developers mailing list: chromium-dev@googlegroups.com
> View archives, change email options, or unsubscribe:
>http://groups.google.com/group/chromium-dev
>

-- 
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev

[chromium-dev] Re: Changes to using threads in the browser process

[Huan Ren]

Cool! I could compare the builds before and after these changes to see
what difference it makes. Of course it also prevents future issues.

Huan

On Mon, Nov 2, 2009 at 9:50 PM, John Abd-El-Malek  wrote:
> Over the last week, I've been making some changes to how threads are used in
> the browser process, with the goal of simplifying cross-thread access and
> improving stability.
> The problem
> We were using a number of patterns that were problematic:
> 1) using ChromeThread::GetMessageLoop
>   -this isn't safe, since it could return a valid pointer, but since the
> caller isn't holding on to a lock anymore, the target MessageLoop could be
> destructed while it's being used
> 2) caching of MessageLoop pointers in order to use them later for PostTask
> and friends
>   -this was more efficient previously (more on that later) since using
> ChromeThread::GetMessageLoop involved a lock
>   -but it spread logic about the order of thread destruction all over the
> code.  Code that moved from the IO thread to the file thread and back, in
> order to avoid doing disk access on the IO thread, ended up having to do an
> extra hop through the UI thread on the way back to the IO thread since the
> file thread outlives the Io thread.  Of course, most code learnt this the
> hard way, after doing the straight forward IO->file->IO thread hop and
> updating the code after seeing reliability or user crashes
>   -it made the browser shutdown fragile and hence difficult to update
> 3) file thread hops using RefCountedThreadSafe objects which have
> non-trivial destructors
>   -to reduce jank, frequently an object on the UI or IO thread would execute
> some code on the file thread, then post the result back to the original
> thread.  We make this easy using RefCountedThreadSafe and NewRunnableMethod
> so this pattern happened often, but it's not always safe: NewRunnableMethod
> holds an extra reference on the object to ensure that it doesn't invoke a
> method on a deleted object.  But it's quite possible that before the file
> thread's stack unwinds and it releases the extra reference, that the
> response task on the original thread executed and released its own
> additional reference.  The file thread is then left with the remaining
> reference and the object gets destructed there.  While for most objects this
> is ok, many have non-trivial destructors, with the worst being ones that
> register with the per-thread NotificationService.  Dangling pointers would
> be left behind and tracking these crashes from ChromeBot or the crash dumps
> has wasted several days at least for me.
> 4) having browser code take different code paths if a thread didn't exist
>   -this could be either deceptively harmless  (i.e. execute synchronously
> when it was normally asynchronous), when in fact it makes shutdown slower
> because disk access is moved to the UI thread
>   -it could lead to data loss, if tasks are silently not posted because the
> code assumes this only happens in unit tests, when it could occur on browser
> shutdown as well
>
> The solution
> 1+2: Use ChromeThread::PostTask and friends (i.e. PostDelayedTask,
> DeleteSoon, ReleaseSoon) which are safe and efficient: no locks are grabbed
> if the target thread is known to outlive the current thread.  The four
> static methods have the same signature as the ones from MessageLoop, with
> the addition of the first parameter to indicate the target thread.
> ChromeThread::PostTask(ChromeThread::FILE, FROM_HERE, task);
> 3: If your object must be destructed on a specific thread, use a trait from
> ChromeThread:
> class Foo : public base::RefCountedThreadSafe ChromeThread::DeleteOnIOThread>
> 4: I've removed all the special casing and always made the objects in the
> browser code behave in one way.  If you're writing a unit test and need to
> use an object that goes to a file thread (where before it would proceed
> synchronously), you just need:
> ChromeThread file_thread(ChromeThread::FILE, MessageLoop::current());
> foo->StartAsyncTaskOnFileThread();
> MessageLoop::current()->RunAllPending();
> There are plenty of examples now in the tests.
>
> Gotchas
> -PostTask will silently delete a task if the thread doesn't exist.  This is
> done to avoid having all the code that uses it have special cases if the
> target thread exists or not, and to make Valgrind happy.  As usual, the task
> for DeleteSoon/ReleaseSoon doesn't do anything in its destructor, so this
> won't cause unexpected behavior with them.  But be wary of posted Task
> objects which have non-trivial destructors or smart pointers as members.
>  I'm still on the fence about this, since while the latter is theoretical
> now, it could lead to problems in the future.  I might change it so that the
> tasks are not deleted when I'm ready for more Valgrind fun.
> If you absolutely must know if a task was posted or not, you can check the
> return value of PostTask and friends.  But note that even if the task was
> posted successfully, th

[chromium-dev] Re: inconsistent file_util::CopyDirectory() behavior

[Huan Ren]

file_util::CopyDirectory is used by Windows installer so we really
need to make sure the change does not break installer.

Huan

On Mon, Oct 12, 2009 at 3:52 PM, Steven Knight  wrote:
> Our existing unit tests for file_util::CopyDirectory() do not test its
> behavior when the destination directory already exists.  The following CL:
>
> http://codereview.chromium.org/271060/show
>
> which adds unit tests for already-existing destination directories, shows
> that Windows and POSIX behave differently for recursive copies when the
> destination directory already exists.  Specifically:
>
> On Windows, a call to file_util::CopyDirectory() with recursive copy enabled
> will copy the directory name itself to ay(already-existing) destination
> directory.  POSIX systems will copy the *contents* of the directory to the
> destination.  That is, given a 'src' directory containing two files and a
> call like:
>
>     file_util::CopyDirectory('src'
> , 'existing_dest_dir', true);
>
> On Windows we create 'existing_dest_dir/src/{file1,file2}' while on POSIX we
> create just 'existing_dest_dir/{file1,file2}'.
>
> This has come up for memory_test, which uses CopyDirectory() to copy its
> checked-in cached user data dir to a freshly-created temporary directory.
> At some point the Windows version of memory_test.cc was broken by this
> inconsistency.  We haven't noticed because the "breakage" fails to load the
> (cached) pages, but still returns memory size data and doesn't cause the
> test itself to fail.
>
> Based on the fact that memory_test.cc originally worked on Windows, it seems
> like the POSIX behavior is correct/intended (especially since it's
> consistent with the behavior when the directory doesn't exist).  Any
> disagreement?  If not, I'll fix Windows accordingly.
>
>     --SK
> >
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Paper about DRAM error rates

[Huan Ren]

It will be helpful to get our own measurement on database failures.
Carlos just added something like that.

Huan

On Tue, Oct 6, 2009 at 3:49 PM, John Abd-El-Malek  wrote:
> Saw this on
> slashdot: http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf
> The conclusion is "an average of 25,000–75,000 FIT (failures in time per
> billion hours of operation) per Mbit".
> On my machine the browser process is usually > 100MB, so that averages out
> to 176 to 493 error per year, with those numbers having big variance
> depending on the machine.  Since most users don't have ECC, which means this
> will lead to corruption.  Sqlite is a heavy user of memory, so even if it's
> 1/4 of the 100MB, that means we'll see an average of 40-120 errors naturally
> because of faulty DIMMs.
> Given that sqlite corruption means (repeated) crashing of the browser
> process, it seems this data heavily suggests we should separate sqlite code
> into a separate process.  The IPC overhead is negligible compared to disk
> access.  My hunch is that the complexity is also not that high, since the
> code that deals with it is already asynchronous since we don't use sqlite on
> the UI/IO threads.
> What do others think?
> >
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Build failure

[Huan Ren]

On Thu, Oct 1, 2009 at 10:02 PM, Bala  wrote:
>
> Hi All,
>
> I tried to build chromium by following the Windows Build Instructions
> for the first time today :). But unfortunately after opening the
> chrome.sln file when I build the project, I got three build errors.
> All the build errors reported the same problem "The binary is not a
> valid windows image"
>
> Build O/P snippet:
> 4>-- Build started: Project: chrome_dll, Configuration: Debug
> Win32 --
> 4>Embedding manifest...
> 4>mt.exe : general error c101008d: Failed to write the updated
> manifest to the resource of file ".\Debug\chrome.dll". The binary is
> not a valid Windows image.

It turns out this failure is usually caused by anti-virus software
activities. Disable it from scanning the source dir and try again.

Huan

> 4>Build log was saved at "file://C:\Bala\chromium\chromium.r27598\home
> \chrome-svn\tarball\chromium\src\chrome\Debug\obj\chrome_dll
> \BuildLog.htm"
> 4>chrome_dll - 1 error(s), 0 warning(s)
> 5>-- Build started: Project: generate_profile, Configuration:
> Debug Win32 --
> 5>Embedding manifest...
> 5>mt.exe : general error c101008d: Failed to write the updated
> manifest to the resource of file ".\Debug\generate_profile.exe". The
> binary is not a valid Windows image.
> 5>Build log was saved at "file://C:\Bala\chromium\chromium.r27598\home
> \chrome-svn\tarball\chromium\src\chrome\Debug\obj\generate_profile
> \BuildLog.htm"
> 5>generate_profile - 1 error(s), 0 warning(s)
> .
> .
> 7>-- Build started: Project: browser_tests_dll, Configuration:
> Debug Win32 --
> 7>Embedding manifest...
> 7>mt.exe : general error c101008d: Failed to write the updated
> manifest to the resource of file ".\Debug\browser_tests.dll". The
> binary is not a valid Windows image.
> 7>Build log was saved at "file://C:\Bala\chromium\chromium.r27598\home
> \chrome-svn\tarball\chromium\src\chrome\Debug\obj\browser_tests_dll
> \BuildLog.htm"
> 7>browser_tests_dll - 1 error(s), 0 warning(s)
>
> After googling I found this link "http://www.wintellect.com/CS/blogs/
> jrobbins/archive/2009/01/24/the-case-of-the-corrupt-pe-
> binaries.aspx".
>
> The workaround as stated in the blog is to change linker option from /
> DYNAMICBASE:NO to /DYNAMICBASE. But the projects which reported errors
> (chrome_dll, generate_profile and browser_test_dll) all have their
> linker option set to /DYNAMICBASE only.
>
> So, please let me know how to resolve this build error?
>
> Waiting avidly for your reply :)
>
> Thanks,
> Bala.
>
> >
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Missing symbols for Chrome 2.0.172.43

[Huan Ren]

On Sat, Sep 12, 2009 at 5:49 PM, yuhong  wrote:
>
> Yes, notice the symbol path that begins with 
> http://build.chromium.org/buildbot/symsrv/.
>
> On Sep 12, 4:00 pm, Amit Joshi  wrote:
>> Are you sure you have added chromium debug symbol server to the symbol path?
>> At least from the following output, I don't see an attempt to connect
>> to it.More
>> information 
>> at:http://dev.chromium.org/developers/how-tos/debugging#TOC-Debugging-wi...
>> 
>>
>>
>>
>> On Sat, Sep 12, 2009 at 1:56 PM, yuhong  wrote:
>>
>> > SYMCHK output:
>> > DBGHELP: c:\windows\symbols\chrome_dll.pdb - file not found
>> > DBGHELP: c:\windows\symbols\dll\chrome_dll.pdb - file not found
>> > DBGHELP: c:\windows\symbols\symbols\dll\chrome_dll.pdb - file not
>> > found
>> > SYMSRV:  c:\downloads\symbols\chrome_dll.pdb
>> > \B9CAB1AA626A4C32BD2E681FDD12495D2\chrome_dll.pdb not found
>> > SYMSRV:
>> >http://msdl.microsoft.com/download/symbols/chrome_dll.pdb/B9CAB1AA626...
>> > not found
>> > SYMSRV:  c:\downloads\symbols\chrome_dll.pdb
>> > \B9CAB1AA626A4C32BD2E681FDD12495D2\chrome_dll.pdb not found
>> > SYMSRV:
>> >http://build.chromium.org/buildbot/symsrv/chrome_dll.pdb/B9CAB1AA626A...
>> > not found
>> > SYMSRV:  c:\downloads\symbols\chrome_dll.pdb
>> > \B9CAB1AA626A4C32BD2E681FDD12495D2\chrome_dll.pdb not found
>> > SYMSRV:
>> >http://symbols.mozilla.org/firefox/chrome_dll.pdb/B9CAB1AA626A4C32BD2...
>> > not found
>> > DBGHELP: C:\Users\bob\AppData\Local\Google\Chrome\Application
>> > \2.0.172.43\chrome_dll.pdb - file not found
>> > DBGHELP: C:\b\slave\chrome-official-2\build\src\chrome\Release
>> > \chrome_dll.pdb - file not found
>> > *** ERROR: Symbol file could not be found.  Defaulted to export
>> > symbols for C:\Users\bob\AppData\Local\Google\Chrome\Application
>> > \2.0.172.43\chrome.dll -
>> > DBGHELP: chrome_5c34 - export symbols
>> > DBGHELP: c:\windows\symbols\chrome_exe.pdb - file not found
>> > DBGHELP: c:\windows\symbols\exe\chrome_exe.pdb - file not found
>> > DBGHELP: c:\windows\symbols\symbols\exe\chrome_exe.pdb - file not
>> > found
>> > SYMSRV:  c:\downloads\symbols\chrome_exe.pdb
>> > \BA04199F772D437C8BE4076986127B4E2\chrome_exe.pdb not found
>> > SYMSRV:
>> >http://msdl.microsoft.com/download/symbols/chrome_exe.pdb/BA04199F772...
>> > not found
>> > SYMSRV:  c:\downloads\symbols\chrome_exe.pdb
>> > \BA04199F772D437C8BE4076986127B4E2\chrome_exe.pdb not found
>> > SYMSRV:
>> >http://build.chromium.org/buildbot/symsrv/chrome_exe.pdb/BA04199F772D...
>> > not found
>> > SYMSRV:  c:\downloads\symbols\chrome_exe.pdb
>> > \BA04199F772D437C8BE4076986127B4E2\chrome_exe.pdb not found
>> > SYMSRV:
>> >http://symbols.mozilla.org/firefox/chrome_exe.pdb/BA04199F772D437C8BE...
>> > not found
>> > DBGHELP: C:\Users\bob\AppData\Local\Google\Chrome\Application
>> > \chrome_exe.pdb - file not found
>> > DBGHELP: C:\b\slave\chrome-official-2\build\src\chrome\Release
>> > \chrome_exe.pdb - file not found
>> > *** ERROR: Symbol file could not be found.  Defaulted to export
>> > symbols for C:\Users\bob\AppData\Local\Google\Chrome\Application
>> > \chrome.exe -
>> > DBGHELP: chrome - export symbols
> >
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: new round of flakiness, IsStringASCII this time

[Huan Ren]

This script is enabled for all bots now.

Huan

On Mon, Aug 17, 2009 at 1:39 PM, Huan Ren  wrote:
> UI test can not walk the stack as for now. There is a script to print
> stack traces of all crash dumps after the test is finished. This one
> from the latest chromium cycle:
>
> http://build.chromium.org/buildbot/waterfall/builders/Chromium%20XP/builds/6674/steps/Process%20Dumps/logs/stdio
>
>
>
> On Mon, Aug 17, 2009 at 10:38 AM, Darin Fisher wrote:
>> It would be great if the UI test framework printed out a stack trace when a
>> [D]CHECK fails.
>> -Darin
>>
>>
>> On Mon, Aug 17, 2009 at 9:35 AM, Paweł Hajdan Jr. 
>> wrote:
>>>
>>> I'm seeing a lot of these in the logs (most important message is bold):
>>>
>>>
>>> C:\b\slave\chromium-dbg-builder\build\src\chrome\browser\sessions\session_restore_uitest.cc(385):
>>> error: Value of: last_tab->Reload()
>>>   Actual: false
>>> Expected: true
>>> C:\b\slave\chromium-dbg-builder\build\src\chrome\test\ui\ui_test.cc(485):
>>> error: Value of: automation()->GetBrowserWindowCount(&window_count)
>>>   Actual: false
>>> Expected: true
>>> C:\b\slave\chromium-dbg-builder\build\src\chrome\test\ui\ui_test.cc(148):
>>> error: Value of: assertions.size()
>>>   Actual: 1
>>> Expected: expected_errors_
>>> Which is: 0
>>> The following error(s) occurred in the application during this test:
>>>
>>> [FATAL:string_util.cc(515)] Check failed: IsStringASCII(utf16).
>>> C:\b\slave\chromium-dbg-builder\build\src\chrome\test\ui\ui_test.cc(162):
>>> error: Value of: actual_crashes
>>>   Actual: 1
>>> Expected: expected_crashes_
>>> Which is: 0
>>> Encountered an unexpected crash in the program during this test.
>>>
>>> Do you know how this can happen? (intermittently!) Do you have some idea
>>> how to debug it?
>>>
>>
>>
>> >>
>>
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Just got a backtrace from flaky RenderViewTest

[Huan Ren]

I have seen it on chromebot recently as well. One instance:

http://chromebot/buildsummary?id=buildbot_25671_ext

But the stack is not trustworthy given the large number of function offset.

Huan


On Wed, Sep 9, 2009 at 11:08 AM, Paweł Hajdan Jr.
 wrote:
> So, RenderViewTest crashes very often. Now, thanks to maruel, we have a
> stack trace:
>
> [--] 16 tests from RenderViewTest
> [ RUN  ] RenderViewTest.OnLoadAlternateHTMLText
> Backtrace:
>   GetStackFrames [0x00FF6085+306229]
>   _sbrk [0x00A7E5EE+70254]
>   _sbrk [0x00A7E63E+70334]
>   _sbrk [0x00A7E763+70627]
>   _sbrk [0x00AB052E+274862]
>   _sbrk [0x00A7EA20+71328]
>   _sbrk [0x00A7EE50+72400]
>   _sbrk [0x00A7F747+74695]
>   (No symbol) [0x00431868]
>   MallocExtension::SetMemoryReleaseRate [0x00924756+2127654]
>   MallocExtension::SetMemoryReleaseRate [0x00925699+2131561]
>   MallocExtension::SetMemoryReleaseRate [0x0091FCB5+2108549]
>   MallocExtension::SetMemoryReleaseRate [0x00925E93+2133603]
>   MallocExtension::SetMemoryReleaseRate [0x00925FFC+2133964]
>   MallocExtension::SetMemoryReleaseRate [0x008CF119+1777897]
>   MallocExtension::SetMemoryReleaseRate [0x008CF34E+1778462]
>   (No symbol) [0x005D66E9]
>   RegisterWaitForInputIdle [0x7C817067+73]
>
> Does anyone have some ideas why is it failing?
> >
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Running UI test in parallel (experimental)

[Huan Ren]

On Thu, Aug 6, 2009 at 1:06 PM, John Abd-El-Malek  wrote:

> This is very cool, but I ran into a few problems when I tried to run it:
> a:\chrome2\src\chrome>tools\test\smoketests.py --tests=ui
>
> >> You must have your local path of trunk/src/tools/python added to your
> PYTHONPATH.<<
>
> Traceback (most recent call last):
>   File "A:\chrome2\src\chrome\tools\test\smoketests.py", line 32, in
> 
> import google.httpd_utils
> ImportError: No module named google.httpd_utils
>
>
> hmmm, never needed PYTHONPATH set before.  Can't this script set it itself?
>  Setting it manually will fail when I move depot locations etc..  But
> anyways, I set it and then
>
> a:\chrome2\src\chrome>set PYTHONPATH=a:\chrome\src\tools\python
>
> a:\chrome2\src\chrome>tools\test\smoketests.py --tests=ui
> Traceback (most recent call last):
>   File "A:\chrome2\src\chrome\tools\test\smoketests.py", line 274, in
> 
> result = main(options, args)
>   File "A:\chrome2\src\chrome\tools\test\smoketests.py", line 155, in main
> sys.path.insert(0, _BuildbotScriptPath('slave'))
>   File "A:\chrome2\src\chrome\tools\test\smoketests.py", line 84, in
> _BuildbotScriptPath
> 'scripts', sub_dir)
>   File "a:\chrome\src\tools\python\google\path_utils.py", line 72, in
> FindUpward
> parent = FindUpwardParent(start_dir, *desired_list)
>   File "a:\chrome\src\tools\python\google\path_utils.py", line 55, in
> FindUpwardParent
> (desired_path, start_dir))
> google.path_utils.PathNotFound: Unable to find tools\buildbot\scripts\slave
> above A:\chrome2\src\chrome\tools\test
>
>
>
> tools\buildbot isn't in the public tree I think, since I don't find it
> here: http://src.chromium.org/viewvc/chrome/trunk/src/chrome/tools/.  So
> external contributors can't use this?  Also, why should this script depend
> on the buildbot scripts?  I don't have them so I can't try this out.
>

It is externally available.
http://src.chromium.org/viewvc/chrome/trunk/tools/buildbot/
<http://src.chromium.org/viewvc/chrome/trunk/tools/buildbot/>

>
> Can't we just have a minimal python script that runs ui_tests in parallel?
>

Pam wrote the original smoketests.py. Pam, is it easy to drop those
dependencies? Otherwise, I will write a minimal python script.

Huan


>
> On Wed, Jul 29, 2009 at 3:28 PM, Huan Ren  wrote:
>
>>
>> I just checked in a change to run ui_tests in parallel based on
>> sharding mechanism provided by GTest. Each ui_tests instance has its
>> own user data dir, and the number of ui_tests instances is
>> NUMBER_OF_PROCESSORS. I have updated
>> src/chrome/tools/test/smoketests.py so you can run it through command
>> line:
>>
>> python.exe smoketests.py --tests=ui [--verbose]
>>
>> Running ui_tests.exe directly is still the old behavior of
>> sequentially running. On my 4 core machine, the running time has been
>> reduced by half, from 832 secs to 443 secs. But I need to make sure
>> all tests can run reliably in this parallel fashion. So if you try it
>> out, I will be interested to know how fast the performance is improved
>> and what additional tests are failing.
>>
>> Huan
>>
>> P.S. this change is for Windows platform as I think Linux/Mac is
>> already using GTest sharding.
>>
>> >>
>>
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: new round of flakiness, IsStringASCII this time

[Huan Ren]

UI test can not walk the stack as for now. There is a script to print
stack traces of all crash dumps after the test is finished. This one
from the latest chromium cycle:

http://build.chromium.org/buildbot/waterfall/builders/Chromium%20XP/builds/6674/steps/Process%20Dumps/logs/stdio



On Mon, Aug 17, 2009 at 10:38 AM, Darin Fisher wrote:
> It would be great if the UI test framework printed out a stack trace when a
> [D]CHECK fails.
> -Darin
>
>
> On Mon, Aug 17, 2009 at 9:35 AM, Paweł Hajdan Jr. 
> wrote:
>>
>> I'm seeing a lot of these in the logs (most important message is bold):
>>
>>
>> C:\b\slave\chromium-dbg-builder\build\src\chrome\browser\sessions\session_restore_uitest.cc(385):
>> error: Value of: last_tab->Reload()
>>   Actual: false
>> Expected: true
>> C:\b\slave\chromium-dbg-builder\build\src\chrome\test\ui\ui_test.cc(485):
>> error: Value of: automation()->GetBrowserWindowCount(&window_count)
>>   Actual: false
>> Expected: true
>> C:\b\slave\chromium-dbg-builder\build\src\chrome\test\ui\ui_test.cc(148):
>> error: Value of: assertions.size()
>>   Actual: 1
>> Expected: expected_errors_
>> Which is: 0
>> The following error(s) occurred in the application during this test:
>>
>> [FATAL:string_util.cc(515)] Check failed: IsStringASCII(utf16).
>> C:\b\slave\chromium-dbg-builder\build\src\chrome\test\ui\ui_test.cc(162):
>> error: Value of: actual_crashes
>>   Actual: 1
>> Expected: expected_crashes_
>> Which is: 0
>> Encountered an unexpected crash in the program during this test.
>>
>> Do you know how this can happen? (intermittently!) Do you have some idea
>> how to debug it?
>>
>
>
> >
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Temporarily disable tcmalloc in build

[Huan Ren]

- The crash rates on chromebot are comparable for builds with and
without tcmalloc. The crash rate for tcmalloc build is higher but it
seems within error margin.
- Wtih tcmalloc disabled, there are some promising data from Purify
test run and ChromeBot run with full page heap enabled. These are
still under further investigation.
- The performance results are same as last time we compared. With
tcmalloc, the most perf gain is from page-cycler-moz: 15% improvement
(with 25% more memory usage). The perfor gain on page-cycler-more-js
is 4%, as we see tcmalloc has no perf impact on V8 benchmark. The
performance gain on DOM is significant: from score 325 to 414.

r22251 re-enables tcmalloc on trunk. We can merge 22080 to dev branch if needed.

Huan

On Fri, Jul 31, 2009 at 6:51 PM, cpu wrote:
>
> What are the results of this experiment?
>
> On Jul 30, 12:15 pm, Huan Ren  wrote:
>> I just submitted a change (22080) that disables tcmalloc used on
>> Windows platform. The plan is keeping it in trunk for 24 hours and
>> then reverting it. The intentions are
>>    - Having another round of performance comparison between build with
>> and w/o tcmalloc.
>>    - Having a full run of UI test under purify with tcmalloc disabled.
>>    - Getting a verified CL in case we'd like to build an alternative
>> dev build w/o tcmalloc for A/B test.
>>
>> As a head up, the performance, stability, and purify test results
>> could be different during the period.
>>
>> Huan
> >
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Temporarily disable tcmalloc in build

[Huan Ren]

I just submitted a change (22080) that disables tcmalloc used on
Windows platform. The plan is keeping it in trunk for 24 hours and
then reverting it. The intentions are
   - Having another round of performance comparison between build with
and w/o tcmalloc.
   - Having a full run of UI test under purify with tcmalloc disabled.
   - Getting a verified CL in case we'd like to build an alternative
dev build w/o tcmalloc for A/B test.

As a head up, the performance, stability, and purify test results
could be different during the period.

Huan

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Running UI test in parallel (experimental)

[Huan Ren]

Each gets it own profile thus different browser process.

Huan

On Wed, Jul 29, 2009 at 6:01 PM, Erik Kay wrote:
> Does each shard get its own profile?  Or do they share?  Does this mean that
> when you shard they're sharing the same browser process?
> Erik
>
> On Wed, Jul 29, 2009 at 3:28 PM, Huan Ren  wrote:
>>
>> I just checked in a change to run ui_tests in parallel based on
>> sharding mechanism provided by GTest. Each ui_tests instance has its
>> own user data dir, and the number of ui_tests instances is
>> NUMBER_OF_PROCESSORS. I have updated
>> src/chrome/tools/test/smoketests.py so you can run it through command
>> line:
>>
>> python.exe smoketests.py --tests=ui [--verbose]
>>
>> Running ui_tests.exe directly is still the old behavior of
>> sequentially running. On my 4 core machine, the running time has been
>> reduced by half, from 832 secs to 443 secs. But I need to make sure
>> all tests can run reliably in this parallel fashion. So if you try it
>> out, I will be interested to know how fast the performance is improved
>> and what additional tests are failing.
>>
>> Huan
>>
>> P.S. this change is for Windows platform as I think Linux/Mac is
>> already using GTest sharding.
>>
>> >>
>
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Running UI test in parallel (experimental)

[Huan Ren]

I just checked in a change to run ui_tests in parallel based on
sharding mechanism provided by GTest. Each ui_tests instance has its
own user data dir, and the number of ui_tests instances is
NUMBER_OF_PROCESSORS. I have updated
src/chrome/tools/test/smoketests.py so you can run it through command
line:

python.exe smoketests.py --tests=ui [--verbose]

Running ui_tests.exe directly is still the old behavior of
sequentially running. On my 4 core machine, the running time has been
reduced by half, from 832 secs to 443 secs. But I need to make sure
all tests can run reliably in this parallel fashion. So if you try it
out, I will be interested to know how fast the performance is improved
and what additional tests are failing.

Huan

P.S. this change is for Windows platform as I think Linux/Mac is
already using GTest sharding.

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] A few notes on Reliability FIXIT

[Huan Ren]

We have triaged most of FIXIT bugs. A few quick notes for the week:
1. Fix P1 crash bugs first. FYI, Anthony's script cuts off crashes in
following way:
  - Top 25 P1
- Top 50 P2 (exception to P1 for regressions)
- Else P3

2. Consider off loading your FIXIT bugs early. Please take a look at
reliability FIXIT bugs assigned to you and punt the ones you don't have time
to work on. Please CC huanr or laforge when doing so.

3. There are still Coverity
review bugs unassigned. Those are good candidates for people looking for
FIXITs to work on.

Regards,

Huan and Anthony

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Reliability Fixit Week

[Huan Ren]

Next week June 29 - July 2 is the Fixit week focusing on reliability.

All crash bugs are Fixit work items.

http://code.google.com/p/chromium/issues/list?can=2&q=label:Crash&sort=owner

In addition, there are purify, valgrind, and coverity issues labeled for
this round of Fixit.

http://code.google.com/p/chromium/issues/list?can=2&q=coverity%20OR%20valgrind%20OR%20purify%20label:fixit&sort=owner&;

We welcome *all* chromium contributors to participate. For committers
(including provisional committers), please plan to set aside about 2 days
for the next week to fix the above bugs.

The process of this round of Fixit is same as what we did before and I list
here for people recently joined the chromium development community.

1. If you have not done so, please look through the list and take the bugs
you'd like to work on.
2. If you are overloaded with Fixit bugs, find new owners for some bugs or
simply unassign yourself.
3. During the week, we will assign unclaimed bugs. For this reason, you may
want to take the bug you'd like to work on yourself instead of being
assigned with random bugs.
4. As our development culture, be prepared to work on bugs across all
chromium code base. If you are assigned with bugs in an unfamiliar area,
consider the following options:
- Identify the best person to fix the bug and assign the bug to him/her
if the person has cycles.
- Do as much as you can to help out, for example, analyzing crash dumps,
querying crash server to find common pattern across crash dumps, narrowing
down the revision range that causes the bug, searching through reports from
user group and bug database to get additional information, finding repro
steps, constructing reduced test cases, etc.
- Understand the code and fix it.

Huan and Anthony.

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Purify/Valgrind FIXIT

[Huan Ren]

For people looking for FIXIT items next week, these are lists of
Purify/Valgrind
FIXIT bugs without owner assigned.

http://code.google.com/p/chromium/issues/list?can=2&q=purify+label:fixit+-has:owner&sort=pri&colspec=ID+Stars+Pri+Area+Type+Status+Summary+Modified+Owner+Mstone&x=mstone&y=area&cells=tiles

http://code.google.com/p/chromium/issues/list?can=2&q=valgrind+label:fixit+-has:owner&sort=pri&colspec=ID+Stars+Pri+Area+Type+Status+Summary+Modified+Owner+Mstone&x=mstone&y=area&cells=tiles

Huan

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: What's the TabRestoreUITest bug 1215881?

[Huan Ren]

Are you looking at the latest version of file  tab_restore_uitest.cc? That
comment was long gone.

Huan

On Wed, Jun 17, 2009 at 9:37 AM, Dan Kegel  wrote:

>
> On Tue, Jun 16, 2009 at 12:04 PM, pi wrote:
> > tab_restore_uitest.cc:
> >
> > TEST_F(TabRestoreUITest, RestoreToDifferentWindow) {
> >  // This test is disabled on win2k. See bug 1215881.
> >
> > I cant find anything about  bug 1215881 on
> http://code.google.com/p/chromium/issues/list,
> > where to find it?
>
> That's an internal bug tracker, mostly for really old bugs.
> Since we don't support win2k, that comment can probably be deleted.
>
> >
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: some trival details

[Huan Ren]

2009/6/15 

>  I began to read chromium's source codes, and got caught in these trival
> questions:
>
> 1. what's gfx? I always see that.
>
> 2. what's the usage of .gyp files?
>
> 3.Is gmock used to help automation test? What else does chrome use for
> automation test? I'm curious about that.
>
> 4.gcapi confused me, what is it?
>
This is also part of installer. It provides APIs for third party to
distribute Chrome.


> 5.mini_installer.exe and setup.exe? I can't execute them. What're they used
> for?
>
> 6.what is courgette? and used for?
> >
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Progress indicator for mini_installer on Windows

[Huan Ren]

We have UI constraint on mini_installer. But maybe we can do something
for a chromium build. Feel free to file a request.

Huan

On Thu, Apr 23, 2009 at 7:36 PM, Robert Dailey  wrote:
>
> Hi,
>
> I notice that when I install Chromium via the mini_installer (on the
> continuous builds website for Windows), I get no progress indication.
> Is this by design? I can never know for sure when the installer has
> completed, I'm always having to guess. Would it be unreasonable to
> create a feature request for a small dialog with a progress bar? At
> least this way I'll be able to know for sure when the installer has
> completed.
>
> Thanks for your time.
> >
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: UI test improvement

[Huan Ren]

Right it is generated but then next UI test case will clean up the
directory to detect new crashes. It is easy to change the behavior to
something like backing up crash dumps at the end of each test case.
What we need is some place in try server or buildbot to save the crash
dumps and adding a link in the test output. Maybe there are some other
better options for try server since it does not build with symbols.
What about buildbot?

Huan

On Fri, Apr 17, 2009 at 12:00 PM, Marc-Antoine Ruel  wrote:
> The memory dumps _are_ generated. They are just not accessible at the moment.
>
> For example on XP, they are on %HOMEPATH%\Local Settings\Application
> Data\Chromium\User Data\Crash Reports.
>
> If there is interest, we could investigate to have the crash dumps
> available for public consumption but in general, if the bug is not
> reproducible locally, it's easier to just debug on the slave.
> Debugging a crash dump requires the symbols, which is a relatively
> large download, slowing down the efficiency of this method. So I'm not
> sure it's a good idea at all.
>
> The try server (at least on windows) don't build with symbols to be
> faster so the memory dumps wouldn't be of much use.
>
> M-A
>
> On Thu, Apr 16, 2009 at 6:22 PM, Huan Ren  wrote:
>>
>> I am trying to improve the UI automation framework with the goal of
>> reducing disabled UI test from 42 to 10 and running time (xp release)
>> from 400s to 200s. Following is a list of issues I found so far (I am
>> still studying them):
>>
>> 1. Many UI commands are executed asynchronously, and test client
>> sleeps and polls (not always polls actually) the result. This process
>> is slow and error prone. And we need different timeout for release,
>> debug, and purify.
>> 2. The mechanism to send command to certain window/tab is not
>> reliable. For example, I found GetLastActiveBrowserWindow and
>> ActivateTab don't work as intended.
>> 3. UI_tests.exe, crash_service.exe, and chrome.exe share the same
>> mutex when writing log file. This makes things slow. In addition, when
>> a process crashes/shutdowns while writing log files, other process
>> will get WAIT_ABANDONED when waiting for mutex and this leads to an
>> infinite loop.
>> 4. If Chrome crashes when running the test on buildbot, we can not
>> investigate the cause. This might be exact the situation we'd like the
>> test to catch but we have to disable the test most of the time.
>>
>> The biggest pain I have is that when UI tests fail/crash on buildbot
>> or try server, we have little information to investigate. I would
>> image lots of people faced similar situation in the past and ended up
>> with disabling the test. I'd think we need to address this issue first
>> before making good improvement on UI tests. If we have good debugging
>> infrastructure in place, fixing UI flakiness is much easier. As a
>> minimal, we need to save crash dumps. Any thoughts on this?
>>
>> Huan
>>
>> >>
>>
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] UI test improvement

[Huan Ren]

I am trying to improve the UI automation framework with the goal of
reducing disabled UI test from 42 to 10 and running time (xp release)
from 400s to 200s. Following is a list of issues I found so far (I am
still studying them):

1. Many UI commands are executed asynchronously, and test client
sleeps and polls (not always polls actually) the result. This process
is slow and error prone. And we need different timeout for release,
debug, and purify.
2. The mechanism to send command to certain window/tab is not
reliable. For example, I found GetLastActiveBrowserWindow and
ActivateTab don't work as intended.
3. UI_tests.exe, crash_service.exe, and chrome.exe share the same
mutex when writing log file. This makes things slow. In addition, when
a process crashes/shutdowns while writing log files, other process
will get WAIT_ABANDONED when waiting for mutex and this leads to an
infinite loop.
4. If Chrome crashes when running the test on buildbot, we can not
investigate the cause. This might be exact the situation we'd like the
test to catch but we have to disable the test most of the time.

The biggest pain I have is that when UI tests fail/crash on buildbot
or try server, we have little information to investigate. I would
image lots of people faced similar situation in the past and ended up
with disabling the test. I'd think we need to address this issue first
before making good improvement on UI tests. If we have good debugging
infrastructure in place, fixing UI flakiness is much easier. As a
minimal, we need to save crash dumps. Any thoughts on this?

Huan

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: purecall exceptions and the manbearpig

[Huan Ren]

On Fri, Apr 3, 2009 at 12:15 PM, Huan Ren  wrote:
> Based on what I saw in the bug, it looks like an exception happening
> during CALL instruction may lead to PureCall().
>
> For example, an object obj has been freed and later on someone calls
> obj->func(). Then the assembly code looks like this:
>
> // ecx: pointer to obj which is in memory
> // [ecx]: supposed to be pointer to vtable, it has invalid value since
> obj is freed
> // edx: now has pointer to vtable, which is invalid
> mov edx,dword ptr [ecx]
>
> // deref the vtable and make the call
> call dword ptr [edx+4]
>
> When a (hardware) exception happens during the call instruction, the
> control will be eventually transfered to the routine handling this
> type of exception which I *think* is PureCall().
>
> Huan
>
> On Fri, Apr 3, 2009 at 11:26 AM, Ricardo Vargas  wrote:
>> I certainly don't want to imply that it is the case with this particular
>> bug, but I have seen crashes when the cause of the problem is using an
>> object that was previously deleted (and only end up with this exception when
>> all the planets are properly aligned). I guess that it depends on the actual
>> class hierarchy of the objects in question, but I'd think that "simple"
>> examples end up on a lot of crashes right after the cl that exposes the
>> problem.
>>
>> On Fri, Apr 3, 2009 at 12:52 AM, Dean McNamee  wrote:
>>>
>>> You could, however, corrupt the vtable pointer (not the vtable).  Say
>>> somehow 32 was added to it, now the table is misaligned, and you might
>>> get a purecall, etc.  Not sure that's likely at all though.
>>>
>>> Since  the vtable pointer is the first field, it seems ripe for
>>> problems w/ use after free, etc.  I kinda doubt that's what's
>>> happening here though.  Anyone who is working on one of these can bug
>>> me and I'll look at the crash dump.
>>>
>>> On Fri, Apr 3, 2009 at 7:24 AM, Tommi  wrote:
>>> > On Thu, Apr 2, 2009 at 7:09 PM, cpu  wrote:
>>> >>
>>> >>
>>> >>
>>> >> On Apr 2, 3:53 pm, Nicolas Sylvain  wrote:
>>> >> > Another simple(r) example
>>> >> > :http://msdn.microsoft.com/en-us/library/t296ys27(VS.80).aspx
>>> >> >
>>> >> > <http://msdn.microsoft.com/en-us/library/t296ys27(VS.80).aspx>But, as
>>> >> > discussed in bug 8544, we've see many purecall crashes that happens
>>> >> > and
>>> >> > we
>>> >> > don't
>>> >> > think it's related to virtual functions. The only thing I can think
>>> >> > of
>>> >> > is
>>> >> > that the vtable is corrupted. (overwritten or freed)
>>> >> >
>>> >> > Does it not make sense?
>>> >>
>>> >> I don't think you can overwrite a vtables because they should be in
>>> >> the code section of the executable (the pages marked as read-execute),
>>> >> they are known at compile time and it would not make sense to
>>> >> construct them on the fly.
>>> >>
>>> >> But if you know of a case then that would be very interesting.
>>> >
>>> >
>>> > yes they should be protected with read/execute and besides, you'd have
>>> > to
>>> > overwrite entries in the vtable with a pointer to __purecall for that to
>>> > happen
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> >
>>> >> > Nicolas
>>> >> >
>>> >> >
>>> >> >
>>> >> > On Thu, Apr 2, 2009 at 1:54 PM, cpu  wrote:
>>> >> >
>>> >> > > After reading some speculation in bugs such as
>>> >> > >http://code.google.com/p/chromium/issues/detail?id=8544I felt
>>> >> > > compelled to dispel some myths and misunderstandings about the
>>> >> > > origin
>>> >> > > and meaning of the mythical _purecall_ exception. My hope is that
>>> >> > > then
>>> >> > > you can spot the problems in our source code and fix them. Sorry
>>> >> > > for
>>> >> > > the long post.
>>> >> >
>>> >> > > So first of all, what do you see when you get this error? if you
>>> &