(2013/02/01 11:47), ISHIKAWA, Chiaki wrote:
(2013/02/01 11:34), Joshua Cranmer wrote:
On 1/31/2013 8:26 PM, ISHIKAWA, Chiaki wrote:
(2013/02/01 1:48), Joshua Cranmer wrote:
On 1/31/2013 9:41 AM, ISHIKAWA, Chiaki wrote:
Sorry, it may not have been ldap code (my memory is now hazy).
But please take a look at this code. I am using comm-central code
for development/debugging TB.
https://mxr.mozilla.org/comm-central/source/mozilla/security/nss/lib/base/hash.c#53
This is NSS code, which is a separate codebase included in mozilla-central and
updated periodically.
Judging from the context, this is someone using the low 32 bits as a hash key;
since it's a hashtable, I'm going to
assume that it's expected to have some amount of collisions, so the conversion
of a possibly-64-bit pointer to a 32-bit
value is "safe" here.
Yes, I was also expecting to see the collision resolution.
But, if my reading was correct, in one place, I thought the
equality of the hash value was deemed instantly to be the equality of the
object.
(That is why I thought something was fishy, and I assume that the coder
thought it would work only under in 32bit address space. Thus the check for
32bit
address space.)
I should have raised the flag then and there. But I went on other things, and
later when I learned the availability of 64bit version of TB, I was surprised.
It could be that my reading of "equality check" may have been incorrect.
Anyway, I assume that the code would bomb out by PR_ASSERT() if incorrect usage
of this funciton in 64 bit address space is encountered at run time, and
assuming that it has not happened to anybody (?), then probably
this part of the code base is not used in production TB.
That assert says that sizeof(PLHashNumber) == sizeof(PRUint32). Which is always
true thanks to
<https://mxr.mozilla.org/comm-central/source/mozilla/nsprpub/lib/ds/plhash.h#18>.
Hmm. I will re-read this and make sure if I understand this correctly.
TIA
(This is lengthy. Sorry about it.)
Hi,
I am still looking at the 64 bit cleanness issue of TB.
Setting the hash table issue aside (I can't still find
where the smoking gun is: it should be searchable using
the compiler warnings which initially led me to the place),
I am testing the local build of
TB under 64 bit linux using "make mozmill" test suite.
The result is not very promizing.
1. make mozmill seems to get hung occasionally. This never happened
under 32-bit linux. It could be related to available memory issue,
but I am not so sure. It seems that 64 bit may have uncovered
some hidden issues in TB itself and make mozmill test harness also.
2. In the session log under 64-bit linux, there are error messages
(ASSERTIONS and ERROR messages) that I have NOT noticed in the
session log of "make mozmill" under 32-bit linux at all (for the last
few months).
These ASSERTIONS and ERROR messages only seen in 64-bit build
are very troubling, especially the failure of iconv routine in particular.
(Not sure if this is particularly observed under Debign GNU/Linux 64-bit and not
under other linux 64-bit OS distribution.)
In one session log of "make mozmill" of 64 bit TB, I counted 15 repetitions of
###!!! ASSERTION: iconv failed: 'Error', file
/home/ishikawa/TB-NEW/TB-3HG/new-src/mozilla/xpcom/io/nsNativeCharsetUtils.cpp, line 541
that I didn't see in 32 bit session log at all(!).
Given that mail is about exchanging text, failure of iconv() sounds ominious.
So I inserted dump probes into the problematic source file, and found
that iconv seems to return an error value. I also called abort()
when the error is observed so that I can trace the stack using gdb.
I have found that
"make mozmill" test suite allows the connection to the aborted TB image for 300
seconds before moving to the next test target. During that 300 seconds,
I can use gdb to connect to the terminated process image and look at the stack,
etc..
[] Error value which seems to be returned by iconv(). (see the log in the
following)
./x86_64-linux-gnu/bits/errno.h:# define EILSEQ 84 /* Illegal byte
sequence. */
It certainly looks like the value returned by failing iconv().
[] Modified xp_iconv() :
(Notice that in the error check, *outputLeft was originally
checked even if outputLeft is NULL. So I inserted
extra check in there.)
static inline size_t
xp_iconv(iconv_t converter,
const char **input,
size_t *inputLeft,
char **output,
size_t *outputLeft)
{
size_t res, outputAvail = outputLeft ? *outputLeft : 0;
res = iconv(converter, ICONV_INPUT(input), inputLeft, output, outputLeft);
if (res == (size_t) -1) {
// on some platforms (e.g., linux) iconv will fail with
// E2BIG if it cannot convert _all_ of its input. it'll
// still adjust all of the in/out params correctly, so we
// can ignore this error. the assumption is that we will
// be called again to complete the conversion.
#if DEBUG
fprintf(stderr,
"iconv returned -1, errno=%d, outputLeft =%p, "
"*outputLeft=%d, outputAvail=%d\n",
errno, outputLeft, outputLeft ? *outputLeft : 0,
outputAvail);
#endif
/* do not access *outputLeft if outputLeft is NULL */
if ((errno == E2BIG) && (outputLeft == NULL || (*outputLeft <
outputAvail)))
res = 0;
#if DEBUG
else
abort();
#endif
}
return res;
}
[] Excerpt from the session log under 64-bit linux.
(Now I noticed that this particular instance of the error occurs near
the end of shutdown process, so something is not probably going well
due to the shutdown sequence problem discussed often in mozilla
newsgroup these days. But, still, I didn't see the failure of
iconv() under 32-bit linux at all so there may be 64-bit specific issue here.)
--DOCSHELL 0x408cd30 == 4 [id = 7]
--DOCSHELL 0x40a2720 == 3 [id = 8]
--DOCSHELL 0x2ddab10 == 2 [id = 2]
--DOCSHELL 0x40c77e0 == 1 [id = 10]
WARNING: An event was posted to a thread that will never run it (rejected): file
/home/ishikawa/TB-NEW/TB-3HG/new-src/mozilla/xpcom/threads/nsThread.cpp, line 378
WARNING: unable to post continuation event: file
/home/ishikawa/TB-NEW/TB-3HG/new-src/mozilla/xpcom/io/nsStreamUtils.cpp, line 445
WARNING: An event was posted to a thread that will never run it (rejected): file
/home/ishikawa/TB-NEW/TB-3HG/new-src/mozilla/xpcom/threads/nsThread.cpp, line 378
WARNING: unable to post continuation event: file
/home/ishikawa/TB-NEW/TB-3HG/new-src/mozilla/xpcom/io/nsStreamUtils.cpp, line 445
iconv returned -1, errno=84, outputLeft =0x7fffe8950068, *outputLeft=3957,
outputAvail=4096
Program ../../.././mozilla/dist/bin/thunderbird (pid = 20083) received signal 6.
Stack:
UNKNOWN [/lib/x86_64-linux-gnu/libpthread.so.0 +0x0000F030]
gsignal+0x00000035 [/lib/x86_64-linux-gnu/libc.so.6 +0x00032475]
abort+0x00000180 [/lib/x86_64-linux-gnu/libc.so.6 +0x000356F0]
UNKNOWN [/home/ishikawa/TB-NEW/TB-3HG/objdir-tb3/mozilla/dist/bin/libxul.so
+0x0268DE58]
UNKNOWN [/home/ishikawa/TB-NEW/TB-3HG/objdir-tb3/mozilla/dist/bin/libxul.so
+0x0268E065]
UNKNOWN [/home/ishikawa/TB-NEW/TB-3HG/objdir-tb3/mozilla/dist/bin/libxul.so
+0x007B69BB]
UNKNOWN [/home/ishikawa/TB-NEW/TB-3HG/objdir-tb3/mozilla/dist/bin/libxul.so
+0x007AB50F]
UNKNOWN [/home/ishikawa/TB-NEW/TB-3HG/objdir-tb3/mozilla/dist/bin/libxul.so
+0x007B62B9]
XRE_main+0x0000013B
[/home/ishikawa/TB-NEW/TB-3HG/objdir-tb3/mozilla/dist/bin/libxul.so +0x007B6645]
UNKNOWN [../../.././mozilla/dist/bin/thunderbird +0x00001F1E]
__libc_start_main+0x000000FD [/lib/x86_64-linux-gnu/libc.so.6 +0x0001EEAD]
UNKNOWN [../../.././mozilla/dist/bin/thunderbird +0x00001949]
Sleeping for 300 seconds.
Type 'gdb ../../.././mozilla/dist/bin/thunderbird 20083' to attach your
debugger to this thread.
Done sleeping...
WARNING: shutting down early because of crash!: file
/home/ishikawa/TB-NEW/TB-3HG/new-src/mozilla/dom/plugins/ipc/PluginModuleChild.cpp, line 701
WARNING: plugin process _exit()ing: file
/home/ishikawa/TB-NEW/TB-3HG/new-src/mozilla/dom/plugins/ipc/PluginModuleChild.cpp, line 666
WARNING | test-lwthemes.js::test_lightweight_themes_install_and_undo | (SKIP)
True
TEST-START |
/home/ishikawa/TB-NEW/TB-3HG/new-src/mail/test/mozmill/content-tabs/test-lwthemes.js
| teardownModule
TEST-PASS | /home/ishikawa/TB-NEW/TB-3HG/new-src/mail/test/mozmill/content-tabs/test-lwthemes.js |
test-lwthemes.js::teardownModule
...
[] gdb session when the error occured..
This time, gdb was not so useful since I compiled the object using
-O2 instead of -O. I will report the result with the "-O" compilation.
$ gdb thunderbird 20083 <=== I use the PID of the TB process as suggested by
"make mozmill" test harness.
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from
/home/ishikawa/TB-NEW/TB-3HG/objdir-tb3/mozilla/dist/bin/thunderbird...done.
Attaching to program:
/home/ishikawa/TB-NEW/TB-3HG/objdir-tb3/mozilla/dist/bin/thunderbird, process
20083
Reading symbols from /lib/x86_64-linux-gnu/libpthread.so.0...Reading symbols from
/usr/lib/debug/lib/x86_64-linux-gnu/libpthread-2.13.so...done.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x2b66c2600700 (LWP 20122)]
[New Thread 0x2b66c2c01700 (LWP 20120)]
[New Thread 0x2b66c2a00700 (LWP 20116)]
[New Thread 0x2b66bcf0b700 (LWP 20114)]
[New Thread 0x2b66c1e3b700 (LWP 20106)]
[New Thread 0x2b66c180f700 (LWP 20105)]
[New Thread 0x2b66c140d700 (LWP 20103)]
[New Thread 0x2b66c120c700 (LWP 20102)]
[New Thread 0x2b66c0c00700 (LWP 20101)]
[New Thread 0x2b66bd300700 (LWP 20097)]
[New Thread 0x2b66b5c00700 (LWP 20093)]
[New Thread 0x2b66b5800700 (LWP 20092)]
[New Thread 0x2b66b52c0700 (LWP 20091)]
[New Thread 0x2b66b50bf700 (LWP 20090)]
[New Thread 0x2b66b4a7d700 (LWP 20089)]
[New Thread 0x2b66b4589700 (LWP 20088)]
[New Thread 0x2b66b3d28700 (LWP 20086)]
[New Thread 0x2b66b3b27700 (LWP 20085)]
done.
Loaded symbols for /lib/x86_64-linux-gnu/libpthread.so.0
Reading symbols from /lib/x86_64-linux-gnu/libdl.so.2...Reading symbols from
/usr/lib/debug/lib/x86_64-linux-gnu/libdl-2.13.so...done.
done.
Loaded symbols for /lib/x86_64-linux-gnu/libdl.so.2
Reading symbols from /lib/x86_64-linux-gnu/libm.so.6...Reading symbols from
/usr/lib/debug/lib/x86_64-linux-gnu/libm-2.13.so...done.
done.
[... so many library symbols loaded...
I wish libxul.so is split because it is SO HUGE,
I also noticed audio and video libraries and I wish that audio and
video support
can be taken out from TB by compile time option to make TB
a sleek mail client, but I digress.]
Loaded symbols for /usr/lib/x86_64-linux-gnu/libFLAC.so.8
Reading symbols from /usr/lib/x86_64-linux-gnu/libvorbisenc.so.2...(no
debugging symbols found)...done.
Loaded symbols for /usr/lib/x86_64-linux-gnu/libvorbisenc.so.2
0x00002b66a12b4bdd in nanosleep () at ../sysdeps/unix/syscall-template.S:82
82 ../sysdeps/unix/syscall-template.S: そのようなファイルやディレクトリはありません.
(gdb) where
#0 0x00002b66a12b4bdd in nanosleep ()
at ../sysdeps/unix/syscall-template.S:82
#1 0x00002b66a12b4a50 in __sleep (seconds=<optimized out>)
at ../sysdeps/unix/sysv/linux/sleep.c:138
#2 0x00002b66a351c13f in ah_crap_handler (signum=6)
at
/home/ishikawa/TB-NEW/TB-3HG/new-src/mozilla/toolkit/xre/nsSigHandlers.cpp:88
#3 0x00002b66a3526487 in nsProfileLock::FatalSignalHandler (signo=6,
info=0x7fffe894fbf0, context=0x7fffe894fac0)
at
/home/ishikawa/TB-NEW/TB-3HG/objdir-tb3/mozilla/toolkit/profile/nsProfileLock.cpp:190
#4 <signal handler called>
#5 0x00002b66a123b475 in *__GI_raise (sig=<optimized out>)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#6 0x00002b66a123e6f0 in *__GI_abort () at abort.c:92
#7 0x00002b66a53ebe58 in xp_iconv (outputLeft=0x7fffe8950068,
output=<optimized out>, inputLeft=0x7fffe8950060, input=<optimized out>,
converter=<optimized out>)
at
/home/ishikawa/TB-NEW/TB-3HG/new-src/mozilla/xpcom/io/nsNativeCharsetUtils.cpp:142
#8 nsNativeCharsetConverter::UnicodeToNative (
this=this@entry=0x7fffe89500b7, input=input@entry=0x7fffe89500c0,
inputLeft=inputLeft@entry=0x7fffe89500b8,
output=output@entry=0x7fffe89500c8,
outputLeft=outputLeft@entry=0x7fffe89500bc)
at
/home/ishikawa/TB-NEW/TB-3HG/new-src/mozilla/xpcom/io/nsNativeCharsetUtils.cpp:556
#9 0x00002b66a53ec065 in NS_CopyUnicodeToNative (input=..., output=...)
at
/home/ishikawa/TB-NEW/TB-3HG/new-src/mozilla/xpcom/io/nsNativeCharsetUtils.cpp:857
#10 0x00002b66a35149bb in WriteConsoleLog ()
at
/home/ishikawa/TB-NEW/TB-3HG/new-src/mozilla/toolkit/xre/nsConsoleWriter.cpp:87
#11 0x00002b66a350950f in ScopedXPCOMStartup::~ScopedXPCOMStartup (
this=0x1d0ce40, __in_chrg=<optimized out>)
at
/home/ishikawa/TB-NEW/TB-3HG/new-src/mozilla/toolkit/xre/nsAppRunner.cpp:1122
#12 0x00002b66a35142b9 in XREMain::XRE_main (this=this@entry=0x7fffe8951450,
argc=argc@entry=6, argv=argv@entry=0x7fffe8952848,
aAppData=aAppData@entry=0x1c638f0)
at
/home/ishikawa/TB-NEW/TB-3HG/new-src/mozilla/toolkit/xre/nsAppRunner.cpp:3915
#13 0x00002b66a3514645 in XRE_main (argc=6, argv=0x7fffe8952848,
aAppData=0x1c638f0, aFlags=<optimized out>)
at
/home/ishikawa/TB-NEW/TB-3HG/new-src/mozilla/toolkit/xre/nsAppRunner.cpp:4093
#14 0x0000000000401f1e in do_main (argv=0x7fffe8952848, argc=6,
exePath=0x7fffe8951720 "/home/ishikawa/TB-NEW/TB-3HG/objdir-tb3/mozilla/dist/bin/") at
/home/ishikawa/TB-NEW/TB-3HG/new-src/mail/app/nsMailApp.cpp:111
#15 main (argc=6, argv=0x7fffe8952848)
at /home/ishikawa/TB-NEW/TB-3HG/new-src/mail/app/nsMailApp.cpp:200
(gdb) up
#1 0x00002b66a12b4a50 in __sleep (seconds=<optimized out>)
at ../sysdeps/unix/sysv/linux/sleep.c:138
138 ../sysdeps/unix/sysv/linux/sleep.c: そのようなファイルやディレクトリはありません.
(gdb) up
#2 0x00002b66a351c13f in ah_crap_handler (signum=6)
at
/home/ishikawa/TB-NEW/TB-3HG/new-src/mozilla/toolkit/xre/nsSigHandlers.cpp:88
88 sleep(_gdb_sleep_duration);
(gdb) up
#3 0x00002b66a3526487 in nsProfileLock::FatalSignalHandler (signo=6,
info=0x7fffe894fbf0, context=0x7fffe894fac0)
at
/home/ishikawa/TB-NEW/TB-3HG/objdir-tb3/mozilla/toolkit/profile/nsProfileLock.cpp:190
190 oldact->sa_handler(signo);
(gdb) up
#4 <signal handler called>
(gdb) up
#5 0x00002b66a123b475 in *__GI_raise (sig=<optimized out>)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
64 ../nptl/sysdeps/unix/sysv/linux/raise.c: そのようなファイルやディレクトリはありません.
(gdb) up
#6 0x00002b66a123e6f0 in *__GI_abort () at abort.c:92
92 abort.c: そのようなファイルやディレクトリはありません.
(gdb) up
#7 0x00002b66a53ebe58 in xp_iconv (outputLeft=0x7fffe8950068,
output=<optimized out>, inputLeft=0x7fffe8950060, input=<optimized out>,
converter=<optimized out>)
at
/home/ishikawa/TB-NEW/TB-3HG/new-src/mozilla/xpcom/io/nsNativeCharsetUtils.cpp:142
142 abort();
(gdb) list
[The dump was produced before I realied the reference to
possibly NULL outputLeft issue.]
137 #endif
138 if ((errno == E2BIG) && (*outputLeft < outputAvail))
139 res = 0;
140 #if DEBUG
141 else
142 abort();
143 #endif
144
145 }
146 return res;
(gdb) print *input
value has been optimized out
(gdb) up
#8 nsNativeCharsetConverter::UnicodeToNative (
this=this@entry=0x7fffe89500b7, input=input@entry=0x7fffe89500c0,
inputLeft=inputLeft@entry=0x7fffe89500b8,
output=output@entry=0x7fffe89500c8,
outputLeft=outputLeft@entry=0x7fffe89500bc)
at
/home/ishikawa/TB-NEW/TB-3HG/new-src/mozilla/xpcom/io/nsNativeCharsetUtils.cpp:556
556 res = xp_iconv(gUnicodeToNative, (const char **) input, &inLeft,
output, &outLeft);
(gdb) print input
$1 = (const PRUnichar **) 0x7fffe89500c0
(gdb) print *input
$2 = (const PRUnichar *) 0x36677c6
(gdb) quit
A debugging session is active.
Inferior 1 [process 20083] will be detached.
Quit anyway? (y or n) n
Not confirmed.
(gdb) up
#9 0x00002b66a53ec065 in NS_CopyUnicodeToNative (input=..., output=...)
at
/home/ishikawa/TB-NEW/TB-3HG/new-src/mozilla/xpcom/io/nsNativeCharsetUtils.cpp:857
857 nsresult rv = conv.UnicodeToNative(&buf, &bufLeft, &p,
&tempLeft);
(gdb) up
#10 0x00002b66a35149bb in WriteConsoleLog ()
at
/home/ishikawa/TB-NEW/TB-3HG/new-src/mozilla/toolkit/xre/nsConsoleWriter.cpp:87
87 NS_CopyUnicodeToNative(msg, nativemsg);
(gdb) print msg
$3 = {<nsString> = {<nsAString_internal> = {mData = 0x36676b0, mLength = 200,
mFlags = 9}, <No data fields>}, <No data fields>}
(gdb) list
82 nsAutoCString nativemsg;
83
84 for (uint32_t i = 0; i < mcount; ++i) {
85 rv = messages[i]->GetMessageMoz(getter_Copies(msg));
86 if (NS_SUCCEEDED(rv)) {
87 NS_CopyUnicodeToNative(msg, nativemsg);
88 PR_fprintf(file, "%s" NS_LINEBREAK, nativemsg.get());
89 }
90 NS_IF_RELEASE(messages[i]);
91 }
(gdb) print i
$4 = <optimized out>
(gdb) quit
A debugging session is active.
Inferior 1 [process 20083] will be detached.
Quit anyway? (y or n) y
Detaching from program:
/home/ishikawa/TB-NEW/TB-3HG/objdir-tb3/mozilla/dist/bin/thunderbird, process
20083
ishikawa@debian-amd64-vmplayer:/TB-NEW/TB-3HG/objdir-tb3/mozilla/dist/bin$
As you can see, "-O2" optimized out many local variables and so I am
recompiling the object using -O and get back with more informative
result.
Still, does TB 64-bit version work for other people without any
noticeable issues? (I am surprised at the "failure" of "make mozmill"
of TB 64-bit version so far.)
TIA
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
