Re: Use of chunksize before initialization
On 03/21/2015 11:31 PM, Konstantin Belousov wrote: On Sat, Mar 21, 2015 at 11:20:26AM +0200, Ivan A. Kosarev wrote: On 03/21/2015 03:02 AM, Konstantin Belousov wrote: On Fri, Mar 20, 2015 at 03:59:52PM +0200, Ivan A. Kosarev wrote: #12 0x0008011b428d in malloc_init_hard () at jemalloc_jemalloc.c:698 #13 malloc_init () at jemalloc_jemalloc.c:296 #14 0x000801243ea2 in ?? () from /lib/libc.so.7 #15 0x0008006a5400 in ?? () #16 0x00080089e5b0 in ?? () from /libexec/ld-elf.so.1 #17 0x7fffe0b0 in ?? () #18 0x000801139d06 in _init () from /lib/libc.so.7 #19 0x7fffe0b0 in ?? () The backtrace is strange. Did you compiled malloc with the debugging symbols, while keep rest of libc without -g ? I've just added the -g flag to CC_FLAGS in the Makefile and made sure to install an unstripped version of the .so . I could investigate more on why the early calls omit debug symbols, if it does any matter. I want to understand at what stage of the initialization the access happens. This is why I want to see the complete backtrace. It is jemalloc_constructor() that calls malloc_init(), so it should be called directly by the loader. -- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Use of chunksize before initialization
On 03/21/2015 03:02 AM, Konstantin Belousov wrote: On Fri, Mar 20, 2015 at 03:59:52PM +0200, Ivan A. Kosarev wrote: #12 0x0008011b428d in malloc_init_hard () at jemalloc_jemalloc.c:698 #13 malloc_init () at jemalloc_jemalloc.c:296 #14 0x000801243ea2 in ?? () from /lib/libc.so.7 #15 0x0008006a5400 in ?? () #16 0x00080089e5b0 in ?? () from /libexec/ld-elf.so.1 #17 0x7fffe0b0 in ?? () #18 0x000801139d06 in _init () from /lib/libc.so.7 #19 0x7fffe0b0 in ?? () The backtrace is strange. Did you compiled malloc with the debugging symbols, while keep rest of libc without -g ? I've just added the -g flag to CC_FLAGS in the Makefile and made sure to install an unstripped version of the .so . I could investigate more on why the early calls omit debug symbols, if it does any matter. Does it happen always, on only for the early initialization of the mutexes ? I'm not sure I understand the whole logic of the initialization process, but we could put a statement initializing the chunksize variable to 0 to the beginning of malloc_init_hard() and see if the assertion (or any other before it) fails. Since my suspicion is that the variable get random values at base_boot(), the presence of the failure depends on random factors. For a simple one-line program calling malloc() it is known to not to fail, of course. I should be able to to more tests on Mon. It might be related to r276630. Can you test on, say, 10.1 ? The Tsan tests mentioned below that cause mass (alignment != 0) failures are known to work fine on 10.1. : jemalloc_chunk.c:152: Failed assertion: "alignment != 0" Here's more of failures of this kind around: http://lab.llvm.org:8011/builders/sanitizer_x86_64-freebsd/builds/4758/steps/make-check-tsan/logs/stdio Can you please let me know if the analysis is correct and there's something to fix about initialization of the variable? Backtrace looks valid. Thanks. -- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Use of chunksize before initialization
Hello everybody,
The malloc_init_hard() function defined in jemalloc_jemalloc.c, FreeBSD
11 r277486 reads:
static bool
malloc_init_hard(void)
{
...
if (base_boot()) {
malloc_mutex_unlock(&init_lock);
eturn (true);
}
if (chunk_boot()) {
malloc_mutex_unlock(&init_lock);
return (true);
}
...
The second call initializes the 'chunksize' global variable defined in
jemalloc_chunk.c:
bool
chunk_boot(void)
{
/* Set variables according to the value of opt_lg_chunk. */
chunksize = (ZU(1) << opt_lg_chunk);
assert(chunksize >= PAGE);
...
However, it seems the first call to base_boot() depends on that variable
already:
(gdb) bt
#0 thr_kill () at thr_kill.S:3
#1 0x000801241408 in __raise (s=6) at /usr/src/lib/libc/gen/raise.c:51
#2 0x0041d817 in __interceptor_raise () at
/usr/home/ik/llvm/llvm.current/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:2097
#3 0x00080123f969 in abort () at /usr/src/lib/libc/stdlib/abort.c:65
#4 0x0041c5d9 in __interceptor_abort () at
/usr/home/ik/llvm/llvm.current/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:1851
#5 0x0008011a8d64 in __je_chunk_alloc (size=,
alignment=, base=, zero=,
dss_prec=dss_prec_disabled) at jemalloc_chunk.c:150
#6 0x0008011a9bfc in base_pages_alloc (minsize=128) at
jemalloc_base.c:35
#7 __je_base_alloc (size=) at jemalloc_base.c:57
#8 0x0008011a9c96 in __je_base_calloc (number=,
size=6) at jemalloc_base.c:74
#9 0x0008008ae548 in mutex_init (calloc_cb=0x0, mutex=out>, mutex_attr=) at
/usr/src/lib/libthr/thread/thr_mutex.c:145
#10 _pthread_mutex_init_calloc_cb (mutex=0x801487c90, calloc_cb=0x0) at
/usr/src/lib/libthr/thread/thr_mutex.c:229
#11 0x0008011a18da in __je_malloc_mutex_init (mutex=0x18744) at
jemalloc_mutex.c:97
#12 0x0008011b428d in malloc_init_hard () at jemalloc_jemalloc.c:698
#13 malloc_init () at jemalloc_jemalloc.c:296
#14 0x000801243ea2 in ?? () from /lib/libc.so.7
#15 0x0008006a5400 in ?? ()
#16 0x00080089e5b0 in ?? () from /libexec/ld-elf.so.1
#17 0x7fffe0b0 in ?? ()
#18 0x000801139d06 in _init () from /lib/libc.so.7
#19 0x7fffe0b0 in ?? ()
Note that base_pages() calls chunk_alloc() with chucksize used as the
alignment value:
static bool
base_pages_alloc(size_t minsize)
{
...
base_pages = chunk_alloc(csize, chunksize, true, &zero,
chunk_dss_prec_get());
...
and the latter tests it against zero:
void *
chunk_alloc(size_t size, size_t alignment, bool base, bool *zero,
dss_prec_t dss_prec)
{
...
assert(alignment != 0);
so we sometimes we end up with:
: jemalloc_chunk.c:152: Failed assertion: "alignment != 0"
Here's more of failures of this kind around:
http://lab.llvm.org:8011/builders/sanitizer_x86_64-freebsd/builds/4758/steps/make-check-tsan/logs/stdio
Can you please let me know if the analysis is correct and there's
something to fix about initialization of the variable?
Thanks.
--
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: libthr and main thread stack size
On 08/13/2014 01:40 AM, Dimitry Andric wrote: On 08 Aug 2014, at 13:22, Konstantin Belousov wrote: On Fri, Aug 08, 2014 at 12:32:56PM +0400, Ivan A. Kosarev wrote: On 08/08/2014 09:28 AM, Konstantin Belousov wrote: On Thu, Aug 07, 2014 at 04:18:12PM +0400, Ivan A. Kosarev wrote: Hello, According to libthr's thr_init.c (the 9.2 version) init_main_thread() allocates s.c. "red zone" below the main stack in order to protect other stacks. The size of the main stack is determined by the _thr_stack_initial variable that is declared extern though it doesn't seem it can be changed. The value of the variable is set to 4M on 64-bit platforms which is obviously not sufficient for the most of real programs. Can anyone please confirm that there is no way to increase the stack size for the main thread and thus any program linked against libthr has only a few megabytes of stack memory for its main thread--whatever the system stack size (ulimit -s) is set to? Yes, there is no way to change the main thread stack clamping. Could you provide a reasonable use case for the 4MB stack ? Traversing trees with recursive functions or on-stack grammar parsers? I just ran into a similar issue while running one of clang 3.5's test cases (see http://llvm.org/PR20635 ). On i386, it used up approximately 2MB of stack, then ran into the guard page, and segfaulted. I was quite amazed to find out that ulimit -s didn't help at all, until I remembered this thread. :-) Yes, that was the point: at least some of real programs consume more stack space than what we have with the hardcoded libthr's limits. Also, it looks a bit weird to me that adding thread support/linking against libthr affects how much stack space we can consume and whether it can be controlled with ulimit. The original issue I've run into is one of the Asan's tests: http://reviews.llvm.org/D4562 -- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: libthr and main thread stack size
On 08/08/2014 09:28 AM, Konstantin Belousov wrote: On Thu, Aug 07, 2014 at 04:18:12PM +0400, Ivan A. Kosarev wrote: Hello, According to libthr's thr_init.c (the 9.2 version) init_main_thread() allocates s.c. "red zone" below the main stack in order to protect other stacks. The size of the main stack is determined by the _thr_stack_initial variable that is declared extern though it doesn't seem it can be changed. The value of the variable is set to 4M on 64-bit platforms which is obviously not sufficient for the most of real programs. Can anyone please confirm that there is no way to increase the stack size for the main thread and thus any program linked against libthr has only a few megabytes of stack memory for its main thread--whatever the system stack size (ulimit -s) is set to? Yes, there is no way to change the main thread stack clamping. Could you provide a reasonable use case for the 4MB stack ? Traversing trees with recursive functions or on-stack grammar parsers? Anyway, I somewhat sympathize to the idea to stop clamping the main thread stack, and to not reuse it for other threads stack carving. This also means that non-main threads stack allocator should stop tracking the explicit location for the stacks and rely on vm mmap(2) base selection instead. Yes, that would solve the problem. I do not know the motivations why the current scheme of stacks allocation was chosen. The changes do not look too involved. Thanks a lot. -- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
libthr and main thread stack size
Hello, According to libthr's thr_init.c (the 9.2 version) init_main_thread() allocates s.c. "red zone" below the main stack in order to protect other stacks. The size of the main stack is determined by the _thr_stack_initial variable that is declared extern though it doesn't seem it can be changed. The value of the variable is set to 4M on 64-bit platforms which is obviously not sufficient for the most of real programs. Can anyone please confirm that there is no way to increase the stack size for the main thread and thus any program linked against libthr has only a few megabytes of stack memory for its main thread--whatever the system stack size (ulimit -s) is set to? Thanks. -- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Demangling issues with libcxxrt.so.1 on v9.2
Hello everybody,
It seems there are problems with demandling some kinds of names with
libcxxrt.so.1 on FreeBSD 9.2 (I didn't test other versions yet).
This program:
---
#include
#include
extern "C" char* __cxa_demangle(const char* mangled_name,
char* buf, size_t* n, int* status);
void test(const char *mangled) {
int status = 0;
char *DemangledName = __cxa_demangle(mangled, NULL, NULL, &status);
printf("%s: status %d", mangled, status);
if(status == 0)
printf("; demangled: '%s'", DemangledName);
free(DemangledName);
printf("\n");
}
int main(void) {
test("_Z9NullDerefPi");
test("_ZL9NullDerefPi");
test("_ZN8DeepFreeILi0EE4freeEPc");
test("_ZN8DeepFreeILi13EE4freeEPc");
test("_ZN8DeepFreeILi36EE4freeEPc");
return 0;
}
---
outputs:
---
_Z9NullDerefPi: status 0; demangled: 'NullDeref(int*)'
_ZL9NullDerefPi: status -2
_ZN8DeepFreeILi0EE4freeEPc: status 0; demangled: 'DeepFree<0E>::free(char*)'
_ZN8DeepFreeILi13EE4freeEPc: status 0; demangled:
'DeepFree<13E>::free(char*)'
_ZN8DeepFreeILi36EE4freeEPc: status 0; demangled:
'DeepFree<36E>::free(char*)'
---
Note that it fails to demangle the local name in the 2nd line and adds
extra 'E' character in DeepFree<...E>s.
The case with the local name is not critical, but the case with
DeepFree<...E> prevents LLVM's address sanitizer tests from passing on
FreeBSD so the question is: is there a chance the defect will be
resolved any time soon or should I try to prepare a fix to speed up the
process?
Thanks a lot.
--
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Intercepting calls in PIC mode
Hello,
Consider the following:
---
#include
#include
extern "C" void* memset(void *block, int c, size_t size)
__attribute__((weak, alias("__int_memset"), visibility("default")));
extern "C" __attribute__((visibility("default")))
void* __int_memset(void *block, int c, size_t size) {
puts("Hello");
return NULL;
}
int main()
{
void *(*F)(void *b, int c, size_t len) = memset;
char a[5];
memset(a, 0, sizeof(a));
F(a, 0, sizeof(a));
return 0;
}
---
It intercepts the memset() calls without issue on both x86-64 FreeBSD
9.2 and Linux. However, with the -fPIC option specified in the cc's
command line, only the first (direct) call work on FreeBSD, but not the
second (indirect) one. Note is that on Linux both the calls are
intercepted--no matter whether the -fPIC option is specified or not.
The question is: is there a way to intercept indirect calls on FreeBSD?
Thanks,
--
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
