Hello Michael,
I have started to investigate the memory consumption issue in more detail, and
can confirm it is leaking around 32 bytes per call of say_hello(). That is
about the size of the User helloworld_object and the user->name wchar_t which
is allocated in the client loop, but clearly must be a copy of that data if
this is the issue. The leak traceback is below (for a run with a loop length of
1000):
32,768 bytes in 4 blocks are possibly lost in loss record 567 of 567
at malloc (vg_replace_malloc.c:195)
by apr_pool_create_ex (apr_pools.c:344)
by new_queue (etch_queue.c:59)
by new_mailbox_a (etch_plain_mailbox.c:119)
by new_mailbox (etch_plain_mailbox.c:82)
by pmboxmgr_transport_call (etch_plain_mailbox_manager.c:543)
by tcpdelsvc_begincall (etch_transport.c:525)
by etchremote_begincall (etch_remote.c:129)
by helloworld_remote_begin_server_say_hello (helloworld_remote_server.c:217)
by helloworld_remote_server_say_hello (helloworld_remote_server.c:276)
by main (helloworld_client_main.c:183)
The leak size at this location varies depending on the number of times I
iterate over the say_hello() invocation. Here are the leak sizes from this
routine at various loop durations:
500 -> 16,384 bytes lost
1,000 -> 32,768 bytes lost
5,000 -> 155,648 bytes lost
10,000 -> 319,488 bytes lost
However, this memory is released upon a call to etch_runtime_shutdown(), so you
cannot see the leak unless you interrupt the program execution (or watch the
memory grow...).
As for my platform:
- Linux CentOS 5.4 (32 bit i686)
- libapr-1.4.5
- libapr-util-1.3.12
- libapr-iconv-1.2.1
- etch (latest from SVN)
- gcc 4.1.2
I hope that the traceback may give you some clue as to the source of the leak.
Otherwise, I will dig deeper when I find more time to focus on this again.
Viele Gruesse,
--thomas
-----Original Message-----
From: Michael Fitzner [mailto:[email protected]]
Sent: Wednesday, September 14, 2011 6:34 AM
To: [email protected]
Subject: AW: Etch/C Memory Consumption
Hi Thomas,
Your code should be correct and no memory leaks in this part. Could you provide
me with some more information about what systems do you use (Linux, Windows)
and a call stack of you memory leak if available. I will also try to reproduce
your test.
Thanks
Michael
-----Ursprüngliche Nachricht-----
Von: Thomas Marsh [mailto:[email protected]]
Gesendet: Dienstag, 13. September 2011 16:47
An: [email protected]
Betreff: RE: Etch/C Memory Consumption
Hello Martijn,
Thanks for your response. However, this is not the source of the leak. There is
a clear etch_object_destroy in the helloworld_remote_begin_server_say_hello()
which deallocates any parameters to the methods (meaning you will get a
segfault if you try to reuse the parameters which now no longer point to valid
memory). This usage semantic is also clearly stated in the C Binding notes
>From http://incubator.apache.org/etch/c-binding-tips-tricks.html:
The C Binding for Etch has the following memory management rules:
Implementation side: Parameters of functions have to be destroyed by
the function implementation using etch_object_destroy.
Caller side: Result Objects have to be freed by the caller using
etch_object_destroy. Parameters of calls will be freed by the runtime
automatically.
To reiterate, program memory usage in this very simple usage scenario _must_ be
stable. I suspect an ever growing hash table, or some other similar culprit
within the runtime. Can any of the Etch/C binding developers comment?
Thanks, and best regards,
--thomas
-----Original Message-----
From: Martijn Dashorst [mailto:[email protected]]
Sent: Tuesday, September 13, 2011 7:23 AM
To: [email protected]
Subject: Re: Etch/C Memory Consumption
And user->name = new_stringw(L"User"); will do so as well
Martijn
On Tue, Sep 13, 2011 at 1:52 PM, Martijn Dashorst <[email protected]>
wrote:
> user = new_helloworld_user()
>
> will allocate memory for each pass through the loop.
>
> Martijn
>
> On Mon, Sep 12, 2011 at 11:19 PM, Thomas Marsh <[email protected]>
> wrote:
>> Hello all,
>>
>> I have a question about memory consumption in the Etch/C runtime based on
>> behavior we are seeing within our C client. I have modified the C
>> implementation of the HelloWorld example in the distribution to run an
>> infinite loop of requests:
>>
>> In the main() routine of helloworld_client_main.c:
>>
>> ...
>> while (1) {
>> user = new_helloworld_user();
>> user->id = 5;
>> user->name = new_stringw(L"User");
>> result = remote->say_hello(remote,
>> user);
>> if (is_etch_exception(result)) {
>> ...
>> }
>> printf("%S\n", result->v.valw);
>> etch_object_destroy(result);
>> }
>> ...
>>
>> While running this, I see that the memory consumption of the client
>> continually grows. (In this example, it grows by about 1 mB every 5
>> seconds.) I cannot see the memory leak when testing with valgrind, so it
>> would suggest that the leak is in Etch managed memory which is cleared at
>> exit.
>>
>> My understanding is that the call to remote->say_hello() should delegate
>> responsibility of deallocation of the parameters to the Etch runtime, and
>> that the client code is only responsible for deallocating the result object.
>> The memory use should be stable within this tight loop. Can anyone comment
>> on the potential cause of the memory consumption?
>>
>> Thanks, and best regards,
>>
>> --thomas
>>
>
>
>
> --
> Become a Wicket expert, learn from the best: http://wicketinaction.com
>
--
Become a Wicket expert, learn from the best: http://wicketinaction.com
