Hi,
So finally I got a little hint of the problem from what Robert described
earlier in the mail. Thank you so much Robert.
Looks like patterndb instance is not getting freed.
node.c
---------
PatternDB *patterndb;
int load_pattern_db(const gchar* file, key_value_cb cb)
{
if(patterndb != NULL){
printf("Patterndb Free called\n"); <<< Not getting printed
pattern_db_free(patterndb);
}
patterndb = pattern_db_new();
pattern_db_reload_ruleset(patterndb, configuration, file);
pattern_db_set_emit_func(patterndb, pdbtool_pdb_emit_accumulate, cb);
return 0;
}
patterndb is a global variable in C wrapper code that internally calls some
syslog-ng library api's.Since load_pattern_db() method is getting called
from a single goroutine every 3 mins, patterndb instance is not getting
free because the statement inside if clause ('if(patterndb != NULL)') is
not getting printed when I call 'load_pattern_db()' method.Looks like that
is the leak here.
1)Can someone please help me understand the problem in detail as in why am
I facing this issue?
2)Though patterndb instance is a global variable in the C wrapper code, why
is it not getting freed?
3)How can I fix this issue?
Thanks,
Nitish
On Mon, Mar 16, 2020 at 8:17 PM Nitish Saboo <nitish.sabo...@gmail.com>
wrote:
> Hi Robert,
>
> Sorry I did not understand your point completely.
> I have a global variable patterndb on C side and It is getting called from
> a single goroutine every 3 mins. Why do I need to synchronize it?
> Even though the goroutine gets pinned to different threads, it can access
> the same global variable every time and free it ...right ?
>
> Thanks,
> Nitish
>
>
> On Mon, Mar 16, 2020 at 8:10 PM Robert Engels <reng...@ix.netcom.com>
> wrote:
>
>> Yes, you have a shared global variable you need to synchronize.
>>
>> On Mar 16, 2020, at 9:35 AM, Nitish Saboo <nitish.sabo...@gmail.com>
>> wrote:
>>
>>
>> Hi,
>>
>> Are you saying it is working as expected?
>>
>> Thanks,
>> Nitish
>>
>> On Mon, Mar 16, 2020 at 7:42 PM Volker Dobler <dr.volker.dob...@gmail.com>
>> wrote:
>>
>>> On Monday, 16 March 2020 14:25:52 UTC+1, Nitish Saboo wrote:
>>>>
>>>> Hi,
>>>>
>>>> I upgraded the go version and compiled the binary against go version
>>>> 'go version go1.12.4 linux/amd64'.
>>>> I ran the program for some time. I made almost 30-40 calls to the
>>>> method Load_Pattern_Db().
>>>> The program starts with 6% Mem Usage. The memory usage increases only
>>>> when I call 'LoadPatternDb()' method and LoadPatternDb() method is called
>>>> by a goroutine at regular intervals of 3 minutes(making use of ticker here
>>>> ).
>>>>
>>>> What I observed is:
>>>>
>>>> 1)After almost 16-17 calls to the method 'LoadPatternDb(), the memory
>>>> usage got almost constant at 29%. But I did not expect the program to take
>>>> this much memory.
>>>> When I restart the service the Mem Usage again starts with 6%.
>>>>
>>>> a) Is this the sign of memory leaking?
>>>>
>>>
>>> No, as explained above.
>>>
>>>
>>>>
>>>> b) Till this moment I did not see memory getting reclaimed or going
>>>> down but it did become constant.
>>>> As mentioned by experts above, the same sort of behavior is seen here.
>>>> But I did not expect the memory usage to grow this much. Is this expected?
>>>>
>>> Yes. (Well, no. But your gut feeling of how much memory
>>> should grow is not a suitable benchmark to compare
>>> actual growth to.)
>>>
>>>
>>>>
>>>> 2)I will run mem-profiling at intervals(10 minutes, 100 minutes..etc)
>>>> as mentioned in the earlier email.
>>>>
>>>> a) Which all mem-stats variables should I look into for debugging this
>>>> kind of behavior?
>>>>
>>> Alloc/HeapAlloc
>>> But probably this is plain useless as nothing here indicates
>>> that you do have any memory issues.
>>>
>>> V.
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/e664151d-474d-4c1d-ae1d-979dc6975469%40googlegroups.com
>>> <https://groups.google.com/d/msgid/golang-nuts/e664151d-474d-4c1d-ae1d-979dc6975469%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/CALjMrq7EuvpFBaAQCJfO_QhkW8ceac8oEv-oFq9GPsik%3D5GNkw%40mail.gmail.com
>> <https://groups.google.com/d/msgid/golang-nuts/CALjMrq7EuvpFBaAQCJfO_QhkW8ceac8oEv-oFq9GPsik%3D5GNkw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/CALjMrq4Kz7_S_gWiWyS%3Dn%2BEsX%3D7rYvHhKcX7MGkg_O4oQXwo4A%40mail.gmail.com.
