Hi Michael,
Thanks for pointing it out.I am a beginner with Go and CGO and have never
worked on corruption issues with C.
I am familiar with python and Java.So just seeking help from the experts to
find the pain points that should be looked into.
Hence, asking questions to Ian as in where all I should look for the
corruption issues.
Thanks,
Nitish
On Fri, Jul 12, 2019 at 9:51 PM Michael Jones <michael.jo...@gmail.com>
wrote:
> It is really impossible to help you debug what is to the thousands of
> readers here an invisible unseen program. You have a logic error. You don't
> know where it is (of course, you are busy trying to find it) but we can't
> know either as we don't see the code. General advice, as always, is to
> "trap" the bug in the smallest stand-alone code that you can. That will
> focus your search.
>
> You just shared this:
>
> Following is one of the methods called before this method :
>
> func (obj SyslogParser) InitializeEngine() {
> patterndb := C.CString(obj.Patterndb)
> defer C.free(unsafe.Pointer(patterndb))
> modulepath := C.CString(obj.Modulepath)
> defer C.free(unsafe.Pointer(modulepath))
> C.initialize_engine(patterndb, modulepath)
> }
>
> Not deallocating the memory for the struct can be the issue for 'fatal
> error: bin/main': double free or corruption (fasttop)' ?
>
> Is this right? Is it correct to call C.initialize_engine(a,b) then
> immediately call C.free(a) and C.free(b)? It might be fine if your
> initialize engine clones/duplicates/or otherwise uses a & b just once. But
> if not, then of course A & B will both instantly become death traps for
> your C code during runtime of the engine, for any allocations that are
> where A & B were located, and potentially when your "engine" is deallocated
> later on.
>
> Just one example of might be fine, might be disaster, no way for us to
> tell. It seems that you have a logic problem in the program, not a C-to-Go
> or Go-to-C interfacing problem. You're going to need to study your code
> (unseen by us) yourself to see what that is. I think so anyway. Maybe Ian
> is so smart that he can debug this for you by ESP. ;-)
>
> Michael
>
> On Fri, Jul 12, 2019 at 8:15 AM Nitish Saboo <nitish.sabo...@gmail.com>
> wrote:
>
>> Hi Ian,
>>
>> > syslogparser.go
>> > =============
>> >
>> > func (obj SyslogParser) LoadPatternDB(opts Syslog, workerId int) {
>> log.Info("Loading pattern db with path:", opts.Patterndb)
>> > patterndbpath := C.CString(opts.Patterndb) . <<<<<<<<<<<<<<< . STEP 1
>> > defer C.free(unsafe.Pointer(patterndbpath)) . <<<<<<<<<<<<<<<< STEP 2
>> > InitStruct := (*C.Accumulatedparams)(C.calloc(1,
>> C.sizeof_struct_Accumulatedparams))
>> > InitStruct.callback = (C.key_value_cb)(C.callback)
>> > InitStruct.data = C.int(workerId)
>> > C.load_pattern_db(patterndbpath,
>> (*C.Accumulatedparams)(unsafe.Pointer(InitStruct)), C.int(workerId))
>> > }
>> >
>> > Since I am running this program in multiple go routines, when we do
>> above STEP 1 :
>> >
>> > 1)Will the C memory allocate different addresses for 'patterndbpath'
>> in both the routines ...Am I right ?
>>
>> Yes.
>>
>> > 2)You said 'It may be calling free twice on the same pointer', but I am
>> assigning a different pointer in STEP 1 everytime for each routine...Am I
>> right ?
>>
>> Yes, you will see a different pointer each time you call C.calloc, and
>> in the code that I see that pointer is only freed once, but I have no
>> idea what the rest of your code is doing.
>>
>> I am calling free only once and that is for 'patterndbpath' in this
>> particular method.In the entire codebase I am NOT calling free for
>> InitStruct (Memory allocated for the struct)
>> I am pointing out this particular method 'LoadPatternDB' of Go code
>> base because I had put a 'Info' log in this method at the starting and
>> that got printed before the corruption error came up.
>> And my testing failed at the third iteration:
>>
>> 1)WorkerID0 completed the work
>> 2)WorkerID1 completed the work
>> 3)WorkerID1 failed in this particular method with 'fatal error:
>> bin/main': double free or corruption (fasttop)'
>>
>> Following is one of the methods called before this method :
>>
>> func (obj SyslogParser) InitializeEngine() {
>> patterndb := C.CString(obj.Patterndb)
>> defer C.free(unsafe.Pointer(patterndb))
>> modulepath := C.CString(obj.Modulepath)
>> defer C.free(unsafe.Pointer(modulepath))
>> C.initialize_engine(patterndb, modulepath)
>> }
>>
>> Not deallocating the memory for the struct can be the issue for 'fatal
>> error: bin/main': double free or corruption (fasttop)' ?
>>
>> > 3)Is it possible that one Go routine has already freed up the address
>> that the other Go routine is trying to free and that is causing this
>> 'fatal error: bin/main': double free or corruption (fasttop)'
>>
>> Yes, that is possible. It won't happen with the code I see above but
>> I have no idea what the rest of your code is doing.
>>
>> How can it not happen with this code ?I did not understand this point.
>> Can you please clarify this point ?
>>
>> > Is this because of the race condition between two routines that this
>> error is seen ?
>>
>> It is possible.
>>
>> > 4)If you show us a small, complete, self-contained example, that does
>> not call any other code, then we may be able to see the problem.
>> > >> As you have mentioned problem is happening when we call 'C.free'.Do
>> you mean by not calling internal syslog-ng methods and just calling a
>> normal C program with multiple go routines ?
>> > Please correct me here if I am wrong
>>
>> I mean any small, self-contained example where we can see all the code.
>>
>> >>>I will try to send a very small, basic code but not sure how to
>> reproduce the issue with that.
>>
>> You seem to be looking at a memory corruption or double free problem.
>> These are normal, ordinary, problems when writing C code. We can't
>> debug your problem for you.
>>
>> On Fri, Jul 12, 2019 at 7:54 PM Ian Lance Taylor <i...@golang.org> wrote:
>>
>>> On Thu, Jul 11, 2019 at 11:41 PM Nitish Saboo <nitish.sabo...@gmail.com>
>>> wrote:
>>> >
>>> > syslogparser.go
>>> > =============
>>> >
>>> > func (obj SyslogParser) LoadPatternDB(opts Syslog, workerId int) {
>>> > patterndbpath := C.CString(opts.Patterndb) . <<<<<<<<<<<<<<< . STEP 1
>>> > defer C.free(unsafe.Pointer(patterndbpath)) . <<<<<<<<<<<<<<<< STEP 2
>>> > InitStruct := (*C.Accumulatedparams)(C.calloc(1,
>>> C.sizeof_struct_Accumulatedparams))
>>> > InitStruct.callback = (C.key_value_cb)(C.callback)
>>> > InitStruct.data = C.int(workerId)
>>> > C.load_pattern_db(patterndbpath,
>>> (*C.Accumulatedparams)(unsafe.Pointer(InitStruct)), C.int(workerId))
>>> > }
>>> >
>>> > Since I am running this program in multiple go routines, when we do
>>> above STEP 1 :
>>> >
>>> > 1)Will the C memory allocate different addresses for 'patterndbpath'
>>> in both the routines ...Am I right ?
>>>
>>> Yes.
>>>
>>> > 2)You said 'It may be calling free twice on the same pointer', but I
>>> am assigning a different pointer in STEP 1 everytime for each routine...Am
>>> I right ?
>>>
>>> Yes, you will see a different pointer each time you call C.calloc, and
>>> in the code that I see that pointer is only freed once, but I have no
>>> idea what the rest of your code is doing.
>>>
>>> > 3)Is it possible that one Go routine has already freed up the address
>>> that the other Go routine is trying to free and that is causing this
>>> 'fatal error: bin/main': double free or corruption (fasttop)'
>>>
>>> Yes, that is possible. It won't happen with the code I see above but
>>> I have no idea what the rest of your code is doing.
>>>
>>> > Is this because of the race condition between two routines that this
>>> error is seen ?
>>>
>>> It is possible.
>>>
>>> > 4)If you show us a small, complete, self-contained example, that does
>>> not call any other code, then we may be able to see the problem.
>>> > >> As you have mentioned problem is happening when we call 'C.free'.Do
>>> you mean by not calling internal syslog-ng methods and just calling a
>>> normal C program with multiple go routines ?
>>> > Please correct me here if I am wrong
>>>
>>> I mean any small, self-contained example where we can see all the code.
>>>
>>> You seem to be looking at a memory corruption or double free problem.
>>> These are normal, ordinary, problems when writing C code. We can't
>>> debug your problem for you.
>>>
>>> Ian
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/CALjMrq6eM1_ykJYmNVYqnFnjxqRvRCJF8Bqgh6k2N7kBV%2B_3Dw%40mail.gmail.com
>> <https://groups.google.com/d/msgid/golang-nuts/CALjMrq6eM1_ykJYmNVYqnFnjxqRvRCJF8Bqgh6k2N7kBV%2B_3Dw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
> --
>
> *Michael T. jonesmichael.jo...@gmail.com <michael.jo...@gmail.com>*
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/CALjMrq4ZC_2dwC3F0%3DJnphWd7K8TjzRvbtVP08k93pTqTTRE1g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
