Hi,
I have the following function 'LoadPatternDB' where I am allocating memory
for a C struct.My C struct contains the callback function.
I have the function 'LoadPatternDB' that is running in multiple go
routines.I have written a C wrapper that internally uses syslog-ng methods.
While running the method 'LoadPatternDB' in syslogparser.go with multiple
go routines, I get the following error.
**** Error in `fatal error: bin/main': double free or corruption
(fasttop)unexpected signal during runtime execution: 0x00007fd2b8588060 ****
My queries are:
1)When do we get this error ?
2)Is it because of not deallocating the struct memory, I am getting this
error ?But at the same time I am allocating separate memories for each of
the go routines.
3)Is it because of the memory crunch on the system that can lead to this
error ?
4)The issues is seen with only multiple go routines.Single iteration of the
code is working fine.
5)Can I not call Go code and C code using multiple threads ?
Can some one please guide me here ?
syslog-node.c
===========
PatternDB *patterndb[WORKERIDS];
int load_pattern_db(const gchar* filename, key_value_cb cb, int workerId)
{
if(patterndb[workerId] != NULL){
pattern_db_free(patterndb[workerId]);
}
patterndb[workerId] = pattern_db_new();
pattern_db_reload_ruleset(patterndb[workerId], configuration, filename);
pattern_db_set_emit_func(patterndb[workerId], pdbtool_pdb_emit_accumulate,
cb);
return 0;
}
gboolean pdbtool_accumulate_fields(NVHandle handle, const gchar *name,
const gchar *value, gssize length, gpointer user_data)
{
struct Accumulatedparams *params = user_data;
key_value_cb cb = params->callback;
cb(name, value, length, params->data);
return FALSE;
}
void pdbtool_pdb_emit_accumulate(LogMessage *msg, gboolean synthetic,
gpointer user_data)
{
nv_table_foreach(msg->payload, logmsg_registry, pdbtool_accumulate_fields,
user_data);
}
syslogparser.go
=============
func (obj SyslogParser) LoadPatternDB(opts Syslog, workerId int) {
patterndbpath := C.CString(opts.Patterndb)
defer C.free(unsafe.Pointer(patterndbpath))
InitStruct := (*C.Accumulatedparams)(C.calloc(1,
C.sizeof_struct_Accumulatedparams))
InitStruct.callback = (C.key_value_cb)(C.callback)
InitStruct.data = C.int(workerId)
C.load_pattern_db(patterndbpath,
(*C.Accumulatedparams)(unsafe.Pointer(InitStruct)), C.int(workerId))
}
//export Result
func Result(key *C.char, value *C.char, value_len C.size_t, mapid C.int) {
fmt.Println("Map Id: " + strconv.Itoa(int(mapid)))
value_field := C.GoStringN(value, C.int(value_len))
key_field := C.GoString(key)
remap, ok := constants.FIELD_MAPPINGS[key_field]
if ok {
Check.result[int(mapid)][remap] = value_field
//check.result[remap] = value_field
} else {
//check.result[key_field] = value_field
Check.result[int(mapid)][key_field] = value_field
}
}
cfuncs.go
-========
package lib
/*
#include <stdio.h>
// The gateway function
void callback(char *key, char *value, size_t value_len, int mapid)
{
//printf("C.callOnMeGo_cgo(): called");
void Result(const char *key, const char *value, size_t value_len, int
mapid);
Result(key, value, value_len, mapid);
}
*/
import "C"
syslog-node.h
============
#ifndef TEST_H_INCLUDED
#define TEST_H_INCLUDED
#include <stdlib.h>
typedef void (*key_value_cb)(const char* key, const char* value, size_t
value_len, int data);
typedef struct Accumulatedparams{
key_value_cb callback;
int data;
}Accumulatedparams;
int load_pattern_db(const char* filename, Accumulatedparams *cb, int
workerId);
#endif
On Fri, Jul 12, 2019 at 12:32 AM Nitish Saboo <nitish.sabo...@gmail.com>
wrote:
> This is the code snippet I was talking about:
>
> store := make([]*C.struct_Accumulatedparams, 2)
>
>
> func (obj Syslogparser) LoadPatternDB(opts Syslog) {
> >
> > patterndb_path := C.CString(opts.patterndb)
> > store[0] = (*C.Accumulatedparams)(C.calloc(1,
> C.sizeof_struct_Accumulatedparams))
> > store[0].callback = (C.key_value_cb)(C.callOnMeGo_cgo)
> > store[0].data = C.int(1)
> > C.load_pattern_db(patterndb_path,
> (*C.Accumulatedparams)(unsafe.Pointer(store[0])))
> > }
>
>
> On Fri, 12 Jul 2019, 00:22 Nitish Saboo, <nitish.sabo...@gmail.com> wrote:
>
>> Hi Ian,
>>
>> Can you please guide me with the correct way to store the address of a C
>> struct in a Go array so that I can use it later for deallocating the memory?
>>
>> Thanks,
>> Nitish
>>
>> On Fri, 12 Jul 2019, 00:04 Ian Lance Taylor, <i...@golang.org> wrote:
>>
>>> On Thu, Jul 11, 2019 at 10:58 AM Nitish Saboo <nitish.sabo...@gmail.com>
>>> wrote:
>>> >
>>> > pattern_db_set_emit_func(patterndb[workerId],
>>> pdbtool_pdb_emit_accumulate, cb);
>>> > suggests that cb is being saved away for future use.
>>> >
>>> > >>>>yes cb is being saved away and is being used for future use for
>>> callbacks to Go code.
>>>
>>> Then clearly the Go code must not immediately free the C memory. That
>>> is no different from how C code would work.
>>>
>>>
>>> > syslog-node.h
>>> > ============
>>> >
>>> > #ifndef TEST_H_INCLUDED
>>> > #define TEST_H_INCLUDED
>>> >
>>> > #include <stdlib.h>
>>> > typedef void (*key_value_cb)(const char* key, const char* value,
>>> size_t value_len, int data);
>>> > typedef struct Accumulatedparams{
>>> > key_value_cb callback;
>>> > int data;
>>> > }Accumulatedparams;
>>> > int initialize_engine(const char* filename, const char* module_path);
>>> > int reload_pattern_db(const char* filename, Accumulatedparams *cb,
>>> int workerId);
>>> > int load_pattern_db(const char* filename, Accumulatedparams *cb, int
>>> workerId);
>>> >
>>> > >>>>>If I declare the array in the following manner:
>>> >
>>> > result := make([]*C.struct_Accumulatedparams)
>>> > store[0] = (*C.Accumulatedparams)(C.calloc(1,
>>> C.sizeof_struct_Accumulatedparams))
>>> >
>>> > Here also the code gets compiled successfully, just that the GoLand
>>> IDE shows store[0].callback and store[0].data in red color saying
>>> Unresolved reference 'callback' and Unresolved reference 'data'
>>> respectively.Not sure how the code compiles if the IDE shows it in red
>>> color. Binary(./main) works fine
>>>
>>> I don't know what the GoLand IDE does. I don't see any
>>> store[0].callback or store[0].data in that code snippet.
>>>
>>>
>>> > Though this works,
>>> >
>>> > store[0] = (*C.Accumulatedparams)(C.calloc(1,
>>> C.sizeof_struct_Accumulatedparams))
>>> > InitStruct := store[0]
>>> > InitStruct.callback = (C.key_value_cb)(C.callOnMeGo_cgo)
>>> > InitStruct.data = C.int(1)
>>> >
>>> > Here the code compiles successfully and there is no complain from
>>> GoLand IDE as well.Binary (./main) too works fine
>>> >
>>> > So what i thought to do :
>>> >
>>> > type Store struct{
>>> > result []*C.struct_Accumulatedparams
>>> > data []C.int
>>> > callback []C.key_value_cb
>>> > }
>>> >
>>> > var store = Store{result: make([]*C.struct_Accumulatedparams, 2),
>>> data: make([]C.int ,2), callback:make([]C.key_value_cb, 2)}
>>> >
>>> > func (obj Syslogparser) LoadPatternDB(opts Syslog) {
>>> >
>>> > patterndb_path := C.CString(opts.patterndb)
>>> > store.result[0] = (*C.Accumulatedparams)(C.calloc(1,
>>> C.sizeof_struct_Accumulatedparams)) <<<<<<
>>> > store.callback[0] = (C.key_value_cb)(C.callOnMeGo_cgo) <<<<<
>>> > store.data[0] = C.int(1)
>>> > C.load_pattern_db(patterndb_path,
>>> (*C.Accumulatedparams)(unsafe.Pointer(store.result[0])))
>>> > }
>>> >
>>> > Here the code gets compiled successfully. But I am getting the
>>> following error when I run the binary './main'
>>> >
>>> > 1In reload PatternDb:
>>> > PatternDb :/home/nsaboo/Documents/goworkspace/src/poc/reload.xml
>>> > ModulePath :usr/local/lib/syslog-ng
>>> > fatal error: unexpected signal during runtime execution
>>> > [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x0]
>>> >
>>> > runtime stack:
>>> > runtime.throw(0x4fa203, 0x2a)
>>> > /usr/local/go/src/runtime/panic.go:617 +0x72
>>> > runtime.sigpanic()
>>> > /usr/local/go/src/runtime/signal_unix.go:374 +0x4a9
>>> >
>>> > goroutine 1 [syscall]:
>>> > runtime.cgocall(0x4b8fb0, 0xc000042cc8, 0x4d9280)
>>> > /usr/local/go/src/runtime/cgocall.go:128 +0x5b fp=0xc000042c98
>>> sp=0xc000042c60 pc=0x404f6b
>>> > main._Cfunc_execute_percustomer_db(0x7fb0fb5a5fa0, 0x1d5,
>>> 0x7fb0fb8259b0, 0x8)
>>> > _cgo_gotypes.go:121 +0x45 fp=0xc000042cc8 sp=0xc000042c98 pc=0x4b5ca5
>>> > main.Syslogparser.ParseMessagePerCustomer(0xc0000a2000, 0x36,
>>> 0x4f6b0c, 0x18, 0x4f3fc4, 0x8, 0xc0000d0000, 0x1d5)
>>> > /home/nsaboo/Documents/goworkspace/src/poc/main.go:132 +0x109
>>> fp=0xc000042d38 sp=0xc000042cc8 pc=0x4b6f09
>>> > main.perCustomerParsing(...)
>>> > /home/nsaboo/Documents/goworkspace/src/poc/main.go:249
>>> > main.main()
>>> > /home/nsaboo/Documents/goworkspace/src/poc/main.go:232 +0x6e1
>>> fp=0xc000042f98 sp=0xc000042d38 pc=0x4b78e1
>>> > runtime.main()
>>> > /usr/local/go/src/runtime/proc.go:200 +0x20c fp=0xc000042fe0
>>> sp=0xc000042f98 pc=0x42cd9c
>>> > runtime.goexit()
>>> > /usr/local/go/src/runtime/asm_amd64.s:1337 +0x1 fp=0xc000042fe8
>>> sp=0xc000042fe0 pc=0x4548c1
>>> > }
>>> >
>>> >
>>> > Where am i going wrong here ?
>>>
>>> Somewhere your C code is getting a NULL pointer dereference.
>>>
>>> I don't see much in these questions that seems specific to Go.
>>>
>>> Ian
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/CALjMrq4GDLv1CZVUL47U1tN0Qn%3DAjYEdq%3D-naQGT%3DZ_jMi0wpQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
