pattern_db_set_emit_func(patterndb[workerId], pdbtool_pdb_emit_accumulate, cb); suggests that cb is being saved away for future use.
>>>>yes cb is being saved away and is being used for future use for callbacks to Go code. Yes, it's fine to store the address of the C struct on the Go side, if that helps. syslog-node.h ============ #ifndef TEST_H_INCLUDED #define TEST_H_INCLUDED #include <stdlib.h> typedef void (*key_value_cb)(const char* key, const char* value, size_t value_len, int data); typedef struct Accumulatedparams{ key_value_cb callback; int data; }Accumulatedparams; int initialize_engine(const char* filename, const char* module_path); int reload_pattern_db(const char* filename, Accumulatedparams *cb, int workerId); int load_pattern_db(const char* filename, Accumulatedparams *cb, int workerId); >>>>>If I declare the array in the following manner: result := make([]*C.struct_Accumulatedparams) store[0] = (*C.Accumulatedparams)(C.calloc(1, C.sizeof_struct_Accumulatedparams)) *Here also the code gets compiled successfully, just that the GoLand IDE shows **store[0].callback and **store[0].data in red color saying **Unresolved reference 'callback' and Unresolved reference 'data' respectively.Not sure how the code compiles if the IDE shows it in red color. Binary(./main) works fine* Though this works, store[0] = (*C.Accumulatedparams)(C.calloc(1, C.sizeof_struct_Accumulatedparams)) InitStruct := store[0] InitStruct.callback = (C.key_value_cb)(C.callOnMeGo_cgo) InitStruct.data = C.int(1) *Here the code compiles successfully and there is no complain from GoLand IDE as well.Binary (./main) too works fine* So what i thought to do : type Store struct{ result []*C.struct_Accumulatedparams data []C.int callback []C.key_value_cb } var store = Store{result: make([]*C.struct_Accumulatedparams, 2), data: make([]C.int ,2), callback:make([]C.key_value_cb, 2)} func (obj Syslogparser) LoadPatternDB(opts Syslog) { patterndb_path := C.CString(opts.patterndb) store.result[0] = (*C.Accumulatedparams)(C.calloc(1, C.sizeof_struct_Accumulatedparams)) <<<<<< store.callback[0] = (C.key_value_cb)(C.callOnMeGo_cgo) <<<<< store.data[0] = C.int(1) C.load_pattern_db(patterndb_path, (*C.Accumulatedparams)(unsafe.Pointer(store.result[0]))) } *Here the code gets compiled successfully. But I am getting the following error when I run the binary './main'* 1In reload PatternDb: PatternDb :/home/nsaboo/Documents/goworkspace/src/poc/reload.xml ModulePath :usr/local/lib/syslog-ng fatal error: unexpected signal during runtime execution [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x0] runtime stack: runtime.throw(0x4fa203, 0x2a) /usr/local/go/src/runtime/panic.go:617 +0x72 runtime.sigpanic() /usr/local/go/src/runtime/signal_unix.go:374 +0x4a9 goroutine 1 [syscall]: runtime.cgocall(0x4b8fb0, 0xc000042cc8, 0x4d9280) /usr/local/go/src/runtime/cgocall.go:128 +0x5b fp=0xc000042c98 sp=0xc000042c60 pc=0x404f6b main._Cfunc_execute_percustomer_db(0x7fb0fb5a5fa0, 0x1d5, 0x7fb0fb8259b0, 0x8) _cgo_gotypes.go:121 +0x45 fp=0xc000042cc8 sp=0xc000042c98 pc=0x4b5ca5 main.Syslogparser.ParseMessagePerCustomer(0xc0000a2000, 0x36, 0x4f6b0c, 0x18, 0x4f3fc4, 0x8, 0xc0000d0000, 0x1d5) /home/nsaboo/Documents/goworkspace/src/poc/main.go:132 +0x109 fp=0xc000042d38 sp=0xc000042cc8 pc=0x4b6f09 main.perCustomerParsing(...) /home/nsaboo/Documents/goworkspace/src/poc/main.go:249 main.main() /home/nsaboo/Documents/goworkspace/src/poc/main.go:232 +0x6e1 fp=0xc000042f98 sp=0xc000042d38 pc=0x4b78e1 runtime.main() /usr/local/go/src/runtime/proc.go:200 +0x20c fp=0xc000042fe0 sp=0xc000042f98 pc=0x42cd9c runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1337 +0x1 fp=0xc000042fe8 sp=0xc000042fe0 pc=0x4548c1 } *Where am i going wrong here ?* Thanks, Nitish On Thu, 11 Jul 2019, 23:03 Ian Lance Taylor, <i...@golang.org> wrote: > On Thu, Jul 11, 2019 at 9:27 AM Nitish Saboo <nitish.sabo...@gmail.com> > wrote: > > > > Following is my C Code: > > > > PatternDB *patterndb[WORKERIDS]; > > > > int load_pattern_db(const gchar* filename, key_value_cb cb, int workerId) > > { > > if(patterndb[workerId] != NULL){ > > pattern_db_free(patterndb[workerId]); > > } > > patterndb[workerId] = pattern_db_new(); > > pattern_db_reload_ruleset(patterndb[workerId], configuration, filename); > > pattern_db_set_emit_func(patterndb[workerId], > pdbtool_pdb_emit_accumulate, cb); > > return 0; > > } > > > > This is just the C wrapper that internally call syslog-ng methods. > > > > Since I am using an array of patterndbs, I am calling the C code with > multiple Go routines. > > > > Single iteration of the code is working fine. > > > > The issue is seen only with multiple go routines. Can we store the > address of the C struct (when memory is allocated using calloc) on Go side > in an array so that I can deallocate the memory using 'defer > C.free(unsafe.Pointer(InitStruct))' when the callback is done?This should > make multiple go routines to work...Am I right ? > > I don't know what this code does, but the line > > pattern_db_set_emit_func(patterndb[workerId], pdbtool_pdb_emit_accumulate, > cb); > > suggests that cb is being saved away for future use. If that is > indeed the case then freeing the memory in Go will cause a dangling > pointer, leading to crashes as you see. > > You should also double-check that each separate goroutine is using a > different "workerId", as duplication of workerId values could > certainly cause trouble when running concurrently. > > Yes, it's fine to store the address of the C struct on the Go side, if > that helps. > > Ian > > > > > On Thu, Jul 11, 2019 at 9:39 PM Ian Lance Taylor <i...@golang.org> > wrote: > >> > >> On Thu, Jul 11, 2019 at 7:16 AM Nitish Saboo <nitish.sabo...@gmail.com> > wrote: > >> > > >> > I have the following function 'LoadPatternDB' where I am allocating > memory for a C struct.My C struct contains the callback function. > >> > > >> > 1)I cannot deference the struct in the same func 'LoadPatternDB' > because I get the following error when my callback function is called from > C code: > >> > > >> > signal SIGSEGV: segmentation violation code=0x1 addr=0x7f1e00415341 > pc=0x7f1e00415341] > >> > > >> > 2)The same code is running in multiple go routines.And because of > that if I don't deallocate the memory I get the following error: > >> > > >> > *** Error in `fatal error: bin/main': double free or corruption > (fasttop)unexpected signal during runtime execution: 0x00007fd2b8588060 *** > >> > > >> > My question is: > >> > > >> > How can I store the address of the C struct (when memory is allocated > using calloc) on Go side in an array so that I can deallocate the memory > using 'defer C.free(unsafe.Pointer(InitStruct))' when the callback is done. > >> > > >> > > >> > 'InitStruct := (*C.Accumulatedparams)(C.calloc(1, > C.sizeof_struct_Accumulatedparams))' >>>> I want to store this address on > Go side. > >> > > >> > > >> > Can some please guide me here ? > >> > > >> > > >> > syslogparser.go > >> > ============= > >> > > >> > func (obj SyslogParser) LoadPatternDB(opts Syslog, workerId int) { > >> > patterndbpath := C.CString(opts.Patterndb) > >> > defer C.free(unsafe.Pointer(patterndbpath)) > >> > InitStruct := (*C.Accumulatedparams)(C.calloc(1, > C.sizeof_struct_Accumulatedparams)) > >> > //defer C.free(unsafe.Pointer(InitStruct)) . <<<<<<<<<<<<<<<<<<<<<<I > cannot do this here > >> > InitStruct.callback = (C.key_value_cb)(C.callback) > >> > InitStruct.data = C.int(workerId) > >> > C.load_pattern_db(patterndbpath, > (*C.Accumulatedparams)(unsafe.Pointer(InitStruct)), C.int(workerId)) > >> > } > >> > > >> > > >> > //export Result > >> > func Result(key *C.char, value *C.char, value_len C.size_t, mapid > C.int) { > >> > fmt.Println("Map Id: " + strconv.Itoa(int(mapid))) > >> > value_field := C.GoStringN(value, C.int(value_len)) > >> > key_field := C.GoString(key) > >> > remap, ok := constants.FIELD_MAPPINGS[key_field] > >> > if ok { > >> > Check.result[int(mapid)][remap] = value_field > >> > //check.result[remap] = value_field > >> > } else { > >> > //check.result[key_field] = value_field > >> > Check.result[int(mapid)][key_field] = value_field > >> > } > >> > } > >> > > >> > > >> > cfuncs.go > >> > -======== > >> > package lib > >> > > >> > /* > >> > > >> > #include <stdio.h> > >> > > >> > // The gateway function > >> > void callback(char *key, char *value, size_t value_len, int mapid) > >> > { > >> > //printf("C.callOnMeGo_cgo(): called"); > >> > void Result(const char *key, const char *value, size_t value_len, int > mapid); > >> > Result(key, value, value_len, mapid); > >> > } > >> > */ > >> > import "C" > >> > > >> > syslog-node.h > >> > ============ > >> > > >> > #ifndef TEST_H_INCLUDED > >> > #define TEST_H_INCLUDED > >> > > >> > #include <stdlib.h> > >> > > >> > typedef void (*key_value_cb)(const char* key, const char* value, > size_t value_len, int data); > >> > typedef struct Accumulatedparams{ > >> > key_value_cb callback; > >> > int data; > >> > }Accumulatedparams; > >> > int initialize_engine(const char* filename, const char* module_path); > >> > int reload_pattern_db(const char* filename, Accumulatedparams *cb, > int workerId); > >> > int load_pattern_db(const char* filename, Accumulatedparams *cb, int > workerId); > >> > > >> > #endif > >> > >> > >> I don't see anything obviously wrong in your Go code, but it's > >> impossible to answer your question without knowing what the C code > >> does. What you describe is consistent with C code that expects to > >> retain the memory passed to load_pattern_db, or alternatively is also > >> consistent with C code that does not expect to be called concurrently > >> from multiple threads of execution. > >> > >> Ian > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CALjMrq5g1uNN-gFzL_GJ7sEqeYyGvwtbRAAJ-c3%3DfRSGfzaNAw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.