That's very helpful, thanks very much!
On Wednesday, September 28, 2016 at 10:33:04 PM UTC+1, Yichao Yu wrote: > > On Wed, Sep 28, 2016 at 5:16 PM, Christian Rorvik > <christia...@gmail.com <javascript:>> wrote: > > Thanks! I'll try this tomorrow. So similar to what you've done here? > > > https://github.com/yuyichao/FunctionWrappers.jl/blob/master/src/FunctionWrappers.jl#L40 > > > Correct. Also https://github.com/JuliaPy/PyCall.jl/pull/267 > > > > > I can't remember if I actually called that explicitly, but I believe > what I > > ended up with was similar to here: > > http://julialang.org/blog/2013/05/callback, where they use > > unsafe_pointer_to_objref to recover the function object. > > unsafe_pointer_to_objref is ok. It's unsafe in the sense that you > almost guarantee a segfault if it is not used correctly. > pointer_from_objref is less unsafe in that you might not get a > segfault if the GC is nice to you today. > > Ref https://github.com/JuliaLang/julia/issues/15857 for why > pointer_from_objref is bad if you call it on arbitrary object (mostly > immutables). Essentially the compiler is free to give you garbage if > you call it on immutables or the pointer you get back might not be the > pointer that you stored somewhere else since the pointer value is > supposed to be insignificant, only the content is, for immutables. > > > > > On Wednesday, September 28, 2016 at 10:06:51 PM UTC+1, Yichao Yu wrote: > >> > >> On Wed, Sep 28, 2016 at 5:00 PM, Christian Rorvik > >> <christia...@gmail.com> wrote: > >> > I don't have the code at hand right now (it's at work), but what I > was > >> > doing > >> > was something like > >> > > >> > type Wrapper{Arg} > >> > cfun::Ptr{Void} > >> > cobj::Ptr{Void} > >> > obj > >> > > >> > function Wrapper{T}(f::T) > >> > return new(cfunction(call_wrapper, Void, (Ref{T}, Arg)), > >> > pointer_from_objref(f), f) > >> > end > >> > end > >> > >> This is wrong, You shouldn't ever call `pointer_from_objref` for > >> unknown object. The right way to handl this is > >> > >> type Wrapper{Arg} > >> cfun::Ptr{Void} > >> cobj::Ptr{Void} > >> obj > >> function Wrapper{T}(f::T) > >> to_root = Base.cconvert(Ref{T}, f) > >> ptr = Base.unsafe_convert(Ref{T}, to_root) > >> return new(cfunction(call_wrapper, Void, (Ref{T}, Arg)), > >> Ptr{Void}(ptr), to_root) > >> end > >> end > >> > >> > >> > > >> > Where call_wrapper would invoke the closure. I had to change > >> > call_wrapper to > >> > receive the closure as Ptr{T} and call unsafe_pointer_to_objref and > then > >> > call it. > >> > > >> > > >> > On Wednesday, September 28, 2016 at 9:50:47 PM UTC+1, Yichao Yu > wrote: > >> >> > >> >> On Wed, Sep 28, 2016 at 4:36 PM, Christian Rorvik > >> >> <christia...@gmail.com> wrote: > >> >> > GC of code turned out to be a red herring. After isolating the > >> >> > instance > >> >> > of a > >> >> > call that was crashing, and getting right up to it, GDB was kind > >> >> > enough > >> >> > to > >> >> > get its knickers in a twist and segfault itself every time I > stepped > >> >> > instruction by instruction to find the precise point of failure > >> >> > (backtrace > >> >> > after segfault looked pretty meaningless). > >> >> > > >> >> > What it tunred out to be a was a Ref vs Ptr problem, where in > Julia I > >> >> > have a > >> >> > callback wrapper that receives a pointer to the closure. I was > taking > >> >> > this > >> >> > as Ref{T} and invoking it with a pointer from the C code. This > worked > >> >> > all > >> >> > fine and dandy, but somehow caused the object to be collected (I'm > >> >> > guessing). I can see how what I did was wrong, but I can't see how > it > >> >> > would > >> >> > lead to something being collected when there still exists roots > >> >> > pointing > >> >> > to > >> >> > it. If anything I would possibly expect a leak. Nonetheless, bit > of a > >> >> > wild > >> >> > goose chase. > >> >> > >> >> It's unclear what exactly you mean but just to be clear > `cfunction(f, > >> >> Ret, Tuple{Ref{Int}})` expect a pointer to a julia boxed `Int` that > is > >> >> rooted somewhere during the execution of the julia callback > function. > >> >> It is illegal to pass it a bare C `intptr_t*` (that doesn't point to > >> >> julia memory of a boxed Int) and it is illegal to generates it with > >> >> `ccall(..., (Ptr{Int},), &i)`. > >> >> > >> >> > > >> >> > > >> >> > On Wednesday, September 28, 2016 at 5:22:49 PM UTC+1, Christian > >> >> > Rorvik > >> >> > wrote: > >> >> >> > >> >> >> I'm not yet a 100% sure what is happening in my application, but > I'm > >> >> >> embedding Julia and using cfunction() to generate C-callable > >> >> >> functions > >> >> >> that > >> >> >> at a later time invoke the registered julia code. I use a fairly > >> >> >> standard > >> >> >> pattern I think, of passing a pointer to a julia object that is > >> >> >> retain > >> >> >> in C > >> >> >> and later passed down to the callback created by cfunction, and > from > >> >> >> there > >> >> >> resolved to the original real Julia reference type, invoking a > >> >> >> closure. > >> >> >> The > >> >> >> closure itself is retained in some global state to prevent > garbage > >> >> >> collection. What appears to be happening however, is the code at > the > >> >> >> landing > >> >> >> site for the cfunction returned pointer is at some point garbage > >> >> >> collection > >> >> >> (or at least corrupted), as my program, after a while of running, > >> >> >> will > >> >> >> segfault upon invoking the callback (after many previously > >> >> >> successful > >> >> >> callbacks). It segfaults because it hits invalid code, and it's > not > >> >> >> that > >> >> >> some state is missing when running the code. > >> >> >> > >> >> >> Is this to be be expected, and what's the right way to ensure the > >> >> >> code > >> >> >> isn't GC'd? >
