Re: Chicken git egg: bug & patch
Hi Megane, Like Evan says, the libgit2 project doesn't keep your pointers around so that should be safe. But thanks for pointing that out, errors caused by these things are very hard to track down. I'd actually never thought about a off-gc memory heap before. That is a very interesting idea! I'm sure it's overkill for the git egg, but I will definitely keep it in mind on my next memory management adventures. K. On Mon, Jun 14, 2021, 10:42 Evan Hanson wrote: > I think we can probably just use blobs and locatives to keep things > simple, without needing to involve finalizers or similar tricks. This > library is pretty good about leaving memory management up to the client, > and they provide an object for indirection (the git_buf) when that's not > possible. I'll have a look when I get a chunk of spare time. > > Thank you Kristian for digging into this, it is very helpful. > > Evan > >
Re: Chicken git egg: bug & patch
I think we can probably just use blobs and locatives to keep things simple, without needing to involve finalizers or similar tricks. This library is pretty good about leaving memory management up to the client, and they provide an object for indirection (the git_buf) when that's not possible. I'll have a look when I get a chunk of spare time. Thank you Kristian for digging into this, it is very helpful. Evan
Re: Chicken git egg: bug & patch
Kristian Lein-Mathisen writes: > Hi, > > Indeed. We could malloc and set-finalizer! and that should work and be > safe. However, as far as I understand, this approach has some disadvantages: > > - malloc is relatively slow compared to chicken's internal allocation (eg > make-string) that uses the stack > - there is a practical limit on finalizers that Chicken can handle > - large number of set-finalizer!s will slow things down > > I forgot to mention that I don't want to go for this approach in this case > because it will impact us heavily, since we allocate oid objects a lot. I > believe that is why Evan went for the alternative make-primitive method. So > I think we really _want_ to involve the Chicken GC, but we need to give it > some more info than just a #. > > I've used this make-blob approach that is in the attached patches before, > with success. Unfortunately, it doesn't integrate well with the foreigners > egg, but I hope it is still a good candidate here for the oid, buf and > revspec structures. > Hi, Yes, that's all true. I've only used the finalizer approach with 1000s of finalizers per second. At this scale the impact is negligible. If you're generating millions of tiny objects you could also consider using off heap object pools. If your objects live long enough to get into the CHICKEN heap they will have a GC impact. Also worth repeating: passing pointers to CHICKEN objects handled by the GC to foreign code is unsafe if said pointers are stored and accessed after the foreign code returns. GC may move the objects and make the pointers invalid. It's OK if you know the foreign library you're binding doesn't do this, which I assume is the case here.
Re: Chicken git egg: bug & patch
Hi, Indeed. We could malloc and set-finalizer! and that should work and be safe. However, as far as I understand, this approach has some disadvantages: - malloc is relatively slow compared to chicken's internal allocation (eg make-string) that uses the stack - there is a practical limit on finalizers that Chicken can handle - large number of set-finalizer!s will slow things down I forgot to mention that I don't want to go for this approach in this case because it will impact us heavily, since we allocate oid objects a lot. I believe that is why Evan went for the alternative make-primitive method. So I think we really _want_ to involve the Chicken GC, but we need to give it some more info than just a #. I've used this make-blob approach that is in the attached patches before, with success. Unfortunately, it doesn't integrate well with the foreigners egg, but I hope it is still a good candidate here for the oid, buf and revspec structures. K. On Mon, Jun 14, 2021, 07:10 megane wrote: > > Kristian Lein-Mathisen writes: > > > From what I gather, there is no way to allocate memory and return a > pointer > > to it, safely, in CHICKEN. > > Won't the garbage collector potentially overwrite whatever region was > > allocated since it has no way of knowing it? > > There's allocate from chicken.memory that does just that. It allocates > using 'malloc', and chicken GC will have no knowledge or way of > interfering with the memory. Free the memory using 'free' from the same > module. > >
