Re: [Chicken-users] memory monitoring and leak debugging? (should the advice be in a web page?)

[Kristian Lein-Mathisen]

Hi Daniel,

Sorry for the late reply, I'm on vacation.



Q1: no, you are not calling back into Scheme (you'd need define-external or
something).
Q2: What it seems you are doing there is construct a compound object using
the Chicken C-api. I try to avoid doing that.

Constructing Scheme objects in C is easy to get wrong. Everything is
allocated on the stack so I think you have to use foreign-primitive to be
safe, not foreign-lambda (I think). This may be an easier way:

You can define-external a simple scheme procedure that takes the three ints
and returns list of them:

(define-external
  (list_int3 (int a) (int b) (int c))
scheme-object
   (list a b c))
;;C_word  list_int3(int t0,int t1,int t2)

And use this to produce your compound object. However, keep in mind that
the GC will move your objects around: Don't store lst anywhere in C between
Scheme callbacks because it will be invalidated after gc. Note that this
does violates the "don't use callbacks" advice!

Another trick which is useful to be aware of is using let-location. You can
allocate  objects in Scheme and pass them as pointers to your C procedures,
which in turn can mutate them and we can multiple return values from Scheme
side. I'm doing that here, for example:

https://github.com/kristianlm/chicken-stb-image/blob/79475a5d256ef3d8bd7e932b0d16c10e0bd9e1e3/stb-image.scm#L14

My advice for beginners would be:

## don't allocate compound scheme objects in C because it is hard
but allocating "C objects" is usually not that hard though. if a library
allocates a struct for you, go for it. and you can even use set-finalizer!
on it.

## avoid Scheme callbacks if you can because it makes things hard

## whenever you can, allocate in Scheme and mutate in C
because it is generally safe and fast. I'm doing that here, for example:
https://github.com/kristianlm/chicken-minissh/blob/8d5cccb9a561704aacf1464a8e3d0315499ec2c6/chacha20.scm#L10

## box C structs in define-record's even if you feel it is slow
so you don't start mixing up types

Safety is always harder and more important that it seems. I've had so many
cases where I have considered my bindings complete only to find I get the
occational segfault - typically caused by the GC moving object around. But
reproducing GC-related bugs can be hard.

Hope this helps a bit!
K.

On Wed, Aug 7, 2019, 23:10 Daniel Ortmann  wrote:

> I have Scheme calling the C yylex() and can easily get all of the token
> ints.
>
> I am trying to return '(tok,lineno) '(tok,lineno,intval) or
> '(tok,lineno,strval)
> C_word *ptr, *sptr;
> C_word lst;
> ...
> case TOK_G_R_SECT:
> printf("gettoken:(TOK_G_R_SECT,line,text) = (%2d,%3d,'%s')\n",
> tok, lineno, yylval.sptr);
> ptr = C_alloc(C_SIZEOF_LIST(3));
> lst = C_list(, 3, C_fix(tok), C_fix(lineno), C_fix(0));
> /*C_string2(, yylval.sptr));*/
> return callin(lst);
>
> Question 1:
> Does this stumbling attempt fall under that warning about not calling from
> C to Scheme?  Or is that warning for something else?
>
> Question 2:
> Do I need a bit better magic words to get that 'lst =' line working?  Or
> am I missing something more major?
>
> Thank you!
>
>
> On 8/7/19 3:00 PM, Kristian Lein-Mathisen wrote:
>
>
> Hi Daniel and welcome to the Chicken mailing list.
>
> Another thing to keep in mind is that the Chicken garbage-collector will
> move objects around during gc. That can cause a lot of hard-to-find bugs,
> and is probably one of the reasons for Joerg advice on not calling back
> into Scheme from your foreign-lambdas.
>
> Another cause of problems with the relocations is that you cannot
> reference Chicken objects in C long-term, because the gc won't be able to
> update those pointers.
>
> Having said that, the ffi in Chicken is really nice to use, so much that I
> often use it to explore new C api's.
>
> And also, I sometimes use valgrind to check for memory leaks.
>
> Best of luck!
> K.
>
> On Tue, Aug 6, 2019, 22:18 Jörg F. Wittenberger <
> joerg.wittenber...@softeyes.net> wrote:
>
>> Hello Daniel,
>>
>> welcome here.
>>
>> Since CHICKEN compiles into C, all the tools you are used with C to use
>> are still there.
>>
>> Personally I'm not a fan of fancy debuggers, since most of the things I
>> write tend to depend on external (network) events.  I'd welcome tips
>> how to automate those jobs using better tools than printing log
>> messages.
>>
>> Memory use in code mixing C and CHICKEN Scheme can be hairy.  I tend to
>> recommend to abstain from calling back from C into Scheme until you
>> know what you are doing.
>> http://wiki.call-cc.org/man/5/C%20interface#notes
>> 
>>
>> Otherwise I used to run my code under valgrind, 

Re: [Chicken-users] memory monitoring and leak debugging? (should the advice be in a web page?)

[felix . winkelmann]

> That is possible, but it is quite hard to use correctly in my opinion. It's
> also not in core anymore. If I understand things correctly, there is a
> performance penalty in every new gc root. Maybe some of the core team
> members can comment. To me, object-evict has been a last-resort (and as
> such, I've never used it) but I don't know how others might feel about it.
> 

"object-evict" moves an object into statically allocated memory, it does 
not create a GC root. There are issues with object-identity for symbols
and also the eviction recurses into all data contained in the object. It
is most useful for number arrays and data that contains only numeric 
or simple values.

But Kristian is quite right - it's a low-level tool for very specific 
situations. Using "foreign-lambda*" with callbacks into Scheme
is a cleaner way of handling allocation inside foreign code.


felix


___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] memory monitoring and leak debugging? (should the advice be in a web page?)

[Kristian Lein-Mathisen]

That is possible, but it is quite hard to use correctly in my opinion. It's
also not in core anymore. If I understand things correctly, there is a
performance penalty in every new gc root. Maybe some of the core team
members can comment. To me, object-evict has been a last-resort (and as
such, I've never used it) but I don't know how others might feel about it.

K.


On Wed, Aug 7, 2019, 22:52 Dan Leslie  wrote:

> Isn't it possible to pin items, and avoid these relocation and garbage
> collection issues, with object-evict?
>
> https://wiki.call-cc.org/eggref/5/object-evict
>
> -Dan
>
>
> Sent with ProtonMail  Secure Email.
>
> ‐‐‐ Original Message ‐‐‐
> On Wednesday, August 7, 2019 1:00 PM, Kristian Lein-Mathisen <
> kristianl...@gmail.com> wrote:
>
>
> Hi Daniel and welcome to the Chicken mailing list.
>
> Another thing to keep in mind is that the Chicken garbage-collector will
> move objects around during gc. That can cause a lot of hard-to-find bugs,
> and is probably one of the reasons for Joerg advice on not calling back
> into Scheme from your foreign-lambdas.
>
> Another cause of problems with the relocations is that you cannot
> reference Chicken objects in C long-term, because the gc won't be able to
> update those pointers.
>
> Having said that, the ffi in Chicken is really nice to use, so much that I
> often use it to explore new C api's.
>
> And also, I sometimes use valgrind to check for memory leaks.
>
> Best of luck!
> K.
>
> On Tue, Aug 6, 2019, 22:18 Jörg F. Wittenberger <
> joerg.wittenber...@softeyes.net> wrote:
>
>> Hello Daniel,
>>
>> welcome here.
>>
>> Since CHICKEN compiles into C, all the tools you are used with C to use
>> are still there.
>>
>> Personally I'm not a fan of fancy debuggers, since most of the things I
>> write tend to depend on external (network) events.  I'd welcome tips
>> how to automate those jobs using better tools than printing log
>> messages.
>>
>> Memory use in code mixing C and CHICKEN Scheme can be hairy.  I tend to
>> recommend to abstain from calling back from C into Scheme until you
>> know what you are doing.
>> http://wiki.call-cc.org/man/5/C%20interface#notes
>>
>> Otherwise I used to run my code under valgrind, which helped me a lot
>> to catch some errors.
>>
>> Best Regards
>>
>> /Jörg
>>
>> Am Tue, 6 Aug 2019 10:37:06 -0500
>> schrieb Daniel Ortmann :
>>
>> > Hello all,
>> > I am new to Chicken Scheme and experimenting with binding scheme to a
>> > C scanner built with Flex.  The results are fast but I feel the need
>> > to monitor memory use and watch for leaks.
>> >
>> > The only relevant thing I find on call-cc.org is this url:
>> > http://wiki.call-cc.org/chicken-for-emacs-lisp-programmers#tooling
>> >
>> > What are your experiences, tools, and practices with debugging mixed
>> > Scheme + C code?
>>
>>
>>
>> ___
>> Chicken-users mailing list
>> Chicken-users@nongnu.org
>> https://lists.nongnu.org/mailman/listinfo/chicken-users
>>
>
>
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] memory monitoring and leak debugging? (should the advice be in a web page?)

[Daniel Ortmann]

I have Scheme calling the C yylex() and can easily get all of the token
ints.

I am trying to return '(tok,lineno) '(tok,lineno,intval) or
'(tok,lineno,strval)
    C_word *ptr, *sptr;
    C_word lst;
...
    case TOK_G_R_SECT:
    printf("gettoken:(TOK_G_R_SECT,line,text) =
(%2d,%3d,'%s')\n", tok, lineno, yylval.sptr);
    ptr = C_alloc(C_SIZEOF_LIST(3));
    lst = C_list(, 3, C_fix(tok), C_fix(lineno), C_fix(0));
/*C_string2(, yylval.sptr));*/
    return callin(lst);

Question 1:
Does this stumbling attempt fall under that warning about not calling
from C to Scheme?  Or is that warning for something else?

Question 2:
Do I need a bit better magic words to get that 'lst =' line working?  Or
am I missing something more major?

Thank you!


On 8/7/19 3:00 PM, Kristian Lein-Mathisen wrote:
>
> Hi Daniel and welcome to the Chicken mailing list.
>
> Another thing to keep in mind is that the Chicken garbage-collector
> will move objects around during gc. That can cause a lot of
> hard-to-find bugs, and is probably one of the reasons for Joerg advice
> on not calling back into Scheme from your foreign-lambdas.
>
> Another cause of problems with the relocations is that you cannot
> reference Chicken objects in C long-term, because the gc won't be able
> to update those pointers.
>
> Having said that, the ffi in Chicken is really nice to use, so much
> that I often use it to explore new C api's.
>
> And also, I sometimes use valgrind to check for memory leaks.
>
> Best of luck!
> K.
>
> On Tue, Aug 6, 2019, 22:18 Jörg F. Wittenberger
>  > wrote:
>
> Hello Daniel,
>
> welcome here.
>
> Since CHICKEN compiles into C, all the tools you are used with C
> to use
> are still there.
>
> Personally I'm not a fan of fancy debuggers, since most of the
> things I
> write tend to depend on external (network) events.  I'd welcome tips
> how to automate those jobs using better tools than printing log
> messages.
>
> Memory use in code mixing C and CHICKEN Scheme can be hairy.  I
> tend to
> recommend to abstain from calling back from C into Scheme until you
> know what you are doing.
> http://wiki.call-cc.org/man/5/C%20interface#notes
> 
> 
>
> Otherwise I used to run my code under valgrind, which helped me a lot
> to catch some errors.
>
> Best Regards
>
> /Jörg
>
> Am Tue, 6 Aug 2019 10:37:06 -0500
> schrieb Daniel Ortmann  >:
>
> > Hello all,
> > I am new to Chicken Scheme and experimenting with binding scheme
> to a
> > C scanner built with Flex.  The results are fast but I feel the need
> > to monitor memory use and watch for leaks.
> >
> > The only relevant thing I find on call-cc.org
> 
> 
> is this url:
> >
> http://wiki.call-cc.org/chicken-for-emacs-lisp-programmers#tooling
> 
> 
> >
> > What are your experiences, tools, and practices with debugging mixed
> > Scheme + C code?
>
>
>
> ___
> Chicken-users mailing list
> Chicken-users@nongnu.org 
> https://lists.nongnu.org/mailman/listinfo/chicken-users
> 
> 
>
>
> ___
> Chicken-users mailing list
> Chicken-users@nongnu.org
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.nongnu.org_mailman_listinfo_chicken-2Dusers=DwIGaQ=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE=Dkq-B4wf3U4UeyggH34kSw8HWPy7CqI3u3RwPXz2f5o=uGR1RxlutvwNyGUJ8DkkNMO13p4pIcq2C8qZP_SlF2Q=zCUp8Z1mO63DzrUtaQUUKKKpB3z_x70a3OgFBS5Unn4=
>  

___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] memory monitoring and leak debugging? (should the advice be in a web page?)

[Dan Leslie]

Isn't it possible to pin items, and avoid these relocation and garbage 
collection issues, with object-evict?

https://wiki.call-cc.org/eggref/5/object-evict

-Dan

Sent with [ProtonMail](https://protonmail.com) Secure Email.

‐‐‐ Original Message ‐‐‐
On Wednesday, August 7, 2019 1:00 PM, Kristian Lein-Mathisen 
 wrote:

> Hi Daniel and welcome to the Chicken mailing list.
>
> Another thing to keep in mind is that the Chicken garbage-collector will move 
> objects around during gc. That can cause a lot of hard-to-find bugs, and is 
> probably one of the reasons for Joerg advice on not calling back into Scheme 
> from your foreign-lambdas.
>
> Another cause of problems with the relocations is that you cannot reference 
> Chicken objects in C long-term, because the gc won't be able to update those 
> pointers.
>
> Having said that, the ffi in Chicken is really nice to use, so much that I 
> often use it to explore new C api's.
>
> And also, I sometimes use valgrind to check for memory leaks.
>
> Best of luck!
> K.
>
> On Tue, Aug 6, 2019, 22:18 Jörg F. Wittenberger 
>  wrote:
>
>> Hello Daniel,
>>
>> welcome here.
>>
>> Since CHICKEN compiles into C, all the tools you are used with C to use
>> are still there.
>>
>> Personally I'm not a fan of fancy debuggers, since most of the things I
>> write tend to depend on external (network) events.  I'd welcome tips
>> how to automate those jobs using better tools than printing log
>> messages.
>>
>> Memory use in code mixing C and CHICKEN Scheme can be hairy.  I tend to
>> recommend to abstain from calling back from C into Scheme until you
>> know what you are doing.
>> http://wiki.call-cc.org/man/5/C%20interface#notes
>>
>> Otherwise I used to run my code under valgrind, which helped me a lot
>> to catch some errors.
>>
>> Best Regards
>>
>> /Jörg
>>
>> Am Tue, 6 Aug 2019 10:37:06 -0500
>> schrieb Daniel Ortmann :
>>
>>> Hello all,
>>> I am new to Chicken Scheme and experimenting with binding scheme to a
>>> C scanner built with Flex.  The results are fast but I feel the need
>>> to monitor memory use and watch for leaks.
>>>
>>> The only relevant thing I find on call-cc.org is this url:
>>> http://wiki.call-cc.org/chicken-for-emacs-lisp-programmers#tooling
>>>
>>> What are your experiences, tools, and practices with debugging mixed
>>> Scheme + C code?
>>
>> ___
>> Chicken-users mailing list
>> Chicken-users@nongnu.org
>> https://lists.nongnu.org/mailman/listinfo/chicken-users___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] memory monitoring and leak debugging? (should the advice be in a web page?)

[Kristian Lein-Mathisen]

Hi Daniel and welcome to the Chicken mailing list.

Another thing to keep in mind is that the Chicken garbage-collector will
move objects around during gc. That can cause a lot of hard-to-find bugs,
and is probably one of the reasons for Joerg advice on not calling back
into Scheme from your foreign-lambdas.

Another cause of problems with the relocations is that you cannot reference
Chicken objects in C long-term, because the gc won't be able to update
those pointers.

Having said that, the ffi in Chicken is really nice to use, so much that I
often use it to explore new C api's.

And also, I sometimes use valgrind to check for memory leaks.

Best of luck!
K.

On Tue, Aug 6, 2019, 22:18 Jörg F. Wittenberger <
joerg.wittenber...@softeyes.net> wrote:

> Hello Daniel,
>
> welcome here.
>
> Since CHICKEN compiles into C, all the tools you are used with C to use
> are still there.
>
> Personally I'm not a fan of fancy debuggers, since most of the things I
> write tend to depend on external (network) events.  I'd welcome tips
> how to automate those jobs using better tools than printing log
> messages.
>
> Memory use in code mixing C and CHICKEN Scheme can be hairy.  I tend to
> recommend to abstain from calling back from C into Scheme until you
> know what you are doing.
> http://wiki.call-cc.org/man/5/C%20interface#notes
>
> Otherwise I used to run my code under valgrind, which helped me a lot
> to catch some errors.
>
> Best Regards
>
> /Jörg
>
> Am Tue, 6 Aug 2019 10:37:06 -0500
> schrieb Daniel Ortmann :
>
> > Hello all,
> > I am new to Chicken Scheme and experimenting with binding scheme to a
> > C scanner built with Flex.  The results are fast but I feel the need
> > to monitor memory use and watch for leaks.
> >
> > The only relevant thing I find on call-cc.org is this url:
> > http://wiki.call-cc.org/chicken-for-emacs-lisp-programmers#tooling
> >
> > What are your experiences, tools, and practices with debugging mixed
> > Scheme + C code?
>
>
>
> ___
> Chicken-users mailing list
> Chicken-users@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/chicken-users
>
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] memory monitoring and leak debugging? (should the advice be in a web page?)

[Jörg F. Wittenberger]

Hello Daniel,

welcome here.

Since CHICKEN compiles into C, all the tools you are used with C to use
are still there.

Personally I'm not a fan of fancy debuggers, since most of the things I
write tend to depend on external (network) events.  I'd welcome tips
how to automate those jobs using better tools than printing log
messages.

Memory use in code mixing C and CHICKEN Scheme can be hairy.  I tend to
recommend to abstain from calling back from C into Scheme until you
know what you are doing.
http://wiki.call-cc.org/man/5/C%20interface#notes

Otherwise I used to run my code under valgrind, which helped me a lot
to catch some errors.

Best Regards

/Jörg

Am Tue, 6 Aug 2019 10:37:06 -0500
schrieb Daniel Ortmann :

> Hello all,
> I am new to Chicken Scheme and experimenting with binding scheme to a
> C scanner built with Flex.  The results are fast but I feel the need
> to monitor memory use and watch for leaks.
> 
> The only relevant thing I find on call-cc.org is this url:
> http://wiki.call-cc.org/chicken-for-emacs-lisp-programmers#tooling
> 
> What are your experiences, tools, and practices with debugging mixed
> Scheme + C code?



___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

[Chicken-users] memory monitoring and leak debugging? (should the advice be in a web page?)

[Daniel Ortmann]

Hello all,
I am new to Chicken Scheme and experimenting with binding scheme to a C
scanner built with Flex.  The results are fast but I feel the need to
monitor memory use and watch for leaks.

The only relevant thing I find on call-cc.org is this url:
http://wiki.call-cc.org/chicken-for-emacs-lisp-programmers#tooling

What are your experiences, tools, and practices with debugging mixed
Scheme + C code?

Thank you!


___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users