Re: [racket-users] Does Racket offer anything for implementing a language with its own GC that could manage GPU memory?

[Gustavo Massaccesi]

> Also I forgot to note that I also want my language to have indentation
> sensitive syntax. Would that be doable in Racket?

It's possible (but I never tried it). In the official distribution there
are a few languages implemented in Racket that doesn't look like Racket at
all. For example https://docs.racket-lang.org/algol60/index.html and
https://docs.racket-lang.org/honu/index.html . I also remember a recent
talk by Mathew Flatt about how to implement a non standard language (with
princess and castles ...) but I can't find a link now.

Gustavo


On Sun, Feb 11, 2018 at 2:19 PM, Marko Grdinic  wrote:

> Also I forgot to note that I also want my language to have indentation
> sensitive syntax. Would that be doable in Racket?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to racket-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Does Racket offer anything for implementing a language with its own GC that could manage GPU memory?

[Matthias Felleisen]

> On Feb 11, 2018, at 12:18 PM, Marko Grdinic  wrote:
> 
> 
> Well, another problem I have is with the language's compiler which is not 
> necessarily slow, but the issue is that I have no idea how fast it is 
> relative to what could be. The .NET profiler is useless and keeps showing me 
> broken stacks due to all the tail recursion thereby making bottom up analysis 
> useless. I suppose you wouldn't know without experience at both, but does 
> Racket have good facilities for optimizing compile times such as during macro 
> expansion? If I could get some indication that I could make the compiler a 
> lot faster by rewriting it in Racket that would be strong motivator for me to 
> do so


The good news is that you can write any module-level static analysis that you 
want and then use the results during expansion. The not so good news is that we 
are only exploring this idea now in any depth and it’s all somewhat ad hoc. 

See Turnstile for how to do this for types. 

See Hackett for eliminating the top-tier of Turnstile and just exploiting the 
basic idea. 

— Matthias

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Does Racket offer anything for implementing a language with its own GC that could manage GPU memory?

[Marko Grdinic]

Also I forgot to note that I also want my language to have indentation 
sensitive syntax. Would that be doable in Racket?

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Does Racket offer anything for implementing a language with its own GC that could manage GPU memory?

[Marko Grdinic]

>
> What is the backend for Spiral -- a combination of the CPU and a GPU? 


Yeah. The main one is the F# one, but it is easy to write Cuda kernels as 
well. In fact, much easier than in C++ since they can capture values in 
lexical scope.

Here are 

 
all of them that I've written so far.  They look quite nice right? 
Everything in that module in fact works and feedforward nets are not a 
problem for the library. Spiral's tensors make indexing much simpler than 
in C++ and they are generic in the type, layout and the numbers of 
dimensions.

The two backends are like 500 lines of code each so it would not be a 
problem to add more should the need arise.

The standard way of implementing a Racket language X is to parse and 
> compile/expand a source file into an existing Racket language (which 
> eventually will be compiled to bytecode and then jitted). If one takes care 
> to keep source locations around, IDE integration will be automatic. 
>
 

This approach (the compiled source uses Racket primitives) will allocate 
> values on the standard heap and use the built-in garbage collector in 
> Racket. It is possible to use low-level primitives to allocate memory that 
> the Garbage collector won't touch though. See "Inside: Racket C API".


Would it be possible for Racket to save the results of a partial macro 
expansion, such as for an optimization pass to a file without me having to 
define a custom file format and serialization functions? That would save me 
a lot of work. Right now Spiral is slow due to having to parse and optimize 
the standard library on each iteration.

The reason why I am asking for an extensible GC is because GCs can do 
things like compact the memory and trigger when it is getting low.

The most I can get with .NET at the moment would be its safe handles. They 
act as wrappers around unmanaged resources, but my sense is that they won't 
be enough to let me manage GPU memory because they cannot do any of the 
above, at most they can act as safeguards. I also would not mind if the GPU 
GC tracked resources separately from the rest as I'd want to trigger it 
after every NN pass because that is the point where memory should be freed 
in bulk in most cases and I do not want it to go over the entire working 
set...though if it did I suppose it would not be a problem given how little 
heap allocation Spiral actually does at runtime.

Basically, I want GC, I want to tune it for my specific need and I want to 
tune it for Spiral's strengths. Spiral is not a dynamically typed language, 
but a statically typed one with intensional polymorphism and first class 
staging which might offer some interesting avenues of approaching the 
problem.

Well, another problem I have is with the language's compiler which is not 
necessarily slow, but the issue is that I have no idea how fast it is 
relative to what could be. The .NET profiler is useless and keeps showing 
me broken stacks due to all the tail recursion thereby making bottom up 
analysis useless. I suppose you wouldn't know without experience at both, 
but does Racket have good facilities for optimizing compile times such as 
during macro expansion? If I could get some indication that I could make 
the compiler a lot faster by rewriting it in Racket that would be strong 
motivator for me to do so.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Does Racket offer anything for implementing a language with its own GC that could manage GPU memory?

[Jens Axel Søgaard]

2018-02-11 14:38 GMT+01:00 Marko Grdinic :

> I first asked this on SO
> 
>  and
> got the advice to ask here on the mailing list. Compared to when I asked I
> do know how malloc is implemented now, but since it is a pile of hacks I am
> going to have to make the thing I had in mind previously. It is
> disappointing. I'd hoped it would be something I could not have come up on
> my own because nothing I've come up feels like a solution.
>
> Setting that aside, would it be possible to do what I asked in the title
> for a language like Spiral 
> ?
>
> Even if it were possible I am not saying I'd defect to right away - the
> kind of nets I want to make for my ML library are not so complex that they
> would require full GC, but I might consider it in the future when the
> demands they start placing on me grow bigger and there will be new chip
> architectures that would require me to break up with Cuda.
>

What is the backend for Spiral -- a combination of the CPU and a GPU?


> Probably I could implement GC for GPU memory on top of the language even
> now, but it would be complicated and slow, so I'd rather avoid it unless it
> were absolutely necessary.
>
> Also while GC is most important to me personally, maybe Racket could offer
> me other things like IDE integration for the language and the ability to
> compile the prepasses into a module without me having to make my own
> bytecode? I am guessing in Racket that would be something like a partial
> macro expansion?
>

The standard way of implementing a Racket language X is to parse and
compile/expand a source file into
an existing Racket language (which eventually will be compiled to bytecode
and then jitted).
If one takes care to keep source locations around, IDE integration will be
automatic.

This approach (the compiled source uses Racket primitives) will allocate
values on the standard heap and
use the built-in garbage collector in Racket. It is possible to use
low-level primitives to allocate
memory that the Garbage collector won't touch though. See "Inside: Racket C
API".

http://docs.racket-lang.org/inside/index.html

-- 
Jens Axel Søgaard

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[racket-users] Does Racket offer anything for implementing a language with its own GC that could manage GPU memory?

[Marko Grdinic]

I first asked this on SO 

 and 
got the advice to ask here on the mailing list. Compared to when I asked I 
do know how malloc is implemented now, but since it is a pile of hacks I am 
going to have to make the thing I had in mind previously. It is 
disappointing. I'd hoped it would be something I could not have come up on 
my own because nothing I've come up feels like a solution.

Setting that aside, would it be possible to do what I asked in the title 
for a language like Spiral ?

Even if it were possible I am not saying I'd defect to right away - the 
kind of nets I want to make for my ML library are not so complex that they 
would require full GC, but I might consider it in the future when the 
demands they start placing on me grow bigger and there will be new chip 
architectures that would require me to break up with Cuda.

Probably I could implement GC for GPU memory on top of the language even 
now, but it would be complicated and slow, so I'd rather avoid it unless it 
were absolutely necessary.

Also while GC is most important to me personally, maybe Racket could offer 
me other things like IDE integration for the language and the ability to 
compile the prepasses into a module without me having to make my own 
bytecode? I am guessing in Racket that would be something like a partial 
macro expansion?

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.