[go-nuts] Re: Starlight - run python inside Go to extend your applications - easily

[michal]

On Tuesday, December 11, 2018 at 3:35:37 AM UTC+1, Jason E. Aten wrote:
>
> For full, actual python inside Go, one could combine:
>
> (a) https://github.com/iodide-project/pyodide  has the python scientific 
> stack compiled to wasm (python + numpy + scipy + numplotlib)
>
> and
>
> (b) either https://github.com/go-interpreter/wagon or 
> https://github.com/perlin-network/life 
> :
>  
> each provide a wasm byte-code interpreter as a Go library.
>

Thanks for sharing that, that's some fresh thinking here.
 

>
>
> On Monday, December 10, 2018 at 2:39:09 AM UTC-6, mic...@scylladb.com 
> wrote:
>>
>>
>>
>> On Friday, December 7, 2018 at 9:05:02 PM UTC+1, Nate Finch wrote:
>>>
>>> I’d like to announce starlight - 
>>> https://github.com/starlight-go/starlight.
>>>
>>>
>>> Starlight wraps google’s Go implementation of the starlark python 
>>> dialect  (most notably found in 
>>> the Bazel build tool). Starlight makes it super easy for users to extend 
>>> your application by writing simple python scripts that interact seamlessly 
>>> with your current Go code… with no boilerplate on your part.
>>>
>>
>> Do you think it is suitable for porting python applications?
>> Usually you go through cgo like this 
>> https://www.datadoghq.com/blog/engineering/cgo-and-python/ it could be 
>> an interesting alternative. 
>>  
>>
>>>
>>> *Parser by google*
>>>
>>> The parser and runner are maintained by google’s bazel team, which write 
>>> starlark-go. Starlight is a wrapper on top of that, which makes it so much 
>>> easier to use starlark-go. The problem with the starlark-go API is that it 
>>> is more built to be a used as configuration, so it assumes you want to get 
>>> information out of starlark and into Go. It’s actually pretty difficult to 
>>> get Go information into a starlark script…. unless you use starlight.
>>>
>>> *Easy two-way interaction*
>>>
>>>
>>> Starlight has adapters that use reflection to automatically make any Go 
>>> value usable in a starlark script. Passing an *http.Request into a 
>>> starlark script? Sure, you can do name = r.URL.Query()["name"][0] in 
>>> the python without any work on your part.
>>>
>>> Starlight is built to *just work* the way you hope it’ll work. You can 
>>> access any Go methods or fields, basic types get converted back and forth 
>>> seamlessly… and even though it uses reflection, it’s not as slow as you’d 
>>> think. A basic benchmark wrapping a couple values and running a starlark 
>>> script to work with them runs in a tiny fraction of a millisecond.
>>>
>>> The great thing is that the changes made by the python code are 
>>> reflected in your go objects, just as if it had been written in Go. So, set 
>>> a field on a pointer to a struct? Your go code will see the change, no 
>>> additional work needed.
>>>
>>> *100% Safe*
>>>
>>>
>>> The great thing about starlark and starlight is that the scripts are 
>>> 100% safe to run. By default they have no access to other parts of your 
>>> project or system - they can’t write to disk or connect to the internet. 
>>> The only access they have to the outside is what you give them. Because of 
>>> this, it’s safe to run untrusted scripts (as long as you’re not giving them 
>>> dangerous functions to run, like os.RemoveAll). But at the same time, 
>>> if you’re only running trusted scripts, you can give them whatever you want 
>>> (http.Get? Sure, why not?)
>>>
>>> *Caching*
>>>
>>>
>>> In a production environment, you probably want to only read a script 
>>> once and parse it once. You can do that with starlight’s Cache. This 
>>> cache takes a list of directories to look in for scripts, which it will 
>>> read and parse on-demand, and then store the parsed object in memory for 
>>> later use. It also uses a cache for any load() calls the scripts use to 
>>> load scripts they depend on.
>>>
>>> *Work Ongoing*
>>>
>>>
>>> Starlight is still a work in progress, so don’t expect the API to be 
>>> perfectly stable quite yet. But it’s getting pretty close, and there 
>>> shouldn’t be any earth shattering changes, but definitely pin your imports. 
>>> Right now it’s more about finding corner cases where the starlight wrappers 
>>> don’t work quite like you’d expect, and supporting the last few things that 
>>> aren’t implemented yet (like channels).
>>>
>>>
>>> *Example*
>>>
>>>
>>> Here's a simple example of how easy it is to extend the behavior of your 
>>> application with a python script.  Just pass starlight whatever go values 
>>> you want your python script to act on, and any changes the python code 
>>> makes get reflected in your go code.  
>>>
>>>
>>> package main
>>>
>>> import (
>>> "fmt"
>>> "log"
>>> "time"
>>>
>>> "github.com/starlight-go/starlight"
>>> )
>>>
>>> // Starlight makes it easy to get values in and out of your starlark 
>>> scripts.

[go-nuts] Re: Starlight - run python inside Go to extend your applications - easily

[Jason E. Aten]

For full, actual python inside Go, one could combine:

(a) https://github.com/iodide-project/pyodide  has the python scientific 
stack compiled to wasm (python + numpy + scipy + numplotlib)

and

(b) either https://github.com/go-interpreter/wagon or 
https://github.com/perlin-network/life: each provide a wasm byte-code 
interpreter as a Go library.


On Monday, December 10, 2018 at 2:39:09 AM UTC-6, mic...@scylladb.com wrote:
>
>
>
> On Friday, December 7, 2018 at 9:05:02 PM UTC+1, Nate Finch wrote:
>>
>> I’d like to announce starlight - 
>> https://github.com/starlight-go/starlight.
>>
>>
>> Starlight wraps google’s Go implementation of the starlark python dialect 
>>  (most notably found in the Bazel 
>> build tool). Starlight makes it super easy for users to extend your 
>> application by writing simple python scripts that interact seamlessly with 
>> your current Go code… with no boilerplate on your part.
>>
>
> Do you think it is suitable for porting python applications?
> Usually you go through cgo like this 
> https://www.datadoghq.com/blog/engineering/cgo-and-python/ it could be an 
> interesting alternative. 
>  
>
>>
>> *Parser by google*
>>
>> The parser and runner are maintained by google’s bazel team, which write 
>> starlark-go. Starlight is a wrapper on top of that, which makes it so much 
>> easier to use starlark-go. The problem with the starlark-go API is that it 
>> is more built to be a used as configuration, so it assumes you want to get 
>> information out of starlark and into Go. It’s actually pretty difficult to 
>> get Go information into a starlark script…. unless you use starlight.
>>
>> *Easy two-way interaction*
>>
>>
>> Starlight has adapters that use reflection to automatically make any Go 
>> value usable in a starlark script. Passing an *http.Request into a 
>> starlark script? Sure, you can do name = r.URL.Query()["name"][0] in the 
>> python without any work on your part.
>>
>> Starlight is built to *just work* the way you hope it’ll work. You can 
>> access any Go methods or fields, basic types get converted back and forth 
>> seamlessly… and even though it uses reflection, it’s not as slow as you’d 
>> think. A basic benchmark wrapping a couple values and running a starlark 
>> script to work with them runs in a tiny fraction of a millisecond.
>>
>> The great thing is that the changes made by the python code are reflected 
>> in your go objects, just as if it had been written in Go. So, set a field 
>> on a pointer to a struct? Your go code will see the change, no additional 
>> work needed.
>>
>> *100% Safe*
>>
>>
>> The great thing about starlark and starlight is that the scripts are 100% 
>> safe to run. By default they have no access to other parts of your project 
>> or system - they can’t write to disk or connect to the internet. The only 
>> access they have to the outside is what you give them. Because of this, 
>> it’s safe to run untrusted scripts (as long as you’re not giving them 
>> dangerous functions to run, like os.RemoveAll). But at the same time, if 
>> you’re only running trusted scripts, you can give them whatever you want (
>> http.Get? Sure, why not?)
>>
>> *Caching*
>>
>>
>> In a production environment, you probably want to only read a script once 
>> and parse it once. You can do that with starlight’s Cache. This cache 
>> takes a list of directories to look in for scripts, which it will read and 
>> parse on-demand, and then store the parsed object in memory for later use. 
>> It also uses a cache for any load() calls the scripts use to load 
>> scripts they depend on.
>>
>> *Work Ongoing*
>>
>>
>> Starlight is still a work in progress, so don’t expect the API to be 
>> perfectly stable quite yet. But it’s getting pretty close, and there 
>> shouldn’t be any earth shattering changes, but definitely pin your imports. 
>> Right now it’s more about finding corner cases where the starlight wrappers 
>> don’t work quite like you’d expect, and supporting the last few things that 
>> aren’t implemented yet (like channels).
>>
>>
>> *Example*
>>
>>
>> Here's a simple example of how easy it is to extend the behavior of your 
>> application with a python script.  Just pass starlight whatever go values 
>> you want your python script to act on, and any changes the python code 
>> makes get reflected in your go code.  
>>
>>
>> package main
>>
>> import (
>> "fmt"
>> "log"
>> "time"
>>
>> "github.com/starlight-go/starlight"
>> )
>>
>> // Starlight makes it easy to get values in and out of your starlark 
>> scripts.
>> // Just pass in pointers to values that you want changed, or callback 
>> functions
>> // that propagate data.
>>
>> // In theory, starlight also returns all global variables set by the 
>> script, but
>> // in real programs, you need well-defined outputs for your calling code 
>> to act on.
>> // If I write a script that creates a variable called nate_is_awesome = 
>> 1337 ... your
>> // 

Re: [go-nuts] Re: Starlight - run python inside Go to extend your applications - easily

[Nate Finch]

Justin got this 100% correct.  Starlight and Starlark are not intended to 
be used to just run existing python code.  there's a ton they don't 
support, so pretty much any non-trivial python program would not be 
compatible.

It's really more of a way to give users the ability to write small to 
moderately sized scripts to customize behavior of your application without 
having to recompile and potentially without even having to restart the 
application.

On Monday, December 10, 2018 at 4:58:48 AM UTC-5, Justin Israel wrote:
>
>
>
> On Mon, Dec 10, 2018, 9:39 PM > wrote:
>
>>
>>
>> Do you think it is suitable for porting python applications?
>> Usually you go through cgo like this 
>> https://www.datadoghq.com/blog/engineering/cgo-and-python/ it could be 
>> an interesting alternative. 
>>  
>>
>
> The documentation for Starlark says it is a subset of python, originally 
> targeted at being a configuration language for Bazel. So it wouldn't be 
> complete enough to straight port full Python applications, unless you 
> really backed alot of it with Go code. The cgo approach gives you full 
> access to the CPython API and to embed an interpreter. 
>
> Starlight does sound interesting though for allowing specific extension 
> scripts for a Go application.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Starlight - run python inside Go to extend your applications - easily

[Raffaele Sena]

If you want a full python interpreter, you should look at
https://github.com/go-python/gpython, but gpython also doesn't have a full
standard library implemented.

-- Raffaele


On Mon, Dec 10, 2018 at 1:58 AM Justin Israel 
wrote:

>
>
> On Mon, Dec 10, 2018, 9:39 PM  wrote:
>
>>
>>
>> On Friday, December 7, 2018 at 9:05:02 PM UTC+1, Nate Finch wrote:
>>>
>>> I’d like to announce starlight -
>>> https://github.com/starlight-go/starlight.
>>>
>>>
>>> Starlight wraps google’s Go implementation of the starlark python
>>> dialect  (most notably found in
>>> the Bazel build tool). Starlight makes it super easy for users to extend
>>> your application by writing simple python scripts that interact seamlessly
>>> with your current Go code… with no boilerplate on your part.
>>>
>>
>> Do you think it is suitable for porting python applications?
>> Usually you go through cgo like this
>> https://www.datadoghq.com/blog/engineering/cgo-and-python/ it could be
>> an interesting alternative.
>>
>>
>
> The documentation for Starlark says it is a subset of python, originally
> targeted at being a configuration language for Bazel. So it wouldn't be
> complete enough to straight port full Python applications, unless you
> really backed alot of it with Go code. The cgo approach gives you full
> access to the CPython API and to embed an interpreter.
>
> Starlight does sound interesting though for allowing specific extension
> scripts for a Go application.
>
>
>>> *Parser by google*
>>>
>>> The parser and runner are maintained by google’s bazel team, which write
>>> starlark-go. Starlight is a wrapper on top of that, which makes it so much
>>> easier to use starlark-go. The problem with the starlark-go API is that it
>>> is more built to be a used as configuration, so it assumes you want to get
>>> information out of starlark and into Go. It’s actually pretty difficult to
>>> get Go information into a starlark script…. unless you use starlight.
>>>
>>> *Easy two-way interaction*
>>>
>>>
>>> Starlight has adapters that use reflection to automatically make any Go
>>> value usable in a starlark script. Passing an *http.Request into a
>>> starlark script? Sure, you can do name = r.URL.Query()["name"][0] in
>>> the python without any work on your part.
>>>
>>> Starlight is built to *just work* the way you hope it’ll work. You can
>>> access any Go methods or fields, basic types get converted back and forth
>>> seamlessly… and even though it uses reflection, it’s not as slow as you’d
>>> think. A basic benchmark wrapping a couple values and running a starlark
>>> script to work with them runs in a tiny fraction of a millisecond.
>>>
>>> The great thing is that the changes made by the python code are
>>> reflected in your go objects, just as if it had been written in Go. So, set
>>> a field on a pointer to a struct? Your go code will see the change, no
>>> additional work needed.
>>>
>>> *100% Safe*
>>>
>>>
>>> The great thing about starlark and starlight is that the scripts are
>>> 100% safe to run. By default they have no access to other parts of your
>>> project or system - they can’t write to disk or connect to the internet.
>>> The only access they have to the outside is what you give them. Because of
>>> this, it’s safe to run untrusted scripts (as long as you’re not giving them
>>> dangerous functions to run, like os.RemoveAll). But at the same time,
>>> if you’re only running trusted scripts, you can give them whatever you want
>>> (http.Get? Sure, why not?)
>>>
>>> *Caching*
>>>
>>>
>>> In a production environment, you probably want to only read a script
>>> once and parse it once. You can do that with starlight’s Cache. This
>>> cache takes a list of directories to look in for scripts, which it will
>>> read and parse on-demand, and then store the parsed object in memory for
>>> later use. It also uses a cache for any load() calls the scripts use to
>>> load scripts they depend on.
>>>
>>> *Work Ongoing*
>>>
>>>
>>> Starlight is still a work in progress, so don’t expect the API to be
>>> perfectly stable quite yet. But it’s getting pretty close, and there
>>> shouldn’t be any earth shattering changes, but definitely pin your imports.
>>> Right now it’s more about finding corner cases where the starlight wrappers
>>> don’t work quite like you’d expect, and supporting the last few things that
>>> aren’t implemented yet (like channels).
>>>
>>>
>>> *Example*
>>>
>>>
>>> Here's a simple example of how easy it is to extend the behavior of your
>>> application with a python script.  Just pass starlight whatever go values
>>> you want your python script to act on, and any changes the python code
>>> makes get reflected in your go code.
>>>
>>>
>>> package main
>>>
>>> import (
>>> "fmt"
>>> "log"
>>> "time"
>>>
>>> "github.com/starlight-go/starlight"
>>> )
>>>
>>> // Starlight makes it easy to get values in and out of your starlark
>>> scripts.
>>> // Just pass in pointers 

Re: [go-nuts] Re: Starlight - run python inside Go to extend your applications - easily

[Justin Israel]

On Mon, Dec 10, 2018, 9:39 PM  wrote:

>
>
> On Friday, December 7, 2018 at 9:05:02 PM UTC+1, Nate Finch wrote:
>>
>> I’d like to announce starlight -
>> https://github.com/starlight-go/starlight.
>>
>>
>> Starlight wraps google’s Go implementation of the starlark python dialect
>>  (most notably found in the Bazel
>> build tool). Starlight makes it super easy for users to extend your
>> application by writing simple python scripts that interact seamlessly with
>> your current Go code… with no boilerplate on your part.
>>
>
> Do you think it is suitable for porting python applications?
> Usually you go through cgo like this
> https://www.datadoghq.com/blog/engineering/cgo-and-python/ it could be an
> interesting alternative.
>
>

The documentation for Starlark says it is a subset of python, originally
targeted at being a configuration language for Bazel. So it wouldn't be
complete enough to straight port full Python applications, unless you
really backed alot of it with Go code. The cgo approach gives you full
access to the CPython API and to embed an interpreter.

Starlight does sound interesting though for allowing specific extension
scripts for a Go application.


>> *Parser by google*
>>
>> The parser and runner are maintained by google’s bazel team, which write
>> starlark-go. Starlight is a wrapper on top of that, which makes it so much
>> easier to use starlark-go. The problem with the starlark-go API is that it
>> is more built to be a used as configuration, so it assumes you want to get
>> information out of starlark and into Go. It’s actually pretty difficult to
>> get Go information into a starlark script…. unless you use starlight.
>>
>> *Easy two-way interaction*
>>
>>
>> Starlight has adapters that use reflection to automatically make any Go
>> value usable in a starlark script. Passing an *http.Request into a
>> starlark script? Sure, you can do name = r.URL.Query()["name"][0] in the
>> python without any work on your part.
>>
>> Starlight is built to *just work* the way you hope it’ll work. You can
>> access any Go methods or fields, basic types get converted back and forth
>> seamlessly… and even though it uses reflection, it’s not as slow as you’d
>> think. A basic benchmark wrapping a couple values and running a starlark
>> script to work with them runs in a tiny fraction of a millisecond.
>>
>> The great thing is that the changes made by the python code are reflected
>> in your go objects, just as if it had been written in Go. So, set a field
>> on a pointer to a struct? Your go code will see the change, no additional
>> work needed.
>>
>> *100% Safe*
>>
>>
>> The great thing about starlark and starlight is that the scripts are 100%
>> safe to run. By default they have no access to other parts of your project
>> or system - they can’t write to disk or connect to the internet. The only
>> access they have to the outside is what you give them. Because of this,
>> it’s safe to run untrusted scripts (as long as you’re not giving them
>> dangerous functions to run, like os.RemoveAll). But at the same time, if
>> you’re only running trusted scripts, you can give them whatever you want (
>> http.Get? Sure, why not?)
>>
>> *Caching*
>>
>>
>> In a production environment, you probably want to only read a script once
>> and parse it once. You can do that with starlight’s Cache. This cache
>> takes a list of directories to look in for scripts, which it will read and
>> parse on-demand, and then store the parsed object in memory for later use.
>> It also uses a cache for any load() calls the scripts use to load
>> scripts they depend on.
>>
>> *Work Ongoing*
>>
>>
>> Starlight is still a work in progress, so don’t expect the API to be
>> perfectly stable quite yet. But it’s getting pretty close, and there
>> shouldn’t be any earth shattering changes, but definitely pin your imports.
>> Right now it’s more about finding corner cases where the starlight wrappers
>> don’t work quite like you’d expect, and supporting the last few things that
>> aren’t implemented yet (like channels).
>>
>>
>> *Example*
>>
>>
>> Here's a simple example of how easy it is to extend the behavior of your
>> application with a python script.  Just pass starlight whatever go values
>> you want your python script to act on, and any changes the python code
>> makes get reflected in your go code.
>>
>>
>> package main
>>
>> import (
>> "fmt"
>> "log"
>> "time"
>>
>> "github.com/starlight-go/starlight"
>> )
>>
>> // Starlight makes it easy to get values in and out of your starlark
>> scripts.
>> // Just pass in pointers to values that you want changed, or callback
>> functions
>> // that propagate data.
>>
>> // In theory, starlight also returns all global variables set by the
>> script, but
>> // in real programs, you need well-defined outputs for your calling code
>> to act on.
>> // If I write a script that creates a variable called nate_is_awesome =
>> 1337 ... your
>> // 

[go-nuts] Re: Starlight - run python inside Go to extend your applications - easily

[michal]

On Friday, December 7, 2018 at 9:05:02 PM UTC+1, Nate Finch wrote:
>
> I’d like to announce starlight - https://github.com/starlight-go/starlight
> .
>
>
> Starlight wraps google’s Go implementation of the starlark python dialect 
>  (most notably found in the Bazel 
> build tool). Starlight makes it super easy for users to extend your 
> application by writing simple python scripts that interact seamlessly with 
> your current Go code… with no boilerplate on your part.
>

Do you think it is suitable for porting python applications?
Usually you go through cgo like 
this https://www.datadoghq.com/blog/engineering/cgo-and-python/ it could be 
an interesting alternative. 
 

>
> *Parser by google*
>
> The parser and runner are maintained by google’s bazel team, which write 
> starlark-go. Starlight is a wrapper on top of that, which makes it so much 
> easier to use starlark-go. The problem with the starlark-go API is that it 
> is more built to be a used as configuration, so it assumes you want to get 
> information out of starlark and into Go. It’s actually pretty difficult to 
> get Go information into a starlark script…. unless you use starlight.
>
> *Easy two-way interaction*
>
>
> Starlight has adapters that use reflection to automatically make any Go 
> value usable in a starlark script. Passing an *http.Request into a 
> starlark script? Sure, you can do name = r.URL.Query()["name"][0] in the 
> python without any work on your part.
>
> Starlight is built to *just work* the way you hope it’ll work. You can 
> access any Go methods or fields, basic types get converted back and forth 
> seamlessly… and even though it uses reflection, it’s not as slow as you’d 
> think. A basic benchmark wrapping a couple values and running a starlark 
> script to work with them runs in a tiny fraction of a millisecond.
>
> The great thing is that the changes made by the python code are reflected 
> in your go objects, just as if it had been written in Go. So, set a field 
> on a pointer to a struct? Your go code will see the change, no additional 
> work needed.
>
> *100% Safe*
>
>
> The great thing about starlark and starlight is that the scripts are 100% 
> safe to run. By default they have no access to other parts of your project 
> or system - they can’t write to disk or connect to the internet. The only 
> access they have to the outside is what you give them. Because of this, 
> it’s safe to run untrusted scripts (as long as you’re not giving them 
> dangerous functions to run, like os.RemoveAll). But at the same time, if 
> you’re only running trusted scripts, you can give them whatever you want (
> http.Get? Sure, why not?)
>
> *Caching*
>
>
> In a production environment, you probably want to only read a script once 
> and parse it once. You can do that with starlight’s Cache. This cache 
> takes a list of directories to look in for scripts, which it will read and 
> parse on-demand, and then store the parsed object in memory for later use. 
> It also uses a cache for any load() calls the scripts use to load scripts 
> they depend on.
>
> *Work Ongoing*
>
>
> Starlight is still a work in progress, so don’t expect the API to be 
> perfectly stable quite yet. But it’s getting pretty close, and there 
> shouldn’t be any earth shattering changes, but definitely pin your imports. 
> Right now it’s more about finding corner cases where the starlight wrappers 
> don’t work quite like you’d expect, and supporting the last few things that 
> aren’t implemented yet (like channels).
>
>
> *Example*
>
>
> Here's a simple example of how easy it is to extend the behavior of your 
> application with a python script.  Just pass starlight whatever go values 
> you want your python script to act on, and any changes the python code 
> makes get reflected in your go code.  
>
>
> package main
>
> import (
> "fmt"
> "log"
> "time"
>
> "github.com/starlight-go/starlight"
> )
>
> // Starlight makes it easy to get values in and out of your starlark 
> scripts.
> // Just pass in pointers to values that you want changed, or callback 
> functions
> // that propagate data.
>
> // In theory, starlight also returns all global variables set by the 
> script, but
> // in real programs, you need well-defined outputs for your calling code 
> to act on.
> // If I write a script that creates a variable called nate_is_awesome = 
> 1337 ... your
> // go code probably isn't going to care that the variable exists.
>
> // The best way to do it is to write a "results" struct that you pass in, 
> just
> // as you would for any other function.
>
> type Page struct {
> Name string
> Date time.Time
> Contents string
> IsDraft bool
> }
>
> const code = `
> def run():
> if "nate" in page.Name:
>  # capitalize words
>  page.Name = page.Name.title()
> page.Name += " " + page.Date.Format("2006/01/02")
> page.IsDraft = False
> run()
> `
>
> func main() {
> p := {
> Name: