Re: [go-nuts] Some questions about GC and map[int]int

[Kurtis Rader]

You only need to ask for help once. You sent essentially the same message
twice. Also, your example program doesn't provide any time for the GC to
run in any meaningful manner as far as I can tell. So I am confused how
your trivial program relates to a real world program. That is, while simple
reproductions of a problem are always a good idea that does not seem to be
the case with your example reproduction. No real program would instantiate
a huge map then immediately terminate. I think you need to provide more
context and a more realistic program to reproduce the problem.

On Thu, Nov 23, 2023 at 10:10 PM 'Zhao Weng' via golang-nuts <
golang-nuts@googlegroups.com> wrote:

> Dear Gophers:
>
> I have some questions about GC and map[int]int, please help.
>
> consider the following program:
>
> ```go
> package main
>
> import (
> "fmt"
> "os"
> "runtime/pprof"
> )
>
> func main() {
> f, err := os.OpenFile("memory.pb.gz", os.O_RDWR|os.O_CREATE|os.O_TRUNC,
> 0666)
> if err != nil {
> panic(err)
> }
>
> m := make(map[int]int)
> for i := 0; i < 10e6; i++ {
> m[i] = 2 * i
> }
>
> if err := pprof.WriteHeapProfile(f); err != nil {
> panic(err)
> }
> }
> ```
>
> after inserting 10 million key/value pair, the result of this memory
> profile show inuse_objects count ~ 100k
>
> ```bash
> go tool pprof -inuse_objects memory.pb.gz
> Type: inuse_objects
> Time: Nov 23, 2023 at 3:40pm (CST)
> Entering interactive mode (type "help" for commands, "o" for options)
> (pprof) top
> Showing nodes accounting for 127450, 100% of 127450 total
>   flat  flat%   sum%cum   cum%
> 127450   100%   100% 127450   100%  main.main
>
> ```
>
> in my view this inuse_objects is found by GC by following the bucket and
> old_buckets pointer of hmap
>
> ```go
> // A header for a Go map.
> type hmap struct {
> // Note: the format of the hmap is also encoded in
> cmd/compile/internal/reflectdata/reflect.go.
> // Make sure this stays in sync with the compiler's definition.
> count int // # live cells == size of map. Must be first (used by len()
> builtin)
> flags uint8
> B uint8 // log_2 of # of buckets (can hold up to loadFactor * 2^B items)
> noverflow uint16 // approximate number of overflow buckets; see
> incrnoverflow for details
> hash0 uint32 // hash seed
>
> buckets unsafe.Pointer // array of 2^B Buckets. may be nil if count==0.
> oldbuckets unsafe.Pointer // previous bucket array of half the size,
> non-nil only when growing
> nevacuate uintptr // progress counter for evacuation (buckets less than
> this have been evacuated)
>
> extra *mapextra // optional fields
> }
> ```
>
> but according to this doc: https://go101.org/optimizations/6-map.html
>
> If the key type and element type of a map both don't contain pointers,
> then in the scan phase of a GC cycle, the garbage collector will not scan
> the entries of the map. This could save much time.
>
> My question is:
> maybe GC should not follow the bucket and old_buckets pointer of hmap?
> or the doc above just forbid GC from scan the entries not the bucket and
> old_buckets pointer of hmap, and larger map[int]int do increase GC scan
> overhead by having more buckets and old_buckets?
>
> Thanks
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/b7049ce2-d5a8-49bc-ae44-7a73777494f4n%40googlegroups.com
> 
> .
>


-- 
Kurtis Rader
Caretaker of the exceptional canines Junior and Hank

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CABx2%3DD96%3DSXs-3PfE_ayMRxJrKmuobUhA%3D6cizx_YROQ9T7DYA%40mail.gmail.com.

[go-nuts] Some questions about GC and map[int]int, please help

['Zhao Weng' via golang-nuts]

Dear Gophers:

I have some questions about GC and map[int]int, please help.

consider the following program:

```go
package main

import (
"fmt"
"os"
"runtime/pprof"
)

func main() {
f, err := os.OpenFile("memory.pb.gz", os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0666
)
if err != nil {
panic(err)
}

m := make(map[int]int)
for i := 0; i < 10e6; i++ {
m[i] = 2 * i
}

if err := pprof.WriteHeapProfile(f); err != nil {
panic(err)
}
}
```

after inserting 10 million key/value pair, the result of this memory 
profile show inuse_objects count ~ 100k

```bash
go tool pprof -inuse_objects memory.pb.gz
Type: inuse_objects
Time: Nov 23, 2023 at 3:40pm (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 127450, 100% of 127450 total
  flat  flat%   sum%cum   cum%
127450   100%   100% 127450   100%  main.main
 0 0%   100% 127450   100%  runtime.main
(pprof) 
```

in my view this inuse_objects is found by GC by following the bucket and 
old_buckets pointer of hmap

```go
// A header for a Go map.
type hmap struct {
// Note: the format of the hmap is also encoded in 
cmd/compile/internal/reflectdata/reflect.go.
// Make sure this stays in sync with the compiler's definition.
count int // # live cells == size of map. Must be first (used by len() 
builtin)
flags uint8
B uint8 // log_2 of # of buckets (can hold up to loadFactor * 2^B items)
noverflow uint16 // approximate number of overflow buckets; see 
incrnoverflow for details
hash0 uint32 // hash seed

buckets unsafe.Pointer // array of 2^B Buckets. may be nil if count==0.
oldbuckets unsafe.Pointer // previous bucket array of half the size, 
non-nil only when growing
nevacuate uintptr // progress counter for evacuation (buckets less than 
this have been evacuated)

extra *mapextra // optional fields
}
```

but according to this doc: https://go101.org/optimizations/6-map.html

If the key type and element type of a map both don't contain pointers, then 
in the scan phase of a GC cycle, the garbage collector will not scan the 
entries of the map. This could save much time.

My question is:
maybe GC should not follow the bucket and old_buckets pointer of hmap?
or the doc above just forbid GC from scanning the entries not the bucket 
and old_buckets pointer of hmap, and larger map[int]int do increase GC scan 
overhead by having more buckets and old_buckets?

Thanks

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/36fec7cb-6808-4a47-85ba-754963a6ddc5n%40googlegroups.com.

[go-nuts] Some questions about GC and map[int]int

['Zhao Weng' via golang-nuts]

Dear Gophers:

I have some questions about GC and map[int]int, please help.

consider the following program:

```go
package main

import (
"fmt"
"os"
"runtime/pprof"
)

func main() {
f, err := os.OpenFile("memory.pb.gz", os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0666
)
if err != nil {
panic(err)
}

m := make(map[int]int)
for i := 0; i < 10e6; i++ {
m[i] = 2 * i
}

if err := pprof.WriteHeapProfile(f); err != nil {
panic(err)
}
}
```

after inserting 10 million key/value pair, the result of this memory 
profile show inuse_objects count ~ 100k

```bash
go tool pprof -inuse_objects memory.pb.gz 
Type: inuse_objects
Time: Nov 23, 2023 at 3:40pm (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 127450, 100% of 127450 total
  flat  flat%   sum%cum   cum%
127450   100%   100% 127450   100%  main.main
 
```

in my view this inuse_objects is found by GC by following the bucket and 
old_buckets pointer of hmap

```go
// A header for a Go map.
type hmap struct {
// Note: the format of the hmap is also encoded in 
cmd/compile/internal/reflectdata/reflect.go.
// Make sure this stays in sync with the compiler's definition.
count int // # live cells == size of map. Must be first (used by len() 
builtin)
flags uint8
B uint8 // log_2 of # of buckets (can hold up to loadFactor * 2^B items)
noverflow uint16 // approximate number of overflow buckets; see 
incrnoverflow for details
hash0 uint32 // hash seed

buckets unsafe.Pointer // array of 2^B Buckets. may be nil if count==0.
oldbuckets unsafe.Pointer // previous bucket array of half the size, 
non-nil only when growing
nevacuate uintptr // progress counter for evacuation (buckets less than 
this have been evacuated)

extra *mapextra // optional fields
}
```

but according to this doc: https://go101.org/optimizations/6-map.html

If the key type and element type of a map both don't contain pointers, then 
in the scan phase of a GC cycle, the garbage collector will not scan the 
entries of the map. This could save much time.

My question is:
maybe GC should not follow the bucket and old_buckets pointer of hmap?
or the doc above just forbid GC from scan the entries not the bucket and 
old_buckets pointer of hmap, and larger map[int]int do increase GC scan 
overhead by having more buckets and old_buckets?

Thanks

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b7049ce2-d5a8-49bc-ae44-7a73777494f4n%40googlegroups.com.

[go-nuts] Some questions about GC and map[int]int

['Zhao Weng' via golang-nuts]

Dear Gophers:

I have some questions about GC and map[int]int, please help.

consider the following program:

```go
package main

import (
"fmt"
"os"
"runtime/pprof"
)

func main() {
f, err := os.OpenFile("memory.pb.gz", os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0666
)
if err != nil {
panic(err)
}

m := make(map[int]int)
for i := 0; i < 10e6; i++ {
m[i] = 2 * i
}

if err := pprof.WriteHeapProfile(f); err != nil {
panic(err)
}
}
```

after inserting 10 million key/value pair, the result of this memory 
profile show inuse_objects count ~ 100k
[image: inuse_objects.png]

in my view this inuse_objects is found by GC by following the bucket and 
old_buckets pointer of hmap

```go
// A header for a Go map.
type hmap struct {
// Note: the format of the hmap is also encoded in 
cmd/compile/internal/reflectdata/reflect.go.
// Make sure this stays in sync with the compiler's definition.
count int // # live cells == size of map. Must be first (used by len() 
builtin)
flags uint8
B uint8 // log_2 of # of buckets (can hold up to loadFactor * 2^B items)
noverflow uint16 // approximate number of overflow buckets; see 
incrnoverflow for details
hash0 uint32 // hash seed

buckets unsafe.Pointer // array of 2^B Buckets. may be nil if count==0.
oldbuckets unsafe.Pointer // previous bucket array of half the size, 
non-nil only when growing
nevacuate uintptr // progress counter for evacuation (buckets less than 
this have been evacuated)

extra *mapextra // optional fields
}
```

but according to this doc: https://go101.org/optimizations/6-map.html

If the key type and element type of a map both don't contain pointers, then 
in the scan phase of a GC cycle, the garbage collector will not scan the 
entries of the map. This could save much time.

My question is:
maybe GC should not follow the bucket and old_buckets pointer of hmap?
or the doc above just forbid GC from scan the entries not the bucket and 
old_buckets pointer of hmap, and larger map[int]int do increase GC scan 
overhead by having more buckets and old_buckets?

Thanks

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/db9507eb-fd64-46c0-867a-f4eaf5c78b38n%40googlegroups.com.

[go-nuts] Some question about GC and map[int]int

['Zhao Weng' via golang-nuts]

consider the following program:

```go
package main

import (
"fmt"
"os"
"runtime/pprof"
)

func main() {
f, err := os.OpenFile("memory.pb.gz", os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0666
)
if err != nil {
panic(err)
}

m := make(map[int]int)
for i := 0; i < 10e6; i++ {
m[i] = 2 * i
}

if err := pprof.WriteHeapProfile(f); err != nil {
panic(err)
}
}
```

after inserting 10 million key/value pair, the result of this memory 
profile show inuse_objects count ~ 100k[image: inuse_objects.png]

in my view this inuse_objects is found by GC by following the bucket and 
old_buckets pointer of hmap

```go
// A header for a Go map.
type hmap struct {
// Note: the format of the hmap is also encoded in 
cmd/compile/internal/reflectdata/reflect.go.
// Make sure this stays in sync with the compiler's definition.
count int // # live cells == size of map. Must be first (used by len() 
builtin)
flags uint8
B uint8 // log_2 of # of buckets (can hold up to loadFactor * 2^B items)
noverflow uint16 // approximate number of overflow buckets; see 
incrnoverflow for details
hash0 uint32 // hash seed

buckets unsafe.Pointer // array of 2^B Buckets. may be nil if count==0.
oldbuckets unsafe.Pointer // previous bucket array of half the size, 
non-nil only when growing
nevacuate uintptr // progress counter for evacuation (buckets less than 
this have been evacuated)

extra *mapextra // optional fields
}
```

but according to this doc: https://go101.org/optimizations/6-map.html 


If the key type and element type of a map both don't contain pointers, then 
in the scan phase of a GC cycle, the garbage collector will not scan the 
entries of the map. This could save much time.

This tip is also valid for other kinds of container in Go, such as slices, 
arrays and channels.

My question is:
maybe GC should not follow the bucket and old_buckets pointer of hmap?
or the doc above just forbid GC from scan the entries not the bucket and 
old_buckets pointer of hmap, and larger map[int]int do increase GC scan 
overhead by having more buckets and old_buckets?

Thanks

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/32427592-36e4-4262-8904-e2d68b14f7e6n%40googlegroups.com.

[go-nuts] Changing PWD ruins test caching

[Kevin Burke]

We have some tests that are pretty slow. I would like to use Go's test 
caching to skip them in our CI environment, if nothing in the code or the 
environment has changed. Our CI environment has a set of _agents_, each of 
which can run multiple jobs simultaneously. As a result, they check out 
code into a different directory each time they run a job.

Go's test caching checks every env var loaded by the test program. If any 
of them change, then Go assumes that the cache is busted and the test must 
be run.

Any program that imports "os" hits this logic in the os package, which 
checks the value of the $PWD environment variable:

// We query the working directory at init, to use it later to search for the
// executable file
// errWd will be checked later, if we need to use initWd
var initWd, errWd = Getwd()

So checking code out to different directories means that any program that 
imports "os" cannot have test caching. This seems like a shame because the 
code is all laid out in the same place in the working directory.

Has anyone tried to fix this issue? Do you think this is worth trying to 
patch or modify the test caching behavior in Go itself? I could solve this 
by running a chroot or another layer of Docker, of course, but I'd prefer 
not to do these because of the difficulty of getting data in and out of 
each one, communicating with other Docker containers, etc.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/93132985-d1cc-4bdc-abfb-6fb519d62c54n%40googlegroups.com.

Re: [go-nuts] Go/Python interop

[Sebastien Binet]

On Thu Nov 23, 2023 at 15:23 CET, 'Michael Knyszek' via golang-nuts wrote:
> Also, I'd like to clarify:
>
> - [David]: is it a good idea to use cgo for Go-Python interop?
> - [Michael]: no. better with pipe or RPC
>
> I'm wrong about this. A conversation after the meeting clarified a
> misunderstanding I had about Go->Python calls specifically. Both
> Go->Python
> and Python->Go with cgo may very well be preferable to pipe/RPC in many
> circumstances. :)

I think that, at least for the ML/AI use cases (where the API is python-based), 
tackling this via the cgo dance will be cumbersome:

to access the data or call the methods/functions, one will need to go
through the C.PyXYZ calls (even though most of the ML/AI modules are 
implemented in C for performances)

this means a world of C.PyIncref/Decref, C-callbacks, ...
(having done something along these lines with go-python/gopy (a set of tools to 
automatically expose a Go package as a CPython module), I speak from "some" 
experience)

so, I'd tend to agree with your previous self :)
what did change your mind ?

-s

> On Thursday, November 23, 2023 at 3:40:35 AM UTC-5 Sebastien Binet
> wrote:
>
> > On Thu Nov 23, 2023 at 01:24 CET, Eli Bendersky wrote:
> > > On Wed, Nov 22, 2023 at 11:31 AM Sebastien Binet
> > > 
> > > wrote:
> > >
> > > > Hi there,
> > > >
> > > > In this week "compiler minutes" [1], one can read:
> > > >
> > > > """
> > > > - Go on future platforms (RAM efficiency. NUMA?)
> > > > - (maybe) Go-Python interop for AI-powered applications
> > > > - [David]: is it a good idea to use cgo for Go-Python interop?
> > > > - [Michael]: no. better with pipe or RPC
> > > > """
> > > >
> > > > Would it be possible to have a bit more informations ?
> > > > What kind of interop is it ? Exchanging binary data ? On disk ?
> > > > Establishing a protobuf-based-like standard ?
> > > > Something else ?
> > > >
> > >
> > > All of it, maybe :-)
> > > We're just exploring the issue, throwing ideas around. There are many
> > > potential options, each with its own tradeoffs in terms of performance
> > > vs. effort.
> >
> > depending on the timescale, one could also imagine having a Go-based 
> > python VM (e.g. github.com/go-python/gpython) that can run a limited set 
> > of python modules (even the C-based ones, like pypy did at some point ?).
> > alternatively, "just" rely on pickle-based, Apache Arrow-based or numpy 
> > array-based exchanged data.
> >
> > for Arrow and numpy, there are already packages that do offer a fair 
> > amount of interop:
> >
> > - https://github.com/apache/arrow/tree/main/go
> > - https://github.com/sbinet/npyio/ (shameless plug.)
> >
> > (we could imagine also adding some buffer protocol implementation à la 
> > PEP-3118)
> >
> > for pickle, gpython has support up to protocol=3, and gopickle seems to 
> > have support for up to 5:
> > - https://github.com/nlpodyssey/gopickle
> > (adding support for array.array to gopickle was relatively straightforward)
> >
> > -s
> >
>
> --
> You received this message because you are subscribed to the Google
> Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/cf461932-52bb-4ac1-9690-053bec7e432an%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CX6CITBAJ3LS.2HFXLFQCV58PQ%40cern.ch.

Re: [go-nuts] a simple linux audio player pkg?

['Mark' via golang-nuts]

I just converted the []float32 to []byte (see function below) and it works.
But the sound produced while recognizable is very staticy and I don't know 
why.

func floats32ToBytes(fs []float32) []byte {
var buf bytes.Buffer
for _, f := range fs {
if err := binary.Write(&buf, binary.LittleEndian, f); err != nil {
panic(err)
}
}
return buf.Bytes()
}

On Thursday, November 23, 2023 at 10:03:24 AM UTC Mark wrote:

> I've now tried using those libraries, but there seems to be an 
> incompatibility []float32 vs []byte.
> Here's the error:
>
> ./play.go:24:29: cannot use reader (variable of type *oggvorbis.Reader) as 
> io.Reader value in argument to otoCtx.NewPlayer: *oggvorbis.Reader does not 
> implement io.Reader (wrong type for method Read)
> have Read([]float32) (int, error)
> want Read([]byte) (int, error)
>
> And here's the code (main.go):
>
> package main
>
> import (
> "log"
> "os"
> "time"
>
> //"github.com/ebitengine/oto/v3"
> "github.com/hajimehoshi/oto/v2"
> "github.com/jfreymuth/oggvorbis"
> )
>
> func main() {
> log.SetFlags(0)
> file, err := os.Open(os.Args[0])
> checkErr(err)
> defer file.Close()
> reader, err := oggvorbis.NewReader(file)
> checkErr(err)
> otoCtx, readyCh, err := oto.NewContext(reader.SampleRate(),
> reader.Channels(), 2)
> checkErr(err)
> <-readyCh // wait for h/w
> player := otoCtx.NewPlayer(reader)
> defer player.Close()
> player.Play()
> for player.IsPlaying() {
> time.Sleep(time.Millisecond * 10)
> }
> }
>
> func checkErr(err error) {
> if err != nil {
> log.Fatal(err)
> }
> }
>
> On Wednesday, November 22, 2023 at 10:31:25 PM UTC Raffaele Sena wrote:
>
>> I have used github.com/jfreymuth/oggvorbis to read the ogg file (and 
>> convert to PCM) and github.com/ebitengine/oto/v3 to play the PCM.
>> I don't know of a full ogg player in Go
>>
>>
>> On Wed, Nov 22, 2023 at 2:02 PM 'Mark' via golang-nuts <
>> golan...@googlegroups.com> wrote:
>>
>>> Is there a simple vorbis/oga audio player package for Go that works on 
>>> Linux.
>>> The only API I need is something like this:
>>>
>>> player.SetFilename(string) error // string is say tune.ogg; stops 
>>> playing previous if any
>>> player.Play(float32) error // plays current filename from given second 
>>> e.g. 0.0 for the beginning
>>> player.Pause() (float32, error) // pauses current playing and returns 
>>> position
>>> player.Resume() error // resumes playing
>>> player.Secs() float32 // returns current position
>>>
>>> The ones I've seen on awsome go either don't seem to play ogg format or 
>>> are far more sophisticated than I need.
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to golang-nuts...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/a38e3a9b-a357-4c85-aa4d-a7a4322ec216n%40googlegroups.com
>>>  
>>> 
>>> .
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/4bbdeb78-6ac8-49c3-b71f-a270bb090a38n%40googlegroups.com.

Re: [go-nuts] Go/Python interop

['Michael Knyszek' via golang-nuts]

Also, I'd like to clarify:

- [David]: is it a good idea to use cgo for Go-Python interop?
- [Michael]: no. better with pipe or RPC

I'm wrong about this. A conversation after the meeting clarified a 
misunderstanding I had about Go->Python calls specifically. Both Go->Python 
and Python->Go with cgo may very well be preferable to pipe/RPC in many 
circumstances. :)
On Thursday, November 23, 2023 at 3:40:35 AM UTC-5 Sebastien Binet wrote:

> On Thu Nov 23, 2023 at 01:24 CET, Eli Bendersky wrote:
> > On Wed, Nov 22, 2023 at 11:31 AM Sebastien Binet
> > 
> > wrote:
> >
> > > Hi there,
> > >
> > > In this week "compiler minutes" [1], one can read:
> > >
> > > """
> > > - Go on future platforms (RAM efficiency. NUMA?)
> > > - (maybe) Go-Python interop for AI-powered applications
> > > - [David]: is it a good idea to use cgo for Go-Python interop?
> > > - [Michael]: no. better with pipe or RPC
> > > """
> > >
> > > Would it be possible to have a bit more informations ?
> > > What kind of interop is it ? Exchanging binary data ? On disk ?
> > > Establishing a protobuf-based-like standard ?
> > > Something else ?
> > >
> >
> > All of it, maybe :-)
> > We're just exploring the issue, throwing ideas around. There are many
> > potential options, each with its own tradeoffs in terms of performance
> > vs. effort.
>
> depending on the timescale, one could also imagine having a Go-based 
> python VM (e.g. github.com/go-python/gpython) that can run a limited set 
> of python modules (even the C-based ones, like pypy did at some point ?).
> alternatively, "just" rely on pickle-based, Apache Arrow-based or numpy 
> array-based exchanged data.
>
> for Arrow and numpy, there are already packages that do offer a fair 
> amount of interop:
>
> - https://github.com/apache/arrow/tree/main/go
> - https://github.com/sbinet/npyio/ (shameless plug.)
>
> (we could imagine also adding some buffer protocol implementation à la 
> PEP-3118)
>
> for pickle, gpython has support up to protocol=3, and gopickle seems to 
> have support for up to 5:
> - https://github.com/nlpodyssey/gopickle
> (adding support for array.array to gopickle was relatively straightforward)
>
> -s
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/cf461932-52bb-4ac1-9690-053bec7e432an%40googlegroups.com.

Re: [go-nuts] a simple linux audio player pkg?

['Mark' via golang-nuts]

I've now tried using those libraries, but there seems to be an 
incompatibility []float32 vs []byte.
Here's the error:

./play.go:24:29: cannot use reader (variable of type *oggvorbis.Reader) as 
io.Reader value in argument to otoCtx.NewPlayer: *oggvorbis.Reader does not 
implement io.Reader (wrong type for method Read)
have Read([]float32) (int, error)
want Read([]byte) (int, error)

And here's the code (main.go):

package main

import (
"log"
"os"
"time"

//"github.com/ebitengine/oto/v3"
"github.com/hajimehoshi/oto/v2"
"github.com/jfreymuth/oggvorbis"
)

func main() {
log.SetFlags(0)
file, err := os.Open(os.Args[0])
checkErr(err)
defer file.Close()
reader, err := oggvorbis.NewReader(file)
checkErr(err)
otoCtx, readyCh, err := oto.NewContext(reader.SampleRate(),
reader.Channels(), 2)
checkErr(err)
<-readyCh // wait for h/w
player := otoCtx.NewPlayer(reader)
defer player.Close()
player.Play()
for player.IsPlaying() {
time.Sleep(time.Millisecond * 10)
}
}

func checkErr(err error) {
if err != nil {
log.Fatal(err)
}
}

On Wednesday, November 22, 2023 at 10:31:25 PM UTC Raffaele Sena wrote:

> I have used github.com/jfreymuth/oggvorbis to read the ogg file (and 
> convert to PCM) and github.com/ebitengine/oto/v3 to play the PCM.
> I don't know of a full ogg player in Go
>
>
> On Wed, Nov 22, 2023 at 2:02 PM 'Mark' via golang-nuts <
> golan...@googlegroups.com> wrote:
>
>> Is there a simple vorbis/oga audio player package for Go that works on 
>> Linux.
>> The only API I need is something like this:
>>
>> player.SetFilename(string) error // string is say tune.ogg; stops playing 
>> previous if any
>> player.Play(float32) error // plays current filename from given second 
>> e.g. 0.0 for the beginning
>> player.Pause() (float32, error) // pauses current playing and returns 
>> position
>> player.Resume() error // resumes playing
>> player.Secs() float32 // returns current position
>>
>> The ones I've seen on awsome go either don't seem to play ogg format or 
>> are far more sophisticated than I need.
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/a38e3a9b-a357-4c85-aa4d-a7a4322ec216n%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b719492d-bead-4391-8371-2a6489f9cf46n%40googlegroups.com.

Re: [go-nuts] Go/Python interop

[Sebastien Binet]

On Thu Nov 23, 2023 at 01:24 CET, Eli Bendersky wrote:
> On Wed, Nov 22, 2023 at 11:31 AM Sebastien Binet
> 
> wrote:
>
> > Hi there,
> >
> > In this week "compiler minutes" [1], one can read:
> >
> > """
> > - Go on future platforms (RAM efficiency. NUMA?)
> > - (maybe) Go-Python interop for AI-powered applications
> > - [David]: is it a good idea to use cgo for Go-Python interop?
> > - [Michael]: no. better with pipe or RPC
> > """
> >
> > Would it be possible to have a bit more informations ?
> > What kind of interop is it ? Exchanging binary data ? On disk ?
> > Establishing a protobuf-based-like standard ?
> > Something else ?
> >
>
> All of it, maybe :-)
> We're just exploring the issue, throwing ideas around. There are many
> potential options, each with its own tradeoffs in terms of performance
> vs. effort.

depending on the timescale, one could also imagine having a Go-based python VM 
(e.g. github.com/go-python/gpython) that can run a limited set of python 
modules (even the C-based ones, like pypy did at some point ?).
alternatively, "just" rely on pickle-based, Apache Arrow-based or numpy 
array-based exchanged data.

for Arrow and numpy, there are already packages that do offer a fair amount of 
interop:

- https://github.com/apache/arrow/tree/main/go
- https://github.com/sbinet/npyio/ (shameless plug.)

(we could imagine also adding some buffer protocol implementation à la PEP-3118)

for pickle, gpython has support up to protocol=3, and gopickle seems to have 
support for up to 5:
- https://github.com/nlpodyssey/gopickle
(adding support for array.array to gopickle was relatively straightforward)

-s

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CX621G9YBBIW.2PJ6K1VMFR9SR%40cern.ch.

Re: [go-nuts] a simple linux audio player pkg?

['Mark' via golang-nuts]

Thank you, I'll try them. (Well, for oto I'll try 
https://pkg.go.dev/github.com/hajimehoshi/oto/v2 since that seems to have 
more importers)

On Wednesday, November 22, 2023 at 10:31:25 PM UTC Raffaele Sena wrote:

> I have used github.com/jfreymuth/oggvorbis to read the ogg file (and 
> convert to PCM) and github.com/ebitengine/oto/v3 to play the PCM.
> I don't know of a full ogg player in Go
>
>
> On Wed, Nov 22, 2023 at 2:02 PM 'Mark' via golang-nuts <
> golan...@googlegroups.com> wrote:
>
>> Is there a simple vorbis/oga audio player package for Go that works on 
>> Linux.
>> The only API I need is something like this:
>>
>> player.SetFilename(string) error // string is say tune.ogg; stops playing 
>> previous if any
>> player.Play(float32) error // plays current filename from given second 
>> e.g. 0.0 for the beginning
>> player.Pause() (float32, error) // pauses current playing and returns 
>> position
>> player.Resume() error // resumes playing
>> player.Secs() float32 // returns current position
>>
>> The ones I've seen on awsome go either don't seem to play ogg format or 
>> are far more sophisticated than I need.
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/a38e3a9b-a357-4c85-aa4d-a7a4322ec216n%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/85e7b5c4-7224-4f79-accd-ef1b4f095425n%40googlegroups.com.