Re: [go-nuts] Re: Optional dependencies in module

[Arnaud Delobelle]

On Tue, 8 Feb 2022 at 20:55, Manlio Perillo  wrote:

> Note that a single repository can have multiple modules, so you can make 
> cmd/golua-repls a separate module, adding github.com/arnodel/golua as a 
> dependency.
> See https://go.dev/ref/mod#vcs-version.

That is good to know!  My question was relayed on Gophers slack
(https://gophers.slack.com/archives/C0VPK4Z5E/p1644352629693789) and
there was some good advice.  What I took from it was that the best
approach is probably a single module in a single repo with a clear
message in the README explaining that the "core" packages have no
external dependencies so that people browsing the module have less
chance of being misled by the contents of go.mod

Cheers,

-- 
Arnaud

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAJ6cK1bk%2B3AQh3AMP3o9vpy%3DCvUvPYYS%3DqfKGAGnSukujcfXCQ%40mail.gmail.com.

[go-nuts] Optional dependencies in module

[Arnaud Delobelle]

Hi all,

Here is my issue: I have a Go module at github.com/arnodel/golua which
implements a Lua compiler / runtime.  It has no external dependencies
(strictly, there is a dependency on another small module of mine
implementing strftime) and that is easily verifiable by looking at the
go.mod file.  As a typical usage of Lua is to embed it into another
program, it is important to me that it is clear that there are no
dependencies brought in when doing that.

On the other had, I want to add a REPL command which will have a
dependency on a third party terminal management package
(https://github.com/gdamore/tcell) because it would be silly of me to
reimplement that!  This REPL is completely optional. The simplest
solution I see is to add a `cmd/golua-repl` package to my module
implementing the REPL.  Why it seems the right solution:

+ It is nice because installing the REPL at a given version of the
module will always use the Lua VM for the same version.

+ No need to have two repositories, two modules and keep dependencies
up to date.  That would create a confusing situation for me and for
users.

+ Because of dependency pruning, I believe that users who e.g. want to
embed the Lua runtime into their program will not download the extra
REPL dependencies, because the runtime package doesn't actually import
them (as long as they use recent Go).

OTOH

- The go.mod file of my golua module will now show the extra
dependency and its dependencies.  This will make it much less evident
to users that those dependencies are not applicable to the golua
runtime packages.

To me this feels like a big negative point and it makes me very
hesitant to take this approach.  So I would like to ask for some
wisdom on this problem.
- Am I overstating the problem?
- Are there good mitigations that people know work?
- Is there another simple approach that would keep go.mod of my module clean?

As stated above, I don't think two repositories with a golua module
and a golua-repl module are very attractive solutions, and it seems
that having two modules in one repository is not a very good approach
either.  And these are the only other approaches I can think of.

Thanks in advance,

Arnaud Delobelle

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAJ6cK1Z%3DyD_dBLYueH8d%3DudGkiTpSz29LesDqc5-K8a9ws2e-Q%40mail.gmail.com.

Re: [go-nuts] Re: slices grow at 25% after 1024 but why 1024?

[Arnaud Delobelle]

Nice!

Arnaud

On Tue, 7 Sep 2021, 18:02 Keith Randall,  wrote:

> Sounds good. CL up for review at
> https://go-review.googlesource.com/c/go/+/347917
>
> On Mon, Sep 6, 2021 at 7:30 PM Arnaud Delobelle  wrote:
>
>> If the growing function is currently
>>
>> f(x) = 2x for x < 1024
>> f(x) = x + x/4 otherwise
>>
>> (Which I haven't checked), couldn't a simple way be to use e.g.
>>
>> f(x) = 2xfor x < 1024
>> f(x) = x + x/4 + 768 otherwise
>>
>> Then
>>
>> f(1023) = 2046
>> f(1024) = 2048
>>
>> So the function is monotonic (and kind of "smooth")
>>
>> On Tue, 7 Sep 2021, 02:17 'Keith Randall' via golang-nuts, <
>> golang-nuts@googlegroups.com> wrote:
>>
>>> I don't think this is an important thing to fix, but I agree it is a bit
>>> odd. If there's a simple way to restore monotonicity we'll consider it.
>>> A similar issue: https://github.com/golang/go/issues/41239
>>>
>>> On Sunday, September 5, 2021 at 8:59:01 AM UTC-7 jake...@gmail.com
>>> wrote:
>>>
>>>> You are 100% correct. I missed that value. oops
>>>>
>>>> On Sunday, September 5, 2021 at 10:16:08 AM UTC-4 arn...@gmail.com
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Sun, 5 Sep 2021, 14:59 jake...@gmail.com, 
>>>>> wrote:
>>>>> [...]
>>>>>
>>>>>> In the example given  (https://play.golang.org/p/RJbEkmFsPKM
>>>>>> <https://play.golang.org/p/RJbEkmFsPKM>), the capacities *are 
>>>>>> *"monotonically
>>>>>> increasing", as no number in the second column is smaller than the one
>>>>>> before it.
>>>>>>
>>>>>
>>>>> Nitpick: that is not true.  I copy-pasted the output of the playground
>>>>> below.
>>>>>
>>>>> 0 8
>>>>> 100 208
>>>>> 200 416
>>>>> 300 640
>>>>> 400 896
>>>>> 500 1024
>>>>> 600 1280
>>>>> 700 1408
>>>>> 800 1792
>>>>> 900 2048
>>>>> 1000 2048
>>>>> 1100 1408 <-- at this point the new cap is less than for the previous
>>>>> row
>>>>> 1200 1536
>>>>> 1300 1792
>>>>> 1400 1792
>>>>> 1500 2048
>>>>> 1600 2048
>>>>> 1700 2304
>>>>> 1800 2304
>>>>> 1900 2688
>>>>>
>>>>> Regards,
>>>>>
>>>>> Arnaud
>>>>>
>>>>> On Sunday, September 5, 2021 at 7:02:43 AM UTC-4 kortschak wrote:
>>>>>>
>>>>>>> On Sun, 2021-09-05 at 03:51 -0700, Brian Candler wrote:
>>>>>>> > I'm not sure you're clear about what "monotonically increasing"
>>>>>>> > means.
>>>>>>> >
>>>>>>> > Are you saying that there are some cases where append() results in
>>>>>>> > the allocated size of a slice *shrinking*? If so, please
>>>>>>> > demonstrate.
>>>>>>>
>>>>>>> I think he means that the cap of the appended slice is not a
>>>>>>> monotonically increasing function of the cap of the input slice.
>>>>>>>
>>>>>>> https://play.golang.org/p/RJbEkmFsPKM
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "golang-nuts" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to golang-nuts...@googlegroups.com.
>>>>>>
>>>>> To view this discussion on the web visit
>>>>>> https://groups.google.com/d/msgid/golang-nuts/912453d5-2f2f-43b2-b65f-ce27e95752e9n%40googlegroups.com
>>>>>> <https://groups.google.com/d/msgid/golang-nuts/912453d5-2f2f-43b2-b65f-ce27e95752e9n%40googlegroups.com?utm_medium=email_source=footer>
>>>>>> .
>>>>>>
>>>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/fa360f3d-e23c-4c07-9505-9f89bd155bb8n%40googlegroups.com
>>> <https://groups.google.com/d/msgid/golang-nuts/fa360f3d-e23c-4c07-9505-9f89bd155bb8n%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAJ6cK1aAonAjO7rd8-wnnVeLAFX4HixFUk7s3hJnEqtwRaz4uA%40mail.gmail.com.

Re: [go-nuts] Re: slices grow at 25% after 1024 but why 1024?

[Arnaud Delobelle]

If the growing function is currently

f(x) = 2x for x < 1024
f(x) = x + x/4 otherwise

(Which I haven't checked), couldn't a simple way be to use e.g.

f(x) = 2xfor x < 1024
f(x) = x + x/4 + 768 otherwise

Then

f(1023) = 2046
f(1024) = 2048

So the function is monotonic (and kind of "smooth")

On Tue, 7 Sep 2021, 02:17 'Keith Randall' via golang-nuts, <
golang-nuts@googlegroups.com> wrote:

> I don't think this is an important thing to fix, but I agree it is a bit
> odd. If there's a simple way to restore monotonicity we'll consider it.
> A similar issue: https://github.com/golang/go/issues/41239
>
> On Sunday, September 5, 2021 at 8:59:01 AM UTC-7 jake...@gmail.com wrote:
>
>> You are 100% correct. I missed that value. oops
>>
>> On Sunday, September 5, 2021 at 10:16:08 AM UTC-4 arn...@gmail.com wrote:
>>
>>>
>>>
>>> On Sun, 5 Sep 2021, 14:59 jake...@gmail.com,  wrote:
>>> [...]
>>>
 In the example given  (https://play.golang.org/p/RJbEkmFsPKM
 ), the capacities *are 
 *"monotonically
 increasing", as no number in the second column is smaller than the one
 before it.

>>>
>>> Nitpick: that is not true.  I copy-pasted the output of the playground
>>> below.
>>>
>>> 0 8
>>> 100 208
>>> 200 416
>>> 300 640
>>> 400 896
>>> 500 1024
>>> 600 1280
>>> 700 1408
>>> 800 1792
>>> 900 2048
>>> 1000 2048
>>> 1100 1408 <-- at this point the new cap is less than for the previous row
>>> 1200 1536
>>> 1300 1792
>>> 1400 1792
>>> 1500 2048
>>> 1600 2048
>>> 1700 2304
>>> 1800 2304
>>> 1900 2688
>>>
>>> Regards,
>>>
>>> Arnaud
>>>
>>> On Sunday, September 5, 2021 at 7:02:43 AM UTC-4 kortschak wrote:

> On Sun, 2021-09-05 at 03:51 -0700, Brian Candler wrote:
> > I'm not sure you're clear about what "monotonically increasing"
> > means.
> >
> > Are you saying that there are some cases where append() results in
> > the allocated size of a slice *shrinking*? If so, please
> > demonstrate.
>
> I think he means that the cap of the appended slice is not a
> monotonically increasing function of the cap of the input slice.
>
> https://play.golang.org/p/RJbEkmFsPKM
>
>
> --
 You received this message because you are subscribed to the Google
 Groups "golang-nuts" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to golang-nuts...@googlegroups.com.

>>> To view this discussion on the web visit
 https://groups.google.com/d/msgid/golang-nuts/912453d5-2f2f-43b2-b65f-ce27e95752e9n%40googlegroups.com
 
 .

>>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/fa360f3d-e23c-4c07-9505-9f89bd155bb8n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAJ6cK1Zk2UcUz37Bcf-z_7wM8%3D3jKJpQvv2d5nE-vsTVVRjLAQ%40mail.gmail.com.

Re: [go-nuts] Re: slices grow at 25% after 1024 but why 1024?

[Arnaud Delobelle]

On Sun, 5 Sep 2021, 14:59 jake...@gmail.com,  wrote:
[...]

> In the example given  (https://play.golang.org/p/RJbEkmFsPKM
> ), the capacities *are *"monotonically
> increasing", as no number in the second column is smaller than the one
> before it.
>

Nitpick: that is not true.  I copy-pasted the output of the playground
below.

0 8
100 208
200 416
300 640
400 896
500 1024
600 1280
700 1408
800 1792
900 2048
1000 2048
1100 1408 <-- at this point the new cap is less than for the previous row
1200 1536
1300 1792
1400 1792
1500 2048
1600 2048
1700 2304
1800 2304
1900 2688

Regards,

Arnaud

On Sunday, September 5, 2021 at 7:02:43 AM UTC-4 kortschak wrote:
>
>> On Sun, 2021-09-05 at 03:51 -0700, Brian Candler wrote:
>> > I'm not sure you're clear about what "monotonically increasing"
>> > means.
>> >
>> > Are you saying that there are some cases where append() results in
>> > the allocated size of a slice *shrinking*? If so, please
>> > demonstrate.
>>
>> I think he means that the cap of the appended slice is not a
>> monotonically increasing function of the cap of the input slice.
>>
>> https://play.golang.org/p/RJbEkmFsPKM
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/912453d5-2f2f-43b2-b65f-ce27e95752e9n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAJ6cK1a38cQ%3D55GCW113ygm3bV29FL1VZYD9Jh9mQfQ8C1PUmw%40mail.gmail.com.

Re: [go-nuts] Re: WASM Performance

[Arnaud Delobelle]

Hi Stephen,

I haven't really looked at your code in detail but one obvious
difference between your version and the original is the rendering
code.  Are you certain that the slowness is not confined to your
rendering code (I have no reason to believe it is btw)?

Here is a suggestion.  I have had a good experience using a 2d library
called ebiten (https://github.com/hajimehoshi/ebiten).  It abstracts
the rendering and can target native or browsers via wasm.  In my
experience performance when targeting the browser has been acceptable,
so you could write an implementation of television.PixelRenderer
backed by ebiten.  You could then compile both to native and wasm and
see if there is still a big performance difference?

HTH

Arnaud

On Sat, 4 Sept 2021 at 11:12, Stephen Illingworth
 wrote:
>
> I don't think I'm going to be able to make any progress with this. I chopped 
> away enough code so that it compiles with TinyGo. It works but it's even 
> slower.
>
> I was hoping to find a way of profiling the WASM code but I see DWARF support 
> for WASM binaries is still a work in progress. 
> https://github.com/golang/go/issues/33503 I don't know enough about WASM to 
> be able to contribute to that issue unfortunately.
>
> Thanks to anyone who looked at this.
> On Friday, 3 September 2021 at 11:20:26 UTC+1 stephen.t@gmail.com wrote:
>>
>> To follow up on this I should clarify what my questions are:
>>
>> 1) How much of a performance drop (when compared to AMD64 for example) 
>> should I expect when compiling to the WASM target?
>>
>> 2) Is there anything obvious I can do to counter any performance drops?
>>
>> And I suppose this is a non-Go question, but:
>>
>> 3) I know nothing about WASM beyond the bare minimum. How can I profile and 
>> understand the compiled WASM binary? Is it possible to use the pprof tool in 
>> some way?
>>
>>
>> On Fri, Sep 3, 2021 at 10:40 AM Stephen Illingworth 
>>  wrote:
>>>
>>>
>>>
>>>
>>> On Fri, Sep 3, 2021 at 10:15 AM Brian Candler  wrote:

 Could you explain a bit more about what you're comparing?

 - Is the wasm version running in a browser? If so, which one? Or have you 
 got a way to run wasm directly on the host (in which case, what is it)?
>>>
>>>
>>> Running it in Firefox (78.13.0esr) and Chromium (92.0.4515.159)
>>>

 - How is the linux/amd64 version running, if it's not talking to a 
 DOM-type environment?  If the native version is still using syscall/js, 
 then how is it doing so?  Or is the native version in a different repo?

 - By "the parent emulator project" do you just mean web2600 itself?
>>>
>>>
>>> Web2600 is using the emulation core of the parent project
>>>
>>> https://github.com/JetSetIlly/Gopher2600
>>>
>>> The parent project runs on the desktop. It currently uses SDL and OpenGL 
>>> etc. but it is designed to allow different methods of presentation.
>>>
>>> Web2600 is using the core of the emulator (ie. the non-presentation parts) 
>>> and so doesn't use SDL or OpenGL.
>>>
>>> For presentation, the television package in the core emulation allows you 
>>> to register "PixelRenderers". So Web2600 adds itself as a pixel renderer. 
>>> The implemented SetPixels(), NewFrame() (etc.) functions will then talk to 
>>> the DOM as appropriate.
>>>
>>> The web version works but is just exceedingly slow by comparison.
>>>
>>>
> --
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/6bb486eb-7481-4356-94cd-29c365c02416n%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAJ6cK1abWOrcv%3DWx-foEeDEqBCZro%2BEYgse_j4uwshBxyG2XJA%40mail.gmail.com.

Re: [go-nuts] Contrary to popular opinion...

[Arnaud Delobelle]

E.g. compare

a = 1
b = 2

And

a = 1 b = 2


They do no mean the same in Go.

Arnaud
On Sun, 28 Feb 2021, 08:05 'Dan Kortschak' via golang-nuts, <
golang-nuts@googlegroups.com> wrote:

> On Sun, 2021-02-28 at 08:40 +0100, Jan Mercl wrote:
> > Actually Go has that problem as well, just a thousand times smaller.
>
> I'm curious where the meaningful whitespace is in Go (for amounts
> differences in number greater than 1).
>
> > FTR, I also think Python's approach to white space is a failure. But
> > its popularity seems to prove people think otherwise.
>
> The problem with python's approach is that it operated on the basis
> that error correcting redundancy in source code is not valuable. It is.
>
> Dan
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/baa6db5212f6c2d71e6d011faec073a558b02adb.camel%40kortschak.io
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAJ6cK1YWwGX0SZcJJMmoHWwLCWufhxmQiB%2B%3D%3D2LcuB9J%2BtOJ4g%40mail.gmail.com.

Re: [go-nuts] type checking custom interface{} is not working as expected

[Arnaud Delobelle]

You need you Json type to be a concrete type, e.g. (if the underlying type
is a string)

type Json string

Then users can try to assert that what they get is of concrete type Json

Cheers,

Arnaud

On Tue, 16 Feb 2021, 08:20 Santhosh Kumar T, 
wrote:

> I am not using either json.Marshaler or json.Unmarshaller
>
> my project implements mysql binlog replication protocol
>
> i have a method:   fn nextRow() []interafce{}
> which returns values in row as slice.
> the row might contain VARCHAR and JSON type columns in mysql table
> definition
> if actual json value for a row is string, then nextRow() return a slice
> containing two strings
> to distinguish i am trying to returns []interface{}{"value1",
> Json("value2")}
> but this seems does not solve the issue, since user of my library cannot
> check if it is json value or not
>
> thanks
> Santhosh
>
> On Tuesday, February 16, 2021 at 1:32:07 PM UTC+5:30
> axel.wa...@googlemail.com wrote:
>
>> An interface type-assertion asserts that the dynamic value in the
>> interface (in this case it's `string`) implements the interface you are
>> asserting (in this case `myinterface`). As `myinterface` has no methods,
>> every value implements that interface.
>>
>> In general, a type-assertion will assert things about the dynamic value
>> stored in an interface. In that code, the dynamic value is `string` *either
>> way*. In one case, you assign a `string` to `interface{}`, in the other you
>> assign a `string` to `myinterface`. But both will ultimately store a
>> `string`. That's also why your first case, using `mystring` works as
>> expected - in that case, the dynamic type is `string` in one case and
>> `mystring` in the other (note that the dynamic type is *always* a "concrete
>> type", not an interface).
>>
>> In general, ways to get around this is to either a) not use an interface,
>> or b) give that interface a method only specific types implement, or c)
>> store a `*myinterface` - it's a pointer, not an interface, thus a concrete
>> type, which can be distinguished. However, in your case, neither of these
>> strategies would likely be fruitful. The `json` package needs to know how
>> to store the values it decodes and it won't be able to, if it doesn't have
>> a concrete type or a `json.Unmarshaler`.
>>
>> So in your specific case, you might want to consider using something like
>>
>> type struct {
>> s string
>> n int64
>> isString bool
>> }
>>
>> and have that implement `json.Marshaler` and `json.Unmarshaler`.
>>
>> On Tue, Feb 16, 2021 at 8:47 AM Santhosh Kumar T 
>> wrote:
>>
>>> I have a function call nextRow which returns []interface{}
>>>
>>> i want to add support for json, since json value can be string, number
>>> which conflicts
>>> with native types, i used following:
>>> type Json interface{}
>>> and returning json values as:
>>> return Json(v)
>>> but in type checking it fails.
>>>
>>> i extract minimal code to explain my issue:
>>>https://play.golang.org/p/AAbeOzH-SHE
>>>
>>> in the above example, it prints: v3 is myinterface
>>> but v3 is regular interface{} not of type myinterface
>>>
>>> could someone help in resolving the issue.
>>> Is this expected behavior. what can i do to workaround if so
>>>
>>> thanks
>>> Santhosh
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/bcaa91b2-38ab-4713-93c6-34dd55792f4dn%40googlegroups.com
>>> 
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/41a88e9f-8301-4717-b07c-b067c687ad1cn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAJ6cK1Y7-SYpJYp_Bpswz9Z1MD5pFdzvbvwRxnC14wg5boLfog%40mail.gmail.com.

Re: [go-nuts] Error handling

[Arnaud Delobelle]

I do sometimes do something similar, but without the check() function.
The exit points are explicit, it is guaranteed that errors will be
wrapped.

func do(name string) (err error) {
defer PWrapf(, "do(%s)", name)

s, err := works(name);
if err != nil {
return err
}

// ...
}

// PWrapf applies Wrapf to an error pointer
func PWrapf(err *error, format string, args ...interface{}) {
*err = errors.Wrapf(*err, format, args...)
}

On Sun, 14 Feb 2021 at 01:45, Michael MacInnis
 wrote:
>
> I've been playing around with reducing error handling boilerplate using 
> standard language constructs.
>
> I'm currently doing something that looks like this:
>
> import (
> "github.com/michaelmacinnis/handle"
> )
>
> func do(name string) (err error) {
> check, handle := handle.Errorf(, "do(%s)", name); defer handle()
>
> s, err := works(name); check(err)
>
> // ...
> }
>
> Other than the named return value and check being a hidden return, are there 
> reasons I would want to avoid doing this? I assume others have tried similar 
> things but I haven't stumbled across any similar packages.
>
> Before using it in production I would probably want a linter that checks to 
> make sure that the statement after handle.Error or handle.Errorf is defer 
> blah, where blah is the name given to the second value returned by those 
> functions and that all of this happens at the start of a function.
>
> Michael.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/cbe53d07-4f99-49fa-a708-dcb85b1aff5bn%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAJ6cK1ZgvpGV_vUFwHc%2BbadJXcoJO%3Dwv4-tfaWTYqjQbM%2BOMMg%40mail.gmail.com.

Re: [go-nuts] Possible Go 2 proposal for built-in Remove method for Slices.

[Arnaud Delobelle]

On Sat, 6 Feb 2021 at 11:25, Martin Schnabel  wrote:
>
>
>
> On 06.02.21 03:32, Wojciech S. Czarnecki wrote:
> > Dnia 2021-02-04, o godz. 00:30:57
> > Selahaddin Harmankaya  napisał(a):
> >
> >> There are obviously more things to consider
> >
> > Slice is not an array, it is a _view_into_ an array. Many such views
> > into the same array may exist simultaneously.
> >
> > A "remove element from _slice_" operation always must make a new array
> > and copy all elements that you intend to stay. Current idiom using append
> > will do this for you while being frank about costs.
>
> Maybe I misunderstand what you wrote, but as long as the slice has
> enough capacity the array will be reused and not not copied:
>
> https://play.golang.org/p/sevpWixzw6V
>

Yes, given this and e.g. https://play.golang.org/p/Dyh9Dzd-1zq , in my
opinion it's hard to argue against having slice manipulation
primitives with clearer semantics.  Having generics will allow this
though!

-- 
Arnaud

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAJ6cK1aP2XEoGFjE3j6huy9gsszQPQGk0Aa0kCZGPL2AO8z8_A%40mail.gmail.com.

Re: [go-nuts] Re: Quick question about the generics alternatives that have been considered

[Arnaud Delobelle]

Hello

FWIW I had a go at imagining a way to make a variant of your idea
(package-level parametrisation) work.  At the time my principal worry
was contracts and I was keen to use interfaces instead to express
parametric constraints:
https://arnodel.github.io/marooned/go/generics/2020/06/06/go-spec2.html

I posted it on golang-nuts a while ago.  Here is a link to Ian Lance
Taylor's reply on this list pointing out some shortcomings:
https://groups.google.com/g/golang-nuts/c/Plq8DkScDW8/m/ig5NuDpXBgAJ

Although I don't think my proposal was very good, I also think it was
an attempt to limit the power of generics in the language.  I have a
feeling that in order to avoid abuse of generic code you need to make
it less easy to reach, especially less easy to write generic code
(using it should be easy I think!).  Restricting to package-level
parametricity means that there is an obvious trade-off in writing
generic code: you have to make a package for it.

Cheers,

Arnaud




On Wed, 20 Jan 2021 at 19:42, atd...@gmail.com  wrote:
>
>
> It didn't seem to me like proposal-killers enoguh so much so that they might 
> not have liked some of the design choices they entailed.
>
> The proble with the current running proposal is that it is a generalization 
> fo what the compiler does. It definitely  permits the spread of generic 
> functions across package boundaries since they can be exported. Only that the 
> type inference allows to not have to express the constraints in most cases. 
> This is still a complication.
>
> What I'd like to know is what are the complexities in a 
> package-parametrization. Indeed it would require additional syntax for 
> package import, there would still be a problem with zero-values return. But 
> it could be so much more simple other than that.
>
> The alternative is doing what the compiler does and dealing with constraint 
> on kinds. Which is a bit more complex.
>
> As far as deriving constraints from code, I think that it's actually 
> completely different at the package level than doing it at a higher 
> granularity ( functions) because a function is not a compilation unit.
> It is similar to essentialy replacing a name and building the package.
> At the function level, I would presume that  more information about the flow 
> of data would be required.
>
> As a note, I think that with the first iteration of type aliases, people had 
> started experimenting with such parameterization(I think in the image 
> package). Seems a bit more intuitive.
> Obviously, the difference is that packages are uniquely identified by their 
> names which is why, the names have to be internally parameterized as well.
>
> I think that dividing the design space for parametricity into simple code 
> parametrization at the package level and type list interfaces on the other 
> hand could be clearer and easier to handle but I'd like to know what are the 
> pain points.
>
>
> On Wednesday, January 20, 2021 at 8:22:07 PM UTC+1 axel.wa...@googlemail.com 
> wrote:
>>
>> The proposal-killers are mentioned - it is awkward for different 
>> instantiations of the same package to interact and there is no reason to 
>> believe generic usecases will delineate neatly into packages. It sounds like 
>> you do not agree that those are proposal-killers. And of course, you are 
>> welcome to write down your own design and maybe you even see something they 
>> didn't see. But you should be prepared that ultimately, in a "agree to 
>> disagree" situation, if the Go team feels it is awkward, their opinion will 
>> be what tips the scales.
>>
>> On Wed, Jan 20, 2021 at 7:09 PM atd...@gmail.com  wrote:
>>>
>>> Also, the distinction I see is that it would not be the same instantiation 
>>> of the same package as much as the creation of new packages ( the package 
>>> names should be parameterized too depending on the input)
>>
>>
>> Just to point out the obvious, that this means you still have to make up a 
>> syntax for type-parameters and instantiations. So, this is not a simple 
>> matter of importing a regular package.
>>
>>> Having different packages for different kinds seems to be a good separation 
>>> of concerns to me. Quite the opposite from mixing package boundaries with 
>>> types, I think it's rather the opposite. (the oppsoite being to allow 
>>> generic functions spreading across package boundaries)
>>
>>
>> Generic functions do not spread across package boundaries. Every generic 
>> function and every generic type will still live in a single package.
>>
>>>
>>> it's more complex to write (constraints)
>>
>>
>> FTR, the way to express constraints is largely independent on whether you 
>> constrain types on a per-function/type or per-package level.
>> For example, you suggested constraints should be derived implicitly based on 
>> the operations used in the package. But you can do that with generic 
>> functions too. For example, a previous incarnation of the generics design 
>> would have allowed you to specify 

Re: [go-nuts] Runtime cost of type assertions

[Arnaud Delobelle]

Thanks for the explanations.  Given reflect and the plugin mechanism, it 
seems like compile time analysis is not a very attractive option.  It is a 
shame in my case because I really feel like using an interface is the 
natural solution to my problem in Go, but the runtime cost of type 
assertion is too high for me to ignore (which means I can still use 
interfaces but need to work around type assertion).

Another possibility may be that at the first type assertion x.(I), the 
runtime learns that the concrete type of x satisfies interface I, so that 
next time an interface value with the same concrete type as x is coerced to 
I, it can take a short path.  I don't know enough about the runtime to know 
whether that is feasible at all!

I don't have any stats, but my personal experience tells me that when you 
do a type assertion to a non-empty interface, there are only going to be a 
few concrete types in your program that can satisfy it (although I guess 
this may become less true with the advent of parametric types).

-- 
Arnaud
On Wednesday, 30 December 2020 at 16:35:41 UTC Keith Randall wrote:

> Go currently doesn't do any optimizations based on the universe of types 
> seen in the program. This is both because reflect can create new ones, as 
> you mentioned, and also because package plugin can effectively "discover" 
> new ones.
>
> The Go compiler + "go build" are organized to compile one package at a 
> time. There's no global prepass at the moment which could discover such 
> facts anyway. We could add one, and bail if either reflect or plugin are 
> imported, but that seems unlikely to apply in many cases (reflect in 
> particular is imported indirectly by lots of things).
>
> On Wednesday, December 30, 2020 at 6:25:11 AM UTC-8 Sebastien Binet wrote:
>
>> it's because (well, one of the reasons, rather) we didn't find a great 
>> API to create a new type and add methods to it:
>>
>> - one needs a new typename
>> - one needs a set of methods that have the receiver type as first 
>> argument.
>>
>> you're a bit in a chicken-and-egg situation because it would be great to 
>> be able to create a type with *all* its methods known at the time of the 
>> type creation, so a type couldn't "gain" new methods (and thus implement 
>> new interfaces) during the course of the execution of a program.
>>
>> having a kind of "start-new-type", "add new methods", "seal-type" API is 
>> error prone.
>>
>> alternatively, one could use a "type builder" type:
>>
>> type TypeBuilder struct { .. }
>> func NewTypeBuilder(name string, kind reflect.Kind) *TypeBuilder { ... }
>>
>> // still the issue of how to address the receiver (ptr? value?)
>> // and its type within the 'fct' reflect.Value possibly created
>> // via a reflect.MakeFunc.
>> func (bldr *TypeBuilder) AddMethod(name string, fct reflect.Value) { ... }
>>
>> // Build seals the type and returns the finalized named type.
>> func (bldr *TypeBuilder) Build() reflect.Type { ... }
>>
>>
>>
>> but at the time, this kind of API was departing a bit from what we had in 
>> reflect.
>>
>> -s
>> ‐‐‐ Original Message ‐‐‐
>> On Tuesday, December 29th, 2020 at 6:21 PM, 'Axel Wagner' via golang-nuts 
>>  wrote:
>>
>> On Tue, Dec 29, 2020 at 6:01 PM Arnaud Delobelle  
>> wrote:
>>
>>>
>>>
>>> On Tuesday, 29 December 2020 at 16:25:41 UTC axel.wa...@googlemail.com 
>>> wrote:
>>>
>>>> On Tue, Dec 29, 2020 at 4:37 PM Arnaud Delobelle  
>>>> wrote:
>>>>
>>>>> Question 1: I *think* that the compiler has all the information 
>>>>> necessary to implement type assertion to the Cont interface as I have, 
>>>>> i.e. 
>>>>> it knows only 3 types implement that interface, so could it not do the 
>>>>> optimisation on my behalf?
>>>>>
>>>>  
>>>>>
>>>> Question 2: Or is it possible that other Go values can be made at 
>>>>> runtime that would implement this interface but not be one of the three 
>>>>> known types that implement it? 
>>>>>
>>>>
>>>> Yes, re 2. `reflect` can create new types at runtime. AFAIK the 
>>>> implementation for interface-type-assertions is basically to look it up in 
>>>> a global hashmap, which is pre-seeded with compile-time known types and 
>>>> then gets filled on each (successful) inteface-type-assertion with the 
>>>> correct method tables. But, I'm handwaving.
>&g

Re: [go-nuts] Runtime cost of type assertions

[Arnaud Delobelle]

On Tuesday, 29 December 2020 at 16:25:41 UTC axel.wa...@googlemail.com 
wrote:

> On Tue, Dec 29, 2020 at 4:37 PM Arnaud Delobelle  wrote:
>
>> Question 1: I *think* that the compiler has all the information necessary 
>> to implement type assertion to the Cont interface as I have, i.e. it knows 
>> only 3 types implement that interface, so could it not do the optimisation 
>> on my behalf?
>>
>  
>>
> Question 2: Or is it possible that other Go values can be made at runtime 
>> that would implement this interface but not be one of the three known types 
>> that implement it? 
>>
>
> Yes, re 2. `reflect` can create new types at runtime. AFAIK the 
> implementation for interface-type-assertions is basically to look it up in 
> a global hashmap, which is pre-seeded with compile-time known types and 
> then gets filled on each (successful) inteface-type-assertion with the 
> correct method tables. But, I'm handwaving.
>
>
Ok, I have just looked at the docs for the reflect package, but I can't see 
a way to create a type that implements anything but the empty interface.  
Is that correct?  In that case, wouldn't it mean that it is known at 
compile time what types implement a given (non-empty) interface?

Edit: I see that reflect.StructOf allows creation of struct types out of 
StructField specifications, which have an Anonymous boolean field.  I 
imagine that the created struct will inherit the methods of embedded types, 
so it may implement non empty interfaces.  I'm interested in valid 
use-cases for this, as it seems to be the thing that prevents this 
optimisation from being possible.

Under these assumptions, it might be *possible* to first check against the 
> statically known types and only fall back on the map if none of that 
> matches. But it doesn't seem 100% clear to me that that's always faster.
>
> I think concrete type-assertions will always be faster than interface 
> type-assertions though - for a concrete type-assertions, it's really just 
> comparing the two type-pointers and copying the value (which, in your case, 
> is a pointer itself), whereas for an interface type-assertion, the actual 
> method table must be assembled or looked up.
>
 
Yes, that's what I was imagining, and why I decided to try coercion to 
concrete type instead!

-- 
Arnaud

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/e6a8cd0f-bf8c-46ee-9435-48394a120cb5n%40googlegroups.com.

Re: [go-nuts] Runtime cost of type assertions

[Arnaud Delobelle]

On Tuesday, 29 December 2020 at 16:25:41 UTC axel.wa...@googlemail.com 
wrote:

> On Tue, Dec 29, 2020 at 4:37 PM Arnaud Delobelle  wrote:
>
>> Question 1: I *think* that the compiler has all the information necessary 
>> to implement type assertion to the Cont interface as I have, i.e. it knows 
>> only 3 types implement that interface, so could it not do the optimisation 
>> on my behalf?
>>
>  
>>
> Question 2: Or is it possible that other Go values can be made at runtime 
>> that would implement this interface but not be one of the three known types 
>> that implement it? 
>>
>
> Yes, re 2. `reflect` can create new types at runtime. AFAIK the 
> implementation for interface-type-assertions is basically to look it up in 
> a global hashmap, which is pre-seeded with compile-time known types and 
> then gets filled on each (successful) inteface-type-assertion with the 
> correct method tables. But, I'm handwaving.
>
> Under these assumptions, it might be *possible* to first check against the 
> statically known types and only fall back on the map if none of that 
> matches. But it doesn't seem 100% clear to me that that's always faster.
>
> I think concrete type-assertions will always be faster than interface 
> type-assertions though - for a concrete type-assertions, it's really just 
> comparing the two type-pointers and copying the value (which, in your case, 
> is a pointer itself), whereas for an interface type-assertion, the actual 
> method table must be assembled or looked up.
>
> But, here's a question: If `v.iface` always contains a `Cont` - at least 
> that's what I assume from the fact that you don't use a ,ok assertion - why 
> not just declare it as such, instead of `interface{}`?
>
>>
>>
It doesn't always contain a Cont.  Sometimes it should, which is why this 
function panics (programming error).  There is also TryCont() which returns 
a boolean as well and doesn't panic. 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d4cab7ff-30dc-4fdd-b656-c89aa7d1aa8dn%40googlegroups.com.

[go-nuts] Runtime cost of type assertions

[Arnaud Delobelle]

Hi there!

Looking at performance bottlenecks for my implementation of Lua in Go 
([1]), I found that type assertions can have a significant cost, which 
feels unnecessary to me.  I couldn't explain it without quite a lot of 
context, which makes my post quite long - sorry about that!

I have a Value type that holds Lua values:

// A Value is a runtime value.
type Value struct {
scalar uint64
iface interface{}
}

As Lua is dynamically typed, values can hold any type.  In the Lua runtime 
implementation, there are "type assertion" functions to convert a Value to 
a specific Go type, among which is the Cont interface type ([2]) which 
represents a continuation:

func (v Value) AsCont() Cont {
return v.iface.(Cont)
}

This function is called every time a Lua function is called and turns out 
to be costly when many function calls are made.  I know exactly what types 
implement the Cont interface, so I tried an other implementation of AsCont 
as follows:

func (v Value) AsCont2() Cont {
switch cont := v.iface.(type) {
case *GoCont:
return cont
case *LuaCont:
return cont
case *Termination:
return cont
default:
// Only the types above implement the Cont interface
panic("value is not a continuation")
}
}

Here is a benchmark comparing AsCont and AsCont2.

func BenchmarkAsCont(b *testing.B) {
v1 := ContValue(new(GoCont))
v2 := ContValue(new(LuaCont))
v3 := ContValue(new(Termination))
b.ResetTimer()
for i := 0; i < b.N; i++ {
_ = v1.AsCont()
_ = v2.AsCont()
_ = v3.AsCont()
}
}

func BenchmarkAsCont2(b *testing.B) {
v1 := ContValue(new(GoCont))
v2 := ContValue(new(LuaCont))
v3 := ContValue(new(Termination))
b.ResetTimer()
for i := 0; i < b.N; i++ {
_ = v1.AsCont2()
_ = v2.AsCont2()
_ = v3.AsCont2()
}
}

$ go test -run='^$' -bench '^(BenchmarkAsCont)' 
github.com/arnodel/golua/runtime
goos: darwin
goarch: amd64
pkg: github.com/arnodel/golua/runtime
cpu: Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz
BenchmarkAsCont-8   5279850420.96 ns/op
BenchmarkAsCont2-8  4330326862.866 ns/op
PASS
ok  github.com/arnodel/golua/runtime2.908s

There's a very significant difference in this benchmark, and in some "real 
tests" benchmarking Lua code  I get ~15% speedup, which is pretty good.  
Now here are my questions.

Question 1: I *think* that the compiler has all the information necessary 
to implement type assertion to the Cont interface as I have, i.e. it knows 
only 3 types implement that interface, so could it not do the optimisation 
on my behalf?

Question 2: Or is it possible that other Go values can be made at runtime 
that would implement this interface but not be one of the three known types 
that implement it?

Question 3: Is it possible that there is something dodgy going on with the 
benchmarks, with some code being optimised away - if so, how can I check 
that?

Thanks in advance for any insights!

-- 
Arnaud

[1] https://github.com/arnodel/golua

[2] Definition of the Cont interface type:
type Cont interface {
Push(Value)
PushEtc([]Value)
RunInThread(*Thread) (Cont, *Error)
Next() Cont
DebugInfo() *DebugInfo
}



-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/efcf0a84-4e7d-4241-9f5a-994774a7f14dn%40googlegroups.com.

Re: [go-nuts] Re: [generics] Print[T Stringer](s []T) vs Print(s []Stringer)

[Arnaud Delobelle]

I understand the difference, but that doesn't prevent me from having to 
choose between the two implementations.  To simplify greatly, and as you 
pointed out in your reply, there is a tension between "simplicity" 
(non-generic) and "performance" (generic), and that is where I fear the 
friction will come from. 

Looking beyond the syntax, the code for both implementations expresses 
exactly the same thing.  So in a way it is the syntax that forces the 
choice to be made at the time the function is written.  What I would like 
is not to have to make the choice at the time that I write the function, 
but perhaps this wish is impossible given the current Go syntax.

Arnaud

On Sunday, 27 December 2020 at 17:32:48 UTC k.alex...@gmail.com wrote:

> Since in this case the use of generics doesn't let you do anything new, I 
> would argue that the KISS principle applies and the non-generic version 
> should be preferred.
>
> I think a linter can be added to go vet to warn about instances like this 
> one (where the type parameter can be replaced by the type bound) to 
> encourage simplicity.
>
> But as Ian pointed out, in the version that takes a slice argument, using 
> generics does allow you to do something you couldn't do otherwise (without 
> reallocating the contents of the slice to effect a "cast" to []Stringer). 
> In this case using generics actually makes the caller's job simpler and 
> improves performance by avoiding the need for reallocation.
>
> On Sun, Dec 27, 2020 at 4:23 AM Arnaud Delobelle  wrote:
>
>> In my opinion, the issue is that there are two ways to express (almost) 
>> the same thing and that in itself creates friction in the language.
>>
>> There may be a valid reason to choose one version over the other, but 
>> every time it will be necessary to review the pros and cons of each 
>> alternative.
>>
>> If we could have "generics" without having to make this choice it would 
>> unblock the whole thing as far as I am concerned.
>>
>> Cheers
>>
>> On Sun, 27 Dec 2020, 05:25 K. Alex Mills,  wrote:
>>
>>> While it depends on the final generics implementation, my understanding 
>>> of how things stand now is that Print would compile down to a separate 
>>> chunk of binary for each type T that is used. For instance, if you used 
>>> Print[A] and Print[B] in your code, they would each refer to separate 
>>> binary implementations in which T is replaced by A and B, respectively.
>>>
>>> Printi does not do this, so you should see a smaller binary. 
>>>
>>> IIRC, Printi also has to do a bit of work to lookup the Stringer method 
>>> on the type inhabiting the interface. I don't think that creates a 
>>> significant performance hit, but I might be understating the overhead of 
>>> interface dispatch. A benchmark would help here (alas, I am on my phone).
>>>
>>> With respect for the concerns mentioned above, I don't see an argument 
>>> for preferring Print over Printi. However, there may other concerns which I 
>>> am unaware of.
>>>
>>> On Sat, Dec 26, 2020, 9:58 PM Elliot  wrote:
>>>
>>>> If we remove slice from OP's example:
>>>>
>>>> https://go2goplay.golang.org/p/KSJpRw1Lrmm
>>>>
>>>> func Print[T Stringer](s T) {
>>>> fmt.Print(s.String())
>>>> }
>>>>
>>>> func Printi(s Stringer) {
>>>> fmt.Print(s.String())
>>>> }
>>>>
>>>> Are these two equivalent? When should one be chosen over the other?
>>>>
>>>> On Thursday, 24 December 2020 at 04:41:16 UTC+8 Henrik Johansson wrote:
>>>>
>>>>> Why will interfaces be more idiomatic once generics lands? It remains 
>>>>> to be seen I guess but I could very well see the other way become the 
>>>>> idiom.
>>>>>
>>>>> On Wed, 23 Dec 2020, 21:20 wilk,  wrote:
>>>>>
>>>>>> On 23-12-2020, Ian Lance Taylor wrote:
>>>>>> > On Wed, Dec 23, 2020 at 9:54 AM wilk  wrote:
>>>>>> >>
>>>>>> >> https://go2goplay.golang.org/p/fTW3hJYNgfU
>>>>>> >>
>>>>>> >> type Stringer interface {
>>>>>> >>String() string
>>>>>> >> }
>>>>>> >>
>>>>>> >> Print[T Stringer](s []T)
>>>>>> >>
>>>>>> >> Print(s []Stringer)
>>>>>> >>
>>>>>> >> Bot

Re: [go-nuts] Re: [generics] Print[T Stringer](s []T) vs Print(s []Stringer)

[Arnaud Delobelle]

In my opinion, the issue is that there are two ways to express (almost) the
same thing and that in itself creates friction in the language.

There may be a valid reason to choose one version over the other, but every
time it will be necessary to review the pros and cons of each alternative.

If we could have "generics" without having to make this choice it would
unblock the whole thing as far as I am concerned.

Cheers

On Sun, 27 Dec 2020, 05:25 K. Alex Mills,  wrote:

> While it depends on the final generics implementation, my understanding of
> how things stand now is that Print would compile down to a separate chunk
> of binary for each type T that is used. For instance, if you used Print[A]
> and Print[B] in your code, they would each refer to separate binary
> implementations in which T is replaced by A and B, respectively.
>
> Printi does not do this, so you should see a smaller binary.
>
> IIRC, Printi also has to do a bit of work to lookup the Stringer method on
> the type inhabiting the interface. I don't think that creates a significant
> performance hit, but I might be understating the overhead of interface
> dispatch. A benchmark would help here (alas, I am on my phone).
>
> With respect for the concerns mentioned above, I don't see an argument for
> preferring Print over Printi. However, there may other concerns which I am
> unaware of.
>
> On Sat, Dec 26, 2020, 9:58 PM Elliot  wrote:
>
>> If we remove slice from OP's example:
>>
>> https://go2goplay.golang.org/p/KSJpRw1Lrmm
>>
>> func Print[T Stringer](s T) {
>> fmt.Print(s.String())
>> }
>>
>> func Printi(s Stringer) {
>> fmt.Print(s.String())
>> }
>>
>> Are these two equivalent? When should one be chosen over the other?
>>
>> On Thursday, 24 December 2020 at 04:41:16 UTC+8 Henrik Johansson wrote:
>>
>>> Why will interfaces be more idiomatic once generics lands? It remains to
>>> be seen I guess but I could very well see the other way become the idiom.
>>>
>>> On Wed, 23 Dec 2020, 21:20 wilk,  wrote:
>>>
 On 23-12-2020, Ian Lance Taylor wrote:
 > On Wed, Dec 23, 2020 at 9:54 AM wilk  wrote:
 >>
 >> https://go2goplay.golang.org/p/fTW3hJYNgfU
 >>
 >> type Stringer interface {
 >>String() string
 >> }
 >>
 >> Print[T Stringer](s []T)
 >>
 >> Print(s []Stringer)
 >>
 >> Both forms works.
 >> How to prevent double way to do the same things that can be
 confusing ?
 >
 > Both forms work but they mean two different things.
 >
 > Print(s []Stringer) takes a slice of the type Stringer.
 >
 > Print[T Stringer](s []T) takes a slice of some type T, where T
 > implements Stringer.
 >
 > For example, if MyInt implements Stringer, and I have a []MyInt, then
 > I can call Print[T Stringer](s []T) but I can't call Print(s
 > []Stringer), because a []Stringer is not a []MyInt.

 I understand the differences. But i'm affraid someone who never used
 Go before will use type parameters instead of interface which is more
 idiomatic i think.
 I mean it will be difficult to say, you could use type parameters but
 you should use interface, or something like that...
 I'm speaking about ease of learn Go2.

 --
 wilk

 --
 You received this message because you are subscribed to the Google
 Groups "golang-nuts" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to golang-nuts+unsubscr...@googlegroups.com.

>>> To view this discussion on the web visit
 https://groups.google.com/d/msgid/golang-nuts/rs08pp%24p8m%241%40ciao.gmane.io
 .

>>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/d044ae30-7254-4a86-9cba-1bc18eeb7fefn%40googlegroups.com
>> 
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CALJzkY-asEOYK1_zgVzNJ4B37u17QX0hZr5vZGADe-vEJgTtQA%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this 

Re: [go-nuts] Obtaining an efficient hash for Go values

[Arnaud Delobelle]

Lua `next` is a function that takes two arguments: a table and a key and 
returns the another key (or nil).  Unlike an iterator, it holds no 
context.  It is guaranteed that you will iterate over all keys if you 
repeatedly call next as follows

k = next(t)
while k ~= nil do
k = next(t, k)
end

If, however, you were to insert new items into t in the body of this loop, 
this guaranteed would no longer hold.  This is what is meant by "undefined 
behaviour", as far as I understand.

Arnaud

On Thursday, 24 December 2020 at 15:55:27 UTC Jan Mercl wrote:

> On Thu, Dec 24, 2020 at 4:08 PM Arnaud Delobelle  wrote:
>
> > The order is undefined but stable (provided you don't insert new 
> values), and accessible via the `next` function. Here is an example:
>
> Quoting from the provided link:
>
> """"
> next (table [, index])
>
> Allows a program to traverse all fields of a table. Its first argument
> is a table and its second argument is an index in this table. next
> returns the next index of the table and its associated value. When
> called with nil as its second argument, next returns an initial index
> and its associated value. When called with the last index, or with nil
> in an empty table, next returns nil. If the second argument is absent,
> then it is interpreted as nil. In particular, you can use next(t) to
> check whether a table is empty.
>
> The order in which the indices are enumerated is not specified, even
> for numeric indices. (To traverse a table in numerical order, use a
> numerical for.)
>
> The behavior of next is undefined if, during the traversal, you assign
> any value to a non-existent field in the table. You may however modify
> existing fields. In particular, you may clear existing fields.
> """"
>
> The word 'stable' does not seem to be present in this part of the
> specs at all, so only "The order in which the indices are enumerated
> is not specified, even for numeric indices." remains to be considered.
> That said, I see no problem with implementing it by iterating a Go
> map.
>
> Or is the Lua specification perhaps incomplete? Maybe people
> incorrectly rely on the behavior of a particular implementation.
> That's why the Go map iteration is intentionally randomized, BTW.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/4aa8f174-2e3b-4124-9e55-44b415cce398n%40googlegroups.com.

Re: [go-nuts] Obtaining an efficient hash for Go values

[Arnaud Delobelle]

On Thursday, 24 December 2020 at 14:31:22 UTC Jan Mercl wrote:

> On Thu, Dec 24, 2020 at 3:12 PM Arnaud Delobelle  
> wrote: 
> > That's interesting, thanks. Although I don't think it fits my use case 
> because I need something stronger than an iterator: a function that given a 
> table and a key returns the next key in the table (as this is a public API 
> that Lua provides, see https://www.lua.org/manual/5.3/manual.html#pdf-next). 
> From my point of view this is an unfortunate API! Nevertheless it is what 
> it is... I haven't found even a hacky way to obtain that for a Go map. That 
> is why I think I need to make my own hashtable implementation 
>
> The linked pdf seems to indicate that Lua's 'next' can be implemented 
> using Go range over a map with no problem because the iteration order 
> is undefined in Lua as well. 
>
> What am I missing? 
>

The order is undefined but stable (provided you don't insert new values), 
and accessible via the `next` function.  Here is an example:

$ lua
Lua 5.3.5  Copyright (C) 1994-2018 Lua.org, PUC-Rio
> t = { x = 1, y = 2, z = 3 }
> next(t, 'y')
z   3
> next(t, 'z')
x   1
> next(t, 'y')
z   3
> next(t, 'z')
x   1

I don't understand how to do that with a map iterator unless get a new 
iterator each time and consume it till I get the key I want to find the 
next key for (which would be very inefficient), or there is a way to 
initialise the iterator at a given key - but I haven't found how to do the 
latter.

Arnaud

 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/31930063-9c33-4e0f-8372-ee5ae991b3e1n%40googlegroups.com.

Re: [go-nuts] Obtaining an efficient hash for Go values

[Arnaud Delobelle]

I did have a look at hash/maphash, but I didn't think it was the right 
shaped API for what I am doing.  In short, I would have to contrive to 
provide words to the package as byte slices, which then would turn them 
back into words under the hood.  That feels like a superfluous round trip, 
given that the table is at the heart of everything in Lua as it is the only 
data structure available (so it is used to implement objects, modules, 
namespaces...).  I haven't discarded it yet but I wanted to explore other 
possibilities before doing some benchmarking.

Arnaud

On Thursday, 24 December 2020 at 10:45:30 UTC axel.wa...@googlemail.com 
wrote:

> On Thu, Dec 24, 2020 at 11:19 AM Arnaud Delobelle  
> wrote:
>
>> Does that sound like a sensible approach?  I.e. is it safe enough to use 
>> the go:linkname directive, and do those seem like the right functions to 
>> call to obtain a good hash?
>>
>
> It is probably better to use hash/maphash, if you can.
> https://golang.org/pkg/hash/maphash/
> It's the intended use-case for that package and not subject to changes in 
> the runtime. However, the downside is that it doesn't allow all the same 
> values to be used as keys as Go maps do (you have to do the mapping from 
> values to []byte manually), so it might not be suitable.
>  
> Personally, I feel that go:linkname and similar trickery is too much black 
> magic to actually use. But TBF, I might be too biased on cleanliness over 
> performance and other concerns.
>
>
>> TIA
>>
>> -- 
>> Arnaud
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/79e3f124-037c-4c10-a7b6-42f496bd26b6n%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/golang-nuts/79e3f124-037c-4c10-a7b6-42f496bd26b6n%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/53df26d1-cacc-48ed-9355-38dd46cddef4n%40googlegroups.com.

Re: [go-nuts] Obtaining an efficient hash for Go values

[Arnaud Delobelle]

(I am replying inline to make the context of my comment clear, although it 
seems common practice to top-post in this forum - is there any guidance 
about this?)

Arnaud

On Thursday, 24 December 2020 at 10:31:54 UTC kortschak wrote:

> You can also use the internal map implementation and make us of the 
> runtime's map iterator. This is relatively straightforward at the cost 
> of vigilance for changes in the runtime. 
>
> Here is an example (note that yo need a .S file as well to get the 
> go:linkname magic to work). 
> https://github.com/gonum/gonum/blob/master/graph/iterator/map.go 
>
>
That's interesting, thanks.  Although I don't think it fits my use case 
because I need something stronger than an iterator: a function that given a 
table and a key returns the next key in the table (as this is a public API 
that Lua provides, see 
https://www.lua.org/manual/5.3/manual.html#pdf-next).  From my point of 
view this is an unfortunate API!  Nevertheless it is what it is...  I 
haven't found even a hacky way to obtain that for a Go map.  That is why I 
think I need to make my own hashtable implementation
 

> (we back this with a reflect equivalent that it provided by 
> https://golang.org/pkg/reflect/#MapIter. 
>
> Dan 
>
> On Thu, 2020-12-24 at 02:18 -0800, Arnaud Delobelle wrote: 
> > Hi there! 
> > 
> > In my continued efforts to improve the performance of my Go Lua 
> > implementation [1], I have reached a bottleneck which causes me a 
> > quandary. 
> > 
> > Lua has a data structure which is called 'table', which is 
> > essentially a hashmap. So far I have implemented it as a Go map, 
> > which works OK. However there is significant overhead coming from 
> > the fact that Lua has a `next` function that allows getting the 
> > "next" key-value pair in a table after a given one: `next(t, key)`. 
> > As far as I can tell Go doesn't allow this so if I want to use a Go 
> > map, I also have to keep track of the next key for each key, which 
> > doubles the memory requirement, necessitates more accounting in the 
> > code and makes iteration via `next` slower. 
> > 
> > So I am looking at not using the builtin Go map and making my own 
> > hashtable implementation. However, because the keys are still made 
> > of Go values, I would like to benefit from the quick hashing that 
> > maps use. After some poking around in the implementation of map (and 
> > discovering the //go:linkname compiler directive), I think that I can 
> > do this: 
> > 
> > 
> > // This is the Lua Value type. The scalar part contains the payload 
> > of int64, float64 or bool for quicker access and minimising 
> > allocations. 
> > type Value struct { 
> > scalar uint64 
> > iface interface{} 
> > } 
> > 
> > 
> > //go:linkname goRuntimeInt64Hash runtime.int64Hash 
> > //go:noescape 
> > func goRuntimeInt64Hash(i uint64, seed uintptr) uintptr 
> > 
> > //go:linkname goRuntimeEfaceHash runtime.efaceHash 
> > //go:noescape 
> > func goRuntimeEfaceHash(i interface{}, seed uintptr) uintptr 
> > 
> > // Hash returns a hash for the value. 
> > func (v Value) Hash() uintptr { 
> > if v.scalar != 0 { 
> > return goRuntimeInt64Hash(v.scalar, 0) 
> > } 
> > return goRuntimeEfaceHash(v.iface, 0) 
> > } 
> > 
> > Does that sound like a sensible approach? I.e. is it safe enough to 
> > use the go:linkname directive, and do those seem like the right 
> > functions to call to obtain a good hash? 
> > 
> > TIA 
> > 
> > -- 
> > Arnaud 
> > -- 
> > You received this message because you are subscribed to the Google 
> > Groups "golang-nuts" group. 
> > To unsubscribe from this group and stop receiving emails from it, 
> > send an email to golang-nuts...@googlegroups.com. 
> > To view this discussion on the web visit 
> > 
> https://groups.google.com/d/msgid/golang-nuts/79e3f124-037c-4c10-a7b6-42f496bd26b6n%40googlegroups.com
>  
> > . 
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ee5e1f22-32a4-4adb-b1f0-52bf7148fa9bn%40googlegroups.com.

[go-nuts] Obtaining an efficient hash for Go values

[Arnaud Delobelle]

Hi there!

In my continued efforts to improve the performance of my Go Lua 
implementation [1], I have reached a bottleneck which causes me a quandary.

Lua has a data structure which is called 'table', which is essentially a 
hashmap.  So far I have implemented it as a Go map, which works OK.  
However there is significant overhead coming from the fact that Lua has a 
`next` function that allows getting the "next" key-value pair in a table 
after a given one: `next(t, key)`.  As far as I can tell Go doesn't allow 
this so if I want to use a Go map, I also have to keep track of the next 
key for each key, which doubles the memory requirement, necessitates more 
accounting in the code and makes iteration via `next` slower.

So I am looking at not using the builtin Go map and making my own hashtable 
implementation.  However, because the keys are still made of Go values, I 
would like to benefit from the quick hashing that maps use.  After some 
poking around in the implementation of map (and discovering the 
//go:linkname compiler directive), I think that I can do this:


// This is the Lua Value type.  The scalar part contains the payload of 
int64, float64 or bool for quicker access and minimising allocations.
type Value struct {
scalar uint64
iface interface{}
}


//go:linkname goRuntimeInt64Hash runtime.int64Hash
//go:noescape
func goRuntimeInt64Hash(i uint64, seed uintptr) uintptr

//go:linkname goRuntimeEfaceHash runtime.efaceHash
//go:noescape
func goRuntimeEfaceHash(i interface{}, seed uintptr) uintptr

// Hash returns a hash for the value.
func (v Value) Hash() uintptr {
if v.scalar != 0 {
return goRuntimeInt64Hash(v.scalar, 0)
}
return goRuntimeEfaceHash(v.iface, 0)
}

Does that sound like a sensible approach?  I.e. is it safe enough to use 
the go:linkname directive, and do those seem like the right functions to 
call to obtain a good hash?

TIA

-- 
Arnaud

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/79e3f124-037c-4c10-a7b6-42f496bd26b6n%40googlegroups.com.

[go-nuts] Re: Interfaces holding integers and memory allocations

[Arnaud Delobelle]

Luckily, I have the "no scalar" version with a build tag.  Here is a simple 
benchmark:

func BenchmarkValue(b *testing.B) {
for n := 0; n < b.N; n++ {
sv := IntValue(0)
for i := 0; i < 1000; i++ {
iv := IntValue(int64(i))
sv, _ = add(nil, sv, iv) // add is the "real" lua runtime 
function that adds two numeric values.
}
}
}

Results with the "scalar" version

$ go test -benchmem -run=^$ -bench '^(BenchmarkValue)$' ./runtime
goos: darwin
goarch: amd64
pkg: github.com/arnodel/golua/runtime
cpu: Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz
BenchmarkValue-8  122995  9494 ns/op   0 
B/op  0 allocs/op
PASS
ok  github.com/arnodel/golua/runtime1.415s

Results without the "scalar" version (noscalar build tag)

$ go test -benchmem -run=^$ -tags noscalar  -bench '^(BenchmarkValue)$' 
./runtime
goos: darwin
goarch: amd64
pkg: github.com/arnodel/golua/runtime
cpu: Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz
BenchmarkValue-8   37407 32357 ns/op   13768 
B/op   1721 allocs/op
PASS
ok  github.com/arnodel/golua/runtime1.629s

That looks like a pretty big improvement :)

The improvement is also significant in real workloads but not.so dramatic 
(given they don't spend all their time manipulating scalar values!)

On Monday, 21 December 2020 at 21:02:26 UTC ben...@gmail.com wrote:

> Nice! Do you have any benchmarks on how much faster the "scalar" version 
> is than the non-scalar?
>
> On Tuesday, December 22, 2020 at 12:58:19 AM UTC+13 arn...@gmail.com 
> wrote:
>
>> Just an update (in case anyone is interested!).  I went for the approach 
>> described below of having a Value type holding a scalar for quick access to 
>> values that fit in 64 bits (ints, floats, bools) and an interface fo for 
>> the rest.
>>
>> type Value struct {
>> scalar uint64
>> iface interface{}
>> }
>>
>> That significantly decreased memory management pressure on the program 
>> for many workloads, without having to manage a pool of say integer values.  
>> It also had the consequence of speeding up many arithmetic operations.  
>> Thanks all for your explanations and suggestions!
>>
>> -- 
>> Arnaud
>>
>> On Wednesday, 16 December 2020 at 11:15:32 UTC Arnaud Delobelle wrote:
>>
>>> Ah interesting, I guess that could mean I would need to switch to using 
>>> reflect.Value as the "value" type in the Lua runtime.  I am unclear about 
>>> the performance consequences, but I guess I could try to measure that.
>>>
>>> Also, looking at the implementation of reflect, its seems like the Value 
>>> type I suggested in my reply to Ben [1] is a "special purpose" version of 
>>> reflect.Value - if you squint at it from the right angle!
>>>
>>> -- 
>>> Arnaud
>>>
>>> [1]
>>> type Value struct {
>>> scalar uint64
>>> iface interface{}
>>> }
>>> On Wednesday, 16 December 2020 at 00:56:52 UTC Keith Randall wrote:
>>>
>>>> Unfortunately for you, interfaces are immutable. We can't provide a 
>>>> means to create an interface from a pointer, because then the user can 
>>>> modify the interface using the pointer they constructed it with (as you 
>>>> were planning to do).
>>>>
>>>> You could use a modifiable reflect.Value for this.
>>>>
>>>> var i int64  = 77
>>>> v := reflect.ValueOf().Elem()
>>>>
>>>> At this point, v now has .Type() of int64, and is settable.
>>>>
>>>> Note that to get the value you can't do v.Interface().(int64), as that 
>>>> allocates. You need to use v.Int().
>>>> Of course, reflection has its own performance gotchas. It will solve 
>>>> this problem but may surface others.
>>>> On Tuesday, December 15, 2020 at 12:04:54 PM UTC-8 ben...@gmail.com 
>>>> wrote:
>>>>
>>>>> Nice project!
>>>>>
>>>>> It's a pity Go doesn't have C-like unions for cases like this (though 
>>>>> I understand why). In my implementation of AWK in Go, I modelled the 
>>>>> value 
>>>>> type as a pseudo-union struct, passed by value:
>>>>>
>>>>> type value struct {
>>>>> typ valueType // Type of value (Null, Str, Num, NumStr)
>>>>> s   string// String value (for typeStr)
>>>>> n   

[go-nuts] Re: Interfaces holding integers and memory allocations

[Arnaud Delobelle]

Just an update (in case anyone is interested!).  I went for the approach 
described below of having a Value type holding a scalar for quick access to 
values that fit in 64 bits (ints, floats, bools) and an interface fo for 
the rest.

type Value struct {
scalar uint64
iface interface{}
}

That significantly decreased memory management pressure on the program for 
many workloads, without having to manage a pool of say integer values.  It 
also had the consequence of speeding up many arithmetic operations.  Thanks 
all for your explanations and suggestions!

-- 
Arnaud

On Wednesday, 16 December 2020 at 11:15:32 UTC Arnaud Delobelle wrote:

> Ah interesting, I guess that could mean I would need to switch to using 
> reflect.Value as the "value" type in the Lua runtime.  I am unclear about 
> the performance consequences, but I guess I could try to measure that.
>
> Also, looking at the implementation of reflect, its seems like the Value 
> type I suggested in my reply to Ben [1] is a "special purpose" version of 
> reflect.Value - if you squint at it from the right angle!
>
> -- 
> Arnaud
>
> [1]
> type Value struct {
> scalar uint64
> iface interface{}
> }
> On Wednesday, 16 December 2020 at 00:56:52 UTC Keith Randall wrote:
>
>> Unfortunately for you, interfaces are immutable. We can't provide a means 
>> to create an interface from a pointer, because then the user can modify the 
>> interface using the pointer they constructed it with (as you were planning 
>> to do).
>>
>> You could use a modifiable reflect.Value for this.
>>
>> var i int64  = 77
>> v := reflect.ValueOf().Elem()
>>
>> At this point, v now has .Type() of int64, and is settable.
>>
>> Note that to get the value you can't do v.Interface().(int64), as that 
>> allocates. You need to use v.Int().
>> Of course, reflection has its own performance gotchas. It will solve this 
>> problem but may surface others.
>> On Tuesday, December 15, 2020 at 12:04:54 PM UTC-8 ben...@gmail.com 
>> wrote:
>>
>>> Nice project!
>>>
>>> It's a pity Go doesn't have C-like unions for cases like this (though I 
>>> understand why). In my implementation of AWK in Go, I modelled the value 
>>> type as a pseudo-union struct, passed by value:
>>>
>>> type value struct {
>>> typ valueType // Type of value (Null, Str, Num, NumStr)
>>> s   string// String value (for typeStr)
>>> n   float64   // Numeric value (for typeNum and typeNumStr)
>>> }
>>>
>>> Code here: 
>>> https://github.com/benhoyt/goawk/blob/22bd82c92461cedfd02aa7b8fe1fbebd697d59b5/interp/value.go#L22-L27
>>>
>>> Initially I actually used "type Value interface{}" as well, but I 
>>> switched to the above primarily to model the funky AWK "numeric string" 
>>> concept. However, I seem to recall that it had a significant performance 
>>> benefit too, as passing everything by value avoided a number of allocations.
>>>
>>> Lua has more types to deal with, but you could try something similar. Or 
>>> maybe include int64 (for bool as well) and string fields, and everything 
>>> else falls back to interface{}? It'd be a fairly large struct, so not sure 
>>> it would help ... you'd have to benchmark it. But I'm thinking something 
>>> like this:
>>>
>>> type Value struct {
>>> typ valueType
>>> i int64 // for typ = bool, integer
>>> s string // for typ = string
>>> v interface{} // for typ = float, other
>>> }
>>>
>>> -Ben
>>>
>>> On Wednesday, December 16, 2020 at 6:50:05 AM UTC+13 arn...@gmail.com 
>>> wrote:
>>>
>>>> Hi
>>>>
>>>> The context for this question is that I am working on a pure Go 
>>>> implementation of Lua [1] (as a personal project).  Now that it is more or 
>>>> less functionally complete, I am using pprof to see what the main CPU 
>>>> bottlenecks are, and it turns out that they are around memory management.  
>>>> The first one was to do with allocating and collecting Lua "stack frame" 
>>>> data, which I improved by having add-hoc pools for such objects.
>>>>
>>>> The second one is the one that is giving me some trouble. Lua is a 
>>>> so-called "dynamically typed" language, i.e. values are typed but 
>>>> variables 
>>>> are not.  So for easy interoperability with Go I implemented Lua values 
>&g

Re: [go-nuts] Performance issue with os.File.Write

[Arnaud Delobelle]

I agree that it's very sensible this way!  My confusion stemmed for a 
combination of blind spots in both Go and C (in fact it was somewhere in my 
head that the File API provided unbuffered access, just not accessible at 
that time!).  Thanks all for clarifying.

Arnaud

On Monday, 21 December 2020 at 07:17:15 UTC di...@samantree.com wrote:

> As Jan said "apples and oranges", in this case comparing *os.Stdout with C 
> File *stdout is not fair. The equivalent of *os.Stdout in C is the 
> filedescriptor 1 (STDOUT macro), and the equivalent of *os.Stdout.Write is 
> write(2) (the syscall), not fwrite or fputs. If you retry your 
> microbenchmark using the syscall write(2) with filedescriptor 1, you'll 
> probably see very similar (if not identical) results.
>
> Personally I prefer the explicit buffering proposed by the Go standard 
> library, rather than the implicit one of the C stdio.h header. When I was 
> starting using C, I've already been bitten by the fact that my fwrites were 
> being implicitly buffered, until I discovered the setbuf(3) function and 
> read the documentation. Having different default buffering schemes 
> depending on the output used (yes stderr is not buffered by default), I 
> find more confusing.
>
> On Sun, 20 Dec 2020 at 21:32, ben...@gmail.com  wrote:
>
>> And os.Stdout (and friends) are all regular *os.File objects (which as 
>> Jan said, don't buffer). It was non-intuitive to me that stdout didn't 
>> buffer by default, because it's such a bad thing for efficiently writing 
>> lots of output, but I guess it makes sense when you want terminal output to 
>> appear right away. So I realized it made sense, and gives you more control. 
>> And it's so easy to wrap it in a bufio.NewWriter() ... Flush() if you need 
>> buffering.
>>
>> I ran into this exact same issue when implementing GoAWK ... a 10-line 
>> fix gave me a 10x speedup. 
>> https://github.com/benhoyt/goawk/commit/60745c3503ba3d99297816f5c7b5364a08ec47ab
>>
>> -Ben
>>
>> On Monday, December 21, 2020 at 12:27:43 AM UTC+13 arn...@gmail.com 
>> wrote:
>>
>>> Ah, that is it, thank you!
>>>
>>> On Sunday, 20 December 2020 at 11:06:05 UTC Jan Mercl wrote:
>>>
>>>> On Sun, Dec 20, 2020 at 11:53 AM Arnaud Delobelle  
>>>> wrote: 
>>>>
>>>> > TLDR; a simple test program appears to show that Go's 
>>>> (*os.File).Write is 10x slower than C's fputs (on MacOS). 
>>>>
>>>> Apples and oranges. fputs buffers, os.File does not. Rewrite the 
>>>> benchmark using bufio. 
>>>>
>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/909fa4da-1c74-4c33-98c1-185e6bae9d40n%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/golang-nuts/909fa4da-1c74-4c33-98c1-185e6bae9d40n%40googlegroups.com?utm_medium=email_source=footer>
>> .
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/e8c2a879-db37-489a-a7a7-0f70f3a89d2fn%40googlegroups.com.

Re: [go-nuts] Performance issue with os.File.Write

[Arnaud Delobelle]

@Ben interesting, I did something similar and also ended up with a 64KB 
buffer (seemed like the default of 4KB didn't work very well in my 
context).  How did you decide of the buffer size?

Also, there is something that I don't understand.  The default buffer size 
works really well for a toy example (like the one I posted in my original 
question), but the improvements are much less dramatic in my program, for 
the same pattern of writing to stdout.  I can't work out a reason for this 
yet.

On Sunday, 20 December 2020 at 20:31:52 UTC ben...@gmail.com wrote:

> And os.Stdout (and friends) are all regular *os.File objects (which as Jan 
> said, don't buffer). It was non-intuitive to me that stdout didn't buffer 
> by default, because it's such a bad thing for efficiently writing lots of 
> output, but I guess it makes sense when you want terminal output to appear 
> right away. So I realized it made sense, and gives you more control. And 
> it's so easy to wrap it in a bufio.NewWriter() ... Flush() if you need 
> buffering.
>
> I ran into this exact same issue when implementing GoAWK ... a 10-line fix 
> gave me a 10x speedup. 
> https://github.com/benhoyt/goawk/commit/60745c3503ba3d99297816f5c7b5364a08ec47ab
>
> -Ben
>
> On Monday, December 21, 2020 at 12:27:43 AM UTC+13 arn...@gmail.com wrote:
>
>> Ah, that is it, thank you!
>>
>> On Sunday, 20 December 2020 at 11:06:05 UTC Jan Mercl wrote:
>>
>>> On Sun, Dec 20, 2020 at 11:53 AM Arnaud Delobelle  
>>> wrote: 
>>>
>>> > TLDR; a simple test program appears to show that Go's (*os.File).Write 
>>> is 10x slower than C's fputs (on MacOS). 
>>>
>>> Apples and oranges. fputs buffers, os.File does not. Rewrite the 
>>> benchmark using bufio. 
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/30bc991b-1a94-4ce7-bc0d-fea10777b233n%40googlegroups.com.

Re: [go-nuts] Performance issue with os.File.Write

[Arnaud Delobelle]

Ah, that is it, thank you!

On Sunday, 20 December 2020 at 11:06:05 UTC Jan Mercl wrote:

> On Sun, Dec 20, 2020 at 11:53 AM Arnaud Delobelle  
> wrote:
>
> > TLDR; a simple test program appears to show that Go's (*os.File).Write 
> is 10x slower than C's fputs (on MacOS).
>
> Apples and oranges. fputs buffers, os.File does not. Rewrite the
> benchmark using bufio.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/f841a530-f80e-4c1c-95cc-3428d8db9617n%40googlegroups.com.

[go-nuts] Performance issue with os.File.Write

[Arnaud Delobelle]

Hi there!

TLDR; a simple test program appears to show that Go's (*os.File).Write is 
10x slower than C's fputs (on MacOS).

While doing some benchmarking for my lua implementation in Go [1], I found 
very big differences between C Lua and and golua for benchmarks that do a 
lot of output to stdout.  Using pprof, I found that my implementation 
spends a lot of its time in syscall.  I couldn't see an obvious reason why 
so I decided to make a minimal example.  It is a program that writes the 
string "Hello There" one million times to stdout:

 test.go 
package main

import "os"

func main() {
hello := []byte("Hello There\n") // To make it fairer
for i := 0; i < 1000; i++ {
os.Stdout.Write(hello)
}
}
- /test.go 

To compare with, here what I think is the equivalent in C:

 test.c 
#include 

int main() {
for (int i = 0; i < 1000; i++) {
fputs("Hello There\n", stdout);
}
return 0;
}
 /test.c 

I compared those using multitime [2], using both go 1.15.6 and the beta1 
release of go 1.16, using the following steps (I am using gvm to select 
different Go versions).

- Compile the Go version using go 1.15 and go 1.16, and the C version using 
clang.

$ gvm use go1.16beta1
Now using version go1.16beta1
$ go version && go build -o test-go1.16 test.go
go version go1.16beta1 darwin/amd64

$ gvm use go1.15.6
Now using version go1.15.6
$ go version && go build -o test-go1.15 test.go
go version go1.15.6 darwin/amd64

$ clang -o test-c test.c

- Check that the C version and the Go version output the same amount of 
data to stdout:

$ ./test-c | wc -c
 12000
$ ./test-go1.15 | wc -c
 12000

- Run each executable 5 times

$ cat >cmds < -q ./test-c
> -q ./test-go1.15
> -q ./test-go1.16
> EOF
$ multitime -b cmds -n 5
===> multitime results
1: -q ./test-c
MeanStd.Dev.Min Median  Max
real0.524   0.070   0.476   0.492   0.662   
user0.475   0.011   0.465   0.472   0.495   
sys 0.011   0.002   0.009   0.011   0.014   

2: -q ./test-go1.15
MeanStd.Dev.Min Median  Max
real5.986   0.125   5.861   5.947   6.186   
user3.717   0.040   3.677   3.715   3.788   
sys 2.262   0.034   2.221   2.260   2.314   

3: -q ./test-go1.16
MeanStd.Dev.Min Median  Max
real5.958   0.160   5.781   5.941   6.213   
user3.706   0.094   3.624   3.638   3.855   
sys 2.258   0.069   2.200   2.215   2.373   

There is no significant difference between 1.15 and 1.16, but both are more 
than 10 times slower than the C version.  Why is it so? Is there something 
that I can do to overcome this performance penalty?  Any insights would be 
appreciated.

FWIW, I am running these on MacOS Catalina
$ uname -v
Darwin Kernel Version 19.6.0: Thu Oct 29 22:56:45 PDT 2020; 
root:xnu-6153.141.2.2~1/RELEASE_X86_64

(sorry I haven't got easy access to a Linux box to run this on).

-- 
Arnaud Delobelle

[1] https://github.com/arnodel/golua
[2] https://tratt.net/laurie/src/multitime/multitime.1.html

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/c7966990-d873-4d29-a5aa-9c52642d98fdn%40googlegroups.com.

[go-nuts] Re: Interfaces holding integers and memory allocations

[Arnaud Delobelle]

y create a 
>>> pool of available values because Go presents say Value(int64(1000)) as an 
>>> immutable object to me, so I cannot keep it around for later use to hold 
>>> the integer 1001.  To be more explicit
>>>
>>> // Go code
>>> i := int64(1000)
>>> v := Value(i) // This triggers an allocation (because the interface 
>>> needs a pointer)
>>> // Here the Lua runtime can work with v (containing 1000)
>>> j := i + 1
>>> // Even though v contains a pointer to a heap location, I cannot 
>>> modify it
>>> v := Value(j) // This triggers another allocation
>>> // Here the Lua runtime can work with v (containing 1001)
>>>
>>>
>>> I could perhaps use a pointer to an integer to make a Value out of.  
>>> This would allow reuse of the heap location.
>>>
>>> // Go code
>>> p :=new(int64) // Explicit allocation
>>> vp := Value(p)
>>> i :=int64(1000)
>>> *p = i // No allocation
>>> // Here the Lua runtime can work with vp (contaning 1000)
>>> j := i + 1
>>> *p = j // No allocation
>>> // Here the Lua runtime can work with vp (containing 1001)
>>>
>>> But the issue with this is that Go interoperability is not so good, as 
>>> Go int64 now map to (interfaces holding) *int64 in the Lua runtime.
>>>
>>> However, as I understand it, in reality interfaces holding an int64 and 
>>> an *int64 both contain the same thing (with a different type annotation): a 
>>> pointer to an int64.
>>>
>>> Imagine that if somehow I had a function that can turn an *int64 to a 
>>> Value holding an int64 (and vice-versa):
>>>
>>> func Int64PointerToInt64Iface(p *int16) interface{} {
>>> // returns an interface that has concrete type int64, and points 
>>> at p
>>> }
>>>
>>> func int64IfaceToInt64Pointer(v interface{}) *int64 {
>>> // returns the pointer that v holds
>>> }
>>>
>>>  then I would be able to "pool" the allocations as follows:
>>>
>>> func NewIntValue(n int64) Value {
>>> v = getFromPool()
>>> if p == nil {
>>> return Value(n)
>>> }
>>> *p = n
>>> return Int64PointerToint64Iface(p)
>>> }
>>>
>>> func ReleaseIntValue(v Value) {
>>> addToPool(Int64IPointerFromInt64Iface(v))
>>> }
>>>
>>> func getFromPool() *int64 {
>>> // returns nil if there is no available pointer in the pool
>>> }
>>>
>>> func addToPool(p *int64) {
>>> // May add p to the pool if there is spare capacity.
>>> }
>>>
>>> I am sure that this must leak an abstraction and that there are good 
>>> reasons why this may be dangerous or impossible, but I don't know what the 
>>> specific issues are.  Could someone enlighten me?
>>>
>>> Or even better, would there be a different way of modelling Lua values 
>>> that would allow good Go interoperability and allow controlling heap 
>>> allocations?
>>>
>>> If you got to this point, thank you for reading!
>>>
>>> Arnaud Delobelle
>>>
>>> [1] https://github.com/arnodel/golua
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/3bb7d415-1be3-4b5d-9379-558b2d59b2ban%40googlegroups.com.

Re: [go-nuts] Re: Interfaces holding integers and memory allocations

[Arnaud Delobelle]

(sorry about the code formatting gone wrong, I replied in gmail it it seems 
to have removed all indentation!)

On Wednesday, 16 December 2020 at 10:15:07 UTC Arnaud Delobelle wrote:

> Hi Ben, that's an interesting idea. I considered it at the start but
> didn't go for it in the end (I can't remember why exactly, probably
> because that would make it quite a big struct for Lua). There is a
> possibility that I could adapt it a bit and have something like
>
> type Value struct {
> scalar uint64
> iface interface{}
> }
>
> The type could be always obtained from the iface field (it would be
> its concrete type), but the value could be encoded in the scalar field
> for a few types such as int64, float64, bool. There would be no
> storage overhead for int64 and floa64, as the extra 8 bytes used for
> the scalar field are saved by having a "constant" iface field. The
> overhead for other non-scalar values would be only 8 bytes.
>
> I would need some reusable "dummy" interface values for the types
> encoded in the scalar:
>
> var (
> dummyInt64 interface{} = int64(0)
> dummyFloat64 interface{} = float64(0)
> dummyBool interface{} = false
> )
>
> Then I could create Value instances like this:
>
> func IntValue(n int64) Value {
> return Value{uint64(n), dummyInt64}
> }
>
> func FloatValue(f float64) Value {
> return Value{*(*uint64)(unsafe.Pointer()), dummyFloat64}
> }
>
> func BoolValue(b bool) Value {
> var s uint64
> if b {
> s = 1
> }
> return Value{s, dummyBool}
> }
>
> func StringValue(s string) Value {
> return Value{iface: s}
> }
>
> func TableValue(t Table) Value {
> return Value{iface: t}
> }
>
> We could obtain the type of Values like this:
>
> type ValueType uint8
>
> const (
> IntType ValueType = iota
> FloatType
> BoolType
> StringType
> TableType
> )
>
> func (v Value) Type() ValueType {
> switch v.iface.(type) {
> case int64:
> return IntType
> case float64:
> return FloatType
> case bool:
> return BoolType
> case string:
> return StringType
> case Table:
> return TableType
> default:
> panic("invalid type")
> }
> }
>
> Methods like this could extract the concrete value out a Value instance:
>
> func (v Value) AsInt() int64 {
> return int64(v.scalar)
> }
>
> func (v Value) AsFloat() float64 {
> return *(*float64)(unsafe.Pointer())
> }
>
> func (v Value) AsBool() bool {
> return v.scalar != 0
> }
>
> func (v Value) AsString() string {
> return v.iface.(string)
> }
>
> func (v Value) AsTable() Table {
> return v.iface.(Table)
> }
>
> Interoperability with Go code is not as good but still OK. There is
> no need to maintain a pool of reusable values, which is a bonus. I'll
> have to see how much modification to the codebase it requires, but
> that sounds interesting.
>
> -- 
> Arnaud
>
> On Tue, 15 Dec 2020 at 20:06, ben...@gmail.com  wrote:
> >
> > Nice project!
> >
> > It's a pity Go doesn't have C-like unions for cases like this (though I 
> understand why). In my implementation of AWK in Go, I modelled the value 
> type as a pseudo-union struct, passed by value:
> >
> > type value struct {
> > typ valueType // Type of value (Null, Str, Num, NumStr)
> > s string // String value (for typeStr)
> > n float64 // Numeric value (for typeNum and typeNumStr)
> > }
> >
> > Code here: 
> https://github.com/benhoyt/goawk/blob/22bd82c92461cedfd02aa7b8fe1fbebd697d59b5/interp/value.go#L22-L27
> >
> > Initially I actually used "type Value interface{}" as well, but I 
> switched to the above primarily to model the funky AWK "numeric string" 
> concept. However, I seem to recall that it had a significant performance 
> benefit too, as passing everything by value avoided a number of allocations.
> >
> > Lua has more types to deal with, but you could try something similar. Or 
> maybe include int64 (for bool as well) and string fields, and everything 
> else falls back to interface{}? It'd be a fairly large struct, so not sure 
> it would help ... you'd have to benchmark it. But I'm thinking something 
> like this:
> >
> > type Value struct {
> > typ valueType
> > i int64 // for typ = bool, integer
> > s string // for typ = string
> > v interface{} // for typ = float, other
> > }
> >
> > -Ben
> >
> > On Wednesday, December 16, 2020 at 6:50:05 AM UTC+13 arn...@gmail.com 
> wrote:
> >>
> >> Hi
> >>
> >> The context for this question is that I am working on a pure Go 
> implementation of Lua [1] (as a personal 

Re: [go-nuts] Re: Interfaces holding integers and memory allocations

[Arnaud Delobelle]

>
>> // Go code
>> type Value interface{}
>>
>> The scalar Lua types are simply implemented as int64, float64, bool, string 
>> with their type "erased" by putting them in a Value interface.  The problem 
>> is that the Lua runtime creates a great number of short lived Value 
>> instances.  E.g.
>>
>> -- Lua code
>> for i = 0, 10 do
>> n = n + i
>> end
>>
>> When executing this code, the Lua runtime will put the values 0 to 1 billion 
>> into the register associated with the variable "i" (say, r_i).  But because 
>> r_i contains a Value, each integer is converted to an interface which 
>> triggers a memory allocation.  The critical functions in the Go runtime seem 
>> to be convT64 and mallocgc.
>>
>> I am not sure how to deal with this issue.  I cannot easily create a pool of 
>> available values because Go presents say Value(int64(1000)) as an immutable 
>> object to me, so I cannot keep it around for later use to hold the integer 
>> 1001.  To be more explicit
>>
>> // Go code
>> i := int64(1000)
>> v := Value(i) // This triggers an allocation (because the interface 
>> needs a pointer)
>> // Here the Lua runtime can work with v (containing 1000)
>> j := i + 1
>> // Even though v contains a pointer to a heap location, I cannot modify 
>> it
>> v := Value(j) // This triggers another allocation
>> // Here the Lua runtime can work with v (containing 1001)
>>
>>
>> I could perhaps use a pointer to an integer to make a Value out of.  This 
>> would allow reuse of the heap location.
>>
>> // Go code
>> p :=new(int64) // Explicit allocation
>> vp := Value(p)
>> i :=int64(1000)
>> *p = i // No allocation
>> // Here the Lua runtime can work with vp (contaning 1000)
>> j := i + 1
>> *p = j // No allocation
>> // Here the Lua runtime can work with vp (containing 1001)
>>
>> But the issue with this is that Go interoperability is not so good, as Go 
>> int64 now map to (interfaces holding) *int64 in the Lua runtime.
>>
>> However, as I understand it, in reality interfaces holding an int64 and an 
>> *int64 both contain the same thing (with a different type annotation): a 
>> pointer to an int64.
>>
>> Imagine that if somehow I had a function that can turn an *int64 to a Value 
>> holding an int64 (and vice-versa):
>>
>> func Int64PointerToInt64Iface(p *int16) interface{} {
>> // returns an interface that has concrete type int64, and points at p
>> }
>>
>> func int64IfaceToInt64Pointer(v interface{}) *int64 {
>> // returns the pointer that v holds
>> }
>>
>>  then I would be able to "pool" the allocations as follows:
>>
>> func NewIntValue(n int64) Value {
>> v = getFromPool()
>> if p == nil {
>> return Value(n)
>> }
>> *p = n
>> return Int64PointerToint64Iface(p)
>> }
>>
>> func ReleaseIntValue(v Value) {
>> addToPool(Int64IPointerFromInt64Iface(v))
>> }
>>
>> func getFromPool() *int64 {
>> // returns nil if there is no available pointer in the pool
>> }
>>
>> func addToPool(p *int64) {
>> // May add p to the pool if there is spare capacity.
>> }
>>
>> I am sure that this must leak an abstraction and that there are good reasons 
>> why this may be dangerous or impossible, but I don't know what the specific 
>> issues are.  Could someone enlighten me?
>>
>> Or even better, would there be a different way of modelling Lua values that 
>> would allow good Go interoperability and allow controlling heap allocations?
>>
>> If you got to this point, thank you for reading!
>>
>> Arnaud Delobelle
>>
>> [1] https://github.com/arnodel/golua
>
> --
> You received this message because you are subscribed to a topic in the Google 
> Groups "golang-nuts" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/golang-nuts/163s0WdXYIU/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to 
> golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/dcd07f38-1ead-4359-90f3-f6b514c7d541n%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAJ6cK1bpOETKSJv4%2BEEvyMtX-FVBG2PvxzEgv5Wi_sK%3Dgm95rg%40mail.gmail.com.

[go-nuts] Interfaces holding integers and memory allocations

[Arnaud Delobelle]

Hi

The context for this question is that I am working on a pure Go 
implementation of Lua [1] (as a personal project).  Now that it is more or 
less functionally complete, I am using pprof to see what the main CPU 
bottlenecks are, and it turns out that they are around memory management.  
The first one was to do with allocating and collecting Lua "stack frame" 
data, which I improved by having add-hoc pools for such objects.

The second one is the one that is giving me some trouble. Lua is a 
so-called "dynamically typed" language, i.e. values are typed but variables 
are not.  So for easy interoperability with Go I implemented Lua values 
with the type

// Go code
type Value interface{}

The scalar Lua types are simply implemented as int64, float64, bool, string 
with their type "erased" by putting them in a Value interface.  The problem 
is that the Lua runtime creates a great number of short lived Value 
instances.  E.g.

-- Lua code
for i = 0, 10 do
n = n + i
end

When executing this code, the Lua runtime will put the values 0 to 1 
billion into the register associated with the variable "i" (say, r_i).  But 
because r_i contains a Value, each integer is converted to an interface 
which triggers a memory allocation.  The critical functions in the Go 
runtime seem to be convT64 and mallocgc.

I am not sure how to deal with this issue.  I cannot easily create a pool 
of available values because Go presents say Value(int64(1000)) as an 
immutable object to me, so I cannot keep it around for later use to hold 
the integer 1001.  To be more explicit

// Go code
i := int64(1000)
v := Value(i) // This triggers an allocation (because the interface 
needs a pointer)
// Here the Lua runtime can work with v (containing 1000)
j := i + 1
// Even though v contains a pointer to a heap location, I cannot modify 
it
v := Value(j) // This triggers another allocation
// Here the Lua runtime can work with v (containing 1001)


I could perhaps use a pointer to an integer to make a Value out of.  This 
would allow reuse of the heap location.

// Go code
p :=new(int64) // Explicit allocation
vp := Value(p)
i :=int64(1000)
*p = i // No allocation
// Here the Lua runtime can work with vp (contaning 1000)
j := i + 1
*p = j // No allocation
// Here the Lua runtime can work with vp (containing 1001)

But the issue with this is that Go interoperability is not so good, as Go 
int64 now map to (interfaces holding) *int64 in the Lua runtime.

However, as I understand it, in reality interfaces holding an int64 and an 
*int64 both contain the same thing (with a different type annotation): a 
pointer to an int64.

Imagine that if somehow I had a function that can turn an *int64 to a Value 
holding an int64 (and vice-versa):

func Int64PointerToInt64Iface(p *int16) interface{} {
// returns an interface that has concrete type int64, and points at 
p
}

func int64IfaceToInt64Pointer(v interface{}) *int64 {
// returns the pointer that v holds
}

 then I would be able to "pool" the allocations as follows:

func NewIntValue(n int64) Value {
v = getFromPool()
if p == nil {
return Value(n)
}
*p = n
return Int64PointerToint64Iface(p)
}

func ReleaseIntValue(v Value) {
addToPool(Int64IPointerFromInt64Iface(v))
}

func getFromPool() *int64 {
// returns nil if there is no available pointer in the pool
}

func addToPool(p *int64) {
// May add p to the pool if there is spare capacity.
}

I am sure that this must leak an abstraction and that there are good 
reasons why this may be dangerous or impossible, but I don't know what the 
specific issues are.  Could someone enlighten me?

Or even better, would there be a different way of modelling Lua values that 
would allow good Go interoperability and allow controlling heap allocations?

If you got to this point, thank you for reading!

Arnaud Delobelle

[1] https://github.com/arnodel/golua

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/5c782f9f-e442-427e-ae92-d7285fb55ca3n%40googlegroups.com.

Re: [go-nuts] Go Generics using interfaces - alternative take

[Arnaud Delobelle]

I was planning to read the updated proposal in more details before 
replying, but I haven't been able to find the time sadly, so I'm doing it 
anyway :), mostly inline.  There are a couple of general points I'd like to 
put forward first.

- What I'm trying to push towards is not my particular approach, but more 
simplicity while retaining enough usefulness (that is, being a good fit for 
the established good use cases for generics in Go).  I think that when you 
explore how to introduce generics into Go, it's important to remember how 
effective it has been without generics (and partly because of its lack of 
generics), and to try not to upset that balance.

- There is one aspect of the current proposal that makes me think that we 
can find a better manifestation of generics in Go.  To take the one of the 
simplest examples, the Stringer example:

type Stringer interface {
String() string
}

Here is the generic Stringify implementation

func Stringify(type T Stringer)(s []T) (ret []string) {
for _, v := range s {
ret = append(ret, v.String())
}
return ret
}

Here is the "plain" non-generic one

func Stringify(s []Stringer) (ret []string) {
for _, v := range s {
ret = append(ret, v.String())
}
return ret
}

The two implementations above express the same intent, even have the same 
body, but they are not the same thing!  Obviously you can use 
Stringify(Stringer) to obtain the second one from the first but to me it 
feels like we are missing a trick, a better perspective that would make 
this simplest of examples more natural.

- shortly after the contracts proposal was published, I found that I was 
unhappy with the overlap between contracts and interfaces, wrote an 
explanation of the problem and imagined a rebalancing of the proposal which 
is in essence the new one (ie. interfaces can be generic).  I think it's 
useful because it presents a rationale for "transforming" the earlier 
contracts proposal into the new interface-based one, so I've posted it 
here: 
https://arnodel.github.io/marooned/go/generics/2020/06/21/go-against-contracts.html

Kind regards,

Arnaud Delobelle

On Wednesday, 17 June 2020 at 06:10:08 UTC+1 Ian Lance Taylor wrote:

> On Tue, Jun 16, 2020 at 1:54 PM Arnaud Delobelle 
>  wrote: 
> > 
> > I noticed today the blog post about the revised Generics proposal. What 
> got me excited is this extract at the start: 
> > 
> > The biggest change is that we are dropping the idea of contracts. The 
> difference between contracts and interface types was confusing, so we’re 
> eliminating that difference. 
> > 
> > To me the lack of orthogonality between contracts and interfaces was the 
> major issue with the previous proposal and I have been musing with ways of 
> resolving it for a while, so I am very pleased that the proposal is going 
> down this path! 
> > 
> > Last month I decided to write a post about a way of introducing Generics 
> into Go using interfaces, which I called "Package Specialization". 
> > 
> > Headline features are: 
> > 
> > use interfaces to express parametric types; 
> > no new keyword; 
> > the only new syntax is “package specialization”: pkg(T=int, 
> Y=*MyType).Func(); 
> > generic code which doesn’t use the package specialization syntax is 
> already valid Go. 
> > 
> > The full post can be read here: 
> https://arnodel.github.io/marooned/go/generics/2020/06/06/go-spec2.html 
> > 
> > I was toying with the idea of submitting it here for feedback, to try to 
> move the discussion in that direction. Well, I guess today's updated 
> proposal steals my thunder! Essentially this is the same idea, but with a 
> different manifestation in the language - I thought that submitting it 
> anyway may provide a useful comparison point to help gauge the updated 
> proposal better. 
> > 
> > Note that sadly, I do not have much time to polish a proposal or to 
> follow discussions in general on this or other golang-related discussion 
> lists, so I apologize in advance if I am inadvertently re-hashing ideas 
> that have been put forward and dismissed in the past, or if my blog post is 
> a little low on detail (it was meant to be an exploration of a possible 
> approach). 
>
> Thanks. I hadn't seen that before. 
>
> A difficulty with package specialization is when you want to write a 
> List(List(int)). That is, a List where each element of the List is a 
> List(int). 


That would be

 list(T = list(T = int).List).List,

which I agree is not the most palatable.  Of course one could defined an 
intermediary type

  type intList = list(T = int).List
 

> Or, when you want to write a transformation function as in 
>
> https://go.googlesource.com/proposal/+/refs/heads/master/design/go2draft-type-parameters.md#list-transform.
>  

[go-nuts] Go Generics using interfaces - alternative take

[Arnaud Delobelle]

Hi everyone,

I noticed today the blog post about the revised Generics proposal.  What 
got me excited is this extract at the start:

*The biggest change is that we are dropping the idea of contracts. The 
difference between contracts and interface types was confusing, so we’re 
eliminating that difference.*

To me the lack of orthogonality between contracts and interfaces was the 
major issue with the previous proposal and I have been musing with ways of 
resolving it for a while, so I am very pleased that the proposal is going 
down this path!

Last month I decided to write a post about a way of introducing Generics 
into Go using interfaces,  which I called "Package Specialization".

Headline features are:

   - *use interfaces to express parametric types*;
   - no new keyword;
   - the only new syntax is “package specialization”: pkg(T=int, 
   Y=*MyType).Func();
   - generic code which doesn’t use the package specialization syntax is 
   already valid Go.

The full post can be read here: 
https://arnodel.github.io/marooned/go/generics/2020/06/06/go-spec2.html

I was toying with the idea of submitting it here for feedback, to try to 
move the discussion in that direction.  Well, I guess today's updated 
proposal steals my thunder! Essentially this is the same idea, but with a 
different manifestation in the language - I thought that submitting it 
anyway may provide a useful comparison point to help gauge the updated 
proposal better.

Note that sadly, I do not have much time to polish a proposal or to follow 
discussions in general on this or other golang-related discussion lists, so 
I apologize in advance if I am inadvertently re-hashing ideas that have 
been put forward and dismissed in the past, or if my blog post is a little 
low on detail (it was meant to be an exploration of a possible approach).

Kinds regards,

Arnaud Delobelle

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d47d543e-1ac0-4145-ab4a-11c20aea462bn%40googlegroups.com.